From patchwork Wed Dec 1 14:30:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12650033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEA12C433FE for ; Wed, 1 Dec 2021 14:37:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 417BC6B0087; Wed, 1 Dec 2021 09:31:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D4706B0092; Wed, 1 Dec 2021 09:31:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1B206B0089; Wed, 1 Dec 2021 09:31:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0071.hostedemail.com [216.40.44.71]) by kanga.kvack.org (Postfix) with ESMTP id DCAB76B0088 for ; Wed, 1 Dec 2021 09:31:05 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A84D289D49 for ; Wed, 1 Dec 2021 14:30:55 +0000 (UTC) X-FDA: 78869462070.15.CC5BECF Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf09.hostedemail.com (Postfix) with ESMTP id 732C030017D9 for ; Wed, 1 Dec 2021 14:30:56 +0000 (UTC) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1B1EOuJI023540; Wed, 1 Dec 2021 14:30:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=RZT1kr5PGIoOsnaO5c86U9aVkvMOoA5jUV9pXKZXd2Q=; b=cAJrQlnrSGPKANZ1N+tewsZMfBeRwc3U0wGwKfZvPzDzYNP+o2HllcBB9NIWVtprMjYH zEGjrx3Z19Mo0ontPYbPj2bNP4wmxvnp1jY4ncJVHhRvYNzbf2yihvV4mvESOJk4Tsdi WI01REqhw2H7kPpE2F1M0Mfnmg8N8gBkMqtl2L1+vzTN4CT21pLoYcBw5zLvs3lv3yZ/ bT9cfjC4jyJE2mnwWBv41+nN1428zx4niCS/Q6sl/+j8O6BbjO9rBm427w0z1sNAsBg5 jEWeHNHJ/enbmNtlQnFMfE/3FLCrwiYdgD8u/6ajr4FTMrE8+RZAtdlRsaQVWi0G1bUX Ww== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3cp9gkgghx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 01 Dec 2021 14:30:46 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1B1EBoq7037533; Wed, 1 Dec 2021 14:30:44 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2046.outbound.protection.outlook.com [104.47.57.46]) by userp3030.oracle.com with ESMTP id 3ck9t1v74d-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 01 Dec 2021 14:30:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nHbI9wkqMYKJv5zNf/FJo0qiVeZ7BFDJdrFVVsjr5j9FsdtzqCgYpI2V4GXLWF8+7sXwk49hzdHC2KwPjbgxTBXla9QKqF/W2Pm0509LwSEoSkGZR5EPVSHkP+cCWJbMMEOQHohfJ7hgGDrhsuWA7wzSTv+N55R+x27Dh7UYvd3Tn6YmpIEqtMgxh/tuEVRk/Zc3zpKWDx/w7NscwdSGhdXNoYW2y0sUeDgZ4rf0H4KofT8Zhj6HCQjJA4pnXupcgP5jZ5L6bJxyrUuJPEdlrHuXm00vhbyUrmfr/lKEjx5kLH0RFY+F0jFqmZPHU9NQtAYyJyFr2JJMqZfrs3Mnaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RZT1kr5PGIoOsnaO5c86U9aVkvMOoA5jUV9pXKZXd2Q=; b=KiiuesEtG7xMcP6bC6rHI1qpPcDTJUPX1CufEQSZlEprMb9VJzE3BwU+N9t3/X1Tv6U1CYkYYkI2dHFAkMsYbsPr5uyJXF85+/mOi5EqAV/XsP51L9awgVAmcBdFyB6B5KLz5GJdrERAND6vBWiEGcsXrxbSVJn6x/PRzryRGuUr1SmxsPXRXxaYdTANzfrhucsk07vTm+f0cPSl0FXFixvwE9zVuee+zhScGbevBCt2x4y2FMdlxeM0oCdfbz7sfY14XoKOfK+Qg/1vx10Mj1T9T82n9a9Olp+Ga+P7n4Ja5cWJNrh3PgXF3yqTCXP93qqu1QSxNO9+jxCPfuZdFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RZT1kr5PGIoOsnaO5c86U9aVkvMOoA5jUV9pXKZXd2Q=; b=KSVmwN/SxqeLN4zXmlQOX2BXXdZ8ltIMmwy7RFz1ke9xl1jMF/0XisoxKhG7AIwqyZQx9/vOmLNyelme46yGxCX6+IU6EvGIy2tbkWLgnF23g3SCGnp7kWDByXfOra6vzYl40DFzlQeZXPbRGIBvEsJh9P7a/wkUDvrvNFUXTHg= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by SN6PR10MB2464.namprd10.prod.outlook.com (2603:10b6:805:4d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.22; Wed, 1 Dec 2021 14:30:41 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::c4d0:8291:84d:ac66]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::c4d0:8291:84d:ac66%7]) with mapi id 15.20.4734.027; Wed, 1 Dec 2021 14:30:41 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton CC: Song Liu , Davidlohr Bueso , "Paul E . McKenney" , Matthew Wilcox , Laurent Dufour , David Rientjes , Axel Rasmussen , Suren Baghdasaryan , Vlastimil Babka , Rik van Riel , Peter Zijlstra , Michel Lespinasse , Jerome Glisse , Minchan Kim , Joel Fernandes , Rom Lemarchand , Liam Howlett Subject: [PATCH v4 56/66] mm/mlock: Use maple tree iterators instead of vma linked list Thread-Topic: [PATCH v4 56/66] mm/mlock: Use maple tree iterators instead of vma linked list Thread-Index: AQHX5r/0n+VjSgFv/U2n/DOQtwVgxg== Date: Wed, 1 Dec 2021 14:30:15 +0000 Message-ID: <20211201142918.921493-57-Liam.Howlett@oracle.com> References: <20211201142918.921493-1-Liam.Howlett@oracle.com> In-Reply-To: <20211201142918.921493-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.30.2 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1dcafc40-109c-43f0-c44a-08d9b4d72717 x-ms-traffictypediagnostic: SN6PR10MB2464: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2331; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: /RZES8BdRt78tWya3kPLOc6Cx7saGrA7HBq49L60lNPPtQM6TqmC8dSgl7AoNPt2nEQmcAEkGPdxD/r6gL0civv3YhKMgPbMcK1QMS3wELWWSJ/CPQmhHklXA5qCFDFCVb+HOi722C0E934cGUOUr7DhjEHjEUP5YM6MIhqQJc6rAHQHeJ6JXiefDLUmzUTECs+lvf4jEKF+Y7Rf3In9AWS2TZWczr5qrxV7VCwffC0remLVRDc4nJuyIqr0j3IdRI2aHZJFBlsb786jVLDKA4yDPqy5D24GHwrPMRBKTRiBu1cJVEMKvlZfbCgKMSpLap5PQdzulYs/CCV3/40imU+Uqz7krT725GhvmZ5/SCnd9O1cbP0Dl+it0dX1HcbtOCsTpAtX47Ab5Ckrx2deezE5+tTBmi5HVKCVNeZYQ+0X77PgGrdvG8cp/FAJj058QlcE73q9eJanR4a1RDdqeRiyTnB2psb9yWacoGMWDEob/6uFh2PbEqkcDPTCA4zuj0jQGh4TDK5xMKvm6oPv9txfJqR1DZDps0g3MLHHXufigQuY4HlRyXyN1XaXM7M0sCqLEAnHXS7+tNig13utLT6a0NBWSz/H7bU69fs6F3CLgdTX7u8Bvqeaju8ClLqo9+nC9MJK990FnAPffEoe9wdM4inzpUO5B6QMluFPse+l2IJ0HB3FtFx7RBkOzTPScA5buSgTfi72Ca8GJfVHzg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(91956017)(6512007)(2906002)(1076003)(5660300002)(76116006)(83380400001)(186003)(6666004)(4326008)(26005)(38070700005)(36756003)(8676002)(7416002)(54906003)(107886003)(2616005)(316002)(86362001)(122000001)(44832011)(508600001)(8936002)(64756008)(66446008)(110136005)(38100700002)(66946007)(71200400001)(66556008)(6486002)(66476007)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?zdb4m6/tCqMZbEEnsD4qbAc?= =?iso-8859-1?q?x1bVlmbb8dWctc83rYhSEn2qofsO/AVidvLp5qSQT1L/KuS8HWBhE+Lrt0So?= =?iso-8859-1?q?kMwMQ6ZbGUdtoto7/8q/An6HqBR/7vVaoZEyr6sF43m5C7/kJF2Y4GbYPRw+?= =?iso-8859-1?q?IysW6/++xLbHWeE53saSHCeG8MZCfWZIVqJwy+aulNrUPsJ9Ir1j+/uYNnoC?= =?iso-8859-1?q?nml7972VVD8lXXzH/RZuSqjI4iqTDpBs0N+qbn5HdZpgSxjs2Tx//tD3QBTw?= =?iso-8859-1?q?wyVf2nQIvhadNhF56QjmzyUsiX88a3PD7LlgBbtGH4JCJcObGQVf6oRJVnLM?= =?iso-8859-1?q?dVbG/QtKMEGvLpVPfd/6rxbL22fZ2Vnpi2mreuu8GDiCWfaLlrk8BOTNEOuM?= =?iso-8859-1?q?k5Jxy/U4kKXS8KXpWtwDceTkkO7Ot2Z42MzgBIue5Wk+piScp5/Fv36eflKh?= =?iso-8859-1?q?zs7NAvMdrJ1PoZ+/fO+SZAn+uuYc3QkjaoY3TlmY0upfQd69V9gFEFYfTMsA?= =?iso-8859-1?q?Pz+Jr896qSLX/61ZbV8CbymFfQT3zwG8IO4wlXAUg1pyf7puOkSJkEzN5IGS?= =?iso-8859-1?q?1sHje+YqX54CelAcEphMrNx4WuN5oih9LrdV894oigc7DS36BRdzsvULX/qN?= =?iso-8859-1?q?rK0fM9Yi/GV04StCFYClXVwLvwfpZ19jv+MzKezk4pezRv/bxqDHnbZ5D4D4?= =?iso-8859-1?q?BWAO/8E88qggf5sJAqFR34h8QCKUdTDJrpyQF62oPmyWyq1o62+GdKOtE0qC?= =?iso-8859-1?q?Pf3LFUFFPVWK1hnhHKMD8SbHXasvB5QrABvevHKv20b8dfN73Qa3g08kSy53?= =?iso-8859-1?q?Xq5ROewrQowvel4KMI7WY+QHez9ytabV6S2JAKA0m8FvcedHEp18S8qFygZ7?= =?iso-8859-1?q?oOedK1yIwxM/GFuEJW7qrH/ka9oCFDj27MiHM+N7sWL1lLEdPQfjGBV94hbj?= =?iso-8859-1?q?6XoEjf/AxRJxF6Z9214fQdNNfJAzYlWG5p8jbs5Yh5hp+FWuhPLxjNeVWAne?= =?iso-8859-1?q?0uoVwvPSlanQp7MPmu4ydQ914yzqSbYvamP4X1xtaXmUU3gDvwHYk+uzv0Xq?= =?iso-8859-1?q?V+pJf42HQtZpMqygP5WZ76wT7wd/NEzt+NWY0QN0aVweKVKSSNKV+mSK4cPG?= =?iso-8859-1?q?atODAO8RTz3T/v2nEn+K/KmGt+ZcXFTgUxYlNhBnGTmyl4sMJq3JzV+gIMK0?= =?iso-8859-1?q?P3IYMr7gRk02ncFGYilgsA+fh5O4HJaMXrKpW0ggyqWB6LojM5duw/vkqQ/Q?= =?iso-8859-1?q?EgMKDhbiRVUWmavrh9C46PXhpLtNNTmcCPYDG4D3cHMh6cxlvWTyx4+Ce8C+?= =?iso-8859-1?q?Yc8XPgOrK6udvdzsMGKr0LQdEyZmThdzydKkJ1AMyAgsWthHf9/lMRQnAAzJ?= =?iso-8859-1?q?OvQ7aT2OXUS7dezwSvo/XH7XpRTij1f6x1s9pxhn3RCLXeVr9kNUW/FBO6PR?= =?iso-8859-1?q?14fial6wUUWkjDQo+1CNk3vvmI0ijSGeUegwxRyoqWRcH40szbBbTAaaeX5N?= =?iso-8859-1?q?pazTU7o/Mg/2hTQerI0bnOibZ98NLWy6wNl/uAE1un1gtiex91PyRgJy7yTM?= =?iso-8859-1?q?IBWhmMZ+AyT9ASvoiR1PDiOhq8TmmV/ERI00GWHdNXFe9Bs7WEnsjbG1BTT0?= =?iso-8859-1?q?viL/skLT45ckwpXYJl0jS1F3KYYPxVVV9/A9KIQC6u7h7uFBMfiR2gp54UNo?= =?iso-8859-1?q?=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1dcafc40-109c-43f0-c44a-08d9b4d72717 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Dec 2021 14:30:15.2973 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: SONUEqX8TiwyGd7x6oL7/4lBr6fawu6bA31FuskNMLzfwGkYAiMJd4o/fDdSxmVLuUVb3YPThiNu8DOyfySM9g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2464 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10184 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 malwarescore=0 spamscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112010081 X-Proofpoint-ORIG-GUID: gNURNBZ4Em8LUnk1dBri3SYEPTPyX0wp X-Proofpoint-GUID: gNURNBZ4Em8LUnk1dBri3SYEPTPyX0wp X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 732C030017D9 X-Stat-Signature: k3km8rj8i6d7z9a7cw1jcpkhdwp7t3z4 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=cAJrQlnr; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="KSVmwN/S"; spf=none (imf09.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=liam.howlett@oracle.com; dmarc=pass (policy=none) header.from=oracle.com X-HE-Tag: 1638369056-696781 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Liam R. Howlett" Signed-off-by: Liam R. Howlett --- mm/mlock.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/mlock.c b/mm/mlock.c index e263d62ae2d0..feb691eb4c64 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -563,6 +563,7 @@ static int apply_vma_lock_flags(unsigned long start, size_t len, unsigned long nstart, end, tmp; struct vm_area_struct *vma, *prev; int error; + MA_STATE(mas, ¤t->mm->mm_mt, start, start); VM_BUG_ON(offset_in_page(start)); VM_BUG_ON(len != PAGE_ALIGN(len)); @@ -571,11 +572,11 @@ static int apply_vma_lock_flags(unsigned long start, size_t len, return -EINVAL; if (end == start) return 0; - vma = find_vma(current->mm, start); - if (!vma || vma->vm_start > start) + vma = mas_walk(&mas); + if (!vma) return -ENOMEM; - prev = vma->vm_prev; + prev = mas_prev(&mas, 0); if (start > vma->vm_start) prev = vma; @@ -597,7 +598,7 @@ static int apply_vma_lock_flags(unsigned long start, size_t len, if (nstart >= end) break; - vma = prev->vm_next; + vma = find_vma(prev->vm_mm, prev->vm_end); if (!vma || vma->vm_start != nstart) { error = -ENOMEM; break; @@ -618,15 +619,12 @@ static unsigned long count_mm_mlocked_page_nr(struct mm_struct *mm, { struct vm_area_struct *vma; unsigned long count = 0; + MA_STATE(mas, &mm->mm_mt, start, start); if (mm == NULL) mm = current->mm; - vma = find_vma(mm, start); - if (vma == NULL) - return 0; - - for (; vma ; vma = vma->vm_next) { + mas_for_each(&mas, vma, start + len) { if (start >= vma->vm_end) continue; if (start + len <= vma->vm_start) @@ -741,6 +739,7 @@ static int apply_mlockall_flags(int flags) { struct vm_area_struct *vma, *prev = NULL; vm_flags_t to_add = 0; + unsigned long addr = 0; current->mm->def_flags &= VM_LOCKED_CLEAR_MASK; if (flags & MCL_FUTURE) { @@ -759,7 +758,7 @@ static int apply_mlockall_flags(int flags) to_add |= VM_LOCKONFAULT; } - for (vma = current->mm->mmap; vma ; vma = prev->vm_next) { + mt_for_each(¤t->mm->mm_mt, vma, addr, ULONG_MAX) { vm_flags_t newflags; newflags = vma->vm_flags & VM_LOCKED_CLEAR_MASK;