From patchwork Tue Oct 5 01:30:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12535339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6CA2C433FE for ; Tue, 5 Oct 2021 01:35:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7A845610C8 for ; Tue, 5 Oct 2021 01:35:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7A845610C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 0027494003A; Mon, 4 Oct 2021 21:32:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EDB00940037; Mon, 4 Oct 2021 21:32:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5DC794003B; Mon, 4 Oct 2021 21:32:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id 98318940037 for ; Mon, 4 Oct 2021 21:32:09 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5A5DB181D302B for ; Tue, 5 Oct 2021 01:32:09 +0000 (UTC) X-FDA: 78660657978.03.66518C5 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf06.hostedemail.com (Postfix) with ESMTP id E2C3B801C349 for ; Tue, 5 Oct 2021 01:32:08 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1951Ooio023948; Tue, 5 Oct 2021 01:31:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=V5qpaCz/mr8CT4B2S43tOfxGzkL2jGk0OwfTrdxVPZ8=; b=aNO/CCohT23d337zBef8aKdGiB6reAInUkEphAdYcV7SVg7J96IeVGxlMbX6yrg2yaUA Le5zsMDLFRj1wclWNSzUM3mkv3WbEKrLn4XwF5Pfp8bSwHf+A/Al5OP9/OskcWtPwF/g Rz16pEkOtqHWWYCxZoSu57+mRMp69gkRPxKDD82JANirE98vvG743wc+1UNIRDseYpmb k3KaP5RKDhIKi/AjyZMZkrjs/g5bgm3//20c+Mvl8HDnQZdx7FU+DIUsa1nr2OVMu2uE Aqmgy1BEmor6pumqMgzoKaK+rXNUcXDoH66yWj1BbyAqOlcM6dZe8F8RReU/861+NgHb iQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3bg43gkph8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 05 Oct 2021 01:31:27 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1951Tu3q056799; Tue, 5 Oct 2021 01:31:26 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2175.outbound.protection.outlook.com [104.47.58.175]) by aserp3030.oracle.com with ESMTP id 3bev7sgru8-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 05 Oct 2021 01:31:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dGJGPOSPjDBTQHTTfzfzB5EU/rthn3ZBl/Mey41W3gWPIowdN4I+yls2jTNwbXtv7sW3isf0tZdaSvnHuRNheuZONM/sniSsWC9lK04PCzsw5NWbYZYDzxwB3Sv6sLFxHhi1yJTCvcAyOCfW0vV0gUzcE7YhC+YkpfMtH/EpJF4N83qUXlnUQMPLJie3l6N9ZWJDxzzIfhv3dt+Cx+Lgr1YhDdZye0C7671DAfCL3zdBHe+4XJVuzGVsL+brL+GM6vShnUFiPcKYJ5xjP18MG6j/+t36uXfCMgBiOC4vwlHqrI5d5TKt3jOFCK7av0/wINcncyialZU9M3IoD4XkOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V5qpaCz/mr8CT4B2S43tOfxGzkL2jGk0OwfTrdxVPZ8=; b=gfO0M8Vu3KRFSFsYWgBsUqxALpA1e0eptL6QUVmOllQpf7Dv+hTf5j1nxl6azot2/mGRsaiRsn+zVBEoLg9xtDdPmzQBvEnWBPWGuntUJlEvqICJTPP6dApVf8tRGmPwflZ5J6NdkSY34951m+jCOOOiPe7KWZPn+hY0XK9qrq7GMuZcBNUUKklnDsyR+bzMaF1Udjxu1Fv5a9EoIc6ms1rWkjMMOV2tFVUHosU8oNlIdpQ3DayqZdwEYMYQrO3MjzLWQ1M64PbXyj7z1P42nKkC7sPNEIZfRYAkPwfPohqmRbc1k1fu1kMWhD+fEC0EwZp5KBEb5AKJDeWOQIO3hw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V5qpaCz/mr8CT4B2S43tOfxGzkL2jGk0OwfTrdxVPZ8=; b=dx/jByQ29B9U0QkU5tLuHehYBV0y9NdsfM20bi8bppJxPvNgqy7xPt9jyg8buMjUuwELMTkxn91P2XAi+Be07/85MFYCgQF7SDr24mgHJ3uheeFLjssrTfpwQ4eLf3ICrTXzXAI8aI+N/QvOVS+Q1TBeLYoPHgKrMtz+6Zd64P8= Received: from BL0PR10MB3011.namprd10.prod.outlook.com (2603:10b6:208:7e::29) by BLAPR10MB5044.namprd10.prod.outlook.com (2603:10b6:208:326::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15; Tue, 5 Oct 2021 01:31:22 +0000 Received: from BL0PR10MB3011.namprd10.prod.outlook.com ([fe80::6d61:54c2:40f0:93a]) by BL0PR10MB3011.namprd10.prod.outlook.com ([fe80::6d61:54c2:40f0:93a%5]) with mapi id 15.20.4566.017; Tue, 5 Oct 2021 01:31:22 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton , David Hildenbrand , Douglas Gilbert CC: Song Liu , Davidlohr Bueso , "Paul E . McKenney" , Matthew Wilcox , David Rientjes , Axel Rasmussen , Suren Baghdasaryan , Vlastimil Babka , Rik van Riel , Peter Zijlstra Subject: [PATCH v3 39/66] fs/userfaultfd: Stop using vma linked list. Thread-Topic: [PATCH v3 39/66] fs/userfaultfd: Stop using vma linked list. Thread-Index: AQHXuYijZR18QGOVckCImC/Z6w8VfQ== Date: Tue, 5 Oct 2021 01:30:53 +0000 Message-ID: <20211005012959.1110504-40-Liam.Howlett@oracle.com> References: <20211005012959.1110504-1-Liam.Howlett@oracle.com> In-Reply-To: <20211005012959.1110504-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.30.2 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7dc322b0-0428-4cc7-c74d-08d9879fd6dc x-ms-traffictypediagnostic: BLAPR10MB5044: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2887; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: bk6p1mbJYFOZcrQeWVEfUWdVlOnAOhiiemxvFrAfbvFh2r/eu2D3AKBVJScAWT4K+xq49MxgY6uLP9KMp64387PjzSnm2yYVgQvFG3V6wacbf51crtTafINcX9b5yxFvUjdzVfYLzBm/JlTV7Q34imH4mbwVLyLp6J2j3n2CVTPz9WyIGWXX+286JjIH/x6NKdtfwzOeg7ksj3lhGyBFYykrqCbiPQxOinezjfTTYOwke3mAxWPcROP6n15JE2/qVma9eRePXemmWXIS7ehMcqoo00y2HGPOPa5M0OIWrorH1gq8nqdOvw3BcEqJ/g+wTccAoMgXlYmZqEY7tEe/V2Gi2+vOGivaHzL9ZmQn+UAVHIE6yDDtGQAaQ2fKoIrgYDO6RdGtNHTedZ0iaf/ZlPelolRnnyjXMJnCskiCVSnKLQwWLiJ4j51IynRKRYmEDh2FtgulIjctwXlbWV47UGK9lGINXY1rEVHoykpC1HUcUyfuzHVv9cpF/B0FeJ4mRYXTSIr+nNwyb0OMvy4xeT1NccOCd4vWCZK8yYsORNXUKwscGclVWIsLaefzD7z4lJ92skAZfqBzVd6NEZozHGlRwgmNXsnKOyNtz+8oPl8lIYDpNjXJtWNIgYOByyKzAxZf6TX1omnjMa7Brl0ILltJjjgLDzvwMNaWMATRO2GvnCvuIcHdjLaLt0unk4xCzzQjvRJ6gJrSYUWB9PZjQQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR10MB3011.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(110136005)(71200400001)(8676002)(86362001)(6666004)(6506007)(6512007)(36756003)(316002)(8936002)(54906003)(2906002)(83380400001)(186003)(66446008)(122000001)(26005)(7416002)(6486002)(4326008)(508600001)(2616005)(66476007)(44832011)(38070700005)(76116006)(5660300002)(91956017)(64756008)(66556008)(66946007)(1076003)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?KVDHBCex7kylyeru821i8Ok?= =?iso-8859-1?q?KmikBI2KjuSiWjV4Ai6e0QUtGIJoAuiOmsvll5j+YGc21HSVQsITjE9djbz6?= =?iso-8859-1?q?V3CI8hCYz+H9ps1nwMGJPoLmqYJGZIxOr3LSSuA8Unlv/V8/Rq1LDtnFcAJT?= =?iso-8859-1?q?FnsWdKsL/pCYdkFNDY71SqqNC0lvRHyhQt7+/PZg0UsvH8J8HF3SHGIl2U/u?= =?iso-8859-1?q?t2PGUb2AG/2wbbLSDCswi/NEtUxDwkd/nfHtbSTxa0OvcvCVRm3w6SOusEpz?= =?iso-8859-1?q?UXbssyk6e/+JHtoq8knUMaOgFexFILvUtZ94ruq2J+z9QI+wuKoaYBt1AmUc?= =?iso-8859-1?q?zB8KqmxgBzE/6ucb7jaCCOcpBAKpFSNvRKy/xZ6xkH8mKRa3y/Kpk45yvBEb?= =?iso-8859-1?q?OD8E0TZQxq5AKZAARDIG4GjOP4vGkZrF/qN7U5zSA80NSZAMYSD62c4f3ZuE?= =?iso-8859-1?q?ouXd9LicIFK9jaMGohoitEtA+jpCgzw9m9FGiJIpsRbimvDh5H9CpeF5yn0f?= =?iso-8859-1?q?JcN9bF3kjmunagAjocTHI9gCoSVwyoPeD/My58wlPOouHNHg4Cr7ahAg7j4G?= =?iso-8859-1?q?cs0no5UyGhmC8xWQta9DJyZJkZhpkbv+LXkNmKxx9OkBhfzdrddEr40unv6v?= =?iso-8859-1?q?9/MUCba7ASjWOeI4LiZxTiVgOJdl6E5+OI/pp0ODhf/xTLQDeVteQq2ZHhKj?= =?iso-8859-1?q?Spoixa/7USNHaX20AFw73smzX8QxmYUvuLsImzVQVyCKDBNTW07AoXkQT+Vd?= =?iso-8859-1?q?tdrWPpdNRmSznPwtJpvTbWFWdygruVxtOnMCdqeD3of2MbcxlHHeRozrNxqi?= =?iso-8859-1?q?Mm+0nEroifH4kL5ndeeWWQUxhQRyBqK+moHT8U7cgYXlsv071gruNmnRosMY?= =?iso-8859-1?q?V8fclgQD5rHDlh6y/j6Y9wsWOrfhrLV35Oh6Kcyk1KWq4KjmenhVY84M7CiS?= =?iso-8859-1?q?RCTo2E46EzwiPRXfwQsyt+bPb82cgV2OY/SMtMjraFgGkSrPtdUz2R09cMvd?= =?iso-8859-1?q?vjswXdLcpbOz23+5LsOO0mjaNFJ9fxCbtV2w4z4CTx0mmxFo2XyhvowM3t19?= =?iso-8859-1?q?lLVxsD97axYxSTVaEsUk3Y2siIbznfMZaj3aaRgpxvV3l2KE5YEpfxIimFtb?= =?iso-8859-1?q?8x0PzVs911CYMkJ/QE7fBb8EoPpOBHBSXou87mBrt2SIMZr4ECZC4SodrIjD?= =?iso-8859-1?q?1CuehLLUMks93Aaj+bwPCQ86o4i33caV1eQsa00fCb8OU7won3380C/px1aU?= =?iso-8859-1?q?QtF2cBe7wSUR33ot0MVFHOxnILR5el2ZKt+RHbL+LGTOWK+40QGR5x1FXzHn?= =?iso-8859-1?q?DFNe3l0uc7Zl8YdjqijODJ5TUaaLT30E0dFuJeigqGu7QdZduY5vvgnOJ4hm?= =?iso-8859-1?q?H?= x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BL0PR10MB3011.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7dc322b0-0428-4cc7-c74d-08d9879fd6dc X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Oct 2021 01:30:53.4693 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4QCvqUQ7LqVFn5FnMzZfD+htxdWvsdxYTFwDY746LIOrhEJfqFtpf0n6WbjW1pSwxdrsCHR3uV9ySrG7nGLcnw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5044 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10127 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110050007 X-Proofpoint-GUID: 4vPbTnq9msns1Oj9RshWAjZgW4pDuO6e X-Proofpoint-ORIG-GUID: 4vPbTnq9msns1Oj9RshWAjZgW4pDuO6e X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E2C3B801C349 X-Stat-Signature: aptco587xru6bqw4ccmcrawnhd1jgtja Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b="aNO/CCoh"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="dx/jByQ2"; dmarc=pass (policy=none) header.from=oracle.com; spf=none (imf06.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=liam.howlett@oracle.com X-HE-Tag: 1633397528-148380 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Liam R. Howlett" Don't use the mm_struct linked list or the vma->vm_next in prep for removal Signed-off-by: Liam R. Howlett --- fs/userfaultfd.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 003f0d31743e..bd181f922999 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -606,14 +606,18 @@ static void userfaultfd_event_wait_completion(struct userfaultfd_ctx *ctx, if (release_new_ctx) { struct vm_area_struct *vma; struct mm_struct *mm = release_new_ctx->mm; + MA_STATE(mas, &mm->mm_mt, 0, 0); /* the various vma->vm_userfaultfd_ctx still points to it */ mmap_write_lock(mm); - for (vma = mm->mmap; vma; vma = vma->vm_next) + mas_lock(&mas); + mas_for_each(&mas, vma, ULONG_MAX) { if (vma->vm_userfaultfd_ctx.ctx == release_new_ctx) { vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX; vma->vm_flags &= ~__VM_UFFD_FLAGS; } + } + mas_unlock(&mas); mmap_write_unlock(mm); userfaultfd_ctx_put(release_new_ctx); @@ -798,7 +802,10 @@ int userfaultfd_unmap_prep(struct vm_area_struct *vma, unsigned long start, unsigned long end, struct list_head *unmaps) { - for ( ; vma && vma->vm_start < end; vma = vma->vm_next) { + MA_STATE(mas, &vma->vm_mm->mm_mt, vma->vm_start, vma->vm_start); + + rcu_read_lock(); + mas_for_each(&mas, vma, end) { struct userfaultfd_unmap_ctx *unmap_ctx; struct userfaultfd_ctx *ctx = vma->vm_userfaultfd_ctx.ctx; @@ -817,6 +824,7 @@ int userfaultfd_unmap_prep(struct vm_area_struct *vma, unmap_ctx->end = end; list_add_tail(&unmap_ctx->list, unmaps); } + rcu_read_unlock(); return 0; } @@ -848,6 +856,7 @@ static int userfaultfd_release(struct inode *inode, struct file *file) /* len == 0 means wake all */ struct userfaultfd_wake_range range = { .len = 0, }; unsigned long new_flags; + MA_STATE(mas, &mm->mm_mt, 0, 0); WRITE_ONCE(ctx->released, true); @@ -863,9 +872,11 @@ static int userfaultfd_release(struct inode *inode, struct file *file) * taking the mmap_lock for writing. */ mmap_write_lock(mm); + mas_lock(&mas); prev = NULL; - for (vma = mm->mmap; vma; vma = vma->vm_next) { + mas_for_each(&mas, vma, ULONG_MAX) { cond_resched(); + BUG_ON(!!vma->vm_userfaultfd_ctx.ctx ^ !!(vma->vm_flags & __VM_UFFD_FLAGS)); if (vma->vm_userfaultfd_ctx.ctx != ctx) { @@ -885,6 +896,7 @@ static int userfaultfd_release(struct inode *inode, struct file *file) vma->vm_flags = new_flags; vma->vm_userfaultfd_ctx = NULL_VM_UFFD_CTX; } + mas_unlock(&mas); mmap_write_unlock(mm); mmput(mm); wakeup: @@ -1281,6 +1293,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx, bool found; bool basic_ioctls; unsigned long start, end, vma_end; + MA_STATE(mas, &mm->mm_mt, 0, 0); user_uffdio_register = (struct uffdio_register __user *) arg; @@ -1323,6 +1336,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx, goto out; mmap_write_lock(mm); + mas_lock(&mas); vma = find_vma_prev(mm, start, &prev); if (!vma) goto out_unlock; @@ -1348,7 +1362,8 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx, */ found = false; basic_ioctls = false; - for (cur = vma; cur && cur->vm_start < end; cur = cur->vm_next) { + mas_set(&mas, vma->vm_start); + mas_for_each(&mas, cur, end) { cond_resched(); BUG_ON(!!cur->vm_userfaultfd_ctx.ctx ^ @@ -1466,9 +1481,10 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx, skip: prev = vma; start = vma->vm_end; - vma = vma->vm_next; + vma = vma_next(mm, vma); } while (vma && vma->vm_start < end); out_unlock: + mas_unlock(&mas); mmap_write_unlock(mm); mmput(mm); if (!ret) { @@ -1511,6 +1527,7 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, bool found; unsigned long start, end, vma_end; const void __user *buf = (void __user *)arg; + MA_STATE(mas, &mm->mm_mt, 0, 0); ret = -EFAULT; if (copy_from_user(&uffdio_unregister, buf, sizeof(uffdio_unregister))) @@ -1529,6 +1546,7 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, goto out; mmap_write_lock(mm); + mas_lock(&mas); vma = find_vma_prev(mm, start, &prev); if (!vma) goto out_unlock; @@ -1554,7 +1572,8 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, */ found = false; ret = -EINVAL; - for (cur = vma; cur && cur->vm_start < end; cur = cur->vm_next) { + mas_set(&mas, vma->vm_start); + mas_for_each(&mas, cur, end) { cond_resched(); BUG_ON(!!cur->vm_userfaultfd_ctx.ctx ^ @@ -1640,9 +1659,10 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, skip: prev = vma; start = vma->vm_end; - vma = vma->vm_next; + vma = vma_next(mm, vma); } while (vma && vma->vm_start < end); out_unlock: + mas_unlock(&mas); mmap_write_unlock(mm); mmput(mm); out: