From patchwork Wed Dec 1 14:30:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12650071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 564A0C433EF for ; Wed, 1 Dec 2021 14:48:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9012A6B00A4; Wed, 1 Dec 2021 09:32:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8AFDE6B00A5; Wed, 1 Dec 2021 09:32:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DB976B00A6; Wed, 1 Dec 2021 09:32:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 5DE0B6B00A4 for ; Wed, 1 Dec 2021 09:32:23 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1ED7F18036D45 for ; Wed, 1 Dec 2021 14:32:13 +0000 (UTC) X-FDA: 78869465346.28.FBAFFE3 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf17.hostedemail.com (Postfix) with ESMTP id 93BD3F00216C for ; Wed, 1 Dec 2021 14:31:30 +0000 (UTC) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1B1ECcS6002533; Wed, 1 Dec 2021 14:31:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=pJpse2ykoht5HtOmSmcEVVPiH7WSqrXnU0XB278v4/o=; b=xf43J+yHHBVB16VnrRgR/8wUE8kW3mEA+oN8g9bUhbDj6MHIm4cXnmCPJl0jlUaiwldm 9QemlK6zcyjsvCvb+uYljN1p9WLim8rpKKK/ly0wDkmArkzh/WovTf8iP2JxwDf0cT17 9HfzzbASjFF0hUNsaOGbSfrCIPdqgDDWcEINgPddJEZ9wRZZveYIkTg6SPU//5sDPsTi hK2ZORdANv+fcaoOj9EylfOjRnJh1PuQSApvxpXQOSTM5mSn+gvpVSywCTgdT8GFLamg PphD6JDIg3sLK4Xa5GN4Cj9UbvisDaADXB6+fyWAXs9pG9grjJ3Q3wMh/Ie5qA96BB97 Iw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3cmvmx2273-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 01 Dec 2021 14:31:20 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1B1EBlwa109866; Wed, 1 Dec 2021 14:31:18 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3020.oracle.com with ESMTP id 3cnhveqdaw-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 01 Dec 2021 14:31:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=REYGqLc4FPm3tVPZft8qACevkNDhapwliai41Z0WUnvIRotlOCq8IUI3ZmMoHeXAQxtm+wnuSJWa5kWrVgtvKm01tWyfBxL1Eg/zzcbACvN2zgGKp6aocJ2fVjWw1bUH0QJkbglE01Wz6fLlHqrZZVfGB1u0w1bXQMBSBYNwINmUdZxoRiZK9E3CzLFQQt1PnPsvgiB5FeqIcxPk4qi02/GCFHqsc8010k43bMe24rilypWHZKG+yT+Mrz7KqlYL0mfNYpb3vZGHWKHAkr+9o0PyxXCOwmbC8JIq2kZncvx9Dq3MScjXHGLRGTEPUfJYq8gphEPtVhWCWYyWpdFlhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pJpse2ykoht5HtOmSmcEVVPiH7WSqrXnU0XB278v4/o=; b=EmwXSTqZvUgNYGH/31QFzheY76/qUAysxVMhSai/l+e4F0TMzcX7YVoH0EK2pqk0PJQlE3RMq8uGXuuVt5RroNMJsOiXPUSoH+uxTRG4ZNZfm223Yp8aw8iHf1qO10J/VT9nFqrTxlQOpVA9nZ05Y/HKjKkW0rm1p+FdHa6p68BIZhQlkgf+a36zFN4iF8TkbL2HcqOQaLNjS33urznp2T/qDZwPDiI9SpynbReRr9JuvKVnLxib5o8diAg7vK+wnOB64LdqH2+4GNYL68udclO2knvCKXuSWdRkWkHzRPau+z2I02XjNFNUHmGLWN/foFOuRnInXmi3k0ueLLJd0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pJpse2ykoht5HtOmSmcEVVPiH7WSqrXnU0XB278v4/o=; b=Fuz2Ri6mpNdTv8GYuWOMUE+j5KFJ8PO9DmBDsOtiaYl8lCx3hL4e93qIvLCE83uosM7UEaE5FWw+W8nHEAxks0pZdeXyBN+xlc5lpt1Zs/YM7q7pPBhaJYYvJIFkmVIHrnDkwCZ7WfTSQvtKRwGaxb51EvQVcOyGrLD6T7rLGSw= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by SN6PR10MB2464.namprd10.prod.outlook.com (2603:10b6:805:4d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.22; Wed, 1 Dec 2021 14:31:13 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::c4d0:8291:84d:ac66]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::c4d0:8291:84d:ac66%7]) with mapi id 15.20.4734.027; Wed, 1 Dec 2021 14:31:13 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton CC: Song Liu , Davidlohr Bueso , "Paul E . McKenney" , Matthew Wilcox , Laurent Dufour , David Rientjes , Axel Rasmussen , Suren Baghdasaryan , Vlastimil Babka , Rik van Riel , Peter Zijlstra , Michel Lespinasse , Jerome Glisse , Minchan Kim , Joel Fernandes , Rom Lemarchand , Liam Howlett Subject: [PATCH v4 64/66] nommu: Remove uses of VMA linked list Thread-Topic: [PATCH v4 64/66] nommu: Remove uses of VMA linked list Thread-Index: AQHX5r/8HxPnsXSv40y1GuowyJAhjg== Date: Wed, 1 Dec 2021 14:30:27 +0000 Message-ID: <20211201142918.921493-65-Liam.Howlett@oracle.com> References: <20211201142918.921493-1-Liam.Howlett@oracle.com> In-Reply-To: <20211201142918.921493-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.30.2 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fbb3a0ca-2e82-4046-17b6-08d9b4d73a43 x-ms-traffictypediagnostic: SN6PR10MB2464: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:854; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: xcz7ArqRtCvXvI9cyJ5eIVdzdFuqeBpvugTZIoZQKHYVxSDanQitZPA0jMTWJJ/kL7INN9K9DbSt5E+e8TWrkS3WQ64FzZnp8g6gnFOXwyAr/Y3vLwf7tQjOhfSlOk12k2zDa/TrubeJOVSk3ANnYVc9amVXuTnzEu8F4swHplivewWGTFQKL6hI/Y4ca5Vbmz+FY+DMyTN8Y21abAGaYedyxQlZ+acfh5gU2AlFFgXAJcMX3uJvW9ZVInP+vgTEPakAdIK1cIEShbiI+8wdE7KIkoDVaGA0/wxCVDdCRCDoEGJ74tgUu2fWRRS3dy6WH1WP9AKDLbaiVPSG4AUB39KYjST/v+2jaC744D/dUWfbvEF5NfVjxRmM9fSEuaNPQcjWSqZ+WacW5dmnqWm7UNSrgxk6mymNrwpMWWMPho6H2pZsibCV44/lwPTu3HdzQE1lqogx1neJSpnIHV7CJ7DH+UnK+MUrBLuWj321JalEornogFmwYTkANf8hpSv+Am2dorHMDhfEbN+5oGIC7Ebmo/zSl/4+lsx32D9TYmOGGBuCiKu55ADENkDVcdEX4aL/bwqriytFJ3z7eKdzOeomPkNn+1EALO2LAOZ8xvivckSx7ysqdB09NiYaZ3UyyLzitcnOQjaOMXTowGqpVf36weY5YpD3CeWF4X+KiZ3AbSChCzbOIsztFlCSwdewAftrpL1tkqHs089DR3fN5Q== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(91956017)(6512007)(2906002)(1076003)(5660300002)(76116006)(83380400001)(186003)(6666004)(4326008)(26005)(38070700005)(36756003)(8676002)(7416002)(54906003)(107886003)(2616005)(316002)(86362001)(122000001)(44832011)(508600001)(8936002)(64756008)(66446008)(110136005)(38100700002)(66946007)(71200400001)(66556008)(6486002)(66476007)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?mHpL9Q1x0nHQxpkla00XRBJ?= =?iso-8859-1?q?yk5aO591ERlA7sHl3SOz+I7hHq6pGjDgmCXqZD6doYbyLYsvKmtkIK+nlxzt?= =?iso-8859-1?q?+J2UAhKMtzq59r9I3k0YnVcUF7olczyg8zAPxZS4a9elB6RhJwOq6hD6lK9N?= =?iso-8859-1?q?hWB02+geiCboQawSjKcu0Cs0qKZUIWa0XTzpNkwZqZdozsryywwR0IVNhyXO?= =?iso-8859-1?q?geqjF3utQ2jt2EorrT645KfN0E1UWRMcxz+NS8ZYOx5XuLZuDKb0UeIzn+8c?= =?iso-8859-1?q?OoSq7GeYyCOKXpzgPLLK/dem/tXO2cmuHKuObErJ4P9wLCOmbGn3Najxx9T/?= =?iso-8859-1?q?cbUr/BqdeqDiZRnvxbRf85ERoRF2crm8jq9/bRNAOKJC/vPl3udYSTjBJJDe?= =?iso-8859-1?q?XasBl/aTV/wjEJoNXX6L116rCFrang5RxRgRMs7OgW2dxLqfMqEkFrjuBJxx?= =?iso-8859-1?q?jCjp+XjSeDGTG23ZAPC5VFP+PltoL26lkzhRsxmh43zd2AlsQ3+w3kjbgMa3?= =?iso-8859-1?q?/MEqpWNyjkw7yANy1wm3NTHf8t3L7XDQHYV9mVkXdhAkSgcwZaqks8IbRVV3?= =?iso-8859-1?q?kYF6Q7tCja8m/DSHAQxriTew3htWZJfmxLB3y8JaKXQ/Q4SrmE1M9ng3DtzY?= =?iso-8859-1?q?o4NDF6MQKIn5MwPftojYGKxD1F3o+cMWT2lIXaUM31VnMWbSn4T88P4n9feo?= =?iso-8859-1?q?HUHmsEau0s6HTx6nq7HtfpuyAFN0qOsM5Sakt+BUonj7nPR7C6suQcXnuJdT?= =?iso-8859-1?q?MDKMQopqbzZpF5cQHJ9pONbANE5Rq1Pjq0jqX+HUwz/5yWlDLA3nEkr/ivFO?= =?iso-8859-1?q?7p8+bPnDkNWAjKaoCyr4wZiwk9F1WN7oRKLjxtQHDE18qXO89jTVTKOssUqE?= =?iso-8859-1?q?ioAkx60s0EQ+tmTcWd/ZqTF8QuHaCJlpWGh6+lwhFb/71IutNJtS4KJV0mPF?= =?iso-8859-1?q?9rhqVZC8HFoUUVG2os/OCrB2dFy/TvEVnx+jb615+qDhcEff+3p2SD1jU4fl?= =?iso-8859-1?q?9EnwUhq697QuhihrIfS5La2jTDvyccqgGKNCyw4gfyH07OMSL13byW/+ahIB?= =?iso-8859-1?q?NYj5qhBmka1SkiA4U/R/RDFLLhimvwgVxElrGQF4m4xVTCOT9o5XhgSEAbH6?= =?iso-8859-1?q?U7gWuOsoNqe7FvdOHrAYLzMlX4ct/a7YKaRQIsPrPZkxsId967mpMDmJxgSi?= =?iso-8859-1?q?zZsc9Vr47VZ4MBHMfbF7dBUtZrgI/u8E6L+SUo13Df7ADk9nTu6MrtUDXhI8?= =?iso-8859-1?q?0BDvfP1H6nUPsz5B9mQWqJzv5I1myOlBRcwS8jysWEAClqygjhHQD3ob2tOG?= =?iso-8859-1?q?Syqj25fcR7tNRK1a1ehXY3LQ86RNTIslrmP8nOs1TdzRU6COzMBdmI4whDaz?= =?iso-8859-1?q?OnbCveHQuGsQHXFGslizLrmMGsaLykNiCHeNkIoAZO80FmkKmlSaCvvqg3IA?= =?iso-8859-1?q?k4WKqjfX+fIkRbkeX7o56xfjmi4U1DoGNMYQTJu7mjCklANzIgnSFpmE4/kE?= =?iso-8859-1?q?mZFAtbBn8k2yOL+b4xrjhR4lZhgfPP9GGQk+epVPPzzJDXmjvQQngCWtWtSE?= =?iso-8859-1?q?NIyiVd2bsY21KblMGtDa/72qmLpXAQyhbDma7FzXdRAjSr/gjrBgzTlrycG5?= =?iso-8859-1?q?L8/3Pf4FgPhwKIRKjqHg4UolpFxuetZMTfUNkb3xXLg6yUjIhVwWTbqKY6qE?= =?iso-8859-1?q?=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: fbb3a0ca-2e82-4046-17b6-08d9b4d73a43 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Dec 2021 14:30:27.8781 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: nf35+jZm0pujtB2ym3Ku3fj8LrJC64gvEOiAPaNentHJVcbYaex0S62XYpCrh/ZJW4yXqnEzoMMzFQM3k86Qxg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2464 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10184 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112010081 X-Proofpoint-ORIG-GUID: BcSXbRvgMHqcV1iv8sUXop8WXNkCUeqM X-Proofpoint-GUID: BcSXbRvgMHqcV1iv8sUXop8WXNkCUeqM X-Rspamd-Queue-Id: 93BD3F00216C X-Stat-Signature: aobzaff89jwomn1qbry91wkk8r8j5b6p Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=xf43J+yH; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=Fuz2Ri6m; dmarc=pass (policy=none) header.from=oracle.com; spf=none (imf17.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.165.32) smtp.mailfrom=liam.howlett@oracle.com X-Rspamd-Server: rspam02 X-HE-Tag: 1638369090-918639 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Use the maple tree or VMA iterator instead. This is faster and will allow us to shrink the VMA. Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Liam R. Howlett Acked-by: Vlastimil Babka --- mm/nommu.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/mm/nommu.c b/mm/nommu.c index 11ae550834cd..acb9aafb0afc 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -1370,6 +1370,7 @@ static int shrink_vma(struct mm_struct *mm, */ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, struct list_head *uf) { + MA_STATE(mas, &mm->mm_mt, start, start); struct vm_area_struct *vma; unsigned long end; int ret; @@ -1381,7 +1382,7 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, struct list end = start + len; /* find the first potentially overlapping VMA */ - vma = find_vma(mm, start); + vma = mas_find(&mas, end - 1); if (!vma) { static int limit; if (limit < 5) { @@ -1400,7 +1401,7 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, struct list return -EINVAL; if (end == vma->vm_end) goto erase_whole_vma; - vma = vma->vm_next; + vma = mas_next(&mas, end - 1); } while (vma); return -EINVAL; } else { @@ -1449,6 +1450,7 @@ SYSCALL_DEFINE2(munmap, unsigned long, addr, size_t, len) */ void exit_mmap(struct mm_struct *mm) { + VMA_ITERATOR(vmi, mm, 0); struct vm_area_struct *vma; if (!mm) @@ -1456,12 +1458,14 @@ void exit_mmap(struct mm_struct *mm) mm->total_vm = 0; - while ((vma = mm->mmap)) { - mm->mmap = vma->vm_next; + mmap_write_lock(mm); + for_each_vma(vmi, vma) { delete_vma_from_mm(vma); delete_vma(mm, vma); cond_resched(); } + __mt_destroy(&mm->mm_mt); + mmap_write_unlock(mm); } int vm_brk(unsigned long addr, unsigned long len)