From patchwork Tue Jun 21 20:47:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12889806 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C917C433EF for ; Tue, 21 Jun 2022 20:48:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E03018E0044; Tue, 21 Jun 2022 16:48:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D12468E0043; Tue, 21 Jun 2022 16:48:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8600B8E0042; Tue, 21 Jun 2022 16:48:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3C20A8E0040 for ; Tue, 21 Jun 2022 16:48:01 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 1C71D601CF for ; Tue, 21 Jun 2022 20:48:01 +0000 (UTC) X-FDA: 79603429962.11.D106C97 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf02.hostedemail.com (Postfix) with ESMTP id A9EBA80024 for ; Tue, 21 Jun 2022 20:48:00 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25LJ8opG004752; Tue, 21 Jun 2022 20:47:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=JEfA7TueiDgfp5h1U06IfqgtjfOFfAG9vBixdDcPJwc=; b=IrbmpkDlcb5DWVd0imRzcxpreDebFPg3ui1nh4C8kFVOvIpiZImatwsf7mqfRXARkTQm 1p+HLjMYubYRqpKTvA4ClUwhdSUyM6Tp+bWSgUcp+gapuQ/fBjXrHVIC9uYGpLpaUajr yrydLBLYnECbn3BbL7zqC8kPwD/HZ3kwBSO9ukz+1SzwsqE/8GBhUHXfAI0DgUkS4LoT tpDg64LCkXh6cNRARNtMV60hiseiGJFpAkGochttC12eK9ev+uoD2WsHHmOxfTwAiU4A RgKtBv9EmiKfXQgdwVrJFl76+5wNP3MRVyqtRw8de/me2OV2w3LZ4pTrGUkkbwDw4Rje tw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3gs54cpnx3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 21 Jun 2022 20:47:53 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 25LKeiYF001682; Tue, 21 Jun 2022 20:47:52 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2107.outbound.protection.outlook.com [104.47.70.107]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3gtf5cutt1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 21 Jun 2022 20:47:52 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CCALOa1LK/nBKGf9feE6p/DoUAlUpP6SwTPsC/BzOwTmKlk2A7Tme6ssmqy4a6S0IIBdow4YKP/f7Z/TmCnI14eMeBFRkwZDbanGMjKKc6K/RiFUoRsUoWhCYW+Pkyi3SluZzgLuFCD6XXquVbASLKgj4p0IX6uF9JJJbGhr2RjPRj9cm8x6xWB4septQzbw4dEEH4BU049h+LTIwXW0TS0LcTFPdRtt6x0t+EX+LLv0avaHG2wP+4SaIkNO6mYSx5u5CkNaAWDxiHq6ciMcW9YncAFSIzmdbXkOo2KNHpCwutg2x8E+btkUs4BSX0kUV+1qiC6NEnWot+GthdwIKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JEfA7TueiDgfp5h1U06IfqgtjfOFfAG9vBixdDcPJwc=; b=XEkqOH+0N42wuE9SM/JmAxj7gYpv7PBAKnT0I003kMPWpPdDaqfueF0fmPo7cC/W9D5TNqJFAAm4tn79+GzYMQG1TKthz1bripA2W7HcYe9qeCAWaECjjozuuNAVrXQ4CMQ+BY4VpfElRsK58CovQYOinRUbN5tzP6pSgx6q7HjnJN1Kf1c1pSN2XR/6VjIqLRHBxD81fccP7asb8XOJ3lgIbqhPgguc5tQxHNGAOB0BTjriEuHDNSncjHCnKS5QN6Jwr5qPP7WSYjYPSl0schsa85M5moeWL2Okl5kk/kQErxuo/lzpqTRFeKG7XltT8eylzABCBEcjnIFecT9nmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JEfA7TueiDgfp5h1U06IfqgtjfOFfAG9vBixdDcPJwc=; b=EC2ANukTeM8DkZ/eqyWReOz/NggRA+fiBFAdvhHZqzL5cf5Lq8J3QlS/f4OayRSt6E5j/IgWNEnFJvVGI30LxbHfSrg+fJlQkHgZ1M46EQVWFBPwgINHw3GsRvOBE/weaDz2eM/xwE03NBZ2DTW+mlGayY2rwTYNhGElEn+Chq4= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by DS7PR10MB5392.namprd10.prod.outlook.com (2603:10b6:5:297::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.14; Tue, 21 Jun 2022 20:47:50 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::f59a:175d:d24:949c]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::f59a:175d:d24:949c%7]) with mapi id 15.20.5353.022; Tue, 21 Jun 2022 20:47:49 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton , "damon @ lists . linux . dev" , SeongJae Park , David Hildenbrand Subject: [PATCH v10 43/69] fs/proc/task_mmu: stop using linked list and highest_vm_end Thread-Topic: [PATCH v10 43/69] fs/proc/task_mmu: stop using linked list and highest_vm_end Thread-Index: AQHYhbAS9Zaz8pdLnkepDbelP4KxUA== Date: Tue, 21 Jun 2022 20:47:07 +0000 Message-ID: <20220621204632.3370049-44-Liam.Howlett@oracle.com> References: <20220621204632.3370049-1-Liam.Howlett@oracle.com> In-Reply-To: <20220621204632.3370049-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.35.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f1d39e52-2936-4fa8-ad71-08da53c74de8 x-ms-traffictypediagnostic: DS7PR10MB5392:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qrW5pzuyBlFQonOr242Be8cMWQ7jn8KsO4+ACGlb9Gj/OMy5PnZyaFYThrmYjVtg8VD+dkQmjGfNIz27aaE/3FcErqFJErJ03oUqgIe2uJlmH/zE1rd69pk9WRR3HGso+hsAFF8YqKIQBoQCrDogDk2ViEyvhgfe/2b0Suj7Y5isYp3yQpLKfcMlLorrD8Ks34E6VHXyTafA8AHigD9zHyC4TkYasW1RIW3a/8b5TKf6GrljJ71lFamBEG3LBkkdxaKUGtcxN4kMvwKRcT2RyiDKJEpa+vM3niu2ob0vF/L+CC4JK7MsWSWXUeXFjakw/eMLgdp5AX3GylDmziBDibvDno6+i6IS1oUSg+Gg00mhqYzqqDpMc3h/tm/vNgUUE4rAJCZnWeYQPp/BR5xBIUQiIhejC0z4uaVYNrSUU2UBhT2OJEBi1yc0MZQLoywNdieaJsVZStUd7PSlV71knLf9aK7rpQ5kSKtC6U3pIRBKD8Cd1CafD3rVJ1rjOFRRJ3WlcuC5BxGpAaO0wkBAgUhuf76/Hm2xpkZEd/Us4rpJxdYETYKoFr35Xm/0Z0ObCloawvEqfn/e3viGJcPX7V09DDX26TZfTqWIXcMVZpzOpFgRy5Pd8vrW7AhG7LwvmgrJ7e7KdGE/f5UCHriOM6X50em8LmY0gGwYfmTciLLOCpXplvRvS2FFnAaFpWdRZz1PrNyZT8+GCqs/ijFtl6ep3DksFg9CNhtPkmLu37Tc0skRunj/4GSqzWS5BrkkdjF5y+pwFYCPxfUQbbv9fi8RdzwNkX5upDwSIww8kiQ= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(39860400002)(396003)(366004)(376002)(346002)(136003)(41300700001)(6486002)(38070700005)(2616005)(186003)(86362001)(26005)(6512007)(38100700002)(122000001)(36756003)(1076003)(83380400001)(316002)(66446008)(66946007)(2906002)(64756008)(44832011)(6666004)(6506007)(8936002)(66556008)(76116006)(66476007)(966005)(478600001)(8676002)(91956017)(71200400001)(110136005)(5660300002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?quqQr3yfXCI0nXUPywTjNyy?= =?iso-8859-1?q?1jWid/LSAS6xuy1jYzDAhO7UQBju5QaHYtnTHkxLpR8L8R8o+vL6Sy/OAp94?= =?iso-8859-1?q?ouTuXLUdb1wXWVBPo19sYh7cGdX8nO+x8Fc55KDaYiSl8iRrFLJfksAKbvR6?= =?iso-8859-1?q?bNNaSck3isS9Z2uvA6Kj2fR+n0pNLuB+ZXosAA2n8ElccixDOp4jdmcrYFfn?= =?iso-8859-1?q?W1Xm4D40Mix28KHP02wO1nIXKxB7yy5BNn8CilBnhvEsQXn0HYILCvR0dnTe?= =?iso-8859-1?q?rBcnbB16jwZtRg3SFKeIXrpMxKyyFrb3P2vudZif2plneTp5gDioPox8Vdf+?= =?iso-8859-1?q?wObRMM+VClZoiD1lqax8YVFJo+X0tjoUZ8yAabHOUYAnjJ/9zftA3UKVBelA?= =?iso-8859-1?q?4SyI6EvvwkjLkZsd0u23v/I86IT4pSxWVB747WOzgfyDzqwFDb7zb9bIubSE?= =?iso-8859-1?q?orAU4+ENEpjM0vYmBREkjjh3gznENtFGXK7C1ANzrOz4qtWfLs6tIfVbeaut?= =?iso-8859-1?q?d3HKTPoBlaf7yyhASrp7Wk3hY3RKJeQk8oi4+yqTKpG085DCSRZ6m0d64Z0v?= =?iso-8859-1?q?HV2h6eTPztNkTfrdRhmU+kMjW6cH/O1G1sYsN9Erf57tosjPPehMY3W/jNXM?= =?iso-8859-1?q?ukq01fp3rSBKVJ1Nm2T97krT71ynsJVoygboXBF9hlHcuIQmiEWq8Aqkqw5c?= =?iso-8859-1?q?NM0MC1dj8xMiBuYoTKj3hr4/04DnEH72orvwnkbyCnVK0WXaOVd+NrSxkKlh?= =?iso-8859-1?q?aZLpPUmKeCVE0njtHKMab9eLhbpXNSRFL4lDG7Q6zSE0qbGFMuR0fD/Yn4xv?= =?iso-8859-1?q?p3ioVUPUH9i5AOzXzxXAMlvGrnV0FWcOTs4PkEdYsrLONEd/J0d3YIlLsSUU?= =?iso-8859-1?q?Ot1q/WxhNnmr6r5Le0aJXlg4i4YZJZDJT5gLmobYyA8GLxGYudv9CrpZtEVO?= =?iso-8859-1?q?mgHOH9m9SgknQbBnzsiE4PYKeo7Jrv7DKqay35LH31lN6H4L5IliU2Kxue1n?= =?iso-8859-1?q?oDRaL0Kx72HajhDJZYIpWEriLKtj3p7rAYCgJ2hWQ/setcqhzkSc/DWQEolQ?= =?iso-8859-1?q?esIs2WB7848nvqa+UzwKFBAYV3Kn1cDb9EvuIHgmg0t23qlubbsXEpTEVOyz?= =?iso-8859-1?q?QXLDVYQ3LPicpUMZ9Pfjyl3fCBikp8H/wW7e//xpVnaa5HX5kWlHhbQDw2KV?= =?iso-8859-1?q?sMbkfkeOaLBBA3IE6q06esJ8Av2XufTvz+nnBdoBWckhHlPMNhMkFHezQ4qA?= =?iso-8859-1?q?viFNK/iO8n1vV4ehUpHbdcIS3TDsOZUAGIbeQivJmpeyWxEnCYIvlIcb3vVh?= =?iso-8859-1?q?u8U7q3Q8Ubk9vbEbBF41+48WHLvDYWeMknfYrCXY6py6kL4POb3jubVvqpKb?= =?iso-8859-1?q?F4j+vdpUckkcZQ/pDsbgLh8NVR1te0Iu0JCyDTKNC2tXGe0WseME5WDzAfuw?= =?iso-8859-1?q?oVX5ED3UwI+qbML95dR+AU8n8NHD6vPH3J6dkLhWTLYwgjE05WIJ3HqGR4q5?= =?iso-8859-1?q?ed+TXzGK30JVih8nfPOdargWZz/0sAOSHvho+auz0+yx32V7GhFU+rg2Kh4Q?= =?iso-8859-1?q?QpbJtGSZdA2k88i38icbIoUWZR9QcihbPK7PAsSg1iSlVl7QwSXxsogtedPJ?= =?iso-8859-1?q?bQsxTUhTB2PuOayNYZdg18MO+xr1o4b/BX/zURpmfTWNfypz1eI91mCFnuaK?= =?iso-8859-1?q?hrwN2tExwp42WbCVnjct2My5rhp/PN/cjzCGL5j5lXzQ2kOrQeczeE7L7a6o?= =?iso-8859-1?q?oDQlVrFAfbyCjhG2RC4ysVnO2tF8m9f3JzWy1YQkSjHyZMwA2Y5nK5H1x5v+?= =?iso-8859-1?q?fntKgmSQ=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f1d39e52-2936-4fa8-ad71-08da53c74de8 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2022 20:47:07.3316 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: v5drfWDuEiT1HBeu47g5k2wszvFJGBLF+bWN/iuJ1RfcPGLWRwFFj6imkQLDS+A9cmEBo/hWG7/oQbCWv9qtqA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB5392 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883 definitions=2022-06-21_09:2022-06-21,2022-06-21 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=974 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2206210087 X-Proofpoint-GUID: C_Y2lG0ORKGqhkRsSV7LCRLUxl1smF9C X-Proofpoint-ORIG-GUID: C_Y2lG0ORKGqhkRsSV7LCRLUxl1smF9C ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1655844480; a=rsa-sha256; cv=pass; b=RIHIAv3GwjCJ1AsSQZ/kLBiIe4jVpZmwpjLLGZoHUMohpI1Pq4XW/onUkhk2Vb7hiCAcwi LGxxWheGHKMwO03txBc7fTX7dLP41GvtvsCpVUEd1+PQzp+k4+xhnT/GVMOfLSzyys2YtD FLok0NLp8KXGnXC2r4PHZ2mNEiiRV6o= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=IrbmpkDl; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=EC2ANukT; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf02.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=liam.howlett@oracle.com; dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655844480; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JEfA7TueiDgfp5h1U06IfqgtjfOFfAG9vBixdDcPJwc=; b=A4I4+lltUXvRVQ9qVMDBZ5AfOp8EfHqxQgdZRb6rWOY+kuaFsWjvlRV9D/Dxz9JE/QPDS+ KtfDxOysuHSuvbJZC7o3ewdgdTnBkciv3gAvF71d1+P9C9981vH2I8+J/ThjXsvOlDYFJG fg/rRzYztbmsYaD3Aw1qfZqCYWDjnPY= X-Stat-Signature: sd54z6b9wajmy9nr5b1nmh7mabmhn6nc X-Rspamd-Queue-Id: A9EBA80024 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=IrbmpkDl; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=EC2ANukT; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf02.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=liam.howlett@oracle.com; dmarc=pass (policy=none) header.from=oracle.com X-Rspamd-Server: rspam07 X-Rspam-User: X-HE-Tag: 1655844480-393839 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Remove references to mm_struct linked list and highest_vm_end for when they are removed Link: https://lkml.kernel.org/r/20220504011345.662299-28-Liam.Howlett@oracle.com Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Liam R. Howlett Cc: Catalin Marinas Cc: David Howells Cc: SeongJae Park Cc: Vlastimil Babka Cc: Will Deacon Cc: Davidlohr Bueso Signed-off-by: Andrew Morton --- fs/proc/internal.h | 2 +- fs/proc/task_mmu.c | 73 ++++++++++++++++++++++++++-------------------- 2 files changed, 42 insertions(+), 33 deletions(-) diff --git a/fs/proc/internal.h b/fs/proc/internal.h index 06a80f78433d..f03000764ce5 100644 --- a/fs/proc/internal.h +++ b/fs/proc/internal.h @@ -285,7 +285,7 @@ struct proc_maps_private { struct task_struct *task; struct mm_struct *mm; #ifdef CONFIG_MMU - struct vm_area_struct *tail_vma; + struct vma_iterator iter; #endif #ifdef CONFIG_NUMA struct mempolicy *task_mempolicy; diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 616913bb1fa7..f9c9abb50bb7 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -123,12 +123,26 @@ static void release_task_mempolicy(struct proc_maps_private *priv) } #endif +static struct vm_area_struct *proc_get_vma(struct proc_maps_private *priv, + loff_t *ppos) +{ + struct vm_area_struct *vma = vma_next(&priv->iter); + + if (vma) { + *ppos = vma->vm_start; + } else { + *ppos = -2UL; + vma = get_gate_vma(priv->mm); + } + + return vma; +} + static void *m_start(struct seq_file *m, loff_t *ppos) { struct proc_maps_private *priv = m->private; unsigned long last_addr = *ppos; struct mm_struct *mm; - struct vm_area_struct *vma; /* See m_next(). Zero at the start or after lseek. */ if (last_addr == -1UL) @@ -152,31 +166,21 @@ static void *m_start(struct seq_file *m, loff_t *ppos) return ERR_PTR(-EINTR); } + vma_iter_init(&priv->iter, mm, last_addr); hold_task_mempolicy(priv); - priv->tail_vma = get_gate_vma(mm); - - vma = find_vma(mm, last_addr); - if (vma) - return vma; + if (last_addr == -2UL) + return get_gate_vma(mm); - return priv->tail_vma; + return proc_get_vma(priv, ppos); } static void *m_next(struct seq_file *m, void *v, loff_t *ppos) { - struct proc_maps_private *priv = m->private; - struct vm_area_struct *next, *vma = v; - - if (vma == priv->tail_vma) - next = NULL; - else if (vma->vm_next) - next = vma->vm_next; - else - next = priv->tail_vma; - - *ppos = next ? next->vm_start : -1UL; - - return next; + if (*ppos == -2UL) { + *ppos = -1UL; + return NULL; + } + return proc_get_vma(m->private, ppos); } static void m_stop(struct seq_file *m, void *v) @@ -872,16 +876,16 @@ static int show_smaps_rollup(struct seq_file *m, void *v) { struct proc_maps_private *priv = m->private; struct mem_size_stats mss; - struct mm_struct *mm; + struct mm_struct *mm = priv->mm; struct vm_area_struct *vma; - unsigned long last_vma_end = 0; + unsigned long vma_start = 0, last_vma_end = 0; int ret = 0; + MA_STATE(mas, &mm->mm_mt, 0, 0); priv->task = get_proc_task(priv->inode); if (!priv->task) return -ESRCH; - mm = priv->mm; if (!mm || !mmget_not_zero(mm)) { ret = -ESRCH; goto out_put_task; @@ -894,8 +898,13 @@ static int show_smaps_rollup(struct seq_file *m, void *v) goto out_put_mm; hold_task_mempolicy(priv); + vma = mas_find(&mas, 0); + + if (unlikely(!vma)) + goto empty_set; - for (vma = priv->mm->mmap; vma;) { + vma_start = vma->vm_start; + do { smap_gather_stats(vma, &mss, 0); last_vma_end = vma->vm_end; @@ -904,6 +913,7 @@ static int show_smaps_rollup(struct seq_file *m, void *v) * access it for write request. */ if (mmap_lock_is_contended(mm)) { + mas_pause(&mas); mmap_read_unlock(mm); ret = mmap_read_lock_killable(mm); if (ret) { @@ -947,7 +957,7 @@ static int show_smaps_rollup(struct seq_file *m, void *v) * contains last_vma_end. * Iterate VMA' from last_vma_end. */ - vma = find_vma(mm, last_vma_end - 1); + vma = mas_find(&mas, ULONG_MAX); /* Case 3 above */ if (!vma) break; @@ -961,11 +971,10 @@ static int show_smaps_rollup(struct seq_file *m, void *v) smap_gather_stats(vma, &mss, last_vma_end); } /* Case 2 above */ - vma = vma->vm_next; - } + } while ((vma = mas_find(&mas, ULONG_MAX)) != NULL); - show_vma_header_prefix(m, priv->mm->mmap->vm_start, - last_vma_end, 0, 0, 0, 0); +empty_set: + show_vma_header_prefix(m, vma_start, last_vma_end, 0, 0, 0, 0); seq_pad(m, ' '); seq_puts(m, "[rollup]\n"); @@ -1258,6 +1267,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, return -ESRCH; mm = get_task_mm(task); if (mm) { + MA_STATE(mas, &mm->mm_mt, 0, 0); struct mmu_notifier_range range; struct clear_refs_private cp = { .type = type, @@ -1277,7 +1287,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, } if (type == CLEAR_REFS_SOFT_DIRTY) { - for (vma = mm->mmap; vma; vma = vma->vm_next) { + mas_for_each(&mas, vma, ULONG_MAX) { if (!(vma->vm_flags & VM_SOFTDIRTY)) continue; vma->vm_flags &= ~VM_SOFTDIRTY; @@ -1289,8 +1299,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, 0, NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } - walk_page_range(mm, 0, mm->highest_vm_end, &clear_refs_walk_ops, - &cp); + walk_page_range(mm, 0, -1, &clear_refs_walk_ops, &cp); if (type == CLEAR_REFS_SOFT_DIRTY) { mmu_notifier_invalidate_range_end(&range); flush_tlb_mm(mm);