From patchwork Mon Aug 22 15:06:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12950722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C0E7C32774 for ; Mon, 22 Aug 2022 15:06:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F070C6B0088; Mon, 22 Aug 2022 11:06:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EB7006B0089; Mon, 22 Aug 2022 11:06:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6C888D0002; Mon, 22 Aug 2022 11:06:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AF7ED6B0088 for ; Mon, 22 Aug 2022 11:06:37 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9025816123E for ; Mon, 22 Aug 2022 15:06:37 +0000 (UTC) X-FDA: 79827555234.12.1005871 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf02.hostedemail.com (Postfix) with ESMTP id 2807A80055 for ; Mon, 22 Aug 2022 15:06:36 +0000 (UTC) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27MEphqZ016577; Mon, 22 Aug 2022 15:06:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2022-7-12; bh=Ngf70k76kRpRObHjvGvMUliwq0ETt1cTVrTppxrchEY=; b=pBRQHdLwmLWNKLGXxJwrsbL+11Vmyl5DMar6AKVtimbslOMbPErIzD1dZpK9zUAdk3uk RtayBgyIsd+NXCjpuc71U6Nrj4Zunpu07qR/Q+EZ1GtLQoHllmLSoy5P7190/moOV3kX FltsCmslge21s47G2xx2nlM6Ty52wLbqHkRFjnoDQC2o35poAlSiFBqwo+TKeMBbOLks LlTVZHI8CWpPEZtw8w1viCn4A5Pj7FTOqdVemsSJSHCH0Qc18Kr/+UE4YGQCLy0zrng5 jdDs4lr3i/5oFtePH24bwslmRcCU3M0Kna7K83pZy9d1wC1Xq0XwwNfwfJx3SmlGCq1o 6w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3j4bx681j7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 22 Aug 2022 15:06:30 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 27ME4Nab031868; Mon, 22 Aug 2022 15:06:29 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2168.outbound.protection.outlook.com [104.47.55.168]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3j3mjd3986-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 22 Aug 2022 15:06:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dsfKkaOnZFpkZMOZYmiIn4LMsGGkfTz0sKpfICClgcStTt+H8seCr0itNWYgfge6NxW+r8/WHDhmoAmz/KKsGIGbX/ln7uGLrCc6I2wHpgOR7VgVGBy/KP9okodL8jW2AqCN5b6bpZNo5TyaiSBL/tjMHdyuP+9pbF1rIB+C7/7xiRZGIcoxA4OOqQTNs06LZvkdo96UXjiJ9F8uGFVneFlhCbi5Tia9wGCfq+aNkyfYVmuUm8PHJ46/CYCYRXkIGbM3eEv9Rk4LqpFaElAiLv2UMyJ+JLgsrn8EKdrfa1rdeZIrFMlHhwQY/TourD2z+DhKSSZX1RqOsGZQRh9zhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ngf70k76kRpRObHjvGvMUliwq0ETt1cTVrTppxrchEY=; b=cXl1edtt21rkTGnIKCzTOPb0VCpAqcV/zXnT6W5rkkWwAIw3q/7BTRcJd3HTL5dk6ZMcvLThOX6VBfcO51O+hjUjicse6Aaks3Wpt7vePoiIB7rHCFQDIH4g8YdSQBjAuNZo0jwP/ns6RULLlnShJLw6oyKXcXf7xIMTa0NftmHfET0zt/WOX9CeeiYUTSukDlrscW1NnBjnUzPD3FsFAh/gi1cgFU3MNvPQAOi4qfL3tfewEXZW2+YPWoFZp1nkVaXdgZeLS0UlfkNVbGBdMjl4oYvil55jPKW3pEUQt+WT0/iGP4+mcOY4phNmnt4Cv7wlriDJxKXCP/U3Ot6wFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ngf70k76kRpRObHjvGvMUliwq0ETt1cTVrTppxrchEY=; b=D646UGegXKMvsgaBWeD0W5tSK8zyPqs4BKFHnrdEsKGfpLJ5TR8QiXKPDeP95EWVPw7opWJumvvBZnOgExL6fVRgG9+WBBmy3+1+pgWQ2CQ4ytQ7tn6gzmWS6L3CnGbPJW/0NAqQLUEfZCFFWVJOTlm1ZGQg3sBMhVihBdRsOBo= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by DM5PR10MB1433.namprd10.prod.outlook.com (2603:10b6:3:e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5546.21; Mon, 22 Aug 2022 15:06:27 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::3d34:ebb5:d9df:98b3]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::3d34:ebb5:d9df:98b3%5]) with mapi id 15.20.5546.023; Mon, 22 Aug 2022 15:06:27 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton CC: "Matthew Wilcox (Oracle)" , Liam Howlett Subject: [PATCH v13 43/70] fs/proc/task_mmu: stop using linked list and highest_vm_end Thread-Topic: [PATCH v13 43/70] fs/proc/task_mmu: stop using linked list and highest_vm_end Thread-Index: AQHYtji/m5mzimtmq0eu1ZrHTOCI6Q== Date: Mon, 22 Aug 2022 15:06:25 +0000 Message-ID: <20220822150128.1562046-44-Liam.Howlett@oracle.com> References: <20220822150128.1562046-1-Liam.Howlett@oracle.com> In-Reply-To: <20220822150128.1562046-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.35.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 897e1438-579d-418b-8def-08da844fe321 x-ms-traffictypediagnostic: DM5PR10MB1433:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: pP+SDxtkOULcuYVpns6mntc/WxA9qwZf+o0EoqvmafPr0NR+mfVVvwt/tcZRj3kOGky/N3iPKcGBKdNlCDQSN33KafL2fj00ZE1AlXrytKHuxC0LxrvrS2LOh35FRHuhGjwPNy5eTN7ar8abDCb86aqnqEaPhR4vs0igbTy+B7xelfUkt6K0cOzQ0MpfCnAAkAmoQ1/qrWHiXTNiOcvWcC1fWNgi73iR6Aof3g7dbDIlm08Y1xP6ufXg5iBzLjC9Fe7W23SIlofspAVYdv/+D1jL0fyqujYiCZZjoM7C1HL/JEDhuy3hV9Ll2E1iUpmEfMSbCMOOII8R7HxYWLIAELAjTrgOUjNjyY6e+tiZPSOKlkExS22Ab2o6kd6dEANZWzKQz6AZE6+y58v8OAWavGAgHA7FLoI2uYLw2xc5dgIEbZ+9mKhUebgOgbquClo3Op0T3NpYSQnTsBidof9bp9Sc3t9q8qfP8K/Hg69HEmxdK74y5zRbmYBW/b5qwFmu7N9OKHGQDll40n3Bdh+L5bImT66D9Vsi4dPr9vcbhCEd5d7Ge0xVUvQ9kmL2SosFTtAgCih+xgLC79sqfkCMW2c055tWYcXqQwuQOSRpB7MLEB05j4N30QIsSS8rXm2Hn4q8KHLveAmdgcMoBJRRwcmELor1+bfb8CFCUH5rYYFIr4dhnxA57VWzym5C8xMuWyVWJ/7swYGlf4PIR0sfT5mhW/whocntfebBxyH3uUHvd6QWd4gsmuhC4H4tPcf2OORhQELG4JdsJ8Zx2CH4og== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(346002)(376002)(396003)(136003)(39860400002)(366004)(5660300002)(8936002)(38100700002)(122000001)(36756003)(41300700001)(44832011)(6486002)(316002)(71200400001)(76116006)(91956017)(110136005)(8676002)(4326008)(64756008)(66446008)(66476007)(66556008)(66946007)(54906003)(478600001)(186003)(1076003)(107886003)(6512007)(26005)(83380400001)(2616005)(6506007)(2906002)(38070700005)(86362001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?x+xy7aFS4EHXUM2qojcFCX5?= =?iso-8859-1?q?MNVIRQL9TVaC44XQeu3IlEcHc1Um4M/qgBpEXOvr3SE6qSJJzvYeMBqGmwxa?= =?iso-8859-1?q?zo3jj5vP+YQ29Bmn3gk8rGAqWWu37WLAq+cPVCwF47AN8CayKjxKu4lg/WoU?= =?iso-8859-1?q?KVHCa5DO9wnEj7/NcFixjvsmKIMwMC8RF0rwuvwAfiQxmzpGECQS3Tqj1nnP?= =?iso-8859-1?q?ToVm/3aou2mnS1S40G6iJ1u3m2Ge5cICQg9UYU8ow7k48xeCu5/dq9B1kz/H?= =?iso-8859-1?q?OS+BCh9GQHr4qDK6pum9A26sWQAxwQO89y9eIC2gjfq7TMfcEuzcLHlTGpyW?= =?iso-8859-1?q?bLUy3e+OBnlYYLwAkFv5sQph1pmL9EqQ1tAzlgUbla6C4czIJVpGvI5BCeFf?= =?iso-8859-1?q?K7fH1k42X3KPHOXUMCTjO+Eu5cSp58aj+iIblA+vviZe9QDj9abiE7Z+yHbO?= =?iso-8859-1?q?ck5CejFJIfmhYYnO4JWwtPu2SKcsiylgNemSAO2dB3hWeSwh3aolq7/QcDEh?= =?iso-8859-1?q?LEgvTrUAd7gW2minNhDh/b46ARxwm9q536NQq4k64JswSYZfrnx3TvDn4xTg?= =?iso-8859-1?q?MAjh3yNw5dvL2OKy4knYEVM+/5QFF/X282iZBgHDKZwQkkJjRdWRVcB5ydff?= =?iso-8859-1?q?UolhQF9VMpKaKtG/DlE812R0nrcgYGhZsuFOxE1JgAtIX7oUi8lg9415VF43?= =?iso-8859-1?q?/OW0dyOxdIdU7C3E+xJUCYDGcMYgpQH/9dPki2KP7KpWJBVdLHg20VtzlG3A?= =?iso-8859-1?q?2+/XcKLYlzs53+Age7t5b8Wa3eICUrRLY8dv7sMRYIXwdt/6+wa+fQIjcJM8?= =?iso-8859-1?q?cm53L3xwbdSm1QURtaw9h0z1tCf1H36sjDC9Dgh1Eua0Tys9WuFIEGuhpm2q?= =?iso-8859-1?q?2NdUFENqUgTlIca1R9Fs9N0p6wVzt+KbJSqEnSil3qR0kE4Cwuo8mXofm7BB?= =?iso-8859-1?q?qLXHDFgqT2Aayi0tZF8ohf7GiWIEywqV5Vv18YhYKwfgtyY4h/OFG3sK04Uo?= =?iso-8859-1?q?K+/ML+rfia34D7AxLw3VeML6U2qTPsv7uDY7vScv7HqZwDAOhHcyjplkMW9i?= =?iso-8859-1?q?5SinBXUbiZnht0dzb6/AM/xaakjp9wCh3L4+7MfzWVQhn9BIqGz0OADUYmhN?= =?iso-8859-1?q?gx7yPb9rdycAoXbEYc9wzRk7sRCvB9R6MO17yO9QtlP+IZikxNOOoaShRAhn?= =?iso-8859-1?q?8QMyif7u8iroxi9l8ss7LicCT92YXYN4igfwMfzd+T2NxgJXz0Gpqvm49Xgx?= =?iso-8859-1?q?/fQzQBVitTMLr2J7iXoF56i7uYMDgN72CAH5l78rvXRnsufZppBuYLEQcjTc?= =?iso-8859-1?q?5m4Z9Clem9LfKs2FjZKIhBUStY/TRdstPenwra/vPlRHaeV0DaoHpIcvXc4W?= =?iso-8859-1?q?Pk1hzlzJeXgGkpLZDDkpOHGgNaBhZcd4j3piabM0ppNTc9kAMV+MWyIsfyO1?= =?iso-8859-1?q?ZgDSjP9sRMCL4T6+KKGidDr59eqiKSbLSGixkLXprGq3+bcdzlmbrj7oQIom?= =?iso-8859-1?q?O1pZxJgloyyiNGxExo/4IalPimCzq3oJlLE6UnYEMDY3YPDGZmkQrGJtRjwk?= =?iso-8859-1?q?V+qr9mnsW0Ij+sN0eUG4ZVW/zSb/iBJHjQN5Pam6DI24l/dfopTrl+AXoryz?= =?iso-8859-1?q?YFA43QGqB2IdwNUz1OFuuCIHrCQdvelTphBWgsw=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 897e1438-579d-418b-8def-08da844fe321 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Aug 2022 15:06:26.0194 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: N8vZKg7Anw5fHfVUaGj5CUdAPfIDU7pyoHMGj+pvGtEprN1IYyUfhQlw99IhjYoRfdoIv6ybU1/gtaQTxn59Xg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR10MB1433 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-22_09,2022-08-22_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2207270000 definitions=main-2208220065 X-Proofpoint-ORIG-GUID: lPC9tCxAgUcy7jdkNKI_9E1hZD5rO8FM X-Proofpoint-GUID: lPC9tCxAgUcy7jdkNKI_9E1hZD5rO8FM ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1661180797; a=rsa-sha256; cv=pass; b=3ggmYeTUYoRKyFEVaVInOjqoYXJWuQDE8ffdf3IVxeNGis0qu15inl6++DMnT6K+GZ9Cfh 5L59dUUjxLUQj+M9QOxBJV81pOk8pvYB+4p5ciZ/pZV+z8tzFgDNJOzdRKiVmHHyeCNNwT ICsTunpv3fbTc1R2oQVZrLFpMBtJ+/U= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=pBRQHdLw; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=D646UGeg; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf02.hostedemail.com: domain of liam.howlett@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=liam.howlett@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661180797; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ngf70k76kRpRObHjvGvMUliwq0ETt1cTVrTppxrchEY=; b=Vrl0abYkYKNzkuuTZqt/nWOC0EHs8xBvTGf5BsMTtU8SOlePpAEGya/sz/9MlKvsZXewgU dOTuz/FjTWFy2kJLks7/WQMZqv/BltYxsmsSaMtO3Y3BEFP46SWme2Jct+QRekOREzgJcl A/bbN0E0SVihWgp8xRlO2Ou7wi90Y2Q= X-Rspam-User: Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=pBRQHdLw; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=D646UGeg; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf02.hostedemail.com: domain of liam.howlett@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=liam.howlett@oracle.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2807A80055 X-Stat-Signature: 7b4heo9oj3u3jwx73cjcprk5xgzfk91p X-HE-Tag: 1661180796-595886 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Remove references to mm_struct linked list and highest_vm_end for when they are removed Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Liam R. Howlett --- fs/proc/internal.h | 2 +- fs/proc/task_mmu.c | 73 ++++++++++++++++++++++++++-------------------- 2 files changed, 42 insertions(+), 33 deletions(-) diff --git a/fs/proc/internal.h b/fs/proc/internal.h index 06a80f78433d..f03000764ce5 100644 --- a/fs/proc/internal.h +++ b/fs/proc/internal.h @@ -285,7 +285,7 @@ struct proc_maps_private { struct task_struct *task; struct mm_struct *mm; #ifdef CONFIG_MMU - struct vm_area_struct *tail_vma; + struct vma_iterator iter; #endif #ifdef CONFIG_NUMA struct mempolicy *task_mempolicy; diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 9f70bc1c2766..8b4f3073f8f5 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -123,12 +123,26 @@ static void release_task_mempolicy(struct proc_maps_private *priv) } #endif +static struct vm_area_struct *proc_get_vma(struct proc_maps_private *priv, + loff_t *ppos) +{ + struct vm_area_struct *vma = vma_next(&priv->iter); + + if (vma) { + *ppos = vma->vm_start; + } else { + *ppos = -2UL; + vma = get_gate_vma(priv->mm); + } + + return vma; +} + static void *m_start(struct seq_file *m, loff_t *ppos) { struct proc_maps_private *priv = m->private; unsigned long last_addr = *ppos; struct mm_struct *mm; - struct vm_area_struct *vma; /* See m_next(). Zero at the start or after lseek. */ if (last_addr == -1UL) @@ -152,31 +166,21 @@ static void *m_start(struct seq_file *m, loff_t *ppos) return ERR_PTR(-EINTR); } + vma_iter_init(&priv->iter, mm, last_addr); hold_task_mempolicy(priv); - priv->tail_vma = get_gate_vma(mm); - - vma = find_vma(mm, last_addr); - if (vma) - return vma; + if (last_addr == -2UL) + return get_gate_vma(mm); - return priv->tail_vma; + return proc_get_vma(priv, ppos); } static void *m_next(struct seq_file *m, void *v, loff_t *ppos) { - struct proc_maps_private *priv = m->private; - struct vm_area_struct *next, *vma = v; - - if (vma == priv->tail_vma) - next = NULL; - else if (vma->vm_next) - next = vma->vm_next; - else - next = priv->tail_vma; - - *ppos = next ? next->vm_start : -1UL; - - return next; + if (*ppos == -2UL) { + *ppos = -1UL; + return NULL; + } + return proc_get_vma(m->private, ppos); } static void m_stop(struct seq_file *m, void *v) @@ -876,16 +880,16 @@ static int show_smaps_rollup(struct seq_file *m, void *v) { struct proc_maps_private *priv = m->private; struct mem_size_stats mss; - struct mm_struct *mm; + struct mm_struct *mm = priv->mm; struct vm_area_struct *vma; - unsigned long last_vma_end = 0; + unsigned long vma_start = 0, last_vma_end = 0; int ret = 0; + MA_STATE(mas, &mm->mm_mt, 0, 0); priv->task = get_proc_task(priv->inode); if (!priv->task) return -ESRCH; - mm = priv->mm; if (!mm || !mmget_not_zero(mm)) { ret = -ESRCH; goto out_put_task; @@ -898,8 +902,13 @@ static int show_smaps_rollup(struct seq_file *m, void *v) goto out_put_mm; hold_task_mempolicy(priv); + vma = mas_find(&mas, 0); + + if (unlikely(!vma)) + goto empty_set; - for (vma = priv->mm->mmap; vma;) { + vma_start = vma->vm_start; + do { smap_gather_stats(vma, &mss, 0); last_vma_end = vma->vm_end; @@ -908,6 +917,7 @@ static int show_smaps_rollup(struct seq_file *m, void *v) * access it for write request. */ if (mmap_lock_is_contended(mm)) { + mas_pause(&mas); mmap_read_unlock(mm); ret = mmap_read_lock_killable(mm); if (ret) { @@ -951,7 +961,7 @@ static int show_smaps_rollup(struct seq_file *m, void *v) * contains last_vma_end. * Iterate VMA' from last_vma_end. */ - vma = find_vma(mm, last_vma_end - 1); + vma = mas_find(&mas, ULONG_MAX); /* Case 3 above */ if (!vma) break; @@ -965,11 +975,10 @@ static int show_smaps_rollup(struct seq_file *m, void *v) smap_gather_stats(vma, &mss, last_vma_end); } /* Case 2 above */ - vma = vma->vm_next; - } + } while ((vma = mas_find(&mas, ULONG_MAX)) != NULL); - show_vma_header_prefix(m, priv->mm->mmap->vm_start, - last_vma_end, 0, 0, 0, 0); +empty_set: + show_vma_header_prefix(m, vma_start, last_vma_end, 0, 0, 0, 0); seq_pad(m, ' '); seq_puts(m, "[rollup]\n"); @@ -1262,6 +1271,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, return -ESRCH; mm = get_task_mm(task); if (mm) { + MA_STATE(mas, &mm->mm_mt, 0, 0); struct mmu_notifier_range range; struct clear_refs_private cp = { .type = type, @@ -1281,7 +1291,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, } if (type == CLEAR_REFS_SOFT_DIRTY) { - for (vma = mm->mmap; vma; vma = vma->vm_next) { + mas_for_each(&mas, vma, ULONG_MAX) { if (!(vma->vm_flags & VM_SOFTDIRTY)) continue; vma->vm_flags &= ~VM_SOFTDIRTY; @@ -1293,8 +1303,7 @@ static ssize_t clear_refs_write(struct file *file, const char __user *buf, 0, NULL, mm, 0, -1UL); mmu_notifier_invalidate_range_start(&range); } - walk_page_range(mm, 0, mm->highest_vm_end, &clear_refs_walk_ops, - &cp); + walk_page_range(mm, 0, -1, &clear_refs_walk_ops, &cp); if (type == CLEAR_REFS_SOFT_DIRTY) { mmu_notifier_invalidate_range_end(&range); flush_tlb_mm(mm);