From patchwork Wed Dec 1 14:30:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12650029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5FBCC433F5 for ; Wed, 1 Dec 2021 14:36:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFDB06B0085; Wed, 1 Dec 2021 09:31:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AAD4E6B0087; Wed, 1 Dec 2021 09:31:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FFB46B0088; Wed, 1 Dec 2021 09:31:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id 7DD226B0085 for ; Wed, 1 Dec 2021 09:31:05 -0500 (EST) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 44F8089D2D for ; Wed, 1 Dec 2021 14:30:55 +0000 (UTC) X-FDA: 78869462070.31.93AA114 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf27.hostedemail.com (Postfix) with ESMTP id 131F2700194F for ; Wed, 1 Dec 2021 14:30:53 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1B1EK9EZ012065; Wed, 1 Dec 2021 14:30:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=JzKUWwbZwn43FV3hmpFRmczudy7ssUxHOA8AReXzeoU=; b=njLxwZm3MZ0EhK/NSsZxy4rBxKGyPptTOvexaXGorJQEpxqZIzoUalg4YV0uOZ4M/TP7 ynC0xyKJkIO66nt4enxdQHmDxVLhdIjwXeA46FFhr7DjeZTYPyx3+epRK/DLlALn3qvm lBYVy3L4yepfTuX9xBzZcSiVpR5hEA0GpEiLkzk5o6fefrw725AmpH0tBMBmrGQf4e45 5qC9vKD5NmsLiTByPrPnpr81UEDKXqBGmTuA7egOlNTA+RZJQlvlnm5u6ho65esyLUBR JFjJCyloQc/rCTKjQ/ML3cakD9L5E/Von+BSrgGGgtlKJiY6CKpGyRS/Q9KskQGOQp5E Ag== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3cp7t1h0x3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 01 Dec 2021 14:30:44 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1B1EBoq5037533; Wed, 1 Dec 2021 14:30:43 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2046.outbound.protection.outlook.com [104.47.57.46]) by userp3030.oracle.com with ESMTP id 3ck9t1v74d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 01 Dec 2021 14:30:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SiiOmnOau9piVf25dgFTIcFgMdLuxDEpQ0zGYVTIZ8uKKX3WLuYg3k5Qbkai7UkArCSVBFkh3AOOf+/tnEbzfxwNY3wEubWVNL/Xs0Yxj3yqx2J+eK4BhewNIff17w7jwIbjyhIcEFzfKG5eyfqp3iHqJ6QYCc+9VS6TGx5rThN8e5WQ4tPRHOaeXGF3Yt/CaOkctigg/ezz9vUnQnEPolagLZTcynki7FzBTcrvjl63x2n9BM0krJDWk9yPCNxJkmPmyYJ6llzat2bQiYWpt8AVO3tPYlqz/Rb8smRAjfWReL5wBOdV2xB03cNj6ajlCgvZrgDHHTSTkHoxdRKzDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JzKUWwbZwn43FV3hmpFRmczudy7ssUxHOA8AReXzeoU=; b=NIJkvcBNFOmLuRw0PmwI79LhzMk1Z37jHI569rdHLafqhEwsuGQPLXhBbIbPtLWE5s4+2g/MuHGIQI4/fZGn11vfCnXJKFphqmzmHXei0iryeHJNuVKT8VJCd1vuDQOhWUrKIachycOmFbugde3L+ju45zGzqnM7X0MzT5mCCCRQJpnJPKOyLjJ8j3KflXypdW2bFsE+ltQeZdNhtkn6l/5vGUKfuqDlQxsujBPtLWNVyX/MJS+xORyX0+puHD4HPe1bK6AmU+RWBK5TLB1MsKuHdFTtwB8meFVXuYZEEfxkqSsN1zgbjtPqfdU3rwu4SiTmfmakVKEHThzbHfb4zg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JzKUWwbZwn43FV3hmpFRmczudy7ssUxHOA8AReXzeoU=; b=CTSTLJJzZmUQh3EQ8/Ja+xtxCuApm8hqujy8oi/U056SgBShgSmt6hoTVzGT4yDrhevjYpfeNI7kl+ujXxslD0PgzVcEsOHGbPkQMjmCg0ALkeQzDhCa0oJpe2tD6DBSwNv4QJ2xZn1bVxBU4eJdclmaU+ZVlOkyrceAI1usj4Q= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by SN6PR10MB2464.namprd10.prod.outlook.com (2603:10b6:805:4d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.22; Wed, 1 Dec 2021 14:30:41 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::c4d0:8291:84d:ac66]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::c4d0:8291:84d:ac66%7]) with mapi id 15.20.4734.027; Wed, 1 Dec 2021 14:30:41 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton CC: Song Liu , Davidlohr Bueso , "Paul E . McKenney" , Matthew Wilcox , Laurent Dufour , David Rientjes , Axel Rasmussen , Suren Baghdasaryan , Vlastimil Babka , Rik van Riel , Peter Zijlstra , Michel Lespinasse , Jerome Glisse , Minchan Kim , Joel Fernandes , Rom Lemarchand , Liam Howlett Subject: [PATCH v4 55/66] mm/mempolicy: Use maple tree iterators instead of vma linked list Thread-Topic: [PATCH v4 55/66] mm/mempolicy: Use maple tree iterators instead of vma linked list Thread-Index: AQHX5r/0Jk+tXlVP4UiIr9o8XNMVtg== Date: Wed, 1 Dec 2021 14:30:14 +0000 Message-ID: <20211201142918.921493-56-Liam.Howlett@oracle.com> References: <20211201142918.921493-1-Liam.Howlett@oracle.com> In-Reply-To: <20211201142918.921493-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.30.2 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8fb325f3-220e-47c4-bc84-08d9b4d726cc x-ms-traffictypediagnostic: SN6PR10MB2464: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: OBKyYaOwDjxoVUmlIhbwwNd+Urk2P8P6zFQ2iFLIlEXtxXknPNIaK+njLrygpymwSa/j86hey334Fjv7rSCfCvcsVZvL8f6hxqK+rRQAdwAEQPJTGGE4uKjpFOL838/vuMGC1J54acuFTGKHEAObSzqlt75GVlcqQWYyLZZLRLSuUiRJkQw/lhIByploiu89JVvZkPFcFfNJXyYWUJjiyFkccy4PuqbKG8wr+/e0BNM1Co67EcFLwaMayhodDC7JQTXOIFP/nVtYIrNP1pYJVOF9lUfkbdPNpukTFb6PhV86Tnx5cz3cAKj2fi49S9MGNpDnGmu7tOfZoco+mtFabuuRw2mjGt/D5nEdJ90Wg23myJ5EqAfa1Le2uiWykUKkTbVhGvwjq6WBirixn0DVBBH88uSmcoxNTqISBYY6URoZ0LsXdXFrY0KIj2KAWwFlAoP6zAyxHDZ/3SODElnh77Y+TU4NLOwgSLb6UhCBkjtHlQWbg+jpEqLXt8/LGmUCNMA/I2ySeU8qLfK5molVomf6HI7QmDGSNkrQm0+zMuAxXRfUxrlCfcTXqTD17Y75T318r8VjVN+kiQ1Eue53cyRYoQs03L2Q5hdJKTyBRNL0iWYfR5+22vk43p8YbpYkolbKpKreVVNpFkZX5hoduVGMx5+M41h+4MXFSDiTOHfxltThpsaRpXtIkgzCIcwZIPnni8+u7tUs5A0tqwnCbA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(91956017)(6512007)(2906002)(1076003)(5660300002)(76116006)(83380400001)(186003)(6666004)(4326008)(26005)(38070700005)(36756003)(8676002)(7416002)(54906003)(107886003)(2616005)(316002)(86362001)(122000001)(44832011)(508600001)(8936002)(64756008)(66446008)(110136005)(38100700002)(66946007)(71200400001)(66556008)(6486002)(66476007)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?y11ZEHZTDQ3sMyU+7ziiM50?= =?iso-8859-1?q?Z4F7PebItQSRP3emY9KjGi3RIvV9r11witPWiapZ1A3oU7MyDlFWdowesZ/j?= =?iso-8859-1?q?Ob+Gh6LdmoLnOtczD0ue71lvly/NCFJCsNYQT+AVdId5fAVtVSaIedpLqzjk?= =?iso-8859-1?q?E24de7d3/WCmICG0cEVi+0bsStwD0soH/9zhKm02SvNBiF+Nlczk6EPVbwqO?= =?iso-8859-1?q?SGi0ZTM8G6bnI5e1CsgZTXiR23R/RffD7/VMVuSzPl3XVA7RG9iL7a/gc2SB?= =?iso-8859-1?q?Ri0+fTRca5C+VUEpk24xvek87gC7zemr3O5ULf+XWfphMaLDvzvSgr0j8NyD?= =?iso-8859-1?q?m0VoP6BfaEm88EZm59ZC+N1ZyaoWckZDEdlee4KQoT9eXdrqn6xPFkMG8Fg1?= =?iso-8859-1?q?oez+iDZL6rb+jD7xnoY4xmCmMwnpWs6fH9JniaWPXgB8iryZKldt1gFot1zn?= =?iso-8859-1?q?tdcqIKuRxu4df1bQ0FM4YNkyu1G3rp78JDkgifMzxruT5IiRXg31lZHWXkeP?= =?iso-8859-1?q?uOM3PDP10iwrmYuaULJHZa9c7JTO9klyC+pUAotFj1ELwmsAseckfIhFuqLc?= =?iso-8859-1?q?KcHCWbFLkzmmULoMhfcRu6LXPTvsleZX4y3eY8kCklcrjL6dv587uZUyPPUt?= =?iso-8859-1?q?zpmf3npY0IgLYYcU90gHkU/p61ZDTk1JptOzLQsOYNXoq4ZbE6uRsbn5TG+w?= =?iso-8859-1?q?0FkFETg+8/SySXwgEFYxANvbHdeK6XT4ad/kUa9338eQTcj0l0Ys/7klVyAd?= =?iso-8859-1?q?KV/rccGZNDrlIadvpU17yITdG4osY8i5WWKBfN1bn5xoXOpC5QQOe0B1MvQ7?= =?iso-8859-1?q?dLu9zktQwIUVfWZBiy0Gs6LRSpuL0dctGqLWS7+5BgrTamQk1KmnnFwQ1t+X?= =?iso-8859-1?q?kgSTQjki+rbK4pe3DtB/ZPFp19OHx2HMlEjLTqwQu7BHyRSsLtd9qBxUZ/TQ?= =?iso-8859-1?q?KqlF7uJ2BJzo8uQOtGHykRdATuLDUwVohL0hzocNjDvjw49FqPFLxyazCCSU?= =?iso-8859-1?q?0KztFJO301Nm7UD1/wmlDVCcEXaToU/kGQikVKtuN+jHInljEhNMPcdkniAS?= =?iso-8859-1?q?b6IBhsWlUrYovyZ7X48g4+S/BDFcLlm4FPokyb5uX18NyjAQVoyonB3ZBI9Q?= =?iso-8859-1?q?ckceo1SHavXdCbn2vmewXgEJAS0r8mxyEJzsDK+4Q1Ky9wg0aNtCOKisVuPD?= =?iso-8859-1?q?+BgnpjhTjqZexQqA6h3CqbGX7/hFDxwakSl5TcPpxcYKoz/TIedbNGBLvlCT?= =?iso-8859-1?q?2j1RQWh/ptqCdAxm0MP9HCjGMh3P3LDYxUQkDjTZIyxab9uJH4mtt0zeYnzN?= =?iso-8859-1?q?uWhA3Jj5wjAhbK9kwUAw3xVS+FimoiZxyBzr8/9G00M70NstcPmmEnIdQVtT?= =?iso-8859-1?q?iN9jttfjSwU+8205KzzGVb5txI2ejhGB9echpfYVqFWlor509pcuu0mBTtOx?= =?iso-8859-1?q?NQ550aOHkDn4dSO0+OlMvJMNpgmT9KKEmnqoeZpmzkwNcrHaVVfFAS4EHGFW?= =?iso-8859-1?q?63OViKiAkacCSjU+TGqrpcfQHX8oFO7KFsSL1McpCWL4CbwX2gnmZIp7YENA?= =?iso-8859-1?q?FxBET1g9aqys+qSBxnSHvnoCx3bSHW/MkUoVlamln4mjgldVZQwx24AFw/Vq?= =?iso-8859-1?q?UlEq4ATIepQToXyzy9Hkw7nOSNAeaY8gn9h+KJaazrHNYuM8g5gN6WSUZ1lI?= =?iso-8859-1?q?=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8fb325f3-220e-47c4-bc84-08d9b4d726cc X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Dec 2021 14:30:14.9615 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: KRSp+VlBAPRP69FnfGvQg2gapkMlN88enjtU59UFUlwkePoTL4dKwsPdvwR0Hq3TI5Ks7teTJMbJ8GIi+P8kOA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2464 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10184 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 malwarescore=0 spamscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2112010081 X-Proofpoint-ORIG-GUID: qneRTRhRpyVDpLmMLc6zIalFPhFNJltK X-Proofpoint-GUID: qneRTRhRpyVDpLmMLc6zIalFPhFNJltK X-Rspamd-Queue-Id: 131F2700194F X-Stat-Signature: 1cad93aw9r3hf3kkn539sp4gymi8f9mx Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=njLxwZm3; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=CTSTLJJz; dmarc=pass (policy=none) header.from=oracle.com; spf=none (imf27.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=liam.howlett@oracle.com X-Rspamd-Server: rspam02 X-HE-Tag: 1638369053-363049 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Liam R. Howlett" Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Liam R. Howlett --- mm/mempolicy.c | 53 ++++++++++++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 25 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 10e9c87260ed..0e2d918f4f30 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -377,9 +377,10 @@ void mpol_rebind_task(struct task_struct *tsk, const nodemask_t *new) void mpol_rebind_mm(struct mm_struct *mm, nodemask_t *new) { struct vm_area_struct *vma; + MA_STATE(mas, &mm->mm_mt, 0, 0); mmap_write_lock(mm); - for (vma = mm->mmap; vma; vma = vma->vm_next) + mas_for_each(&mas, vma, ULONG_MAX) mpol_rebind_policy(vma->vm_policy, new); mmap_write_unlock(mm); } @@ -652,7 +653,7 @@ static unsigned long change_prot_numa(struct vm_area_struct *vma, static int queue_pages_test_walk(unsigned long start, unsigned long end, struct mm_walk *walk) { - struct vm_area_struct *vma = walk->vma; + struct vm_area_struct *next, *vma = walk->vma; struct queue_pages *qp = walk->private; unsigned long endvma = vma->vm_end; unsigned long flags = qp->flags; @@ -667,9 +668,10 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, /* hole at head side of range */ return -EFAULT; } + next = find_vma(vma->vm_mm, vma->vm_end); if (!(flags & MPOL_MF_DISCONTIG_OK) && ((vma->vm_end < qp->end) && - (!vma->vm_next || vma->vm_end < vma->vm_next->vm_start))) + (!next || vma->vm_end < next->vm_start))) /* hole at middle or tail of range */ return -EFAULT; @@ -783,28 +785,24 @@ static int vma_replace_policy(struct vm_area_struct *vma, static int mbind_range(struct mm_struct *mm, unsigned long start, unsigned long end, struct mempolicy *new_pol) { - struct vm_area_struct *next; + MA_STATE(mas, &mm->mm_mt, start - 1, start - 1); struct vm_area_struct *prev; struct vm_area_struct *vma; int err = 0; pgoff_t pgoff; - unsigned long vmstart; - unsigned long vmend; - - vma = find_vma(mm, start); - VM_BUG_ON(!vma); - prev = vma->vm_prev; - if (start > vma->vm_start) - prev = vma; + prev = mas_find_rev(&mas, 0); + if (prev && (start < prev->vm_end)) + vma = prev; + else + vma = mas_next(&mas, end - 1); - for (; vma && vma->vm_start < end; prev = vma, vma = next) { - next = vma->vm_next; - vmstart = max(start, vma->vm_start); - vmend = min(end, vma->vm_end); + do { + unsigned long vmstart = max(start, vma->vm_start); + unsigned long vmend = min(end, vma->vm_end); if (mpol_equal(vma_policy(vma), new_pol)) - continue; + goto next; pgoff = vma->vm_pgoff + ((vmstart - vma->vm_start) >> PAGE_SHIFT); @@ -812,8 +810,8 @@ static int mbind_range(struct mm_struct *mm, unsigned long start, vma->anon_vma, vma->vm_file, pgoff, new_pol, vma->vm_userfaultfd_ctx); if (prev) { + mas_pause(&mas); vma = prev; - next = vma->vm_next; if (mpol_equal(vma_policy(vma), new_pol)) continue; /* vma_merge() joined vma && vma->next, case 8 */ @@ -823,19 +821,23 @@ static int mbind_range(struct mm_struct *mm, unsigned long start, err = split_vma(vma->vm_mm, vma, vmstart, 1); if (err) goto out; + mas_pause(&mas); } if (vma->vm_end != vmend) { err = split_vma(vma->vm_mm, vma, vmend, 0); if (err) goto out; + /* mas_pause() unnecessary as the loop is ending */ } - replace: +replace: err = vma_replace_policy(vma, new_pol); if (err) goto out; - } +next: + prev = vma; + } while ((vma = mas_next(&mas, end - 1)) != NULL); - out: +out: return err; } @@ -1053,6 +1055,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest, int flags) { nodemask_t nmask; + struct vm_area_struct *vma; LIST_HEAD(pagelist); int err = 0; struct migration_target_control mtc = { @@ -1068,8 +1071,9 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest, * need migration. Between passing in the full user address * space range and MPOL_MF_DISCONTIG_OK, this call can not fail. */ + vma = find_vma(mm, 0); VM_BUG_ON(!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL))); - queue_pages_range(mm, mm->mmap->vm_start, mm->task_size, &nmask, + queue_pages_range(mm, vma->vm_start, mm->task_size, &nmask, flags | MPOL_MF_DISCONTIG_OK, &pagelist); if (!list_empty(&pagelist)) { @@ -1198,13 +1202,12 @@ static struct page *new_page(struct page *page, unsigned long start) { struct vm_area_struct *vma; unsigned long address; + MA_STATE(mas, ¤t->mm->mm_mt, start, start); - vma = find_vma(current->mm, start); - while (vma) { + mas_for_each(&mas, vma, ULONG_MAX) { address = page_address_in_vma(page, vma); if (address != -EFAULT) break; - vma = vma->vm_next; } if (PageHuge(page)) {