Message ID | 20210428153542.2814175-73-Liam.Howlett@Oracle.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show
Return-Path: <SRS0=wPXd=JZ=kvack.org=owner-linux-mm@kernel.org> X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,PDS_BAD_THREAD_QP_64,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85992C433ED for <linux-mm@archiver.kernel.org>; Wed, 28 Apr 2021 15:38:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E52406023C for <linux-mm@archiver.kernel.org>; Wed, 28 Apr 2021 15:38:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E52406023C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 60473940010; Wed, 28 Apr 2021 11:37:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E09B94000E; Wed, 28 Apr 2021 11:37:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40A3D940010; Wed, 28 Apr 2021 11:37:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 1A0F494000E for <linux-mm@kvack.org>; Wed, 28 Apr 2021 11:37:39 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0DB9A12D2 for <linux-mm@kvack.org>; Wed, 28 Apr 2021 15:37:36 +0000 (UTC) X-FDA: 78082180512.04.6EE4E0A Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf20.hostedemail.com (Postfix) with ESMTP id D4D8D3C7 for <linux-mm@kvack.org>; Wed, 28 Apr 2021 15:37:26 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 13SFNedL010520; Wed, 28 Apr 2021 15:37:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=aojRHjAmDWZB31YVUdOjTPD0S2QtERZ5qmPhmsp2Gqg=; b=GNZU4+a8ML9C80GnKdCrchJKnAIuUBI5hPc60KqRSJRHumkqrJlMviDvk0nGNMxyUjAo gFKLXVordbmObTVqZETqpFpt1b6AXx+zzXnRBfPtfGyxWjaVlTc51YlkUGD9U5mWYxNP Y4gLFZ9fEg8YwPcZGOGcIJ9EUSzXKPg7dmdum9rAVtU37IcoB1h5xbgdCDb5+BJYvtvj XpA0PrIXwwQE5O5mvsiKVNRbVt14qkhiAR3+SnsxclKom7jPzX+oELYxORJGayGxs+22 9+r+gNdkkLN0eVMVm0nyEeVYSALxE8luaGYi/64iVd7QO4RIMCWHBc7yGEohtyL7dY9G Cg== Received: from oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 387891060f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Apr 2021 15:37:28 +0000 Received: from aserp3020.oracle.com (aserp3020.oracle.com [127.0.0.1]) by pps.podrdrct (8.16.0.36/8.16.0.36) with SMTP id 13SFbScp033031; Wed, 28 Apr 2021 15:37:28 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2172.outbound.protection.outlook.com [104.47.58.172]) by aserp3020.oracle.com with ESMTP id 384b58qpxp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Apr 2021 15:37:28 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MWE3zwIT1WeTWmVKIvIIGct04cIW/oLFSTLkjeq5i85LWtll2EEKSXWeq2BfgY6YxTmosLOaHbs78+iklidg7PknvVPwZhk8zSN9lnirO/JR727WK96zdUIOOJDv1+S7phUEG+3KeqIiybJKeFLV/9QXnKSwMyXG2pkCIhAPO9sU7pywPig5ewx17ccd11zE9eLsc7bNkwTeYEXvsnwcaG0blBz1Hc00JS/PlR2phz/5vDQzBdBOAjqaHVzUU4XK0u0S9qg1mWfYBAIqmhDtWOLGkbSjqdE5ypMXeMFl+lPyuYsiG8d4BA6B844Xr0W2l41QQsrLhOUzDhUoJ5OC2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aojRHjAmDWZB31YVUdOjTPD0S2QtERZ5qmPhmsp2Gqg=; b=Wq1HRIpRMLRfHKZUWgQenleyEhvsfrM8JcqkaKJYb1PnGLjVxScx6fOOBVQhY8rny91Yhn5APV6bcrsj0o9E8nxnSqsYxZ3BoQYbPqXrdpK4MDgc8HMLqBWD6BoH3y+2KByWmGoN8l1H/8C6axVeZGeAJasD2X+F33lbSpIQlSw7xNJJtTFN4eQio34EY0mY2jl2aHeyLw3iQumASMmnc2nhbLRrqyBX/KMyaKepcL3TPgHQjf3e+HfCyVoZIID9sThL7ABoPBHVtbeEH7dA6/J7h8V3OX/uEH9oCscLzv5GbK/saVtNGFmoj1K2qog66aZCB7SBYlArjbLxrMQXcw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aojRHjAmDWZB31YVUdOjTPD0S2QtERZ5qmPhmsp2Gqg=; b=NK31drPe82B6ogFNEnQD8PrYtjjdXxnVCmVOyVKtKP9HKpkC9bm2wydYSP1yt+nNX3D/hD31AT8q45eqCYIKJetGjXK0Voa/7wM1MlOnMpKgtYNc+boK8ly8pOA+lErUwF1DRoUqBS+2JHhvfkEyPIr6Ai2qImeMo1setUGzygI= Received: from MWHPR10MB1582.namprd10.prod.outlook.com (2603:10b6:300:22::8) by CO1PR10MB4548.namprd10.prod.outlook.com (2603:10b6:303:97::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.20; Wed, 28 Apr 2021 15:37:23 +0000 Received: from MWHPR10MB1582.namprd10.prod.outlook.com ([fe80::353a:1802:6e91:1811]) by MWHPR10MB1582.namprd10.prod.outlook.com ([fe80::353a:1802:6e91:1811%8]) with mapi id 15.20.4065.027; Wed, 28 Apr 2021 15:37:23 +0000 From: Liam Howlett <liam.howlett@oracle.com> To: "maple-tree@lists.infradead.org" <maple-tree@lists.infradead.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Andrew Morton <akpm@linux-foundation.org> CC: Song Liu <songliubraving@fb.com>, Davidlohr Bueso <dave@stgolabs.net>, "Paul E . McKenney" <paulmck@kernel.org>, Matthew Wilcox <willy@infradead.org>, Laurent Dufour <ldufour@linux.ibm.com>, David Rientjes <rientjes@google.com>, Axel Rasmussen <axelrasmussen@google.com>, Suren Baghdasaryan <surenb@google.com>, Vlastimil Babka <vbabka@suse.cz>, Rik van Riel <riel@surriel.com>, Peter Zijlstra <peterz@infradead.org>, Michel Lespinasse <walken.cr@gmail.com>, Liam Howlett <liam.howlett@oracle.com> Subject: [PATCH 72/94] kernel/sched/fair: Use maple tree iterators instead of linked list Thread-Topic: [PATCH 72/94] kernel/sched/fair: Use maple tree iterators instead of linked list Thread-Index: AQHXPEQ+GPt/wrmyP0ysDAwuPfKHtQ== Date: Wed, 28 Apr 2021 15:36:23 +0000 Message-ID: <20210428153542.2814175-73-Liam.Howlett@Oracle.com> References: <20210428153542.2814175-1-Liam.Howlett@Oracle.com> In-Reply-To: <20210428153542.2814175-1-Liam.Howlett@Oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.30.2 authentication-results: lists.infradead.org; dkim=none (message not signed) header.d=none;lists.infradead.org; dmarc=none action=none header.from=oracle.com; x-originating-ip: [23.233.25.87] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2d5a1966-7e70-4d2c-b863-08d90a5b84a0 x-ms-traffictypediagnostic: CO1PR10MB4548: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: <CO1PR10MB45484C9EFB66522863292951FD409@CO1PR10MB4548.namprd10.prod.outlook.com> x-ms-oob-tlc-oobclassifiers: OLM:655; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Tov92UsF8yKUwzZ+IK4ZP+/sIC9XByTAuxY6XCAKsk8cuKJDzH4g0XU/SXYiALfKnPk6UKe0Mic3Q0sjkqbBB2gH5kzuNHN+ANKzTJ0q5Spl+/I8sxXKdkRuhuQCL7gD6HKdkcZFWkIbMTIUyDd656VFXyTpU80Sob7+FfHvUHRuC1rdWYCbu4uGGK8W2YppLmSD6pEMOBXdhCJMoVvoUCQqhrEYWZYdlHlgXb7HryOB7fV53B/Qfe3P9+cczI/4Y/m3yvjtAE4T7RIPrpnHBNFgrIsez0XcBr/8UzYmkFLZSArfmA9a7iVWRYI3eh/VYASKHT/pE74CMUTe+a8ZMTnBIpTRDV56fpI9nrH/6WUUmTdutsALSlgj47d1bu1CTaUYb80rKsGvk34A5HX0rHDQr66b63T9zgyP1Jyg+kg0YCPDK1i7fkfLEVxRF5rPyCz47xHSQHW+h4hshtY1t0HesF92B1k5feaFiYokG607U/YXWqvOJENgtTA1kDa3jHLovvm0G5dLRBXNVCOOAKDUOokbOSbyKCQXbjmZNT9OpHXIJ0IS8nZifmJL7RgPTK2lizwYtovH7AdjhLjIduMWrSOehAdGRvuZeJIzFsM= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR10MB1582.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(376002)(366004)(136003)(396003)(346002)(39860400002)(6512007)(66946007)(8936002)(186003)(83380400001)(54906003)(86362001)(478600001)(122000001)(4326008)(76116006)(1076003)(107886003)(7416002)(110136005)(38100700002)(26005)(2906002)(2616005)(8676002)(66446008)(6486002)(71200400001)(5660300002)(316002)(44832011)(64756008)(91956017)(66476007)(66556008)(36756003)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?q?hRC9V3vSu87JEeeT3Zxbt7KvS?= =?iso-8859-1?q?Fp8QUlcO81UGUWAfVJOTS7MSaJwGFLrt6DnHxzk9FpxcVs24M8fJ2tRI7mfi?= =?iso-8859-1?q?N+7VSNOh/v3ncqBC9p8CvAa30atbHhnrq2GoFEQsctqjNjppSiTutiMRyBK9?= =?iso-8859-1?q?8dwjamW0hlyAIZJuQ2DFBlcqphMf9UGk7fiekaZJiJabUCEtmYpemcSbjBbx?= =?iso-8859-1?q?6UGyyXyscx1h+uzMdnGTcCvDS9R2dQejyZUyXF98byHRgC28Se6JdQuqP/9i?= =?iso-8859-1?q?447Rl2EQrACwGpoHhVmRaKhoJW4KliPCb2JASykXeTmYXwjj6JK332R/ILRg?= =?iso-8859-1?q?S1pJFZh4f05fOjklvx5eUcqqDHSdhjtQ+PpMtW9QvyPmEgPZitmkA31p/Ek8?= =?iso-8859-1?q?w7+1qLrqBW8ihM3jPd4CKdTB/XHvDqsamFNjZveqyPDEtBW4KC71kyUBSW2K?= =?iso-8859-1?q?M9VS1bqZFKlx0grA/R5KmgbGi5WZgdrAs7YAtqJZudYEOD7U0/kInJjz/MHg?= =?iso-8859-1?q?ujO89VfCX7vKVbrqK88f+k30j+oSOX11cbjg1W+Cilg6ELchfZ9acDXu8kQH?= =?iso-8859-1?q?jOlJLgw4EvBrmJm9I0IluQOeEptOMXAwRhaapIpDWaWIXwU1a9VTYxxErOC8?= =?iso-8859-1?q?br2rFHY14oLmPg6qVvJWLLFKiEjSyYUzSiTUKSK5VoejqCWfp7AOE/RoEfMr?= =?iso-8859-1?q?t01EZGYqSSf2fpINJ0zowhT4SewpAEX5UCXmC7eUDWm0QCG8nnPzBkEDHO7Q?= =?iso-8859-1?q?ljumt+GBon2Prkg8IXt8ptOB8OACZJFB/llhkReYMmHHXcwTducCBImZ6m9O?= =?iso-8859-1?q?+nWZ8WE+SdCM5m/KP6h4WWqKNbhTZeX+UgjF+fmt6g11gLiU0Oqp0jJPb1AQ?= =?iso-8859-1?q?dtMPt4v/sa/V6jBj3MXsKwdOX3SG+Dpro69WDF8asj0+eNcTRU/sdP+WpgWC?= =?iso-8859-1?q?KEDVK86dVsi2nZFyBxpRZztfXHaH/Yw9TsSvs1aIe0ranq6DLkpd0LCxClKh?= =?iso-8859-1?q?WxvqkAe9D88GvbkM9p89uJe3TAFAgVSJNJlPGQ46Sk7Ia1QygjhC/kNFktjA?= =?iso-8859-1?q?gEoqXC3fTlTTeZjlP5VqgNy5iQLWrwIOS3mq6UcEeYX3a5hwOX7QXYQkdaud?= =?iso-8859-1?q?SYsr4FQcmedV1WEZaRjVA6grC8geSaq/fdjfZdapcQOC0lOzy1cCuS+V4T9t?= =?iso-8859-1?q?/snwR/hIlvdjt+wUgmspFM4PC0WU0Aw8e9ZZxnvSKVamUi0o2528DhJhdZ1f?= =?iso-8859-1?q?70mCrW52Ylp0qSQkgtbjQilaRIO512vPsQ+XyRyx+lQ1GxQJsWHb5AoN6Wx5?= =?iso-8859-1?q?XF0BvlUwXA3aeAr/DWd074MCkA2jKUrAOjSPHbxwLlpiu0yf9v9fXG7CUUj?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR10MB1582.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2d5a1966-7e70-4d2c-b863-08d90a5b84a0 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Apr 2021 15:36:23.3168 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: //0+D11Sye+oj1/baisy8Ce38Nh4U0bQoNhzR+zVYli7XsvHUb7+PxxdRlBvmXTQ+Fwe+AJhOJisWG5h4FTgaA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4548 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9968 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 mlxscore=0 malwarescore=0 phishscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104280101 X-Proofpoint-GUID: VNkDgr0n8_WgotltFtV9i1yg8zWwAH4O X-Proofpoint-ORIG-GUID: VNkDgr0n8_WgotltFtV9i1yg8zWwAH4O X-Rspamd-Queue-Id: D4D8D3C7 X-Stat-Signature: e69bthro3igsn9zohdcnqtwkgaqr4ecs X-Rspamd-Server: rspam02 Received-SPF: none (oracle.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from="<liam.howlett@oracle.com>"; helo=mx0b-00069f02.pphosted.com; client-ip=205.220.177.32 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619624246-578195 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: <linux-mm.kvack.org> |
Series |
Introducing the Maple Tree
|
expand
|
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9b8ae02f1994..db403de2131e 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2726,6 +2726,7 @@ static void task_numa_work(struct callback_head *work) unsigned long start, end; unsigned long nr_pte_updates = 0; long pages, virtpages; + MA_STATE(mas, &mm->mm_mt, 0, 0); SCHED_WARN_ON(p != container_of(work, struct task_struct, numa_work)); @@ -2778,13 +2779,17 @@ static void task_numa_work(struct callback_head *work) if (!mmap_read_trylock(mm)) return; - vma = find_vma(mm, start); + + rcu_read_lock(); + mas_set(&mas, start); + vma = mas_find(&mas, ULONG_MAX); if (!vma) { reset_ptenuma_scan(p); start = 0; - vma = mm->mmap; + mas_set(&mas, start); } - for (; vma; vma = vma->vm_next) { + + mas_for_each(&mas, vma, ULONG_MAX) { if (!vma_migratable(vma) || !vma_policy_mof(vma) || is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) { continue; @@ -2829,7 +2834,9 @@ static void task_numa_work(struct callback_head *work) if (pages <= 0 || virtpages <= 0) goto out; + rcu_read_unlock(); cond_resched(); + rcu_read_lock(); } while (end != vma->vm_end); } @@ -2844,6 +2851,7 @@ static void task_numa_work(struct callback_head *work) mm->numa_scan_offset = start; else reset_ptenuma_scan(p); + rcu_read_unlock(); mmap_read_unlock(mm); /*
Signed-off-by: Liam R. Howlett <Liam.Howlett@Oracle.com> --- kernel/sched/fair.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)