From patchwork Wed May 4 01:13:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12840762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8E02C433F5 for ; Fri, 6 May 2022 08:36:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE6416B0071; Fri, 6 May 2022 04:36:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E952B6B0073; Fri, 6 May 2022 04:36:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0DE66B0074; Fri, 6 May 2022 04:36:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BE7786B0071 for ; Fri, 6 May 2022 04:36:37 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 8C08C80A44 for ; Fri, 6 May 2022 08:36:37 +0000 (UTC) X-FDA: 79434662034.12.5AD34ED Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf26.hostedemail.com (Postfix) with ESMTP id 752EF140006 for ; Fri, 6 May 2022 08:36:34 +0000 (UTC) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 243NIL3H004092; Wed, 4 May 2022 01:13:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=OSTa9kaHYK2ZqH1nXuy4YN0cTqCfCvNxmCWDsGTmK/w=; b=DCckkX+RBGLikzVMS0SrnzidKuESLe6viC70e1TMdJxrIhgCoKSkbodKVmJkEMXeFwR6 G78+6cYi43WL9m1yRXPns5fK5Xo5W1/4+zBDo6SLaqMUX2TDZBZ2h6gd32Q67gIxOjKB e33gVVhEXu9sTGy2dQqyI5PmRYco4TmXL5V7HaJR9h8rhI74CBhQgisRFHbOfR7FN8kC M/j7WgA4Ksn4ygvKB/NXH6TNF2ZibXvEJdI7F0B7d98XffHjhu6YIIO2CY0JZEt0yjDG Yl1PW81FY53DsxJw+vBJaPn+NU1lFfR0dSPYpn2a0n2tSPCEkDjFrL640jYlrwVnQUOx Kg== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3frw0apxyf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 04 May 2022 01:13:53 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 2441BRJQ013094; Wed, 4 May 2022 01:13:51 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2105.outbound.protection.outlook.com [104.47.58.105]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fsvbmv3ke-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 04 May 2022 01:13:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JKXpgBTAxUf+nx/tbdHriDPwE6ld16+79Zhds857m3eJAkVXXZ3HouT3xaL5OwfTm+A9N+bK2RzHYpC5tACsZ89rwJvy5wRG7xaCui8RqKlLsi6sdSRPRZDlec707Y3uzDn6Qf7ra7bogTYggdpaPfBL8jtfespSRdHgZ0hQRrxElvbbz2RlZWnrkf5JdGWV9VsP1mhPz6uRi8bm5iTWKcB4wR7eEhx8cZK9WvbffZpdA7m+hakF+vyr96cM2JeomdoEVJdX1m9t1n1OwCaiWYmV+LwmTpIM07RGLI7wSH84MQySSycHG3rimT4HoVat0kgZBLGG8bMnBwrZqGBIFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OSTa9kaHYK2ZqH1nXuy4YN0cTqCfCvNxmCWDsGTmK/w=; b=EEy6mGD/r+UUsP29cpwITGimAoFtx3KRbf5ecyUX45aNrRRNj9sGChUzuM9lNX2Cb/1RhDMkGpXCoxZj64RWAxQYfybieRrIZsZJdaNnPKTzKb6Zs+oZzeuU04RNGAQ6NU7Fh91rw741zRmSWcyXAHh+yPqTNGOtJ3rkt2tjylueCr+mpg6kzktQhQvrMwtSsOeWIVMRzK7ax6pgQAEvEag8HF8DsPa+fKerP0vHaUIKLG5VM42bYQuFJ4dhkHpaTWupFSMXLTRGYaXmgLXTeMN67uCziIXdVV1Jb1sTzflwG3c05Dz3aOYJ7aL0Hg3JSJZfNnT8mtp3RPoNrndKxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OSTa9kaHYK2ZqH1nXuy4YN0cTqCfCvNxmCWDsGTmK/w=; b=cUCS/bYCr/yHIuaA111E6mT8aBFC9HJ8lK47q6FrdpZw9Z0/DAz6ofFweD5IQ95IiT1QcypHyqUs8/zTG/99JDE9lUqfhRJNzXDin9VvCy8GTSshfWwQ2JD/aPgWAXtVqZhYplcXMuL8+z5bmmOsCJEIKU7uo5PNeVkUzKnIcig= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by PH7PR10MB5830.namprd10.prod.outlook.com (2603:10b6:510:127::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Wed, 4 May 2022 01:13:50 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::318c:d02:2280:c2c]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::318c:d02:2280:c2c%7]) with mapi id 15.20.5206.024; Wed, 4 May 2022 01:13:50 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton Subject: [PATCH v9 18/69] mmap: change zeroing of maple tree in __vma_adjust() Thread-Topic: [PATCH v9 18/69] mmap: change zeroing of maple tree in __vma_adjust() Thread-Index: AQHYX1Q2zOYtx7lWNEypQfVNSHXowg== Date: Wed, 4 May 2022 01:13:50 +0000 Message-ID: <20220504011345.662299-3-Liam.Howlett@oracle.com> References: <20220504002554.654642-1-Liam.Howlett@oracle.com> <20220504011345.662299-1-Liam.Howlett@oracle.com> In-Reply-To: <20220504011345.662299-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.35.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 12cb40c4-2c4d-4e50-cad1-08da2d6b58db x-ms-traffictypediagnostic: PH7PR10MB5830:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wPlpHJDev6qnCH/Cy2EophXKV6sQ3tF+rGSFytKvr7+Y19Dhvrf3sXSB5rZ9pt3tmmiGvR4qkU4Wx/UPcydN9TrbT+v5LmOfW9eJR1zxO1Y9UDqwkibXqVi6V7eouJFCKl3ypUDr7xtkdCkQugUlyi+vjb+DuRsfOrGOZk20fiIlzYYu6Gl9OKx+Fd8MDwD8HiQ6WikFRdJARnkY2HdfmR6Yrf2fiZhANysV8SAsiTGpauNedpBrFXz+02xzLZ9yPVaJ6NWc8nnbAaLmJbU40fdRyHlTCfmCEIpdphZPzW+/zTdql1WEIg45XHIixDOaKScJGIUpAL9ptQGt80VIITacVNoxacpFtAi3AlptYXTHo4n1GPgJRV4fWeOK7L5mtpQbyMAe/PtnhG2i4xyPhZTi6pl+JGC6Br6agL/Dnvp/zXzHZYYKhbSOmAQDDrrqC50I0240bNSnHduQ96hnB/LJgjoIsalkOOymLr2hCA3VGTCLQsN/bLrisGVt8ZKzYpbRY/8/I1Kcqg2SbevwaJTqjYgPOTp/vR0dSHOKdQt06X2aaWfgyjPI2vFBizCtPqQGp05N0c8IU4tteXeQPF33txCSBskVnoyMz7+74xFDoZn2LEm0AScgB+0Rtn+qlnGQv2q8aXRcicaSZz1feGwv3XS80FeEmZOnAQlIWr0W7o0zdj7FY69TS1jujSzTQ41QoajR9W58/imeJzw4cA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(71200400001)(83380400001)(38100700002)(38070700005)(122000001)(316002)(110136005)(44832011)(2906002)(8676002)(5660300002)(64756008)(66476007)(36756003)(8936002)(91956017)(66556008)(2616005)(66446008)(66946007)(76116006)(1076003)(186003)(6512007)(6486002)(508600001)(26005)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?/CI9kQHKZye9YcxWACWtjik?= =?iso-8859-1?q?s1wcs9Wrn4h/xlOX/gRYuWKm4VsCuRm7mkBgAATjx5vtM0X0Yo8SibrPrt1U?= =?iso-8859-1?q?OOagrf5dU9bWvWPI63znneYXN0jeBnf8GksUFAQbxBAT0lZzXZV5gWxItTam?= =?iso-8859-1?q?tDZZRdWEIuZN9Wbag9Xq3q0pJEpy1Avr3YwAD4W7bTUSyS2qJ2a2wchKk5gc?= =?iso-8859-1?q?rZDa7UVFQWqh5c8rbDLOxdX2rLaOiO6AxJEdmNrpGrrN/dDTxdqRYq3Q2Vs1?= =?iso-8859-1?q?E3631/E9ap03vMy5XFbWwbLFri9QWT8zbOC/IAiDn6ntX0IcY5dHaKbvVqtz?= =?iso-8859-1?q?jqrHEiOF8HKc/4Kmdmq/ncxgghZSPKUUG4RoMs4cZ3IhTCbSww+e3YVvPgsd?= =?iso-8859-1?q?FjXxYVLq23UpdghpmltRzekauVwcIxatX6Pwx0hjDM9v2qIYf+81jBBNYr/Q?= =?iso-8859-1?q?8jOZrolfhv04Xla2Fu9BfYvuuBpi/9Kokz2FzNrLdNDd6rUcly4ET00C8pIV?= =?iso-8859-1?q?2kAEllNjwrvSfLzLYtj9uJpKTo9XB98F1J/xAmWHHXGCWKnlT5+RefzkDtA1?= =?iso-8859-1?q?icVCHKGPQM3iNZjsLVKoNmCHJUtbtkTv6bllwwRK3csUme+PWodAghkrIWdN?= =?iso-8859-1?q?UqlllVoaQMSkxknjyPXSDiLW9OZ7Q3Sm7/Bal3+7+grJHmoNoFXHQ/6ndUmo?= =?iso-8859-1?q?vbUpbkLN5xpERp9Ht1rOkfiVFaB3+L2HMLhjyQmqxeeQZOZjJ5lrlotphxBM?= =?iso-8859-1?q?RmjkCfWcEMctG/4gNpfsy1jaIEJymPaJNlScNzqCq2xScrKelCGa3ym9imHv?= =?iso-8859-1?q?Hl/bWHFy9wLPFpOmUPJW13X/S6XQpRYCowd+WluIXfiXOHO/OdAkaKc/O35c?= =?iso-8859-1?q?XX3UIkdoYwuCoh1YOdvW/9QTaGeFkRboNR6T7M7VtkMMlOKrug8Hc95JrIGy?= =?iso-8859-1?q?eIDkOctCjUkal6cRvos4l6km6JQdwrrA+4oIf7HMuRN+5lJaoECKZ3+8BKW+?= =?iso-8859-1?q?cVyEnJXHuu4P9cE2uqrMY3iB38nnCBa3XvEniqnb7Nk2oYEQwT3oNqmW4uiY?= =?iso-8859-1?q?I52rJiuJqMw9wT1nxKNyPjFtU6mGUXjv2txgLXm7GcnzMdCvQFZac4XknpsN?= =?iso-8859-1?q?VFc/fkbU2PDPXgPXp+ilvBW1kNNEWEiKZbWY89u3AZdkrA6ErWDbDkAvTzbC?= =?iso-8859-1?q?tZr2geSRbLX02iuaIYOtfxVNnwhslvOKcJLXQBxJD6FDiVG33L2zrbx0jp12?= =?iso-8859-1?q?P8PzoSJzmDunpWdsAB85ZK5btA93iO1vCh/9xuUGPzRqTzCVB8XUVIjMTHCB?= =?iso-8859-1?q?0SuLRnbU2Abr00kIx7Ng028gzki3ODLlXWmALme8hN8hLfoxaTSEubLNdcKe?= =?iso-8859-1?q?oN9hhlKaxToTdDCcVrIOd6m0mcMJIw8U6wlg/vjkbxY9d4BtWP1zGC6MAKn6?= =?iso-8859-1?q?gVf7AjCKlFx95gEIwhFLq24RREi5A8kSWsdhTn+SZiQ9yZZVduGNCXb50Edj?= =?iso-8859-1?q?D/kgKonX7P0f6FNVBS8bGKApw0e0rRoCXw8m/+5EIjL28U4hSYH++S7bEwZv?= =?iso-8859-1?q?xi3Y0TKf2HvT+9+Sj4s4gXewssO/Fj+dbmwxatY2JDMxYlc75UJLrTFFB387?= =?iso-8859-1?q?rl4wiOt1MiUgH0S0BSVlY60ulnyn4OcAVjAjatkZO6f6/sRwDCusf6F/HXTZ?= =?iso-8859-1?q?puahx4T7yUybbvpbqY3KDPKIirHbyAMD6SbThThLa+smPOSxKz4fTRFXYjA6?= =?iso-8859-1?q?fJyYQ+NsJ8TW49Jqv5YbeU7WA9KUSH5GwNktKuBP4sZd4nJWnpuimeDHeAF0?= =?iso-8859-1?q?5gr/CKGs=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 12cb40c4-2c4d-4e50-cad1-08da2d6b58db X-MS-Exchange-CrossTenant-originalarrivaltime: 04 May 2022 01:13:50.1271 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rnUpz+4dGpslNU3O6HrLzdYexHUeOUsuXslGBwF1AffTZmYaGfb1FeoLFCx+klcSUKlt2qYfeGnK5LJ2n9wTmg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB5830 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-05-03_10:2022-05-02,2022-05-03 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 spamscore=0 malwarescore=0 adultscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205040006 X-Proofpoint-GUID: InAmIyOke9iiIifciXoAXPRJkxgzKFdE X-Proofpoint-ORIG-GUID: InAmIyOke9iiIifciXoAXPRJkxgzKFdE X-Rspamd-Queue-Id: 752EF140006 X-Stat-Signature: nnsgcmncn9g41n96cxa8ddimmh4zahso X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=DCckkX+R; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="cUCS/bYC"; spf=none (imf26.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.165.32) smtp.mailfrom=liam.howlett@oracle.com; dmarc=pass (policy=none) header.from=oracle.com X-Rspamd-Server: rspam09 X-HE-Tag: 1651826194-333339 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Liam R. Howlett" Only write to the maple tree if we are not inserting or the insert isn't going to overwrite the area to clear. This avoids spanning writes and node coealescing when unnecessary. The change requires a custom search for the linked list addition to find the correct VMA for the prev link. Signed-off-by: Liam R. Howlett --- mm/mmap.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 44f9f4b5411e..6f1d72172ef6 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -613,11 +613,11 @@ static int vma_link(struct mm_struct *mm, struct vm_area_struct *vma, * mm's list and the mm tree. It has already been inserted into the interval tree. */ static void __insert_vm_struct(struct mm_struct *mm, struct ma_state *mas, - struct vm_area_struct *vma) + struct vm_area_struct *vma, unsigned long location) { struct vm_area_struct *prev; - mas_set(mas, vma->vm_start); + mas_set(mas, location); prev = mas_prev(mas, 0); vma_mas_store(vma, mas); __vma_link_list(mm, vma, prev); @@ -647,6 +647,7 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long start, int remove_next = 0; MA_STATE(mas, &mm->mm_mt, 0, 0); struct vm_area_struct *exporter = NULL, *importer = NULL; + unsigned long ll_prev = vma->vm_start; /* linked list prev. */ if (next && !insert) { if (end >= next->vm_end) { @@ -778,15 +779,27 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long start, } if (start != vma->vm_start) { - if (vma->vm_start < start) + if ((vma->vm_start < start) && + (!insert || (insert->vm_end != start))) { vma_mas_szero(&mas, vma->vm_start, start); - vma_changed = true; + VM_WARN_ON(insert && insert->vm_start > vma->vm_start); + } else { + vma_changed = true; + } vma->vm_start = start; } if (end != vma->vm_end) { - if (vma->vm_end > end) - vma_mas_szero(&mas, end, vma->vm_end); - vma_changed = true; + if (vma->vm_end > end) { + if (!insert || (insert->vm_start != end)) { + vma_mas_szero(&mas, end, vma->vm_end); + VM_WARN_ON(insert && + insert->vm_end < vma->vm_end); + } else if (insert->vm_start == end) { + ll_prev = vma->vm_end; + } + } else { + vma_changed = true; + } vma->vm_end = end; if (!next) mm->highest_vm_end = vm_end_gap(vma); @@ -821,7 +834,7 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long start, * us to insert it before dropping the locks * (it may either follow vma or precede it). */ - __insert_vm_struct(mm, &mas, insert); + __insert_vm_struct(mm, &mas, insert, ll_prev); } if (anon_vma) { @@ -908,6 +921,7 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long start, if (insert && file) uprobe_mmap(insert); + mas_destroy(&mas); validate_mm(mm); return 0; }