From patchwork Sun Jun 18 06:57:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13283769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E37B2EB64D7 for ; Sun, 18 Jun 2023 06:57:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED3C06B0071; Sun, 18 Jun 2023 02:57:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E83B56B0072; Sun, 18 Jun 2023 02:57:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4AB78E0001; Sun, 18 Jun 2023 02:57:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C4FF06B0071 for ; Sun, 18 Jun 2023 02:57:26 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 89A681A03CE for ; Sun, 18 Jun 2023 06:57:26 +0000 (UTC) X-FDA: 80914962492.30.D5892FC Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf28.hostedemail.com (Postfix) with ESMTP id D3472C0006 for ; Sun, 18 Jun 2023 06:57:24 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=f8hZG5EV; spf=pass (imf28.hostedemail.com: domain of 306qOZAoKCI4G6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=306qOZAoKCI4G6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687071444; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=myYEJQ4cXoxihGQz14f2NgKY1m5nccoqkiKaHRCGLnA=; b=H8xciudEItIXwLui8jkBY7XdCyBPZKUjGDAUGSERbMDwcVRFp9YQ7buIdLdMYZMpvjv1Rn leSpXYaECAAxs/g2T39dFa7xJIcppyBHDgIvP3Ay0Lue3NbWiFrsgY+Pi5DxU3RzKsk+6X biJRSHWBH4liKWFvVJO705NfmxZw8AU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687071444; a=rsa-sha256; cv=none; b=uuSB+QN6gjjDnr1E7gdyjK+1b2WMzOzmWbzbXpara19ObBH6t/bmVJvrBQbKhKK1fbAZGm 1tXLpzBspumsHoR1FIGHLni6G0CtcbRWcF+OOKRafC8DcMkgi5Z/taKZ4JJ+GeaLDce7cV 8IjUOtFEimbp3wtJddFFToCfjrotanw= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=f8hZG5EV; spf=pass (imf28.hostedemail.com: domain of 306qOZAoKCI4G6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=306qOZAoKCI4G6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-5704991ea05so31833657b3.1 for ; Sat, 17 Jun 2023 23:57:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687071444; x=1689663444; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=myYEJQ4cXoxihGQz14f2NgKY1m5nccoqkiKaHRCGLnA=; b=f8hZG5EV1v8ur0Uofbbm4KrkwSDeIuesyp1vSzziXgT41RJNgW7DnUeNXNS4qH2atl I1UDna+07xwVf6Drz61wbRkC4OLnbYypo8GOdxdYQ2Lltrp+/ubLWkOdxAlLpaPnx0XK OnBtKJXwjNtbvMz7a4xeAzeCbfZGpawFhJjfWKNbr+iKy2S8LrQJCqhHLw8pIbZrn3lI HCQaHctYUwL2cnhuule9c5VbVqClwrtziyAi1OYAcqH7md99Qjvpipq4fPTdcehGbfF/ TiZ6ruO+/dS5pL9VmyaTRN3fonOjPmJHMzh6mwNAU/xR9LLzxDNdxBlnlinQJGc9RZhP ALzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687071444; x=1689663444; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=myYEJQ4cXoxihGQz14f2NgKY1m5nccoqkiKaHRCGLnA=; b=RR+M/vKDsa7E2n+zjLHHTGyfwRZOrRsrwTWi9IIm+ZxL5dIgU97iLWw9b8Yc0sXnMh 6fayfOrmzFR3oCwrc61vQUaN6zII7TdSCiiFEyAg6RVSI/cf249THY0q50LPYaFc/AE4 s9EhqlynWPHA9WO9vOD87OT9s3MRJMrBElpbnxLDqSRxXW6ak0smdDtChYIriWenv1WJ P4PmBExBbObLFn0naC+eKbxJ7GSKEL5pQ2qTykgmOG/oyvCcFHC4s1S+VyIweqQBTr7m sL/AinuYTp5TcI8gfu0VjVX2kE4yBogdPA3kVT4NGoqrEjhr1yTrGpOdklxan8U4CA3C qGfw== X-Gm-Message-State: AC+VfDyJ1AtU06ytk09rLHXh7e+dpOVxmqVKLUcdS7C5wJbWUYhxEByY 4OFCY9JVpGQpTBiH4NjtZhMbZb+3h6gM2Lqp X-Google-Smtp-Source: ACHHUZ58bBzWyWP210cPGPFNgbxSxsLYVqUXiVVtq+n6XV4dr+17yNK31Laxw3pjV5T0hOoBcXTORg7IVJ2Q5Cv6 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a81:ae0d:0:b0:573:285a:c2a3 with SMTP id m13-20020a81ae0d000000b00573285ac2a3mr699145ywh.1.1687071443846; Sat, 17 Jun 2023 23:57:23 -0700 (PDT) Date: Sun, 18 Jun 2023 06:57:19 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230618065719.1363271-1-yosryahmed@google.com> Subject: [RFC PATCH 0/5] mm/mlock: new mlock_count tracking scheme From: Yosry Ahmed To: Andrew Morton Cc: Yu Zhao , "Jan Alexander Steffens (heftig)" , Steven Barrett , Brian Geffon , "T.J. Alumbaugh" , Gaosheng Cui , Suren Baghdasaryan , "Matthew Wilcox (Oracle)" , "Liam R. Howlett" , David Hildenbrand , Jason Gunthorpe , Mathieu Desnoyers , David Howells , Hugh Dickins , Greg Thelen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: D3472C0006 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 6kkiez5ijy9dkh1uix7r85cr915r358j X-HE-Tag: 1687071444-815104 X-HE-Meta: U2FsdGVkX1+ZvB/MXDQ6ciG9RgW/bT5oQwEWdKNZbUrnmjAguYxeWu+mEOzlTKLasamFPqGcj9dyWY20OY0TgmMl2zv8C/8X/Kuk8OGjxoNYRPDCgsKGRUeUwkCj0jhA0+ir4Tg0O2JAncd208LTdAeg1egaJX6zTbxbYAcM0p4BSoF2jbcMuPV1XCdcLaKST/kpzf1zdJL5dMdd38PlZR6VDFFTbPOExK+oe4pIH7OqQqVcZwqe51gwyodNt5caap2C0U27ZaxouoDumjeTJ74cQViLDcIuuBsLBdfiIz34I5oD8EnwrOxevTBVxJ3BYfGiS6ENX3XJ0IqqEVvaLRicbBgltr5M1L8nCGKLPYlBNbbBW0KpYTzbyjS39pwDTn1ux07ivgBpr2XeWdIyh6mIkkmaSTpNUSJ62KjDX5PYG6sfh4eWY/HT3Ppvoi6MXL3XWxPDOFXnt0c90gD4YKYWLskw+R7nxQFqGUSP1+lIGX9TYBlSg9yHwIKbBaHLkfdKYZL49qhM1He1AW7+MUbdHZL10tZJ/GD6r5L95Lk/IXWlfJc4SvJfD1eUd5i1a6waBqOVEk5Z6Vad+8EpkcW23bEt12JOvYywz2Zd4deC4OeDtPZJFmrnYwrEr5lT0Y7xqF/oC/wuI3r+g8Q/ws5vYc7N7MO6mh4BzpTIR6wPPORImy/6hE74nv8/FfRnNdKnr/jsWH01Wzw4UiZWouYlZG9G14zyIeh5ZAhImEFU0di4pvlhru6lfIe9x8HsPryXIh1mn1y4kL5VMTat/TXU/VcQdcOah42Nwo5Bi9Urt9in98JS87MSTe5cVdnqVa7jBb5xkFEDGcpnOsInQGMkD9ntU5c0R41sgAExmwHUtxN4biY8eEivLpTLpDcJ7hHNQYsU+PA1QvdDSGXUR/qlexq7V3qcT5FGi5typx2yHq3Fgpyzgxa4pqZXayczVILYL5wXIETkW1BE+qB y2IohuAw 5TbEqffGHJoyt3QC30rLJR//W0WrwoWk73h48XgeAwXNMA9nooUc6PdX1Fh9t6I7Ddil9qC+LE7/hVK8oriP+bpfyCFLPiHXNMA9D2ZTCrOb1Hxd7MC+J4G720t2CoxJRISPHms62i6KFAR0NfirEAXLDY19L+kCC25NaP2fDUKQzUQ+qdM75YguRg9nnZXzE44jIMV4V/XusMWGLKgrRRgRPSAEIGNY5kHp/Sn4wpYy6DNGcp5mVslj+Dym/aBGHvntEAyL2VC2zAST5UuL1uaBjznRG+GJLUCnXhx1sqBCCnjs765jryb41XnLtHwEi3x7F72dlce71KN5nXJQYllrgfjlJxNvx+H+hk+3MtdX9f/LYbYXZmImtLzehdwkfEu2/vecSw0Py2k5Nl6VeHBD6wpvEG+zf8eFB0Xt0UXrX2pnkb4XjXoG0ZNBM88uvmspMlqeeRVGC9rf+Lq7OXpD1pi89g6sAF4ufd6TNwFD4yWNqS4aXpMfKZ32QVeORw6FN2vZraWCC98sMqejLfWY5pc38d3DV3E7UjZG26myanFAQW/33DspLSUByVLya4U9K6TLyFvj0EKhupVCrdF//wsbHkbrpdmDuo0st9vzCA6v4LIl9F1bHWFqmkCzDofVebud7NRBOhndpM/lPpOlnOXnqMypF5MdaJ9ZoCH+GLoMYw7jZTBUxXo6gTIq8JgUfInp9eI/ypP/D3ybkDyz3UxWw0aSYKbuhbrNnS72s7qbq+0I8MHjQD1Ed5ENMYhWVvZvB09BBD5E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series attempts to rework the mlock_count tracking scheme to avoid overlaying page->lru. The main goal is to revive the unevictable LRU, which would be useful for upcoming work for offline memcgs recharging [1]. For that work, we need to be able to find all the pages charged to a memcg, and iterating the LRUS is the most efficient way to do it. With the current mlock_count scheme, the unevictable LRU is imaginary, as page->mlock_count overlays page->lru. The proposed scheme overloads page->_mapcount to track mlock_count for order-0 pages, slightly similar to how page->_refcount is overloaded for pincount. More details in patch 1. Another advantage of this series is that we do not have to reset the mlock_count everytime we isolate an mlocked page from the LRU. This means we can more reliably track the mlock_count -- we are less likely to prematurely munlock() a page. We also do not need to re-initialize the mlock_count every time we add an mlocked page to the LRUs, or every time we found that it was reset during mlock/munlock. The lack of re-initialization slightly simplifies the mlock_count logic. The complexity is also more contained within mm/mlock.c. This series is based on v6.4-rc6, and has been tested with the mlock selftests (though I had to rebase to v6.2 to get those selftests working). The series is broken up as follows: - Patch 1 is the actual rework of the mlock_count scheme. - Patch 2 handles the case where a page might be mistaknely stranded as mlocked indefinetly if it was mapped a very large number of times. - Patch 3 adds a WARN_ON() in case a very large number of mappings can be mistakenly interpreted as an mlock_count. - Patch 4 revives the unevictable LRU. - Patch 5 reverts a patch that was part of the original mlock_count series [2] that is no longer needed now. [1]https://lore.kernel.org/linux-mm/CAJD7tkb56gR0X5v3VHfmk3az3bOz=wF2jhEi+7Eek0J8XXBeWQ@mail.gmail.com/ [2]https://lore.kernel.org/linux-mm/55a49083-37f9-3766-1de9-9feea7428ac@google.com/ Yosry Ahmed (5): mm/mlock: rework mlock_count to use _mapcount for order-0 folios mm/mlock: fixup mlock_count during unmap mm/mlock: WARN_ON() if mapcount overflows into mlock_count mm/vmscan: revive the unevictable LRU Revert "mm/migrate: __unmap_and_move() push good newpage to LRU" include/linux/mm.h | 31 ++++++-- include/linux/mm_inline.h | 11 +-- include/linux/mm_types.h | 24 +----- mm/huge_memory.c | 5 +- mm/migrate.c | 24 +++--- mm/mlock.c | 150 +++++++++++++++++++++++++++++++++----- mm/mmzone.c | 8 -- mm/rmap.c | 3 + mm/swap.c | 8 -- 9 files changed, 174 insertions(+), 90 deletions(-)