From patchwork Fri Jun 30 02:04:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13297510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3A87EB64D9 for ; Fri, 30 Jun 2023 02:04:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 13AA98D0002; Thu, 29 Jun 2023 22:04:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C41E8D0001; Thu, 29 Jun 2023 22:04:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7F288D0002; Thu, 29 Jun 2023 22:04:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D56488D0001 for ; Thu, 29 Jun 2023 22:04:42 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 92C90B013F for ; Fri, 30 Jun 2023 02:04:42 +0000 (UTC) X-FDA: 80957770404.25.CAB12D1 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf21.hostedemail.com (Postfix) with ESMTP id C82B91C0018 for ; Fri, 30 Jun 2023 02:04:40 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=kfu4pQEa; spf=pass (imf21.hostedemail.com: domain of 3NzieZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3NzieZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688090680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=GPwwDnKV3zLsafzwT78L/ly5VBmF6yg5GTSdgrnhrkg=; b=StlrO65u/amAer62qAeWRZ2EoUi/cKkPfYvt+ry/e5IlnlufSZXTcs/wHdc6lKU7D7UFzF CE+AGBBwUnbOS5SXfAwmaHOjhF9zKsmhGBO8I0ogSLQgk8fAvO1KqANAs+NrfnMgag3k3N NFZV5txhMzEczNYND5V7lfLrZ2/I9/Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688090680; a=rsa-sha256; cv=none; b=rI0rSROAc35tbtE3OV7idzv9FowG1u1+zUwn0GmlZovl8ubtfL6G6V44kGqwIqUJwOJkxC Vrxtf51FfHaMKbvGWGUcM1J07RXiCG77Vfvd1E0hkbfinu5QeU7H7BbcELP4/MPnCbEQ2M LMgxfRvghC1J33c6JVWG91vL96Mp0p4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=kfu4pQEa; spf=pass (imf21.hostedemail.com: domain of 3NzieZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3NzieZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-c38a4d22b39so1184727276.0 for ; Thu, 29 Jun 2023 19:04:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688090680; x=1690682680; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=GPwwDnKV3zLsafzwT78L/ly5VBmF6yg5GTSdgrnhrkg=; b=kfu4pQEacyKKHj9b/2HiXqrRq9FshER4qY3suCOAtPq6+5e5lG4n35E9cJBLZ1bKYZ RbfY0xOZmqANZpHbwlHjy5y8qWKF98CV1clCDfR5WzRrby/lWJRpQa4LlrMlj28jbsNI ltNIusl/Ne+jF0KZ5IPryUdJU11jGiyAhj2znXrFGKQTkisMzxD/R1rebP6yb1KWUDSR 2ADGw9E6PU4cO3hLHRB8xbSvP0y1wd5sYL4E3pbPQ2qoNsbpfgOALFrv+LHeZ5VR5pTg ntc6D8SUq0XZRsOh3u9F8ajl86QckBCaSKw+uTmZ/fnnxBmS7P3yyGeWylNLPLTJpm0q cmVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688090680; x=1690682680; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=GPwwDnKV3zLsafzwT78L/ly5VBmF6yg5GTSdgrnhrkg=; b=g/U+70fC+tjNSanAD3YhkhlxfbtFvq+W9ZP/L3aHYq9kK9fhMAtoIydyiXCLX26cMp yQCLOj5D42H1S3286yj4IwMBfLMphrjY2lRVK2sFYVDFs5nyi015l4uwIxWg4kmivhCF B6mpOCRrTdO5B+mtD2+ytgxtuo8o9yujCXSKoQW74q5HWyFmP6p7WpzacZ0cXd+m99ZK OHTL0ApVWZowhTR2cpltDI+W7rw5AzXSgW6m13civUqLPhKqgABKXsLbFL6ep0a5UCeK eKDeuuD1dRUnMVm2YcdpQ4BRLuhIXh2Sv7M1G32fI0VfudHaqzHFZZXqxzs9L9BydOYF 9Iig== X-Gm-Message-State: ABy/qLaN6nM/QSBsOCN4iL+H07pTi0I9Bpzj++G+Pf1txiN5ObRO6c0k CSGjXhYSmuhDUEhzWeYTJlH2Si+5pqw= X-Google-Smtp-Source: APBJJlG3yK/4SAhx4gxDQI0ZWRIfNvm139giJ2STexbUqk9qFqQjDK5gzi78T9+ZkFmf7uZf5UkwRTC4Hvo= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:1f11:a3d0:19a9:16e5]) (user=surenb job=sendgmr) by 2002:a25:a249:0:b0:c39:d6f6:481f with SMTP id b67-20020a25a249000000b00c39d6f6481fmr11525ybi.10.1688090679885; Thu, 29 Jun 2023 19:04:39 -0700 (PDT) Date: Thu, 29 Jun 2023 19:04:29 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230630020436.1066016-1-surenb@google.com> Subject: [PATCH v6 0/6] Per-VMA lock support for swap and userfaults From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, hdanton@sina.com, apopple@nvidia.com, peterx@redhat.com, ying.huang@intel.com, david@redhat.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, pasha.tatashin@soleen.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com X-Rspamd-Queue-Id: C82B91C0018 X-Rspam-User: X-Stat-Signature: zu5bmws1jrz777i3gp8hqzyn3uuf4mnq X-Rspamd-Server: rspam03 X-HE-Tag: 1688090680-110120 X-HE-Meta: U2FsdGVkX19ke5q71ytUVXM3cp8IgOTLwdkQr/QWr/mOa+l/spLqkhl3IC0wrWJ/VR38aFsxVevxGCe3HSHa9ypfxJHZ6JJ3BEIzrOK8E4RBHbgLocw20t9nVCpNmno1+y5eaKgY7ljBRnpRRny31GQVI82TpAvjzJC6b950gIffeXjG0XHkcfLtrmKKkBKkvfyweHgJxNlk0soiUBMJoDQb1B8dxawISOh29SPp2/Gj7cpjvJCUwN48/OcKN9A5/yLOu3iygyygqzSuXA9KDMS0zlgfFFwbi94O9wzXfWDs0qFpxIALlCJLz8It/0ffLsWXxqhh2jotkcFVKumvG02+ZH/sGbLn3h/aV47oVzkFXu5+mpO3/B5j1pQPveWiaC56SrRwpZEWu06J3KEEC1N0013pSdaIZ/Wt+OzayvTvncN1/ZVSQ01MORdRT+EV8915U8kSvqm0cm22+0spHbMuOKDqf7tX3t2OriYCEVQzNCo2I+s9w8YXMJdehiwz5KnQcF79lzi90NyVIoGNoSxlv4/UBygbYya6/Smrqx943/MJTQMZvbFEeIrbxc30vv4vNakHXM2aDzPORitpHltsFGBcrDWSGgSjvbZpCpmwU1rNXn15dkmd5HglkaP9kBlurino/c7kzf4+yVP8iKHEwwL2M5v2CEXQB6GGNVtQpvZNOzyovHIH6IJ323OXgkQXIFrs1MyGycmg8goot5T51sUnJKuR2uUxEIUAtAXX0wT/rAd4h1VSRX70AJEKI91xlrrPflbeeG9D61LCuuswUvNEaM6tWLuvWiuEeDj6ajoDA1dYR1LeSUE80LUK1f9jVHuEh6BzfdlxL95oWyCmazp5py8mnGDUP7y+NbHb8IVjc75TWdgECrgGgHFt1O4N+XQFDd5G2tXaXyyjfhFQ1fCbVEKi7ePW4KVGewfWj1/4M9UfJaLEKfpFjfKiwkltQ9NkHYeWMO4P3DP 3OyAmsTx Y/45ydAreQodJQWtOMTU6CTV2A2sTFR5GB2McuHK6ehnoNj+SYJ99+Z8+uJvkELnBQb8OZb9K5qrNamjnkzkI5EKzpUEEmyIAjrT+ECnhyZcrLAi/Z7Uy7pmGpaXEhMR2ZRmz4R+wUHJOw+dJ3pOiRvPZ0S95Y2Y91sEqmRQ7iEasiG9NJ0BTnxH3AfU+1qAXdfTV4JqSqSfBhyNSUgYS61vpM+UGQKybcsaZxlv7fEQ4saoiPftHCp2q3E07/dgRL0r3vj9/b+LmZAY/r0D5kzsAR2dT4CeeyD6Mzj1QAc5ZDrBq7qDJT4qT2tQZfKJw3ViNvFEPM6BmudogSeXgOnSbkQX1FHTfdLDdhMhZo984NO3+0oucRXNORC4kjvmYW3lca8UOX+Ew+l9tCPHGlerXRothwKCO9E5bnn9GLwFr67uHNwrYhAKZZEdPOLfI/AEP+KRW8214m6P9tBWdjyJ11fcf1VUoZllQVT0kAmujE2ByV/YnRF0a/3gDzw3c+zgiXf4ZBwEi7qqiGau3LEO96zdHR602mUujLI2yn+AYP/1xF/BrNucyAbOcuA/eXPuYMhzqoieT2zV7FbeTCRt/Z5k7moz33E+GyXDHN+5sC8DHaqfbsbGTEkevUZaSgY8n6jDFzvh2ewKCiiUkhPxCRDQ9laLyZpuj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When per-VMA locks were introduced in [1] several types of page faults would still fall back to mmap_lock to keep the patchset simple. Among them are swap and userfault pages. The main reason for skipping those cases was the fact that mmap_lock could be dropped while handling these faults and that required additional logic to be implemented. Implement the mechanism to allow per-VMA locks to be dropped for these cases. First, change handle_mm_fault to drop per-VMA locks when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED to be consistent with the way mmap_lock is handled. Then change folio_lock_or_retry to accept vm_fault and return vm_fault_t which simplifies later patches. Finally allow swap and uffd page faults to be handled under per-VMA locks by dropping per-VMA and retrying, the same way it's done under mmap_lock. Naturally, once VMA lock is dropped that VMA should be assumed unstable and can't be used. Changes since v5 posted at [2] - 6/6 moved changes in sanitize_fault_flags into 3/6, per Peter Xu - rebased over Linus' ToT Note: patch 3/6 will cause a trivial merge conflict in arch/arm64/mm/fault.c when applied over mm-unstable branch due to a patch from ARM64 tree [3] which is missing in mm-unstable. [1] https://lore.kernel.org/all/20230227173632.3292573-1-surenb@google.com/ [2] https://lore.kernel.org/all/20230628172529.744839-1-surenb@google.com/ [3] https://lore.kernel.org/all/20230524131305.2808-1-jszhang@kernel.org/ Suren Baghdasaryan (6): swap: remove remnants of polling from read_swap_cache_async mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED mm: drop per-VMA lock when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED mm: change folio_lock_or_retry to use vm_fault directly mm: handle swap page faults under per-VMA lock mm: handle userfaults under VMA lock arch/arm64/mm/fault.c | 3 ++- arch/powerpc/mm/fault.c | 3 ++- arch/s390/mm/fault.c | 3 ++- arch/x86/mm/fault.c | 3 ++- fs/userfaultfd.c | 34 ++++++++++++---------------- include/linux/mm.h | 37 ++++++++++++++++++++++++++++++ include/linux/mm_types.h | 3 ++- include/linux/pagemap.h | 9 ++++---- mm/filemap.c | 37 +++++++++++++++--------------- mm/madvise.c | 4 ++-- mm/memory.c | 49 ++++++++++++++++++++++------------------ mm/swap.h | 1 - mm/swap_state.c | 12 ++++------ 13 files changed, 118 insertions(+), 80 deletions(-)