From patchwork Fri Jun 9 00:51:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13273051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB07BC7EE23 for ; Fri, 9 Jun 2023 00:52:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 495FD8E0002; Thu, 8 Jun 2023 20:52:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4461D8E0001; Thu, 8 Jun 2023 20:52:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30D718E0002; Thu, 8 Jun 2023 20:52:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1F2258E0001 for ; Thu, 8 Jun 2023 20:52:06 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EB29BAF008 for ; Fri, 9 Jun 2023 00:52:05 +0000 (UTC) X-FDA: 80881382610.09.09AAEFF Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf28.hostedemail.com (Postfix) with ESMTP id 3915CC0007 for ; Fri, 9 Jun 2023 00:52:04 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=ImpLDGn3; spf=pass (imf28.hostedemail.com: domain of 3s3eCZAYKCNYKMJ6F38GG8D6.4GEDAFMP-EECN24C.GJ8@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3s3eCZAYKCNYKMJ6F38GG8D6.4GEDAFMP-EECN24C.GJ8@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686271924; a=rsa-sha256; cv=none; b=6JZ4DvqoPmkHBwkvIp4wYAyZNjsd0nCThTI/JvTAiWkvjV8rIQNfQYPWHU99+v6cx4GHPw lycDrUjhrzdLcXA1wWwQU7MkL1RIbR+ATBw5IJ6r6fSzsxpzqiiJ07NCjeUmr7uwEXkm0B kldu5YouFvlco6VpCYIJo0WdnkYVAwM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=ImpLDGn3; spf=pass (imf28.hostedemail.com: domain of 3s3eCZAYKCNYKMJ6F38GG8D6.4GEDAFMP-EECN24C.GJ8@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3s3eCZAYKCNYKMJ6F38GG8D6.4GEDAFMP-EECN24C.GJ8@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686271924; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=TthVOGU5NBXTdUCbenudXv/zsuPKI5dT0elSKLsw5KY=; b=i8kHzr8hawfdHC7ZraXXIpuujaA8AfLYA0bOoVIXkMposgLcepZ6hC+22+rkoviBs6ioOk 69G+kFv9z2Y5kSO7Dv1StFemD+Qa6TQCoAmgP1npJ0Hxu4orKFywyjEznHohXCkPi8moD4 iDpraj3tfz2kBU3T9dvluf6CLiWdVaQ= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-568a8704f6dso16121117b3.1 for ; Thu, 08 Jun 2023 17:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686271923; x=1688863923; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=TthVOGU5NBXTdUCbenudXv/zsuPKI5dT0elSKLsw5KY=; b=ImpLDGn3lmP9gv5tkN/NEWkOq1CUqj8TD3qghFzsulypsFg5z+NBVRVHcl4W+zs9NS a/AUxP3USFFEem5uKIozHHNBnxBibW9dCIUuyoz/7IPe89xOljWkVvW9V6osOlolfUne DQLjQqvvd5IXZIwOo9ZRrUtWG6YL+hMBO0XMhyX9ft2y5anTxTVz9LsrpUIxcTGwDtT+ +rMTNXGaBaHLPmEulzNcpPMC8knZz8tOalmVU/5YVSXK0/OGPREjG3YZwobjrPaMaWCr SQOISk0fFfxmV88xE6geZP9//tFPMygeFZNtEH0SbRgZnobBVXTiCsqOvEWJz55HHPRP 8ZIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686271923; x=1688863923; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=TthVOGU5NBXTdUCbenudXv/zsuPKI5dT0elSKLsw5KY=; b=XSwDvRz5lBrHnMfhcFAK2LSq0Y0T1d4ZxwClPLiw7rUcNCz+4ikPKFYxkUKS8kTxI3 TzTB34MZ34c+QRulXMIadz6h1UWcRV9Kdis6WbJ9AKSP7923SWo/zDB24IQB+ZOEXHmV FFNu5ws2u4fzZYIJe3r+oePLCNQaAgwwLIrPNkTcklnkFOnvQ2+IijWH6LZED/kqB8hN ZwwG5ioCS4vlAg9qY/atZzmfR7lS8b+A9ORfKqfFaq7ZOB48ILSOPbBVjWHGeK86sOVW G9Z58MekJ3FCk8XtjqS1SR+7nSmcTU/KizYJt4/NJH1bTlXBaGADPNLHb3IIZUrJ14sY +74g== X-Gm-Message-State: AC+VfDyPdpb5agNgHUrA7drCnpgMYC1tR1S+l4yxhaF4LSnGGLLBVqKz fF/cvVYIHItuAjtMT5S8qB9ekXzy1YY= X-Google-Smtp-Source: ACHHUZ5NZ4g1X3BYYVvHaQ5ca+C3geF48LesY4562DfpiKDGzLPRjCuGKlBWzQoWjUktyTRs7xzqWuvyq0c= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:c03e:d3b7:767a:9467]) (user=surenb job=sendgmr) by 2002:a81:ac43:0:b0:568:9bcc:5e16 with SMTP id z3-20020a81ac43000000b005689bcc5e16mr712323ywj.2.1686271923250; Thu, 08 Jun 2023 17:52:03 -0700 (PDT) Date: Thu, 8 Jun 2023 17:51:52 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230609005158.2421285-1-surenb@google.com> Subject: [PATCH v2 0/6] Per-vma lock support for swap and userfaults From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, hdanton@sina.com, apopple@nvidia.com, peterx@redhat.com, ying.huang@intel.com, david@redhat.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, pasha.tatashin@soleen.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3915CC0007 X-Stat-Signature: py6kbqqusn53pd59cm34iimucn1hhd73 X-Rspam-User: X-HE-Tag: 1686271924-539890 X-HE-Meta: U2FsdGVkX1+nRYqf0N1W8veG/jMBGu+b6s6dx8lymhzwJ1cQHsWZ/qcRAetILNT2a+Vl0byeQetavP179UoMnk04uJikfnx1CSqZMMOt51xhAx9s63EvV9jgLgiJXGcX7aq/szyg/RbKhfe6CgjcJchF2fHPJY21vxYdE6djMKjOpZwKqy64dVZK2LVq97p4lEPiFZfVCfODat0+0mAT6Wt4BqNSL6HDLQijztFODNf9fSmd7BVWDKO3QXtGrWcZsVNceRmgOYlKFb01qZf8E5RUgFPgBQhxYxWPSodalrsMAqyanhLQfEpo8jXsqefEj5IGu3Psdw9w+iimIBcCPRzb5tIVbsglJFKNNXU7iVetfZXI33BQY/mHeNjORbqgI0g4kItTRQZu/w+BVLbF2dXRpvEgbvymBUtv081MfZ8Js0vk0xGOTxCRGbzruJYGX548hH9fs8Hxy9p/S6vhK/N+kN9MpeQ6QAVMwN2CrGtMiQm++XJcyTZ3STudcMVwGdwWrM8kUcw1o2lOBJDuumJgD447WUbuT4640cPIeq8kz3JwrLEXLkUsjqf9B8hLwRjvFS35WIdVNPvHypznTQCRtYpRyoDlq4qAFT88j94Hbd8/AK9ufZTeaZmpndLy7qBfcEhkMo6XwbgW1FW5Dyih1nv1o+JkrVzRcJqV7kdbiXzBA8rwgIpFGxvZLsa2XD0VZNjYTNOw0+MYQJpiiWUnKmxbnwFungAejhzaHUNcFTP6WdVRlL2ibTJseCpqwjqM8SICLX5i/Mk/YkzZDLj2B1xXzF0buoNR3cNTfHQoAJl9tFrl8I6Bh11mvSXD2j9uh5Hsz8GdzfFuYQ1cnZrSYcjqULECC2LLHjPNtCI7Xlklq4yiaFTzgzmIzpYShri0EfiVvnb863CphuaJDSbaUIqKdKvnzCN1q8YpJI/RSF8dQAGtzLjnlT4rvi9cRi4vaJovvHRK4W20nqW uNGOVk5J uJ5m+quwVczkwsnarh76UMbvh9s2DN9s0rsqVBBK+fhtgVrz4HTRHqWCTbZMrqo6qjJQV9Yo2jcZyL46cPnSHS0eYbAHAFCpBzxvZaho03Mv0PWZqN6Zs5HHEOl+T+S9VnUEADAjGvzalStfw0/zEnpmHuw5JutmubJjWKb+417Jd1VKIy+j+7O3ruV6+oY0W5EttEyppsi8oC+eSm8963M6Axdc6SREWoVipKXX9bHZFGj9x8Q+YWwkMt7qe1AW0wR98US81eenMXfajg7rVilopzaE36bQuDr5oIfpLyd4KflDu2jyjEeGx6OYWEijhmTmTjzilvwe+6bCS9lv9J/dTkU9nDRMhKItIO9urnxEqa55Yaq9mcKc+2G5iVTIbzwhcnb1crpS4FLdOoaKAFWiTN8ZC0t5vff7MnRrZC/8hIxN8VyUAlHMMQ5/EjYMo+Nvrm5PipMb5i7sjfaVh09GjENfU6VBn4JYMG9LYoOrj1d4kEufCYgzSv1gTT63224I3F7OX8F59DBvaySk4mrV7BDTp45sf1iHxP8S8sUDlKXfI278ah3UViNvPfkJQJNPmQOT/F7PGBMVF5UkuThIl0fPJ/mBJFaDQqHQoH61wFdRHU9yXEdaHUtN2Za9+02lGLRd7wMoadKfDJKL+t0YLCUGKxAvWzVISmI7kZxdbV9ChMY3uRk/AktibsWuqh3UU5UBET389KIosgWDV6F0/y0X/pEE5ICup X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When per-vma locks were introduced in [1] several types of page faults would still fall back to mmap_lock to keep the patchset simple. Among them are swap and userfault pages. The main reason for skipping those cases was the fact that mmap_lock could be dropped while handling these faults and that required additional logic to be implemented. Implement the mechanism to allow per-vma locks to be dropped for these cases. When that happens handle_mm_fault returns new VM_FAULT_VMA_UNLOCKED vm_fault_reason bit along with VM_FAULT_RETRY to indicate that VMA lock was dropped. Naturally, once VMA lock is dropped that VMA should be assumed unstable and can't be used. Changes since v1 posted at [2] - New patch 1/6 to remove do_poll parameter from read_swap_cache_async(), per Huang Ying - New patch 3/6 to separate VM_FAULT_COMPLETED addition, per Alistair Popple - Added comment for VM_FAULT_VMA_UNLOCKED in 4/6, per Alistair Popple - New patch 6/6 to handle userfaults under VMA lock Note: I tried implementing Matthew's suggestion in [3] to add vmf_end_read but that gets quite messy since it would require changing code for every architecture when we change handle_mm_fault interface. Note: patch 4/6 will cause a trivial merge conflict in arch/arm64/mm/fault.c when applied over mm-unstable branch due to a patch from ARM64 tree [4] which is missing in mm-unstable. [1] https://lore.kernel.org/all/20230227173632.3292573-1-surenb@google.com/ [2] https://lore.kernel.org/all/20230501175025.36233-1-surenb@google.com/ [3] https://lore.kernel.org/all/ZFEeHqzBJ6iOsRN+@casper.infradead.org/ [4] https://lore.kernel.org/all/20230524131305.2808-1-jszhang@kernel.org/ Suren Baghdasaryan (6): swap: remove remnants of polling from read_swap_cache_async mm: handle swap page faults under VMA lock if page is uncontended mm: add missing VM_FAULT_RESULT_TRACE name for VM_FAULT_COMPLETED mm: drop VMA lock before waiting for migration mm: implement folio wait under VMA lock mm: handle userfaults under VMA lock arch/arm64/mm/fault.c | 3 ++- arch/powerpc/mm/fault.c | 3 ++- arch/s390/mm/fault.c | 3 ++- arch/x86/mm/fault.c | 3 ++- fs/userfaultfd.c | 42 ++++++++++++++++++----------------- include/linux/mm_types.h | 7 +++++- include/linux/pagemap.h | 14 ++++++++---- mm/filemap.c | 37 +++++++++++++++++++------------ mm/madvise.c | 4 ++-- mm/memory.c | 48 ++++++++++++++++++++++------------------ mm/swap.h | 1 - mm/swap_state.c | 12 +++++----- 12 files changed, 103 insertions(+), 74 deletions(-)