From patchwork Mon Feb 27 17:36:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13153984 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACEE3C64ED6 for ; Mon, 27 Feb 2023 17:37:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B73C280005; Mon, 27 Feb 2023 12:37:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 43E0B280001; Mon, 27 Feb 2023 12:37:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30730280005; Mon, 27 Feb 2023 12:37:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1DB1F280001 for ; Mon, 27 Feb 2023 12:37:30 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C5E7AC0A4F for ; Mon, 27 Feb 2023 17:37:29 +0000 (UTC) X-FDA: 80513778618.23.3B1ACFD Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf11.hostedemail.com (Postfix) with ESMTP id F242940019 for ; Mon, 27 Feb 2023 17:37:27 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=oOzu68rh; spf=pass (imf11.hostedemail.com: domain of 3Vur8YwYKCEUz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3Vur8YwYKCEUz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677519448; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2RRQaU516ZwpTlJ/IPA+DGjHt1op6pQWjJsnPIcYfuA=; b=1/1XCHJvZ7BtY4ajBdLTrLfXuQ4sWVyFRxXEUCHZFF4c53fTzsCC9JyyO7+Azz2nTwoEDh cjzE8OH8RyJdEAdsk0b/nxmf1cFmPIJNJ23ZLYPm5JwjJ1ID7gzWPjCidSZzFgavcjhxHl oNzTcgIvyuo8wx8M+ddSCdPuVtP0hRk= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=oOzu68rh; spf=pass (imf11.hostedemail.com: domain of 3Vur8YwYKCEUz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3Vur8YwYKCEUz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677519448; a=rsa-sha256; cv=none; b=LrNr1Ou9VnFe01/rQeJdnEDNV3yYjD8bE43UM3KLeNejRZEJ4JNtsUD3HZu2SoFnZoP8br /O0v+VZi8MuDIiJNCe1vXUc2EgtimVHWypeT5ftYtPJWcuPOyCa2kXn+UbpGFbkD6Ayjb2 0+7AiXXa02pXzdhTzW4GjikAriKG7U4= Received: by mail-pg1-f201.google.com with SMTP id s187-20020a635ec4000000b00502f5c8f5eeso2175097pgb.14 for ; Mon, 27 Feb 2023 09:37:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2RRQaU516ZwpTlJ/IPA+DGjHt1op6pQWjJsnPIcYfuA=; b=oOzu68rhqFvQUE3xPXRgC65GI9mEeLvvYxOtiIzpOqVj22Kyf5G6v2nWjVGklLxMvP HmGHsiBCF1KS106jCim6ns3MdM+isHFIB2cW4BN/lGs8dwWcRP4ETfa+ag2t0c8ooDBq IbM6nmgazseNTIyYCuuyg4z6GzI60o3WZCSdFyA+NgXZEZsvKFhXlvHjg7yVejPrN9tm lKdZz9wSXLCbvII2MlT1JIIh3E3Pzy0qhZRWTGIGZxssznyrIga0ZMQq7hZXO4CZPVOT 3YmHDK/l7fYrKAyqOPLY4x/QliUuYuXk7UtTVeDvzP6SJegWWyEijbowKlqNMovXBpqB mtGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2RRQaU516ZwpTlJ/IPA+DGjHt1op6pQWjJsnPIcYfuA=; b=AV3Gin21mnCJcqMpL1VbuqHlcyvgHR26W2n9ym5OHAxusak/MFS1p+KtlK0TJT5chr ZerNtkF8DIQ1kMNC+ktSqH8cMhl6N0vYHAw+TjBs9D/L8uM7mHcNDeOkXC1Jbcxd5E4i PEzCuaygHS5puge3dpnxrq8/75U0zFavn2nIw9By0HBLfs6BL+bjjaeHXAMO/Prpm2I7 evKRPW/GOllZ+eZuNvKo65yJWQe2uNTwaA0RncCEqo+R6a9Q0h3vDrz2U9/mZX70CI6e FCQw4h8ORVxW3+exxvDhmX5eyiKKx9jqc2xtCXN0rGLrO30jSuJGsUiiayKtafZMKlid orvw== X-Gm-Message-State: AO0yUKXzgZy7mQ55ChigMk7LKN56Q6BpGgnGPHilKGEgwn4yd02/4SHD ZYT34mSJIq9qA9xAdM7b0rtwDS5+dKw= X-Google-Smtp-Source: AK7set/2J8KUumsEZ/0orhhZ+Y6s/zP6QMsk4a+yqlkwHuIWJVWfr/HftGogZhR0NE5sVdiuRfgOT6Lze5Y= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:e1f6:21d1:eead:3897]) (user=surenb job=sendgmr) by 2002:a62:8387:0:b0:593:974c:cba7 with SMTP id h129-20020a628387000000b00593974ccba7mr6413186pfe.5.1677519446727; Mon, 27 Feb 2023 09:37:26 -0800 (PST) Date: Mon, 27 Feb 2023 09:36:20 -0800 In-Reply-To: <20230227173632.3292573-1-surenb@google.com> Mime-Version: 1.0 References: <20230227173632.3292573-1-surenb@google.com> X-Mailer: git-send-email 2.39.2.722.g9855ee24e9-goog Message-ID: <20230227173632.3292573-22-surenb@google.com> Subject: [PATCH v4 21/33] mm/mmap: prevent pagefault handler from racing with mmu_notifier registration From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, chriscli@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, michalechner92@googlemail.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com, Suren Baghdasaryan X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: F242940019 X-Rspam-User: X-Stat-Signature: orfbi1epaze6jbidjpzdiijt7qpoij5t X-HE-Tag: 1677519447-944288 X-HE-Meta: U2FsdGVkX1/9wbWRuLj4OE6iKRv9LJb99qEoWAPcgCo0zxMgIe+Q02OZbH7ihdYDAagJA023Qkv4vwkNMqAcW/gQwYP2zkFOTmNrjaSkZ+SKXJW/IL29SPkxppMtHW/qLu4yPUBBJtK7l+FHwhVT1nUO5IAs/ZLmpqSCK7GCxezjECNPnN4z78dUg/wR7T5FXoviT/JqdO2oOjt2erq1cwZSi7tOUcdW2eB+u8L8lPRwnKQYF4cOBmKGcq4ppeO9/lVih1Ylc9BohzECTbMXEQ55YaB+I1IOEtDm/gLWL3ON1vGQyRr/ueTX2vx17wtOaKdCPuVVWcZLBtTCL3UOCWQ5tun0ryCwR+yzgDyF8EhUTO5W4JAPnH4hh+guu9G5dXf/Oid/DcK1fXHWRTaX/U5Ux1OR7ZB7llGxagGK1NUoBrmH9F1d47trIjYJJNsDUQiO06RJ3IWRnp+NZ5oMU0WM1enwk1vMdSFJPZYZbdl7DlFSyFuRjzjk7zHR45xoMzptm6it2VqYHybJ0Q+6W01C6rxoOcpB+PyCou18Rdkp+k73VJsWdRUB8wWGdf57NhyzemikyN7h88DYvGD2DpTE0uWM44IrR5lVVwhDzhJ3bzNHeU5KuMoQihPNUXZUTQz8IgwtVps7qkW4ZZCWKtHk63QNts4XxzJT/RqAc6ooOF4wo/VuLTbQZq6e/xQDzlmRQrmYYLXH14V2LmzOtNVntf2ikV/FRE4EFX7p50/8T1QkDaUtB5BuIrsLgubtiaD8JXPkH+1+cVaD1wO9WbWRVHZBR/43zuSpvb85eRGB3mH6Zff+MyAtX9rl2r5/pUjNI4GTtnhXMyc86LiQTFX0n9ugh6ffi0McCw9c2a2EYAhon9MwA0gCxDIzL/XagRaEu5HMF4H0F0ZBeP9GlhpesVAz0+HGNxWYS2x9cBQvOCUygctug7fI1x8P531cVvEImalEbfNLx5bakFB y/UU5vje 7UOrN6cJ6evzB9zqE1kKTlzZ5Hj5r0wnbP56ImydBzaAvPcCEiqzMTIxT7z6fmYXWRMmWiXDof+Pqdk+aRpUoKAX2EOtTyVI3QVyoffE3ORg+peprL7az9cUmh4kJ8Jub7BeZ0jKKn+j/3rUrp3SER/ALs1a0vtvXldI0W8cJcyVyfH5ZCZPf8U5h+AfSR2ZIo/dWfMA85yHGxahUZrJ3ZoUWB0y2q8VzxOUSSl5fA2w3bBn3Go0Si5c5lZa7Ac+2BwyKEHmkB2aEIamHuYWitJQtWNsvBfIhzvg6Rk46FOHkzqjWVCHXJuVpEy21pkgrjxG6/CNa2WZzG48AOGUlMBP4nUZA21lHO47ghg+NTWePiIRGql9RsNm9DY/k5ZieVRMMi+xFT6XPIH4eFV80iF81LhelPMj14sMA8FWSF9Mvnzv93oHEMfWtGQD09qQ93pkfKYTnA4lTNFONwr7illYuklfiNHSPg5jhQXdXWo8sz0QJjafBEcRtwpJ3o0tfgOcYMxUCnMYKy8Dpm2B0XuIwKN3wXX05euEydB7rDWjOYP/jsCudTAxw0jHfBgxL20KPObCVZ3lWly6fezm9jn4ibJYfHfvFcaGXDBUrLuinR+mp67rgtzexRF4N+faPtMJ4aPwjXjr+jgz59XFWnYmqNaITszMb9AgWKazFcv9RtQ4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Page fault handlers might need to fire MMU notifications while a new notifier is being registered. Modify mm_take_all_locks to write-lock all VMAs and prevent this race with page fault handlers that would hold VMA locks. VMAs are locked before i_mmap_rwsem and anon_vma to keep the same locking order as in page fault handlers. Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index ec745586785c..b947d82e8522 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3486,6 +3486,7 @@ static void vm_lock_mapping(struct mm_struct *mm, struct address_space *mapping) * of mm/rmap.c: * - all hugetlbfs_i_mmap_rwsem_key locks (aka mapping->i_mmap_rwsem for * hugetlb mapping); + * - all vmas marked locked * - all i_mmap_rwsem locks; * - all anon_vma->rwseml * @@ -3508,6 +3509,13 @@ int mm_take_all_locks(struct mm_struct *mm) mutex_lock(&mm_all_locks_mutex); + mas_for_each(&mas, vma, ULONG_MAX) { + if (signal_pending(current)) + goto out_unlock; + vma_start_write(vma); + } + + mas_set(&mas, 0); mas_for_each(&mas, vma, ULONG_MAX) { if (signal_pending(current)) goto out_unlock; @@ -3597,6 +3605,7 @@ void mm_drop_all_locks(struct mm_struct *mm) if (vma->vm_file && vma->vm_file->f_mapping) vm_unlock_mapping(vma->vm_file->f_mapping); } + vma_end_write_all(mm); mutex_unlock(&mm_all_locks_mutex); }