From patchwork Fri Aug 4 15:27:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13342034 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE58DC04A6A for ; Fri, 4 Aug 2023 15:27:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F264E8D0006; Fri, 4 Aug 2023 11:27:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED5668E0001; Fri, 4 Aug 2023 11:27:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D774A8D0014; Fri, 4 Aug 2023 11:27:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C97948D0006 for ; Fri, 4 Aug 2023 11:27:37 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9D2E9C1535 for ; Fri, 4 Aug 2023 15:27:37 +0000 (UTC) X-FDA: 81086801754.06.9937E94 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf20.hostedemail.com (Postfix) with ESMTP id 9769E1C001C for ; Fri, 4 Aug 2023 15:27:35 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="jZlRzwl/"; spf=pass (imf20.hostedemail.com: domain of 35hjNZAYKCHkprobkYdlldib.Zljifkru-jjhsXZh.lod@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=35hjNZAYKCHkprobkYdlldib.Zljifkru-jjhsXZh.lod@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691162855; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LfajP5IIbFpg2Y7BQRxvk1sHiLCqjf/rsCkC3jBCtoI=; b=lWagMTxTnY7m4y+oE64RibVRdrv/uVBLfGc4YuI5lnCjS9ziG01zPlT+j9P+lEQkgZYgqU O7BHJW+4V9KUBBUXs7/l4JXIQkRcUCL7VdbsNIiTst0NlMfBxIxU/9VSHoHFuhRmoCiMuL 6rF5AVggYUjCZZBG+0Jsk/PPs46QqC0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691162855; a=rsa-sha256; cv=none; b=lRuPQ/tFcHcV/FHTWJT9xONun2MMiWlfZnrwX0jLqHR1ax6Lwp9OwWuODyqCqLCVfexl7T S5qXdH+o/ym50uLWgOJ7RyG9crFtoB65yFBDhr7+rgWq7sBnk6g4AVxfgoA14405Z9qXO9 zgDorsZq8cl0VO4910ZDUjjUGGX3Ko4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="jZlRzwl/"; spf=pass (imf20.hostedemail.com: domain of 35hjNZAYKCHkprobkYdlldib.Zljifkru-jjhsXZh.lod@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=35hjNZAYKCHkprobkYdlldib.Zljifkru-jjhsXZh.lod@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-5866e5f6e40so51348037b3.0 for ; Fri, 04 Aug 2023 08:27:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691162854; x=1691767654; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LfajP5IIbFpg2Y7BQRxvk1sHiLCqjf/rsCkC3jBCtoI=; b=jZlRzwl/sZxeKY0Q22GmSPxgFIqyr2sjFjG2dJWAnROoCqd561GTzeSUKMtg/2mqt1 ggyBLLrU1fznQHJEpV6iArmu4q/1ElFb+5c8i813a5QvYsRPPftZDP04yYsFkEpBfTFv biqZHQTtxZS7XMj4IKe/OZIiQ2Hzx3Sc4KiQkuiuVJ83Jibh5wp9rdvvqebLNTk9QL7W NA08SkaqEcu5wURtIv2bzCzal6v9uxf83gQzzLLvQVI6Kfk5jLDfOArO5bHqXdZSMQ1C ssQWic0dvDYtH0HIwM19ShxQ3BVwkNI58J3XyBg9Vj3lhfWdSX5jmfTA1qFRH0njGemz MjSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691162854; x=1691767654; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LfajP5IIbFpg2Y7BQRxvk1sHiLCqjf/rsCkC3jBCtoI=; b=C8/oNXu+GQxOUdiH0U6pLc0+4wnnxIDzugLgsJQ+MfRa+9R7Uc80ItinF1JGwpq+x7 6CJ9bBVZsH4LA7SxLhWSNhS3y46ryDJVtaDvx6I47E4cesZqoruZZzD8IPynJGLmudZN xR92WH0vTOCAevlYXBWNFtzi361/RushCL4hmgiBxOykwsaDPqyQI8Cr2S9OiFDJwTmA JACyVamVihLHXkgn56AqZByVeyre73dh9Vy6HN1Z8LLIjJ+8NrcwnwvKlqbSpJJrB6Tl L8TNIhFXu5IRH3l0rp8et+Y+i06D5v5rU56nKas/FJgGP+k52xq6yL4NNfh2vs5WL0aU EB8Q== X-Gm-Message-State: AOJu0Yz5NYNUlmU4C7hsJLEQ+HykOdNWiolo+Luwv6feY7C+K9W97gkc oxn01bay2U4u772MRUOG4tcOGqjgMqE= X-Google-Smtp-Source: AGHT+IHa9ndTaenWukiguHAw5+MvFDtv8DkvZBd31Rp3irRnrIogFfiHRjgfkWXCeFLeklAX7f0FtCBLNkg= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:43a7:a50f:b0fd:a068]) (user=surenb job=sendgmr) by 2002:a05:690c:448a:b0:583:9d8d:fb0d with SMTP id gr10-20020a05690c448a00b005839d8dfb0dmr1503ywb.0.1691162854651; Fri, 04 Aug 2023 08:27:34 -0700 (PDT) Date: Fri, 4 Aug 2023 08:27:21 -0700 In-Reply-To: <20230804152724.3090321-1-surenb@google.com> Mime-Version: 1.0 References: <20230804152724.3090321-1-surenb@google.com> X-Mailer: git-send-email 2.41.0.585.gd2178a4bd4-goog Message-ID: <20230804152724.3090321-4-surenb@google.com> Subject: [PATCH v4 3/6] mm: replace mmap with vma write lock assertions when operating on a vma From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: torvalds@linux-foundation.org, jannh@google.com, willy@infradead.org, liam.howlett@oracle.com, david@redhat.com, peterx@redhat.com, ldufour@linux.ibm.com, vbabka@suse.cz, michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, hannes@cmpxchg.org, dave@stgolabs.net, hughd@google.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, stable@vger.kernel.org, kernel-team@android.com, Suren Baghdasaryan , "Liam R . Howlett" X-Rspamd-Queue-Id: 9769E1C001C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: jsifbrqf1y3b3xwcojqp13hgbakh5dm3 X-HE-Tag: 1691162855-585624 X-HE-Meta: U2FsdGVkX1+X/bnr7jB7h/hO3TvlE3S/72d25/eS7W5yhS8OHMnC/bYDBU0MppaqAt6tkhiZnrpcxC/lozoTpfaQ6WvDyeQAn0R1DjrNsKb3nGQ9BLxNg2hEWpi0N7YjTGiEfVPpJABlQnRdSns2NGfvftOMluFxvTURVEvwQrDoWFdDFwPFdYIZKe+lT1+qxIA4ypDkkJVI2+K1Syv6i6C8kJ9TO+W58w7sCMRhdi+yXntAWqZH0Yqm4yCHN9x3fCUSxL5LjZpWsvX7XsJnJ2380GaxYKjwoPrsOETX1B7k4OINoEh9yYpS0rA3FDVllyTSc1plouMLFSjyDMvd1OWzncHlrkq4H2XvA3wWdGmZXn9WfW54xfsej5i5Fc8hmFj0+QSf7xaKFXIGaL5J6RMo4wrq824F1ujIbuvvQZD6oeS6d1F9+P07x/Y2s15QMG1BRWP1qynGR9iAyMq8rC+vFvE1rYfUZjc4I9B7xSq2WbSELDqW4Dn13Eo/fylgmNEE+UWjag7fe0lw4AzwW1T4/Aw8wtg6Q/isUWBXBZ6v0EmfOK1mNfOYzL10IJALbJNmVQh7tg+UGEcFTKQ9IpO8psk81RkxRPRW/L7r8Hd8L5wHuN7/GqACstxHq0eClxE/i1WYY0AJsOIaADDbCvZhS9lT0F2y3Hlqmvb1sDE70YbenkYMK/L1/z+HwD2wDC5AEO0ZM4AQP/KUZS/+Wtml2onRxleYOveRdwxY0zUjfpo8Vrh8zuJdVoIL+KwwFhws6Y8kH6ZzoPZYVqDXp1K7HkzUy8J9ott1Wx0csV6k/UnxPV/4gT79Kjv2aL865BssUjxKSZ3OBWXC9yD4rozcLw19ue0CGKqpFpTbay4SwAfZADGfm9DwaNXLYhYRfCSCpPYezZb9N9CcxHPefp5PbbaUoGgcSJl7WdIS+pdw22UA9piYGGEwEyS+wRbknic1Yj4z5a2Qm97T1G0 Dp8hSMXF KpZYyUtg757AjuoGnQGfyVVtFCEZyCUqjh5QAoZ3JEVDU0caJriCZ5qzHA7hq/1cJLMtwcQcbZKNnGMwaDpWIO39YKEiXiEpVw8FFXF6WyAvTesIby785kpBPS/InQlyk0gB6mkMXjgNAidBUbBzHVxw26zkfZuoTspTPPiHOiodAxhI6KHfmqBueIA3zlEdwo14+7zIwCvAEQ1KeljX0GsnavRG5FGfLXswCaktpnve7dGtNtRnAU/Gx7HNYXcPUuZmQlGUDLbxvsordrWSwFepLh6atv6FkIAVieT71nzZoNMVo3z//nFXBIVHm6rhx3wIgvc/UZZRv76ZW0kp82X0Apu3a0zWTs59C3EKyKpvy2UhDWxlW9FckKtFTGrJ/ddhuA0OsWDqSy71aEKd9MqZjJCyKuJ7SVyNxpdrbf2e808qM9wp5Vsx89SKedzyYGaN6QPLnKo0ZBPWoVs1G/x9GCpHdY8NoU1O08nrCpAyXwS5R0hPxkrRshH8AYQTaXHqPWjt5alIPbbPXXF9J8J/sLYQKZ5mdPGzlPkw5C5dbJ6lPLwCW7aG0q5zl0rTb6WE687MYXdQWOko8YicT8Z5ajzX0zLpaFcgodBFq1+NH2vBX7Tp/XrjZWrp8Yw8RE11Q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Vma write lock assertion always includes mmap write lock assertion and additional vma lock checks when per-VMA locks are enabled. Replace weaker mmap_assert_write_locked() assertions with stronger vma_assert_write_locked() ones when we are operating on a vma which is expected to be locked. Suggested-by: Jann Horn Signed-off-by: Suren Baghdasaryan Reviewed-by: Liam R. Howlett --- mm/hugetlb.c | 2 +- mm/khugepaged.c | 5 +++-- mm/memory.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 64a3239b6407..1d871a1167d8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5028,7 +5028,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, src_vma->vm_start, src_vma->vm_end); mmu_notifier_invalidate_range_start(&range); - mmap_assert_write_locked(src); + vma_assert_write_locked(src_vma); raw_write_seqcount_begin(&src->write_protect_seq); } else { /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 78c8d5d8b628..1e43a56fba31 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1495,7 +1495,7 @@ static int set_huge_pmd(struct vm_area_struct *vma, unsigned long addr, }; VM_BUG_ON(!PageTransHuge(hpage)); - mmap_assert_write_locked(vma->vm_mm); + vma_assert_write_locked(vma); if (do_set_pmd(&vmf, hpage)) return SCAN_FAIL; @@ -1525,7 +1525,7 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v pmd_t pmd; struct mmu_notifier_range range; - mmap_assert_write_locked(mm); + vma_assert_write_locked(vma); if (vma->vm_file) lockdep_assert_held_write(&vma->vm_file->f_mapping->i_mmap_rwsem); /* @@ -1570,6 +1570,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, int count = 0, result = SCAN_FAIL; int i; + /* Ensure vma can't change, it will be locked below after checks */ mmap_assert_write_locked(mm); /* Fast check before locking page if already PMD-mapped */ diff --git a/mm/memory.c b/mm/memory.c index 603b2f419948..652d99b9858a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1312,7 +1312,7 @@ copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) * Use the raw variant of the seqcount_t write API to avoid * lockdep complaining about preemptibility. */ - mmap_assert_write_locked(src_mm); + vma_assert_write_locked(src_vma); raw_write_seqcount_begin(&src_mm->write_protect_seq); }