From patchwork Tue Oct 4 00:37:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12997902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 971EBC4332F for ; Tue, 4 Oct 2022 00:37:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DDC06B0074; Mon, 3 Oct 2022 20:37:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 267056B0075; Mon, 3 Oct 2022 20:37:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01B0E6B0078; Mon, 3 Oct 2022 20:37:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BF8256B0074 for ; Mon, 3 Oct 2022 20:37:13 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8ACD68033A for ; Tue, 4 Oct 2022 00:37:13 +0000 (UTC) X-FDA: 79981402746.11.B42AFE3 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 37C16C0010 for ; Tue, 4 Oct 2022 00:37:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664843832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=agJ87AhkrOU4WXbJTuk9++7as6a20Yx3hvs9+IM86N0=; b=b/iPtf+JA0AP1JX43pGCuUZuyxPyRpP7kbweCep9Cu3WMMtcPG+knu+6w6sfyOOHCNYBqB YwYwH38j/iLAuXZeU6pyTVSfzmrSExBHIjp++ppX1K/AW6bsEzhLIHRfkc84u2jb3DVUXf UAIznu2I/4rwYZsR6fhsWYG66I8+GxE= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-638-o5X4LcEHNjOcW36D8mxBkg-1; Mon, 03 Oct 2022 20:37:09 -0400 X-MC-Unique: o5X4LcEHNjOcW36D8mxBkg-1 Received: by mail-qk1-f199.google.com with SMTP id h8-20020a05620a284800b006b5c98f09fbso10522340qkp.21 for ; Mon, 03 Oct 2022 17:37:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=agJ87AhkrOU4WXbJTuk9++7as6a20Yx3hvs9+IM86N0=; b=4WPuiNln8txV987AD7Jt7Q3THaZS3BKsK3QdAlTAIQIdnHicB0QozbLAFzXUxvAbC6 MVUikRwfmpzB6uHgXRZKAVeUUsniCphATTCp40BRkn7/rajECYpH/aD+4ffaVRYW0a/s xdYwdAzrcciAaoVxMFeqUn7p/xjgysUUxVOyXTEkDdZhKMZHDgAV/hl399LPMEQgO4ky lBEyzc7BKer0SAqa+dK+gmXE1yUPoqOblvH2Ba5FQoS6Kuhwce4C87YSRhBF5hbrH/fB iMSMy/llge3f/GP6mQxX7RuuYMmGWKnNaCZ4MkfezqEhZofMp/kMQwdbgbOtFfrheUMh shEw== X-Gm-Message-State: ACrzQf36qhCUfeKbfzuAuCrk/VOhk5RCptfqfZwe3a4ykv06QfymO/V3 idoxPhUvPcJemaTfZeRRNtKHQY26BYCi0L3ptXOrtF8wdSrlCEXh17u/XvW4GIMo8eECE554Hut kyPM9J4FjELk= X-Received: by 2002:a05:620a:2809:b0:6bc:60fa:6b93 with SMTP id f9-20020a05620a280900b006bc60fa6b93mr15045485qkp.254.1664843829350; Mon, 03 Oct 2022 17:37:09 -0700 (PDT) X-Google-Smtp-Source: AMsMyM74Q1B/bbLHC2GBtqEZOKQ/JZbvCEg1IiGQX5mCpjocQh5bJvbd6ZJW2ioDiKMXCi//B1kRGA== X-Received: by 2002:a05:620a:2809:b0:6bc:60fa:6b93 with SMTP id f9-20020a05620a280900b006bc60fa6b93mr15045473qkp.254.1664843829115; Mon, 03 Oct 2022 17:37:09 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u30-20020a37ab1e000000b006bb9125363fsm12228104qke.121.2022.10.03.17.37.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Oct 2022 17:37:08 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: peterx@redhat.com, Andrew Morton , David Hildenbrand , Nadav Amit , Mike Kravetz , Andrea Arcangeli , Axel Rasmussen , Mike Rapoport Subject: [PATCH v2 1/3] mm/hugetlb: Fix race condition of uffd missing/minor handling Date: Mon, 3 Oct 2022 20:37:03 -0400 Message-Id: <20221004003705.497782-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221004003705.497782-1-peterx@redhat.com> References: <20221004003705.497782-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664843833; a=rsa-sha256; cv=none; b=ByMEEnMitltbzYXxFtw6tmIoChuRJ1EcDtl3NLa04e0cwt0GfXYjR2fr8AOuCNSKpH7kUO 52kt+tdHb9qQfJrB0bY6yVmassKGClfdtlGKrfM7I/Apv5uW59hke8dDnJ8Xs5BD8A0eBp ovUX8EAVASby2SL2/fhVX7+5T6yv2Ec= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="b/iPtf+J"; spf=pass (imf22.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664843833; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=agJ87AhkrOU4WXbJTuk9++7as6a20Yx3hvs9+IM86N0=; b=iIbAKo/dil0EfI0XggG3b1wAM+Iwrq4h1lw8D/s0BiUvEQwY6167Ouwsx9K5Unu8NeB7yv yjLtYyfYvQIhBr+L+4ZOMRDKUx6Dxn4NRO2/8ojQb7vZCmCNdfGYAOQAPDUNsfbianVZ8H tg4R2gS74GltSiBurJEXnw9ITtRX8zw= X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 37C16C0010 X-Rspam-User: Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="b/iPtf+J"; spf=pass (imf22.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: 9ga73mctaudaqsc41y65aqu94n8zo49u X-HE-Tag: 1664843833-188808 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After the recent rework patchset of hugetlb locking on pmd sharing, kselftest for userfaultfd sometimes fails on hugetlb private tests with unexpected write fault checks. It turns out there's nothing wrong within the locking series regarding this matter, but it could have changed the timing of threads so it can trigger an old bug. The real bug is when we call hugetlb_no_page() we're not with the pgtable lock. It means we're reading the pte values lockless. It's perfectly fine in most cases because before we do normal page allocations we'll take the lock and check pte_same() again. However before that, there are actually two paths on userfaultfd missing/minor handling that may directly move on with the fault process without checking the pte values. It means for these two paths we may be generating an uffd message based on an unstable pte, while an unstable pte can legally be anything as long as the modifier holds the pgtable lock. One example, which is also what happened in the failing kselftest and caused the test failure, is that for private mappings wr-protection changes can happen on one page. While hugetlb_change_protection() generally requires pte being cleared before being changed, then there can be a race condition like: thread 1 thread 2 -------- -------- UFFDIO_WRITEPROTECT hugetlb_fault hugetlb_change_protection pgtable_lock() huge_ptep_modify_prot_start pte==NULL hugetlb_no_page generate uffd missing event even if page existed!! huge_ptep_modify_prot_commit pgtable_unlock() Fix this by recheck the pte after pgtable lock for both userfaultfd missing & minor fault paths. This bug should have been around starting from uffd hugetlb introduced, so attaching a Fixes to the commit. Also attach another Fixes to the minor support commit for easier tracking. Note that userfaultfd is actually fine with false positives (e.g. caused by pte changed), but not wrong logical events (e.g. caused by reading a pte during changing). The latter can confuse the userspace, so the strictness is very much preferred. E.g., MISSING event should never happen on the page after UFFDIO_COPY has correctly installed the page and returned. Cc: Andrea Arcangeli Cc: Mike Kravetz Cc: Axel Rasmussen Cc: Nadav Amit Fixes: 1a1aad8a9b7b ("userfaultfd: hugetlbfs: add userfaultfd hugetlb hook") Fixes: 7677f7fd8be7 ("userfaultfd: add minor fault registration mode") Co-developed-by: Mike Kravetz Signed-off-by: Peter Xu Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 49 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 9679fe519b90..fa3fcdb0c4b8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5521,6 +5521,23 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_area_struct *vma, return ret; } +/* + * Recheck pte with pgtable lock. Returns true if pte didn't change, or + * false if pte changed or is changing. + */ +static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, + pte_t *ptep, pte_t old_pte) +{ + spinlock_t *ptl; + bool same; + + ptl = huge_pte_lock(h, mm, ptep); + same = pte_same(huge_ptep_get(ptep), old_pte); + spin_unlock(ptl); + + return same; +} + static vm_fault_t hugetlb_no_page(struct mm_struct *mm, struct vm_area_struct *vma, struct address_space *mapping, pgoff_t idx, @@ -5562,9 +5579,30 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, goto out; /* Check for page in userfault range */ if (userfaultfd_missing(vma)) { - ret = hugetlb_handle_userfault(vma, mapping, idx, - flags, haddr, address, - VM_UFFD_MISSING); + /* + * Since hugetlb_no_page() was examining pte + * without pgtable lock, we need to re-test under + * lock because the pte may not be stable and could + * have changed from under us. Try to detect + * either changed or during-changing ptes and retry + * properly when needed. + * + * Note that userfaultfd is actually fine with + * false positives (e.g. caused by pte changed), + * but not wrong logical events (e.g. caused by + * reading a pte during changing). The latter can + * confuse the userspace, so the strictness is very + * much preferred. E.g., MISSING event should + * never happen on the page after UFFDIO_COPY has + * correctly installed the page and returned. + */ + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) + ret = hugetlb_handle_userfault( + vma, mapping, idx, flags, haddr, + address, VM_UFFD_MISSING); + else + /* Retry the fault */ + ret = 0; goto out; } @@ -5634,9 +5672,14 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, if (userfaultfd_minor(vma)) { unlock_page(page); put_page(page); - ret = hugetlb_handle_userfault(vma, mapping, idx, - flags, haddr, address, - VM_UFFD_MINOR); + /* See comment in userfaultfd_missing() block above */ + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) + ret = hugetlb_handle_userfault( + vma, mapping, idx, flags, haddr, + address, VM_UFFD_MINOR); + else + /* Retry the fault */ + ret = 0; goto out; } } From patchwork Tue Oct 4 00:37:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12997901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4BDEC433F5 for ; Tue, 4 Oct 2022 00:37:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 242536B0073; Mon, 3 Oct 2022 20:37:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1EEE06B0074; Mon, 3 Oct 2022 20:37:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0940A6B0075; Mon, 3 Oct 2022 20:37:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id ED28D6B0073 for ; Mon, 3 Oct 2022 20:37:12 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C4C1DA0500 for ; Tue, 4 Oct 2022 00:37:12 +0000 (UTC) X-FDA: 79981402704.19.50F06CF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 72C58180018 for ; Tue, 4 Oct 2022 00:37:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664843831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x4xYjXYzlE422UA6NYhxH1mMMh8xu1ZsHfGS6BWev5s=; b=B+wk3O6HNItR/O+4GHzbhwHZV3U5YrGJtkTUzcThGODBdfCePalOQEUr3zByXIvjbmsPXI gzADlGbYBGRPQpNSvfuEzxBd1guZagpJJ5ho2dFinM4Mk/XxCwG10FptfwxYRFKzxyu/ZR ZAGZT0FuNORarDifMCoosSTSOcZsBEw= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-653-ThTcsKajMcCIHgG7AqD6qQ-1; Mon, 03 Oct 2022 20:37:11 -0400 X-MC-Unique: ThTcsKajMcCIHgG7AqD6qQ-1 Received: by mail-qv1-f72.google.com with SMTP id i1-20020a056214020100b004afa3dbf35dso8025019qvt.14 for ; Mon, 03 Oct 2022 17:37:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=x4xYjXYzlE422UA6NYhxH1mMMh8xu1ZsHfGS6BWev5s=; b=dZ6/7gKHtDLlwkEySRz92Kr64b+RC9C1YuRVTF1hAgDzkHpPRDrjaEnoSga6aUsTqr 5sMvHiXCBwwFhkrIpeY+r7SkvLL8Lrp0o7Ce5pi61Y/14/zNYuzHdMj6uu6tU5lvfRBN CHfaYfMrXiJL1SxksBuSAHyW3gSMJUbzAwCJHgwBufWmscui6PoNKzG7t4FezwKKoOaA cPmFluBu+Htd6KQTvkAYBdeTUqSbE1+fZHcQR3C3ShZiMloWuEPAlfgxhWxLAAdBj9wJ IkQI/CR2Dlq8OgwHLAX0qNEBLKeJ5m9Fa/B2niNiuCLhrsO19/69L70V85NN4IV+xJQi 9/eA== X-Gm-Message-State: ACrzQf1xQitp5ec8nDhiX81ePEcInI/UWtT2ZgiShT1pHaMMYaT+27AM dBmQz9+z0y8zVnYMI+Aiy7zxy5SgNBN9FZ9cyaHFMazYSsAOIr1RwxwqeXIb94MBj6QoADPClI8 zgv6+fWkRNz8= X-Received: by 2002:a05:620a:d94:b0:6bc:5a8c:3168 with SMTP id q20-20020a05620a0d9400b006bc5a8c3168mr15260767qkl.56.1664843830681; Mon, 03 Oct 2022 17:37:10 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5mTbnDGFwa4OqFZLWTwpewQHP9t+R+EHTP9kL897GWBWxesSU9GeVQIGXXHofPQ4um9Sh1gw== X-Received: by 2002:a05:620a:d94:b0:6bc:5a8c:3168 with SMTP id q20-20020a05620a0d9400b006bc5a8c3168mr15260759qkl.56.1664843830507; Mon, 03 Oct 2022 17:37:10 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u30-20020a37ab1e000000b006bb9125363fsm12228104qke.121.2022.10.03.17.37.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Oct 2022 17:37:10 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: peterx@redhat.com, Andrew Morton , David Hildenbrand , Nadav Amit , Mike Kravetz , Andrea Arcangeli , Axel Rasmussen , Mike Rapoport Subject: [PATCH v2 2/3] mm/hugetlb: Use hugetlb_pte_stable in migration race check Date: Mon, 3 Oct 2022 20:37:04 -0400 Message-Id: <20221004003705.497782-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221004003705.497782-1-peterx@redhat.com> References: <20221004003705.497782-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664843832; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x4xYjXYzlE422UA6NYhxH1mMMh8xu1ZsHfGS6BWev5s=; b=uJ67EPclaX5v9QmOKvKV/dhhkStBgJpDYsILQZz8U3oRy2WpKmaeuj/o8owkB33lllmts8 Ph62pzT8KHO90sRzqaVLEvPi8cDRzF3Ps8CS3pNeRl/ilZaMV4UIbzW1HBGUEcrDuj+KDF 9od0SSlW7wjR4LFTO4f6kaXZjywAAO4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=B+wk3O6H; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf24.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664843832; a=rsa-sha256; cv=none; b=DAuCagI778ZQ1LGycLZcuD7sUD+Rhg665iEEAa48d+WGr+UjT7LZGwTS9nT0S005e4EbdS r/wz11o4PpM1soK4/2AQ5e79gDwA4N/pV32951cygVu7zZgrQuX1HuUECz99IDxIxDFVOp PmlNaPFArgizSFB7lcUhmTSqN5HlOA0= X-Rspam-User: X-Stat-Signature: 4yidui8ru41ghatazi6p53u86okd8sdd X-Rspamd-Queue-Id: 72C58180018 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=B+wk3O6H; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf24.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam01 X-HE-Tag: 1664843832-644380 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After hugetlb_pte_stable() introduced, we can also rewrite the migration race condition against page allocation to use the new helper too. Signed-off-by: Peter Xu Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand --- mm/hugetlb.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fa3fcdb0c4b8..e762c5369a6f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5620,11 +5620,10 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, * here. Before returning error, get ptl and make * sure there really is no pte entry. */ - ptl = huge_pte_lock(h, mm, ptep); - ret = 0; - if (huge_pte_none(huge_ptep_get(ptep))) + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) ret = vmf_error(PTR_ERR(page)); - spin_unlock(ptl); + else + ret = 0; goto out; } clear_huge_page(page, address, pages_per_huge_page(h)); From patchwork Tue Oct 4 00:37:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12997903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E64DCC433F5 for ; Tue, 4 Oct 2022 00:37:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 80C776B0075; Mon, 3 Oct 2022 20:37:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7BBF76B0078; Mon, 3 Oct 2022 20:37:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65CD86B007B; Mon, 3 Oct 2022 20:37:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 577456B0075 for ; Mon, 3 Oct 2022 20:37:16 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2B036A024D for ; Tue, 4 Oct 2022 00:37:16 +0000 (UTC) X-FDA: 79981402872.26.0A191F7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id C26D9140017 for ; Tue, 4 Oct 2022 00:37:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664843835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/5RzxXUZxi/MrG066VL+DnGEo1qQl33988Rm0NK9+zA=; b=DPf/j94C6+6CGdgVPxH/Nsb88vKxSg+Go+vg29C9i4208SXa2hXJdyuwzdfTbGACy9sa3O f2/tNaiqb8f/Q2P45bAzLj9a+Sp182TYPqJmaAd/NjYsxN06FV23l4KaDN44j2plvyAL4D SBMwr+jjhRLw3PIF/h4Hh5uCsPCazo4= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-627-FKMt01wyPI-HYbqRCYfeWA-1; Mon, 03 Oct 2022 20:37:12 -0400 X-MC-Unique: FKMt01wyPI-HYbqRCYfeWA-1 Received: by mail-qk1-f199.google.com with SMTP id j13-20020a05620a288d00b006be7b2a758fso10282922qkp.1 for ; Mon, 03 Oct 2022 17:37:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=/5RzxXUZxi/MrG066VL+DnGEo1qQl33988Rm0NK9+zA=; b=qfwc6BlcUeG5wuJ3lmGZfsYvxLoeDEoFJ7jrCjTzjSQGZ7Dtplb1sGTfn9+4QElivY 8S/FkCTxH/xHuZynRb6cUcmRhLiP9DGg2v4kOfYLXO78SROlbuw2s9HdJ9HRyK1WqLMP 463WkcEkKcMXXxAEJIf4uVTCQDzRkHy2tvkz5C+tPEKTBB+TjKJCbXblDIlVxZI+PHCV leUAjEyL9b2JP3ngrlnsih+NNxNNo53k4ssDwq5bPLMHOP72azTnzKGrFms8HWft5Ih7 0zznMq2jlIeTPwlknf5lD9D2so3wMJe/6le7fpUPH53PSeAZ3h278SjOqFCAK5+F8R4h 1Heg== X-Gm-Message-State: ACrzQf3wY3Dzgez83iuTb1MaMtp8ai2iiuaSnp6HfSAKl3eu2l8O7Ohb CjeeGUHyAahdnGcjdhby/YEk8BSs0kOZsk0mcq3BDsftKGS9uyhYnQ/IymN/kIuD6+xHGj54dJs knR4+n/dVQPk= X-Received: by 2002:a05:620a:46a4:b0:6ce:c4af:5a54 with SMTP id bq36-20020a05620a46a400b006cec4af5a54mr15459014qkb.377.1664843831965; Mon, 03 Oct 2022 17:37:11 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5UooKxylyfkc7pYh77HC484ZRa3g4v9oRRuF3krQ3VgFlBVG5fQ7DBFiKYYqVi1SHhupwxmg== X-Received: by 2002:a05:620a:46a4:b0:6ce:c4af:5a54 with SMTP id bq36-20020a05620a46a400b006cec4af5a54mr15459007qkb.377.1664843831739; Mon, 03 Oct 2022 17:37:11 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u30-20020a37ab1e000000b006bb9125363fsm12228104qke.121.2022.10.03.17.37.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Oct 2022 17:37:11 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: peterx@redhat.com, Andrew Morton , David Hildenbrand , Nadav Amit , Mike Kravetz , Andrea Arcangeli , Axel Rasmussen , Mike Rapoport Subject: [PATCH v2 3/3] mm/selftest: uffd: Explain the write missing fault check Date: Mon, 3 Oct 2022 20:37:05 -0400 Message-Id: <20221004003705.497782-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221004003705.497782-1-peterx@redhat.com> References: <20221004003705.497782-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664843835; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/5RzxXUZxi/MrG066VL+DnGEo1qQl33988Rm0NK9+zA=; b=hd/gJbuCqvFIypWK1ZL3FnBQjiXdX4NVkaotc6O3cj+GwrqFX9wKFleVgKMSXH729ka73R 4MOq32xBZ1WjF3y/leY9zjv1YqGB3Kfpu+dRNBxHOEkHHNM6Rw6HFfNUjL3lIjqvqDFb1K 1BBnUTWReMZUmb7LZSilNwb/Uy5MQ7s= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="DPf/j94C"; spf=pass (imf23.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664843835; a=rsa-sha256; cv=none; b=GhZomRwoAe4Ch2hl+vuHG+TKjub3jPApOxCHzesRUKuLKpF2XN2wPGI311FDnhQGsLDfxu jHKa5DWQau3wLeNNioBZqhrqEYpc/P/x89I7bh+UnP0B/It4o/XsEIcROJm0RR4o3pCsyC p9dH/t1n1PpiG7dUCfdTgx57wE1fm7Q= X-Rspamd-Queue-Id: C26D9140017 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="DPf/j94C"; spf=pass (imf23.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam06 X-Rspam-User: X-Stat-Signature: tc7wmxh4cbj553rpj7gcye1bkf9xwqeh X-HE-Tag: 1664843835-160592 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It's not obvious why we had a write check for each of the missing messages, especially when it should be a locking op. Add a rich comment for that, and also try to explain its good side and limitations, so that if someone hit it again for either a bug or a different glibc impl there'll be some clue to start with. Signed-off-by: Peter Xu Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand --- tools/testing/selftests/vm/userfaultfd.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c index 74babdbc02e5..297f250c1d95 100644 --- a/tools/testing/selftests/vm/userfaultfd.c +++ b/tools/testing/selftests/vm/userfaultfd.c @@ -774,7 +774,27 @@ static void uffd_handle_page_fault(struct uffd_msg *msg, continue_range(uffd, msg->arg.pagefault.address, page_size); stats->minor_faults++; } else { - /* Missing page faults */ + /* + * Missing page faults. + * + * Here we force a write check for each of the missing mode + * faults. It's guaranteed because the only threads that + * will trigger uffd faults are the locking threads, and + * their first instruction to touch the missing page will + * always be pthread_mutex_lock(). + * + * Note that here we relied on an NPTL glibc impl detail to + * always read the lock type at the entry of the lock op + * (pthread_mutex_t.__data.__type, offset 0x10) before + * doing any locking operations to guarantee that. It's + * actually not good to rely on this impl detail because + * logically a pthread-compatible lib can implement the + * locks without types and we can fail when linking with + * them. However since we used to find bugs with this + * strict check we still keep it around. Hopefully this + * could be a good hint when it fails again. If one day + * it'll break on some other impl of glibc we'll revisit. + */ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) err("unexpected write fault");