From patchwork Tue Oct 4 19:33:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12998646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD842C4332F for ; Tue, 4 Oct 2022 19:34:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E399F6B0073; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DC4216B0075; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B51166B007B; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8E0076B0073 for ; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5D059140EF8 for ; Tue, 4 Oct 2022 19:34:09 +0000 (UTC) X-FDA: 79984267818.04.323B737 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf01.hostedemail.com (Postfix) with ESMTP id 0CFFD4001E for ; Tue, 4 Oct 2022 19:34:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664912048; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NQaRIORouh5PGNwxNNc/P0MIv99YHmLbB7VcHRYE5ek=; b=WO3qtuQattYKX5xKW+yzgwhdGqBjz8CeTZ94XdTA0OgDL7vRit5hZiFkgw7Sesmgzmppgy KluTf5gpqJhNZCVYOTSUkHduJpnBCe3o8Z4jiRu59/awfrLtMa38pmkfIo1d0q/TGnVR1c G6eZaQeY1yAJone/QSg4XLbDVWjYxKc= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-546-9huCqXk5OHOtCsSj6GWn6w-1; Tue, 04 Oct 2022 15:34:04 -0400 X-MC-Unique: 9huCqXk5OHOtCsSj6GWn6w-1 Received: by mail-qk1-f198.google.com with SMTP id bm21-20020a05620a199500b006cf6a722b16so12419877qkb.0 for ; Tue, 04 Oct 2022 12:34:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=NQaRIORouh5PGNwxNNc/P0MIv99YHmLbB7VcHRYE5ek=; b=QtaT7dGaBGj4yf2pYdgJKylaq2YJDXak58OcROjJChAO5c2nE4IzMHVv7QrWRlCHBo 3MD9E2MRMcM/JU6LjloQI1edtalTsuiNtNOd6wJuMjImj1nK3EcRWbHvpnBdsmlMCZjE Dr2estBLz/YK5itahX5QZo4/JOVwv0D90ldxJCdL2aK7WyBJ2WUkwvVJeQKJZ0EKpHSG Mf3F/2yElIvGFHrlyy5ipv4IxYCepaS2c7dXfNGl7jgsOEcymFkkcT4jB/F0Q2eGepx6 sLNAyoXTEf5Cfi0N0tyLzRL0Qct4zH6Y4dWNM8iyeyxRorV08lm+UufvQ7ToqPDG+Who qZUA== X-Gm-Message-State: ACrzQf3+YEdFdR6xgeVfuwQ2m+9BiYCV+enjG4QBENMZSpsTLSQfg2Ti O8JMjpLsCAgIOyVUxGOK9usQk9n7JE7TvxJ0Q6t1vj3UOpDx8VltxhHPX7i56yyAVN2/tsJe+G0 DH1oYrmb/+c7KoggAxXRgQJv/2B53O1bQI0R8d8yRW8Jk4nBniAkrpvuC82IT X-Received: by 2002:ad4:5be1:0:b0:496:a686:2bec with SMTP id k1-20020ad45be1000000b00496a6862becmr21051724qvc.85.1664912044005; Tue, 04 Oct 2022 12:34:04 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5tzG6ZNsh+aR3v41H1W2JmcaMFErSNKPMzqwMZhRmPNkNIpkN7hRNRj2Y0ZR1OtrRNmgqlGQ== X-Received: by 2002:ad4:5be1:0:b0:496:a686:2bec with SMTP id k1-20020ad45be1000000b00496a6862becmr21051683qvc.85.1664912043713; Tue, 04 Oct 2022 12:34:03 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id z5-20020a05622a028500b00342fb07944fsm13299811qtw.82.2022.10.04.12.34.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Oct 2022 12:34:03 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Mike Rapoport , peterx@redhat.com, David Hildenbrand , Andrew Morton , Andrea Arcangeli , Nadav Amit , Axel Rasmussen , Mike Kravetz Subject: [PATCH v3 1/3] mm/hugetlb: Fix race condition of uffd missing/minor handling Date: Tue, 4 Oct 2022 15:33:58 -0400 Message-Id: <20221004193400.110155-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221004193400.110155-1-peterx@redhat.com> References: <20221004193400.110155-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664912049; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NQaRIORouh5PGNwxNNc/P0MIv99YHmLbB7VcHRYE5ek=; b=3DSeKMQ9PVOs7JIz+p+MdrmbNgeOJ4upjqIzwhdd1YnJaF4PbVmYDLSWWZlDnu/g2lUCoe C8hZcC9QZm6pGJr9sHWOKWt/c+UnboAjn7gEVt0YBFrENZu3ueojRhtYBr7Fv30eWlksQM 8+u3ugMHEmOpaUQNTj582rL+2CL6HfQ= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WO3qtuQa; spf=pass (imf01.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664912049; a=rsa-sha256; cv=none; b=vojbuMXzlpHJpzJAznhQbnGPI6ELcSI2E1ujr7cpgR1tZuEIcb1kbQjm9RKrhcoo2TIYpi LH5/rPWjMKfBHZXMs1HbniejkgsaGhD+AlsFtdjVDiXUrCX5HtwY8qZ3gZu6G20cSrRyew 0T80CGvFjAVVa4aesKkbckTRtTBx8cE= X-Stat-Signature: yqfi73oewfozy8i9puorwtmmya13cgtx X-Rspamd-Queue-Id: 0CFFD4001E Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WO3qtuQa; spf=pass (imf01.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam07 X-Rspam-User: X-HE-Tag: 1664912048-851723 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After the recent rework patchset of hugetlb locking on pmd sharing, kselftest for userfaultfd sometimes fails on hugetlb private tests with unexpected write fault checks. It turns out there's nothing wrong within the locking series regarding this matter, but it could have changed the timing of threads so it can trigger an old bug. The real bug is when we call hugetlb_no_page() we're not with the pgtable lock. It means we're reading the pte values lockless. It's perfectly fine in most cases because before we do normal page allocations we'll take the lock and check pte_same() again. However before that, there are actually two paths on userfaultfd missing/minor handling that may directly move on with the fault process without checking the pte values. It means for these two paths we may be generating an uffd message based on an unstable pte, while an unstable pte can legally be anything as long as the modifier holds the pgtable lock. One example, which is also what happened in the failing kselftest and caused the test failure, is that for private mappings wr-protection changes can happen on one page. While hugetlb_change_protection() generally requires pte being cleared before being changed, then there can be a race condition like: thread 1 thread 2 -------- -------- UFFDIO_WRITEPROTECT hugetlb_fault hugetlb_change_protection pgtable_lock() huge_ptep_modify_prot_start pte==NULL hugetlb_no_page generate uffd missing event even if page existed!! huge_ptep_modify_prot_commit pgtable_unlock() Fix this by recheck the pte after pgtable lock for both userfaultfd missing & minor fault paths. This bug should have been around starting from uffd hugetlb introduced, so attaching a Fixes to the commit. Also attach another Fixes to the minor support commit for easier tracking. Note that userfaultfd is actually fine with false positives (e.g. caused by pte changed), but not wrong logical events (e.g. caused by reading a pte during changing). The latter can confuse the userspace, so the strictness is very much preferred. E.g., MISSING event should never happen on the page after UFFDIO_COPY has correctly installed the page and returned. Cc: Andrea Arcangeli Cc: Mike Kravetz Cc: Axel Rasmussen Cc: Nadav Amit Fixes: 1a1aad8a9b7b ("userfaultfd: hugetlbfs: add userfaultfd hugetlb hook") Fixes: 7677f7fd8be7 ("userfaultfd: add minor fault registration mode") Co-developed-by: Mike Kravetz Reviewed-by: Mike Kravetz Signed-off-by: Peter Xu --- mm/hugetlb.c | 59 +++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 52 insertions(+), 7 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 6022dea6a634..1f059acc38f3 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5524,6 +5524,23 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_area_struct *vma, return handle_userfault(&vmf, reason); } +/* + * Recheck pte with pgtable lock. Returns true if pte didn't change, or + * false if pte changed or is changing. + */ +static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, + pte_t *ptep, pte_t old_pte) +{ + spinlock_t *ptl; + bool same; + + ptl = huge_pte_lock(h, mm, ptep); + same = pte_same(huge_ptep_get(ptep), old_pte); + spin_unlock(ptl); + + return same; +} + static vm_fault_t hugetlb_no_page(struct mm_struct *mm, struct vm_area_struct *vma, struct address_space *mapping, pgoff_t idx, @@ -5564,10 +5581,33 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, if (idx >= size) goto out; /* Check for page in userfault range */ - if (userfaultfd_missing(vma)) - return hugetlb_handle_userfault(vma, mapping, idx, - flags, haddr, address, - VM_UFFD_MISSING); + if (userfaultfd_missing(vma)) { + /* + * Since hugetlb_no_page() was examining pte + * without pgtable lock, we need to re-test under + * lock because the pte may not be stable and could + * have changed from under us. Try to detect + * either changed or during-changing ptes and retry + * properly when needed. + * + * Note that userfaultfd is actually fine with + * false positives (e.g. caused by pte changed), + * but not wrong logical events (e.g. caused by + * reading a pte during changing). The latter can + * confuse the userspace, so the strictness is very + * much preferred. E.g., MISSING event should + * never happen on the page after UFFDIO_COPY has + * correctly installed the page and returned. + */ + if (!hugetlb_pte_stable(h, mm, ptep, old_pte)) { + ret = 0; + goto out; + } + + return hugetlb_handle_userfault(vma, mapping, idx, flags, + haddr, address, + VM_UFFD_MISSING); + } page = alloc_huge_page(vma, haddr, 0); if (IS_ERR(page)) { @@ -5633,9 +5673,14 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, if (userfaultfd_minor(vma)) { unlock_page(page); put_page(page); - return hugetlb_handle_userfault(vma, mapping, idx, - flags, haddr, address, - VM_UFFD_MINOR); + /* See comment in userfaultfd_missing() block above */ + if (!hugetlb_pte_stable(h, mm, ptep, old_pte)) { + ret = 0; + goto out; + } + return hugetlb_handle_userfault(vma, mapping, idx, flags, + haddr, address, + VM_UFFD_MINOR); } } From patchwork Tue Oct 4 19:33:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12998645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B9B0C433F5 for ; Tue, 4 Oct 2022 19:34:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9EF366B0074; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CA376B0078; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A8146B0075; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 66F9A6B0073 for ; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6B5D7C0E3D for ; Tue, 4 Oct 2022 19:34:08 +0000 (UTC) X-FDA: 79984267776.18.EF0CDE3 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id EC7D31A000F for ; Tue, 4 Oct 2022 19:34:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664912047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4nzBQiXcgt3UF9Pzmx/yx9uJYBtOMvv3jFBGynNQW0E=; b=UTg/C/cGQWPjMgNX2nPW5K+BPM548NoAcP3aZejjIkAKjBxFcBQRlZ2oKC+dpVIso2Z3DD Nnltr6PTDa9tlDcDyo6zsMgV3p412UPtkfCh8Pkb2XlU8CED8SqvzSiIdqthelNt38fQ1I yOqS0nopnK5Ot77iBqx7sr2n6R45Sqo= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-577-_mQvLJicM7CskDn4CydUpA-1; Tue, 04 Oct 2022 15:34:06 -0400 X-MC-Unique: _mQvLJicM7CskDn4CydUpA-1 Received: by mail-qk1-f200.google.com with SMTP id bk21-20020a05620a1a1500b006be9f844c59so12412857qkb.9 for ; Tue, 04 Oct 2022 12:34:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=4nzBQiXcgt3UF9Pzmx/yx9uJYBtOMvv3jFBGynNQW0E=; b=ieZEU1h9CZTGCsBp+9x0hn25yXplfMqvp34MhV0UWv9ZlfojS96/YxqsUx9jbQlDgT tyepfjfkvnXRqFW2DOKQofRNZDppl46RUJ4+bOIigqFlphNVeFMsmmsYlG1YcORr6fwt rpJaT7kG9vnW6+jDy1ma/4MyE3722OASfTrDdtmPhBE7J6GBIqNbdMb/6HiFNtF7dEFM VPkKXeJA56UALS8+epvFiFuEA+QTwYCPEP0A2gHvt9CNqC6l0cYokA44hGf8kfFJVUbN icBiwGNLqOWUOOWQlQsVnGeIHEsxSGabmDV2XFIqyAGXMo7ffAUa9XC+RVaWz78hujUw OItQ== X-Gm-Message-State: ACrzQf2wDVqJ8yK6SV4wmBgYYhcEnWCg25Jhd+yatsOi0ODeVdpKHVyh +uZfLl/MrP63c6BkN7I+9nEJ92HUcsEeLn2eeEFHUkDSyEnonlOfa8Fgvw+kB6VjFSI/PwkdyGO 1nWK11WuwFneYO1OM7v2vLtrCsVPyrJnIy36RJZ5FfZLc50sHQIW6XGvmmzHB X-Received: by 2002:a05:622a:4c11:b0:35c:bbb2:d177 with SMTP id ey17-20020a05622a4c1100b0035cbbb2d177mr21244346qtb.314.1664912045541; Tue, 04 Oct 2022 12:34:05 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4kmipNt7Cgqge3wbSWm8qUnwYOKzrhBe+9IEVXWDGwMDz0ZKmsU4IruRK26u/ijQtioiyaFw== X-Received: by 2002:a05:622a:4c11:b0:35c:bbb2:d177 with SMTP id ey17-20020a05622a4c1100b0035cbbb2d177mr21244308qtb.314.1664912045240; Tue, 04 Oct 2022 12:34:05 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id z5-20020a05622a028500b00342fb07944fsm13299811qtw.82.2022.10.04.12.34.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Oct 2022 12:34:04 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Mike Rapoport , peterx@redhat.com, David Hildenbrand , Andrew Morton , Andrea Arcangeli , Nadav Amit , Axel Rasmussen , Mike Kravetz Subject: [PATCH v3 2/3] mm/hugetlb: Use hugetlb_pte_stable in migration race check Date: Tue, 4 Oct 2022 15:33:59 -0400 Message-Id: <20221004193400.110155-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221004193400.110155-1-peterx@redhat.com> References: <20221004193400.110155-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664912048; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4nzBQiXcgt3UF9Pzmx/yx9uJYBtOMvv3jFBGynNQW0E=; b=E3F5iX+K1f3X04jSSoERX5kvENiYcDqGTc+RnL+qySWopF6oK4T3AcMmTosOWQ6PO8brTP Y7/r18WquvTLauCjGICuSg2KCd6vCmE4e3tfQ3KTI97gB33wO0NsyidJjZvSyAjPJp0rkD 8yU+xA2Gi1fNaHU9wKbGdcgpvE+oteo= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="UTg/C/cG"; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664912048; a=rsa-sha256; cv=none; b=dnVcWKyO8pBtBeiwwC8UhfNy8FwjNQyH2ky6VScxv2eIVoHdhbF0nEmgY4mG3EsrzarWFR 2Va7BvNFv+renMKREd5eWmBKEQfRaShaw9/RuKAA/ZPrcJI/lzjjZh8zARSdckhH9Ltw9W +aH+nPFXwbFZrz0AXDY3nflt9/MYTuk= X-Rspamd-Queue-Id: EC7D31A000F Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="UTg/C/cG"; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam06 X-Rspam-User: X-Stat-Signature: afcea3tb34zt7qq5rkrjfmbdgop4b56k X-HE-Tag: 1664912047-109238 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After hugetlb_pte_stable() introduced, we can also rewrite the migration race condition against page allocation to use the new helper too. Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- mm/hugetlb.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1f059acc38f3..63fe47a0240a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5623,11 +5623,10 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, * here. Before returning error, get ptl and make * sure there really is no pte entry. */ - ptl = huge_pte_lock(h, mm, ptep); - ret = 0; - if (huge_pte_none(huge_ptep_get(ptep))) + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) ret = vmf_error(PTR_ERR(page)); - spin_unlock(ptl); + else + ret = 0; goto out; } clear_huge_page(page, address, pages_per_huge_page(h)); From patchwork Tue Oct 4 19:34:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12998647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51456C433F5 for ; Tue, 4 Oct 2022 19:34:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64BE28E0001; Tue, 4 Oct 2022 15:34:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 49BF68E0002; Tue, 4 Oct 2022 15:34:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11DD68E0001; Tue, 4 Oct 2022 15:34:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E53006B0078 for ; Tue, 4 Oct 2022 15:34:09 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B87A440179 for ; Tue, 4 Oct 2022 19:34:09 +0000 (UTC) X-FDA: 79984267818.30.66C91A3 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 23E3E18001E for ; Tue, 4 Oct 2022 19:34:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664912048; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ifodv0MqT+m7iheWymJyjwtmpsW2iF/ZrnT+XTcpadM=; b=d9bqvHzOhvjAjaAowAOgLbQHIYHrP2vCIVNUVKlxP2ESgp4TtQejHTmjV6FZeipTXu9Ymz 4xLntxeBBCfYlqudq8UE3uKeuy4CX+b4vUCN4IbkmgDtSYnCcEPNIFhgr3OR7JAsicpi1L 15P+edQIQvH0hZkSqe1jjgaFDu5AuyM= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-645-cZZ7uSVQNDq2mclvU5BzXA-1; Tue, 04 Oct 2022 15:34:07 -0400 X-MC-Unique: cZZ7uSVQNDq2mclvU5BzXA-1 Received: by mail-qk1-f200.google.com with SMTP id bs33-20020a05620a472100b006cef8cfabe2so12450570qkb.12 for ; Tue, 04 Oct 2022 12:34:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=ifodv0MqT+m7iheWymJyjwtmpsW2iF/ZrnT+XTcpadM=; b=OteJk0uXZD3NtEY64CQNc+4S2TMOU1sNVBkh3ACg3BxaYHS2oXj5j12dhLXsj2W3fA VRqPpNngzeT88btqxcnVyJ/Evh+km0Dh4dMV+V0Ih3GSoLTCcFLDqtsG+Tt44f6/vCNI 9sfrpCAbgcUHFEUI3ZW+0I8UT7JD4Y1LqrhHH/GVEkyLYj1w/8oqQ69/9nt+hq1eA8zB 5vvYPHr09K9ASNyj/kmmNKnxWXM5V5FjarC026RoVEXTh+IOLdFFwWjItxK4qX5WTDdL xR9w7TlSt4OWHkx+6lNjYgDcbsU52j/xr5jBrrS2Tt6CjPIL8rfAdcFeeTwDe5ddimiQ qfMw== X-Gm-Message-State: ACrzQf0XrmWrHq7eB5tonkye8ptSgdCphORXWql32pQLJNL9g0MOT6n7 KFEkQbbZRJuhx/SpbTRluiezyQBzCIZPP46Y8c+ubPrn/WurwVf+koCMP06Po6vIjlg+83heX62 NS1xohulpgpBroKIuwAbjgHOTeU/4JlQ/cVIDLZiJcmfd2crhd2KDtk+jpQTi X-Received: by 2002:a05:622a:164e:b0:35b:a852:52ca with SMTP id y14-20020a05622a164e00b0035ba85252camr20719345qtj.2.1664912046785; Tue, 04 Oct 2022 12:34:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4arrLFNIId2qTFtBrd4SBa+FZyl8Drmu4Nnvp22VTiLuMn7OhMRWmkaEkQqUdfhfxWBgrhdQ== X-Received: by 2002:a05:622a:164e:b0:35b:a852:52ca with SMTP id y14-20020a05622a164e00b0035ba85252camr20719325qtj.2.1664912046503; Tue, 04 Oct 2022 12:34:06 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id z5-20020a05622a028500b00342fb07944fsm13299811qtw.82.2022.10.04.12.34.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Oct 2022 12:34:06 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Mike Rapoport , peterx@redhat.com, David Hildenbrand , Andrew Morton , Andrea Arcangeli , Nadav Amit , Axel Rasmussen , Mike Kravetz Subject: [PATCH v3 3/3] mm/selftest: uffd: Explain the write missing fault check Date: Tue, 4 Oct 2022 15:34:00 -0400 Message-Id: <20221004193400.110155-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221004193400.110155-1-peterx@redhat.com> References: <20221004193400.110155-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664912049; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ifodv0MqT+m7iheWymJyjwtmpsW2iF/ZrnT+XTcpadM=; b=2J4RLHSh9VLUoYC+iy84J5tylca22V8on5Qj7n6OJZXa2gbySNexXL29U+J47HYZT2HORa 149pm0TioBycpcWan4Dx7hGHJkRzAZktV/wctGfw04LZvtRRkOua4bUWZOROhYzoFQqz95 RrTNgMvls0N6mAcZUeonLcy/o2EvQSU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=d9bqvHzO; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf24.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664912049; a=rsa-sha256; cv=none; b=hhYg9OZwXgTCy0Jq4jnp47mEh5U4BtvEcKeWH7PTXIVJot9uvUT44wPcmJabPm9B1TzXiI PwkEMNcwiB1+1f8G3XihJhYsIm5qaKgAZaN5NWJ3rdR2e7ImCfXtXNpQN6yJ7CRWy6xUep u5hq1v/vnmGZCUf1eo55HGmKGFGF7Lo= X-Stat-Signature: 9ek3gjx61jn9ngiyuy83kjqfh5z6o176 X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 23E3E18001E Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=d9bqvHzO; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf24.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com X-HE-Tag: 1664912048-856312 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It's not obvious why we had a write check for each of the missing messages, especially when it should be a locking op. Add a rich comment for that, and also try to explain its good side and limitations, so that if someone hit it again for either a bug or a different glibc impl there'll be some clue to start with. Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- tools/testing/selftests/vm/userfaultfd.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c index 74babdbc02e5..297f250c1d95 100644 --- a/tools/testing/selftests/vm/userfaultfd.c +++ b/tools/testing/selftests/vm/userfaultfd.c @@ -774,7 +774,27 @@ static void uffd_handle_page_fault(struct uffd_msg *msg, continue_range(uffd, msg->arg.pagefault.address, page_size); stats->minor_faults++; } else { - /* Missing page faults */ + /* + * Missing page faults. + * + * Here we force a write check for each of the missing mode + * faults. It's guaranteed because the only threads that + * will trigger uffd faults are the locking threads, and + * their first instruction to touch the missing page will + * always be pthread_mutex_lock(). + * + * Note that here we relied on an NPTL glibc impl detail to + * always read the lock type at the entry of the lock op + * (pthread_mutex_t.__data.__type, offset 0x10) before + * doing any locking operations to guarantee that. It's + * actually not good to rely on this impl detail because + * logically a pthread-compatible lib can implement the + * locks without types and we can fail when linking with + * them. However since we used to find bugs with this + * strict check we still keep it around. Hopefully this + * could be a good hint when it fails again. If one day + * it'll break on some other impl of glibc we'll revisit. + */ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) err("unexpected write fault");