From patchwork Mon Oct 3 15:56:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12997650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28A07C4332F for ; Mon, 3 Oct 2022 15:56:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78EA96B0071; Mon, 3 Oct 2022 11:56:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 716E48E0003; Mon, 3 Oct 2022 11:56:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56A6B8E0002; Mon, 3 Oct 2022 11:56:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3004D6B0071 for ; Mon, 3 Oct 2022 11:56:37 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E3965140803 for ; Mon, 3 Oct 2022 15:56:36 +0000 (UTC) X-FDA: 79980090792.24.B4F68E8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 659B38001D for ; Mon, 3 Oct 2022 15:56:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664812595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EmARIE0aFvhZ3doTDEW0tn2T6UXUcOfWmNkW2tjznSw=; b=bHUvrTaCsrCImN1jcOX/ElPYpDtMuerBYvo9iNqRbT68Oxp1SBRqW7SMNy9KlwVqomeJJ0 SfUfd/OXoUfAk8K2imxgcSnGXPuI/ogfXc+8S2jJvLwZrjUYEMJ8+wMtH06BD66C/DPV8G 2Bdwe1mneRzpVgT9l6tRL9Z4wMUDH4E= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-365-TpFn50XwP1O35i8w_sTsjg-1; Mon, 03 Oct 2022 11:56:34 -0400 X-MC-Unique: TpFn50XwP1O35i8w_sTsjg-1 Received: by mail-qv1-f69.google.com with SMTP id ks15-20020a056214310f00b004b18ea4bc05so1621466qvb.22 for ; Mon, 03 Oct 2022 08:56:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=EmARIE0aFvhZ3doTDEW0tn2T6UXUcOfWmNkW2tjznSw=; b=JUSUUFmIHvVoMiSMxFUdOjNDXJxq2wMSKaKTSw4DX/EijWwCz4cJbGKG68WzcMce+J e1XYJUzAOPNXqIXWuNymwyuXP0Zy5vnXAOcYc9+75Xn/p8V7gzZvYFB3+IF8eK0/APpb B4P8p4vxOhxDtKfpGE0fT0DgcKEwb/g7AbY1KiPKMKeHXWp+nkrnwc/YYPftaqnxiqY3 CVQk3oBMaHJtfJV8oAHVXduVvuHLk4I9B4zDotmlyqMUQtQapBVIVJUcY1MxJv7c13xw WFP37JFLzgeoYn08tEovsU7R+2H9AM4+PBLBVqiX4FMjuN7br8TbkOtLamwWbNn64Cd2 0K2g== X-Gm-Message-State: ACrzQf1xsOeCSx4mpKq3gyydzW/tcUAS2GUVbie8e1cJejsA67z1oiMl SLaUP6epj5rjxSmfWFJeZQuGpUMbjHqgQAVbhIm0+N2z5ssUdXJtKXOmkal6ec6yB/3XqeHs9LE yG9h5VILgU9E= X-Received: by 2002:a05:6214:2386:b0:4af:9757:81bd with SMTP id fw6-20020a056214238600b004af975781bdmr16584734qvb.124.1664812594148; Mon, 03 Oct 2022 08:56:34 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7i9AD0amdQk3NjDYMMsoIIttZ30lWkBTlVS+OXE/Wk+5ArMOHq16CUgv3zbMNLCWQAhaWK0Q== X-Received: by 2002:a05:6214:2386:b0:4af:9757:81bd with SMTP id fw6-20020a056214238600b004af975781bdmr16584720qvb.124.1664812593972; Mon, 03 Oct 2022 08:56:33 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id bs18-20020a05620a471200b006cf38fd659asm10956732qkb.103.2022.10.03.08.56.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Oct 2022 08:56:33 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: peterx@redhat.com, Andrea Arcangeli , Mike Rapoport , David Hildenbrand , Andrew Morton , Axel Rasmussen , Mike Kravetz , Nadav Amit Subject: [PATCH 1/3] mm/hugetlb: Fix race condition of uffd missing/minor handling Date: Mon, 3 Oct 2022 11:56:28 -0400 Message-Id: <20221003155630.469263-2-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221003155630.469263-1-peterx@redhat.com> References: <20221003155630.469263-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664812596; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EmARIE0aFvhZ3doTDEW0tn2T6UXUcOfWmNkW2tjznSw=; b=BlTRUA/BJ9RIqrQiqLymmQAHltOQhl6usquoh1R4Ue2/jkBeIIp3hDU+uJb80aZkjTUKL7 p3B4FMRUuiappQboTxK34wFO5hk3FMYUUHdpiAtOf6638gGhW5YnUeLpfjO7oLCBe7HOZl dhuHJnALdkesfxgOWSWOA/ZgM+ExrTs= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bHUvrTaC; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664812596; a=rsa-sha256; cv=none; b=n6i1Zok0oEb9ldzduPzHqaeQ2v6gVUmmmgzwPg7u5iw2fN0w9h40lEFv3lkh+uzZ5jGiNJ T+tjBxshK8TWIrpIG5Xmj9GELXoXmNXGN2doFztYFL4A+BnbDqR9ja9QWiaik+Nu6SsjKg ydX47PPU0xi4SKKxGDKyPR3H0xp82HQ= X-Stat-Signature: qfa43dbkuby1io19i95czdo9bywacrqq X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 659B38001D Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=bHUvrTaC; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com X-HE-Tag: 1664812596-367012 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After the recent rework patchset of hugetlb locking on pmd sharing, kselftest for userfaultfd sometimes fails on hugetlb private tests with unexpected write fault checks. It turns out there's nothing wrong within the locking series regarding this matter, but it could have changed the timing of threads so it can trigger an old bug. The real bug is when we call hugetlb_no_page() we're not with the pgtable lock. It means we're reading the pte values lockless. It's perfectly fine in most cases because before we do normal page allocations we'll take the lock and check pte_same() again. However before that, there are actually two paths on userfaultfd missing/minor handling that may directly move on with the fault process without checking the pte values. It means for these two paths we may be generating an uffd message based on an unstable pte, while an unstable pte can legally be anything as long as the modifier holds the pgtable lock. One example, which is also what happened in the failing kselftest and caused the test failure, is that for private mappings CoW can happen on one page. CoW requires pte being cleared before being replaced with a new page for TLB coherency, but then there can be a race condition: thread 1 thread 2 -------- -------- hugetlb_fault hugetlb_fault private pte RO hugetlb_wp pgtable_lock() huge_ptep_clear_flush pte=NULL hugetlb_no_page generate uffd missing event even if page existed!! set_huge_pte_at pgtable_unlock() Fix this by recheck the pte after pgtable lock for both userfaultfd missing & minor fault paths. This bug should have been around starting from uffd hugetlb introduced, so attaching a Fixes to the commit. Also attach another Fixes to the minor support commit for easier tracking. Note that userfaultfd is actually fine with false positives (e.g. caused by pte changed), but not wrong logical events (e.g. caused by reading a pte during changing). The latter can confuse the userspace, so the strictness is very much preferred. E.g., MISSING event should never happen on the page after UFFDIO_COPY has correctly installed the page and returned. Cc: Andrea Arcangeli Cc: Mike Kravetz Cc: Axel Rasmussen Cc: Nadav Amit Fixes: 1a1aad8a9b7b ("userfaultfd: hugetlbfs: add userfaultfd hugetlb hook") Fixes: 7677f7fd8be7 ("userfaultfd: add minor fault registration mode") Co-developed-by: Mike Kravetz Signed-off-by: Peter Xu --- mm/hugetlb.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 49 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 9679fe519b90..fa3fcdb0c4b8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5521,6 +5521,23 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_area_struct *vma, return ret; } +/* + * Recheck pte with pgtable lock. Returns true if pte didn't change, or + * false if pte changed or is changing. + */ +static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, + pte_t *ptep, pte_t old_pte) +{ + spinlock_t *ptl; + bool same; + + ptl = huge_pte_lock(h, mm, ptep); + same = pte_same(huge_ptep_get(ptep), old_pte); + spin_unlock(ptl); + + return same; +} + static vm_fault_t hugetlb_no_page(struct mm_struct *mm, struct vm_area_struct *vma, struct address_space *mapping, pgoff_t idx, @@ -5562,9 +5579,30 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, goto out; /* Check for page in userfault range */ if (userfaultfd_missing(vma)) { - ret = hugetlb_handle_userfault(vma, mapping, idx, - flags, haddr, address, - VM_UFFD_MISSING); + /* + * Since hugetlb_no_page() was examining pte + * without pgtable lock, we need to re-test under + * lock because the pte may not be stable and could + * have changed from under us. Try to detect + * either changed or during-changing ptes and retry + * properly when needed. + * + * Note that userfaultfd is actually fine with + * false positives (e.g. caused by pte changed), + * but not wrong logical events (e.g. caused by + * reading a pte during changing). The latter can + * confuse the userspace, so the strictness is very + * much preferred. E.g., MISSING event should + * never happen on the page after UFFDIO_COPY has + * correctly installed the page and returned. + */ + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) + ret = hugetlb_handle_userfault( + vma, mapping, idx, flags, haddr, + address, VM_UFFD_MISSING); + else + /* Retry the fault */ + ret = 0; goto out; } @@ -5634,9 +5672,14 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, if (userfaultfd_minor(vma)) { unlock_page(page); put_page(page); - ret = hugetlb_handle_userfault(vma, mapping, idx, - flags, haddr, address, - VM_UFFD_MINOR); + /* See comment in userfaultfd_missing() block above */ + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) + ret = hugetlb_handle_userfault( + vma, mapping, idx, flags, haddr, + address, VM_UFFD_MINOR); + else + /* Retry the fault */ + ret = 0; goto out; } } From patchwork Mon Oct 3 15:56:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12997651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0391BC433F5 for ; Mon, 3 Oct 2022 15:56:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CDDD8E0002; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 654596B0074; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40C828E0002; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 26B106B0073 for ; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E1652C0BF9 for ; Mon, 3 Oct 2022 15:56:38 +0000 (UTC) X-FDA: 79980090876.29.B1BAE15 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 1CCEBA001D for ; Mon, 3 Oct 2022 15:56:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664812597; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x4xYjXYzlE422UA6NYhxH1mMMh8xu1ZsHfGS6BWev5s=; b=ftxIuGaEAu6OG+TQ+sN37uhdOsQ7E0sp4z9WF5f/6S6qnVRVsgFVmKkSsvUnYbSMELrh6a XFYoYlokhHyFOpjQuUWTHt4OIxRKazGx9+oJpts5Vsh4ePYYLXPd1HZlV/lM8AXGhReEyc qPPHZkpG/rMb2RoilbKSgQlQNU6uQCg= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-604-YtAU7hG-OL2bJcLZ_5SfKw-1; Mon, 03 Oct 2022 11:56:35 -0400 X-MC-Unique: YtAU7hG-OL2bJcLZ_5SfKw-1 Received: by mail-qt1-f200.google.com with SMTP id e22-20020ac84b56000000b0035bb64ad562so7549674qts.17 for ; Mon, 03 Oct 2022 08:56:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=x4xYjXYzlE422UA6NYhxH1mMMh8xu1ZsHfGS6BWev5s=; b=EdI8KeRTwJDPf6Urz8j4dtCXhne3RqhE9zVYk45cxbdP0c+wk5uPZsPvZN838s/xRy gqpeYTisCiBLh+X//D10hiQ2jaWF2Npp2LyQei5WxwXxTR1r1sy+dxUdHHFsv/Nz3deC CbXF4FDGbG/bR5EE92zqtqCaR3OI593d4lK9+2Ga1BJC5mXNLK0+KD6CSHsQdoIQVjtf AyMv9AozTOYrnIYOOKksi0Enq8IUdMBtTJx3af4VXr9IjDvtwKgtQNw4lX7BoQ5yHNGK nL4tyrpXMjbQo9qVN+xBJ8EZ1aHc48aClCyalpJmh37+6zbR+dZgbzsQf/evzuK2H2M2 ld3A== X-Gm-Message-State: ACrzQf14NbY9J10TVUTF9/pAvx80Dua8Rzu+J+IA7vt/OKveJjhjOmtU wtt/bI/5F83ETlWfjRT99ZTXzIxtOiL6uYzHQbzyMOBFPBaWZFkhRqiaactzyKA0UlGSj5t/6CW QSjF3zFL7wuo= X-Received: by 2002:a05:622a:11c8:b0:35c:e912:a8ea with SMTP id n8-20020a05622a11c800b0035ce912a8eamr16371754qtk.17.1664812595448; Mon, 03 Oct 2022 08:56:35 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7jmUpIO0vv9JXx3e6EMgXhYeRsvGCHvJYDkuTWLTvnmvX6dtNKe0ySPp+Up7ucpY2Yh41crw== X-Received: by 2002:a05:622a:11c8:b0:35c:e912:a8ea with SMTP id n8-20020a05622a11c800b0035ce912a8eamr16371738qtk.17.1664812595208; Mon, 03 Oct 2022 08:56:35 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id bs18-20020a05620a471200b006cf38fd659asm10956732qkb.103.2022.10.03.08.56.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Oct 2022 08:56:34 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: peterx@redhat.com, Andrea Arcangeli , Mike Rapoport , David Hildenbrand , Andrew Morton , Axel Rasmussen , Mike Kravetz , Nadav Amit Subject: [PATCH 2/3] mm/hugetlb: Use hugetlb_pte_stable in migration race check Date: Mon, 3 Oct 2022 11:56:29 -0400 Message-Id: <20221003155630.469263-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221003155630.469263-1-peterx@redhat.com> References: <20221003155630.469263-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664812598; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x4xYjXYzlE422UA6NYhxH1mMMh8xu1ZsHfGS6BWev5s=; b=bFCxhuoX+ahBymkPc9WNtXxiNDccF/jXiQfcwi4GMoYt9c6alcBKodi/9JEeAgyEKjtXX+ UpzEpTPZ/a9WXHtoaawjdsCPlwKjyct0qdi/iLevckkILPJgvvkPIZ/Ceh/l5f7HHaTYDu DDXq9UWpXR/g089ss3B9VuuEW6D9AVU= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ftxIuGaE; spf=pass (imf25.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664812598; a=rsa-sha256; cv=none; b=QSU+AsjLiYcuVMGED475515OvSnxIyTxVL2jApyBhRFVEDWNSwEEbV1pZ/xvik8WRLlaMg vmMJlEWBLaPkw4wf7e2yNgIOLqSm9vcLtpJcD/erywhGsjBZmUPGIZrTPbDjXv6qykMa5S Xk8QeYmefZelzF1kguvrcxc+W0xLIjI= Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ftxIuGaE; spf=pass (imf25.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 1CCEBA001D X-Rspam-User: X-Stat-Signature: dt1tmoayb9hsjfamd7kezdte6qwhz3pn X-HE-Tag: 1664812597-541439 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: After hugetlb_pte_stable() introduced, we can also rewrite the migration race condition against page allocation to use the new helper too. Signed-off-by: Peter Xu --- mm/hugetlb.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fa3fcdb0c4b8..e762c5369a6f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5620,11 +5620,10 @@ static vm_fault_t hugetlb_no_page(struct mm_struct *mm, * here. Before returning error, get ptl and make * sure there really is no pte entry. */ - ptl = huge_pte_lock(h, mm, ptep); - ret = 0; - if (huge_pte_none(huge_ptep_get(ptep))) + if (hugetlb_pte_stable(h, mm, ptep, old_pte)) ret = vmf_error(PTR_ERR(page)); - spin_unlock(ptl); + else + ret = 0; goto out; } clear_huge_page(page, address, pages_per_huge_page(h)); From patchwork Mon Oct 3 15:56:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12997652 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E817C433FE for ; Mon, 3 Oct 2022 15:56:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D81EE6B0073; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE27A8E0003; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0E6F6B0075; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9D3EF6B0073 for ; Mon, 3 Oct 2022 11:56:39 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 48D491C5B41 for ; Mon, 3 Oct 2022 15:56:39 +0000 (UTC) X-FDA: 79980090918.26.AF0DBD6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id D334440017 for ; Mon, 3 Oct 2022 15:56:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664812598; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/5RzxXUZxi/MrG066VL+DnGEo1qQl33988Rm0NK9+zA=; b=MLutX1nsPeGms5FwSsl0JmsXVFj0AoP8lzGgDk/TtB5zS/YfxRC1r47fNGYAsoBKKktAKg N2r6Sbb4l0/lDAqpA7gMlRUyNj/dNGwKazc/EStk+BLHiCfbHhaVLtvZ2i8lBV75uWy0uO eW0l5/Ksm1N7HFlAdIX8C5Jyk4dN7sQ= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-501-ZMSBlpLDPmS4K6MZl3LTsA-1; Mon, 03 Oct 2022 11:56:37 -0400 X-MC-Unique: ZMSBlpLDPmS4K6MZl3LTsA-1 Received: by mail-qk1-f199.google.com with SMTP id k2-20020a05620a414200b006ceec443c8bso9453322qko.14 for ; Mon, 03 Oct 2022 08:56:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=/5RzxXUZxi/MrG066VL+DnGEo1qQl33988Rm0NK9+zA=; b=y69HTDt0QPz9PbQXcIfo4C2T57HX0cdpT9jBolBMK3WDsk8/d+6yTwX+I6Mt8+yoaK funa2IkczaO5TM6FL/oLQlEqu676QbrwIS0MXrkee7ZErQNx9mljRGMufdIyy5a9Njsb vJEgpIEiKuqbrA+ZlJXxTBB4PcUnWGWePKD2fY+Fvz5pGINp9IMCC+VQSwV3fk0tn+v0 Kd31gy5rpPlPaz1n5i2DaueflIozvmCq2+fWsjaW6TPQuTkEOz6Ir/nt7HYkk3idWlG5 xovXWQR0jcOfMPIXTTHqtoBNqPltJ5wu3TJF3s8LcbjgOp1sHlXboBFw/zBRNFRqpM/n kWiA== X-Gm-Message-State: ACrzQf1ngVmWi5tohQIs8/oL7uhwPPovM4QBauKk/5A/D4bpEOELXHfF M2oNzyKPXgFIQW5XhURmCIWzbC5dT9ROEQMr5JP0KAXDZstkZ3GLbjbB9gamp4ge/22yQeyO7xk QA1DpWv97D6Q= X-Received: by 2002:a05:620a:15d2:b0:6cf:2d38:9c0d with SMTP id o18-20020a05620a15d200b006cf2d389c0dmr14084013qkm.426.1664812596882; Mon, 03 Oct 2022 08:56:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5YdLVN5mgUHqMX9gMAFXtuu3DO1SuWxQNyQcY/m8UTnt2zEXiE/KDGL7J7kdUZbNlU/tPU6w== X-Received: by 2002:a05:620a:15d2:b0:6cf:2d38:9c0d with SMTP id o18-20020a05620a15d200b006cf2d389c0dmr14083998qkm.426.1664812596639; Mon, 03 Oct 2022 08:56:36 -0700 (PDT) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id bs18-20020a05620a471200b006cf38fd659asm10956732qkb.103.2022.10.03.08.56.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Oct 2022 08:56:36 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: peterx@redhat.com, Andrea Arcangeli , Mike Rapoport , David Hildenbrand , Andrew Morton , Axel Rasmussen , Mike Kravetz , Nadav Amit Subject: [PATCH 3/3] mm/selftest: uffd: Explain the write missing fault check Date: Mon, 3 Oct 2022 11:56:30 -0400 Message-Id: <20221003155630.469263-4-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221003155630.469263-1-peterx@redhat.com> References: <20221003155630.469263-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664812598; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/5RzxXUZxi/MrG066VL+DnGEo1qQl33988Rm0NK9+zA=; b=i30/7hIJAxGluVjMbZUy18b9w77Q8Iq3oaJeBGdsbX6dx9oWNf1F4BmO7uIwnws1KHHxOR 5bdykaP/CExAfSmh5PUw1GFuANCFIPq3WzQO7HH/4hjE9v92X17rRbwuMNuaF38CHLN1IS Z4NfWuLuLaUEZZ/uQHE/oSYGSaUitzc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MLutX1ns; spf=pass (imf07.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664812598; a=rsa-sha256; cv=none; b=G6hl98XqfSdCH0PxXHVU60EYFA2s6vTin7AOK74JzIdePOVa4LfiLWDrmbfHeBjBTfQBKn lfWnbH/JIFnhxjrHLO3xGhEgpn5+iCKgoJXo2JBH4JlLfrv0OjkoX2HfibBwlKQPRxnjnc A8LTe7ABDtmh4ny2GPGZMO/fcRj/1eU= X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MLutX1ns; spf=pass (imf07.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D334440017 X-Stat-Signature: hjbd4ic8n8m39wi6w336fpgxpsxxt3py X-HE-Tag: 1664812598-458433 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It's not obvious why we had a write check for each of the missing messages, especially when it should be a locking op. Add a rich comment for that, and also try to explain its good side and limitations, so that if someone hit it again for either a bug or a different glibc impl there'll be some clue to start with. Signed-off-by: Peter Xu --- tools/testing/selftests/vm/userfaultfd.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vm/userfaultfd.c b/tools/testing/selftests/vm/userfaultfd.c index 74babdbc02e5..297f250c1d95 100644 --- a/tools/testing/selftests/vm/userfaultfd.c +++ b/tools/testing/selftests/vm/userfaultfd.c @@ -774,7 +774,27 @@ static void uffd_handle_page_fault(struct uffd_msg *msg, continue_range(uffd, msg->arg.pagefault.address, page_size); stats->minor_faults++; } else { - /* Missing page faults */ + /* + * Missing page faults. + * + * Here we force a write check for each of the missing mode + * faults. It's guaranteed because the only threads that + * will trigger uffd faults are the locking threads, and + * their first instruction to touch the missing page will + * always be pthread_mutex_lock(). + * + * Note that here we relied on an NPTL glibc impl detail to + * always read the lock type at the entry of the lock op + * (pthread_mutex_t.__data.__type, offset 0x10) before + * doing any locking operations to guarantee that. It's + * actually not good to rely on this impl detail because + * logically a pthread-compatible lib can implement the + * locks without types and we can fail when linking with + * them. However since we used to find bugs with this + * strict check we still keep it around. Hopefully this + * could be a good hint when it fails again. If one day + * it'll break on some other impl of glibc we'll revisit. + */ if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) err("unexpected write fault");