From patchwork Fri Nov 18 01:10:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13047535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF758C4332F for ; Fri, 18 Nov 2022 01:10:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A2858E0002; Thu, 17 Nov 2022 20:10:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 652D46B0073; Thu, 17 Nov 2022 20:10:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42FEF8E0002; Thu, 17 Nov 2022 20:10:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 232D36B0071 for ; Thu, 17 Nov 2022 20:10:35 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E6C9C1A0D98 for ; Fri, 18 Nov 2022 01:10:34 +0000 (UTC) X-FDA: 80144782788.09.09827E0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 9988AC000A for ; Fri, 18 Nov 2022 01:10:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668733834; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9OM954c43rVIe2ciAr/4Oap5Q/Q5UCXLA+pQIGS7ALA=; b=EIfjjx7NY+Fi1PVK+JkJnTH4LA5GnBP8uic1aMryn79BQidEjCZnhFLM13lb0xEimcvvoD MgjYKfs5A3MPtL/Skdwju5bWrNXX3eB28pgO2iBQdmF5OSYZtfE54ilZAGPsq4MEp0Fho+ r+/vS7nCRmd/qQfJN3eSnS2ZvuqxH9Y= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-207-63HhgjF_O0aLTzjxlYhqFw-1; Thu, 17 Nov 2022 20:10:33 -0500 X-MC-Unique: 63HhgjF_O0aLTzjxlYhqFw-1 Received: by mail-qk1-f199.google.com with SMTP id bj1-20020a05620a190100b006fa12a05188so4404640qkb.4 for ; Thu, 17 Nov 2022 17:10:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9OM954c43rVIe2ciAr/4Oap5Q/Q5UCXLA+pQIGS7ALA=; b=3+f2A2relQRGRzuJzvmn/DjW1A3XOitxDl3tsQJqdnSyqXetClFMK2K0/65VzrfhBV WsjWKwFSbxdmSHof5qQO1yykGj6JL+lLkEtfAuV5fGKd8SKlAu0DhENve/oyolNDd1Rq phRWwLWYrWYd3oUgq8kKI/VOAz0Q5q/vqpZCPAMxov0s/QrjeEZyNTxPXWuOLY1lFP8K Xse0QmJL1LU8bpeh/kWsoRvrePrL3grLuS2QCuQbYT/RiiM2wPQB3zlsR+b3plIw5UYn OjpuqzJqOxqzA6epsRVpzFyLi+peQoswzBUSGQvemK+4Pvv+OZ1eqhCqoJzZMQb9aHZp 8KGw== X-Gm-Message-State: ANoB5pnPaP25GfSYEN2NbzhU0CV/T8HwFWxNUFaBlYE93Q0a3Vfkg7KD XrlnjxUXIgt1Tx1jKBqBk47jyGMXCqTSUJUkDhyHNW+Yg+/bTs7abKL3emEs1pNU8quKbv7SALC 4nw2mX+pc7iJelI4oB6ZHekVv/XrgAFA2BME/z4uaAmyOPyem8a3C4efLxGcs X-Received: by 2002:a05:620a:260d:b0:6fb:a9af:2238 with SMTP id z13-20020a05620a260d00b006fba9af2238mr4174802qko.457.1668733831852; Thu, 17 Nov 2022 17:10:31 -0800 (PST) X-Google-Smtp-Source: AA0mqf5Aqw8l8m4fWogYc3FoWyNkgwxEBla8HpkXPW9zfRaGkpX2OEDEon4bGw7svTSEDlw3e0SH+w== X-Received: by 2002:a05:620a:260d:b0:6fb:a9af:2238 with SMTP id z13-20020a05620a260d00b006fba9af2238mr4174778qko.457.1668733831551; Thu, 17 Nov 2022 17:10:31 -0800 (PST) Received: from x1n.redhat.com (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id u7-20020a05620a430700b006eed75805a2sm1491342qko.126.2022.11.17.17.10.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Nov 2022 17:10:31 -0800 (PST) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Rik van Riel , Muchun Song , Andrew Morton , peterx@redhat.com, James Houghton , Nadav Amit , Andrea Arcangeli , David Hildenbrand , Miaohe Lin , Mike Kravetz Subject: [PATCH RFC v2 02/12] mm/hugetlb: Move swap entry handling into vma lock for fault Date: Thu, 17 Nov 2022 20:10:15 -0500 Message-Id: <20221118011025.2178986-3-peterx@redhat.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221118011025.2178986-1-peterx@redhat.com> References: <20221118011025.2178986-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-type: text/plain ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EIfjjx7N; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668733834; a=rsa-sha256; cv=none; b=NY5IP0/MsPn9wevEavsZyjFzDk1eFI1i/vgp1D1o/HUVC9gL67xnwLGB7L690qUg61Q/K4 gpYEppiOzd4UEYu8G6+FFRjg9nYbkNQQW9iqMwPsDVsqiZfEKgojkKYNB6o30wALmx055z G7v8xuj5SqKSYSDHmYVRfJYjpbY5Tig= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668733834; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9OM954c43rVIe2ciAr/4Oap5Q/Q5UCXLA+pQIGS7ALA=; b=PdO+81E/DStOBeNBXYnRvWvoeK+RF9ukZ9+F780I7MU2lFjBt86D9yubKnqdTp4gl8XRQT ZWz8bWsV2M3z2MBmzBTW6XzwHhSASxdPXe7rFcuv+qT+a/GM7vm5fl+aeB42Kf7kLpMuDx ZMzb8fb04DZMR9oWtu7l+Cqz9Qj4R90= Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EIfjjx7N; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam02 X-Rspam-User: X-Stat-Signature: wumojbaqdziokqrfxzjsm6jyb1xs6cpx X-Rspamd-Queue-Id: 9988AC000A X-HE-Tag: 1668733834-155478 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In hugetlb_fault(), there used to have a special path to handle swap entry at the entrance using huge_pte_offset(). That's unsafe because huge_pte_offset() for a pmd sharable range can access freed pgtables if without either the walker lock or vma lock. Here the simplest solution for making it safe is just to move the swap handling to be after the vma lock being held. We may need to take the fault mutex on either migration or hwpoison entries now (also the vma lock, but that's really needed), however neither of them is hot path so it should be fine. Signed-off-by: Peter Xu --- mm/hugetlb.c | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c3aab6d5b7aa..62ff3fc51d4e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5824,22 +5824,6 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, int need_wait_lock = 0; unsigned long haddr = address & huge_page_mask(h); - ptep = huge_pte_offset(mm, haddr, huge_page_size(h)); - if (ptep) { - /* - * Since we hold no locks, ptep could be stale. That is - * OK as we are only making decisions based on content and - * not actually modifying content here. - */ - entry = huge_ptep_get(ptep); - if (unlikely(is_hugetlb_entry_migration(entry))) { - migration_entry_wait_huge(vma, ptep); - return 0; - } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) - return VM_FAULT_HWPOISON_LARGE | - VM_FAULT_SET_HINDEX(hstate_index(h)); - } - /* * Serialize hugepage allocation and instantiation, so that we don't * get spurious allocation failures if two CPUs race to instantiate @@ -5886,8 +5870,14 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, * fault, and is_hugetlb_entry_(migration|hwpoisoned) check will * properly handle it. */ - if (!pte_present(entry)) + if (!pte_present(entry)) { + if (unlikely(is_hugetlb_entry_migration(entry))) + migration_entry_wait_huge(vma, ptep); + else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) + ret = VM_FAULT_HWPOISON_LARGE | + VM_FAULT_SET_HINDEX(hstate_index(h)); goto out_mutex; + } /* * If we are going to COW/unshare the mapping later, we examine the