From patchwork Wed Jun 9 04:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 12308825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C63ECC47095 for ; Wed, 9 Jun 2021 04:05:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 59D4E6124C for ; Wed, 9 Jun 2021 04:05:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59D4E6124C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C33DA6B006E; Wed, 9 Jun 2021 00:05:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE3FD6B0070; Wed, 9 Jun 2021 00:05:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A84BA6B0071; Wed, 9 Jun 2021 00:05:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id 75D166B006E for ; Wed, 9 Jun 2021 00:05:25 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 11542ABE8 for ; Wed, 9 Jun 2021 04:05:25 +0000 (UTC) X-FDA: 78232845810.19.AA39F93 Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) by imf10.hostedemail.com (Postfix) with ESMTP id 6F7724202A07 for ; Wed, 9 Jun 2021 04:05:21 +0000 (UTC) Received: by mail-oi1-f182.google.com with SMTP id c13so18297820oib.13 for ; Tue, 08 Jun 2021 21:05:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:subject:message-id:mime-version; bh=GvD7glKDmqy0C8wgTkrBDLR7pK27K361Vqkf6hOXhYA=; b=SAi4kOfFlJAobWQC5FfaMehHga/M3Wa0Hmdh+mQcnYGc/JRs6pihYdMMtSS5M/Bk98 3ALaK8Gue0Us9N3LxFDvt0a9llHAQ9YTg5EhaDN8CsijPeD1+ZGyUm1qvQJu+ix7phVA 43TJUe9702cUG2c0X6jV4mLwZFPhNX8aUnOE1Y2ME19AQT8qXf3qtZ4W34dBW2K4b/KX 13gbj7KSeaxJsLFzRDWNwPFNAjnh1AUTTZJdHQiWoVFHAXvlusvOQvqy9w3B9peDqN6w cLP+BSiKw0K0Szi5lXIWHaekLUwG9hI1jI0eMG6Hcc06ZiGEO/GjWNnik7/M3Uxvh2Bs tBng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version; bh=GvD7glKDmqy0C8wgTkrBDLR7pK27K361Vqkf6hOXhYA=; b=BmL49OjeBJFBoEbIcIkRGk1Qb/uNdWaqGgdWlrKvNQ4m4WnYddPae/F0AOw6lPoTfd FBF9Ml3sOMZt5RFOuzP3zKKJvs0viWOR7tG2h0DmA70PPpdqjuymbye8FhW48LdCCsBw dwZDxAPW4D46JKo/6Y/e75irSKdmM8AZsf2X9hV0ZnIGhxWn+zGbNJ2iQdcvkFeuQLgo /oFUJ2hQF8ltYmiANAvnoDU1ebQlliaBynM6vwgVRnKyeuyeYZ7TiNDBj8Z3VSCQ6oIE I3eLVjNkWBA4ftze5rWaOX40ZfoZQvIZsfwHKsGptGp2uJsMPRY48F1vOe0I2JBTJnE4 DCjw== X-Gm-Message-State: AOAM5309U9B9EkIpxqhYh6BiZQzd0eqvQ44EfpH7D05UXVDTzCHMEUy6 2PNyDyrWW0I254vYcxKDYxptWXUNUVX04g== X-Google-Smtp-Source: ABdhPJz2ZyojP1wY65WXX6sa5VGixFsfCgnnjUYEZuWwujSze0Qk1mzybTYSjorkuWNbORFSP8JjDQ== X-Received: by 2002:aca:af90:: with SMTP id y138mr5150511oie.92.1623211523605; Tue, 08 Jun 2021 21:05:23 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id q22sm3416834otl.11.2021.06.08.21.05.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 21:05:23 -0700 (PDT) Date: Tue, 8 Jun 2021 21:05:21 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.anvils To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 01/10] mm/thp: fix __split_huge_pmd_locked() on shmem migration entry (fwd) Message-ID: <59d94b4-c0dd-310-894-be99416f3c92@google.com> MIME-Version: 1.0 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=SAi4kOfF; spf=pass (imf10.hostedemail.com: domain of hughd@google.com designates 209.85.167.182 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 6F7724202A07 X-Stat-Signature: rfyr5tehtrtmehe7w47khq7rc1utqxfg X-HE-Tag: 1623211521-637405 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ---------- Forwarded message ---------- Date: Tue, 8 Jun 2021 21:00:12 -0700 (PDT) From: Hugh Dickins To: Andrew Morton Cc: Hugh Dickins , Kirill A. Shutemov , Yang Shi , Wang Yugui , Matthew Wilcox , Naoya Horiguchi , Alistair Popple , Ralph Campbell , Zi Yan , Miaohe Lin , Minchan Kim , Jue Wang , Peter Xu , Jan Kara , Shakeel Butt , Oscar Salvador Subject: [PATCH v2 01/10] mm/thp: fix __split_huge_pmd_locked() on shmem migration entry Stressing huge tmpfs page migration racing hole punch often crashed on the VM_BUG_ON(!pmd_present) in pmdp_huge_clear_flush(), with DEBUG_VM=y kernel; or shortly afterwards, on a bad dereference in __split_huge_pmd_locked() when DEBUG_VM=n. They forgot to allow for pmd migration entries in the non-anonymous case. Full disclosure: those particular experiments were on a kernel with more relaxed mmap_lock and i_mmap_rwsem locking, and were not repeated on the vanilla kernel: it is conceivable that stricter locking happens to avoid those cases, or makes them less likely; but __split_huge_pmd_locked() already allowed for pmd migration entries when handling anonymous THPs, so this commit brings the shmem and file THP handling into line. And while there: use old_pmd rather than _pmd, as in the following blocks; and make it clearer to the eye that the !vma_is_anonymous() block is self-contained, making an early return after accounting for unmapping. Fixes: e71769ae5260 ("mm: enable thp migration for shmem thp") Signed-off-by: Hugh Dickins Cc: Acked-by: Kirill A. Shutemov Reviewed-by: Yang Shi --- v2: omit is_huge_zero_pmd() mods (done differently in next), per Kirill mm/huge_memory.c | 27 ++++++++++++++++++--------- mm/pgtable-generic.c | 5 ++--- 2 files changed, 20 insertions(+), 12 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 63ed6b25deaa..42cfefc6e66e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2044,7 +2044,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, count_vm_event(THP_SPLIT_PMD); if (!vma_is_anonymous(vma)) { - _pmd = pmdp_huge_clear_flush_notify(vma, haddr, pmd); + old_pmd = pmdp_huge_clear_flush_notify(vma, haddr, pmd); /* * We are going to unmap this huge page. So * just go ahead and zap it @@ -2053,16 +2053,25 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, zap_deposited_table(mm, pmd); if (vma_is_special_huge(vma)) return; - page = pmd_page(_pmd); - if (!PageDirty(page) && pmd_dirty(_pmd)) - set_page_dirty(page); - if (!PageReferenced(page) && pmd_young(_pmd)) - SetPageReferenced(page); - page_remove_rmap(page, true); - put_page(page); + if (unlikely(is_pmd_migration_entry(old_pmd))) { + swp_entry_t entry; + + entry = pmd_to_swp_entry(old_pmd); + page = migration_entry_to_page(entry); + } else { + page = pmd_page(old_pmd); + if (!PageDirty(page) && pmd_dirty(old_pmd)) + set_page_dirty(page); + if (!PageReferenced(page) && pmd_young(old_pmd)) + SetPageReferenced(page); + page_remove_rmap(page, true); + put_page(page); + } add_mm_counter(mm, mm_counter_file(page), -HPAGE_PMD_NR); return; - } else if (pmd_trans_huge(*pmd) && is_huge_zero_pmd(*pmd)) { + } + + if (pmd_trans_huge(*pmd) && is_huge_zero_pmd(*pmd)) { /* * FIXME: Do we want to invalidate secondary mmu by calling * mmu_notifier_invalidate_range() see comments below inside diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index c2210e1cdb51..4e640baf9794 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -135,9 +135,8 @@ pmd_t pmdp_huge_clear_flush(struct vm_area_struct *vma, unsigned long address, { pmd_t pmd; VM_BUG_ON(address & ~HPAGE_PMD_MASK); - VM_BUG_ON(!pmd_present(*pmdp)); - /* Below assumes pmd_present() is true */ - VM_BUG_ON(!pmd_trans_huge(*pmdp) && !pmd_devmap(*pmdp)); + VM_BUG_ON(pmd_present(*pmdp) && !pmd_trans_huge(*pmdp) && + !pmd_devmap(*pmdp)); pmd = pmdp_huge_get_and_clear(vma->vm_mm, address, pmdp); flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE); return pmd;