From patchwork Tue Nov 22 09:51:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13052114 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E42C2C433FE for ; Tue, 22 Nov 2022 09:51:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 796ED6B0071; Tue, 22 Nov 2022 04:51:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 746776B0073; Tue, 22 Nov 2022 04:51:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60F228E0001; Tue, 22 Nov 2022 04:51:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 527306B0071 for ; Tue, 22 Nov 2022 04:51:56 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 25D141C37A0 for ; Tue, 22 Nov 2022 09:51:56 +0000 (UTC) X-FDA: 80160611832.16.FC9B779 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by imf24.hostedemail.com (Postfix) with ESMTP id DCFD8180012 for ; Tue, 22 Nov 2022 09:51:55 +0000 (UTC) Received: by mail-qt1-f180.google.com with SMTP id s4so8921757qtx.6 for ; Tue, 22 Nov 2022 01:51:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=d3Ih39NMegZAhP2Qu/k/4zxkPIuPc7SBENROtpvFlxw=; b=DSEt13C4pf2sdQMt8lU0G2oqdagEd8Wg/0MSUb37Kr7CrOmTWHPLhkVpmbZciGuNS9 C8t/tI1CS+zRtzxXE5QADcgZLSJCWgPDlj8nmToiDwBydTdUx1eGXnzTKx6d4qr5lTrE DuYsCDI7UEpY3hZJVcgesjAvUuLBj83LldT8WJXRuWNH1z5a4+JfDyC2Uok9yQhaQxqn WTODQQSG5ouJQ13FeuU9VI7mf+8wihbLKoDgPFO4RdB0noECutAd9ilXZ7o85DkLcqsm FJ13Eaqs7CwdkfpBYOyVsc04MGLxm70lB2bNnbYxQOtH2Waeb8NcJ9OdQ84F03J/3/GY 72Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d3Ih39NMegZAhP2Qu/k/4zxkPIuPc7SBENROtpvFlxw=; b=otFbKv9PdJ1Krh0aIiNfu6lekhnwRrn6EkbjjaqPdhZhyMLAddM4uWj0bz0GdVLxdJ yiQIWpQfWGpvzBIGzXcC02t5pOhJlNJuG8Arh0zbfgEE2uhWoBzHX2DXVYgM3lw7aXon XCUF6987ikEnwdTNmciNR11ifliPU5aS+98XoPwerPFLx7u7+GfcLg43APMCM4Ei5b7p tfxnHintlWdnqtLI+ejUhAatyekEYvb4sOSp24q4fij++2xKo+Cp76ZaujEhh8uTAK30 6tz5WDLZOgwVny4KmIGgVtICljzSwtqBXbDM6yobDCWYE3AXGXOUPuepNxjAKBVyv9hT mr2g== X-Gm-Message-State: ANoB5pkokM8ilOD/uRmBDGvIh4CJ5CxrXHmbTvxnb8dUpy+6/k1oLRa+ RKB+ZzOGta4KixsldH7Op7jwIQ== X-Google-Smtp-Source: AA0mqf4JAFxf1BA3eOCanBBGSqdFaxKzFrzfW6x8iVMu2sZfmFx/p6NkCnNFdJDaTwbFQrMWZ+/NGw== X-Received: by 2002:a05:622a:18a7:b0:3a5:62b5:9093 with SMTP id v39-20020a05622a18a700b003a562b59093mr10794835qtc.252.1669110715048; Tue, 22 Nov 2022 01:51:55 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id ci14-20020a05622a260e00b00398df095cf5sm1349951qtb.34.2022.11.22.01.51.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 01:51:53 -0800 (PST) Date: Tue, 22 Nov 2022 01:51:50 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Linus Torvalds , Johannes Weiner , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Vlastimil Babka , Peter Xu , Yang Shi , John Hubbard , Mike Kravetz , Sidhartha Kumar , Muchun Song , Miaohe Lin , Naoya Horiguchi , Mina Almasry , James Houghton , Zach O'Keefe , Yu Zhao , Dan Carpenter , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 3/3] mm,thp,rmap: clean up the end of __split_huge_pmd_locked() In-Reply-To: Message-ID: References: <5f52de70-975-e94f-f141-543765736181@google.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669110715; a=rsa-sha256; cv=none; b=O1YnDVbkWTHJjHFQihPGUG6TzvWYlvNC2Pl/fzGJJBWsmq+7p1cU2Jm1hPRvvoWfCH1vy8 ZdD15Pwq6RI9hqsxzaKDs58h8tj7O316O3h21GGY2Zev7U3AXHkIZ+0W41QqxoPw8yqCFV 61Noc3wC+cQM/16kUnYbg8J2cQDIbkY= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=DSEt13C4; spf=pass (imf24.hostedemail.com: domain of hughd@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669110715; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d3Ih39NMegZAhP2Qu/k/4zxkPIuPc7SBENROtpvFlxw=; b=cch+JsQ/chOPbyPmlqLAAH46YSiKH2l6g9V6yMAR2VrwaVSKDEHUJkRRNIrxRr+YYCpk7k oolHvu97QXXP34EUklCT3WpipE33YI+92/kAvCrfpAr22YafPMmGa5W0rosyUOGKzME5By yAn5J74S8N2pVFJ2uXcZ38Fw9RXE0g0= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DCFD8180012 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=DSEt13C4; spf=pass (imf24.hostedemail.com: domain of hughd@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Stat-Signature: ibuyrswgmbmyw1r475af44nqgjzcb7kw X-HE-Tag: 1669110715-192160 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It's hard to add a page_add_anon_rmap() into __split_huge_pmd_locked()'s HPAGE_PMD_NR set_pte_at() loop, without wincing at the "freeze" case's HPAGE_PMD_NR page_remove_rmap() loop below it. It's just a mistake to add rmaps in the "freeze" (insert migration entries prior to splitting huge page) case: the pmd_migration case already avoids doing that, so just follow its lead. page_add_ref() versus put_page() likewise. But why is one more put_page() needed in the "freeze" case? Because it's removing the pmd rmap, already removed when pmd_migration (and freeze and pmd_migration are mutually exclusive cases). Signed-off-by: Hugh Dickins Acked-by: Kirill A. Shutemov --- v2: same as v1, plus Ack from Kirill mm/huge_memory.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 3dee8665c585..ab5ab1a013e1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2135,7 +2135,6 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, uffd_wp = pmd_uffd_wp(old_pmd); VM_BUG_ON_PAGE(!page_count(page), page); - page_ref_add(page, HPAGE_PMD_NR - 1); /* * Without "freeze", we'll simply split the PMD, propagating the @@ -2155,6 +2154,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, anon_exclusive = PageAnon(page) && PageAnonExclusive(page); if (freeze && anon_exclusive && page_try_share_anon_rmap(page)) freeze = false; + if (!freeze) + page_ref_add(page, HPAGE_PMD_NR - 1); } /* @@ -2210,27 +2211,21 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, entry = pte_mksoft_dirty(entry); if (uffd_wp) entry = pte_mkuffd_wp(entry); + page_add_anon_rmap(page + i, vma, addr, false); } pte = pte_offset_map(&_pmd, addr); BUG_ON(!pte_none(*pte)); set_pte_at(mm, addr, pte, entry); - if (!pmd_migration) - page_add_anon_rmap(page + i, vma, addr, false); pte_unmap(pte); } if (!pmd_migration) page_remove_rmap(page, vma, true); + if (freeze) + put_page(page); smp_wmb(); /* make pte visible before pmd */ pmd_populate(mm, pmd, pgtable); - - if (freeze) { - for (i = 0; i < HPAGE_PMD_NR; i++) { - page_remove_rmap(page + i, vma, false); - put_page(page + i); - } - } } void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,