From patchwork Mon Sep 28 17:54:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 11804459 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09291618 for ; Mon, 28 Sep 2020 17:55:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B13992184D for ; Mon, 28 Sep 2020 17:55:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sent.com header.i=@sent.com header.b="W2K3Us/8"; dkim=temperror (0-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="fxrT9c03" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B13992184D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=sent.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3F65F900006; Mon, 28 Sep 2020 13:55:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D739A900009; Mon, 28 Sep 2020 13:55:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0497900006; Mon, 28 Sep 2020 13:55:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id 76382900007 for ; Mon, 28 Sep 2020 13:55:25 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 31273181AE86D for ; Mon, 28 Sep 2020 17:55:25 +0000 (UTC) X-FDA: 77313222210.04.cake04_4d177fd27183 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 13921800AD3F for ; Mon, 28 Sep 2020 17:55:25 +0000 (UTC) X-Spam-Summary: 1,0,0,9e139273407ec0c7,d41d8cd98f00b204,zi.yan@sent.com,,RULES_HIT:41:355:379:541:800:960:966:968:973:988:989:1260:1261:1311:1314:1345:1359:1437:1515:1535:1542:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:3138:3139:3140:3141:3142:3353:3865:3867:3868:3871:4117:4250:4385:5007:6119:6120:6261:6653:6742:7576:7875:7901:7903:8957:9040:10004:11026:11232:11473:11658:11914:12043:12291:12296:12438:12555:12679:12683:12895:13894:14096:14110:14181:14721:21063:21080:21627:21990:30054:30064,0,RBL:64.147.123.17:@sent.com:.lbl8.mailshell.net-62.18.0.100 64.100.201.100;04yf4pcktwow1tfgnzgojxn6xb6mwocg6wcgg4kqcm8z65fii1kye1ny75xxjij.m6fmz7p9jdriywbpxoy8nka8ftycy86cymt3zpk4e6uuyzxcwto1cograqoua3p.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: cake04_4d177fd27183 X-Filterd-Recvd-Size: 6863 Received: from wnew3-smtp.messagingengine.com (wnew3-smtp.messagingengine.com [64.147.123.17]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Mon, 28 Sep 2020 17:55:24 +0000 (UTC) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.west.internal (Postfix) with ESMTP id 0E19EDAA; Mon, 28 Sep 2020 13:55:21 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Mon, 28 Sep 2020 13:55:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm1; bh=vrkxZJEQEBPuH CVbBAj+BQgl5osnBBmheesqLl7iLcM=; b=W2K3Us/8YRs4FcMLHPHb+2LU+kawc sMht8gZOYrUjngiiuCmcVI/au1+l3/EYEXw3ADll8FIpZr50bHD6fT5kBGhTN90l h6Tqs++h4vj9tTvdKPLg1TfXzfTSCNIH5/NXUbgUesqi7GsOK/20Ch/weCl8MgPk SEH2Pt47IGxrPxRVnrl0V1+Q3c8p0bQqHe3XBO5EW9yKJXCPodqurxUuDbVZ5kHz x08DV22ZFN29KYuOgQ8K+MpaI4R/aD03GWxayBDvQsD6KXUJ6xiaTyGi2B4zbw/b lakaxoyprrosbT1ytZfPWkquE9emOSg99oXwRUq3yUHoL5cTQ4ylCgM7w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=vrkxZJEQEBPuHCVbBAj+BQgl5osnBBmheesqLl7iLcM=; b=fxrT9c03 fdx0/UW/BNcBqstgRcX+vG7MroPZKqt75Q2/VImuzPwmVoM5LmQoF6bbmXCS46b/ Qs2ALEaNwoY8XH+VOQ29WGP81TEs4+5eIgX3y80lZ/CMwBP0PZOurSzLl/nB6ESL mf6dqX8UNIOKsCnKnVXc9jUGmQTYy1BRu7qinJJrEYy7g22ujZC9Og2PvR3/HVq1 r5BGm425X/7lD0LVnwBeD59D+Oe2q3ZvTkiT+EwIvieRDVSZOngAujlbg0ac+MG3 pniuCoN1PO1xOjn3ZYTZeaUhCHpFGhZ0aG8lk1/Z19sY5uuUVCh5trbXHF79RyZw wr3E057cwSrbXg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhhrggfgsedtkeertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeduhfffve ektdduhfdutdfgtdekkedvhfetuedufedtgffgvdevleehheevjefgtdenucfkphepuddv rdegiedruddtiedrudeigeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpeiiihdrhigrnhesshgvnhhtrdgtohhm X-ME-Proxy: Received: from nvrsysarch6.NVidia.COM (unknown [12.46.106.164]) by mail.messagingengine.com (Postfix) with ESMTPA id 1EC933064685; Mon, 28 Sep 2020 13:55:21 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org Cc: "Kirill A . Shutemov" , Roman Gushchin , Rik van Riel , Matthew Wilcox , Shakeel Butt , Yang Shi , Jason Gunthorpe , Mike Kravetz , Michal Hocko , David Hildenbrand , William Kucharski , Andrea Arcangeli , John Hubbard , David Nellans , linux-kernel@vger.kernel.org, Zi Yan Subject: [RFC PATCH v2 09/30] mm: thp: add PUD THP support to zap_huge_pud. Date: Mon, 28 Sep 2020 13:54:07 -0400 Message-Id: <20200928175428.4110504-10-zi.yan@sent.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200928175428.4110504-1-zi.yan@sent.com> References: <20200928175428.4110504-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan Preallocated 513 (1 PMD and 512 PTE) page table pages need to be freed when PUD THP is removed. zap_pud_deposited_table is added to perform the action. Signed-off-by: Zi Yan --- mm/huge_memory.c | 48 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 45 insertions(+), 3 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ea9fbedcda26..76069affebef 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2013,11 +2013,27 @@ spinlock_t *__pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma) } #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD +static inline void zap_pud_deposited_table(struct mm_struct *mm, pud_t *pud) +{ + pgtable_t pgtable; + int i; + + pgtable = pgtable_trans_huge_pud_withdraw(mm, pud); + pmd_free_page_with_ptes(mm, (pmd_t *)page_address(pgtable)); + + mm_dec_nr_pmds(mm); + for (i = 0; i < (1<<(HPAGE_PUD_ORDER - HPAGE_PMD_ORDER)); i++) + mm_dec_nr_ptes(mm); +} + int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, pud_t *pud, unsigned long addr) { + pud_t orig_pud; spinlock_t *ptl; + tlb_change_page_size(tlb, HPAGE_PUD_SIZE); + ptl = __pud_trans_huge_lock(pud, vma); if (!ptl) return 0; @@ -2027,14 +2043,40 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, * pgtable_trans_huge_withdraw after finishing pudp related * operations. */ - pudp_huge_get_and_clear_full(tlb->mm, addr, pud, tlb->fullmm); + orig_pud = pudp_huge_get_and_clear_full(tlb->mm, addr, pud, + tlb->fullmm); tlb_remove_pud_tlb_entry(tlb, pud, addr); if (vma_is_special_huge(vma)) { spin_unlock(ptl); /* No zero page support yet */ + } else if (is_huge_zero_pud(orig_pud)) { + zap_pud_deposited_table(tlb->mm, pud); + spin_unlock(ptl); + tlb_remove_page_size(tlb, pud_page(orig_pud), HPAGE_PUD_SIZE); } else { - /* No support for anonymous PUD pages yet */ - BUG(); + struct page *page = NULL; + int flush_needed = 1; + + if (pud_present(orig_pud)) { + page = pud_page(orig_pud); + page_remove_rmap(page, true); + VM_BUG_ON_PAGE(page_mapcount(page) < 0, page); + VM_BUG_ON_PAGE(!PageHead(page), page); + } else + WARN_ONCE(1, "Non present huge pmd without pmd migration enabled!"); + + if (PageAnon(page)) { + zap_pud_deposited_table(tlb->mm, pud); + add_mm_counter(tlb->mm, MM_ANONPAGES, -HPAGE_PUD_NR); + } else { + if (arch_needs_pgtable_deposit()) + zap_pud_deposited_table(tlb->mm, pud); + add_mm_counter(tlb->mm, MM_FILEPAGES, -HPAGE_PUD_NR); + } + + spin_unlock(ptl); + if (flush_needed) + tlb_remove_page_size(tlb, page, HPAGE_PUD_SIZE); } return 1; }