From patchwork Fri Dec 7 05:41:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10717465 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23EE8109C for ; Fri, 7 Dec 2018 05:42:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 121452EC8A for ; Fri, 7 Dec 2018 05:42:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 030822EC8C; Fri, 7 Dec 2018 05:42:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67EAD2EC8A for ; Fri, 7 Dec 2018 05:42:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B63566B7EA9; Fri, 7 Dec 2018 00:42:06 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AC2786B7EAB; Fri, 7 Dec 2018 00:42:06 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CC9B6B7EAC; Fri, 7 Dec 2018 00:42:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 3619F6B7EA9 for ; Fri, 7 Dec 2018 00:42:06 -0500 (EST) Received: by mail-pf1-f199.google.com with SMTP id 75so2391469pfq.8 for ; Thu, 06 Dec 2018 21:42:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=s55QkRo7bru0QHsNNF3RTjF55E2oTtnwkjkp42PNOvM=; b=CT/mqGOkrLlhnLLvv6jP7Xc8TI6HbDWySh95paRfqgBTIlHKTLT3pnELZbPxmMWa0N yzk2o12gA26at7gVa1VznCsHRvj1ocrCGK1W85kjdSFx2uVjvBdRX2RUv+2tUyn0cwT+ KuOyXukkDcbm+HqiVXiWa3FgBNRY1Wa27+2a/iTPh+4FBX+V6fJ/B93pJvGbRN8hu5ig o4yNV8yPLqWZF2jU2DVF41mXbkwxoHL812TTHCn+c0IXi5PnzoRVOX7bPjbn6pzqHI3f 6kFsuqr4UuzMIdaPv16sYZpNlj49/bD+LLkQLEOcL+XOiyHKOl5kOxlDDC1fkDyPrgqS twTA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AA+aEWby1l8Tn1n4gRVoYpaxnNfzHf+GqUbPAldc2lwGIbuVlku5QLmF CORRtfD/E4XAhvI2RHFAyAX+MEe0kVDBkhrqurrEUjXbXFOyUHc6wqPMDedOyMkPygczqfXeFIm sFhTFRSt1Jb3hRvADiT+SyonV9DKbtgIUJRVUGu8GKqgeFMXHncIVGC3DbV7RLemCnA== X-Received: by 2002:a62:109b:: with SMTP id 27mr897093pfq.227.1544161325856; Thu, 06 Dec 2018 21:42:05 -0800 (PST) X-Google-Smtp-Source: AFSGD/VO79QOx5Df3Wb+2UXUU7cZa/D7zl1zbTKdP5AcD36dF4/2qQi9Wf+Ri96o1NP7GviAGtHK X-Received: by 2002:a62:109b:: with SMTP id 27mr897062pfq.227.1544161324619; Thu, 06 Dec 2018 21:42:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544161324; cv=none; d=google.com; s=arc-20160816; b=W9AfloqFloA8PiOs7EVuCtNoidKifvvy4c6g0tnK98EdqQ7X0tr7q7F1h1sqG7iOmH XeRf+B+seUPKkqTBXq+kRQy+moM9rOamH0pItaUUfglIVlfRU2AsnP6h9MmwfwljGtpP tO0PrnYSbRU+XAszlLOgogF/JESlITg2anxAccrUjuN9tDtItquBlTQW0FBvB83YP5+Z bFKo2/Tu3yfqFoPwPDrGg9Q9dxqrRGgrtEBdM/fQmLauldt9z8Sgaprvwuu1ES0StrU7 v2M5aDyJTDPNOO4t8TMxGuhXxaF94+L/pe2LMTnn7aE410jRGPRDcs4aA/t0jS03QLl2 rq4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=s55QkRo7bru0QHsNNF3RTjF55E2oTtnwkjkp42PNOvM=; b=sUdBkglgAIX7sWG/b0S8caJdxXWlJoxmu6NKN+G+KcoXui6G1af/IuzYATvgmwoBK2 a9YDiBpFZ3m0zjd4mP5a1tWFQB0S/kwLjUSAZXNmPQ1KLjT160ZIUtaDa+zNDeh+5qgD Fl2AZuAQwXx25eJZAoX4II22nPDupyMjLWXUWNIev9HD+Y1CBA1ziJePYOAps5aB1Yui nTAjtGIsC2IVzZz2dMChjRHiFJlFo1NnEJXFmreUOMIR3L1KBcIbPyGIC62oCnMy07r2 FZSYbZZgeSW0kZnEhChD+gyorJoCll0WX7pn0otmQdllBTPOdn8k8KyVK55ovBB9iZKh utgA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga05.intel.com (mga05.intel.com. [192.55.52.43]) by mx.google.com with ESMTPS id cf16si2126256plb.227.2018.12.06.21.42.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 21:42:04 -0800 (PST) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) client-ip=192.55.52.43; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Dec 2018 21:42:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,324,1539673200"; d="scan'208";a="105567298" Received: from yhuang-mobile.sh.intel.com ([10.239.196.133]) by fmsmga007.fm.intel.com with ESMTP; 06 Dec 2018 21:42:01 -0800 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -V8 13/21] swap: Support PMD swap mapping in madvise_free() Date: Fri, 7 Dec 2018 13:41:13 +0800 Message-Id: <20181207054122.27822-14-ying.huang@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20181207054122.27822-1-ying.huang@intel.com> References: <20181207054122.27822-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/huge_memory.c | 52 ++++++++++++++++++++++++++++++++++-------------- mm/madvise.c | 2 +- 2 files changed, 38 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f3c0a9e8fb9a..9cf5d4fa6d98 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1899,6 +1899,15 @@ int do_huge_pmd_swap_page(struct vm_fault *vmf, pmd_t orig_pmd) } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1919,15 +1928,37 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (IS_ENABLED(CONFIG_THP_SWAP) && + !non_swap_entry(entry)) { + /* + * If part of THP is discarded, split the PMD + * swap mapping and operate on the PTEs + */ + if (next - addr != HPAGE_PMD_SIZE) { + __split_huge_swap_pmd(vma, addr, pmd); + goto out; + } + free_swap_and_cache(entry, HPAGE_PMD_NR); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1973,15 +2004,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index cbb3d7e38e51..0c1f96c605f8 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;