From patchwork Wed Oct 10 07:19:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10634125 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87ED4679F for ; Wed, 10 Oct 2018 07:28:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 124E6294F6 for ; Wed, 10 Oct 2018 07:28:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 06271296AF; Wed, 10 Oct 2018 07:28:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 683D2294F6 for ; Wed, 10 Oct 2018 07:27:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A3EB76B0274; Wed, 10 Oct 2018 03:27:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9CCA96B027A; Wed, 10 Oct 2018 03:27:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B8596B0277; Wed, 10 Oct 2018 03:27:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id D8DEF6B0276 for ; Wed, 10 Oct 2018 03:27:42 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id n81-v6so3865039pfi.20 for ; Wed, 10 Oct 2018 00:27:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=mX5ypdDMWV6Io78WDyuMiseXST/DCvpQ080Aii+wG/A=; b=g7fRdMs3eVYW/9OwYAgl7KYT12mtTrq3dDot1DaL0fHcqoIAfYoc9OS+6Tb0Zncjiu ycOQi8avFcAMbfCh3aJG42jWn1xnwYZZxdNS+Ylx8A3hQgFVgQBSJ/sUybWL/TiSohh5 0k54d+fCWR9EIEvOgnTK8ZY13CJYw8AfJf2+Rx5oe7CINDuyMwZ92p+lo5bFOaVnti7c LmKkdl7qxTN+v+bfDQZdQSf7mQC3gqKL8clWHOZ2s8p3t/4CsU82hU/2DpnUNI1wAkKV WJSvxfmcU1i7BlGR2rNufXC5Xv+RpmyTgDXju6fRUYYYrZNVsBW3tuqT9sN3sLxzY+7J NKww== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfohqt84A1Npi6oYYDN72wFKjOta+Q8Vq486QL/35jvBhOrot6J/u PBtxo18tLYv6+LiZYQWPX1022fqozU+n601xH5iZSQG2FkCV5+bW24KDf7sndi3F4Jsv1mK/fqD g1OYNsF9FeJUdUY6nEn3TyG2dnSHgbKCI+ZzgmLKBfjEFQs8fDamtydDEcWGD6s69/w== X-Received: by 2002:a17:902:8f8f:: with SMTP id z15-v6mr30134305plo.305.1539156462562; Wed, 10 Oct 2018 00:27:42 -0700 (PDT) X-Google-Smtp-Source: ACcGV62yoH5WWSWKBKUTJoAkaFqhuZiglvO8fvA87vWcgCjeyI/AVPunmz1i/PX4gOtsFYIdAyT0 X-Received: by 2002:a17:902:8f8f:: with SMTP id z15-v6mr30134271plo.305.1539156461880; Wed, 10 Oct 2018 00:27:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539156461; cv=none; d=google.com; s=arc-20160816; b=aGdgLTOQgTrcAcfhwbJzX11lBSc40+hSrFSEXkn1KwYDLZfWJ3v1crbJYeIqzZGX+Y JfH+T2mMi/zxhybyb60qHrE9SfMBLFuUcab9mhdl19uFKgtOt05iP8nc1tAw8DwoLswa C8M2/Jc72BctLq2AfLUNbR1v9kN1UQHjLctcGAgcmy/c8xQDJyK2GMRjoWfKwtFA5Aog lenWEAvCtMp5C4Uxj8nlZx7JkurHtO5pazWh0feA1Qz02/lX0SCPVN2PMGiy4T/aCN/G Nshbg7Km+f8FlPoq/THU+8nMZOvuYSuJdbYtiG9pXKVuRjS+FWZyNal6r8J3PI6mSfmO lRow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=mX5ypdDMWV6Io78WDyuMiseXST/DCvpQ080Aii+wG/A=; b=x2VXbaL2/nr+KH0JF9WQTCfM9Qh2woFCZsJ6eAyMzSbHILNN57wHGXCBtp85OJHTxd 93CfefdEeBKf3DxRFjOzOHEEnB7jAbbeL/VtmjaFUg2yHZdkJWoztIgdGCZP4RGPso1m xtbtjSyTAKeVT/Dlb9FirdmmKBllu4956Fop4UCiwfTS/ULhk/ngpgElyK82//FfEVO4 dNWcw1mCA6Pwx1JmFie1ct5KwYsB+wNOXVi7lJyNVq1TyWLW8+/YwWDzjku7XOKoOTlP pMYBfCIojGvKLPut5Eato1zQVnINzgGxuKbYJkXX4lZENfQ5oOTgnxEC/IwV3K/6F9y5 3E1w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga06.intel.com (mga06.intel.com. [134.134.136.31]) by mx.google.com with ESMTPS id n84-v6si21364295pfg.127.2018.10.10.00.27.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 Oct 2018 00:27:41 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) client-ip=134.134.136.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.31 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2018 00:27:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,363,1534834800"; d="scan'208";a="93870228" Received: from yhuang-mobile.sh.intel.com ([10.239.198.87]) by fmsmga002.fm.intel.com with ESMTP; 10 Oct 2018 00:19:30 -0700 From: Huang Ying To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan , Daniel Jordan Subject: [PATCH -V6 13/21] swap: Support PMD swap mapping in madvise_free() Date: Wed, 10 Oct 2018 15:19:16 +0800 Message-Id: <20181010071924.18767-14-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20181010071924.18767-1-ying.huang@intel.com> References: <20181010071924.18767-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan Cc: Daniel Jordan --- mm/huge_memory.c | 54 +++++++++++++++++++++++++++++++++++++++--------------- mm/madvise.c | 2 +- 2 files changed, 40 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0ec71f907fa5..60b4105734b1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1891,6 +1891,15 @@ int do_huge_pmd_swap_page(struct vm_fault *vmf, pmd_t orig_pmd) } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1911,15 +1920,39 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (IS_ENABLED(CONFIG_THP_SWAP) && + !non_swap_entry(entry)) { + /* + * If part of THP is discarded, split the PMD + * swap mapping and operate on the PTEs + */ + if (next - addr != HPAGE_PMD_SIZE) { + unsigned long haddr = addr & HPAGE_PMD_MASK; + + __split_huge_swap_pmd(vma, haddr, pmd); + goto out; + } + free_swap_and_cache(entry, HPAGE_PMD_NR); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1965,15 +1998,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index 50282ba862e2..20101ff125d0 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;