From patchwork Wed May 9 08:38:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10388675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 24B0A60353 for ; Wed, 9 May 2018 08:39:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A33F28E13 for ; Wed, 9 May 2018 08:39:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0EEE528E26; Wed, 9 May 2018 08:39:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD0B028E13 for ; Wed, 9 May 2018 08:39:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 015DA6B04A3; Wed, 9 May 2018 04:39:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F06B76B04A7; Wed, 9 May 2018 04:39:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF59D6B04A8; Wed, 9 May 2018 04:39:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id 9A2786B04A3 for ; Wed, 9 May 2018 04:39:48 -0400 (EDT) Received: by mail-pf0-f199.google.com with SMTP id b25so25885080pfn.10 for ; Wed, 09 May 2018 01:39:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=YmUy/qjMSIoIiPIKFht414r5lLt3yMx+zDS+z12pu40=; b=onGExI5mkvTF8iZUHur6aD2KcVKoivn43ACpyoCJZb3G6OuVDfi8NU7lwBbxQaxEDc V7fXIwSR1V+C/2VWpCe4ZHsC5WtBNhbT/qDIdlMOlEQBFDpYIzo2QGsVz7ijs2e2jrRG jybBDNPO8jnbKkhiV7bLb24N8qkqNcq8amgYZ/SCtaXxrbVFsAsIIWDf118CUjTDDrDz gUofrkOlEwcwKEvJbYMJ/PkcjFiRBIdPbLPKMJVt/pJOidPdiNCAludh8SPtG7sHGVpQ 0t2DfkglSQbyeOI1efogvVU6RMlZsN51Vfy4oidkzG2aQza3vFY8itM6nplgvaDGhzF/ +t7w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com X-Gm-Message-State: ALQs6tBkQfKCJ0ZBY8nkBtCq76NsxTyYWnPPH+tSh74cpmkwu5XlBk/n HNxmAwnJMNFHylErp/mmWO/3UwxcrVvlh02mCZuw0cWibQVj35KRWpwOTqcSB4BQu7PJGFcaNmV aMa0GfxdYqr2xGjUYX3+zt5PLG2cJ6pZ4HICoxK1EdydSsROYaxVoR8oR2HULvUJL8Q== X-Received: by 10.98.13.151 with SMTP id 23mr43483690pfn.231.1525855188283; Wed, 09 May 2018 01:39:48 -0700 (PDT) X-Google-Smtp-Source: AB8JxZpMJ6/2IoQ0UD7S2Sel6SB8dLF/ZHTcss6nRtp8vzcPW3Ex06Vgca6tQ4CnnFGgRPt2Rx80 X-Received: by 10.98.13.151 with SMTP id 23mr43483664pfn.231.1525855187444; Wed, 09 May 2018 01:39:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525855187; cv=none; d=google.com; s=arc-20160816; b=PxANwD1m1eikR8bGNdeqcp4PLwfWggg9KBX0Fug3832++ORS0gMKTycSiRmA/Ez3J/ kEkc21FB0B5QS9GzHifLKWLT81hMdq0KnR1mmhdoe0EXH2FwCTVJR2SjGKDLplW+meCr Fw9gmTrl1JL16E0Uc2TJumSaY7TRl4sBQ/7WYCKEa38hsQOtrrq5sozVSwUb1poE41Ms M/P1ChBrxRzspAm2Y58tNDP57kNQxiLXYky9hRRmck0oD0prrvmialYRuZ6iSxjqPpiD msZKIV5SRUmWbfk6Ng9FzH7dg1kX/wVQXBoNyKvbcZe24DB9Sg5sfqhr/GN9xDusMok+ whoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=YmUy/qjMSIoIiPIKFht414r5lLt3yMx+zDS+z12pu40=; b=oYFDIX0xKdsDJAbPNPfE8V1M4fTta12W2nJhf9cJpZm4fRHWtSqAP93aDTOIW4UNPW R2ml45n1AX2W/6dHyRI2J5aoTEDLnhKKscfDd8O6yq6TFuQWu2fn1f3htBBcxlt3mBT3 FwSLPOI4Y3eINtQA2CCSqObNRqcYWbLIkEky4UoQwCPZEW1Ukl/pBv0tqp7Un4xMrHDQ QzSq+yugHNAU5E9i2+U4vZp50QZg2l5lcxhiwgi0+3uEQNrAopzLnhPxKevczEpU2YrW HEU8O6z0yGMdAAyW8z3wZ+I5UCPMfMs8ziggcBY/gKMqM41ijEB3BynmnVCn57pMkzo1 YFSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com Received: from mga18.intel.com (mga18.intel.com. [134.134.136.126]) by mx.google.com with ESMTPS id y11-v6si18624861pgv.473.2018.05.09.01.39.47 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 09 May 2018 01:39:47 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) client-ip=134.134.136.126; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 May 2018 01:39:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,381,1520924400"; d="scan'208";a="52769688" Received: from yhuang-gentoo.sh.intel.com ([10.239.193.148]) by fmsmga004.fm.intel.com with ESMTP; 09 May 2018 01:39:43 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , "Kirill A. Shutemov" , Andrea Arcangeli , Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Dave Hansen , Naoya Horiguchi , Zi Yan Subject: [PATCH -mm -V2 13/21] mm, THP, swap: Support PMD swap mapping in madvise_free() Date: Wed, 9 May 2018 16:38:38 +0800 Message-Id: <20180509083846.14823-14-ying.huang@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180509083846.14823-1-ying.huang@intel.com> References: <20180509083846.14823-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying When madvise_free() found a PMD swap mapping, if only part of the huge swap cluster is operated on, the PMD swap mapping will be split and fallback to PTE swap mapping processing. Otherwise, if all huge swap cluster is operated on, free_swap_and_cache() will be called to decrease the PMD swap mapping count and probably free the swap space and the THP in swap cache too. Signed-off-by: "Huang, Ying" Cc: "Kirill A. Shutemov" Cc: Andrea Arcangeli Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Dave Hansen Cc: Naoya Horiguchi Cc: Zi Yan --- mm/huge_memory.c | 50 +++++++++++++++++++++++++++++++++++--------------- mm/madvise.c | 2 +- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 26f15fe99931..2f6786425249 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1845,6 +1845,15 @@ static inline void __split_huge_swap_pmd(struct vm_area_struct *vma, } #endif +static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) +{ + pgtable_t pgtable; + + pgtable = pgtable_trans_huge_withdraw(mm, pmd); + pte_free(mm, pgtable); + mm_dec_nr_ptes(mm); +} + /* * Return true if we do MADV_FREE successfully on entire pmd page. * Otherwise, return false. @@ -1865,15 +1874,35 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, goto out_unlocked; orig_pmd = *pmd; - if (is_huge_zero_pmd(orig_pmd)) - goto out; - if (unlikely(!pmd_present(orig_pmd))) { - VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - goto out; + swp_entry_t entry = pmd_to_swp_entry(orig_pmd); + + if (is_migration_entry(entry)) { + VM_BUG_ON(!thp_migration_supported()); + goto out; + } else if (thp_swap_supported() && !non_swap_entry(entry)) { + /* If part of THP is discarded */ + if (next - addr != HPAGE_PMD_SIZE) { + unsigned long haddr = addr & HPAGE_PMD_MASK; + + __split_huge_swap_pmd(vma, haddr, pmd); + goto out; + } + free_swap_and_cache(entry, true); + pmd_clear(pmd); + zap_deposited_table(mm, pmd); + if (current->mm == mm) + sync_mm_rss(mm); + add_mm_counter(mm, MM_SWAPENTS, -HPAGE_PMD_NR); + ret = true; + goto out; + } else + VM_BUG_ON(1); } + if (is_huge_zero_pmd(orig_pmd)) + goto out; + page = pmd_page(orig_pmd); /* * If other processes are mapping this page, we couldn't discard @@ -1919,15 +1948,6 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return ret; } -static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd) -{ - pgtable_t pgtable; - - pgtable = pgtable_trans_huge_withdraw(mm, pmd); - pte_free(mm, pgtable); - mm_dec_nr_ptes(mm); -} - int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr) { diff --git a/mm/madvise.c b/mm/madvise.c index d180000c626b..e03e85a20fb4 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -321,7 +321,7 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, unsigned long next; next = pmd_addr_end(addr, end); - if (pmd_trans_huge(*pmd)) + if (pmd_trans_huge(*pmd) || is_swap_pmd(*pmd)) if (madvise_free_huge_pmd(tlb, vma, pmd, addr, next)) goto next;