From patchwork Thu Apr 30 20:41:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11521555 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F9AF13B2 for ; Thu, 30 Apr 2020 20:41:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9077524954 for ; Thu, 30 Apr 2020 20:41:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9077524954 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B3ADA8E0005; Thu, 30 Apr 2020 16:41:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B11C98E0001; Thu, 30 Apr 2020 16:41:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4F3D8E0005; Thu, 30 Apr 2020 16:41:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id 8942A8E0001 for ; Thu, 30 Apr 2020 16:41:32 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5121E181AC537 for ; Thu, 30 Apr 2020 20:41:32 +0000 (UTC) X-FDA: 76765692024.17.cub40_2a6962911a644 X-Spam-Summary: 2,0,0,8eab4f11a341c806,d41d8cd98f00b204,yang.shi@linux.alibaba.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1345:1431:1437:1534:1543:1711:1730:1747:1777:1792:2393:2559:2562:2898:3138:3139:3140:3141:3142:3353:3743:3865:3866:3867:5007:6261:8957:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12555:12895:13161:13229:14096:14181:14721:21060:21080:21451:21627:21990:30003:30054:30070,0,RBL:115.124.30.44:@linux.alibaba.com:.lbl8.mailshell.net-62.20.2.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: cub40_2a6962911a644 X-Filterd-Recvd-Size: 4314 Received: from out30-44.freemail.mail.aliyun.com (out30-44.freemail.mail.aliyun.com [115.124.30.44]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Thu, 30 Apr 2020 20:41:31 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e07484;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0Tx7vcTH_1588279279; Received: from localhost(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0Tx7vcTH_1588279279) by smtp.aliyun-inc.com(127.0.0.1); Fri, 01 May 2020 04:41:27 +0800 From: Yang Shi To: kirill.shutemov@linux.intel.com, hughd@google.com, aarcange@redhat.com, akpm@linux-foundation.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 linux-next PATCH 1/2] mm: khugepaged: add exceed_max_ptes_* helpers Date: Fri, 1 May 2020 04:41:18 +0800 Message-Id: <1588279279-61908-1-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The max_ptes_{swap|none|shared} are defined to tune the behavior of khugepaged. The are checked at a couple of places with open coding. Replace the opencoding to exceed_pax_ptes_{swap|none_shared} helpers to improve the readability. Cc: Kirill A. Shutemov Cc: Hugh Dickins Cc: Andrea Arcangeli Signed-off-by: Yang Shi --- mm/khugepaged.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a02a4c5..0c8d30b 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -339,6 +339,21 @@ struct attribute_group khugepaged_attr_group = { }; #endif /* CONFIG_SYSFS */ +static inline bool exceed_max_ptes_none(unsigned int *nr_ptes) +{ + return (++(*nr_ptes) > khugepaged_max_ptes_none); +} + +static inline bool exceed_max_ptes_swap(unsigned int *nr_ptes) +{ + return (++(*nr_ptes) > khugepaged_max_ptes_swap); +} + +static inline bool exceed_max_ptes_shared(unsigned int *nr_ptes) +{ + return (++(*nr_ptes) > khugepaged_max_ptes_shared); +} + int hugepage_madvise(struct vm_area_struct *vma, unsigned long *vm_flags, int advice) { @@ -604,7 +619,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, if (pte_none(pteval) || (pte_present(pteval) && is_zero_pfn(pte_pfn(pteval)))) { if (!userfaultfd_armed(vma) && - ++none_or_zero <= khugepaged_max_ptes_none) { + !exceed_max_ptes_none(&none_or_zero)) { continue; } else { result = SCAN_EXCEED_NONE_PTE; @@ -624,7 +639,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, VM_BUG_ON_PAGE(!PageAnon(page), page); if (page_mapcount(page) > 1 && - ++shared > khugepaged_max_ptes_shared) { + exceed_max_ptes_shared(&shared)) { result = SCAN_EXCEED_SHARED_PTE; goto out; } @@ -1234,7 +1249,7 @@ static int khugepaged_scan_pmd(struct mm_struct *mm, _pte++, _address += PAGE_SIZE) { pte_t pteval = *_pte; if (is_swap_pte(pteval)) { - if (++unmapped <= khugepaged_max_ptes_swap) { + if (!exceed_max_ptes_swap(&unmapped)) { /* * Always be strict with uffd-wp * enabled swap entries. Please see @@ -1252,7 +1267,7 @@ static int khugepaged_scan_pmd(struct mm_struct *mm, } if (pte_none(pteval) || is_zero_pfn(pte_pfn(pteval))) { if (!userfaultfd_armed(vma) && - ++none_or_zero <= khugepaged_max_ptes_none) { + !exceed_max_ptes_none(&none_or_zero)) { continue; } else { result = SCAN_EXCEED_NONE_PTE; @@ -1286,7 +1301,7 @@ static int khugepaged_scan_pmd(struct mm_struct *mm, } if (page_mapcount(page) > 1 && - ++shared > khugepaged_max_ptes_shared) { + exceed_max_ptes_shared(&shared)) { result = SCAN_EXCEED_SHARED_PTE; goto out_unmap; } @@ -1961,7 +1976,7 @@ static void khugepaged_scan_file(struct mm_struct *mm, continue; if (xa_is_value(page)) { - if (++swap > khugepaged_max_ptes_swap) { + if (exceed_max_ptes_swap(&swap)) { result = SCAN_EXCEED_SWAP_PTE; break; } From patchwork Thu Apr 30 20:41:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 11521559 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9BB1792C for ; Thu, 30 Apr 2020 20:41:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 66DDD206C0 for ; Thu, 30 Apr 2020 20:41:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 66DDD206C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9B46C8E0006; Thu, 30 Apr 2020 16:41:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 964738E0001; Thu, 30 Apr 2020 16:41:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A0ED8E0006; Thu, 30 Apr 2020 16:41:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 740B08E0001 for ; Thu, 30 Apr 2020 16:41:40 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3AA4C4DBF for ; Thu, 30 Apr 2020 20:41:40 +0000 (UTC) X-FDA: 76765692360.04.van33_2b937728d724a X-Spam-Summary: 2,0,0,e7f7ec2eeb67083f,d41d8cd98f00b204,yang.shi@linux.alibaba.com,,RULES_HIT:41:355:379:541:800:960:965:966:968:973:988:989:1260:1261:1345:1359:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2895:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3874:4321:4385:4390:4395:5007:6119:6261:7903:8957:10004:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12555:12895:12986:13255:14096:14181:14721:21060:21080:21451:21627:21795:21966:21987:30003:30051:30054:30070,0,RBL:115.124.30.130:@linux.alibaba.com:.lbl8.mailshell.net-62.20.2.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: van33_2b937728d724a X-Filterd-Recvd-Size: 3677 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Thu, 30 Apr 2020 20:41:38 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01422;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0Tx7vcUa_1588279287; Received: from localhost(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0Tx7vcUa_1588279287) by smtp.aliyun-inc.com(127.0.0.1); Fri, 01 May 2020 04:41:35 +0800 From: Yang Shi To: kirill.shutemov@linux.intel.com, hughd@google.com, aarcange@redhat.com, akpm@linux-foundation.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 linux-next PATCH 2/2] mm: khugepaged: don't have to put being freed page back to lru Date: Fri, 1 May 2020 04:41:19 +0800 Message-Id: <1588279279-61908-2-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1588279279-61908-1-git-send-email-yang.shi@linux.alibaba.com> References: <1588279279-61908-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When khugepaged successfully isolated and copied data from old page to collapsed THP, the old page is about to be freed if its last mapcount is gone. So putting the page back to lru sounds not that productive in this case since the page might be isolated by vmscan but it can't be reclaimed by vmscan since it can't be unmapped by try_to_unmap() at all. Actually if khugepaged is the last user of this page so it can be freed directly. So, clearing active and unevictable flags, unlocking and dropping refcount from isolate instead of calling putback_lru_page(). Cc: Kirill A. Shutemov Cc: Hugh Dickins Cc: Andrea Arcangeli Signed-off-by: Yang Shi --- v2: Check mapcount and skip putback lru if the last mapcount is gone mm/khugepaged.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 0c8d30b..1fdd677 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -559,10 +559,18 @@ void __khugepaged_exit(struct mm_struct *mm) static void release_pte_page(struct page *page) { mod_node_page_state(page_pgdat(page), - NR_ISOLATED_ANON + page_is_file_lru(page), - -compound_nr(page)); - unlock_page(page); - putback_lru_page(page); + NR_ISOLATED_ANON + page_is_file_lru(page), -compound_nr(page)); + + if (total_mapcount(page)) { + unlock_page(page); + putback_lru_page(page); + } else { + ClearPageActive(page); + ClearPageUnevictable(page); + unlock_page(page); + /* Drop refcount from isolate */ + put_page(page); + } } static void release_pte_pages(pte_t *pte, pte_t *_pte, @@ -771,8 +779,6 @@ static void __collapse_huge_page_copy(pte_t *pte, struct page *page, } else { src_page = pte_page(pteval); copy_user_highpage(page, src_page, address, vma); - if (!PageCompound(src_page)) - release_pte_page(src_page); /* * ptl mostly unnecessary, but preempt has to * be disabled to update the per-cpu stats @@ -786,6 +792,8 @@ static void __collapse_huge_page_copy(pte_t *pte, struct page *page, pte_clear(vma->vm_mm, address, _pte); page_remove_rmap(src_page, false); spin_unlock(ptl); + if (!PageCompound(src_page)) + release_pte_page(src_page); free_page_and_swap_cache(src_page); } }