From patchwork Mon Mar 23 05:52:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joonsoo Kim X-Patchwork-Id: 11452381 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 309051731 for ; Mon, 23 Mar 2020 05:52:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E479120714 for ; Mon, 23 Mar 2020 05:52:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RoK9TGay" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E479120714 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D52CD6B000A; Mon, 23 Mar 2020 01:52:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D044E8E0003; Mon, 23 Mar 2020 01:52:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1AF46B000D; Mon, 23 Mar 2020 01:52:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id AA70F6B000A for ; Mon, 23 Mar 2020 01:52:44 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DEF7B180AD811 for ; Mon, 23 Mar 2020 05:52:44 +0000 (UTC) X-FDA: 76625557848.27.whip40_6f420ab8af028 X-Spam-Summary: 2,0,0,053c2b5955dfb100,d41d8cd98f00b204,js1304@gmail.com,,RULES_HIT:2:41:355:379:541:800:960:966:968:973:988:989:1260:1345:1359:1437:1535:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4049:4120:4321:4385:4605:5007:6261:6653:7576:7903:8957:9413:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12517:12519:12555:12679:12895:12986:13255:14096:14394:21080:21444:21451:21627:21666:21740:21990:30054:30070:30075:30090,0,RBL:209.85.214.195:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: whip40_6f420ab8af028 X-Filterd-Recvd-Size: 9764 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Mar 2020 05:52:44 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id t16so5459464plr.8 for ; Sun, 22 Mar 2020 22:52:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8BKTKeS86EzAAVAM7E7CqauMkIwqwO+JtvB/TRaImBU=; b=RoK9TGaySXHN1OlYzFQBROE5RbaQWaFK2mNV/EaBD4a5/kfEW/c7qJyIkkUuWWsxhh P7Xdetk+EgpULzPhTk4qxQkgzm9GbAKOf9OnOFSaLv9G4h0mnNWHYIB1O9meSHUZybol H2iZgraxKYQ451dlIIWdGTDJXQJYcsP6IAf6mP6t8v7t/TTcCMnh7R5jD9tOO52JuQ5W tBKnItUR8Os5uEWS9AKaKv/hQ1XyDJ9shXNtmIQtjdG2c3TA0ooqTOETR+FGT5g++9jY +v+S0ZmpqTwT/ZXBbJXfgRL/ERQGnvi19/1OgL6rZGN5OUidSOFretMddCm2GUPHb3+e 8tzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8BKTKeS86EzAAVAM7E7CqauMkIwqwO+JtvB/TRaImBU=; b=oTueytzmVWFI45NuKpcHoaniL8tZJ93UufeNCs5UR4x2NvlRFS/RDy/Kl6Vl0rhsnj vdAmOydZXGn3RB+cedd9fq8FucwUYMTZ6ugVr/s+3DQHP230ph9WIVXOoonyNy3NxvVx VrMIfRHUCw3YdYF9zDYxB1zr5f7TXB0F9WIRPuDDmpzFt1D13Fq5e4A/i6dqff13u9BT FyggPU1s/RLzTxW5ha0mXq/azrqMryphQIKGUiNoB6YKjhVUwahVBb767Z+3a+pLUmoq eeBUcVZRPJJHQcpJVGMi7h4FVgY4EzkcukpfKprpSvxyYdCEc0vy/BuVP9Vy1febxNRx nQwA== X-Gm-Message-State: ANhLgQ2kH2tpRiWKamKSujwbu52jKTZ+36G4NUsNl/K1cCUzwq02Tx0K O2Ki59jL0K7X/yAxQAYADZQ= X-Google-Smtp-Source: ADFU+vvZLquJa8MhHFazV10Alh3WWus90SRxZ3uUpTHkON3UkFV9f9Ju1Y/jnbCYVMgb0iku1sML6w== X-Received: by 2002:a17:90b:3711:: with SMTP id mg17mr24045432pjb.73.1584942762728; Sun, 22 Mar 2020 22:52:42 -0700 (PDT) Received: from localhost.localdomain ([114.206.198.176]) by smtp.gmail.com with ESMTPSA id y30sm12563058pff.67.2020.03.22.22.52.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 22 Mar 2020 22:52:42 -0700 (PDT) From: js1304@gmail.com X-Google-Original-From: iamjoonsoo.kim@lge.com To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Johannes Weiner , Michal Hocko , Hugh Dickins , Minchan Kim , Vlastimil Babka , Mel Gorman , kernel-team@lge.com, Joonsoo Kim Subject: [PATCH v4 4/8] mm/swapcache: support to handle the exceptional entries in swapcache Date: Mon, 23 Mar 2020 14:52:08 +0900 Message-Id: <1584942732-2184-5-git-send-email-iamjoonsoo.kim@lge.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584942732-2184-1-git-send-email-iamjoonsoo.kim@lge.com> References: <1584942732-2184-1-git-send-email-iamjoonsoo.kim@lge.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joonsoo Kim Swapcache doesn't handle the exceptional entries since there is no case using it. In the following patch, workingset detection for anonymous page will be implemented and it stores the shadow entries as exceptional entries into the swapcache. So, we need to handle the exceptional entries and this patch implements it. Signed-off-by: Joonsoo Kim Acked-by: Johannes Weiner --- include/linux/swap.h | 10 ++++++---- mm/shmem.c | 3 ++- mm/swap_state.c | 26 ++++++++++++++++++++------ mm/vmscan.c | 2 +- 4 files changed, 29 insertions(+), 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 954e13e..273de48 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -408,9 +408,11 @@ extern struct address_space *swapper_spaces[]; extern unsigned long total_swapcache_pages(void); extern void show_swap_cache_info(void); extern int add_to_swap(struct page *page); -extern int add_to_swap_cache(struct page *, swp_entry_t, gfp_t); +extern int add_to_swap_cache(struct page *page, swp_entry_t entry, + gfp_t gfp, void **shadowp); extern int __add_to_swap_cache(struct page *page, swp_entry_t entry); -extern void __delete_from_swap_cache(struct page *, swp_entry_t entry); +extern void __delete_from_swap_cache(struct page *page, + swp_entry_t entry, void *shadow); extern void delete_from_swap_cache(struct page *); extern void free_page_and_swap_cache(struct page *); extern void free_pages_and_swap_cache(struct page **, int); @@ -565,13 +567,13 @@ static inline int add_to_swap(struct page *page) } static inline int add_to_swap_cache(struct page *page, swp_entry_t entry, - gfp_t gfp_mask) + gfp_t gfp_mask, void **shadowp) { return -1; } static inline void __delete_from_swap_cache(struct page *page, - swp_entry_t entry) + swp_entry_t entry, void *shadow) { } diff --git a/mm/shmem.c b/mm/shmem.c index 8793e8c..c6663ad 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1370,7 +1370,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) list_add(&info->swaplist, &shmem_swaplist); if (add_to_swap_cache(page, swap, - __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) { + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN, + NULL) == 0) { spin_lock_irq(&info->lock); shmem_recalc_inode(inode); info->swapped++; diff --git a/mm/swap_state.c b/mm/swap_state.c index 8e7ce9a..f06af84 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -111,12 +111,15 @@ void show_swap_cache_info(void) * add_to_swap_cache resembles add_to_page_cache_locked on swapper_space, * but sets SwapCache flag and private instead of mapping and index. */ -int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp) +int add_to_swap_cache(struct page *page, swp_entry_t entry, + gfp_t gfp, void **shadowp) { struct address_space *address_space = swap_address_space(entry); pgoff_t idx = swp_offset(entry); XA_STATE_ORDER(xas, &address_space->i_pages, idx, compound_order(page)); unsigned long i, nr = compound_nr(page); + unsigned long nrexceptional = 0; + void *old; VM_BUG_ON_PAGE(!PageLocked(page), page); VM_BUG_ON_PAGE(PageSwapCache(page), page); @@ -132,10 +135,17 @@ int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp) goto unlock; for (i = 0; i < nr; i++) { VM_BUG_ON_PAGE(xas.xa_index != idx + i, page); + old = xas_load(&xas); + if (xa_is_value(old)) { + nrexceptional++; + if (shadowp) + *shadowp = old; + } set_page_private(page + i, entry.val + i); xas_store(&xas, page); xas_next(&xas); } + address_space->nrexceptional -= nrexceptional; address_space->nrpages += nr; __mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, nr); ADD_CACHE_INFO(add_total, nr); @@ -155,7 +165,8 @@ int add_to_swap_cache(struct page *page, swp_entry_t entry, gfp_t gfp) * This must be called only on pages that have * been verified to be in the swap cache. */ -void __delete_from_swap_cache(struct page *page, swp_entry_t entry) +void __delete_from_swap_cache(struct page *page, + swp_entry_t entry, void *shadow) { struct address_space *address_space = swap_address_space(entry); int i, nr = hpage_nr_pages(page); @@ -167,12 +178,14 @@ void __delete_from_swap_cache(struct page *page, swp_entry_t entry) VM_BUG_ON_PAGE(PageWriteback(page), page); for (i = 0; i < nr; i++) { - void *entry = xas_store(&xas, NULL); + void *entry = xas_store(&xas, shadow); VM_BUG_ON_PAGE(entry != page, entry); set_page_private(page + i, 0); xas_next(&xas); } ClearPageSwapCache(page); + if (shadow) + address_space->nrexceptional += nr; address_space->nrpages -= nr; __mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, -nr); ADD_CACHE_INFO(del_total, nr); @@ -209,7 +222,7 @@ int add_to_swap(struct page *page) * Add it to the swap cache. */ err = add_to_swap_cache(page, entry, - __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN); + __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN, NULL); if (err) /* * add_to_swap_cache() doesn't return -EEXIST, so we can safely @@ -247,7 +260,7 @@ void delete_from_swap_cache(struct page *page) struct address_space *address_space = swap_address_space(entry); xa_lock_irq(&address_space->i_pages); - __delete_from_swap_cache(page, entry); + __delete_from_swap_cache(page, entry, NULL); xa_unlock_irq(&address_space->i_pages); put_swap_page(page, entry); @@ -418,7 +431,8 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, /* May fail (-ENOMEM) if XArray node allocation failed. */ __SetPageLocked(new_page); __SetPageSwapBacked(new_page); - err = add_to_swap_cache(new_page, entry, gfp_mask & GFP_KERNEL); + err = add_to_swap_cache(new_page, entry, + gfp_mask & GFP_KERNEL, NULL); if (likely(!err)) { /* Initiate read into locked page */ SetPageWorkingset(new_page); diff --git a/mm/vmscan.c b/mm/vmscan.c index 0493c25..9871861 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -909,7 +909,7 @@ static int __remove_mapping(struct address_space *mapping, struct page *page, if (PageSwapCache(page)) { swp_entry_t swap = { .val = page_private(page) }; mem_cgroup_swapout(page, swap); - __delete_from_swap_cache(page, swap); + __delete_from_swap_cache(page, swap, NULL); xa_unlock_irqrestore(&mapping->i_pages, flags); put_swap_page(page, swap); } else {