From patchwork Mon Nov 26 23:27:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB7FF13BB for ; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC6082A5B2 for ; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A09402A63A; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 077022A5D1 for ; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 321F76B4467; Mon, 26 Nov 2018 18:27:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2A9336B4468; Mon, 26 Nov 2018 18:27:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14A916B4469; Mon, 26 Nov 2018 18:27:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id C46696B4467 for ; Mon, 26 Nov 2018 18:27:55 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id v2so10936786plg.6 for ; Mon, 26 Nov 2018 15:27:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=gNGmozdeLydO2lSANHRatkPVmfiAppDs7BSNKAYbn3s=; b=dhM2mXRQFFbFv0Iyvpnp3UBnUotG6fVNg08hpDGwfVj3DMOzg7hJJMDKnuHEEGRFtv HtLQbByJYsyN8p04ZZI6pYAMUIMYAFiFTEKpscT/wQTy9tEgsTe9mCHFDwn/04XGhrnp VqzKod+Xl28mAnthovzpbZ0CeVq8MgapWcOgnKO+09f0+JXGKklbePGY9kNUa40tw3hO zFZaKA0yVfEyvMZZ1hnjxG14e4clwX4p8m2pZK3mS+gLisf6GXU0dQLMMd9+o/4xrWrd WQN15Gl/fVqMyrjHOZjuH//Zg3mnDFCEbHQrnE4dk5hW0k0XgKt2UN7385jimkMyn7iX GvnQ== X-Gm-Message-State: AA+aEWYJLo5PwIN2+GOdAvOapICNoJSvGF+5gsJJ8pvjc8xf5Bd90vd7 h/455ozZH+StUqbI2VgtiAizZXXu486Mq3zprXWakkMEsg/+utuatLOLQXYb/724QgeKNxOl5Ck K6599om+Yk6rQjWqlSwwvrKeHkDUFVRiXLF6UFZ/kwXMWA82ciaPZLWv01NjWtH4hfNFrfwh89S eNTWykSyNQetrNDYTNXuV26Y6cLTVAFVj6NpimrYt8nGd10KzlJTZ4DclnouS0eyuHpmNzz1VWT gpc0IV1nJRF7II9SxQt01IvorD6qb8KimMmN/NQfmaTEcy09Lmydl8oQvJv9eDYHCMfdpIrBiUP dB90MULPfXMSK2wD8XhuGTV3UTCsuFQSfurguo5rWwP6mBeu8w0FIQESty9T9xiLT4PdEK5tkvW x X-Received: by 2002:a17:902:820d:: with SMTP id x13mr30562853pln.229.1543274875428; Mon, 26 Nov 2018 15:27:55 -0800 (PST) X-Received: by 2002:a17:902:820d:: with SMTP id x13mr30562810pln.229.1543274874596; Mon, 26 Nov 2018 15:27:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274874; cv=none; d=google.com; s=arc-20160816; b=Po+4N1Axd6A3EZA6jZvtyW7jCRPMrWT+iRwU5LNsg/2yaJmuPwNioDucQlTABqdRQC c6olFdR+sNtWWxzcccR0zthGVdlf9rxjKPOYFetDVmT4isHJffeiy8iV6oZfZEAwc0PG 8nBgFwHN3p+zhD77Vj2YZkom4VWxnLuBGOs0uwJoIgqJbtX/5cdH3ES+h4mdVPrZyB92 bZggLQlHARekIFGldQElXsYhOKGx+NmOLvd5OPmKKzlagLETx3ckmW9cK6lgiLmoLG5k SiI7AuHxaXoEIN2v2d/BdZsLhi3wvYFP/fOS/GuQio/qKKMxvWe+FmCzX/Lsxf2zxLKG /b7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=gNGmozdeLydO2lSANHRatkPVmfiAppDs7BSNKAYbn3s=; b=nK+ACfqqZJfcTG33F1Zw5DYUrm1lmLTpainoCePqy2w0ellY6Y4biu8h5AZDZ59lmM JDjv8luzd6A6b+evs+1y4nlZkba9yvRqAziCG+n9y6tq6j3pqwiYxHHPOEdnQkdXzSTN cER2sSQS6Li70clrlmwWMtCMtuqvBPPbS+ky7kjTJ/sEPRjuGqQqKaH+oB013Okss37d O6yzMzEevJ0asvRSJXXn003kbzNuZKnGVQF/d1twVFlCtmwFHgygyXwGuYoffrtLQjkG NbPWDHl4yqHKktXfrr5J7svVElbTarZo0qUPj5Dbg/ad0jQfU15xHd1FezUC/2+HRJYP WlhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BJ82sdYv; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 38sor2512811pld.67.2018.11.26.15.27.54 for (Google Transport Security); Mon, 26 Nov 2018 15:27:54 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BJ82sdYv; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=gNGmozdeLydO2lSANHRatkPVmfiAppDs7BSNKAYbn3s=; b=BJ82sdYvHkZVvWoaqZ8ZLjHKNOzu3E/Yf+ZjIH9kQ2E6lziw6F+gXGe/Rh5YBeKTmR zqqPhYumWKpw9UTmzxPsrdc5FDeQSFMCZklqjsynW7+vGbm39FOG3mSmAoJubHbUD8zX 0c1HjKOUbTc2+4uUQJmu1dHs64Eh/WQ7Qp1QxuVow2dSDVncrJHMPqbWPC0KkXc4JJOf YHcvonYDI85TzmaXBO9y5AAGVBuHzBeZ4FK++wuD7NqxlfJbL9nIIVuTg6IQgqDfdKqE oNEeydEMldkJnHATjqkIl7V2Q0o3/7RJfgwmtuJyMgwIIp0lgKZe+JNkjY62U8g042N5 u1gQ== X-Google-Smtp-Source: AFSGD/XOwOs9IeycZUMd03Rn345TXyBLOMkysnyNi/KTJxp8MLM6J53HxXqgChMOf8Y1J8dCWZoJAQ== X-Received: by 2002:a17:902:24d:: with SMTP id 71mr29118169plc.225.1543274873842; Mon, 26 Nov 2018 15:27:53 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id w10-v6sm1930848pfg.1.2018.11.26.15.27.52 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:27:53 -0800 (PST) Date: Mon, 26 Nov 2018 15:27:52 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 07/10] mm/khugepaged: minor reorderings in collapse_shmem() In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Several cleanups in collapse_shmem(): most of which probably do not really matter, beyond doing things in a more familiar and reassuring order. Simplify the failure gotos in the main loop, and on success update stats while interrupts still disabled from the last iteration. Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ --- mm/khugepaged.c | 72 ++++++++++++++++++++++--------------------------- 1 file changed, 32 insertions(+), 40 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1c402d33547e..9d4e9ff1af95 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1329,10 +1329,10 @@ static void collapse_shmem(struct mm_struct *mm, goto out; } + __SetPageLocked(new_page); + __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; - __SetPageSwapBacked(new_page); - __SetPageLocked(new_page); BUG_ON(!page_ref_freeze(new_page, 1)); /* @@ -1366,13 +1366,13 @@ static void collapse_shmem(struct mm_struct *mm, if (index == start) { if (!xas_next_entry(&xas, end - 1)) { result = SCAN_TRUNCATED; - break; + goto xa_locked; } xas_set(&xas, index); } if (!shmem_charge(mapping->host, 1)) { result = SCAN_FAIL; - break; + goto xa_locked; } xas_store(&xas, new_page + (index % HPAGE_PMD_NR)); nr_none++; @@ -1387,13 +1387,12 @@ static void collapse_shmem(struct mm_struct *mm, result = SCAN_FAIL; goto xa_unlocked; } - xas_lock_irq(&xas); - xas_set(&xas, index); } else if (trylock_page(page)) { get_page(page); + xas_unlock_irq(&xas); } else { result = SCAN_PAGE_LOCK; - break; + goto xa_locked; } /* @@ -1408,11 +1407,10 @@ static void collapse_shmem(struct mm_struct *mm, result = SCAN_TRUNCATED; goto out_unlock; } - xas_unlock_irq(&xas); if (isolate_lru_page(page)) { result = SCAN_DEL_PAGE_LRU; - goto out_isolate_failed; + goto out_unlock; } if (page_mapped(page)) @@ -1432,7 +1430,9 @@ static void collapse_shmem(struct mm_struct *mm, */ if (!page_ref_freeze(page, 3)) { result = SCAN_PAGE_COUNT; - goto out_lru; + xas_unlock_irq(&xas); + putback_lru_page(page); + goto out_unlock; } /* @@ -1444,24 +1444,26 @@ static void collapse_shmem(struct mm_struct *mm, /* Finally, replace with the new page. */ xas_store(&xas, new_page + (index % HPAGE_PMD_NR)); continue; -out_lru: - xas_unlock_irq(&xas); - putback_lru_page(page); -out_isolate_failed: - unlock_page(page); - put_page(page); - goto xa_unlocked; out_unlock: unlock_page(page); put_page(page); - break; + goto xa_unlocked; } - xas_unlock_irq(&xas); + __inc_node_page_state(new_page, NR_SHMEM_THPS); + if (nr_none) { + struct zone *zone = page_zone(new_page); + + __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); + __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); + } + +xa_locked: + xas_unlock_irq(&xas); xa_unlocked: + if (result == SCAN_SUCCEED) { struct page *page, *tmp; - struct zone *zone = page_zone(new_page); /* * Replacing old pages with new one has succeeded, now we @@ -1476,11 +1478,11 @@ static void collapse_shmem(struct mm_struct *mm, copy_highpage(new_page + (page->index % HPAGE_PMD_NR), page); list_del(&page->lru); - unlock_page(page); - page_ref_unfreeze(page, 1); page->mapping = NULL; + page_ref_unfreeze(page, 1); ClearPageActive(page); ClearPageUnevictable(page); + unlock_page(page); put_page(page); index++; } @@ -1489,28 +1491,17 @@ static void collapse_shmem(struct mm_struct *mm, index++; } - local_irq_disable(); - __inc_node_page_state(new_page, NR_SHMEM_THPS); - if (nr_none) { - __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); - __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); - } - local_irq_enable(); - - /* - * Remove pte page tables, so we can re-fault - * the page as huge. - */ - retract_page_tables(mapping, start); - /* Everything is ready, let's unfreeze the new_page */ - set_page_dirty(new_page); SetPageUptodate(new_page); page_ref_unfreeze(new_page, HPAGE_PMD_NR); + set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); lru_cache_add_anon(new_page); - unlock_page(new_page); + /* + * Remove pte page tables, so we can re-fault the page as huge. + */ + retract_page_tables(mapping, start); *hpage = NULL; khugepaged_pages_collapsed++; @@ -1543,8 +1534,8 @@ static void collapse_shmem(struct mm_struct *mm, xas_store(&xas, page); xas_pause(&xas); xas_unlock_irq(&xas); - putback_lru_page(page); unlock_page(page); + putback_lru_page(page); xas_lock_irq(&xas); } VM_BUG_ON(nr_none); @@ -1553,9 +1544,10 @@ static void collapse_shmem(struct mm_struct *mm, /* Unfreeze new_page, caller would take care about freeing it */ page_ref_unfreeze(new_page, 1); mem_cgroup_cancel_charge(new_page, memcg, true); - unlock_page(new_page); new_page->mapping = NULL; } + + unlock_page(new_page); out: VM_BUG_ON(!list_empty(&pagelist)); /* TODO: tracepoints */