From patchwork Tue Jul 17 00:55:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10527849 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 79252600D0 for ; Tue, 17 Jul 2018 00:56:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61B5E285A5 for ; Tue, 17 Jul 2018 00:56:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 56380286B4; Tue, 17 Jul 2018 00:56:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B9491285A5 for ; Tue, 17 Jul 2018 00:56:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 37BD56B0270; Mon, 16 Jul 2018 20:55:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 290D36B0271; Mon, 16 Jul 2018 20:55:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 137E96B0272; Mon, 16 Jul 2018 20:55:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f70.google.com (mail-pl0-f70.google.com [209.85.160.70]) by kanga.kvack.org (Postfix) with ESMTP id BB9FB6B0270 for ; Mon, 16 Jul 2018 20:55:58 -0400 (EDT) Received: by mail-pl0-f70.google.com with SMTP id 70-v6so25711937plc.1 for ; Mon, 16 Jul 2018 17:55:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=lsJWFzURDDygLZjKeMTZ1CmJk9mgKzy8bymMHaEObAM=; b=rYp5bFiOe7kLLHFVdpPVFleZJicVWlH+dm1kzEht1ynnaJBwERtRgYseH0wrZaaTca um44vRhgtxa8d8YKOAEc9fiI508/5Q3OoMILbUj8smrxqpAS6WsMvh7mmk82Kpe1eu9s 8nAAF/NG1dA6Qa0CLcSd23e7TMRyGupx91bZqQ/gB8YDJBncCiYzuy1syr87bKRcSJRb EJg+Y5d9nn3DRaXUW7XCx/L7IaZfd+bjjfo695L91Lfi2xIYHX7ed5EP/FETkX+K0Y7r t/8qYLanAxHRU4Tv9tHFveWL03ODn6gpOuWKkRIMHSoTBXaWBaL7qCjKxn1ZnfXWV/+i ZppQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AOUpUlGYigP4pgKFCdfxe9OwDsreozweDFRPg1GhykAzJtvVDc6MJ9wK +NpOBBItxBKRy81KdN2SIfawb1/DMv6ECQ5XXWTjffDziFT0SGexuxOWQWD22YTjPSB4qSU2GWb J07/Ixj3Dynqb/jMk8lvn6Q5kT89D6DphC0t41CbszAdpcntjucm2LX0A65kY4I58KA== X-Received: by 2002:a17:902:76c4:: with SMTP id j4-v6mr18513855plt.19.1531788958454; Mon, 16 Jul 2018 17:55:58 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdvGopI++PuVLAmCaGwO6sukhQcbPgM+KrCNpWupVRL1y6ypf7lT3UIrBsN65mJyOpxG66g X-Received: by 2002:a17:902:76c4:: with SMTP id j4-v6mr18513819plt.19.1531788957716; Mon, 16 Jul 2018 17:55:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531788957; cv=none; d=google.com; s=arc-20160816; b=yZ6D6+EJ0zz3VAFRxAiyH3B44qO14tUn0rPToSU/bnh9sSa8ytLk0dzVVX5CWQ8l4m P8W8XpzqaQNcOs/aYttKzpXxgCcbJsCn5uJJaWG0kMFvx/fXLV3wE7Gc+r2ReA8Wm6DQ 29tIom3pKcBUi2f4IuFQdOZ0Qfs/V3TC4KJpT8D+S+am4ZbdFHTvRetyhyQV+KhkpT0f HF0MDhkUxKC6stVNG/dKQ+cFmwcftNbywBasBJ18Y+tj3ezC3HjEnECB73RIZa0f7Eyb eUg6wDNGNH1nkCMvCjUWnZh50+Wu1lDjF3kfIBuUalHv6QEDqcXgmt3d3Kd65Wa+oZIo BPzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=lsJWFzURDDygLZjKeMTZ1CmJk9mgKzy8bymMHaEObAM=; b=jQxigfOCzBrHyvFNrpJ7xp+MDj8ljkmWpSFsVXLAgqiF1CCgBm/FfiMhM4XeM3hM0y l6UWs2zG0KRMZG4sepAJi1vcno5ennhdR67IbN+RJRtBLb6evlpafCOcENDNusKsAq2h ja6pR7+cPE+iDXI4ulu6gWB8OcVVjyZTrjhGNE1FPpuAyciLM2grBaIppeZVxMmG8GrT otHTM7MyzEWBHFzKJ63QLYm6dRIHPWMUIatb91ZVC/5axYnNaa3BbAtva8r1Bk9qEU/g 3CKlodqoZw7CNh4ueNi0N8+Tjy4M6FQxgjNRz5aIsGUAdWrgLywCROkyS809M3tJabrJ aYzQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga18.intel.com (mga18.intel.com. [134.134.136.126]) by mx.google.com with ESMTPS id bc5-v6si30619643plb.413.2018.07.16.17.55.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Jul 2018 17:55:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) client-ip=134.134.136.126; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jul 2018 17:55:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,363,1526367600"; d="scan'208";a="245788297" Received: from yhuang6-ux31a.sh.intel.com ([10.239.197.184]) by fmsmga005.fm.intel.com with ESMTP; 16 Jul 2018 17:55:55 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Michal Hocko , Johannes Weiner , Shaohua Li , Hugh Dickins , Minchan Kim , Rik van Riel , Daniel Jordan , Dan Williams , Dave Hansen Subject: [PATCH v2 5/7] swap: Unify normal/huge code path in put_swap_page() Date: Tue, 17 Jul 2018 08:55:54 +0800 Message-Id: <20180717005556.29758-6-ying.huang@intel.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20180717005556.29758-1-ying.huang@intel.com> References: <20180717005556.29758-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In this patch, the normal/huge code path in put_swap_page() and several helper functions are unified to avoid duplicated code, bugs, etc. and make it easier to review the code. The removed lines are more than added lines. And the binary size is kept exactly same when CONFIG_TRANSPARENT_HUGEPAGE=n. Signed-off-by: "Huang, Ying" Suggested-by: Dave Hansen Cc: Michal Hocko Cc: Johannes Weiner Cc: Shaohua Li Cc: Hugh Dickins Cc: Minchan Kim Cc: Rik van Riel Cc: Daniel Jordan Cc: Dan Williams --- mm/swapfile.c | 83 ++++++++++++++++++++++++++--------------------------------- 1 file changed, 37 insertions(+), 46 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index a6d8b8117bc5..622edc47b67a 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -205,8 +205,16 @@ static void discard_swap_cluster(struct swap_info_struct *si, #ifdef CONFIG_THP_SWAP #define SWAPFILE_CLUSTER HPAGE_PMD_NR + +#define nr_swap_entries(nr) (nr) #else #define SWAPFILE_CLUSTER 256 + +/* + * Define nr_swap_entries() as constant to let compiler to optimize + * out some code if !CONFIG_THP_SWAP + */ +#define nr_swap_entries(nr) 1 #endif #define LATENCY_LIMIT 256 @@ -1249,18 +1257,7 @@ void swap_free(swp_entry_t entry) /* * Called after dropping swapcache to decrease refcnt to swap entries. */ -static void swapcache_free(swp_entry_t entry) -{ - struct swap_info_struct *p; - - p = _swap_info_get(entry); - if (p) { - if (!__swap_entry_free(p, entry, SWAP_HAS_CACHE)) - free_swap_slot(entry); - } -} - -static void swapcache_free_cluster(swp_entry_t entry) +void put_swap_page(struct page *page, swp_entry_t entry) { unsigned long offset = swp_offset(entry); unsigned long idx = offset / SWAPFILE_CLUSTER; @@ -1269,39 +1266,41 @@ static void swapcache_free_cluster(swp_entry_t entry) unsigned char *map; unsigned int i, free_entries = 0; unsigned char val; - - if (!IS_ENABLED(CONFIG_THP_SWAP)) - return; + int nr = nr_swap_entries(hpage_nr_pages(page)); si = _swap_info_get(entry); if (!si) return; - ci = lock_cluster(si, offset); - VM_BUG_ON(!cluster_is_huge(ci)); - map = si->swap_map + offset; - for (i = 0; i < SWAPFILE_CLUSTER; i++) { - val = map[i]; - VM_BUG_ON(!(val & SWAP_HAS_CACHE)); - if (val == SWAP_HAS_CACHE) - free_entries++; - } - if (!free_entries) { - for (i = 0; i < SWAPFILE_CLUSTER; i++) - map[i] &= ~SWAP_HAS_CACHE; - } - cluster_clear_huge(ci); - unlock_cluster(ci); - if (free_entries == SWAPFILE_CLUSTER) { - spin_lock(&si->lock); + if (nr == SWAPFILE_CLUSTER) { ci = lock_cluster(si, offset); - memset(map, 0, SWAPFILE_CLUSTER); + VM_BUG_ON(!cluster_is_huge(ci)); + map = si->swap_map + offset; + for (i = 0; i < SWAPFILE_CLUSTER; i++) { + val = map[i]; + VM_BUG_ON(!(val & SWAP_HAS_CACHE)); + if (val == SWAP_HAS_CACHE) + free_entries++; + } + if (!free_entries) { + for (i = 0; i < SWAPFILE_CLUSTER; i++) + map[i] &= ~SWAP_HAS_CACHE; + } + cluster_clear_huge(ci); unlock_cluster(ci); - mem_cgroup_uncharge_swap(entry, SWAPFILE_CLUSTER); - swap_free_cluster(si, idx); - spin_unlock(&si->lock); - } else if (free_entries) { - for (i = 0; i < SWAPFILE_CLUSTER; i++, entry.val++) { + if (free_entries == SWAPFILE_CLUSTER) { + spin_lock(&si->lock); + ci = lock_cluster(si, offset); + memset(map, 0, SWAPFILE_CLUSTER); + unlock_cluster(ci); + mem_cgroup_uncharge_swap(entry, SWAPFILE_CLUSTER); + swap_free_cluster(si, idx); + spin_unlock(&si->lock); + return; + } + } + if (nr == 1 || free_entries) { + for (i = 0; i < nr; i++, entry.val++) { if (!__swap_entry_free(si, entry, SWAP_HAS_CACHE)) free_swap_slot(entry); } @@ -1325,14 +1324,6 @@ int split_swap_cluster(swp_entry_t entry) } #endif -void put_swap_page(struct page *page, swp_entry_t entry) -{ - if (!PageTransHuge(page)) - swapcache_free(entry); - else - swapcache_free_cluster(entry); -} - static int swp_entry_cmp(const void *ent1, const void *ent2) { const swp_entry_t *e1 = ent1, *e2 = ent2;