From patchwork Thu Oct 4 02:15:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10625499 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D5B116B1 for ; Thu, 4 Oct 2018 02:29:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B58929130 for ; Thu, 4 Oct 2018 02:29:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2EEE72913D; Thu, 4 Oct 2018 02:29:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADF6029130 for ; Thu, 4 Oct 2018 02:29:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D85AD6B0269; Wed, 3 Oct 2018 22:29:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D354C6B026A; Wed, 3 Oct 2018 22:29:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C4C286B026B; Wed, 3 Oct 2018 22:29:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 848D46B0269 for ; Wed, 3 Oct 2018 22:29:16 -0400 (EDT) Received: by mail-pl1-f200.google.com with SMTP id 3-v6so7158943plq.6 for ; Wed, 03 Oct 2018 19:29:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=/Opoedhp4FZxe8ngemenYgsG6Spmrjeqm772M5fgCWE=; b=TeuUeEx4Ci1vIWanIa17YR7F9X1gu9lVsoCNeB3kL107re0jfdKyxiRy0K7/xP1ftC DT/Cd8ehiiTImgGgns0OQU+IITvezjDfAtZXghwJpohgUqibpfHW+feLhsXog1X9YAxH EbV04M/419dZpcLUbaTjfrHzrynNX6e1RByk3dSFHrKxLOl2O4opa5P7sFObk7WnacS8 kDcH6K6PmKvmh1U/AhAoz3xPcSZmsVrmG32jFr/JDN6UmhX4ks3Q2+R2JMMZ0KfoXe/4 B+tvHyLGx7KwmR71La9bIb7dxLZyegGkcOrWKJ1Xb4rHkgMLUCxuau2rKs6IlXXiflB1 xpug== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ABuFfohuQe15XMbcLQasOtd9UF4Pz5065LlF/BwEtckcu7MIldmwLidq UHaYhtx+HgCk1/IrXdvutII74/iFt0ldcUqxn51zSFdVLFMT5rQlszPD8PfU1bOZDx7uSoGHj1a Mkhuhgme7Y3/OZigg7GlsYZtfgdyXqXVSvIe/tuP7X3ouEUWq98Dt6DpKMI2aKhZTwQ== X-Received: by 2002:a62:438c:: with SMTP id l12-v6mr4461156pfi.175.1538620156189; Wed, 03 Oct 2018 19:29:16 -0700 (PDT) X-Google-Smtp-Source: ACcGV617WXn5bKo2f465XmBkqTokHnP4UcZ8SC0ZC10pWcM1+4pQj2DPF/NecPMojait7Sd72zlF X-Received: by 2002:a62:438c:: with SMTP id l12-v6mr4461123pfi.175.1538620155503; Wed, 03 Oct 2018 19:29:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538620155; cv=none; d=google.com; s=arc-20160816; b=AVU59Q3XYPcvtgVDqzToHOw2/edoNaP+qVmfkZODZnJvqh3Yl+zhwkNBtO4ldU2kVW iEMgsJY/1bxX8jS7qZ/xmmkn98hIwjRVWLoB0chIvBPQ89WxZWKMY/1BuuZ5rNS8fgYE /PBb8Htf9D5Ub5zOrvkMQKq66V9oKkjmdPzkeZMY6O4KNxQFV453oau1OV9I3wPcGcfo pN9i0PbWjCIMoXNS/Y8W5GxbdqCayFHBrRMbvGIqQf8r0o/9gapwCFdknUym/sGZ+AxC lwefsnxqidy6lu1SC6ivb5xSNjvTXgA3L5yCHAaEHEvvtxjDzE1IMtYV8LrciAIzEwBg 5eBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=/Opoedhp4FZxe8ngemenYgsG6Spmrjeqm772M5fgCWE=; b=SeTjwTQVeenvq0pe7ralaKXs3ErIJqBlRVwdbZPlPGDkqMG2eWzatqFK/5gFyCeRRf 5R8NyZS4sNdApu1yLhTu0gm+61LMfK6FB7njxtJ528MiyPaZRwj8DiAK1Ag1fqn1vN4j K3q0z7OaNypiOIxOHvor7KkZs8Io7L3RWeV38czKbOB4KpSwgnAhxhU3E5ckEiTsaRkD idv+OEEEKUwatCs1uBuYmhs1Df4dS6/5GSTOwlq1XOWBlG2NLXJOkGR/qbk5KB3Y5esp Bx3ve/nd7Y6tzprwY8XwJKUwR0qIzkBW1/yOAGaPwS2KoDHQ98EMa9hPsr5rYB3BhLWl 1fYg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga14.intel.com (mga14.intel.com. [192.55.52.115]) by mx.google.com with ESMTPS id 3-v6si3424926plo.318.2018.10.03.19.29.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Oct 2018 19:29:15 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) client-ip=192.55.52.115; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Oct 2018 19:29:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,338,1534834800"; d="scan'208";a="96302448" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga001.jf.intel.com with ESMTP; 03 Oct 2018 19:27:22 -0700 Subject: [PATCH v2 3/3] mm: Maintain randomization of page free lists From: Dan Williams To: akpm@linux-foundation.org Cc: Michal Hocko , Kees Cook , Dave Hansen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, keescook@chromium.org Date: Wed, 03 Oct 2018 19:15:34 -0700 Message-ID: <153861933441.2863953.2686611248399177664.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <153861931865.2863953.11185006931458762795.stgit@dwillia2-desk3.amr.corp.intel.com> References: <153861931865.2863953.11185006931458762795.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When freeing a page with an order >= shuffle_page_order randomly select the front or back of the list for insertion. While the mm tries to defragment physical pages into huge pages this can tend to make the page allocator more predictable over time. Inject the front-back randomness to preserve the initial randomness established by shuffle_free_memory() when the kernel was booted. The overhead of this manipulation is constrained by only being applied for MAX_ORDER sized pages by default. Cc: Michal Hocko Cc: Kees Cook Cc: Dave Hansen Signed-off-by: Dan Williams --- include/linux/mmzone.h | 2 ++ mm/page_alloc.c | 27 +++++++++++++++++++++++++-- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index adf9b3a7440d..4a095432843d 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -98,6 +98,8 @@ extern int page_group_by_mobility_disabled; struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; + u64 rand; + u8 rand_bits; }; /* Used for pages not on another list */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b4a1598fcab5..e659119351ad 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -746,6 +747,22 @@ static inline int page_is_buddy(struct page *page, struct page *buddy, return 0; } +static void add_to_free_area_random(struct page *page, struct free_area *area, + int migratetype) +{ + if (area->rand_bits == 0) { + area->rand_bits = 64; + area->rand = get_random_u64(); + } + + if (area->rand & 1) + add_to_free_area(page, area, migratetype); + else + add_to_free_area_tail(page, area, migratetype); + area->rand_bits--; + area->rand >>= 1; +} + /* * Freeing function for a buddy system allocator. * @@ -851,7 +868,8 @@ static inline void __free_one_page(struct page *page, * so it's less likely to be used soon and more likely to be merged * as a higher order page */ - if ((order < MAX_ORDER-2) && pfn_valid_within(buddy_pfn)) { + if ((order < MAX_ORDER-2) && pfn_valid_within(buddy_pfn) + && order < shuffle_page_order) { struct page *higher_page, *higher_buddy; combined_pfn = buddy_pfn & pfn; higher_page = page + (combined_pfn - pfn); @@ -865,7 +883,12 @@ static inline void __free_one_page(struct page *page, } } - add_to_free_area(page, &zone->free_area[order], migratetype); + if (order < shuffle_page_order) + add_to_free_area(page, &zone->free_area[order], migratetype); + else + add_to_free_area_random(page, &zone->free_area[order], + migratetype); + } /*