From patchwork Sat Sep 15 16:23:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10601553 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2D9A14DA for ; Sat, 15 Sep 2018 16:35:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8ED472A9D9 for ; Sat, 15 Sep 2018 16:35:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 82DCF2AA26; Sat, 15 Sep 2018 16:35:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EBE02A9D9 for ; Sat, 15 Sep 2018 16:35:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F8338E0004; Sat, 15 Sep 2018 12:35:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6A7F98E0001; Sat, 15 Sep 2018 12:35:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 597C08E0004; Sat, 15 Sep 2018 12:35:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 1B9F28E0001 for ; Sat, 15 Sep 2018 12:35:15 -0400 (EDT) Received: by mail-pl1-f199.google.com with SMTP id h4-v6so5766845pls.17 for ; Sat, 15 Sep 2018 09:35:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=6+eKTRMHGxIuQXh7ZEMX8h+NaT+2wxnpQFWD6QFiAhc=; b=DDb2RiAvI3T0IMCWryjQm8yLqE0///PWODG7H0rPoeUEhV4hePdYZEjb6z2WVCkJwY 0bwqx7kcUkc8YmRNjr8SslE900OKGJ2WzCYNnXJze+FIQoXbjcLZWQQ6K+GuSF6ATMBZ bupjW4pEEafjMhcqcNv6ZugqYRobH2nAa+1Y3wSPkhZCHoUv6e8mDV76CjRnLxMDX6h6 rFL53YaEYgF/mGrI9xhvYsVvc2YNImaDlWJW+v1Ws6AN/13pN4Pmm4MZklWOAs6fcdso yPJs/n8KZQoQ9AzHTfB1tw6NvIWm9qLOiDytnGNF7vTi3VnDaP3r2+SfvJqyYbvQFOVc 8aIQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APzg51ClY21p51NpQh+MwKNjFSaJLtrXKxu4LOsssZTIbbSFsQaB2K31 3k5lbyOAA9GQL22Yj5Xjq0stomNY3PyPF3TGIg1VAmyt50HAwJtqnsxhGPOkP/uPkhwriwj9ZKv 03u6+sbfMTnz1BKzLi0kfDxWHCpt9r537TI0xZEtYinRJCmdjcKj/n+rCPssaNhKf9w== X-Received: by 2002:a63:dd49:: with SMTP id g9-v6mr16241270pgj.356.1537029314730; Sat, 15 Sep 2018 09:35:14 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZauKfnHbRmBI14re8Mn9WGjZieQBYcpSfJED3vHkOJJqr6GEx+PrNf7RFiRG2jof/J6J3n X-Received: by 2002:a63:dd49:: with SMTP id g9-v6mr16241238pgj.356.1537029313723; Sat, 15 Sep 2018 09:35:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537029313; cv=none; d=google.com; s=arc-20160816; b=IZvDv2m75jPXujdpP2ekwF+walyK9ywMYrRim+u0V7TO9sfLivT0foTK3otw3ypr4k gOeZz4A/uq9gQu+Io6hD2FchwACh7W1xaqfots+QMqeJygsFFhL/VuhLSv3wIPvlHXoE Hm48ucHzB+/MtB/nlvcP+wORsFF0d1OVb3E3CbfZbK8YcBXQ0U2o3TPiXuyEEqIYqOQh tAozfDiEGWKryrSzWa6iFNZDze5qWZYErdcyVYQo5zyd25EX9OxMYRhYzv5N4PhE/QRH l/mFaiUP8qWiVp3+DdSJZ0MRw6xPpSVsmryTEQ3LFJ0ZjE7j6NB9n/0nkHVyTfVpega2 KjZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject; bh=6+eKTRMHGxIuQXh7ZEMX8h+NaT+2wxnpQFWD6QFiAhc=; b=sS/ropFBj0+ABcEZUujjfI5EfbV/iI91GHI2lNzAS/ewcrsEGhWshEMtIz7Iq//OUv XxuV3xiYGDIi6gnsrUDAC1MV7U03RT0LzurvwE86o2e3pz6Uwtx9SyLxHyRh24GCZUb1 Yr1dTZbF4VRsx4T0RU0ivw7Akd4IcwtVO7rxK+xQmDDIx8QAK6/5SVDxTNorcL+V4Sly TGwal1x33oDE7Gc6T2fbWKBYngz4JyNUTMFw5lEnDwOFtvrCrdDC2Qnmd9YxtWNcumpJ KIHW05sVPqIo5a9wSXzqsoeIcWDuNvkd+evNZjBdjBNR+29i2WRZOzqyHyIpN+A2HNB9 Yrkg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga02.intel.com (mga02.intel.com. [134.134.136.20]) by mx.google.com with ESMTPS id t7-v6si9896179pfh.3.2018.09.15.09.35.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 15 Sep 2018 09:35:13 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) client-ip=134.134.136.20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Sep 2018 09:35:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,377,1531810800"; d="scan'208";a="83785006" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga003.jf.intel.com with ESMTP; 15 Sep 2018 09:35:00 -0700 Subject: [PATCH 3/3] mm: Maintain randomization of page free lists From: Dan Williams To: akpm@linux-foundation.org Cc: Michal Hocko , Kees Cook , Dave Hansen , linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Sat, 15 Sep 2018 09:23:18 -0700 Message-ID: <153702859851.1603922.5390659652135091505.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <153702858249.1603922.12913911825267831671.stgit@dwillia2-desk3.amr.corp.intel.com> References: <153702858249.1603922.12913911825267831671.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When freeing a page with an order >= shuffle_page_order randomly select the front or back of the list for insertion. While the mm tries to defragment physical pages into huge pages this can tend to make the page allocator more predictable over time. Inject the front-back randomness to preserve the initial randomness established by shuffle_free_memory() when the kernel was booted. The overhead of this manipulation is constrained by only being applied for MAX_ORDER sized pages by default. Cc: Michal Hocko Cc: Kees Cook Cc: Dave Hansen Signed-off-by: Dan Williams --- include/linux/mmzone.h | 2 ++ mm/page_alloc.c | 27 +++++++++++++++++++++++++-- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index adf9b3a7440d..4a095432843d 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -98,6 +98,8 @@ extern int page_group_by_mobility_disabled; struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; + u64 rand; + u8 rand_bits; }; /* Used for pages not on another list */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 175f2e5f9e50..33a6b40ae463 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -746,6 +747,22 @@ static inline int page_is_buddy(struct page *page, struct page *buddy, return 0; } +static void add_to_free_area_random(struct page *page, struct free_area *area, + int migratetype) +{ + if (area->rand_bits == 0) { + area->rand_bits = 64; + area->rand = get_random_u64(); + } + + if (area->rand & 1) + add_to_free_area(page, area, migratetype); + else + add_to_free_area_tail(page, area, migratetype); + area->rand_bits--; + area->rand >>= 1; +} + /* * Freeing function for a buddy system allocator. * @@ -851,7 +868,8 @@ static inline void __free_one_page(struct page *page, * so it's less likely to be used soon and more likely to be merged * as a higher order page */ - if ((order < MAX_ORDER-2) && pfn_valid_within(buddy_pfn)) { + if ((order < MAX_ORDER-2) && pfn_valid_within(buddy_pfn) + && order < shuffle_page_order) { struct page *higher_page, *higher_buddy; combined_pfn = buddy_pfn & pfn; higher_page = page + (combined_pfn - pfn); @@ -865,7 +883,12 @@ static inline void __free_one_page(struct page *page, } } - add_to_free_area(page, &zone->free_area[order], migratetype); + if (order < shuffle_page_order) + add_to_free_area(page, &zone->free_area[order], migratetype); + else + add_to_free_area_random(page, &zone->free_area[order], + migratetype); + } /*