From patchwork Tue Nov 10 19:32:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11895163 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CEBE15E6 for ; Tue, 10 Nov 2020 19:32:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DDEB120678 for ; Tue, 10 Nov 2020 19:32:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="f2fiWJjs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDEB120678 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E32B66B005D; Tue, 10 Nov 2020 14:32:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE3196B0068; Tue, 10 Nov 2020 14:32:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D213F6B006C; Tue, 10 Nov 2020 14:32:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0208.hostedemail.com [216.40.44.208]) by kanga.kvack.org (Postfix) with ESMTP id A72726B005D for ; Tue, 10 Nov 2020 14:32:54 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4428F8249980 for ; Tue, 10 Nov 2020 19:32:54 +0000 (UTC) X-FDA: 77469506268.11.army62_2811d33272f7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 24EFA180F8B81 for ; Tue, 10 Nov 2020 19:32:54 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,david@redhat.com,,RULES_HIT:30034:30054:30062:30064:30070,0,RBL:63.128.21.124:@redhat.com:.lbl8.mailshell.net-64.10.201.10 62.18.0.100;04ygwryauq4ismky84wcd9jy9ss58yc8o6j9qa47ybk3rbtd9dzoja7cqseipir.gpf5zy95rqqqmk9m4ux6qzgrfa9idfaqucmnwpcyhgokj46mjaeqdx89zh7tmst.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:70,LUA_SUMMARY:none X-HE-Tag: army62_2811d33272f7 X-Filterd-Recvd-Size: 6436 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 10 Nov 2020 19:32:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605036772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=m4Ph1suYX4ben8sinoTI3mwXxIcZAzftReDT/Kw+gqs=; b=f2fiWJjsxmp+plr2ZWwFx90KXoasEOdrN7znPEcY9FFiiirdHrbCA+yjDdpGP1GMU6GlU5 TKdiMKB0iJcgXgoTD+CwpM0RVxd1vUyxwe1lw+6FLhUkwuc5IGP3tE5Nhkj9HmmeTHbgNi gtz9DP0HKSnCadOmHPQncVNpPjFDRiM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-200-SDgoVi_HO3GfYuNGMHgeGg-1; Tue, 10 Nov 2020 14:32:50 -0500 X-MC-Unique: SDgoVi_HO3GfYuNGMHgeGg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A9DA310082E0; Tue, 10 Nov 2020 19:32:48 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-232.ams2.redhat.com [10.36.114.232]) by smtp.corp.redhat.com (Postfix) with ESMTP id A86255B4A0; Tue, 10 Nov 2020 19:32:41 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Alexander Potapenko , Michal Hocko , Mike Kravetz , Vlastimil Babka , Mike Rapoport , Oscar Salvador , Kees Cook , Michael Ellerman Subject: [PATCH v1] mm/page_alloc: clear pages in alloc_contig_pages() with init_on_alloc=1 or __GFP_ZERO Date: Tue, 10 Nov 2020 20:32:40 +0100 Message-Id: <20201110193240.25401-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: commit 6471384af2a6 ("mm: security: introduce init_on_alloc=1 and init_on_free=1 boot options") resulted with init_on_alloc=1 in all pages leaving the buddy via alloc_pages() and friends to be initialized/cleared/zeroed on allocation. However, the same logic is currently not applied to alloc_contig_pages(): allocated pages leaving the buddy aren't cleared with init_on_alloc=1 and init_on_free=0. Let's also properly clear pages on that allocation path and add support for __GFP_ZERO. With this change, we will see double clearing of pages in some cases. One example are gigantic pages (either allocated via CMA, or allocated dynamically via alloc_contig_pages()) - which is the right thing to do (and to be optimized outside of the buddy in the callers) as discussed in: https://lkml.kernel.org/r/20201019182853.7467-1-gpiccoli@canonical.com This change implies that with init_on_alloc=1 - All CMA allocations will be cleared - Gigantic pages allocated via alloc_contig_pages() will be cleared - virtio-mem memory to be unplugged will be cleared. While this is suboptimal, it's similar to memory balloon drivers handling, where all pages to be inflated will get cleared as well. Cc: Andrew Morton Cc: Alexander Potapenko Cc: Michal Hocko Cc: Mike Kravetz Cc: Vlastimil Babka Cc: Mike Rapoport Cc: Oscar Salvador Cc: Kees Cook Cc: Michael Ellerman Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index eed4f4075b3c..0361b119b74e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8453,6 +8453,19 @@ static int __alloc_contig_migrate_range(struct compact_control *cc, return 0; } +static void __alloc_contig_clear_range(unsigned long start_pfn, + unsigned long end_pfn) +{ + unsigned long pfn; + + for (pfn = start_pfn; pfn < end_pfn; pfn += MAX_ORDER_NR_PAGES) { + cond_resched(); + kernel_init_free_pages(pfn_to_page(pfn), + min_t(unsigned long, end_pfn - pfn, + MAX_ORDER_NR_PAGES)); + } +} + /** * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate @@ -8461,7 +8474,8 @@ static int __alloc_contig_migrate_range(struct compact_control *cc, * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks * in range must have the same migratetype and it must * be either of the two. - * @gfp_mask: GFP mask to use during compaction + * @gfp_mask: GFP mask to use during compaction. __GFP_ZERO clears allocated + * pages. * * The PFN range does not have to be pageblock or MAX_ORDER_NR_PAGES * aligned. The PFN range must belong to a single zone. @@ -8488,7 +8502,7 @@ int alloc_contig_range(unsigned long start, unsigned long end, .mode = MIGRATE_SYNC, .ignore_skip_hint = true, .no_set_skip_hint = true, - .gfp_mask = current_gfp_context(gfp_mask), + .gfp_mask = current_gfp_context(gfp_mask & ~__GFP_ZERO), .alloc_contig = true, }; INIT_LIST_HEAD(&cc.migratepages); @@ -8600,6 +8614,9 @@ int alloc_contig_range(unsigned long start, unsigned long end, if (end != outer_end) free_contig_range(end, outer_end - end); + if (!want_init_on_free() && want_init_on_alloc(gfp_mask)) + __alloc_contig_clear_range(start, end); + done: undo_isolate_page_range(pfn_max_align_down(start), pfn_max_align_up(end), migratetype); @@ -8653,7 +8670,8 @@ static bool zone_spans_last_pfn(const struct zone *zone, /** * alloc_contig_pages() -- tries to find and allocate contiguous range of pages * @nr_pages: Number of contiguous pages to allocate - * @gfp_mask: GFP mask to limit search and used during compaction + * @gfp_mask: GFP mask to limit search and used during compaction. __GFP_ZERO + * clears allocated pages. * @nid: Target node * @nodemask: Mask for other possible nodes *