From patchwork Tue Aug 10 06:26:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 12428059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3B68C4320A for ; Tue, 10 Aug 2021 06:26:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 742F36105A for ; Tue, 10 Aug 2021 06:26:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 742F36105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 9DC918D0001; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F35A6B0075; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 599348D0002; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id 25F736B0075 for ; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id AB96B8249980 for ; Tue, 10 Aug 2021 06:26:26 +0000 (UTC) X-FDA: 78458186772.31.D2F4346 Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by imf08.hostedemail.com (Postfix) with ESMTP id 6B24F3000246 for ; Tue, 10 Aug 2021 06:26:26 +0000 (UTC) Received: by mail-lf1-f49.google.com with SMTP id w20so12645481lfu.7 for ; Mon, 09 Aug 2021 23:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov-name.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yhM9rzQZh1hFlmIsdV4JGrfhMhv28jDR+Y6mMaIyOd4=; b=iI6obK6IDsJ5WRxV6OM1M/tAx2RUXXdpWjOOljATFp3gKmjcJd16cMca3CRczpVk90 vdUUedOazrl3bkG2F0IfOJ6AItLxNYJwCF1n4JKF50+TflcgrRKXqnKIm/J/THTyieZr KsM8HcwID/NK5sqFV1M5hQPxhiER2JGcI0YABCRIa4iVTkIyyqaNxusuU50potUPb3Oi PUB+8f9Ij8rl+y/qgnVCYh7HdM2F7I53V4Dh1DLgERtHiB2qPWoUfEd0GCbBsUCX6DAg /nKOFs0c2RkzNtTU4ew3Npu0B4ABZRqH7MBpxW8V+hsglvKIqewABkD2rTgSDih6C3Up n1uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yhM9rzQZh1hFlmIsdV4JGrfhMhv28jDR+Y6mMaIyOd4=; b=UpnSc/mtqPScd17iniBdCVm4dmso8KCNo2zxhlVclmvTiaWXEssCpZ2kuyy2VDwTzt LmsvQOW72DCEBsGvJBFJCI3UEhCLpLBYGBD8po5ysTPzlccJWA1kJbLKujrMABMeN0HZ YSB4+iUNhdXvXVKYhu8rmhJvhg5BLSnxqTIYQ+FJ/vr+ZuhMdh7TCSiV8jhdl8BLy3la e4rXSTmbUrVl+OdPiZsaIzvdvYrFlyQ1C+NPogXQCYLVKLkeIk57GKpzGV1WUfn4E1KH FclP3bhmn391LBQW/k/Qf5tOM0plppIx/f44vGpfsALSzYhXctOikVROb/p3Mv9BqUQT gIwQ== X-Gm-Message-State: AOAM532mEHDXEIsc3e6azeAfP1nToY/G/EasHH3iUOFdAXuqKceKd7jz YbOn3I96hxsBNPS5cio9MNWSYw== X-Google-Smtp-Source: ABdhPJywOJGNlI7IqnnlU0lsO+YPTdIUQE6GK58teoRywSZHJeH0Ifngw9u6S0AYbWyYjWXtNj2VvA== X-Received: by 2002:a05:6512:e86:: with SMTP id bi6mr21473291lfb.592.1628576784944; Mon, 09 Aug 2021 23:26:24 -0700 (PDT) Received: from box.localdomain ([86.57.175.117]) by smtp.gmail.com with ESMTPSA id w27sm264085lfa.308.2021.08.09.23.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Aug 2021 23:26:22 -0700 (PDT) From: "Kirill A. Shutemov" X-Google-Original-From: "Kirill A. Shutemov" Received: by box.localdomain (Postfix, from userid 1000) id A60D9102BDA; Tue, 10 Aug 2021 09:26:34 +0300 (+03) To: Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel Cc: Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCH 3/5] x86/boot/compressed: Handle unaccepted memory Date: Tue, 10 Aug 2021 09:26:24 +0300 Message-Id: <20210810062626.1012-4-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210810062626.1012-1-kirill.shutemov@linux.intel.com> References: <20210810062626.1012-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6B24F3000246 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=shutemov-name.20150623.gappssmtp.com header.s=20150623 header.b=iI6obK6I; dmarc=none; spf=none (imf08.hostedemail.com: domain of kirill@shutemov.name has no SPF policy when checking 209.85.167.49) smtp.mailfrom=kirill@shutemov.name X-Stat-Signature: ic85sc6bznowf89z5ieq4aixdtuc3it7 X-HE-Tag: 1628576786-106350 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Firmware is responsible for accepting memory where compressed kernel image and initrd land. But kernel has to accept memory for decompression buffer: accept memory just before decompression starts KASLR allowed to use unaccepted memory for output buffer. Signed-off-by: Kirill A. Shutemov --- arch/x86/boot/compressed/bitmap.c | 62 ++++++++++++++++++++ arch/x86/boot/compressed/kaslr.c | 14 ++++- arch/x86/boot/compressed/misc.c | 9 +++ arch/x86/boot/compressed/unaccepted_memory.c | 13 ++++ arch/x86/include/asm/unaccepted_memory.h | 2 + 5 files changed, 98 insertions(+), 2 deletions(-) diff --git a/arch/x86/boot/compressed/bitmap.c b/arch/x86/boot/compressed/bitmap.c index bf58b259380a..ba2de61c0823 100644 --- a/arch/x86/boot/compressed/bitmap.c +++ b/arch/x86/boot/compressed/bitmap.c @@ -2,6 +2,48 @@ /* Taken from lib/string.c */ #include +#include +#include + +unsigned long _find_next_bit(const unsigned long *addr1, + const unsigned long *addr2, unsigned long nbits, + unsigned long start, unsigned long invert, unsigned long le) +{ + unsigned long tmp, mask; + + if (unlikely(start >= nbits)) + return nbits; + + tmp = addr1[start / BITS_PER_LONG]; + if (addr2) + tmp &= addr2[start / BITS_PER_LONG]; + tmp ^= invert; + + /* Handle 1st word. */ + mask = BITMAP_FIRST_WORD_MASK(start); + if (le) + mask = swab(mask); + + tmp &= mask; + + start = round_down(start, BITS_PER_LONG); + + while (!tmp) { + start += BITS_PER_LONG; + if (start >= nbits) + return nbits; + + tmp = addr1[start / BITS_PER_LONG]; + if (addr2) + tmp &= addr2[start / BITS_PER_LONG]; + tmp ^= invert; + } + + if (le) + tmp = swab(tmp); + + return min(start + __ffs(tmp), nbits); +} void __bitmap_set(unsigned long *map, unsigned int start, int len) { @@ -22,3 +64,23 @@ void __bitmap_set(unsigned long *map, unsigned int start, int len) *p |= mask_to_set; } } + +void __bitmap_clear(unsigned long *map, unsigned int start, int len) +{ + unsigned long *p = map + BIT_WORD(start); + const unsigned int size = start + len; + int bits_to_clear = BITS_PER_LONG - (start % BITS_PER_LONG); + unsigned long mask_to_clear = BITMAP_FIRST_WORD_MASK(start); + + while (len - bits_to_clear >= 0) { + *p &= ~mask_to_clear; + len -= bits_to_clear; + bits_to_clear = BITS_PER_LONG; + mask_to_clear = ~0UL; + p++; + } + if (len) { + mask_to_clear &= BITMAP_LAST_WORD_MASK(size); + *p &= ~mask_to_clear; + } +} diff --git a/arch/x86/boot/compressed/kaslr.c b/arch/x86/boot/compressed/kaslr.c index e36690778497..89cdc17dcbf1 100644 --- a/arch/x86/boot/compressed/kaslr.c +++ b/arch/x86/boot/compressed/kaslr.c @@ -729,10 +729,20 @@ process_efi_entries(unsigned long minimum, unsigned long image_size) * but in practice there's firmware where using that memory leads * to crashes. * - * Only EFI_CONVENTIONAL_MEMORY is guaranteed to be free. + * Only EFI_CONVENTIONAL_MEMORY and EFI_UNACCEPTED_MEMORY (if + * supported) are guaranteed to be free. */ - if (md->type != EFI_CONVENTIONAL_MEMORY) + + switch (md->type) { + case EFI_CONVENTIONAL_MEMORY: + break; + case EFI_UNACCEPTED_MEMORY: + if (IS_ENABLED(CONFIG_UNACCEPTED_MEMORY)) + break; continue; + default: + continue; + } if (efi_soft_reserve_enabled() && (md->attribute & EFI_MEMORY_SP)) diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c index 743f13ea25c1..eeefcde8394d 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -18,6 +18,7 @@ #include "../string.h" #include "../voffset.h" #include +#include /* * WARNING!! @@ -435,6 +436,14 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, #endif debug_putstr("\nDecompressing Linux... "); + + if (IS_ENABLED(CONFIG_UNACCEPTED_MEMORY) && + boot_params->unaccepted_memory) { + debug_putstr("Accepting memory... "); + accept_memory((phys_addr_t)output, + (phys_addr_t)output + needed_size); + } + __decompress(input_data, input_len, NULL, NULL, output, output_len, NULL, error); parse_elf(output); diff --git a/arch/x86/boot/compressed/unaccepted_memory.c b/arch/x86/boot/compressed/unaccepted_memory.c index c2eca85b5073..17b70627b0cd 100644 --- a/arch/x86/boot/compressed/unaccepted_memory.c +++ b/arch/x86/boot/compressed/unaccepted_memory.c @@ -34,3 +34,16 @@ void mark_unaccepted(struct boot_params *params, u64 start, u64 num) bitmap_set((unsigned long *)params->unaccepted_memory, start / PMD_SIZE, npages); } + +void accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long *unaccepted_memory; + unsigned int rs, re; + + unaccepted_memory = (unsigned long *)boot_params->unaccepted_memory; + bitmap_for_each_set_region(unaccepted_memory, rs, re, + start / PMD_SIZE, end / PMD_SIZE) { + __accept_memory(rs * PMD_SIZE, re * PMD_SIZE); + bitmap_clear(unaccepted_memory, rs, re - rs); + } +} diff --git a/arch/x86/include/asm/unaccepted_memory.h b/arch/x86/include/asm/unaccepted_memory.h index cbc24040b853..f1f835d3cd78 100644 --- a/arch/x86/include/asm/unaccepted_memory.h +++ b/arch/x86/include/asm/unaccepted_memory.h @@ -9,4 +9,6 @@ struct boot_params; void mark_unaccepted(struct boot_params *params, u64 start, u64 num); +void accept_memory(phys_addr_t start, phys_addr_t end); + #endif