From patchwork Tue Aug 10 06:26:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 12428057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2D1FC4338F for ; Tue, 10 Aug 2021 06:26:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7671C6105A for ; Tue, 10 Aug 2021 06:26:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7671C6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 7BCBE6B0072; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7470D8D0001; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46E186B0078; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 245226B0074 for ; Tue, 10 Aug 2021 02:26:27 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C9331181EC4A2 for ; Tue, 10 Aug 2021 06:26:26 +0000 (UTC) X-FDA: 78458186772.22.2098E1B Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by imf15.hostedemail.com (Postfix) with ESMTP id 8A1CBD004985 for ; Tue, 10 Aug 2021 06:26:26 +0000 (UTC) Received: by mail-lf1-f43.google.com with SMTP id d4so10385962lfk.9 for ; Mon, 09 Aug 2021 23:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov-name.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wBfWTIHuoUkiwmizGdYmrNr5TmKY7E7Jf9eiL0J7uVk=; b=0h4kwTTU9bgsr5+7M2dbKNl6HdsvupLgIPjYSGLs6f8fz3APWo6zJSPPWqNXW4N/Li UunV42SnRXg/QfbOe6YhRwV3FHVUGTQLs/9luwBZqG1bEgUsxo7c8tHkZ+XQFDIuTpU/ dE60LpbSxBQE68n6v6KGu9eg0zHbWg3jw9ysv+LSIJuLieR4bO4yps2uMY/2h7Q1vR2P q/QdcwI+rjdhzncpWhT1oKNYF68WnvwozDBbb0/dok1B+62Pzyuw46SN/Dp7slhPoMUW fd/rPnR6ZCWvPlxEXkK6rmZqS/R7ooKaJrHFi/MtCXKJLO9zPFiKSaryFrn4KeSfJFd0 VQ5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wBfWTIHuoUkiwmizGdYmrNr5TmKY7E7Jf9eiL0J7uVk=; b=psEcionckYcKVdRXV41LQwuj1AU4ds5whAABNlya7d5u8sg7ECKUsR5WAh3zWGgI09 0XqgEluB2Ek+/aG3/fUkBsqJGcutG+zvMxorZE0Bi17OHtfwT/HB6JbfK9DR8CSXMDDi w8FHtr5vcThRcfmA1iaqxi7CgOiIsuyWAREGX7GSr+9SSzxxVRe3FddhfT6PmAEWXXBj KQSECbmfZ+f3AbnaM9rgGAGO+B1OL7olSVO7Lk9UEgc8fjz9dF88LKOEPVwsXOHkjxY8 8EyCHf8Ik9HLwQYaQvf2r+Boax1CVv5e/QAf8kjlvmbB6crmUxjIh6W0aToPaFoxCe15 x4Mg== X-Gm-Message-State: AOAM533yB0vbzF/tCs1VMwpLCD8agzSjDtiO088mah7gnO1+6nH9LoyR 2hPM7OcedIR1KIvjzgZO1O6trw== X-Google-Smtp-Source: ABdhPJwRONZllplmQufLj5RnClnVIfE2Nmqatiq9pV28+ucprKnq9Or8NIypwNI4s9sGELeie9B4GA== X-Received: by 2002:ac2:443a:: with SMTP id w26mr3042148lfl.537.1628576785236; Mon, 09 Aug 2021 23:26:25 -0700 (PDT) Received: from box.localdomain ([86.57.175.117]) by smtp.gmail.com with ESMTPSA id m13sm1948755lfu.5.2021.08.09.23.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Aug 2021 23:26:22 -0700 (PDT) From: "Kirill A. Shutemov" X-Google-Original-From: "Kirill A. Shutemov" Received: by box.localdomain (Postfix, from userid 1000) id AECA5102BDB; Tue, 10 Aug 2021 09:26:34 +0300 (+03) To: Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel Cc: Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCH 4/5] x86/mm: Provide helpers for unaccepted memory Date: Tue, 10 Aug 2021 09:26:25 +0300 Message-Id: <20210810062626.1012-5-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210810062626.1012-1-kirill.shutemov@linux.intel.com> References: <20210810062626.1012-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=shutemov-name.20150623.gappssmtp.com header.s=20150623 header.b=0h4kwTTU; spf=none (imf15.hostedemail.com: domain of kirill@shutemov.name has no SPF policy when checking 209.85.167.43) smtp.mailfrom=kirill@shutemov.name; dmarc=none X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 8A1CBD004985 X-Stat-Signature: t331wxdohz3eony497za9jceqpy6dpii X-HE-Tag: 1628576786-965893 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Core-mm requires few helpers to support unaccepted memory: - accept_memory() checks the range of addresses against the bitmap and accept memory if needed; - maybe_set_page_offline() checks the bitmap and marks a page with PageOffline() if memory acceptance is required on the first allocation of the page. - clear_page_offline() accepts memory for the page and clears PageOffline(). Signed-off-by: Kirill A. Shutemov --- arch/x86/boot/compressed/unaccepted_memory.c | 3 +- arch/x86/include/asm/page.h | 5 ++ arch/x86/include/asm/unaccepted_memory.h | 3 + arch/x86/mm/Makefile | 2 + arch/x86/mm/unaccepted_memory.c | 80 ++++++++++++++++++++ 5 files changed, 91 insertions(+), 2 deletions(-) create mode 100644 arch/x86/mm/unaccepted_memory.c diff --git a/arch/x86/boot/compressed/unaccepted_memory.c b/arch/x86/boot/compressed/unaccepted_memory.c index 17b70627b0cd..818d32169eef 100644 --- a/arch/x86/boot/compressed/unaccepted_memory.c +++ b/arch/x86/boot/compressed/unaccepted_memory.c @@ -13,8 +13,7 @@ void mark_unaccepted(struct boot_params *params, u64 start, u64 num) unsigned int npages; if ((start & PMD_MASK) == (end & PMD_MASK)) { - npages = (end - start) / PAGE_SIZE; - __accept_memory(start, start + npages * PAGE_SIZE); + __accept_memory(start, end); return; } diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 4d5810c8fab7..1e56d76ca474 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -19,6 +19,11 @@ struct page; #include + +#ifdef CONFIG_UNACCEPTED_MEMORY +#include +#endif + extern struct range pfn_mapped[]; extern int nr_pfn_mapped; diff --git a/arch/x86/include/asm/unaccepted_memory.h b/arch/x86/include/asm/unaccepted_memory.h index f1f835d3cd78..712128760131 100644 --- a/arch/x86/include/asm/unaccepted_memory.h +++ b/arch/x86/include/asm/unaccepted_memory.h @@ -6,9 +6,12 @@ #include struct boot_params; +struct page; void mark_unaccepted(struct boot_params *params, u64 start, u64 num); void accept_memory(phys_addr_t start, phys_addr_t end); +void maybe_set_page_offline(struct page *page, unsigned int order); +void clear_page_offline(struct page *page, unsigned int order); #endif diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index b31cb52bf1bd..fe4e16322868 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -57,3 +57,5 @@ obj-$(CONFIG_X86_MEM_ENCRYPT_COMMON) += mem_encrypt_common.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o + +obj-$(CONFIG_UNACCEPTED_MEMORY) += unaccepted_memory.o diff --git a/arch/x86/mm/unaccepted_memory.c b/arch/x86/mm/unaccepted_memory.c new file mode 100644 index 000000000000..e11933f62ead --- /dev/null +++ b/arch/x86/mm/unaccepted_memory.c @@ -0,0 +1,80 @@ +#include +#include +#include +#include + +#include +#include +#include + +static DEFINE_SPINLOCK(unaccepted_memory_lock); + +#define PMD_ORDER (PMD_SHIFT - PAGE_SHIFT) + +static void __accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long *unaccepted_memory; + unsigned int rs, re; + + unaccepted_memory = __va(boot_params.unaccepted_memory); + bitmap_for_each_set_region(unaccepted_memory, rs, re, + start / PMD_SIZE, + DIV_ROUND_UP(end, PMD_SIZE)) { + /* Platform-specific memory-acceptance call goes here */ + panic("Cannot accept memory"); + bitmap_clear(unaccepted_memory, rs, re - rs); + } +} + +void accept_memory(phys_addr_t start, phys_addr_t end) +{ + if (!boot_params.unaccepted_memory) + return; + + spin_lock(&unaccepted_memory_lock); + __accept_memory(start, end); + spin_unlock(&unaccepted_memory_lock); +} + +void __init maybe_set_page_offline(struct page *page, unsigned int order) +{ + unsigned long *unaccepted_memory; + phys_addr_t addr = page_to_phys(page); + bool unaccepted = true; + unsigned int i; + + if (!boot_params.unaccepted_memory) + return; + + unaccepted_memory = __va(boot_params.unaccepted_memory); + spin_lock(&unaccepted_memory_lock); + if (order < PMD_ORDER) { + BUG_ON(test_bit(addr / PMD_SIZE, unaccepted_memory)); + goto out; + } + + for (i = 0; i < (1 << (order - PMD_ORDER)); i++) { + if (!test_bit(addr / PMD_SIZE + i, unaccepted_memory)) { + unaccepted = false; + break; + } + } + + if (unaccepted) + __SetPageOffline(page); + else + __accept_memory(addr, addr + (PAGE_SIZE << order)); +out: + spin_unlock(&unaccepted_memory_lock); +} + +void clear_page_offline(struct page *page, unsigned int order) +{ + phys_addr_t addr = page_to_phys(page); + + /* PageOffline() page on a free list, but no unaccepted memory? Hm. */ + WARN_ON_ONCE(!boot_params.unaccepted_memory); + + accept_memory(addr, addr + (PAGE_SIZE << order)); + __ClearPageOffline(page); +}