From patchwork Thu Mar 30 11:49:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "kirill.shutemov@linux.intel.com" X-Patchwork-Id: 13194075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40FFEC77B60 for ; Thu, 30 Mar 2023 11:50:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C771F6B0081; Thu, 30 Mar 2023 07:50:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4DAD6B0085; Thu, 30 Mar 2023 07:50:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DED26B0081; Thu, 30 Mar 2023 07:50:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8602D6B0081 for ; Thu, 30 Mar 2023 07:50:39 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F159A120918 for ; Thu, 30 Mar 2023 11:50:38 +0000 (UTC) X-FDA: 80625397356.11.244EAA3 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by imf16.hostedemail.com (Postfix) with ESMTP id B5956180004 for ; Thu, 30 Mar 2023 11:50:36 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SrIoRkNm; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf16.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 134.134.136.24) smtp.mailfrom=kirill.shutemov@linux.intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680177037; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8Nf7y2H48SvI7vjZmdAL4egfrBNrBmenWjONYlXJG6k=; b=joLcPZk0fik4OvaCHt13mOb/DMlq8hxWSA2lzaPLBvTk5RDijzIruoNpNEW7ZZbQXj6d8n 5b2/uNMHTE9VaHer4WX+iKRgrQJFqkfWQ272l+wJ/bfPm1Gxgiq7DCBpp72QdocvdliPRz jFWd0g+fN2cmxGrIcULINxMBc7rYZz8= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SrIoRkNm; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf16.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 134.134.136.24) smtp.mailfrom=kirill.shutemov@linux.intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680177037; a=rsa-sha256; cv=none; b=1nGZBu0J1u81YwJfEWg7QkLleRqOXbdKun8LbPL7/rqzrvSOtodKIYZP/iAHwUxkAUF8XO cIu0F783U/U0+paCvoqWOUyraQku+Oq/tKnNAioKm+i2edw+jJ6WEmcSODP07ZOKIVHa6G fbX2o6HTIoeSdFd+e+rGihsRQUKO9hI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680177036; x=1711713036; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wDfbjNCC+5BxiTtoxByvPaljGaBvwiQ8TKW6aMXMCQY=; b=SrIoRkNmkPD0Ykjq5pwDa5A4YFAPkGmUS/Ruld7OZgmKvNsOAjDpfyAa DaaS9fzxpqmUtf+ix1kS/OcP7ucP246S7Y2RmvfZP6e/s4KkfBuSp9Aek wL/a4v9oBEpQVjdDapC/k7I5yzexu1hiCAgiiHJU4hhR5SC6XoXqLyTiL +v+UYpmU8aZOPllIUsPk/8u5vQyhlmyyBgsrHxpNwCOiuiUo9sDQqm5zJ BHqdALWYILlXBppeNUTvjQHM5jdwiy9FWBH31Bkv8jc3bZVSiepa0cddw d1X/8yZ3JKC+QdK2DiEzspWrMUvLofEnCgMzTL+XJVJI9JbhO5uA7+yYy A==; X-IronPort-AV: E=McAfee;i="6600,9927,10664"; a="342756804" X-IronPort-AV: E=Sophos;i="5.98,303,1673942400"; d="scan'208";a="342756804" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2023 04:50:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10664"; a="634856514" X-IronPort-AV: E=Sophos;i="5.98,303,1673942400"; d="scan'208";a="634856514" Received: from ngreburx-mobl.ger.corp.intel.com (HELO box.shutemov.name) ([10.251.209.91]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2023 04:50:18 -0700 Received: by box.shutemov.name (Postfix, from userid 1000) id 60182104788; Thu, 30 Mar 2023 14:50:00 +0300 (+03) From: "Kirill A. Shutemov" To: Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel , Ard Biesheuvel Cc: Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Dario Faggioli , Dave Hansen , Mike Rapoport , David Hildenbrand , Mel Gorman , marcelo.cerri@canonical.com, tim.gardner@canonical.com, khalid.elmously@canonical.com, philip.cox@canonical.com, aarcange@redhat.com, peterx@redhat.com, x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" , Dave Hansen Subject: [PATCHv9 11/14] x86/mm: Avoid load_unaligned_zeropad() stepping into unaccepted memory Date: Thu, 30 Mar 2023 14:49:53 +0300 Message-Id: <20230330114956.20342-12-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330114956.20342-1-kirill.shutemov@linux.intel.com> References: <20230330114956.20342-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B5956180004 X-Stat-Signature: 9gokuxspwiyhxbtscyxetznmq7hzh8x7 X-HE-Tag: 1680177036-710587 X-HE-Meta: U2FsdGVkX19HDb5gOSdUakQiIoVKiCwtiNMAZe6yhXacrxlwMV1AxYzloynzM2THOmeGgaB3oxEbQmIqtp7KBk7pJODPagEdmFY7Y0qXgEd/Lht/MtbO2SdKaysPSXKxzOLJ+9feXIQCKbqcg4dovJbJn2O0akKIf8UIhDlFkNHqHYHhwhVgnm8fuqC5hfKCvT+KVFY0wnwDHdh541Cwo+2mvBXldfqWYwJOuhDbxW+DBef9LIadgTpUYy5rE7sbbXUrRX7caadoSofheyNOQ6zdJEscqu4HZ3kDP4XVq+rxNISsdTM6ZdksyG8tfMiKXO/I9FWSc7TSw2ajt8SgvSy+buP3seO2H1e41xUgmykCNecSvR0niJQIiOiYJsw6cmnsR/WMrfiIkhN+7Q+66qIrg413WvE3kt1OeEAy/XMcAvBjTd28hrHvBRyL3cSzPT2NMJqfKzSy8bGwuppwDcd11nT1cYnsBmUB9qGRnpSSVsBOoVOX3IHwbYadolaR/MJpqa/Whele4dFqvgmkJZ76G23F12QRjZqPA9ijx4WslF5z0K0ZEVHckIN9vvJ+UKGe6Rbo5fNq5fZ76pS/ZnaRV6h0ms/KwgaRdiVsqouTdae6X+1ifLNoIp01Aq9zx0tYQ1ZZC7LDdtUDtTrh+HUrFibe1yNJ03X639kHmxH9IfKybvLmfJAjP8wImTFp5ZcYL/fs18Odrt5IBBMmJnKPKg7PmqRuGT72J7pRk5fETjmYis00tI1Eq/GOCadcRb8bBje5g7uz1NmGkPr7HUd83d4UO93xUx2XgtykD3INV9qs0diQSjemlE6e3C0iAhzfdrGG1o1AATG/f7bB2Wf4T+5FfVyXc276z54xdfq7k8Qdx8RLBDDJi824oIScLP6aBxPb7XdyhgpnM1dRo1eb6srnpFOIvwuDsFwYfHIOeJJAd/eJdGV/3ACwHRXNheYgM+B4kYrv58CqpNx oyr1ogrc 2uJKrT+lWQWnfIE2GwX2Ses/rZ290h2kJbkM6mSACkNIJ8p1MG5AJ470D6+8c5D7bm4RxOIYwgvWTbsbd1o0VdzLbXkAz/E2zRDdA+VaWhjxkWtanlUJDIf6SxXCqpojsXufCQGCvz92JXQS03/ZgvaTsKe220G+UF/ZUpduSGqumUs1nh+HKJVzJLrxq5Qwo1uK9I2DIL3dWPcc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: load_unaligned_zeropad() can lead to unwanted loads across page boundaries. The unwanted loads are typically harmless. But, they might be made to totally unrelated or even unmapped memory. load_unaligned_zeropad() relies on exception fixup (#PF, #GP and now #VE) to recover from these unwanted loads. But, this approach does not work for unaccepted memory. For TDX, a load from unaccepted memory will not lead to a recoverable exception within the guest. The guest will exit to the VMM where the only recourse is to terminate the guest. There are three parts to fix this issue and comprehensively avoid access to unaccepted memory. Together these ensure that an extra "guard" page is accepted in addition to the memory that needs to be used. 1. Implicitly extend the range_contains_unaccepted_memory(start, end) checks up to end+2M if 'end' is aligned on a 2M boundary. It may require checking 2M chunk beyond end of RAM. The bitmap allocation is modified to accommodate this. 2. Implicitly extend accept_memory(start, end) to end+2M if 'end' is aligned on a 2M boundary. 3. Set PageUnaccepted() on both memory that itself needs to be accepted *and* memory where the next page needs to be accepted. Essentially, make PageUnaccepted(page) a marker for whether work needs to be done to make 'page' usable. That work might include accepting pages in addition to 'page' itself. Side note: This leads to something strange. Pages which were accepted at boot, marked by the firmware as accepted and will never _need_ to be accepted might have PageUnaccepted() set on them. PageUnaccepted(page) is a cue to ensure that the next page is accepted before 'page' can be used. This is an actual, real-world problem which was discovered during TDX testing. Signed-off-by: Kirill A. Shutemov Reviewed-by: Dave Hansen --- arch/x86/mm/unaccepted_memory.c | 39 +++++++++++++++++++++++++ drivers/firmware/efi/libstub/x86-stub.c | 7 +++++ 2 files changed, 46 insertions(+) diff --git a/arch/x86/mm/unaccepted_memory.c b/arch/x86/mm/unaccepted_memory.c index 1df918b21469..a0a58486eb74 100644 --- a/arch/x86/mm/unaccepted_memory.c +++ b/arch/x86/mm/unaccepted_memory.c @@ -23,6 +23,38 @@ void accept_memory(phys_addr_t start, phys_addr_t end) bitmap = __va(boot_params.unaccepted_memory); range_start = start / PMD_SIZE; + /* + * load_unaligned_zeropad() can lead to unwanted loads across page + * boundaries. The unwanted loads are typically harmless. But, they + * might be made to totally unrelated or even unmapped memory. + * load_unaligned_zeropad() relies on exception fixup (#PF, #GP and now + * #VE) to recover from these unwanted loads. + * + * But, this approach does not work for unaccepted memory. For TDX, a + * load from unaccepted memory will not lead to a recoverable exception + * within the guest. The guest will exit to the VMM where the only + * recourse is to terminate the guest. + * + * There are three parts to fix this issue and comprehensively avoid + * access to unaccepted memory. Together these ensure that an extra + * "guard" page is accepted in addition to the memory that needs to be + * used: + * + * 1. Implicitly extend the range_contains_unaccepted_memory(start, end) + * checks up to end+2M if 'end' is aligned on a 2M boundary. + * + * 2. Implicitly extend accept_memory(start, end) to end+2M if 'end' is + * aligned on a 2M boundary. (immediately following this comment) + * + * 3. Set PageUnaccepted() on both memory that itself needs to be + * accepted *and* memory where the next page needs to be accepted. + * Essentially, make PageUnaccepted(page) a marker for whether work + * needs to be done to make 'page' usable. That work might include + * accepting pages in addition to 'page' itself. + */ + if (!(end % PMD_SIZE)) + end += PMD_SIZE; + spin_lock_irqsave(&unaccepted_memory_lock, flags); for_each_set_bitrange_from(range_start, range_end, bitmap, DIV_ROUND_UP(end, PMD_SIZE)) { @@ -46,6 +78,13 @@ bool range_contains_unaccepted_memory(phys_addr_t start, phys_addr_t end) bitmap = __va(boot_params.unaccepted_memory); + /* + * Also consider the unaccepted state of the *next* page. See fix #1 in + * the comment on load_unaligned_zeropad() in accept_memory(). + */ + if (!(end % PMD_SIZE)) + end += PMD_SIZE; + spin_lock_irqsave(&unaccepted_memory_lock, flags); while (start < end) { if (test_bit(start / PMD_SIZE, bitmap)) { diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 1643ddbde249..1afe7b5b02e1 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -715,6 +715,13 @@ static efi_status_t allocate_unaccepted_bitmap(struct boot_params *params, return EFI_SUCCESS; } + /* + * range_contains_unaccepted_memory() may need to check one 2M chunk + * beyond the end of RAM to deal with load_unaligned_zeropad(). Make + * sure that the bitmap is large enough handle it. + */ + max_addr += PMD_SIZE; + /* * If unaccepted memory is present, allocate a bitmap to track what * memory has to be accepted before access.