From patchwork Wed Sep 27 15:13:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 9974233 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2CBBC6037F for ; Wed, 27 Sep 2017 15:15:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BF5729209 for ; Wed, 27 Sep 2017 15:15:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0E7632920C; Wed, 27 Sep 2017 15:15:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 22FB729209 for ; Wed, 27 Sep 2017 15:14:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753406AbdI0PO0 (ORCPT ); Wed, 27 Sep 2017 11:14:26 -0400 Received: from mail-by2nam03on0084.outbound.protection.outlook.com ([104.47.42.84]:59616 "EHLO NAM03-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753179AbdI0POP (ORCPT ); Wed, 27 Sep 2017 11:14:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=gvf0WZnwfDx1P8/xSqwYStV8BmrjywwBRjWJ40WcOBI=; b=YnBTza/y059M5EztuXUtCLf7jzujr2HPEM0Fh7SSLYuJJ/lQTWMC2v8E8rGQCUcozx7hqDq6rqzUAveLNxnAok/4+pACn7Ui+67m50nr6S01cijGH+Q09cID9kUXE5OfsK1jabeF60gI50h7V/WkgmBZl1g7vZQvyqqTOwQOPKI= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from ubuntu-010236106000.amd.com (165.204.78.1) by CY1PR12MB0152.namprd12.prod.outlook.com (10.161.173.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Wed, 27 Sep 2017 15:14:04 +0000 From: Brijesh Singh To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Tom Lendacky , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Konrad Rzeszutek Wilk , "Kirill A. Shutemov" , Laura Abbott , Andy Lutomirski , Kees Cook , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Brijesh Singh Subject: [Part1 PATCH v5 12/17] x86/boot: Add early boot support when running with SEV active Date: Wed, 27 Sep 2017 10:13:24 -0500 Message-Id: <20170927151329.70011-13-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20170927151329.70011-1-brijesh.singh@amd.com> References: <20170927151329.70011-1-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: BN6PR04CA0033.namprd04.prod.outlook.com (10.174.93.150) To CY1PR12MB0152.namprd12.prod.outlook.com (10.161.173.22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 13cbe6d1-1f54-4a92-b530-08d505ba6525 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:CY1PR12MB0152; X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0152; 3:hIseDMMM+26DNKdp1l0C8i+BiDRFlPpDtJw7Z/BeQE0p2ikOi/FoYYvOLiIXQ78byqk35kRrH3Blu9k4WZ39RouCVGRecmYgHrH9CtGuiiZVrdYdaO/tZhIvb74ADGdBiouxRIirk4t8KtKSZmQbZp7T/IhCd45yqFx2zeenQuwS1YViQxlZVN/J27YFIYC+HtUp55QNMAdhzFgSLgGE8lVEsBwcgW0E8RuIot5YVOOVMG8q8GkVObeT6TxlH4Mw; 25:RCRA9krYFtjYt8H/nmdsWdq75SxzQIPb4+Wk4ixGy9YreKNoRsExFnTmPi7+uzqIhvHg/ch0resRExK2ERwqhh+phh3kZKgC7/YmVrxTjpa/7CWkP5nufauS/azSSXnNDQCVTI3H6KcW3mivf9wCV6POSqob4acfOolTuRxWsLhsYF90oFWFw0i5B5Rf6j+AnKZQK7b+8AtBJzLdfgUca/HyXwa4AD/jI2iBM7aWE+uwPN19syYx8f9VYzkHX0xtP8TTh8BbaCsfhDVCxf/zthEuiH+RBHUlZlePrybjHNpuXqvQ/u+wzCZtGqzIDg/YCFDw9OmH9iuW9ys7b3QQFg==; 31:aYZid2etXvWG940QaUNc4WZghXDiSnZYYFyVd7hwH2HF48fOVsASg2h80aqYikJGLMapsWMLpPGcpPWYc+t/YOAJnCL/JY5zWU9MEYAg8yXPWduWrjs0TPhv6fV+94w0qIvXxZV34Q5r6fHmrUZLgNlDsX5U2BzaUJ30UURasJox2QULJeC4+wJv5DACoRvfI6rV77YrVmeXwWEpqNCUVwxqXx6M1m/QeUJ8QKAsBEM= X-MS-TrafficTypeDiagnostic: CY1PR12MB0152: X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0152; 20:+9A9k1cNp8jJ4oHPWAsluGVhpJy/Hz7jgwHuCB/mxAlZh6c6c+OMzkHdZ5Fr0WIJSY1t15iji478Nqjh9RLOkX/3OIQCiZxFaqzJxDGo7+LS8aKceb9FqcEUFNWevV9dJIOc1QtLb833wuW1IbYe7zQKfzDM0E60YGBnGPzPsxJZ6+y40z7g0oTICWFS8SW8pUtP72RXufQE/KoxxXgfKYURMWVxAW5V8pFIIYYowbOYoCsyAuqIxy1yENC66E0eDi0iKzPJ7Dfg5CWcymLCa2eRQn9GGWdmv3aXt63reWm4N4jW2EigXk1BVD9joIRu7TSrPEJzYXB/+2NjZ2jsZQlgq0/5quPd5xR2T1w6V1iEdq1BuGydW4kgZmbpima6k1Ig7ii0CIY89krxI3IgzV90wbp2r29mD5SEcXIk8JXOeJAxeQrFf78F8xMwJUQdxFFSAstnzxBrTD22I4zcDzuR06lakiLRYNcXKTn/czHH529LCMZeJYvaYGj43a+/; 4:NzAD7bEc9atAQUv7mR3zDCfPTVrFQIzmCRPUKAe5ZwEdsFpUKoSYHLEj7/7lz7MUFg3xJhg6GqIH+M1qQaXxzZbrE2qVoJQqoCUn0N/if2fxzZJA6K+9ihbgEOjjTv5shoHeH5xXDQ/7lSh0AgyS92GSp9RbFeBCuABuLZJtwID5G81FaCDBvOotNy8OU2yWT2fN0+WVaSK4BOc1jlf6xe/WNlF8P17GMweVAeUwXf4v8jBaXID77Mi8/SvJQIpQs1wSQGu1Fc31ecsEFHyK5kARhsGoAWsewNGkppqzUW9WZA+s4HAm5JTFfATnBFAtXwyag7ztcK3uHBS+Ebvy66qTYQoh/5UekrqZWZmAK9mH/uO5P7QWkhfkHamxv1mfuIb2Tj+JJC1uqXy3eVuceA== X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(146099531331640)(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93001095)(100000703101)(100105400095)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY1PR12MB0152; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY1PR12MB0152; X-Forefront-PRVS: 04433051BF X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(39860400002)(199003)(189002)(3846002)(25786009)(6116002)(23676002)(53416004)(106356001)(8936002)(4326008)(54906003)(1076002)(6486002)(50226002)(8676002)(305945005)(50466002)(81156014)(81166006)(6666003)(68736007)(36756003)(189998001)(7736002)(105586002)(97736004)(2870700001)(53936002)(7416002)(2950100002)(101416001)(478600001)(16526017)(76176999)(50986999)(316002)(47776003)(66066001)(86362001)(33646002)(5660300001)(575784001)(2906002)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR12MB0152; H:ubuntu-010236106000.amd.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTFQUjEyTUIwMTUyOzIzOk02ME9vd2xOU1ZhbnNQempDcjUwSUxjY0F0?= =?utf-8?B?djZhVnR1ZVJacFV6T1pwb09GVEhmMUt3UERHMVhGcHVGemNMYlFGbnNaQk5m?= =?utf-8?B?aThxcnZFRVZhalZVSlB3ZEhidVpxUHJGVGlFdSsxRnkzK0dqalEyZDBsZzdP?= =?utf-8?B?cVlPMG51VWZRd0RUNWJOcVBRTVBDQnpkdU1hWFdYbU1XM0xHOEJzQ0phWnBl?= =?utf-8?B?RUY4SzZWY0JJa1hDSVMwRFhhVVJML2QwZmJnY0YvcTBhUnMyOEJwd25OVUdE?= =?utf-8?B?SUNublFGeER0ajI2dkJjQ2VacllmL3ZVWVFiTktiaytyamhDV2cxbUkxd0V6?= =?utf-8?B?NkZMS1A3N0d1b3lCeDd0dGVPeUVMQVoyME1DRUNZeHIzWFZ6WERpenZIbHF3?= =?utf-8?B?TEdqWS95REpkQXBuMVNwTmZSamxla1lKMU5wYkR5Rmlaa0xIUmVzeHd5S0xF?= =?utf-8?B?eDI4anJTbWNlZHhiYy9NVkJnY0hXWWh1endjdnM4b1hkWnB6Vjd4M2RhNk12?= =?utf-8?B?ZitteklBeTZSVGhWTHhveElEOUJFZXo4dnZlUWZGNWZMUHE1elF0dWl3RW1u?= =?utf-8?B?TmpJeE1nU3o4eWFNTEEvb1F6NThRZ2lDMEU0RW9sT0dWRllXeENhVFV5ejI1?= =?utf-8?B?Mlh6bU1NRGxtTmFKcmxnMHFHU0pBbWI4WGw3WGwzYnhLRkhkV2QwQUZKZzFw?= =?utf-8?B?M3ByY0swdEhMVmM3eVAvYTR0ODNSK0ZBWDllOW9zM0ZaZ3BUUnRESTl4M05J?= =?utf-8?B?dXZWU2xUSFpORlZMMmtidm1QalNzUFBXbVFVWmo2VnhTUms5K20vSVhIZ1li?= =?utf-8?B?Q0I1YzRiS0dJcG5uVlBLVzR5Z0k4aUtqMkdpV0FqZUJQUTIrTml3Y2toREpy?= =?utf-8?B?Z0pqWmNzcmN4YUduMWcyaitiaHFydVpBK296UVc4TTA3anRiVVVjRlk3Zzdi?= =?utf-8?B?UnVNKzlIdHJOcVM0VlZYazNCbklBdFpnKzZTRTlEMm1Nd2Qzb3BTNFRjcTlP?= =?utf-8?B?aWpRSVU2WmpYdWlrRjZaVWxHYjU3c1NGRE94REY0SFBVdmJZOG4zbGdBRWla?= =?utf-8?B?aW5veS9MUWZvSGphK0cza282M010d0Y2N042ODlyYmFiVnV6RVhOUDcwVGps?= =?utf-8?B?MDcvTit6b0Uva2gvYUpFMExwUmg2eC9hYUh1UGpJZldXM2xRK2hENmtyMDBa?= =?utf-8?B?YUhiVGQ3SmE5N2JtZ1lRQjc0VGt3azhseFBVSlN0OGlJZVNPUEN1N05WTlNo?= =?utf-8?B?NDdlb2JVS3VnT2gxUi9ZV1craEc1YmM5SEhXTTI4bHlkMnRrNlBwcHVGN2lK?= =?utf-8?B?cE9KYVQ1YnNlVGZwM0NldEMwTk5FdmI2NVNPWU1paVBYdGlnSHVCZDA2cHdp?= =?utf-8?B?VlpXQTVIM1BBL0E3TVFVaFk4TUpweWJPd0FtZVRFQjV5YndBUFJZRUlESTJD?= =?utf-8?B?Y3VDTU5KM1BCaGQvUzlnWmd0c1dSbUU0T3JSeWg3WlZESmIveVRZY2NFYTJE?= =?utf-8?Q?42oon+LVBNDxsyGxw9uQvQiiY=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0152; 6:uLIFP9UoHNBln5jlg1jFxOIeAXd7vOQA3np6KH7K5TUdQr5c9vtlusbQ8iRZcDn0n1F0ITkAUlBHcg2D7YxqF+E0Cc18EfBX6wlfvozURq1EpTvsdDr7RCcUjbojCNptgto848iUN+CL3nAvMDLYCgEy5nEP9yaxxVnTgJhc3jZDiLXrd5OuuTTdycKEPo9BhsAkCag1psHAnkzLyeXeWbm6gOoN1UNm56bGgM4+wfHMsF4RPlgHL/ybjYg/Wjv3VtXerb/jfGT1BgrCiWqrEqkkHQRJqheNvYs6GTUoCinSSS5QzrIkIQMgcDSULoYPA8UZCgu23rgJb2lQZMrSBg==; 5:33EdWPDmi5R3Vk4TNwqA2zXhDoSDVBell07uEznth0uhybIxlUcVWI7ojecJvAhJFKLLzBu1VQsgoBO49m2NUuMlb6EE8ebZOs4/4QqO4OKEoQD9PC1X1LZhaeoN4Z+xvV0chv8IpTNGDjRhEBxqFg==; 24:S4gI2ag4Mk3c4McuHtLcOm/t+KFpl3XvxjB5wW8717ddsFNjNcD71SscMIbrxioo0eB53WXcmCZ7UUeWUr8JDWu4bPntaMwVpqwsx2wsVu8=; 7:dfKYzNxBzIquUCUpKAverHrnu52vIGvpjNhJL8xrg1/2MeJR7ccorK1wQ9CWZ2BIU/C+7nhHoZyT+d2mBTYdCUH28FMlGGRhbvkxRCINhOFqVzKVW0mdIw2mrHDFO8G3a8UCZOEY3DX1uQFub5a+fibTt5/XajeDQddozZI8IJ+m9cWAKQ/DLaGWdjnzqBZzKfVPX2lpl/ZzMujJIFaGDLAwaOUAGcqkbt4uEPyXYTE= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0152; 20:BMDPXQyojWrOlLQ2L3R1qjALZDqy1sfla1P6nJ8/Se54HuBamNwEj9klOuqJlAaSz6MoMLk1hk+DSQYcEOxn8RltETWhT46KzCOSgIDktkv4O5n9lH6KlNfzn2isvcFca610EK+B5ptwoUZu6AF2BT0i48+FCSNq0S7jeNURtyk0UidmzF8YCVR6UsNqpK5OCEPb95pwzfjs0sWsau5aOma35IYvoSD9QrgDwRukoyDd2x5wxruPJbXYfjkhVFCh X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2017 15:14:04.5256 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB0152 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tom Lendacky Early in the boot process, add checks to determine if the kernel is running with Secure Encrypted Virtualization (SEV) active. Checking for SEV requires checking that the kernel is running under a hypervisor (CPUID 0x00000001, bit 31), that the SEV feature is available (CPUID 0x8000001f, bit 1) and then checking a non-interceptable SEV MSR (0xc0010131, bit 0). This check is required so that during early compressed kernel booting the pagetables (both the boot pagetables and KASLR pagetables (if enabled) are updated to include the encryption mask so that when the kernel is decompressed into encrypted memory, it can boot properly. After the kernel is decompressed and continues booting the same logic is used to check if SEV is active and set a flag indicating so. This allows us to distinguish between SME and SEV, each of which have unique differences in how certain things are handled: e.g. DMA (always bounce buffered with SEV) or EFI tables (always access decrypted with SME). Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Borislav Petkov Cc: Konrad Rzeszutek Wilk Cc: "Kirill A. Shutemov" Cc: Laura Abbott Cc: Andy Lutomirski Cc: Kees Cook Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Tom Lendacky Signed-off-by: Brijesh Singh Reviewed-by: Borislav Petkov --- arch/x86/boot/compressed/Makefile | 1 + arch/x86/boot/compressed/head_64.S | 16 +++++ arch/x86/boot/compressed/mem_encrypt.S | 120 +++++++++++++++++++++++++++++++++ arch/x86/boot/compressed/misc.h | 2 + arch/x86/boot/compressed/pagetable.c | 8 ++- arch/x86/include/asm/msr-index.h | 3 + arch/x86/include/uapi/asm/kvm_para.h | 1 - arch/x86/mm/mem_encrypt.c | 50 +++++++++++--- 8 files changed, 186 insertions(+), 15 deletions(-) create mode 100644 arch/x86/boot/compressed/mem_encrypt.S diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 8a958274b54c..7fc5b7168e4f 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -77,6 +77,7 @@ vmlinux-objs-$(CONFIG_EARLY_PRINTK) += $(obj)/early_serial_console.o vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/kaslr.o ifdef CONFIG_X86_64 vmlinux-objs-$(CONFIG_RANDOMIZE_BASE) += $(obj)/pagetable.o + vmlinux-objs-y += $(obj)/mem_encrypt.o endif $(obj)/eboot.o: KBUILD_CFLAGS += -fshort-wchar -mno-red-zone diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index b4a5d284391c..3dfad60720d0 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -130,6 +130,19 @@ ENTRY(startup_32) /* * Build early 4G boot pagetable */ + /* + * If SEV is active then set the encryption mask in the page tables. + * This will insure that when the kernel is copied and decompressed + * it will be done so encrypted. + */ + call get_sev_encryption_bit + xorl %edx, %edx + testl %eax, %eax + jz 1f + subl $32, %eax /* Encryption bit is always above bit 31 */ + bts %eax, %edx /* Set encryption mask for page tables */ +1: + /* Initialize Page tables to 0 */ leal pgtable(%ebx), %edi xorl %eax, %eax @@ -140,12 +153,14 @@ ENTRY(startup_32) leal pgtable + 0(%ebx), %edi leal 0x1007 (%edi), %eax movl %eax, 0(%edi) + addl %edx, 4(%edi) /* Build Level 3 */ leal pgtable + 0x1000(%ebx), %edi leal 0x1007(%edi), %eax movl $4, %ecx 1: movl %eax, 0x00(%edi) + addl %edx, 0x04(%edi) addl $0x00001000, %eax addl $8, %edi decl %ecx @@ -156,6 +171,7 @@ ENTRY(startup_32) movl $0x00000183, %eax movl $2048, %ecx 1: movl %eax, 0(%edi) + addl %edx, 4(%edi) addl $0x00200000, %eax addl $8, %edi decl %ecx diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S new file mode 100644 index 000000000000..54f5f6625a73 --- /dev/null +++ b/arch/x86/boot/compressed/mem_encrypt.S @@ -0,0 +1,120 @@ +/* + * AMD Memory Encryption Support + * + * Copyright (C) 2017 Advanced Micro Devices, Inc. + * + * Author: Tom Lendacky + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include + +#include +#include +#include + + .text + .code32 +ENTRY(get_sev_encryption_bit) + xor %eax, %eax + +#ifdef CONFIG_AMD_MEM_ENCRYPT + push %ebx + push %ecx + push %edx + push %edi + + /* + * RIP-relative addressing is needed to access the encryption bit + * variable. Since we are running in 32-bit mode we need this call/pop + * sequence to get the proper relative addressing. + */ + call 1f +1: popl %edi + subl $1b, %edi + + movl enc_bit(%edi), %eax + cmpl $0, %eax + jge .Lsev_exit + + /* Check if running under a hypervisor */ + movl $1, %eax + cpuid + bt $31, %ecx /* Check the hypervisor bit */ + jnc .Lno_sev + + movl $0x80000000, %eax /* CPUID to check the highest leaf */ + cpuid + cmpl $0x8000001f, %eax /* See if 0x8000001f is available */ + jb .Lno_sev + + /* + * Check for the SEV feature: + * CPUID Fn8000_001F[EAX] - Bit 1 + * CPUID Fn8000_001F[EBX] - Bits 5:0 + * Pagetable bit position used to indicate encryption + */ + movl $0x8000001f, %eax + cpuid + bt $1, %eax /* Check if SEV is available */ + jnc .Lno_sev + + movl $MSR_AMD64_SEV, %ecx /* Read the SEV MSR */ + rdmsr + bt $MSR_AMD64_SEV_ENABLED_BIT, %eax /* Check if SEV is active */ + jnc .Lno_sev + + movl %ebx, %eax + andl $0x3f, %eax /* Return the encryption bit location */ + movl %eax, enc_bit(%edi) + jmp .Lsev_exit + +.Lno_sev: + xor %eax, %eax + movl %eax, enc_bit(%edi) + +.Lsev_exit: + pop %edi + pop %edx + pop %ecx + pop %ebx + +#endif /* CONFIG_AMD_MEM_ENCRYPT */ + + ret +ENDPROC(get_sev_encryption_bit) + + .code64 +ENTRY(get_sev_encryption_mask) + xor %rax, %rax + +#ifdef CONFIG_AMD_MEM_ENCRYPT + push %rbp + push %rdx + + movq %rsp, %rbp /* Save current stack pointer */ + + call get_sev_encryption_bit /* Get the encryption bit position */ + testl %eax, %eax + jz .Lno_sev_mask + + xor %rdx, %rdx + bts %rax, %rdx /* Create the encryption mask */ + mov %rdx, %rax /* ... and return it */ + +.Lno_sev_mask: + movq %rbp, %rsp /* Restore original stack pointer */ + + pop %rdx + pop %rbp +#endif + + ret +ENDPROC(get_sev_encryption_mask) + + .data +enc_bit: + .int 0xffffffff diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index 766a5211f827..38c5f0e13f7d 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -108,4 +108,6 @@ static inline void console_init(void) { } #endif +unsigned long get_sev_encryption_mask(void); + #endif diff --git a/arch/x86/boot/compressed/pagetable.c b/arch/x86/boot/compressed/pagetable.c index f1aa43854bed..a577329d84c5 100644 --- a/arch/x86/boot/compressed/pagetable.c +++ b/arch/x86/boot/compressed/pagetable.c @@ -76,16 +76,18 @@ static unsigned long top_level_pgt; * Mapping information structure passed to kernel_ident_mapping_init(). * Due to relocation, pointers must be assigned at run time not build time. */ -static struct x86_mapping_info mapping_info = { - .page_flag = __PAGE_KERNEL_LARGE_EXEC, -}; +static struct x86_mapping_info mapping_info; /* Locates and clears a region for a new top level page table. */ void initialize_identity_maps(void) { + unsigned long sev_me_mask = get_sev_encryption_mask(); + /* Init mapping_info with run-time function/buffer pointers. */ mapping_info.alloc_pgt_page = alloc_pgt_page; mapping_info.context = &pgt_data; + mapping_info.page_flag = __PAGE_KERNEL_LARGE_EXEC | sev_me_mask; + mapping_info.kernpg_flag = _KERNPG_TABLE | sev_me_mask; /* * It should be impossible for this not to already be true, diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 17f5c12e1afd..4a4e7d72b565 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -323,6 +323,9 @@ #define MSR_AMD64_IBSBRTARGET 0xc001103b #define MSR_AMD64_IBSOPDATA4 0xc001103d #define MSR_AMD64_IBS_REG_COUNT_MAX 8 /* includes MSR_AMD64_IBSBRTARGET */ +#define MSR_AMD64_SEV 0xc0010131 +#define MSR_AMD64_SEV_ENABLED_BIT 0 +#define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT) /* Fam 17h MSRs */ #define MSR_F17H_IRPERF 0xc00000e9 diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index a965e5b0d328..c202ba3fba0f 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -109,5 +109,4 @@ struct kvm_vcpu_pv_apf_data { #define KVM_PV_EOI_ENABLED KVM_PV_EOI_MASK #define KVM_PV_EOI_DISABLED 0x0 - #endif /* _UAPI_ASM_X86_KVM_PARA_H */ diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index c301cecd7052..bc6ba4cbe9b4 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -311,7 +311,9 @@ void __init mem_encrypt_init(void) if (sev_active()) dma_ops = &sev_dma_ops; - pr_info("AMD Secure Memory Encryption (SME) active\n"); + pr_info("AMD %s active\n", + sev_active() ? "Secure Encrypted Virtualization (SEV)" + : "Secure Memory Encryption (SME)"); } void swiotlb_set_mem_attributes(void *vaddr, unsigned long size) @@ -639,37 +641,63 @@ void __init __nostackprotector sme_enable(struct boot_params *bp) { const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off; unsigned int eax, ebx, ecx, edx; + unsigned long feature_mask; bool active_by_default; unsigned long me_mask; char buffer[16]; u64 msr; - /* Check for the SME support leaf */ + /* Check for the SME/SEV support leaf */ eax = 0x80000000; ecx = 0; native_cpuid(&eax, &ebx, &ecx, &edx); if (eax < 0x8000001f) return; +#define AMD_SME_BIT BIT(0) +#define AMD_SEV_BIT BIT(1) /* - * Check for the SME feature: - * CPUID Fn8000_001F[EAX] - Bit 0 - * Secure Memory Encryption support - * CPUID Fn8000_001F[EBX] - Bits 5:0 - * Pagetable bit position used to indicate encryption + * Set the feature mask (SME or SEV) based on whether we are + * running under a hypervisor. + */ + eax = 1; + ecx = 0; + native_cpuid(&eax, &ebx, &ecx, &edx); + feature_mask = (ecx & BIT(31)) ? AMD_SEV_BIT : AMD_SME_BIT; + + /* + * Check for the SME/SEV feature: + * CPUID Fn8000_001F[EAX] + * - Bit 0 - Secure Memory Encryption support + * - Bit 1 - Secure Encrypted Virtualization support + * CPUID Fn8000_001F[EBX] + * - Bits 5:0 - Pagetable bit position used to indicate encryption */ eax = 0x8000001f; ecx = 0; native_cpuid(&eax, &ebx, &ecx, &edx); - if (!(eax & 1)) + if (!(eax & feature_mask)) return; me_mask = 1UL << (ebx & 0x3f); - /* Check if SME is enabled */ - msr = __rdmsr(MSR_K8_SYSCFG); - if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT)) + /* Check if memory encryption is enabled */ + if (feature_mask == AMD_SME_BIT) { + /* For SME, check the SYSCFG MSR */ + msr = __rdmsr(MSR_K8_SYSCFG); + if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT)) + return; + } else { + /* For SEV, check the SEV MSR */ + msr = __rdmsr(MSR_AMD64_SEV); + if (!(msr & MSR_AMD64_SEV_ENABLED)) + return; + + /* SEV state cannot be controlled by a command line option */ + sme_me_mask = me_mask; + sev_enabled = true; return; + } /* * Fixups have not been applied to phys_base yet and we're running