From patchwork Tue Apr 18 21:21:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 9686401 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B0C7D602C2 for ; Tue, 18 Apr 2017 21:22:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A19142094F for ; Tue, 18 Apr 2017 21:22:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9609A25223; Tue, 18 Apr 2017 21:22:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8161252D5 for ; Tue, 18 Apr 2017 21:22:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757981AbdDRVVk (ORCPT ); Tue, 18 Apr 2017 17:21:40 -0400 Received: from mail-by2nam01on0073.outbound.protection.outlook.com ([104.47.34.73]:11936 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933014AbdDRVVY (ORCPT ); Tue, 18 Apr 2017 17:21:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hlQWsZI+E4D83Iy3JHhsGCmozA+C9BrPwhPwpMk39sw=; b=iIDct7VN0F7N+9Ixt9n2vxS2YSuNcc8M7eouei+YKlv7A2gr3Xkyalt4/HHu++PGNk79n1X0vDkkxTFLYmOsBZAWsaYauGIBuyiHxSxjZZPTt1H2kprH/uU1g6CKVWgyKRw6oC3I2MrIwLNPv5dPz2YJXxixc6fgJ5gbp6CSdGc= Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=amd.com; Received: from tlendack-t1.amdoffice.net (165.204.77.1) by BN6PR12MB1140.namprd12.prod.outlook.com (10.168.226.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1034.10; Tue, 18 Apr 2017 21:21:13 +0000 From: Tom Lendacky Subject: [PATCH v5 27/32] kvm: x86: svm: Enable Secure Memory Encryption within KVM To: , , , , , , , , , CC: Rik van Riel , Radim =?utf-8?b?S3LEjW3DocWZ?= , Toshimitsu Kani , Arnd Bergmann , Jonathan Corbet , Matt Fleming , "Michael S. Tsirkin" , Joerg Roedel , Konrad Rzeszutek Wilk , Paolo Bonzini , Larry Woodman , Brijesh Singh , Ingo Molnar , Borislav Petkov , Andy Lutomirski , "H. Peter Anvin" , Andrey Ryabinin , Alexander Potapenko , Dave Young , Thomas Gleixner , Dmitry Vyukov Date: Tue, 18 Apr 2017 16:21:10 -0500 Message-ID: <20170418212110.10190.47799.stgit@tlendack-t1.amdoffice.net> In-Reply-To: <20170418211612.10190.82788.stgit@tlendack-t1.amdoffice.net> References: <20170418211612.10190.82788.stgit@tlendack-t1.amdoffice.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: CY4PR20CA0038.namprd20.prod.outlook.com (10.172.113.152) To BN6PR12MB1140.namprd12.prod.outlook.com (10.168.226.142) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 358d0c70-c534-43ad-8d91-08d486a0d925 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081)(201703131423075)(201703031133081); SRVR:BN6PR12MB1140; X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1140; 3:SdaiHq4HSUmi32TDg6+N66gyQ5Fa/KCa6DoRJhFttp1C0YAJT1fEySb1HeD1JrZij60bvbLKE2FuSNYH1YE8yeq66I+E6JmhdQMcVzNaluS8fW3s+XvW0xMaUXBj9zNGPKwQpQJILSIqHI/hExzePqpj65xWNgjy96VqQn5GXXclYVeT/oWAT/PJn6J5vv1Ycdr2x/BeYNK6b30zNwfxl/yycX0Y6A2VyCa/srXsTY9EeQ4M+r0GXkRKZV0DWvtxOwXFL/HFD84QhzSZJPSEcIfY8stsQkF5CWgqLiHHolGfR70WYRAlFqVatCKdYt9HXh0U577gJByEOe1AVUVWuZxAGve9kBOdqakx6jS/w54=; 25:wOZnViaIfpHBK0s9rz0Aw7ufuCZhAgoArS8E0oRAcqB4cSEF6fF065N6ldvoA34489kX3/QOLVlHRb1At15T+fPzlIFp/WSUoyT+FUROjUzfO1UIOD+en1TjsRtfzMHimdbjQsVNv8S9p2X02DT2k9w8Gucj6Ne4RAxJvQRpqaXoQwehzFnTbtkb1VAfgLobB0iuX5tELyxXwE6IPhlRQppZk4jskalPcV2uaEm7fgjBMQiFF+R+Qy8rK/K+RZStg9U79Al/uVXql1p/PqPZweTgjJwQumhLpszBpqGlm5ENPHkkmCEvTy6JL0nPAk6VJMRhe59gfaOUk9+Tm6uIYJM+A7jQlDTLLM0ZEUyzrNr6uoiOvetybErj3vtcJdXh4zKfDPHQbZXBOpbhjaeXY1+cYitX4N2KnsTscqUw4T1evEfmCv83o1xv8/CwGhie X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1140; 31:SN81ObtzUNtjJMuU+f1ykY3VKWaRaBOJb7F6nyLc1SD/NYqc3dtZKEU341lVQ5Dr2zDVAwywgHUzHlZcq2f0VdseJLn+XCUAzt0OqxhexWw52qP+mAhInw7v6d2pzhBLkpxpjeLmZmLwkLJK7agB+LugbxRK4XQrBVK+XBDN3SMZc+3s0pYlyYLD00Rx9oeyHDcaEABMkJCRUaUA4iq9/701bSWMV1CxMEVYvUD09Cg=; 20:Y+ah8G4x2Kb8eXjgPnI2tqIauxbQ0kV2J6ScyoSzYV3WGDIdRMc6Ql2YLDhJyJyezeb67Xd0ZoteSJoBwGq2JUH6bH1zelX1Yb/JjIRQiDz//ed44GmLGrE1FosQWCY6JX95UMV8HsbdJba0/JFbLWPnMtpC8qa0tCleu8b4EeQQzl0anGSg4WAIUz3DnrMLzfKACByodR6BNabKy10LdfcmL3qeVd3OC2yzDXsXiHoAS38ib6AqOq+I6yH3CDHwSoEzvmVJFCOd6ukdfBfDZMvc5efs7nxJw8/t94aIEw0Dn7uZJFGffOiBpOh9CXpmZH+flgVeMWn/Ov0gavfQt8GoWV3YSZRnyKGgkKXycOV8WVMPp6qCJhKUuNvM9HDDhaMz0WCbGEf0ZVd8rQMopm9d7cY5VWjgocBgZU4uPpOsiBxgIDVT+2KT/8nOv76JCeUcBhfkWIZDHlvjur5xtIYh2d6z1jX6Of+wRnMHXAmy9OoOCuGu7oQtjhuJ3rW3 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041248)(20161123560025)(20161123555025)(20161123564025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(6072148); SRVR:BN6PR12MB1140; BCL:0; PCL:0; RULEID:; SRVR:BN6PR12MB1140; X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1140; 4:Ew67limSZSilKJrGh3x5GGfekEySlA3Sn/Bud14mLUbiKBxZP//ZXX0HTVKakeBgfZZoQDVK9gTW5uLsP7a6MvpAWjJY/odcX+dj4B1Hb80W9xS+k8hnGqlytx7LbjLR/3+TQ5QqUNqsrpK4fe2Aa6jApN79iJyDW3OlOlGYbw8ZpJwP54uhACdOfLX7cFTpzQlT4zDPDt+BCPMQrLIXbH31XtP8CioGVk3WPD9OMrNrrzJRqtb1RPPbhffYPIT/TH2BqOF5gnrnCPhR59ELv8nA2xglnHZqALxPHI1FH8F/LdnuaUYWoBW/fewUhVwrFmk9PJQScFKX0h1snLzdP1JlZ/Ad5DTbI54+gB0aLzvfXrUHnNrXdCOdlVZS+SCPx5k0n7gevSaaClvU4wQ9H20YPpEx8DFfH0TcFjQwRx7jeZx6VcNJ7nhpnZWbo9/panlw3VjX/ZuglVAsWdkmrhTRi5E1puE8vGv2ZmUuDDTN9GoyFxjJ08+4X8ehdLt0UJ6qGd8oCJi5XUXJr1FvwLIHuTcH38juRQ3EV4HSvOpkKLfAASB1kEghYh0XvQrWbRIXSQCBXTwIDiclsWGdvBufOzrd24NuM74DzwziRURIaaY1sROO8vUhgO8G0jGyV1g/9zBn4E8kAMf1qaZSHEjUd66v38C9IS0MoEbFv2GFLzn0A5HzNYRlB12LRoDAgtlo2d20uFh2fWMpfo5uTJErMeg6jkFbPyUClN/dIPpmsKMUZA19eK7A8NAZH5iV+lgHfMMx1TfmMmZgyqNrIA== X-Forefront-PRVS: 028166BF91 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39410400002)(39400400002)(39450400003)(39850400002)(39840400002)(39860400002)(189998001)(8676002)(81166006)(2906002)(4001350100001)(103116003)(50986999)(54356999)(76176999)(2950100002)(66066001)(47776003)(53936002)(23676002)(2201001)(230700001)(9686003)(54906002)(50466002)(55016002)(53416004)(4326008)(86362001)(42186005)(25786009)(1076002)(6116002)(3846002)(38730400002)(33646002)(7406005)(83506001)(305945005)(7736002)(6506006)(5660300001)(97746001)(7416002)(921003)(71626007)(1121003); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR12MB1140; H:tlendack-t1.amdoffice.net; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjZQUjEyTUIxMTQwOzIzOmhCK25kVEVtaHhyS2trRzZPVEVIZE1KbHhM?= =?utf-8?B?bldMMThDcldzenEwSFdpRXM3eEF6S3d1L0FWZzU5bXRzMkZKbzEvZm1HQjBN?= =?utf-8?B?L21jMUR2TnlTRmc1S0lzYU1XdXpHaDNJdmhkZEdhbjBMWmdjVE5MWFhxbVcz?= =?utf-8?B?ajZHVFlodmw1RDgwcGlFa3pYdm5ENFliUU5wZkNwRkx2MzNabjUrdlBkYmFr?= =?utf-8?B?SjJ1SXd5aDEvRFhPUExpaFRraFpiV3RMWXMrSFRNSkprRm9SUERWNWtYMzJy?= =?utf-8?B?am9BSFlXTkdESjZ3SDFvNVJwYmptZ2h0RkpSSVQrTjJnd3NMcXIySWlnT0hr?= =?utf-8?B?bjlRdDZmeUUyYzZzRFFrUllnL0NsR3BWY3EwNjFucG02cmxjUTNmUTVuaFlP?= =?utf-8?B?SjdYbVd4VHplVTJEN1p1UlMyTit4cGd0SFBQZ3dueWs1UnlGamI1VG40SUd4?= =?utf-8?B?a2U0Z0RwTnN6d0lidGt3TmVwMXd3OElFU0tPaTlMVXNSbnlUa1RFWUExREE5?= =?utf-8?B?Vk9PRzhUQ2J1aEs3ZTBmN0c4eFFjUW50R2NRdVIxOGN3TzY4Y24xMWlmcmpj?= =?utf-8?B?ZHRwRG1zOGs3aTlKdEJNZHBMVHRNQmJsckRYcmdLR1VYNStoeHUwbkUyWCtB?= =?utf-8?B?MjE1SmNjUllMYjVvYUMwOUlHTzJPMUNGK1J2cjdaR1M3aHhtbmkxQ2lyN0Jj?= =?utf-8?B?TjdTMFRpeGZwK0lwV1hKVkhXVTZuSEJ4UmtLT3I5WDR5SDdhYWw0RVRCT2Nj?= =?utf-8?B?SlVPQTJIc3hSSktXOW8yQ2lwYVlOQnh0eWVIbS94OTVBQXBUMXA2TTNQZ2tu?= =?utf-8?B?MEx3WkNLN3BJSy9udzRvRHV5cHY2MmY4SitOS1MzTC9UbnBFWURYS1FLSjBB?= =?utf-8?B?WEdoTWNMTlJvc24yTitwU2NlOStPaHAycVcvc2NHVlVvOWJoN3RrVmJqcXZq?= =?utf-8?B?cHM3bjM5TkwxenQxeWx0cEZrMDBaNXdPem1nNHpXdUdkYktVVkR1bjI2bUVy?= =?utf-8?B?UjFQYk1ma1hRQmxnbXgyOGpVTzhqSGhzMXh6QWNhMXFKOGFFZ0VQNytFaGp5?= =?utf-8?B?NysxSTRtQTBGSlFMb1ArbWJKMGRsVWh0RDlOOTNQU2lpVFlJNmpiWnk2TWtF?= =?utf-8?B?T3F5eEdFQ2RGUCtFdGtobWpZdHRCNElLOG5KdkdWQmNEQWJGMGJrYm5iQjhR?= =?utf-8?B?V2NneHFBSHpRdWNOd093eWwyYzhPMnJmYlhRUDJmSTNOYVVhTy92RVMxQnJL?= =?utf-8?B?bllCSS9mRWF1alFpTG1sVE5PWjdlYzZTMWl3MVBwL1hmSDhjYjZPMC9oMnBp?= =?utf-8?B?cDNubTFXbFF2a0IvQmVlTXR4SHlVbDFyME94VGtuR2czS2JKV0VLempoaXRY?= =?utf-8?B?emZRcUNFQ3F2MmsvekhxQnhCOUlLL2s3SG9XNG5nK3F5Z3FBOVV1V0c1UWVZ?= =?utf-8?B?ekx0MitqZWJoSjFnZnlVM1l2YmFmVXdnVzRnMzlES0ljS2haekViY1N1MC8w?= =?utf-8?B?ZUkrZDJrUVZ2aXZhUHcvL0lRMFhJZjBBcW0vbm5PWkdLRU55cHV1aGpJTVEr?= =?utf-8?B?bWV3UE9selhDSitRTjVKOFBxVmNJZVpaQnRyS0pQRmN3NUY1K2hrbXVrZmRU?= =?utf-8?Q?OK6RxDZXwWTN39+Mjr13?= X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1140; 6:icyXoR4l7Y9Y34WPJdze10BjnUnQt6F/MxiXxci3egrELFsqI9gZ5PBHHst8KmXO7bSdb8fa6jygrOVsiN60T44fnhiGFbUzX3XA9EKAgMY48pwpZONzUAXzrZopjub751MvmqWpQf3dvMDOW5ZwCL/tk7+4pLg5kkoBfjWAtj4wIQr6wIZEbxNEaJMxg3r4punzO58Ic6rR3hF/NxGVu00dM756/vp4Tp3fjPzrH6v9onr5VhodcnkotMr23SCC8bl2E4ZLmfi2vEXeTK0zXwu/KqfFn15mO2mF8HtibtjFWpFBn+4hSuLPQmxbZkpBBBy0Whr0x6mJAKj80lB/NG7cX8d4acAbqdkeRP+/RgNMwXzYbNFbc6qBhMRcwFLwCbolUIp8puYhtQq5oeFJCkzGJnDfF6t9+DBoXWJvjRa7wEj4gqOcPMXqvEdvoDO6WwClwnLma7LZVj+NmDydDdsZD006VtadD4av5vZhIiw=; 5:QhNBFLg++jCXe4ZGDabbnKcj8S0p9lOmLgi8BZo15Hy6RzfmTK/gD5xfJ2/CzBMc5Ru1icS+Vv4f5W2LzJBFRlVqFIXnpB4hQHJcc2+zMrw2IH/LNhPfCeCKQxjXI3A6+kaI7OtH66Wd2sRfVh911g==; 24:RdlIZgODGuFXfeJDjycBaQ4V++7QcMs8QvYyt4sIsJ8JNRJ+0icP2x1XW1s9GdoK4qqPwxADFtb73NX9mgZA79bRLikX+QxIWVSchS6vpes= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN6PR12MB1140; 7:7qb4VNcHjPQjU1gMP4PJt/SBcjYIQmdbFRYd0mnScilzgrA4UH0PRvwrlZMMv1L1ll7XIyUHeLvRHVWZGAQnNwBBUCvsz5/o1X15oCheV3NfREJgbFlThpH+Bi14l8Qk6PtFqttxBTuidquAUYgwpwtcCtqUwW0oc8I9ud4jl18vt8dIiQYFwtchzu9rxCA9yxMStx7DB25UhcQvjNCcttebTu5dHFDy0ssWDcuZJNG5UPRGqDp3ZOcbvMnafaO0J4eskxX8HCU56h2TTOZ8byc4TSan/C6gT8hL5zowKmoGpbmzITojCeZWX+Ym3t7sE8qRZhzMjsOdJcgDmCRnXQ==; 20:qBHD+kw4eAAEI5l8K/rF/Mp7Ngfto3X4WgYUQpFiCwJ10NJeWEg6bW+w3QzqmBLxRNtNP08vANEW5hoBc524gzkdUFqunqxGT6qG+FlKGJ8ZCUxfrokA+jX4Ohdb5W8DBtQk3fcR6Gu5C5HBvlvRXJXn+eZHt8VWa/9QnKi+l+Jpgx7EHQo+C/zTF9a882XihuZ4O3qa0A74+FrAU2Boo3IBMUL6XkkF6cp5yFpU0KCwImvZAGyAphB+0U+wWFi5 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2017 21:21:13.3755 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1140 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Update the KVM support to work with SME. The VMCB has a number of fields where physical addresses are used and these addresses must contain the memory encryption mask in order to properly access the encrypted memory. Also, use the memory encryption mask when creating and using the nested page tables. Signed-off-by: Tom Lendacky --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/mmu.c | 12 ++++++++---- arch/x86/kvm/mmu.h | 2 +- arch/x86/kvm/svm.c | 35 ++++++++++++++++++----------------- arch/x86/kvm/vmx.c | 3 ++- arch/x86/kvm/x86.c | 3 ++- 6 files changed, 32 insertions(+), 25 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 74ef58c..a25576b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1066,7 +1066,7 @@ struct kvm_arch_async_pf { void kvm_mmu_uninit_vm(struct kvm *kvm); void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask, u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask, - u64 acc_track_mask); + u64 acc_track_mask, u64 me_mask); void kvm_mmu_reset_context(struct kvm_vcpu *vcpu); void kvm_mmu_slot_remove_write_access(struct kvm *kvm, diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index ac78105..f362083 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -107,7 +107,7 @@ enum { (((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1)) -#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) +#define PT64_BASE_ADDR_MASK __sme_clr((((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1))) #define PT64_DIR_BASE_ADDR_MASK \ (PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + PT64_LEVEL_BITS)) - 1)) #define PT64_LVL_ADDR_MASK(level) \ @@ -125,7 +125,7 @@ enum { * PT32_LEVEL_BITS))) - 1)) #define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \ - | shadow_x_mask | shadow_nx_mask) + | shadow_x_mask | shadow_nx_mask | shadow_me_mask) #define ACC_EXEC_MASK 1 #define ACC_WRITE_MASK PT_WRITABLE_MASK @@ -184,6 +184,7 @@ struct kvm_shadow_walk_iterator { static u64 __read_mostly shadow_dirty_mask; static u64 __read_mostly shadow_mmio_mask; static u64 __read_mostly shadow_present_mask; +static u64 __read_mostly shadow_me_mask; /* * The mask/value to distinguish a PTE that has been marked not-present for @@ -317,7 +318,7 @@ static bool check_mmio_spte(struct kvm_vcpu *vcpu, u64 spte) void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask, u64 dirty_mask, u64 nx_mask, u64 x_mask, u64 p_mask, - u64 acc_track_mask) + u64 acc_track_mask, u64 me_mask) { if (acc_track_mask != 0) acc_track_mask |= SPTE_SPECIAL_MASK; @@ -330,6 +331,7 @@ void kvm_mmu_set_mask_ptes(u64 user_mask, u64 accessed_mask, shadow_present_mask = p_mask; shadow_acc_track_mask = acc_track_mask; WARN_ON(shadow_accessed_mask != 0 && shadow_acc_track_mask != 0); + shadow_me_mask = me_mask; } EXPORT_SYMBOL_GPL(kvm_mmu_set_mask_ptes); @@ -2383,7 +2385,8 @@ static void link_shadow_page(struct kvm_vcpu *vcpu, u64 *sptep, BUILD_BUG_ON(VMX_EPT_WRITABLE_MASK != PT_WRITABLE_MASK); spte = __pa(sp->spt) | shadow_present_mask | PT_WRITABLE_MASK | - shadow_user_mask | shadow_x_mask | shadow_accessed_mask; + shadow_user_mask | shadow_x_mask | shadow_accessed_mask | + shadow_me_mask; mmu_spte_set(sptep, spte); @@ -2685,6 +2688,7 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep, pte_access &= ~ACC_WRITE_MASK; spte |= (u64)pfn << PAGE_SHIFT; + spte |= shadow_me_mask; if (pte_access & ACC_WRITE_MASK) { diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index ddc56e9..c31b36e 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -48,7 +48,7 @@ static inline u64 rsvd_bits(int s, int e) { - return ((1ULL << (e - s + 1)) - 1) << s; + return __sme_clr(((1ULL << (e - s + 1)) - 1) << s); } void kvm_mmu_set_mmio_spte_mask(u64 mmio_mask); diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 5f48f62..183e458 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -1138,9 +1138,9 @@ static void avic_init_vmcb(struct vcpu_svm *svm) { struct vmcb *vmcb = svm->vmcb; struct kvm_arch *vm_data = &svm->vcpu.kvm->arch; - phys_addr_t bpa = page_to_phys(svm->avic_backing_page); - phys_addr_t lpa = page_to_phys(vm_data->avic_logical_id_table_page); - phys_addr_t ppa = page_to_phys(vm_data->avic_physical_id_table_page); + phys_addr_t bpa = __sme_set(page_to_phys(svm->avic_backing_page)); + phys_addr_t lpa = __sme_set(page_to_phys(vm_data->avic_logical_id_table_page)); + phys_addr_t ppa = __sme_set(page_to_phys(vm_data->avic_physical_id_table_page)); vmcb->control.avic_backing_page = bpa & AVIC_HPA_MASK; vmcb->control.avic_logical_id = lpa & AVIC_HPA_MASK; @@ -1200,8 +1200,8 @@ static void init_vmcb(struct vcpu_svm *svm) set_intercept(svm, INTERCEPT_MWAIT); set_intercept(svm, INTERCEPT_XSETBV); - control->iopm_base_pa = iopm_base; - control->msrpm_base_pa = __pa(svm->msrpm); + control->iopm_base_pa = __sme_set(iopm_base); + control->msrpm_base_pa = __sme_set(__pa(svm->msrpm)); control->int_ctl = V_INTR_MASKING_MASK; init_seg(&save->es); @@ -1334,9 +1334,9 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu) return -EINVAL; new_entry = READ_ONCE(*entry); - new_entry = (page_to_phys(svm->avic_backing_page) & - AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) | - AVIC_PHYSICAL_ID_ENTRY_VALID_MASK; + new_entry = __sme_set((page_to_phys(svm->avic_backing_page) & + AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK) | + AVIC_PHYSICAL_ID_ENTRY_VALID_MASK); WRITE_ONCE(*entry, new_entry); svm->avic_physical_id_cache = entry; @@ -1604,7 +1604,7 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) svm->vmcb = page_address(page); clear_page(svm->vmcb); - svm->vmcb_pa = page_to_pfn(page) << PAGE_SHIFT; + svm->vmcb_pa = __sme_set(page_to_pfn(page) << PAGE_SHIFT); svm->asid_generation = 0; init_vmcb(svm); @@ -1632,7 +1632,7 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); - __free_page(pfn_to_page(svm->vmcb_pa >> PAGE_SHIFT)); + __free_page(pfn_to_page(__sme_clr(svm->vmcb_pa) >> PAGE_SHIFT)); __free_pages(virt_to_page(svm->msrpm), MSRPM_ALLOC_ORDER); __free_page(virt_to_page(svm->nested.hsave)); __free_pages(virt_to_page(svm->nested.msrpm), MSRPM_ALLOC_ORDER); @@ -2301,7 +2301,7 @@ static u64 nested_svm_get_tdp_pdptr(struct kvm_vcpu *vcpu, int index) u64 pdpte; int ret; - ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(cr3), &pdpte, + ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(__sme_clr(cr3)), &pdpte, offset_in_page(cr3) + index * 8, 8); if (ret) return 0; @@ -2313,7 +2313,7 @@ static void nested_svm_set_tdp_cr3(struct kvm_vcpu *vcpu, { struct vcpu_svm *svm = to_svm(vcpu); - svm->vmcb->control.nested_cr3 = root; + svm->vmcb->control.nested_cr3 = __sme_set(root); mark_dirty(svm->vmcb, VMCB_NPT); svm_flush_tlb(vcpu); } @@ -2801,7 +2801,7 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm) svm->nested.msrpm[p] = svm->msrpm[p] | value; } - svm->vmcb->control.msrpm_base_pa = __pa(svm->nested.msrpm); + svm->vmcb->control.msrpm_base_pa = __sme_set(__pa(svm->nested.msrpm)); return true; } @@ -4433,7 +4433,7 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi) pr_debug("SVM: %s: use GA mode for irq %u\n", __func__, irq.vector); *svm = to_svm(vcpu); - vcpu_info->pi_desc_addr = page_to_phys((*svm)->avic_backing_page); + vcpu_info->pi_desc_addr = __sme_set(page_to_phys((*svm)->avic_backing_page)); vcpu_info->vector = irq.vector; return 0; @@ -4484,7 +4484,8 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, struct amd_iommu_pi_data pi; /* Try to enable guest_mode in IRTE */ - pi.base = page_to_phys(svm->avic_backing_page) & AVIC_HPA_MASK; + pi.base = __sme_set(page_to_phys(svm->avic_backing_page) & + AVIC_HPA_MASK); pi.ga_tag = AVIC_GATAG(kvm->arch.avic_vm_id, svm->vcpu.vcpu_id); pi.is_guest_mode = true; @@ -4909,7 +4910,7 @@ static void svm_set_cr3(struct kvm_vcpu *vcpu, unsigned long root) { struct vcpu_svm *svm = to_svm(vcpu); - svm->vmcb->save.cr3 = root; + svm->vmcb->save.cr3 = __sme_set(root); mark_dirty(svm->vmcb, VMCB_CR); svm_flush_tlb(vcpu); } @@ -4918,7 +4919,7 @@ static void set_tdp_cr3(struct kvm_vcpu *vcpu, unsigned long root) { struct vcpu_svm *svm = to_svm(vcpu); - svm->vmcb->control.nested_cr3 = root; + svm->vmcb->control.nested_cr3 = __sme_set(root); mark_dirty(svm->vmcb, VMCB_NPT); /* Also sync guest cr3 here in case we live migrate */ diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 1a471e5..16b8391 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -6480,7 +6480,8 @@ void vmx_enable_tdp(void) enable_ept_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull, 0ull, VMX_EPT_EXECUTABLE_MASK, cpu_has_vmx_ept_execute_only() ? 0ull : VMX_EPT_READABLE_MASK, - enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK); + enable_ept_ad_bits ? 0ull : VMX_EPT_RWX_MASK, + 0ull); ept_set_mmio_spte_mask(); kvm_enable_tdp(); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ccbd45e..2e65335 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -67,6 +67,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include "trace.h" @@ -6090,7 +6091,7 @@ int kvm_arch_init(void *opaque) kvm_mmu_set_mask_ptes(PT_USER_MASK, PT_ACCESSED_MASK, PT_DIRTY_MASK, PT64_NX_MASK, 0, - PT_PRESENT_MASK, 0); + PT_PRESENT_MASK, 0, sme_me_mask); kvm_timer_init(); perf_register_guest_info_callbacks(&kvm_guest_cbs);