From patchwork Fri Oct 20 14:30:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10020359 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4A66A60211 for ; Fri, 20 Oct 2017 14:32:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34EF128D12 for ; Fri, 20 Oct 2017 14:32:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2964328D5B; Fri, 20 Oct 2017 14:32:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9968C28D12 for ; Fri, 20 Oct 2017 14:32:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752969AbdJTObv (ORCPT ); Fri, 20 Oct 2017 10:31:51 -0400 Received: from mail-dm3nam03on0088.outbound.protection.outlook.com ([104.47.41.88]:47104 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752889AbdJTObr (ORCPT ); Fri, 20 Oct 2017 10:31:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Ji/u+gIsrHYg3rK1d4UzuE4et9Yy0dMqCfJXl3Q7w5I=; b=sfqSFdJWnfeUUO72hQPQPbKYSP80/jiFbcinab+W+PzzfvUXL0L/WFieOv1jBPfwXQzlbaPKpBkfV+jkbXR91PF3mn9D+rx87rnoDuej03f3lKznseyRXM2McTPP+DBtGSIyuoKyMq2lDBKvUZ15rsAhGmJlPB5ad02CdhNYbus= Received: from ubuntu-010236106000.amd.com (165.204.78.1) by SN1PR12MB0160.namprd12.prod.outlook.com (10.162.3.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.156.4; Fri, 20 Oct 2017 14:31:31 +0000 From: Brijesh Singh To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Borislav Petkov , Brijesh Singh , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Tom Lendacky Subject: [Part1 PATCH v7 14/17] x86: Add support for changing memory encryption attribute in early boot Date: Fri, 20 Oct 2017 09:30:56 -0500 Message-Id: <20171020143059.3291-15-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171020143059.3291-1-brijesh.singh@amd.com> References: <20171020143059.3291-1-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: BN6PR14CA0029.namprd14.prod.outlook.com (10.171.172.143) To SN1PR12MB0160.namprd12.prod.outlook.com (10.162.3.147) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0441fbc6-6a82-4e4a-24c8-08d517c742a8 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603229); SRVR:SN1PR12MB0160; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 3:pOYdVgHgcpnTPImu/MbzM/CeiB2Ulllllwaw+HVSjGnqzcjy18UeGrQOSeFogzoQzJc2zSp+KvvubA/eXrMFNnSM1aKkColtpAHZ/xbqkvcZsiSogzpqQVTdaN0yYgiQRx6Wbf47LfroWsmCpv8Y373mwFsXIn/twO7tIVKF7QlQf8eDCkhJagZ5q5CGqJPUovt6kSbldYofhKRl/9lcBO73FeCVZn7VQdLMZXN8QBfjLJ3OInNJ3TUPYmRPXVTX; 25:3S4cGrPfCbOlHaLuBzk0IdVhDnG5xjIFlx5GcKlwj967wqQo1POv/6H0wopdrYPEwVYzKfXDnhU6BaH3w3JtpWFSB5+4Hdl5KWCyjlvxmyQtIQ8V06GjGJXN1hoE8fmRak/crpdheV5YDWjlDr8ym3ZSwL7KF5rvuOxeuXvk3cJahZVnOL7TpjMGJCRu+b/8AI9qTpY3eqBYkLznGBA8GWi6w2GLNCw+xK0DAryiLg19RUZjYdA28xET0/INrFwwfJ9/B9PDIO9QSGWFGVGc4AETvqrqezdfYdPiCkJDnX+PTiZltu4RAlfMGGFzrGsnQPPOLblpUOqNGsJ5YCD92w==; 31:Nu5L6pPiP/XztiDjzxhkSBjR9ZKPJpsrvSyuiFIqiCJRzQ2rbad5f1h1G1EvC+EkT+esbUb/svpAu8ktZBH9l2MbB2b4qMMs4VVWgzuhMGJDb7UKAcdR5harlOmbTMTnlvOl5yct/NMu/b7N/S0EHAqIzQbkW7POR/fGWXe8Wt6GzGhgZDw4o0JVG7h4iuaNpS2lcHBm/x1znTVFI4JCdvSkf++7EoZw0kjcZ2SWXqE= X-MS-TrafficTypeDiagnostic: SN1PR12MB0160: X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 20:hxwlel43KSUP70cbAyZJthRWvmgQCTEEeRnBt5QWGEQ6k6V9oOv7G7coZawI/xNDiawYbgRwNMm0kqK3OvzlBdtLfLFNP374Kc0Hyk7b8LiIYdjahqgC+hlvm/5ihjJii+Q2TBgghKdrJZsp8i5XbLsSB7VdTN9FdEYCc9hIIQf1hU1t+RasfWidxiW2HVi0Hm+CywzfJfC/sZ3J2vf7tJ6iUIXao6EhTvHmPuIInkohlsZVRwpsqF/cjuk5UWWqCdRZraraL78G1W9lgPTlATKnvr2hbVz1PGMGD23YfIpOcoqQHH65+B8gjCaMe5liBgRYtPEIDsESY5RgPT3OnQfDtMa9UUylx+gWNYNz/ZPKr7E9FyCMgPbiSgs6Nutd1Nj0DQI+Jb5x7vadnKHl+FcyyJV44uexHMxH5jh5Wp5GzyBw2iFY3cyOJVKHWsB2VHBgg8ALDz874VqjW9SFj7Imv3k56yjSLCdqFO8rRIKnJUKjmeQlpxBEoBn3Qjr8; 4:kcamKICGJ6asCWOpkXLoMJtXNRLIYRTPt9GJQGGdpstU6qt0zi04Ri5rRdsYt4canlPe7i8UQO2SP483UGv5hyFhtLp/Hdd+5TTDoI7On4p1qRFvD0apgL6V+cNFQg8KDoc101OlL+U5qtuQIGMW5nVIJ4YKmhBVuBqWMMRjJOkbs4skABLdxydBaJ2dvzw5ITTNdLSreJwnEX9XKSGnLJnTOJ+4nTae+S3Grzoke8Xz7NllXrdTvAsfpluHO2TD03+5bIl95fqC6lCJYTLy216IMOcEt/selcryKCxNM4WaZLB9BCaOMkhbdYmVy3Ob X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(3231020)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123562025)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:SN1PR12MB0160; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SN1PR12MB0160; X-Forefront-PRVS: 0466CA5A45 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(39860400002)(376002)(346002)(199003)(189002)(54906003)(1076002)(6666003)(189998001)(68736007)(8676002)(305945005)(50226002)(7736002)(101416001)(76176999)(8936002)(97736004)(25786009)(81166006)(81156014)(16526018)(6486002)(478600001)(48376002)(2906002)(50466002)(4326008)(47776003)(66066001)(5660300001)(2950100002)(53936002)(106356001)(3846002)(50986999)(16586007)(53416004)(6116002)(5003940100001)(36756003)(33646002)(86362001)(105586002)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR12MB0160; H:ubuntu-010236106000.amd.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN1PR12MB0160; 23:c/NbTJ1ujQWplgqH1yRiM+6rLlDgS6xQT4xbyl7Hg?= =?us-ascii?Q?4oEPgkcRxq/bfRkClF02ppsZdbUFj3P+ot/KD4yfL8KKJ89t9BNmJohUTNLV?= =?us-ascii?Q?gJVjR2IwPcgKI/2dkPThPEBv4HfJdBFsBARZEcSIt6WFx76T4JLRDZRWZrUY?= =?us-ascii?Q?MsYpXjRyAEGGTVmI5ty14B14l0hf1kXQ0W7UkBieY6jnWgG8GF/whxQilYZU?= =?us-ascii?Q?Ws2ZwAhW4hEapI6I26h1TT4yu4onSzu2TFeWxAIuTiTIghkA8nVtVgl7gItv?= =?us-ascii?Q?a3BQO5BjcWmmGl05puhCq/woWoLJXB0/vqTtjyW4Ypw4dPDp3rLwrp8bera8?= =?us-ascii?Q?lTeJ0W9yJzVxzJrn9iYEFRFIVFVdO3RIoMulyn4PsShVXs1HTdLn/k3IaqJH?= =?us-ascii?Q?1vKkEtH/iFnbihm5Vms0IA/ehAOXS8aUfnHUeNIORovHFvvpEQ744Qei/0uv?= =?us-ascii?Q?619PLuZgs5X2OxgGqTTwFpDDFCsDLMGp5AmrTDGeY8EixZ7DNUFwitws9fDh?= =?us-ascii?Q?irG4oim+HSlUNbUjyUpdvvEc3vusmCBrWGjiNl8T3ZlyPgnP5fuCHyNPJo8c?= =?us-ascii?Q?WzKBfDnk3bl4PIX6G+FeMCklgkgsAM9pry+5dMMWp42nzNw0Rpsltwf3no+4?= =?us-ascii?Q?iSLMM3rdPNdt+ODs5/VPYCvU+tw1CYdanVXXyNtjwDz//hXseVN4pQDDFmaG?= =?us-ascii?Q?3/X6Oy05DQIVsMzJQw3vi9ImH/I6TZ2K1aOx5xkz2FW3/lXcvOnjpr6QfNH1?= =?us-ascii?Q?q6uJXOpb+Iq4CgQBtKNxa2UefEXnvge8pPIHhmU2BWrqPGmRJ4gVkdbkCQBF?= =?us-ascii?Q?ES1KOGf+BjVJAljPMgc+zv4682AosGh7L4sAuYdbvrWO4u49Otn8wCLKw55/?= =?us-ascii?Q?fwMkqAJ4SsuiN0uHygXtGpcHazPGpK3GkykLIzGYe+xh4g8GsIaB1X8HDp2Q?= =?us-ascii?Q?HeUxnvAaf9z++jJrs7i3oIojolH08a6acTgt9ohJBAXCpExqX5KYBZFvDVm7?= =?us-ascii?Q?3ovy5vI/A5rEu42lhmB8ps7CHoIMf2FQn+1Hx4rw9v+6UIQjlFIRIQ12yC+H?= =?us-ascii?Q?zCIJUQ6UyZ4n2qVoitVRrNx9w6zIwUfVZhNhUilVu2S9vChyQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 6:v9licRnhE8/ZvEMVfS4fsm688i2oO1UPKA7VpA0NbjhJL/bnhj0M1bi4k7dcsSZGYWw8gFolOiUAq9/9Y970t3Mg1uUfWH07Vl1owW94/qFlB6DvxDlvHdG+/+kJm5Mrq1AWxBw71DsBy56jsZaoTCmyR+UkBrMEhITa7JxMubjEtJTuaRQ1NmFwtlD5aEyafc3tzleReDS7+1TZZhM5CB/wH1/Mvhrtg89LxI25yGEo+SVqZnFVRaUCeSIkJ65yIPfMF2Dmu+xNzLc67RkuMPuOJ8yQ8cSaSOebTyffLl7z9b8eoUi0amRFFgWrqCZBkF2dEpXeBni2J4gSgcyMjg==; 5:U1yoTZR+TJ0How208e4vIyLtqY0UcKoSi9b5pxahrL330i2/dZpP0qzfysKt5d3p5EXGTPFGN11APfroouI7tyY5zrF8m1vBb0omwkH28NwFiObT853tAcRoYkRMXu4kOUX5kqDksL5JaPFLVFe+cQ==; 24:gWrZGP+qhgHSE6H0+c7LjmCsukiZrNszqybuFYDvaDrrxt9ht4Lj6fSbyrWvnmiGD00InyKAWtaZJ2HyCE1yOtC39Lx2FR74xUC4AtHqEsk=; 7:dgr6yESDrtGisRcMfohMs9EA+7GJ4oLnFdmcjT/NuJXxN6Xg+jYg9s9nRQ2Jpzirbs3riy5xIHDS5EqFSpBD0b9pyvg6zHQmkU1BtLLmVXn6rk3XYOX23n+W3NDiJVZCG+vewEAYlpLzvnJLLA8f5vE/RLgaVhyspTUceqpXNBZptto3vRbH3uJnInD/oeUU3VVoira1tUQPnCbZQfgdNtFbr9nWUIzEvDObfHpI0qM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 20:Z9mdGnjVGIWlACpO5BHYtO0TFIPEXaDWyH0mrhEmAgN+RMZIG3P9vmtBs4Tw6VH+GBp0Ieeuvhvwkx2FHAQYUDT9MGxc3uJmWFFagzDGeyZx1a3p0PFBX2pzRolM+2qs6ogkmfZNZ6sK6GeDOluUp2RHRQFrEio6KejRqWzYoGFz/VuPXDQNkFOxHe5NO0pjtkxoaR792wgAKPLi9VzUrbcZi4hy8zbZX72hXb5/vfhGoRloZKgPqBDtvlPLK0Dc X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2017 14:31:31.8914 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0441fbc6-6a82-4e4a-24c8-08d517c742a8 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0160 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Some KVM-specific custom MSRs share the guest physical address with the hypervisor in early boot. When SEV is active, the shared physical address must be mapped with memory encryption attribute cleared so that both hypervisor and guest can access the data. Add APIs to change the memory encryption attribute in early boot code. Signed-off-by: Brijesh Singh Reviewed-by: Borislav Petkov Tested-by: Borislav Petkov Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Borislav Petkov Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Cc: Tom Lendacky --- arch/x86/include/asm/mem_encrypt.h | 8 +++ arch/x86/mm/mem_encrypt.c | 130 +++++++++++++++++++++++++++++++++++++ 2 files changed, 138 insertions(+) diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index 2b024741bce9..c9459a4c3c68 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -42,6 +42,9 @@ void __init sme_early_init(void); void __init sme_encrypt_kernel(void); void __init sme_enable(struct boot_params *bp); +int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size); +int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size); + /* Architecture __weak replacement functions */ void __init mem_encrypt_init(void); @@ -70,6 +73,11 @@ static inline void __init sme_enable(struct boot_params *bp) { } static inline bool sme_active(void) { return false; } static inline bool sev_active(void) { return false; } +static inline int __init +early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 0; } +static inline int __init +early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; } + #endif /* CONFIG_AMD_MEM_ENCRYPT */ /* diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index 45110b60be21..6889278afad3 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -30,6 +30,8 @@ #include #include +#include "mm_internal.h" + static char sme_cmdline_arg[] __initdata = "mem_encrypt"; static char sme_cmdline_on[] __initdata = "on"; static char sme_cmdline_off[] __initdata = "off"; @@ -260,6 +262,134 @@ static void sev_free(struct device *dev, size_t size, void *vaddr, swiotlb_free_coherent(dev, size, vaddr, dma_handle); } +static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc) +{ + pgprot_t old_prot, new_prot; + unsigned long pfn, pa, size; + pte_t new_pte; + + switch (level) { + case PG_LEVEL_4K: + pfn = pte_pfn(*kpte); + old_prot = pte_pgprot(*kpte); + break; + case PG_LEVEL_2M: + pfn = pmd_pfn(*(pmd_t *)kpte); + old_prot = pmd_pgprot(*(pmd_t *)kpte); + break; + case PG_LEVEL_1G: + pfn = pud_pfn(*(pud_t *)kpte); + old_prot = pud_pgprot(*(pud_t *)kpte); + break; + default: + return; + } + + new_prot = old_prot; + if (enc) + pgprot_val(new_prot) |= _PAGE_ENC; + else + pgprot_val(new_prot) &= ~_PAGE_ENC; + + /* If prot is same then do nothing. */ + if (pgprot_val(old_prot) == pgprot_val(new_prot)) + return; + + pa = pfn << page_level_shift(level); + size = page_level_size(level); + + /* + * We are going to perform in-place en-/decryption and change the + * physical page attribute from C=1 to C=0 or vice versa. Flush the + * caches to ensure that data gets accessed with the correct C-bit. + */ + clflush_cache_range(__va(pa), size); + + /* Encrypt/decrypt the contents in-place */ + if (enc) + sme_early_encrypt(pa, size); + else + sme_early_decrypt(pa, size); + + /* Change the page encryption mask. */ + new_pte = pfn_pte(pfn, new_prot); + set_pte_atomic(kpte, new_pte); +} + +static int __init early_set_memory_enc_dec(unsigned long vaddr, + unsigned long size, bool enc) +{ + unsigned long vaddr_end, vaddr_next; + unsigned long psize, pmask; + int split_page_size_mask; + int level, ret; + pte_t *kpte; + + vaddr_next = vaddr; + vaddr_end = vaddr + size; + + for (; vaddr < vaddr_end; vaddr = vaddr_next) { + kpte = lookup_address(vaddr, &level); + if (!kpte || pte_none(*kpte)) { + ret = 1; + goto out; + } + + if (level == PG_LEVEL_4K) { + __set_clr_pte_enc(kpte, level, enc); + vaddr_next = (vaddr & PAGE_MASK) + PAGE_SIZE; + continue; + } + + psize = page_level_size(level); + pmask = page_level_mask(level); + + /* + * Check whether we can change the large page in one go. + * We request a split when the address is not aligned and + * the number of pages to set/clear encryption bit is smaller + * than the number of pages in the large page. + */ + if (vaddr == (vaddr & pmask) && + ((vaddr_end - vaddr) >= psize)) { + __set_clr_pte_enc(kpte, level, enc); + vaddr_next = (vaddr & pmask) + psize; + continue; + } + + /* + * The virtual address is part of a larger page, create the next + * level page table mapping (4K or 2M). If it is part of a 2M + * page then we request a split of the large page into 4K + * chunks. A 1GB large page is split into 2M pages, resp. + */ + if (level == PG_LEVEL_2M) + split_page_size_mask = 0; + else + split_page_size_mask = 1 << PG_LEVEL_2M; + + kernel_physical_mapping_init(__pa(vaddr & pmask), + __pa((vaddr_end & pmask) + psize), + split_page_size_mask); + } + + ret = 0; + +out: + __flush_tlb_all(); + return ret; +} + +int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size) +{ + return early_set_memory_enc_dec(vaddr, size, false); +} + +int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size) +{ + return early_set_memory_enc_dec(vaddr, size, true); +} + /* * SME and SEV are very similar but they are not the same, so there are * times that the kernel will need to distinguish between SME and SEV. The