From patchwork Tue Sep 4 01:29:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10586441 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 075C7112B for ; Tue, 4 Sep 2018 01:31:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EADF528BC9 for ; Tue, 4 Sep 2018 01:31:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF45F28BDE; Tue, 4 Sep 2018 01:31:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E5FB28BC9 for ; Tue, 4 Sep 2018 01:31:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726039AbeIDFxT (ORCPT ); Tue, 4 Sep 2018 01:53:19 -0400 Received: from mail-bl2nam02on0082.outbound.protection.outlook.com ([104.47.38.82]:42369 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725833AbeIDFxT (ORCPT ); Tue, 4 Sep 2018 01:53:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K2txtTs/dKgYSLSmfQ6fmp9U7m+Xsj1BFCJl0Pp9LE0=; b=ZDUkVMbPEUxPPJyJ80o0T9TMsPWdCsR8PBdgvAEdFCCsRnbMB96KbplEu4xa2n/4/TvP4NUnSyVl9XAzrNPHxlbA5w8HOG/dhdE+CJSrknKRzYPY3jhRnSHZ28ZcY5tiBMoJXLpkLZZB+dd6EwcsPKaKrqdl0CjTeOTvqeV26UQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1101.18; Tue, 4 Sep 2018 01:29:59 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Brijesh Singh , Tom Lendacky , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Sean Christopherson , =?utf-8?b?UmFkaW0g?= =?utf-8?b?S3LEjW3DocWZ?= Subject: [PATCH v4 1/4] x86/mm: Restructure sme_encrypt_kernel() Date: Mon, 3 Sep 2018 20:29:39 -0500 Message-Id: <1536024582-25700-2-git-send-email-brijesh.singh@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> References: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN6PR20CA0066.namprd20.prod.outlook.com (2603:10b6:404:151::28) To SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 88c02af3-d92b-43a5-6474-08d61205ee75 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:SN6PR12MB2687; X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;3:mTc7CAhA6MP1N115JshHI1S1GFvZMdzAzvEYG2/xhqjkrRK9LQVijtQKV2uYmfgsI7NUIkfY6ZWCHoH77t34qfbfnHIz4FUoGDHrfX8/vpmwdjY+tCQ6tb6SwqpIZmh99g2bY0v6Dve18FJEIEokJXf0MpYfILpWPxoWp8v7ZVNDuZGyKWPiaxeJWvjM67bNmqryQXlZvlJs2Sy9fgiSzfMAvqYmvr94vFBKPdTYDTeytjYA6W//6hnlcXlQ9IGZ;25:O9FOw3dn95TJApn0EjzIov1JsAq7vat7GE5JreVK3KfLkYus+Vt0mGg2SAFrHDG56NPI8G4JNnbAm7frmtGuviEnZ5JxsfFydI3H0zP9hLJDUpIpfaVsu6hbuNCmjn0yinIpY22c0EzYz6GSemtd6djIkaV01nCkfxX80nBrjtt540cIlojk03NsHEgyV5+sOxLGs5PzHgQY9T9BegHN7z74iRKwcu+CkGZbY5CgWe5fSvRcMi9tCcctSg6ZtyABJ80fQGXrK5P+ksXAZC3fPJjEqGFK7T4d1OzjwDRxelgeaNHV3KPcimoqm7K27++nwwgNzpdrWoLravBOn+DGZA==;31:Yf5tXUD76O8DQrVY+PM7VXeoVXrfKf0qmci2E57AXsuiyVHP9Pqx445UA9hMcSG7PT/ZacfrTZeS3jECxgvYVlVkVaziU7v1ux7bd+YlIyBjFYCtAYPRW1xHRZnkjaI9r65K4E/4ViCbEyQrrFUmrAZvyWOU6x7T5f8T06wNW3P0dqV+nqBhdOyf/Dp/KJ2D+mi9m7mMM36Bl83OWA6k1Xg8H9r6BAFJP7onWHTEh/M= X-MS-TrafficTypeDiagnostic: SN6PR12MB2687: X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:iSBbeCc6ByWaMRzzehQy7tbVDy0RY39wfpE0Z0OWZx2RFerv0crzIun6EcEaEWHpJroH57jOd4T+vu4/4zdXr0xuhnMFkXKw4yG9hBrsl6XKrjGQU/kxTFSG1dRPU+05VUQ4ObhQ9DNGXgWYYn2B3j4qi9jq9U5DROEunlTRm4yjM7urCJT5gfqt3ovPQQXJ989xqi2Ca24SaScnbd/Clh4ZBbHOEec67Lp8UhYJmbMUUycddnnwjxuAKtQJlmvqepMwI/sDMIR6ePEUSSVePm5m3sO0Aweo46j/yd3jo9zoqV+sP93iKyCh1ISZFwd+iGz8/oXZELD0+p4PnLTBfIVTRG4RMgyrO2rcoXwbAtXTCCrnsTezGis1PWPJoNhR3H+6Pte5nzGqMLOYJ9/LXv0zLZa8z+F+6SCO8n4uPFoBikZnfgfUlzpV4MnyPrpK/ut+6HWKZLlYUmWtYz73mpN2qp5pjvaPC8aR1XB3/1WT6UCF7YA8gPnap914czCn;4:2QgDLuWELV2ZSzxgrf3ht/nq6toSOucOcgR3vVVJeNMdlsotF4GfE+r+oG4WSVBm99zZKOgOIgfxRQfbmartcqn5mZpx6SzThBstecFm8He7qw9N7OUGW3jJXuprNFn4RS8Dw0GLIAwdWAqQYrRZPEtZ965AB8OVw0K5nsNruq/ZAyLAOM2TAKVJz4auqwsJNDEQxokDK+yy/b1baKl8CjFli5MA2wXiQjKn8PxtDoNu/mS4ZuZ08wy2sSmcJ09apVL4d57cB5BW8LgIVYICzyMj/Wp/d9XrQecq94NK9omvDeY07LiEA6e3trzAqXfIpuW28Do1FdZGsYlnziWyTwiYKH+jui1+qpTb7/L8xI1Np4JZhapDAKcjItA11yEg X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3231311)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:SN6PR12MB2687;BCL:0;PCL:0;RULEID:;SRVR:SN6PR12MB2687; X-Forefront-PRVS: 0785459C39 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(366004)(396003)(39860400002)(136003)(376002)(346002)(199004)(189003)(14444005)(6116002)(105586002)(76176011)(7696005)(47776003)(23676004)(66066001)(68736007)(52116002)(2870700001)(53936002)(53416004)(2906002)(316002)(36756003)(3846002)(97736004)(6486002)(50226002)(4326008)(25786009)(54906003)(106356001)(81166006)(81156014)(8936002)(478600001)(2616005)(6666003)(50466002)(86362001)(8676002)(44832011)(5660300001)(186003)(7736002)(305945005)(16526019)(386003)(486006)(476003)(26005)(446003)(11346002)(956004);DIR:OUT;SFP:1101;SCL:1;SRVR:SN6PR12MB2687;H:sbrijesh-desktop.amd.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2687=3B23=3AJ0sYLRg?= =?utf-8?q?XGvQ8XImhg1TIUzpxUSmxdSIYWS9tUi1dE7BHTKOOYd0hDu+WdE3Kepv6I03hJx0F?= =?utf-8?q?pOVYE4qdk2jAMhvavJtTBN+Sy2wBfMT+sEHrJ0i4OsZ3IU2lZMHdaaIV9KwCnDdyD?= =?utf-8?q?ESByEUWqilYHnghp1TSmdt9BQ+Dbalj2XoGz/0vmI8nRCZa9cezE/3RCCuuGs6PWH?= =?utf-8?q?MiqotvawE1I1Y2tq6TtMibzXnMUeh1ugg753giWhyLTFaq/3AoU36HWRo0GyUTcar?= =?utf-8?q?lO1WvKyGhQHWzl7e+wUiTyvyYy43X4cH02/6Sy2nFAekv9LS4bS2AUdCRte+Jdv8o?= =?utf-8?q?1S5ruI6z1saypEQiglTuHYmRY+WOjdF3IH4wW3K8KnhkOHm40CXFWZtxwpfb9uMb+?= =?utf-8?q?g3o2hWfj0CJW9HtGsQogRjLsvRRHa2hQnt+QgRni6Mgw4HdUrO65Ofs8uIIU4FwLG?= =?utf-8?q?zrmvHrlwhmLwdnEqVjADvU2xCdtceGtBdb4Q/thImMGMSHhjbL2AL3S/GkZP3P2d7?= =?utf-8?q?ihVNO0NB4zaQJYQjkP/GqmfGp4easmfr3rELzcKgpkI5xhcXP4kuSTCBQkstnwMbC?= =?utf-8?q?72im0UYUMdEVYxNqtlWjrLNAPl9FiR/VIx4PRJwYVBUk7w0lRAcKiX6AeQHDALc7U?= =?utf-8?q?Tzz9n79qyymobILHT2YKUpn8WYzx/F30zwDEMeXhcpde7LH35j2d7DEfluNQGYl3R?= =?utf-8?q?KKuTiTJys7BIDFJ89WWibs6d8twvJGnE3HgPKamtHL4EShW4h7cGZxpLFY5A0Olnr?= =?utf-8?q?SykS5S/AlOZoELZQpivKruHC6XRRofAFDgcXlwjZZWgeLLzfggrs4eh3NW6/tPwmb?= =?utf-8?q?grWEbBGhu7xzGQAKYw+dB+Mo0S1eijEpg20zzYSS2oaBoms7xRh4c0tZUgXJIyoyW?= =?utf-8?q?wfjAi9w9Z8EPoj4e4lnJtWwZUUEdVfZ4l8DBeWLJbHECYKRvbzibpe24O1VRIp3wd?= =?utf-8?q?vG3LUwtjOeZq8y5T0m2bh569m9SM66rubua0TuW6s/+knZjJn5qSseShkWZzNHBmV?= =?utf-8?q?2qmBxnx/ZYsPoP7VUwLBLtSTaEnZMVYmS2JS81cvm/sc+LqS//cLv4NCV9yhU6YEO?= =?utf-8?q?D1USjBkxifT9dLuauCygnMPY7YqqloStGPWU7tE4K3TmXXYlAezzZ5v50W7HZ1yR2?= =?utf-8?q?zvYAqZmLYd7uCugJiCk4CtlmSMxMCU2/UUhbWCF?= X-Microsoft-Antispam-Message-Info: tQ8nxaHtxHsKe2O5GFkdWtWCk0Ab+LQkj39aevs8tgYNXv9hpijdcPHeAkc3jES6hztdW6dgvNd7vQyNzzpt1Rlu3fG/LnocjAwBeFZkTsHfX37pzz2+awdIqnjYsvQA+PH7OEtjUbD4XEGhpTLe7JTmCQXo/7PNYG7vRVT3lVJVqo9uBhmSVpzYEQnqXbqKqxC5X2MVDDMlRITFgHmRpud4HlPSvgihlv+87Z+isMsBX2Hjtf6jJcwp9XrPBSC9BGwo+t0F39WCQGasOVfHkkCnYoVg5LWR3jswnuZ4MKRhyYBTBWJ5pH9bWJ3wW75Q1k9/93BtItsZpjmqXp5SLlbgkVBz1zAq5W5h5RPJSjE= X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;6:9WiwyTooMFC0e0fPpFzlekrcNtAS1mvW0xOK6/Pl+POyc3UJZGIwV/QzZkWGrlkEjteiCRHEF2OolHLOsGEpQ/qpcCMpx9I2HIIqWmkT5/G/gqaSxRD6QThXQJyLZ61q0R4cuOcesTNw+LSrPG4lSYJxlBxge1BTdtHC+hXmNRieFZHZVU6hIPfSVi9nIkxe7HQF3WZmo4AvwykXeyB4WVpbAadikGGpzK7Md+FMl2JveyfLhipXchUiES4BErpguP20V9knLiIeW6Q0TkPXTBU5+elinnbpYiLEiKm8dfp6cs+vOJ5fyrsEteygM4BDXm4vDvefVqEX9K1x8bGEFQ8TNFGsOICrbm/hUJhVH+uvjhye72TrKJkE+GfQizAmtQjt11ltyy7YC15PcKAPQaYKu4+Ys7hpn+MLXY8WR+ALCuHfaDSVaUlypPMwSrf/RHg8ys/zY/YBcLoKS6rQ4Q==;5:vYgjx7mWcJc6/9A+p+JGZ8zvWzmd94ICJaxrEY9OEWeLuSdLri7SaQSzO5sB5MDbefX+5bQsXUgjBeUm3+q79jC6CnDQhA6Li7LKVqzPjDYJqCvPi/M4zvg/JVOvcHDJ6hInEqdvbmPwsiPO3yINl2eU0GiXkpAqC7E4HAxsV58=;7:wuoZdqaBpC6SVci/GqWz71WCn6UVMgShH3CcsWILq0iqiDAWyL5vJxd9xdo5nAm7OGBZwsHHJqaaEm4J0P0i2ujS7LsQ7L3RFkh07BF0/hp7LRMMP9R07rIkdVhP1LqoFpbiP6abIfMqRVu6pSbzoUNFHuW/F+IkpjSMwt8bPV6n69riaO5WYMI2HQQmuilHcnQvb6IGqEz2uENQEFUd2PxYPHn7dlmuudW/OyMQbeeuFpo720f7qZnrBtBO7WAq SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:vxXBnIyes4b0rtWC8wxHkCs59Dw6aBsudJIRpS1t7SyDSR5nrFkfrnxZfalO5tLvk+No5XJE7nKDV4mP1GtmNhpCX5PJceaifRoYSoskUNaOp/50ZnKISsSj0jipcUgxWX5ZwgTUcFKo/gP2QMdOX0GSGiDszUrSld/Z+YnEzhUgbxVhuijpQH33oEi19++YnkqE5HZ7L1q875Fi+aDe83y0G/mmaXirQYxdWbOWgECD7Mapdk+4lXuItkHgM+Un X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2018 01:29:59.4154 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 88c02af3-d92b-43a5-6474-08d61205ee75 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2687 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Re-arrange the sme_encrypt_kernel() by moving the workarea map/unmap logic in a separate static function. There are no logical changes in this patch. The restructuring will allow us to expand the sme_encrypt_kernel in future. Signed-off-by: Brijesh Singh Reviewed-by: Tom Lendacky Cc: Tom Lendacky Cc: kvm@vger.kernel.org Cc: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Cc: Paolo Bonzini Cc: Sean Christopherson Cc: kvm@vger.kernel.org Cc: "Radim Krčmář" --- arch/x86/mm/mem_encrypt_identity.c | 160 ++++++++++++++++++++++++------------- 1 file changed, 104 insertions(+), 56 deletions(-) diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 7ae3686..92265d3 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -72,6 +72,22 @@ struct sme_populate_pgd_data { unsigned long vaddr_end; }; +struct sme_workarea_data { + unsigned long kernel_start; + unsigned long kernel_end; + unsigned long kernel_len; + + unsigned long initrd_start; + unsigned long initrd_end; + unsigned long initrd_len; + + unsigned long workarea_start; + unsigned long workarea_end; + unsigned long workarea_len; + + unsigned long decrypted_base; +}; + static char sme_cmdline_arg[] __initdata = "mem_encrypt"; static char sme_cmdline_on[] __initdata = "on"; static char sme_cmdline_off[] __initdata = "off"; @@ -266,19 +282,17 @@ static unsigned long __init sme_pgtable_calc(unsigned long len) return entries + tables; } -void __init sme_encrypt_kernel(struct boot_params *bp) +static void __init build_workarea_map(struct boot_params *bp, + struct sme_workarea_data *wa, + struct sme_populate_pgd_data *ppd) { unsigned long workarea_start, workarea_end, workarea_len; unsigned long execute_start, execute_end, execute_len; unsigned long kernel_start, kernel_end, kernel_len; unsigned long initrd_start, initrd_end, initrd_len; - struct sme_populate_pgd_data ppd; unsigned long pgtable_area_len; unsigned long decrypted_base; - if (!sme_active()) - return; - /* * Prepare for encrypting the kernel and initrd by building new * pagetables with the necessary attributes needed to encrypt the @@ -358,17 +372,17 @@ void __init sme_encrypt_kernel(struct boot_params *bp) * pagetables and when the new encrypted and decrypted kernel * mappings are populated. */ - ppd.pgtable_area = (void *)execute_end; + ppd->pgtable_area = (void *)execute_end; /* * Make sure the current pagetable structure has entries for * addressing the workarea. */ - ppd.pgd = (pgd_t *)native_read_cr3_pa(); - ppd.paddr = workarea_start; - ppd.vaddr = workarea_start; - ppd.vaddr_end = workarea_end; - sme_map_range_decrypted(&ppd); + ppd->pgd = (pgd_t *)native_read_cr3_pa(); + ppd->paddr = workarea_start; + ppd->vaddr = workarea_start; + ppd->vaddr_end = workarea_end; + sme_map_range_decrypted(ppd); /* Flush the TLB - no globals so cr3 is enough */ native_write_cr3(__native_read_cr3()); @@ -379,9 +393,9 @@ void __init sme_encrypt_kernel(struct boot_params *bp) * then be populated with new PUDs and PMDs as the encrypted and * decrypted kernel mappings are created. */ - ppd.pgd = ppd.pgtable_area; - memset(ppd.pgd, 0, sizeof(pgd_t) * PTRS_PER_PGD); - ppd.pgtable_area += sizeof(pgd_t) * PTRS_PER_PGD; + ppd->pgd = ppd->pgtable_area; + memset(ppd->pgd, 0, sizeof(pgd_t) * PTRS_PER_PGD); + ppd->pgtable_area += sizeof(pgd_t) * PTRS_PER_PGD; /* * A different PGD index/entry must be used to get different @@ -399,75 +413,109 @@ void __init sme_encrypt_kernel(struct boot_params *bp) decrypted_base <<= PGDIR_SHIFT; /* Add encrypted kernel (identity) mappings */ - ppd.paddr = kernel_start; - ppd.vaddr = kernel_start; - ppd.vaddr_end = kernel_end; - sme_map_range_encrypted(&ppd); + ppd->paddr = kernel_start; + ppd->vaddr = kernel_start; + ppd->vaddr_end = kernel_end; + sme_map_range_encrypted(ppd); /* Add decrypted, write-protected kernel (non-identity) mappings */ - ppd.paddr = kernel_start; - ppd.vaddr = kernel_start + decrypted_base; - ppd.vaddr_end = kernel_end + decrypted_base; - sme_map_range_decrypted_wp(&ppd); + ppd->paddr = kernel_start; + ppd->vaddr = kernel_start + decrypted_base; + ppd->vaddr_end = kernel_end + decrypted_base; + sme_map_range_decrypted_wp(ppd); if (initrd_len) { /* Add encrypted initrd (identity) mappings */ - ppd.paddr = initrd_start; - ppd.vaddr = initrd_start; - ppd.vaddr_end = initrd_end; - sme_map_range_encrypted(&ppd); + ppd->paddr = initrd_start; + ppd->vaddr = initrd_start; + ppd->vaddr_end = initrd_end; + sme_map_range_encrypted(ppd); /* * Add decrypted, write-protected initrd (non-identity) mappings */ - ppd.paddr = initrd_start; - ppd.vaddr = initrd_start + decrypted_base; - ppd.vaddr_end = initrd_end + decrypted_base; - sme_map_range_decrypted_wp(&ppd); + ppd->paddr = initrd_start; + ppd->vaddr = initrd_start + decrypted_base; + ppd->vaddr_end = initrd_end + decrypted_base; + sme_map_range_decrypted_wp(ppd); } /* Add decrypted workarea mappings to both kernel mappings */ - ppd.paddr = workarea_start; - ppd.vaddr = workarea_start; - ppd.vaddr_end = workarea_end; - sme_map_range_decrypted(&ppd); + ppd->paddr = workarea_start; + ppd->vaddr = workarea_start; + ppd->vaddr_end = workarea_end; + sme_map_range_decrypted(ppd); - ppd.paddr = workarea_start; - ppd.vaddr = workarea_start + decrypted_base; - ppd.vaddr_end = workarea_end + decrypted_base; - sme_map_range_decrypted(&ppd); + ppd->paddr = workarea_start; + ppd->vaddr = workarea_start + decrypted_base; + ppd->vaddr_end = workarea_end + decrypted_base; + sme_map_range_decrypted(ppd); - /* Perform the encryption */ - sme_encrypt_execute(kernel_start, kernel_start + decrypted_base, - kernel_len, workarea_start, (unsigned long)ppd.pgd); + wa->kernel_start = kernel_start; + wa->kernel_end = kernel_end; + wa->kernel_len = kernel_len; - if (initrd_len) - sme_encrypt_execute(initrd_start, initrd_start + decrypted_base, - initrd_len, workarea_start, - (unsigned long)ppd.pgd); + wa->initrd_start = initrd_start; + wa->initrd_end = initrd_end; + wa->initrd_len = initrd_len; + + wa->workarea_start = workarea_start; + wa->workarea_end = workarea_end; + wa->workarea_len = workarea_len; + + wa->decrypted_base = decrypted_base; +} +static void __init teardown_workarea_map(struct sme_workarea_data *wa, + struct sme_populate_pgd_data *ppd) +{ /* * At this point we are running encrypted. Remove the mappings for * the decrypted areas - all that is needed for this is to remove * the PGD entry/entries. */ - ppd.vaddr = kernel_start + decrypted_base; - ppd.vaddr_end = kernel_end + decrypted_base; - sme_clear_pgd(&ppd); - - if (initrd_len) { - ppd.vaddr = initrd_start + decrypted_base; - ppd.vaddr_end = initrd_end + decrypted_base; - sme_clear_pgd(&ppd); + ppd->vaddr = wa->kernel_start + wa->decrypted_base; + ppd->vaddr_end = wa->kernel_end + wa->decrypted_base; + sme_clear_pgd(ppd); + + if (wa->initrd_len) { + ppd->vaddr = wa->initrd_start + wa->decrypted_base; + ppd->vaddr_end = wa->initrd_end + wa->decrypted_base; + sme_clear_pgd(ppd); } - ppd.vaddr = workarea_start + decrypted_base; - ppd.vaddr_end = workarea_end + decrypted_base; - sme_clear_pgd(&ppd); + ppd->vaddr = wa->workarea_start + wa->decrypted_base; + ppd->vaddr_end = wa->workarea_end + wa->decrypted_base; + sme_clear_pgd(ppd); /* Flush the TLB - no globals so cr3 is enough */ native_write_cr3(__native_read_cr3()); } +void __init sme_encrypt_kernel(struct boot_params *bp) +{ + struct sme_populate_pgd_data ppd; + struct sme_workarea_data wa; + + if (!sme_active()) + return; + + build_workarea_map(bp, &wa, &ppd); + + /* When SEV is active, encrypt kernel and initrd */ + sme_encrypt_execute(wa.kernel_start, + wa.kernel_start + wa.decrypted_base, + wa.kernel_len, wa.workarea_start, + (unsigned long)ppd.pgd); + + if (wa.initrd_len) + sme_encrypt_execute(wa.initrd_start, + wa.initrd_start + wa.decrypted_base, + wa.initrd_len, wa.workarea_start, + (unsigned long)ppd.pgd); + + teardown_workarea_map(&wa, &ppd); +} + void __init sme_enable(struct boot_params *bp) { const char *cmdline_ptr, *cmdline_arg, *cmdline_on, *cmdline_off; From patchwork Tue Sep 4 01:29:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10586439 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 221B0112B for ; Tue, 4 Sep 2018 01:30:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12C4128BC9 for ; Tue, 4 Sep 2018 01:30:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 066D228BDE; Tue, 4 Sep 2018 01:30:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DECBA28BC9 for ; Tue, 4 Sep 2018 01:30:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726102AbeIDFxV (ORCPT ); Tue, 4 Sep 2018 01:53:21 -0400 Received: from mail-bl2nam02on0082.outbound.protection.outlook.com ([104.47.38.82]:42369 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725990AbeIDFxU (ORCPT ); Tue, 4 Sep 2018 01:53:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BdJUFBsYLNgYsE/wjTdOS5mKAPWVjyMJBQGIgs0NlnI=; b=1Pxc4Ykr0nDMuunsJYmepDMDq+wI77lS58PU0xoK5HybJs0FLI2oQttzztkbfyt3UwW6vGfmgm8iOSJwURhnP3oYlBOTssDyPcv8ZUGChrAV5/9CKiLKFdm0lpiIMW5q/LBUT1ax1VQf+6VErTbkBQgMX/PokR5LYkKZSENB0AE= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1101.18; Tue, 4 Sep 2018 01:30:00 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Brijesh Singh , Tom Lendacky , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Sean Christopherson , =?utf-8?b?UmFkaW0g?= =?utf-8?b?S3LEjW3DocWZ?= Subject: [PATCH v4 2/4] x86/mm: fix sme_populate_pgd() to update page flags Date: Mon, 3 Sep 2018 20:29:40 -0500 Message-Id: <1536024582-25700-3-git-send-email-brijesh.singh@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> References: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN6PR20CA0066.namprd20.prod.outlook.com (2603:10b6:404:151::28) To SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9855e9b2-ee8f-4d0a-32ba-08d61205ef66 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:SN6PR12MB2687; X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;3:SgYDL3VQNE6Fo5lJBDHVVwU2FeSRGhAxlo2gZ5rwa6bq0JCMxrS6bDuTy+bKys3Pl7nYkyapt5+yyK5IvtLFVcDTH0AHd1a4XBjac/fK29yReEQUf9KbMcwBpQ+Iu4V+28w5QcPNYyd5OZ7DpCY/FaN9jLHua5A1TnDXW0H4XPrjxEGg+sQPbIUBnNvO+Xk0/KUhjaQcPQyAkG1CMDuXnDyR8tlRXHZojU/Lnne2fvh1vwUi6caP7tlSy6xHbw0N;25:+YdFcpb4IdwvuCHyqr6TmFVD19e2NWefNKvrt0FH8ydI5hL7Ai8fRMSD/gHHNV2sgg0w+yvilj79GxBP34nX7LlG3NlbbItBdhd68luxwXBQOvvicyluuXrfDqvFmWbmBcsCVmbbaNfotsILn06VMN7lQlnrEuVNTcCoeUqGw/r/2wzsBCfR7Pc6i2DwNUcMEOZEiSBul2BbmYUoyTYb1+kM1WU4wVLY+0KhUeHo6S9ixSX6FF00HfQXyOIxVOs6EH8hqXR5S1Lz0iZltdoc1Moq/F+7L59ap1cuzUhrpk73II6iyER28GcBE5D+BS7KH7fC7N3rJk7hYGg/GW0U6Q==;31:hZCjY0nli7LnYZ8QNtLq66VcbKYZFgg/lNQ8+22N2kGbJyOTFeXW9y3VVD4AT7ZqKw1pZjgAAnbwRMOEDsxeIzHB8xdtFmvbuqYV7CUIDZpYCZ0CBqeRvcG5pKB8roFH0q+vIQwYLd0tVArQsm8TvuN0iIl8rddin72EQMXsfTZUdvims+q8e6oiVgAhbChxgltwN4VH8Uua5V3Wrq+0d3744xA7Uuf3x9pwfIH3YOY= X-MS-TrafficTypeDiagnostic: SN6PR12MB2687: X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:TfEGx8H0QQxwrzJDMpLWQ6ZG3SeOCOeeh1e+FWaS2U7DegHgr1+uiTTlAAUfF2a0Z7D+/9udm2uweDIghpchwlqD1oG2NXAmzyfhr1yJaRLfg3+N04KOGXlW510lX7a4SxLmht0JplTh/wYOzb633YCewDY3LyFkfluLnFi1ZbJoADophJ3o8SO1Miz7rLKwR9iNnER4TET1Bib8qhRlPAJJP0ImYWhsrp2mGc6RYBZIWGCOj2sCqYGPdALKLmjXEGS0e6c4PPxmzEUcBv530VTltHB3bwns8d1Vvzfj7ZBW4f8TZHDWdKnwedrR1XxvyWtE6w5G0rCgW+I0DeCkLYthBlJPdcszbAARtAYqRYSnNsi5+TSQjhYAsbmxEnECjUKNh0EK3oFNc7wjO20kuqU1drhdzM0AFipokaew3G4dM5JH0rtKBMa4LSG2Ks2zLHtmh9NYiiRpGWy46cHrC+28MDOf2G3vrUZiSCWqCrCcKLsM5ae2pGNxFLxx8Bvn;4:dIqmSQXwCG22Y9UJPE6+rkmxQAnntHqPZklHlP+DdaHBeRqaPeRMVLnhGrmXyaMPAuC88m+x5BDRoVAoMTzd9QjK7klFeGT8kK+GIvIbbObKKCeuCgVN3gO2+SMHsONTVSwWPUzo44PzeleHb6Hud52tX8dSQC7cVg1//Lx4ZWWYLGiIH48Qe1ANM7DFMWDv7tAddCmouiKnhyEja8hJanMOwxcvGz4/MvhkLL3brTHpWA48Cqxetw82gANSzJcMxL7MbztZljRKR8eeAHlZahg5T91bCM+c9s7AsaO3yaUEJX6dYp5JF9rxa0YO3c7Wz8S7TdPS+4E/ib5PwrSweVE4znxE94U+ZNT/FjkRoUpK15jYkI2knNgtitLCYFxD X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3231311)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:SN6PR12MB2687;BCL:0;PCL:0;RULEID:;SRVR:SN6PR12MB2687; X-Forefront-PRVS: 0785459C39 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(366004)(396003)(39860400002)(136003)(376002)(346002)(199004)(189003)(14444005)(6116002)(105586002)(76176011)(7696005)(47776003)(23676004)(66066001)(15650500001)(68736007)(52116002)(2870700001)(53936002)(53416004)(2906002)(316002)(36756003)(3846002)(97736004)(6486002)(50226002)(4326008)(25786009)(54906003)(106356001)(81166006)(81156014)(8936002)(478600001)(2616005)(6666003)(50466002)(86362001)(8676002)(44832011)(5660300001)(186003)(7736002)(305945005)(16526019)(386003)(486006)(476003)(26005)(446003)(11346002)(956004);DIR:OUT;SFP:1101;SCL:1;SRVR:SN6PR12MB2687;H:sbrijesh-desktop.amd.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2687=3B23=3AbwbQHQb?= =?utf-8?q?AfvxnyXiUjekTxTOEfviPpiQywx5NqObW4cE2TZ7Lut52EQmkrCrpo9uvpIdFBlqg?= =?utf-8?q?7SdxpBjrS3tJtto3OZpOyaOMrVkcf+GG+Vb9D1EJq19m1hVQRICL1Chpe/sy6HTSQ?= =?utf-8?q?2N1mVng/7nOjyjyq9wRCC7BbOtb4XOnv0Hu5ljn0wZcD0TzLaoKCJA8uihRrD9Kfa?= =?utf-8?q?rUwXo0DDuFZeE8JAXbXxsyk2AWUTNfkBIC8bMphDP9Axq61XGxuIDRIcb5b8Ri0J/?= =?utf-8?q?KbDdmiFf+M5AYJjUY5R+hsrpNfgJApJG7z2GUgnf9KOhMT1NU1Bn4aajuRzzr3H03?= =?utf-8?q?+pPRL5t8KVMZrG5lBhAY2qQMwqTeuBF0h0uxxSk5tDyKeYS5A2JSDfSDoD9ve1AqZ?= =?utf-8?q?U72WwJ/KQK20yWu6AbebLJej1GZnSWVsvqg39MhWUcc+P4VZfLQLDGjiAro0slqin?= =?utf-8?q?GYTFYW6sjntpZrop3+ge6ezYezxxIzBDKcGeBXUirBASPV7BdPX4GE4TguE1pKf8g?= =?utf-8?q?l89EPnO3+xgBw500RfROiu5E4JDDkpnHYV/dMJj2DE59cfnennyZxGtL9wNlh69D2?= =?utf-8?q?/EQkPprTzyoKZsOrDwnmUXKE7mn+YJ74riv32vIgtUjtBswgtwVZ9B79aH8JIOCJ4?= =?utf-8?q?nvcHg41AvRer7inNeJw5xNAswVywOggzTeNq8oU+kcPEK4wTvfTYIo2GmPg0Vxv9l?= =?utf-8?q?0VMbN3Ipa3hTUbMdm7XqePWv9ci0td/ZJ9o+IHdEQHWXQnhK5+3X6kYQMlbX8DvS1?= =?utf-8?q?KZ5E5rom4MXuc4RYm5GVkupCAnwF25Gf3nhJerb3U5C4sdggbleSGitZAAY0NPn9K?= =?utf-8?q?8AC6A1By470h2nHy5ERwY2KKxgVf5P3yQFKDOIk6u/FEUD3HrxV17sxEBLrTn1JQ6?= =?utf-8?q?G9NTNVeZRQSPkU4Pjvs6/RAe2IdVbdiMdb9Nik2NfgBt+m5oEXDXXBOEabkw14vy7?= =?utf-8?q?QP3s3gODb9qZbUwaXgnq+LFYFFLijU1cexvJSIYMooryQgZrJkZhXvUi5TfWIpT2/?= =?utf-8?q?76AzgZwjmWRtIZoNkf2NrbnbJtOfO5AogHSBO3drSNIScKjVj7krvMdhSDVkjo6qp?= =?utf-8?q?99zukNPQ2wx/6MTt/87M+FjP8gS4CPXG2703r3lT3mOrxShUydmDOTcvgFZ4IbBus?= =?utf-8?q?pmBn4mLG31yPL/aDnyb6i44NAZGHqhMXTmVW9YWbIXnlzKoNQKdwwjU6rogjQ=3D?= =?utf-8?q?=3D?= X-Microsoft-Antispam-Message-Info: qs7q7nCLJ1r7vMDjzQG/437ZfgVyI48O6zRkwDgWYADtOXhPs492W4lmj6x8oPAmj+rXM/lNauyDvtEaZn+HmmOeFe+Zd3GV6D2bOcOdZpFvbXuWa3cr6pzX1VowoyBgK2gD4URaFPmokocJiCNphIsQ4fpVyF4tVjBryRJ3RSjHag4LrK3YQ0q3aWY8OUQImQA/DyB2mK4eO8941v/tKt1A8G/IFnvdAGsSd35y6xNAlikWP9cjGQjY7rlXIx6X9CQ9e+LBExXuWDuyWsW2pH3Qo4PJcQv9Vkpb52PeiEJ1dF686bIylE3cm2wV9cVWk7vr5eWglQZcaNYz7zHjsq3x+1EdUPRz+7JwYZaUGgY= X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;6:yYtIS+78pIeK+vGdh9raOX9HvYdbybK6NQSlRvXU0XO3xNI9vjNJXQuITWPSA+V2sHkxWcZZVI4CcZ1uBzYSKv77uzdKNIxaW9OjV/PqMpy9oM3YqDe+gv6/GEe1twsPOZ8a6Z4/TVuwFPjISzik3yVEef+LidnjS+URCwEsQCoy0iNXrIWxSQbpRN72lbSDjMRZY/+1otsrNmotXWgusXd+vaj1ruF9cCD4y3QUwTH0kd3tuaHywXWJoNyHTv9cy1xIaUejhxHt+U/LshUrncKsRtlOqbuRNCzuOhKuwy/h/bl1lFSIiMteOEQjhi/G7uswwcprRMJ6XTPar3mOkqLX7iZZ3AAqGTOjf2bAH27dm9E7nuDK1Hs7m0ScWN5TZtl1kLAue62Tg+g+NdjxrlKxuj6aQ1gabYGEUvCm1ybxv+MHPpIKyWIQKGyPKM3QBOx5DtyriIBcASIqPfIM2g==;5:GSFKWfFFvFLpr1goQT8qnFBsli6V7+8eW0aYdHFQY7a79jyTb/rQs4WmisuhTLq0HWJkaOCtubxhqgJRs1I59LB59IPAvMe0ElOimkfbGlGmlH3ALZqm/NqEq9ZHZ+sl67Ox/DrZDYIJjGmBfHi23m3cufI1bS5xUbGrNp/PHiM=;7:KLPPNaxw7cL/gMdmi/H9KaNY6gLT1dd5BBhwJyQhOtOnrL7338UfUaQZZfmR3F2o3EVTUYmuboyMNPTM7977xJdE9LYqU/JjA2iWmBBealegPspIqommwMmHgQA5Xqwte3W+p5kO1VxJ3Hj5waujkSeX5hhmHnql7GIcPDIECxeiZpQKrddF8dQyuB4bTDUWXTvTg+AoFLnP4r8eMrztyplANljFFZY6T90aZx9mOasD4PNbtZepKMUFhR2XUtvh SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:y0Cz1I6xWKjPbUoca07Fkk3bvCw4G2lKRk7s7K3hcCTni60dHsg0uRi/IQ4MsjVPxXOFcq0EZic02pQi3PT5LOVBpWANzq+m+9MGqBS4BPvgXb689iTKybGThyB7PJZd8A9fzldqY9xKGvHIJL41nk+mBn2NhezfgYgqmc8JyUZ7Tk0UrDiLuMPB/0G3IvVd+3Wx3Ybfl0HRkQzuGhkij3NlccbHJ3bTjCettdWTRemhmNYBYTOwroTRvAPrPD/A X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2018 01:30:00.9936 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9855e9b2-ee8f-4d0a-32ba-08d61205ef66 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2687 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fix sme_populate_pgd() to update page flags if the PMD/PTE entry already exists. Signed-off-by: Brijesh Singh Reviewed-by: Tom Lendacky Cc: Tom Lendacky Cc: kvm@vger.kernel.org Cc: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Cc: Paolo Bonzini Cc: Sean Christopherson Cc: kvm@vger.kernel.org Cc: "Radim Krčmář" --- arch/x86/mm/mem_encrypt_identity.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 92265d3..7659e65 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -154,9 +154,6 @@ static void __init sme_populate_pgd_large(struct sme_populate_pgd_data *ppd) return; pmd = pmd_offset(pud, ppd->vaddr); - if (pmd_large(*pmd)) - return; - set_pmd(pmd, __pmd(ppd->paddr | ppd->pmd_flags)); } @@ -182,8 +179,7 @@ static void __init sme_populate_pgd(struct sme_populate_pgd_data *ppd) return; pte = pte_offset_map(pmd, ppd->vaddr); - if (pte_none(*pte)) - set_pte(pte, __pte(ppd->paddr | ppd->pte_flags)); + set_pte(pte, __pte(ppd->paddr | ppd->pte_flags)); } static void __init __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) From patchwork Tue Sep 4 01:29:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10586437 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D13A112B for ; Tue, 4 Sep 2018 01:30:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DCB928BC9 for ; Tue, 4 Sep 2018 01:30:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 813CA28BDE; Tue, 4 Sep 2018 01:30:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 678B028BC9 for ; Tue, 4 Sep 2018 01:30:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726208AbeIDFxY (ORCPT ); Tue, 4 Sep 2018 01:53:24 -0400 Received: from mail-bl2nam02on0082.outbound.protection.outlook.com ([104.47.38.82]:42369 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725833AbeIDFxX (ORCPT ); Tue, 4 Sep 2018 01:53:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1kAZqKdOFTzMMAAeVXWLDFW+lmC15xXpNRUUXTogFWM=; b=PIkCkJo6vsJYRGlMGVMrxE5m2rrypkTuyFirzQhoVmj4UPclY2fI9ldOV9ue5fu1oOCKFxMuDgxHi0k/xuZRvV7vDVls2+Z6s60Y5NSMchSNFnpqoSypIiIztb+HGAp8MjuDVIgiIjCpB4ehmb8Z+HzfbL1zKDyL9yIX/abT084= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1101.18; Tue, 4 Sep 2018 01:30:02 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Brijesh Singh , Tom Lendacky , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Sean Christopherson , =?utf-8?b?UmFkaW0g?= =?utf-8?b?S3LEjW3DocWZ?= Subject: [PATCH v4 3/4] x86/mm: add .data..decrypted section to hold shared variables Date: Mon, 3 Sep 2018 20:29:41 -0500 Message-Id: <1536024582-25700-4-git-send-email-brijesh.singh@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> References: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN6PR20CA0066.namprd20.prod.outlook.com (2603:10b6:404:151::28) To SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b2c463af-fa96-4294-c6ee-08d61205f03c X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:SN6PR12MB2687; X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;3:usJh4YIqEqCq2qAXP9/QS/fDX0sd+c8QoMQse1sECGc3q5B76rALpOp1ehjSpaDyHqP10hfMtaEADjCpNwDM2MuHMlDwXweacWQV0D+UTxYFWkVyNkrLoMzlM7NOcmwqcWUIgdbKHc3n6myEKGJ1ye35kAEviSoXpAYr4zTlXOH8IbDr+Jf+hxdPsa3Y1XjwRaWT5+22wLkVMreDwDqOyRlV1LPRtEYImBJMWhKnjjuip/dZWKgq9E6bwJQ7CqKH;25:yPdmTo+m1xtumMqK1Ykt93FGPxMtv30WpjvJnyIcjuU8DgKnsgrzitthJdm/PGPC2gId8AXKyGxnW0kSlcBCZxxF1OwlqLszAFugtst2Jw2x+y6LU2iv9tFY6Cl7h7SN1OUCqAXcJkvXrMfz1RQ0ILXDU25gwXvg9zodwBlWjReC1w7HITY9WW4yREoa59hLqtWaRFbn+P+oMxyv7cSUiebMKk0oihYEi4fq/mP+pOb286b4FonIaCc1COXI51xZGaUEcbmQab5z2RfDnUfkLL3ZOnSSPZ6n1oWIX34F7AyQUb6CrVV+C24aitCDgPwukV4CYqxlOxFB3WJFzgmYTQ==;31:iOIHIRHaPZbJMom5XPnHGvdjdljd22xWg/TNXw7zXpRdygIhAG9OCNkYiXVegUb7+yXMPoqhLogUtehJP4qARw9RpZeymbjFic++fDu4w3jjJbd1zYaqK6LSMtlVRJc1ymvgsUcT8AYQW8T93mmhe6aW138NuKl1KxUMXglcyGgeu2FQlp70uVxy6vwc/TRcXmgCgy2n8YB8RPXK7ZgsW8MgfN1nQ0keUIS9Jra89UI= X-MS-TrafficTypeDiagnostic: SN6PR12MB2687: X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:egjf6WYfOr1DX03v8IogGhHIkCTxp+/xuMvBkt7VJ2MJ68h3+FkjHJ6pWrmnBhFhL+JT15B+PWEf/2ol5g+irGMAK04V6vEZZ1EEZqd3KMf5gRfV/+M7lhU6TWcOF3F9vZ/u2JymbSA56FA+GO0GEhaR2X4ytBZ8n/uQA0xiwEhtu/1/dWSwfBhUmk0RG/lRvvENOp0XJ1AYTivkoeE2cLYCzkRJ2/ZM74Mdfv4O67d6DE8jo+mr05j2PrUTZVi6BTlYFe8iEMbZE3qUgFBgg/DvMfw5MiluFUsoPDsOcAd1pRaVcSSd+N/d+uttYqVls/uyUyyK8/Ep2e0uIpqhCmki6cne3Y9qi/NQUfeNO40uvLW1lKFMzWbaEC/Ap8CnqzSYOOKn4hhicHCT4BVWEvsiZ7XjaCbsJT5twl3uEcZOF0dI5Tv1XoCGS/jy/cNchyshdu3FZz1MkTsy4QdZuGB9WTNhZLt+hqpJhMSWtbLMOKPmVeKqJeE2/Vy1ftqJ;4:T028tw0En/A94Xo0hviVjZJqJElFwL1BxtK7reFr4oy+StseiUcfvpdwVy7i+zQKSP8Kj4bdyXdhlZXubw46smNRnDofJ+mtiwpEJwH+IxAykdB6zrmzqvOVVj9OhmCGKSzTzDIMBmtMdow5Zx0N0eGu1TQ8khvqOyKilGP72kesJTkYdjH7Qg7J5mujQSPK6kM8VY8xxAw/5w9ZkFNaHzwauR1DeJX6ogye2zUw9ZsRta5KqS1Fz3NPTg/MJVyRw5sje6c8b+FZ6xvBw1Zfjoh5Yog/vZBg+/L+WNQ1FGmtDfK0rw3I8Ou/UonspcFMgrYeVGYnowC1m1Ve8TU3YIns+MfnKKZ96/4UbUYIP6MZuJbTbli3KL/T6UWQ3RIakH40ZE1f4Qn2fr8fBnQMow== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(163750095850)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3231311)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:SN6PR12MB2687;BCL:0;PCL:0;RULEID:;SRVR:SN6PR12MB2687; X-Forefront-PRVS: 0785459C39 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(366004)(396003)(39860400002)(136003)(376002)(346002)(199004)(189003)(6116002)(105586002)(76176011)(7696005)(47776003)(23676004)(66066001)(68736007)(52116002)(2870700001)(53936002)(53416004)(2906002)(316002)(36756003)(3846002)(97736004)(6486002)(50226002)(4326008)(25786009)(54906003)(106356001)(81166006)(81156014)(8936002)(478600001)(2616005)(6666003)(50466002)(575784001)(86362001)(8676002)(44832011)(5660300001)(186003)(7736002)(305945005)(16526019)(386003)(486006)(476003)(26005)(446003)(11346002)(956004)(142923001)(101420200001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN6PR12MB2687;H:sbrijesh-desktop.amd.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2687=3B23=3ApusQfhV?= =?utf-8?q?SQbz+DgWWwcnhE63WcfFKEEZLTcTN60y4ccRYNKlFoLPn4gHyKQlkuLp/6H/yCewi?= =?utf-8?q?pKIOr+ON4IY0sf0QKuPxJAZkjR5tQY+4pqRAeecIHYelRBkeU2IvZoTTXPpsMwqzs?= =?utf-8?q?9s+gxf1K6QFXtoGD5BJ7XgrXqSxgzaUcNgV0owIZuEhjjVgTBXyLXOLV1p63yN4Nv?= =?utf-8?q?zQwdsrGV3w2qnXPD1WMX0u2daBU5WTKOgJuPAmF5UouIEzueEo+zt5S3ugEIpcMjq?= =?utf-8?q?HIBMHupYQdwCHT+9Z8sxuH9sueWvjFR4SP4MZr1Vk2KVo9DDS0XUaZHNa6fQ1ngDj?= =?utf-8?q?1/LxmoFu837tcp3vhjHwp7Lhn7ziFced2zhue3RkfXgNX6nny017cQsFjHQb8ZYL1?= =?utf-8?q?00IWw5aKB8+8s7ovqwW0Wa3kfboGHVs81iLomyU1MWs9+vxwwY6XHcgfJtdq2Rj2B?= =?utf-8?q?NGHpI7hEaRLgyx4Tl1A63u+Q8gT0PKKnT32xEaABJV9JFgnVl0W133jrysAmmlKTi?= =?utf-8?q?IUXRPu4TdLIKzQ22GVS8yPn7Nxwt7iu+kOC05zog+wfagB8OytpzdBpsrE5xCG5s1?= =?utf-8?q?tQwZEOD4wDn790I6m/5USTlLYusu9TSi7RG8s6O0CnKPRAZt2gLnnMQWEJ93HVSLp?= =?utf-8?q?KcHXwiqg7AzpuE9kM8g7cVgP6Bj8Ri9homPv0XKlGbUGYv7rIiz+CDw482bsM4Yif?= =?utf-8?q?tQhK4ifjF1yxjt5RRNFS/00t0iBczPeBroaaUF2sBn8DBPaDylytJafYyzuJDUs/W?= =?utf-8?q?RoDkk+M4/o2e34xhCgbvrD78/u7B02dq9sPvm32/XGCrEhtFBZeu3+Yp/JbTBq8NP?= =?utf-8?q?eteHWMqFpT7xLOzWzkLKmvDcgVI62j5SCNgDr4mbw5P8tO7zNiS5FEYTX2Ve4zDks?= =?utf-8?q?XV5xckCiIeymKVoeb9sc5AKNQsXI3t6BHcV7stDPgFoXeM+2N18HHZz1VJanyVFzy?= =?utf-8?q?SuQ6L9c/LfsKQRr3V9/kkLCO08nMrf4jw7te9OA4DTBfrqnWwbH4y+KlehjDX4P00?= =?utf-8?q?n6zY5zBbh63M0zUayGgbdb1jg+e5uCKniDtsOc7F5yY+n1+KqIrQ0Mw6eqGLp/EeU?= =?utf-8?q?4IoxY8Uv1cVoEOlneI7+QhtTifvggH5qVIHS26TD0emiBjrPyzVK90YDlPoBFEIWj?= =?utf-8?q?kvWy6WL6TjJlS8WhK9JoS6JlHimKWrsfKx1xyqVaB1i9/hE+W1pj+RL1l9EoOKMyT?= =?utf-8?q?qKkBr+fx2pX5LyC98=3D?= X-Microsoft-Antispam-Message-Info: rM4tN798/NHB1N7Q9hCpdiJ8cRg2dRzdBeMgM6QAaYzCf992D9mEQapMwKyjmrZlIVLrHks3IxbDeovyUWvqThYwKxkd/lB9ihSxb0qZhed+Dp9yV86AmzjOvkXol1rWdlUK/1hYYPCny7cxZxp7mYWRQOoxGx0LndlhRXTA9kDWst3tw5tJHBmeiCmTyZbAzkllz5UBbMXcnoKFrT4pikywFBVAB9AK3Oa0kBJI4bH12DdSdci5tYutVPoj9/BCvSttBbO308Sy4PSHnI+KEUeg5UY/4bYqfDDw19LnKDx34mQ3f3JYaW+dsC2eMjY7nLaB0DzUSb2pZdg8L7dZbth85fZmroO7ROKi6oWhytU= X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;6:JXstH2CRTN00D8BRujBVwn81AJ/4a5wdVFpUWiuLZNWMSfyl262sOUdHC/djerqaH1du7b4pF1FMRBGSoOcYe8ap3srOdzzY8LsAfqEXSfAWXwUPv/UQrCeTT1pvMTDas5bT6E6OvdzQMARZagcrpg99A+aFA8u66qql/OWjT/7ujNQLRjb5ByxB9NwatqD764jNNJl0pRbqO2k5LaZtAd7ydm9KQE2iYY7qhs6CHjJW6Sfey6Q7+6TcOanB4KNMm7HnhqdwVW3ZyCmp3qXmdwKrq9qCMGK2UjM+D2nHTSiShV+ztJNTLLDIZ0NCWiQoXmbQn5RHp8SE56gAp1GbZjb9H4kKbZ3h+WIPb+T7Iez+53WQTHVEf0qzUV9eL0bnZ6h/FO/HkZCo/kmP12cFCim1BqGWm2tKNvcpNdtEinxSNd0Rl9zH9OC+P1oqoWCH6MYJfES1Y8VOrBUYe7mGKw==;5:fyCz999zSJOdAo0VtzF7Z6WZQg7hAgGDsAKvfyfluTeevz6iWDcbpKrepabjO4KcdxBiDc02/nYOs4AEPhvS61VZyLa4EdESTJ9v4VdIttQqGcgLTjcOMzs2BL/JyYeKftSZj5xLxHzJ53B+q33Xj+KF2jsViMYU6gy8VbGWKww=;7:Qf3e3PXtSKC/Hty7HDAO0B4kdns1iZdqTL+GohK8wFFgvDZuwIX4Yfu0LiBOFilNturhnO+EfWDuAqN1/tZ/ePfeCwxchD80N+jBGRD37P0I/WGPDkkTvjfR7z9XbU5wLH1F1bQDEUDv95gecLUaoALwRuYeGHxGul0hdSUInXIBVyBm9tRHB8fO3FhJNDwcFktRYkumzMsa6E7m6KYDynEvTXyuhWqndKwF/kcc+FNAQcL8ZMB/hQQ83Ad41nEj SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:xFkLBY+brCvV9XiOWY8iw8q1Pxmgh4W//EDrgL0whovC5or4yUYq75Xq/QNnCJM8OkWtlyxtmlmFEftzWL1snTI6PgTh3ZtO+m4VAconezOyn1V3tvn68pKLm4UapdwatEWX5cAkbY2M62kuwppOnYabCH8Cmp1SQDlUgzrGeUiq3+/gF/33eantSC+sB+X4+209mv3x2++m8WBPLoNVuz5gRnxB7QJH8W+SzNOXwzMllI2Vwt1z8n69vOmzMn19 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2018 01:30:02.3998 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b2c463af-fa96-4294-c6ee-08d61205f03c X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2687 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP kvmclock defines few static variables which are shared with the hypervisor during the kvmclock initialization. When SEV is active, memory is encrypted with a guest-specific key, and if guest OS wants to share the memory region with hypervisor then it must clear the C-bit before sharing it. Currently, we use kernel_physical_mapping_init() to split large pages before clearing the C-bit on shared pages. But it fails when called from the kvmclock initialization (mainly because memblock allocator is not ready that early during boot). Add a __decrypted section attribute which can be used when defining such shared variable. The so-defined variables will be placed in the .data..decrypted section. This section is mapped with C=0 early during boot, we also ensure that the initialized values are updated to match with C=0 (i.e perform an in-place decryption). The .data..decrypted section is PMD-aligned and sized so that we avoid the need to split the large pages when mapping the section. The sme_encrypt_kernel() was used to perform the in-place encryption of the Linux kernel and initrd when SME is active. The routine has been enhanced to decrypt the .data..decrypted section for both SME and SEV cases. Signed-off-by: Brijesh Singh Reviewed-by: Tom Lendacky Cc: Tom Lendacky Cc: kvm@vger.kernel.org Cc: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Cc: Paolo Bonzini Cc: Sean Christopherson Cc: kvm@vger.kernel.org Cc: "Radim Krčmář" --- arch/x86/include/asm/mem_encrypt.h | 6 +++ arch/x86/kernel/head64.c | 11 +++++ arch/x86/kernel/vmlinux.lds.S | 17 +++++++ arch/x86/mm/mem_encrypt_identity.c | 94 ++++++++++++++++++++++++++++++++------ 4 files changed, 113 insertions(+), 15 deletions(-) diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index c064383..802b2eb 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -52,6 +52,8 @@ void __init mem_encrypt_init(void); bool sme_active(void); bool sev_active(void); +#define __decrypted __attribute__((__section__(".data..decrypted"))) + #else /* !CONFIG_AMD_MEM_ENCRYPT */ #define sme_me_mask 0ULL @@ -77,6 +79,8 @@ early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 0; static inline int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; } +#define __decrypted + #endif /* CONFIG_AMD_MEM_ENCRYPT */ /* @@ -88,6 +92,8 @@ early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; #define __sme_pa(x) (__pa(x) | sme_me_mask) #define __sme_pa_nodebug(x) (__pa_nodebug(x) | sme_me_mask) +extern char __start_data_decrypted[], __end_data_decrypted[]; + #endif /* __ASSEMBLY__ */ #endif /* __X86_MEM_ENCRYPT_H__ */ diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 8047379..af39d68 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -112,6 +112,7 @@ static bool __head check_la57_support(unsigned long physaddr) unsigned long __head __startup_64(unsigned long physaddr, struct boot_params *bp) { + unsigned long vaddr, vaddr_end; unsigned long load_delta, *p; unsigned long pgtable_flags; pgdval_t *pgd; @@ -234,6 +235,16 @@ unsigned long __head __startup_64(unsigned long physaddr, /* Encrypt the kernel and related (if SME is active) */ sme_encrypt_kernel(bp); + /* Clear the memory encryption mask from the .data..decrypted section. */ + if (mem_encrypt_active()) { + vaddr = (unsigned long)__start_data_decrypted; + vaddr_end = (unsigned long)__end_data_decrypted; + for (; vaddr < vaddr_end; vaddr += PMD_SIZE) { + i = pmd_index(vaddr); + pmd[i] -= sme_get_me_mask(); + } + } + /* * Return the SME encryption mask (if SME is active) to be used as a * modifier for the initial pgdir entry programmed into CR3. diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 8bde0a4..78d3169 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -89,6 +89,21 @@ PHDRS { note PT_NOTE FLAGS(0); /* ___ */ } +/* + * This section contains data which will be mapped as decrypted. Memory + * encryption operates on a page basis. Make this section PMD-aligned + * to avoid spliting the pages while mapping the section early. + * + * Note: We use a separate section so that only this section gets + * decrypted to avoid exposing more than we wish. + */ +#define DATA_DECRYPTED \ + . = ALIGN(PMD_SIZE); \ + __start_data_decrypted = .; \ + *(.data..decrypted); \ + . = ALIGN(PMD_SIZE); \ + __end_data_decrypted = .; \ + SECTIONS { #ifdef CONFIG_X86_32 @@ -171,6 +186,8 @@ SECTIONS /* rarely changed data like cpu maps */ READ_MOSTLY_DATA(INTERNODE_CACHE_BYTES) + DATA_DECRYPTED + /* End of data section */ _edata = .; } :data diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 7659e65..08e70ba 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -51,6 +51,8 @@ (_PAGE_PAT | _PAGE_PWT)) #define PMD_FLAGS_ENC (PMD_FLAGS_LARGE | _PAGE_ENC) +#define PMD_FLAGS_ENC_WP ((PMD_FLAGS_ENC & ~_PAGE_CACHE_MASK) | \ + (_PAGE_PAT | _PAGE_PWT)) #define PTE_FLAGS (__PAGE_KERNEL_EXEC & ~_PAGE_GLOBAL) @@ -59,6 +61,8 @@ (_PAGE_PAT | _PAGE_PWT)) #define PTE_FLAGS_ENC (PTE_FLAGS | _PAGE_ENC) +#define PTE_FLAGS_ENC_WP ((PTE_FLAGS_ENC & ~_PAGE_CACHE_MASK) | \ + (_PAGE_PAT | _PAGE_PWT)) struct sme_populate_pgd_data { void *pgtable_area; @@ -231,6 +235,11 @@ static void __init sme_map_range_encrypted(struct sme_populate_pgd_data *ppd) __sme_map_range(ppd, PMD_FLAGS_ENC, PTE_FLAGS_ENC); } +static void __init sme_map_range_encrypted_wp(struct sme_populate_pgd_data *ppd) +{ + __sme_map_range(ppd, PMD_FLAGS_ENC_WP, PTE_FLAGS_ENC_WP); +} + static void __init sme_map_range_decrypted(struct sme_populate_pgd_data *ppd) { __sme_map_range(ppd, PMD_FLAGS_DEC, PTE_FLAGS_DEC); @@ -378,7 +387,10 @@ static void __init build_workarea_map(struct boot_params *bp, ppd->paddr = workarea_start; ppd->vaddr = workarea_start; ppd->vaddr_end = workarea_end; - sme_map_range_decrypted(ppd); + if (sev_active()) + sme_map_range_encrypted(ppd); + else + sme_map_range_decrypted(ppd); /* Flush the TLB - no globals so cr3 is enough */ native_write_cr3(__native_read_cr3()); @@ -435,16 +447,27 @@ static void __init build_workarea_map(struct boot_params *bp, sme_map_range_decrypted_wp(ppd); } - /* Add decrypted workarea mappings to both kernel mappings */ + /* + * When SEV is active, kernel is already encrypted hence mapping + * the initial workarea_start as encrypted. When SME is active, + * the kernel is not encrypted hence add decrypted workarea + * mappings to both kernel mappings. + */ ppd->paddr = workarea_start; ppd->vaddr = workarea_start; ppd->vaddr_end = workarea_end; - sme_map_range_decrypted(ppd); + if (sev_active()) + sme_map_range_encrypted(ppd); + else + sme_map_range_decrypted(ppd); ppd->paddr = workarea_start; ppd->vaddr = workarea_start + decrypted_base; ppd->vaddr_end = workarea_end + decrypted_base; - sme_map_range_decrypted(ppd); + if (sev_active()) + sme_map_range_encrypted(ppd); + else + sme_map_range_decrypted(ppd); wa->kernel_start = kernel_start; wa->kernel_end = kernel_end; @@ -487,28 +510,69 @@ static void __init teardown_workarea_map(struct sme_workarea_data *wa, native_write_cr3(__native_read_cr3()); } +static void __init decrypt_shared_data(struct sme_workarea_data *wa, + struct sme_populate_pgd_data *ppd) +{ + unsigned long decrypted_start, decrypted_end, decrypted_len; + + /* Physical addresses of decrypted data section */ + decrypted_start = __pa_symbol(__start_data_decrypted); + decrypted_end = ALIGN(__pa_symbol(__end_data_decrypted), PMD_PAGE_SIZE); + decrypted_len = decrypted_end - decrypted_start; + + if (!decrypted_len) + return; + + /* Add decrypted mapping for the section (identity) */ + ppd->paddr = decrypted_start; + ppd->vaddr = decrypted_start; + ppd->vaddr_end = decrypted_end; + sme_map_range_decrypted(ppd); + + /* Add encrypted-wp mapping for the section (non-identity) */ + ppd->paddr = decrypted_start; + ppd->vaddr = decrypted_start + wa->decrypted_base; + ppd->vaddr_end = decrypted_end + wa->decrypted_base; + sme_map_range_encrypted_wp(ppd); + + /* Perform in-place decryption */ + sme_encrypt_execute(decrypted_start, + decrypted_start + wa->decrypted_base, + decrypted_len, wa->workarea_start, + (unsigned long)ppd->pgd); + + ppd->vaddr = decrypted_start + wa->decrypted_base; + ppd->vaddr_end = decrypted_end + wa->decrypted_base; + sme_clear_pgd(ppd); +} + void __init sme_encrypt_kernel(struct boot_params *bp) { struct sme_populate_pgd_data ppd; struct sme_workarea_data wa; - if (!sme_active()) + if (!mem_encrypt_active()) return; build_workarea_map(bp, &wa, &ppd); - /* When SEV is active, encrypt kernel and initrd */ - sme_encrypt_execute(wa.kernel_start, - wa.kernel_start + wa.decrypted_base, - wa.kernel_len, wa.workarea_start, - (unsigned long)ppd.pgd); - - if (wa.initrd_len) - sme_encrypt_execute(wa.initrd_start, - wa.initrd_start + wa.decrypted_base, - wa.initrd_len, wa.workarea_start, + /* When SME is active, encrypt kernel and initrd */ + if (sme_active()) { + sme_encrypt_execute(wa.kernel_start, + wa.kernel_start + wa.decrypted_base, + wa.kernel_len, wa.workarea_start, (unsigned long)ppd.pgd); + if (wa.initrd_len) + sme_encrypt_execute(wa.initrd_start, + wa.initrd_start + wa.decrypted_base, + wa.initrd_len, wa.workarea_start, + (unsigned long)ppd.pgd); + } + + /* Decrypt the contents of .data..decrypted section */ + decrypt_shared_data(&wa, &ppd); + teardown_workarea_map(&wa, &ppd); } From patchwork Tue Sep 4 01:29:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10586435 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69A7D920 for ; Tue, 4 Sep 2018 01:30:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E19E28BD5 for ; Tue, 4 Sep 2018 01:30:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3F7DE28BE4; Tue, 4 Sep 2018 01:30:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9E3FA28BD5 for ; Tue, 4 Sep 2018 01:30:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726280AbeIDFxZ (ORCPT ); Tue, 4 Sep 2018 01:53:25 -0400 Received: from mail-bl2nam02on0082.outbound.protection.outlook.com ([104.47.38.82]:42369 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725837AbeIDFxZ (ORCPT ); Tue, 4 Sep 2018 01:53:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6tObxVukwyqWeihVatFhIVvDkEbDhPcuBsbQp6yoncs=; b=gsMAMDnoOTXUa22HtmP9RaqlzsStK/1xpESQXHE+mV7BACwqppy91w7w/fX2Ne9cN2BL2XNG3JefaUyshwnoMtaqhtpqPw8I3FrjJ064UeXpfdI2W7UKwjnKNUGgQOQfaQYQNEObeCANjM8SVw+jfs6HN6uNC8Wtgth4ap3KeoA= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1101.18; Tue, 4 Sep 2018 01:30:03 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Brijesh Singh , Tom Lendacky , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Sean Christopherson , =?utf-8?b?UmFkaW0g?= =?utf-8?b?S3LEjW3DocWZ?= Subject: [PATCH v4 4/4] x86/kvm: use __decrypted attribute in shared variables Date: Mon, 3 Sep 2018 20:29:42 -0500 Message-Id: <1536024582-25700-5-git-send-email-brijesh.singh@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> References: <1536024582-25700-1-git-send-email-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN6PR20CA0066.namprd20.prod.outlook.com (2603:10b6:404:151::28) To SN6PR12MB2687.namprd12.prod.outlook.com (2603:10b6:805:6f::28) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 57f75fe8-c8a0-4a95-f7a5-08d61205f12f X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:SN6PR12MB2687; X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;3:5fM6m96X8o8lYB3XQgeIW6SVVRUTzfSGTiJxDAKXjPv1NIwsX+wPxA4uk4sOb4I9x/yxfNWmSQOWGArc4wvghutLa9nnA4dCK9i+X31xqTJkw/wSwsiBTZGMGiHkFReD0WYqRz7WIu1KUBdRaKx7C/orqfWQ3ZA4CVQjealw1isqiVU+YshZ2fZV0m/fHHaDe5MyBmg3RbP4vR+8BZ9GVAd0lnOHh5C7vyU3BikAvkMVYRj6ZfBcdBOSuXqWgcUd;25:iGqbZIY5wljqHETjuhtumXICAoXdXNTGzIkkV1oZWIK3vUZsz7TpjrHFcT21TO0Yasqb20RTPH4ei58WZTWVOTvLAWH6MB0R0ZeUVf929eY2FoDhCowbrUQgxGBlnlam6VOmnKnUSte29vjZ8n11pLvUAk2WEAWXiOQ9Xod38VO19C73/cI5sl/9SAoaLLARnmQVmyRIx6MlWdJ+QtaZRX5zUYB8SXz+w+6Dt0Zjqcp2fr2mF2MAfBilojFg785UllSInuax4VAaOsoR0BcfEWzksj9K8lKjawf0LdIAzv8VLNV78C+kVmEr2pnpH13IdNcMC+zQ8uGpLBa+sI2XAw==;31:xgpPtl2eTGhGd05MNcKOtrtip2+2D8WHjLOcypu63ko7D9v3FWyOt6qIr/sVQa3byk5fkNI84lHiW0eSY1uuYFG5HZgjuZFWObJ1yVrj/st+TywUV79K6V8GXdA3rj9eX22JP5Sy9+4qnYTwnnWbhz5yJ7EMZKV4r8+rrf4Al3MtdkC1ch6qxoJXYrTjT77jRiLWjjLVfKCVYFfSIv4rBz4jIdywAQ2DjCFSHFCPOLw= X-MS-TrafficTypeDiagnostic: SN6PR12MB2687: X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:z8JlcFiIBC0adFU6b0BQPEVHQBGozBQsFn0QXOHP44mzOEhS0sLakbso1fZDHz7c6w0VaHBW/CcPaj+lbkNBLFTul+ofgKnx/1ueC6Z59WtnqipZt3LjVZPuJwx8oREW3NilL1TJ04P1ImiM9FaFQiQkdoX8YhEmkkLJwDQO0xkXVw4rsv9HqFPhZQq/CzX7TDhkSfWcIuAhgfrHisTEQ6c8MYftobMdAaqfXcR5KB6TK9f0YUkM9RNP5jNzz721yDkfGJiwR3pW37/5mo9tNf1I7LkgR7E+HYmLc+3NiU39IKOt8MyJ0eXPf1YXWo9Twll1mZplv362vPlJeH6oC6P55U52S6C5aS4hPZk6P5vr3FLNuGOXYemmWfXvxb1zvA8DHkVgyaAN8rX5uNuxC6DmGUPwdQDOqhcOwOJrDXDXkvbWkT7p7rGE8ylHMPOamAKHsvBbQgJKVAxSzgTyS7p9U3vMgeGs1fFKU/4sVuqQWX9pNkNlSlLWeYmm/g9T;4:Om5dtLG7qJAaWqEeWejCEGcAYERzFUfpBBfmckPHC74H6p6u0Pk2blszm9mDnwcBozjR8bFS4QILmYiAo7DJGgi9mkpDOXIkgZL7FwojfcY5azdvtW6zqjbWNYiEw8pu7Oh9meLIbije725BPbTJVwXjn9jRj9pcTnrZawvgZ8Iz4RUd55OaR4iYh5Nvfb0JWziEZs4kzCfVxwHU3NK8QbOznatWXQaksYEG89H77r0diPg7kJ4giNHLNRZrp07wrf8I2lbzbDK8PyBsdkfp/iaelJGJ0A89qd4nXu5HrLPk+DgARUABwZ7OF371Nj8Xqq2otHzYgvvsOH6rNaq7P4xcTwiCAbStPWellTs1DzJzGSkpLf5eYKmQ98IPtU11 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3231311)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:SN6PR12MB2687;BCL:0;PCL:0;RULEID:;SRVR:SN6PR12MB2687; X-Forefront-PRVS: 0785459C39 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(366004)(396003)(39860400002)(136003)(376002)(346002)(199004)(189003)(14444005)(6116002)(105586002)(76176011)(7696005)(47776003)(23676004)(66066001)(68736007)(52116002)(2870700001)(53936002)(53416004)(2906002)(316002)(36756003)(3846002)(97736004)(6486002)(50226002)(4326008)(25786009)(54906003)(106356001)(81166006)(81156014)(8936002)(478600001)(2616005)(6666003)(50466002)(86362001)(8676002)(44832011)(5660300001)(186003)(7736002)(305945005)(16526019)(386003)(486006)(476003)(26005)(446003)(11346002)(956004);DIR:OUT;SFP:1101;SCL:1;SRVR:SN6PR12MB2687;H:sbrijesh-desktop.amd.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2687=3B23=3AoW1yXBc?= =?utf-8?q?fiE0Xn2Dsi3Tzd7WJdyXM4ONhz+l/fQozI0xNXV/zZTbAUJpPsXnhHDN0eXOU+pul?= =?utf-8?q?Cwm6BEumzRnu0ZazSScOOOrKP39aepzYCerwDqJIcvwPB8SoIElVyrzpw7LMqZJT/?= =?utf-8?q?tHIeUD1DOHdWycRg0cOB0EAGeLEA3XchnP9YB8GaHVQVmUERth2hJgbzvmuuD+akE?= =?utf-8?q?hJSh0RneR5rR+5wYqOdefkOVu3+UXGLhyseOgBtRocPE5VXOAZnt0NYzGdV+CMs+S?= =?utf-8?q?OCZRaZ9p0hPgarq9HROfIxvT7KDtzkt0HDsvUqPR2jeB7rL5XVrXiLcKd2f+ApSFD?= =?utf-8?q?KhI1baIJHVMiPv83KepuidGVvGfaxmZZB3Qq+oCnfD6mVM1pciROrE68w3R2Rs4q5?= =?utf-8?q?DgFJUHiy2G34o/lEi/8evEHQyc4ntc4pqV2a2KO/23Eo+mrfgsSNfMLggc173NX9n?= =?utf-8?q?Dmy7Q4a9ysHeDqh2bTCT7Ukrq6peDq6sDaT2zpFbn4CwbYfMMsDJBd0R8u+Yrs2gA?= =?utf-8?q?q3avnDO8LaZpnH52jdctlKYgDcF9A9NFPd+nDrA4q2Wmvqd1C7TqgteawM/xYJi+b?= =?utf-8?q?i2L3IrRLA5XC/+nSaG5kDhGMhnXwcnR/0rvgm+SdDKIHs30bmCQzsRnTp374N4IcN?= =?utf-8?q?QmZHHqwfSGjwznLCzLcouwReSqriXaDmYFednXZTWHaRecS2y30XaNVZzZ0UhqwoJ?= =?utf-8?q?aXAK1+IpbsDvW2t2PMun3XP6k6dEPk66vI0a7nhTSXgm6Uta25cSIi5TesiCqDGdp?= =?utf-8?q?bb+oyXBdLd7USn2TtvMSVnz50CS9AQqiWOUPM2dWjVq3xXLZuoY+AlY8z7SDshbq+?= =?utf-8?q?jTuf3VAwFJ8eGSAVycAnEXqPQd6kTwVVMbxijjelAYQvlBUlDhJvSLWN49wKUUcxd?= =?utf-8?q?ThwyGCPeeRqUGTqz9qe9FZwyAorIidLRxb0x1oh4cZGWW3irfXbQuz7jemNTnS02B?= =?utf-8?q?3mUjf6qMCKmAP8jnmjKtbvDGlu3u/wOJOdGPCXPodwTAIC5fxPqqLWM+hwINfjwvg?= =?utf-8?q?RfMsHV+CjrurHVgo6Jycx7uIQywKMxOxQ8uUNnruor3uy6QYc3iwEbthDOVLfvqQF?= =?utf-8?q?QSAUvmlQvxKd7VVRlVUTSSJNtKCo+SXCKV+N/H41yoazehBIJIlcbeOpyESH0DkM9?= =?utf-8?q?xCkDOcFn+sQrCN0Xy2zmrOiaOo4KbrT/oQbyZVf?= X-Microsoft-Antispam-Message-Info: 4XxUlP6/MyWIvNWa9LDxTFpUkKd9x0UT0mI+G5mpWGJVn4irVl2LJH5cUR8e4EYDV75cmqDNJ3yTC8ISCuGq/sqS/NrKD85LIn+B1HDpETY/zqABbbA9l+cec4K6W9puaYCkfpntF54I1oMSyPmW6UBFqXTnz52PBloKqlEd8cLECKG8MERDrnbdYV92qjBIIQT+vzIAil0CKheU+udi4tOWE5OwEyRB6pvlaIsDMAnqI/z0hZNUhxRnvaBTCrq414ZHBdPAGiGdxk8p+BwWC7KM1rLoOmBFlPezOMkWcnFswPANXqCqAocfe+BufRImB9yY4TQjzQrr3Sb6+AIX/51LTwmJ4zzcdI9HLWZlqSc= X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;6:oW38tdx5MQBwrC5pBHYn79/DqDhkzDfcAYmT0SqkvKznV7cgnHNNP5YKjjerQFHU6PB7UtYei5eIJgBWpSM2QI685v/9hikb6Gmbpd1g/eQO9P9/r5UNP9nXJvp9qTUr7Z6AI5wRSY6pNsB39GAJdb0NY3EW978MFL05qReiXwjGeL6X9EyEXBnmuEhnZuAdBgB5qKu9x5DtMt18NP6ht04+pqK95DLhpcaPCbko7RNfZ9ti/coTMEuEL2bGKl95/lzrBiCVpcVqqJYmGLwwcImaw7Y4wZDUxgwP95gSJ4ohFJpJX9KMvp4UYk3bT54y0yNhWg3Bj5RXGprZUiby94eD7RIIPVf4a2/dmgO59m6oRoMKFJsYgs4oy+wepa5YB0YGZzPTsqZNgEs47V9e9FBx7uitigvW4OGH4FPJxITOkvuljRK002SSoGn/WOymyWApSzjh07n7yHlc3tC89w==;5:iujXbbr+Km3KO9ac6k/CBLQ4PNCJyFxQB0jTEv8UaxDhZhTuOhMoRtjpwfmytsPz0mLfFzgQQMdqWKtpCqxleYSUg99FZIvKs4mE5EZ7FRzt6iePk+RF0sfuYOCevO34+0dFMiKQ4ia7tShiyPX88FyRw535KhuAq7WN8fv3gT4=;7:39aJcW+slZR41eUGCsJ2ruCIp7cMToqDN3CQ/sbEScvelZ8ayMhKh+uPtcnTcK96CiiVqJb7wV+BrF9N/GPqUqg11jUcpSX+VxUF5HuBlohxKDMUs96h0zACIspor3ojg3g0I2j9SO1iXj1qZ4T/41V8wR4k0bqzAVOMJsDWZ3iKAzGrcrvCl2W/duOiYl08fwDqY58KqHrGsYd5VyEVLqF8M8y7qV/x33mDQoOCEkDeg9CSoE7prLc/BDsTP+17 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2687;20:fjyHXe2W6jDd9SuJiv47tAxOqosQYx7pmJ4bU3iEaa08tjly/rXGECpxujr78+9y9BT9WuBTC+x23D0FmvjVZkQ+E4lntJ0YW1s9OMYv0B/sME52UDeGtd5Qii0qxKZz8K+LRLBTjRjIMbHOmClnfBKf5VXHVHuWb/EpRIAiE0DfAu39zna9bEDGX9uG3rsSV2OXJk511ZT6SjO2hJuJaTQ1K8Vf+vJlcLOZ8zAAaiPi8PQZoEWw1tmTXEWt9Vy2 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2018 01:30:03.9936 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 57f75fe8-c8a0-4a95-f7a5-08d61205f12f X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2687 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Commit: 368a540e0232 (x86/kvmclock: Remove memblock dependency) caused SEV guest regression. When SEV is active, we map the shared variables (wall_clock and hv_clock_boot) with C=0 to ensure that both the guest and the hypervisor are able to access the data. To map the variables we use kernel_physical_mapping_init() to split the large pages, but splitting large pages requires allocating a new PMD, which fails now that kvmclock initialization is called early during boot. Recently we added a special .data..decrypted section to hold the shared variables. This section is mapped with C=0 early during boot. Use __decrypted attribute to put the wall_clock and hv_clock_boot in .data..decrypted section so that they are mapped with C=0. Signed-off-by: Brijesh Singh Reviewed-by: Tom Lendacky Fixes: 368a540e0232 ("x86/kvmclock: Remove memblock dependency") Cc: Tom Lendacky Cc: kvm@vger.kernel.org Cc: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Cc: Paolo Bonzini Cc: Sean Christopherson Cc: kvm@vger.kernel.org Cc: "Radim Krčmář" --- arch/x86/kernel/kvmclock.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1e67646..08f5f8a 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include @@ -61,8 +62,8 @@ early_param("no-kvmclock-vsyscall", parse_no_kvmclock_vsyscall); (PAGE_SIZE / sizeof(struct pvclock_vsyscall_time_info)) static struct pvclock_vsyscall_time_info - hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __aligned(PAGE_SIZE); -static struct pvclock_wall_clock wall_clock; + hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __decrypted __aligned(PAGE_SIZE); +static struct pvclock_wall_clock wall_clock __decrypted; static DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); static inline struct pvclock_vcpu_time_info *this_cpu_pvti(void) @@ -267,10 +268,29 @@ static int kvmclock_setup_percpu(unsigned int cpu) return 0; /* Use the static page for the first CPUs, allocate otherwise */ - if (cpu < HVC_BOOT_ARRAY_SIZE) + if (cpu < HVC_BOOT_ARRAY_SIZE) { p = &hv_clock_boot[cpu]; - else - p = kzalloc(sizeof(*p), GFP_KERNEL); + } else { + int rc; + unsigned int sz = sizeof(*p); + + if (sev_active()) + sz = PAGE_ALIGN(sz); + + p = kzalloc(sz, GFP_KERNEL); + + /* + * The physical address of per-cpu variable will be shared with + * the hypervisor. Let's clear the C-bit before we assign the + * memory to per_cpu variable. + */ + if (p && sev_active()) { + rc = set_memory_decrypted((unsigned long)p, sz >> PAGE_SHIFT); + if (rc) + return rc; + memset(p, 0, sz); + } + } per_cpu(hv_clock_per_cpu, cpu) = p; return p ? 0 : -ENOMEM;