From patchwork Thu Sep 13 21:51:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10600149 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64984112B for ; Thu, 13 Sep 2018 21:51:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B5C22B3D6 for ; Thu, 13 Sep 2018 21:51:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3CDDE2B4B6; Thu, 13 Sep 2018 21:51:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6962C2B3D6 for ; Thu, 13 Sep 2018 21:51:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728338AbeINDCt (ORCPT ); Thu, 13 Sep 2018 23:02:49 -0400 Received: from mail-bn3nam01on0056.outbound.protection.outlook.com ([104.47.33.56]:7648 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728090AbeINDCs (ORCPT ); Thu, 13 Sep 2018 23:02:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QbLj0XPm6vr8zgXNRzbcdu80J+v5lVg1zpUm8KtD1o4=; b=3d7K+zm9x1lcEBvYYG/oO4JaNXBIVs1U3KT7fQXyoJM7jser9VHn1UXx6b454zgVqNw6m6MfnptBArDbFmoIl5wbFLavgdPHdLY9vihCEa7LKvhaFOdzHZNDDjn7i6vgNM1BWVtiEa+K0r9yxfZnW+Embgmn5Yp3ByW4pGeSnPc= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN6PR12MB2688.namprd12.prod.outlook.com (2603:10b6:805:6f::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1122.16; Thu, 13 Sep 2018 21:51:22 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Brijesh Singh , Tom Lendacky , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Sean Christopherson , =?utf-8?b?UmFkaW0g?= =?utf-8?b?S3LEjW3DocWZ?= Subject: [PATCH v8 1/2] x86/mm: add .bss..decrypted section to hold shared variables Date: Thu, 13 Sep 2018 16:51:10 -0500 Message-Id: <1536875471-17391-2-git-send-email-brijesh.singh@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536875471-17391-1-git-send-email-brijesh.singh@amd.com> References: <1536875471-17391-1-git-send-email-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN7PR10CA0019.namprd10.prod.outlook.com (2603:10b6:406:bc::32) To SN6PR12MB2688.namprd12.prod.outlook.com (2603:10b6:805:6f::29) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dc635e2e-061f-4a61-5577-08d619c30c5e X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:SN6PR12MB2688; X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;3:5MOrOOMmMk74WouE5R0JMVmp1SZ0LR1j5+L3599w+viaDgnwntusbvkZPze5kczVBFVfR3zFLtVMsfNOTrQbmt/0pnKHGUnjzkI6l7gKYca3Dln9rFAtczq96hLr2UUf+k4rm3laQPJsykcTwI4nB8PlEZQ1QWELJk4O9/jfBKX4G19ND8e79M38/FRYj4pYPYIFP5h33xcTC+egb5LkmPBXbiArin9Bwn9R1h/UxYg55CtwnvF7zNocqZ+BQ5+g;25:RJCSM65RLhosnWI3AgoaUO0KCMW/oaV/P7cfF8+ABqF7oTfQjLsl8xw1jf+EmTuy6KJf0/Ie5yUzveb3nTVc5Xr4+tQk9edXyqPhBvlOuxI+IO4ggguXxGoC3wFgP7ALhjeCK7WL67ysyKcfGTY+DwlLjYzqvqhOfxl7TT7bhSn3jgYwvbyg2cNzEE6uLasHf0fTc4m6HSTl4WMoMH1yNboqIrvR0qWXtSNwP0VNBRXb3Lycx8WU9R1ByeBs9g4yLxArqkgn1Pb39cOT+x3n/iHJhnhEfLi84yfeIHiJ92jKpRifyOkjUCYsqNcVofpOgHP2dZtApvdpTNKN5o1D1w==;31:pNHeFBxh3VgTmm3Ni74DUWDXinwqR1RUTs9JRVlexzS8ilrCufDHg0413UzysDbcfy9ZJsWvUDLoNTiwSpptaATG/oUn9uY96sH2cPCAXKQUKN5Eg5Owgr1yLyatMq1cT9ICk1mqUeTbIoerCY8T4or1/t8tjMF+32tgiDfSIwMs9Uhpf1V+RZCy5vvTFg0K0RLc6t6tysCXfVBRXzaAgMXgOw2mkhsXyXFcYQ/toa0= X-MS-TrafficTypeDiagnostic: SN6PR12MB2688: X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;20:z+Igbx+kkbzIWlMaZfYhK8XIqgauoX4BhhT9vLBqBTEVEKj7zPNMR1EoCvQvAkh1ZSV5t8Ky3WNbmNe7zLKChf14yR0o1twrqMpEC2R4e3Q1pmRBcDgpfn7hxWiB/coCKpQAyokT1bQ6nejbSDqKk3zyHZsuwvlr/eWLJ5a920wOYtYf2GharDhpd7aDsXCrecQqURPWvAG51KzPQTkXknsHvmJzpEvenbWUisvQqP/WM1qLE5ZXiTM/tjYpWCgH7COlVjV5e68N/MYssLF0QUPP/BvDI7tlnKZd9eV1vY18rnR6idywxyPGQDIP+B0ua2ZjyAgLm5vphAybMLjQq+iHCkKQinOakCbfzPLgmGLZJT6c9L45mkLdKbjUG7BTHQhg9/WQnlNlBXmyK8PxTPJ1ucU9U+H9YJdInHXnRGtwwZR7xqzxuI+IC8KBTYmTJ78P0Y5W8IiqlQd8vmXInAqIkXftrPrBYmenJdLgPeV1E7mfo6Wy8zH6LmtulpKO;4:I53nLUNp0RrOqrMC1TDVS9/Mrl4100sU+OCUxZvok/FAH7TP06LtlJ4cWu8JQR1fSkrWl2NPVmIpYONx+jI/QIdjO9eLdhCaiu8D46fcVv/uOrwWR1Y2NKGubNeeNbw5LW116Gd3Xnagsgqy0ZHuS7FFtaXtzsARudYx4m2YAtnzYudcV02Cgc9lRTbwZSbFoWskHCbx5Brln3TJyvRZGyDP0B0KP0gS7FhYjMvmIDXpyZKMJjURI76T2/dkWbcZEl1is68Ip8SCfE8ZbJWnkIQqvzCIJ3yDTcOOEbsSNYlCLDHERyaq6+bqRwylUGmI9VFnqVTvkj8FMcQMiV1f4xjnF4Tnd1Vm7nCU+MPmkXk7vVcSqg8icrrsO122ovKh X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(3002001)(10201501046)(93006095)(93001095)(3231311)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(201708071742011)(7699050);SRVR:SN6PR12MB2688;BCL:0;PCL:0;RULEID:;SRVR:SN6PR12MB2688; X-Forefront-PRVS: 07943272E1 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(39860400002)(396003)(136003)(366004)(376002)(346002)(199004)(189003)(26005)(8936002)(81156014)(478600001)(316002)(53936002)(305945005)(76176011)(16526019)(23676004)(52116002)(7696005)(7736002)(36756003)(47776003)(3846002)(6116002)(50226002)(66066001)(186003)(6486002)(5660300001)(2906002)(486006)(68736007)(54906003)(476003)(2870700001)(8676002)(386003)(25786009)(2616005)(50466002)(956004)(81166006)(44832011)(53416004)(4326008)(446003)(105586002)(11346002)(97736004)(6666003)(106356001)(86362001)(142923001)(101420200001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN6PR12MB2688;H:sbrijesh-desktop.amd.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2688=3B23=3ACtgKfPV?= =?utf-8?q?47LwX++41Ht2h/2kElagiC9Xmgrear1Eyg0SLENAjSOREKM+b3ul+ifWPIsh2sK4b?= =?utf-8?q?djPNULEVJA2vDDLG2IcfVutLe28d4SwcD7c6KFm5WU+Y5AZ7MKwmNQxdX/5CXus3e?= =?utf-8?q?qnUJOsk7+fUeSw2HZpgdOa8diq+Ak5pbn6KAD8ijR5vebiOgQnLlWTIyrMsCj+rjR?= =?utf-8?q?7i/CZAAQVcxH4k6R1c/ogiLNaPZYVWgk0/yN5MYxjvXHg60J14Zzlro3b3686X9sU?= =?utf-8?q?sm3D9gvql7p/e7ur/gQc1gUyLUFEQZAHH1tw8Ng+qn5bpLW4PzRNBDyVVe7bItxX7?= =?utf-8?q?C2kVvgfPUumVpblwLO2BHC/i3ELw6pTm3tmrHZRH37CbtC6qeg3it3jb0rurw6vcY?= =?utf-8?q?t8ch/Nlu8vBK82uUwvZWJx24+OzTPgB+XkvQ6e2Qf7/o+4W+VDTvSZk1I8elQQKtM?= =?utf-8?q?DER7uwyXrHbOw7SARm/8dQoM7rI3lkbfgEpAt1WU21QTcc2P2U57mDQdDqisNaTeg?= =?utf-8?q?U7LYS1n1sMZDBAW3P1KWmVCfahNtsuYhTdZBger9Yebre6NyTyCs9f09ylnWzqpTP?= =?utf-8?q?0WcnZd0bRs+JnEcO9/rFbAx5dhUVog2eq+/xG4h5VcfusyvDIjyBNfaTtRBby8ycH?= =?utf-8?q?M17F4V35ZHzQF9HGI3I6Y9LbLbnheGrPz4pKz5ijIF+kdE0wWEl+QIl6sIbhBYzDa?= =?utf-8?q?t4F0vWXoifqvG+9NhpZyLX3t2kg5YriKon5BvteuC5hoh77Vt97jHfxhQwXH4S2Xr?= =?utf-8?q?thPaDjpqhZ6N7u/Ueu69sz+/i9sjw8IiTmngUTN5PnP31oKLmfsj7rFyYnajNG2dq?= =?utf-8?q?l6c9YIw8vAdZhS4Q26VMM+Yez8j1ZYKQzMs15xF7WnYIpG2+HusoCBatNqB1DbwZj?= =?utf-8?q?AyJaP6Jz/v5At+nLCs4GtfGKitlc1cpARyPqnWwKusCaZ6Jd/h1bjzEJZ6hVb3/mj?= =?utf-8?q?xY2CvNMAfY5CzycdA/YnH2vDJal6P7JYsXZCQce4e20V/jLHQP/9Q82ORUlX1j6fD?= =?utf-8?q?2ZewFLWE+JaCytIpiAh+Qa5dhnYKG5MLypyVunf9AACcNNmoPYxDZaPOPZvfb3fqV?= =?utf-8?q?o2YYZMCBCCN/V4SxELY0fvZ41HVUX9993NT1XKLAVmg4ZVsxsqf5dkhr4yCwUtLL+?= =?utf-8?q?FEIREsAA/f1lccq8lpPFTAJWWBuT+9p9w5p7l8tEDZSJ/Xv4X/5RxDCGgRlng=3D?= =?utf-8?q?=3D?= X-Microsoft-Antispam-Message-Info: zJAX8Rjn+Zcg9pdUa7vU0NA2yfX0PWImF2dV4h4qC6hhbigmDzYp2UyDBPm0k27kg9hQaJEgjdRI51XB9uiGo5DG7OqF85GSdYGNk+n7wgYjBfiitxIl+7FLeLtAjW6r/AFdr4yoQ/T44IMfobcjbKvDZLfiP5OGU6sXOd+zrMmne9wQUdpZnx8XZyEAc3nAoqxgNqAsAXZ4IDaum4HiEloJhICdOt61DgX6BNUkmYlXYRRvivGg1DVkqjjN75reja/HMEG+ZHDYjXlxiga6YRE0/P+8MhAQfXnqyxnzodVaZrw9D7P15NHvkhy17AeTNB0jSlNktGpSflpt6aJHqXm0vEa4zTqsN7p9RLbMucs= X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;6:UjI4bg+9e7n8kG806XXZLENCd4Tf5YuamqN5xaKje5QCAPO60A6CNaTKa8vBm0NpSNLCQdpyriuKtF0GoxrfIbOpKRUdTW9YYllkVcLIABKOFPgEUbJyCtJhRVURjpQ34asnMX/GKeZC+HXxTBcu+lmDN+fH75Z6OVTPpgVI/fRbcpiHQrqjx2yvqCHDkfVxPxoq5zciXTg5JlJGDJKHF1lShQ+W2K/36Y8rH429ysYyIBWo0hkbqqrfjPFFe9bSMXCjCQY3s/6ZkVivbCy1JvrPlXI9R+2u1Q3NQbkLPWu2D8wLrq0hr3Dl46h6rTzxf/B5F1sye1RXVnD3OZDXtSUH872v18o1whHh7P/sWq+/kL1/rfijNWHqSVGUOUjEltot1WIkYN57PqtMbHoDCKdXPbC8A9EvAIj82ghHmFvz0ZcVRBFar5qBbw1hdWnC+7/etpubS2g9HM9DN25s5g==;5:LpYz3LS5nmReXbwAlPz0s9XxAeBxRzI87RAq6NoCxSH+6Uvr2+NUQJOFffTgT4qJCtxyKcnTurijIaLwUAHi59jbXjOIqZbcqvcAPoMui3H3FFvFpbdRpC4v8cL0Z9OyqQURs3mNnXUHVsNX2I+KiTcacaUWz91FKmFJIlzsbsI=;7:gwVtJfIcMTqblBnE77TMYPNL1vEAri6p3Zlw4Uk6MFeyVCcnbnVJ7ZiRL4bKMkyqJ7q+wJxkawzifXcOdKsBgdsE3F+iMZMD8NaTFqBcSk9p3fvxwAFCoUB6vVqco3D56YAt8ccYR8wj6/U//S4Am2HDU8BOoSYU4I6Wd7UQIYBXZaexNZ55SgdKdvTRM9tg0zpxZoZMJTwsKKMoNeJclty+cI28LqfeUsIbPbEb83Rm0lA0ZSC8ZxAuBxu/SjpJ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;20:sTYcek0LFP5cIhGEOOmJXEDipobSQXZMeSsYzA4kMOfeXNecx6y+hcLgp/x18VeDhgAitoGTvMS83s6M2PpG+QdF65U8nw6W1xlV9Pu1wO6u3N87g7KrQ8TUmjLSRoQj9in/U4+SmgN8EXi4qLZ1XxgjS0k+Cxjj8h/LgezTDg/8Blxm8chiEXeNMhbftZuq1Fz83WkbpYclboGUTErWLJ2IIBZOSK5DIKRug/z5STbGif6q/5AvXqO/hDsCXrVE X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2018 21:51:22.5462 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dc635e2e-061f-4a61-5577-08d619c30c5e X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2688 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP kvmclock defines few static variables which are shared with the hypervisor during the kvmclock initialization. When SEV is active, memory is encrypted with a guest-specific key, and if the guest OS wants to share the memory region with the hypervisor then it must clear the C-bit before sharing it. Currently, we use kernel_physical_mapping_init() to split large pages before clearing the C-bit on shared pages. But it fails when called from the kvmclock initialization (mainly because the memblock allocator is not ready that early during boot). Add a __bss_decrypted section attribute which can be used when defining such shared variable. The so-defined variables will be placed in the .bss..decrypted section. This section will be mapped with C=0 early during boot. The .bss..decrypted section has a big chunk of memory that may be unused when memory encryption is not active, free it when memory encryption is not active. Signed-off-by: Brijesh Singh Suggested-by: Thomas Gleixner Cc: Tom Lendacky Cc: kvm@vger.kernel.org Cc: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Cc: Paolo Bonzini Cc: Sean Christopherson Cc: "Radim Krčmář" --- arch/x86/include/asm/mem_encrypt.h | 7 +++++++ arch/x86/kernel/head64.c | 16 ++++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ arch/x86/mm/init.c | 4 ++++ arch/x86/mm/mem_encrypt.c | 10 ++++++++++ 5 files changed, 56 insertions(+) diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index c064383..616f8e6 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -48,10 +48,13 @@ int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size); /* Architecture __weak replacement functions */ void __init mem_encrypt_init(void); +void __init mem_encrypt_free_decrypted_mem(void); bool sme_active(void); bool sev_active(void); +#define __bss_decrypted __attribute__((__section__(".bss..decrypted"))) + #else /* !CONFIG_AMD_MEM_ENCRYPT */ #define sme_me_mask 0ULL @@ -77,6 +80,8 @@ early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 0; static inline int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; } +#define __bss_decrypted + #endif /* CONFIG_AMD_MEM_ENCRYPT */ /* @@ -88,6 +93,8 @@ early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; #define __sme_pa(x) (__pa(x) | sme_me_mask) #define __sme_pa_nodebug(x) (__pa_nodebug(x) | sme_me_mask) +extern char __start_bss_decrypted[], __end_bss_decrypted[], __start_bss_decrypted_unused[]; + #endif /* __ASSEMBLY__ */ #endif /* __X86_MEM_ENCRYPT_H__ */ diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 8047379..c16af27 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -112,6 +112,7 @@ static bool __head check_la57_support(unsigned long physaddr) unsigned long __head __startup_64(unsigned long physaddr, struct boot_params *bp) { + unsigned long vaddr, vaddr_end; unsigned long load_delta, *p; unsigned long pgtable_flags; pgdval_t *pgd; @@ -235,6 +236,21 @@ unsigned long __head __startup_64(unsigned long physaddr, sme_encrypt_kernel(bp); /* + * Clear the memory encryption mask from the .bss..decrypted section. + * The bss section will be memset to zero later in the initialization so + * there is no need to zero it after changing the memory encryption + * attribute. + */ + if (mem_encrypt_active()) { + vaddr = (unsigned long)__start_bss_decrypted; + vaddr_end = (unsigned long)__end_bss_decrypted; + for (; vaddr < vaddr_end; vaddr += PMD_SIZE) { + i = pmd_index(vaddr); + pmd[i] -= sme_get_me_mask(); + } + } + + /* * Return the SME encryption mask (if SME is active) to be used as a * modifier for the initial pgdir entry programmed into CR3. */ diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 9c77d2d..0d618ee 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -65,6 +65,23 @@ jiffies_64 = jiffies; #define ALIGN_ENTRY_TEXT_BEGIN . = ALIGN(PMD_SIZE); #define ALIGN_ENTRY_TEXT_END . = ALIGN(PMD_SIZE); +/* + * This section contains data which will be mapped as decrypted. Memory + * encryption operates on a page basis. Make this section PMD-aligned + * to avoid splitting the pages while mapping the section early. + * + * Note: We use a separate section so that only this section gets + * decrypted to avoid exposing more than we wish. + */ +#define BSS_DECRYPTED \ + . = ALIGN(PMD_SIZE); \ + __start_bss_decrypted = .; \ + *(.bss..decrypted); \ + . = ALIGN(PAGE_SIZE); \ + __start_bss_decrypted_unused = .; \ + . = ALIGN(PMD_SIZE); \ + __end_bss_decrypted = .; \ + #else #define X86_ALIGN_RODATA_BEGIN @@ -74,6 +91,7 @@ jiffies_64 = jiffies; #define ALIGN_ENTRY_TEXT_BEGIN #define ALIGN_ENTRY_TEXT_END +#define BSS_DECRYPTED #endif @@ -345,6 +363,7 @@ SECTIONS __bss_start = .; *(.bss..page_aligned) *(.bss) + BSS_DECRYPTED . = ALIGN(PAGE_SIZE); __bss_stop = .; } diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 7a8fc26..faca978 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -815,10 +815,14 @@ void free_kernel_image_pages(void *begin, void *end) set_memory_np_noalias(begin_ul, len_pages); } +void __weak mem_encrypt_free_decrypted_mem(void) { } + void __ref free_initmem(void) { e820__reallocate_tables(); + mem_encrypt_free_decrypted_mem(); + free_kernel_image_pages(&__init_begin, &__init_end); } diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index b2de398..718acdf 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -348,6 +348,16 @@ bool sev_active(void) EXPORT_SYMBOL(sev_active); /* Architecture __weak replacement functions */ +void __init mem_encrypt_free_decrypted_mem(void) +{ + if (mem_encrypt_active()) + return; + + free_init_pages("unused decrypted", + (unsigned long)__start_bss_decrypted_unused, + (unsigned long)__end_bss_decrypted); +} + void __init mem_encrypt_init(void) { if (!sme_me_mask) From patchwork Thu Sep 13 21:51:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10600147 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CBBF17D5 for ; Thu, 13 Sep 2018 21:51:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43B752B5AD for ; Thu, 13 Sep 2018 21:51:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4131F2B5AF; Thu, 13 Sep 2018 21:51:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 927D02B5A9 for ; Thu, 13 Sep 2018 21:51:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728388AbeINDCu (ORCPT ); Thu, 13 Sep 2018 23:02:50 -0400 Received: from mail-bn3nam01on0056.outbound.protection.outlook.com ([104.47.33.56]:7648 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728222AbeINDCt (ORCPT ); Thu, 13 Sep 2018 23:02:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IIRFSon0tHNcv6kJA1Vaiy3U7u4p1v7qrHJatqT/uXI=; b=VGOEjIIRoDVxMOn6/yjo3DnWjSg94wYyyk+yrQvt9x4oYnd2Zlj2C1KMquljGGuuqWEvVUzMxRbTSWO55LncZwht/MIwYKJtAGqRbdNqQkg3slH5DeMHJsD3dvpnKJrlgW3+JXqwLXhVWTJxmOfwe/568iIiUzvqwuWuDVXe6DM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from sbrijesh-desktop.amd.com (165.204.77.1) by SN6PR12MB2688.namprd12.prod.outlook.com (2603:10b6:805:6f::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1122.16; Thu, 13 Sep 2018 21:51:24 +0000 From: Brijesh Singh To: x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Brijesh Singh , Tom Lendacky , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Paolo Bonzini , Sean Christopherson , =?utf-8?b?UmFkaW0g?= =?utf-8?b?S3LEjW3DocWZ?= Subject: [PATCH v8 2/2] x86/kvm: use __bss_decrypted attribute in shared variables Date: Thu, 13 Sep 2018 16:51:11 -0500 Message-Id: <1536875471-17391-3-git-send-email-brijesh.singh@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536875471-17391-1-git-send-email-brijesh.singh@amd.com> References: <1536875471-17391-1-git-send-email-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN7PR10CA0019.namprd10.prod.outlook.com (2603:10b6:406:bc::32) To SN6PR12MB2688.namprd12.prod.outlook.com (2603:10b6:805:6f::29) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5f8843a1-dd53-47b4-7726-08d619c30d6a X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020);SRVR:SN6PR12MB2688; X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;3:50mJue67oIYM9TTL5wL8lcrPtl6GtHBcrXjVhJQYL08cyEjNH6Jifk5LSGPIkFx0vdKi2iomWQUafmWlG/VraYJxeU2+M9NmuuohZX98k9METga9rcl6pusEfcjxElTXissccss70j0zsOLEm3RheDtX19azAZqQjnVXA9nLv61CbssbNKa1PCBY/R7KgsA9eZh+R30ELdxMtmbN+DGh0L7BT5Rho35xz/fppp/EM5CtL0P21fiblFbMGpSRIn0V;25:4gos5NhS20DFTfSQDjepzVVhaXO2yhJ6a+kIufQk0IMmVSLMdzfwY25S4vv+YRDqPFY7N7vHIwhj8zTCGGv+wpjztB3iMbM7IQ9vj69AD92VkEh4E8nPuIPprsVUWhE2T15h9rrDTxWE6kAz+4Mfj0kDx4GNLe1en6ZCe0NNDzKuCySuhYCsHGDIFlp9oEZ0Wo9eNEwwRypsq1uVyUS0sFr8HM5Q1FHTifCHXa9tAeyvYuIaq2oQ3WHWAKFtI3qtc+l7DLucRHxLIkxP2Mx3FDS97n1nCK3ZlSylhl3TAjB+yKbprTCaSTlNt9PiBUlm99eUwt54YyY+IDnkHxSGog==;31:7wZyubEObWD06iiuoZ11wZx0lV/+G8s16YKXrbBocoPiXy64OKMuJMHfPGOjS+mAiXmVSBGH3jwBMLiFhC0lsR8bxsF7yB5sn+CWLvoWNZeiwlBb0RkGrJEf1NKvesmuOV/iOT7lv1lreCvqdcTDS02xAFg9rRLBVzqgc7Wsmn3ZlenoIur4IXwbMzWqJVDaEXVQSwikbWc3T3oTiqHU17QShIZzK6utmwQY44rJpBI= X-MS-TrafficTypeDiagnostic: SN6PR12MB2688: X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;20:FQqaBgRwRIDGF/QCSBVN35LWyxh+g6t1leBrBawBZkYsbl6x0HDlMFwTgvECBG83wAo5FTWC3XNiXyPzMza/XmpwxeKZN5vh3Cd1zyTiBi7YQa9yppC2TkitL6aqC7YPLuONDl/cqUEOkUsvciypn5pTiuMli47s6Z7rPP/R6z4F98Xw4i3EcOg7+4im2Tn6U+mHfFEdurIYIwl6acHNXQhemBv0yD1jfjUncRODQ3Nz2Ce62oAATFpXtaLE0Ug1jbq4Kn8r8yywX70lZ+XVuo9RWnWDopq2wlK/tKfqoSWZzVDAx9dVMEIta3S9GvwVpx1/TE2r9NuWsXKd2XCHLx8Wk1I+9TsqEILhaWcng95qujlzN2nv5CVXXQianD61VDDS1RCvGWSODsKIJluXIACZaBGztefozYe7sCYCGgtTOtrRm1WIfYxQL0QLTlx7CEEUL9OI8hw0PrUFmlvOXZcGz2Ax2jBZA9fPg6JoprQhfGeCtGHs3Osl6IqpvNYE;4:BJvfmCPOC0kswvpleiNT6zz10+nhCn31Q6tT52khprTV9zRFNp86r1mJpFmf9NTEyIM6xjHk+XUZcbD4rhPJ71bTp6vb44fdHVghE+GbxKKKhPblgE1xnpoj8pztml67+cTB/1v2UcDfUdAT4n0+v8ePnHSgR+5EzMScLbo0lAb7+z5ZzDDe5Ut8xdpCx50gf/RCl0dTKY4aV8W0qbN01DSpqLWwd/gVXpUdTUrMI3+WKDaNjeKcMGFdRqrj1KYEd0RZdJfxf4bir2Ez830pGrOcK+wUFAH8qASpbEa6zhqvabDTYxX73A+HUXxNP3GKot8nZJAjunr46wO65cZp5BsTJ1F6sWYAvgXYKtFyzGYdAwB4IMk+nJ9AiOMux1b0 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(3002001)(10201501046)(93006095)(93001095)(3231311)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123560045)(201708071742011)(7699050);SRVR:SN6PR12MB2688;BCL:0;PCL:0;RULEID:;SRVR:SN6PR12MB2688; X-Forefront-PRVS: 07943272E1 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(39860400002)(396003)(136003)(366004)(376002)(346002)(199004)(189003)(26005)(8936002)(81156014)(478600001)(316002)(53936002)(305945005)(76176011)(16526019)(23676004)(52116002)(7696005)(14444005)(7736002)(36756003)(47776003)(3846002)(6116002)(50226002)(66066001)(186003)(6486002)(5660300001)(2906002)(486006)(68736007)(54906003)(476003)(2870700001)(8676002)(386003)(25786009)(2616005)(50466002)(956004)(81166006)(44832011)(53416004)(4326008)(446003)(105586002)(11346002)(97736004)(6666003)(106356001)(86362001);DIR:OUT;SFP:1101;SCL:1;SRVR:SN6PR12MB2688;H:sbrijesh-desktop.amd.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2688=3B23=3ACjorj36?= =?utf-8?q?eBlE36U86VHmHBkeSwXS4spb2BXPVdjxGdQdUgipR6RuFgJn4ANy3timQFlc46SkZ?= =?utf-8?q?ei/6WqtHdVyLTCCumTS/1ayTZw+D7UHj+QTZVBMGgvEOyNFvd3ZwVcDLeKyEsbmvX?= =?utf-8?q?UciZZg6ZlI/GCtKFfuLJoMEDKkTSb7hHljOPTKQWwKGFfRiXBO9jO+OCTv/I6QA77?= =?utf-8?q?ZqoRTBUmvu/9ftnHn7wo9nb1sBb15LHP5ut+98fctxJB9R7KdsWbFMtiOVk8fBGe4?= =?utf-8?q?aaCg/WZRz9iwaa++3332i5lT6b6QbaPgTXm/m96/iOPoalIeuRW73A2S880wNjU74?= =?utf-8?q?w97iclMTynZCHj2O/n5kH7jcqqTikZDmsAxPeC5gclVWIXVbXXsipbXuRvrk+DzVu?= =?utf-8?q?WUUGnhH7f1gHrNCuA/f5wq8FT9KBeAHzloQpm61IcpK0iw0VkGGWLrRG3p1r8H9+7?= =?utf-8?q?xRgF+tPDFvbu+xS/BOZdVaYs47q6HPhMWE9iKCpXT0tQJvVnRD5be6DQJM9iQqJzl?= =?utf-8?q?GU7MmDl54pXe3CDx+Ut9RoOBun3dqm6XwvNsGXm2LiMXxG5ujdOaJ8tIFDvxQU+w/?= =?utf-8?q?Wenv/JlRtQGeN/ctFhcvLroRUTSOTDOjLu5NfmEnymNgR0fKay/msr9nwnF2zHNQr?= =?utf-8?q?pBCbDIK48C8y/cTSgwqCTSrIaLTN5CT5BLWj4XIpS+b2sstocPwhWFPw8xtGwgqnQ?= =?utf-8?q?5ZpmuDiRkA/yBS6tqL7otttSfuxH/kxxBrU/jxdoSI+5NHJTacMc9adFVXmmJ4/An?= =?utf-8?q?toPSl5Vz+/M7gHhy7IolfJErDeJ0MWJMrcNhqsrz3B8xmNppKIbrH5dS7GeSnG5Wi?= =?utf-8?q?IN5Z6Zj8MKYN2KcO0Hx3AX8TwBSsWzkmAm0kE+dvgPAg5KqwJWqWi4yHixeWR6Gs0?= =?utf-8?q?s350OlThq4h+VuN4wqoPpSAk57SPNhLo6RYPunD8nU7/v92ngm3a8LKVd1ovDoWvs?= =?utf-8?q?0ViEmIs6UY18l4v5eiZdZwWqFGC7DKnhy/ox4DkzyKS2tbP8Re5kCEWKLB+951Qpm?= =?utf-8?q?g/d7uL2UJGqH+B+ckqXXI28I8h2wRqv2TsXz38rodB/Rb2w2t7xoQW9ZdlfalCCmo?= =?utf-8?q?o1Ho4WLJadWIquzGkDF0LBaacdLNkM5iO8fvjhoy3ze7m1Cat3DBqwgxVqH0UrrEw?= =?utf-8?q?jYDwWLUF+WRKq74he7p8xAQQco5A/XNB05W+F77?= X-Microsoft-Antispam-Message-Info: qEiLOKxpjjes2nH5xTyeJwHFAEysKkNNihbkhKzYH+TDLA6trAgb/BSZYjqEIyC4R9SLq8E6zu8u080hGqbQ+9OC7EPnq1Dp3aWUnEvvJRZt3lS5aVUxPsfoNoMsSP9MsPM+6xH29AaAPup3zF4zio8Ej+Ap2rnY1lRGiXYHqA+P67QxdMcwcy23kFGu2kfhtzQ5Kt1QQJ/SLAMyCvQLEry7TVMk4Wo9U4Ch2qdY646w7G0WGXbJKpMHiRp3pL1iwEmxOx3F5jTAsWzIM6PMcIYWr1Igrl6WoSOH3xplqc2aP4hwoTuSkcZnNxPwe3eFQGaPv1ClFmg1jvBbEwgUIW4kDC34lxGz/NzcqQlLqHE= X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;6:hRiOzr67T95hrEX7DRlzUo6SfJW8+lsOw+ljXljjsllGOTX5rz8Wxkm0dgnjIzgMQD+d1fbXeCqx/ACf8n86ZaluxRXFF9uhjLzGfxQG4y+UinmOkt2eMbcPl0wqJd1D+Lk1PzKwWZ5hdBWdXFn3mQSfjOegQ4L9jM1IXA88R33CIbU6kcMxJYVBdlx2shgmKd45mjsIQpy3Tq7M5+UDgjFr/HRBBEfZHTAlvRPDBbeLVVNIEa7gH5XgcTSNNTJONMGu1LR0Xg5vwW1NTBX19uXVwauXTn6ncyGWHODBaaS+tGP6jnRuGvcYF0F9BDfmVD/HQK5vKFlvPsw/7HUICKcdlRW/dELO22xH9GfmO1Sb9eEqOI9EuR5ZIdpZbBQPsmsEk6ztTU/mD0Bf5RZWFTxy3fFULEemlCVH/FQlwoEIA1GPyawG0XROA82LOQ67Z0fPDc8Scfztc/NUB8zHOw==;5:nruUCC0n8Od9/yt9V4iQ1LrCBw60ml0iXGcj6zG9h5Sj0XZjNfsojoennyBHkF5gGkvtHyqGdjQzxej6xJdBngrLbpgiRYUEtslwEDSiTQNc9X2qzUI8SpqEjE9o6ebctomG2P8y1QxHhVTlIKkwofxVWw5vY/1Cm8cFE4LsE4M=;7:NyqVN+wuWwCgSBB/iMYSV3Ii7sWToCvofUTHns9JpqxItr6G3+uFBRIEnLj3/seyOCLIgbZBjuFvjgacqyCxfAZcXRxoUrK3FWNwMCJaQCkZrSj/W+S0c60g+q+1RjLf+TvFzuNLeCZDYa8J7vRn/b1FsYyOjFkIme/btUK2+YBZJOx7zYxo9ffrMfJloO89zVDQjWDWXboEVCUqQMJ56GV0MEUA3i063ReYontuyM8jUX58rmFOcNwVTeUSMVxv SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN6PR12MB2688;20:c5GQ+xURApLXWYOoYg32icNkxdpUj0FWFQuF4GRMPr6LiIIyfm0fXCbtDwrtzWzroz38mO0maXXGEzyxtSeWO78jOoJ3+UA3d+dHD3OyefUbhMXe6Zc85ARmx/HbXzbw4cjveTDtwvgkFCAY8jEBPPasP0bPh/xRm1KpDqo8rwmDG87kbZAWJLy3KPPOOf3CBxJuuIeAkT0QxbRC7ytOASsH7JxyH94U2ZwcYVGuXMcZvHz3X9+fB61yREHJGFfY X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2018 21:51:24.2804 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5f8843a1-dd53-47b4-7726-08d619c30d6a X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2688 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The recent removal of the memblock dependency from kvmclock caused a SEV guest regression because the wall_clock and hv_clock_boot variables are no longer mapped decrypted when SEV is active. Use the __bss_decrypted attribute to put the static wall_clock and hv_clock_boot in the .bss..decrypted section so that they are mapped decrypted during boot. In the preparatory stage of CPU hotplug, the per-cpu pvclock data pointer assigns either an element of the static array or dynamically allocated memory for the pvclock data pointer. The static array are now mapped decrypted but the dynamically allocated memory is not mapped decrypted. However, when SEV is active this memory range must be mapped decrypted. Add a function which is called after the page allocator is up, and allocate memory for the pvclock data pointers for the all possible cpus. Map this memory range as decrypted when SEV is active. Signed-off-by: Brijesh Singh Suggested-by: Thomas Gleixner Fixes: 368a540e0232 ("x86/kvmclock: Remove memblock dependency") Cc: Tom Lendacky Cc: kvm@vger.kernel.org Cc: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: linux-kernel@vger.kernel.org Cc: Paolo Bonzini Cc: Sean Christopherson Cc: "Radim Krčmář" --- arch/x86/kernel/kvmclock.c | 42 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index a36b93a..84f29f1 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include @@ -61,9 +62,10 @@ early_param("no-kvmclock-vsyscall", parse_no_kvmclock_vsyscall); (PAGE_SIZE / sizeof(struct pvclock_vsyscall_time_info)) static struct pvclock_vsyscall_time_info - hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __aligned(PAGE_SIZE); -static struct pvclock_wall_clock wall_clock; + hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __bss_decrypted __aligned(PAGE_SIZE); +static struct pvclock_wall_clock wall_clock __bss_decrypted; static DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); +static struct pvclock_vsyscall_time_info *hvclock_mem; static inline struct pvclock_vcpu_time_info *this_cpu_pvti(void) { @@ -236,6 +238,35 @@ static void kvm_shutdown(void) native_machine_shutdown(); } +static void __init kvmclock_init_mem(void) +{ + unsigned int ncpus = num_possible_cpus() - HVC_BOOT_ARRAY_SIZE; + unsigned int order = get_order(ncpus * sizeof(*hvclock_mem)); + struct page *p; + int r; + + p = alloc_pages(GFP_KERNEL, order); + if (p) { + hvclock_mem = page_address(p); + + /* + * hvclock is shared between the guest and the hypervisor, must + * be mapped decrypted. + */ + if (sev_active()) { + r = set_memory_decrypted((unsigned long) hvclock_mem, + 1UL << order); + if (r) { + __free_pages(p, order); + hvclock_mem = NULL; + return; + } + } + + memset(hvclock_mem, 0, PAGE_SIZE << order); + } +} + static int __init kvm_setup_vsyscall_timeinfo(void) { #ifdef CONFIG_X86_64 @@ -250,6 +281,9 @@ static int __init kvm_setup_vsyscall_timeinfo(void) kvm_clock.archdata.vclock_mode = VCLOCK_PVCLOCK; #endif + + kvmclock_init_mem(); + return 0; } early_initcall(kvm_setup_vsyscall_timeinfo); @@ -269,8 +303,10 @@ static int kvmclock_setup_percpu(unsigned int cpu) /* Use the static page for the first CPUs, allocate otherwise */ if (cpu < HVC_BOOT_ARRAY_SIZE) p = &hv_clock_boot[cpu]; + else if (hvclock_mem) + p = hvclock_mem + cpu - HVC_BOOT_ARRAY_SIZE; else - p = kzalloc(sizeof(*p), GFP_KERNEL); + return -ENOMEM; per_cpu(hv_clock_per_cpu, cpu) = p; return p ? 0 : -ENOMEM;