From patchwork Thu Mar 2 15:15:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 9600963 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 542FB60429 for ; Thu, 2 Mar 2017 16:51:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4486126E97 for ; Thu, 2 Mar 2017 16:51:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 378F4285B2; Thu, 2 Mar 2017 16:51:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0815A26E97 for ; Thu, 2 Mar 2017 16:51:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753774AbdCBQva (ORCPT ); Thu, 2 Mar 2017 11:51:30 -0500 Received: from mail-dm3nam03on0050.outbound.protection.outlook.com ([104.47.41.50]:1516 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750787AbdCBQvW (ORCPT ); Thu, 2 Mar 2017 11:51:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=IDCOeVA1gi/ixYxaOXfoiw+dbWwfzH+BMjHgIjKwXEs=; b=FeyLYfLIfN5V9nXx1GGTRShfXnIzSz36xMTq3/KzIFFIUJMAei91vszOG3a2i7ORXg0nMumKPCE68h5d5qDiiGrLf47h+D/NhPv+RZBIl0KJAgA3a7HoNfOzPfBFsWLwQlBdAQU/nApi06cQwmR2xAIo4mLVnIg8WHLnw59RpSU= Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=amd.com; Received: from [127.0.1.1] (165.204.77.1) by CY4PR12MB1607.namprd12.prod.outlook.com (10.172.72.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.933.12; Thu, 2 Mar 2017 15:15:18 +0000 Subject: [RFC PATCH v2 14/32] x86: mm: Provide support to use memblock when spliting large pages From: Brijesh Singh To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Date: Thu, 2 Mar 2017 10:15:15 -0500 Message-ID: <148846771545.2349.9373586041426414252.stgit@brijesh-build-machine> In-Reply-To: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> References: <148846752022.2349.13667498174822419498.stgit@brijesh-build-machine> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: MWHPR10CA0003.namprd10.prod.outlook.com (10.172.48.13) To CY4PR12MB1607.namprd12.prod.outlook.com (10.172.72.16) X-MS-Office365-Filtering-Correlation-Id: e63e9228-9ea7-4dc2-d6d8-08d4617ef279 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081); SRVR:CY4PR12MB1607; X-Microsoft-Exchange-Diagnostics: 1; CY4PR12MB1607; 3:Co3z9vQo61C23f4Aj+cgEn9+qu1gmfKYGQ17mG3U5Qn+TvtqC6jplRGEMbmaYmaaovoPE/dJQGrj+s8rJVKflDpdAMDmwQx8Subc5pK0Dr8y/Hhr5bscck2oAdFe6IKqUDcH6tHYC/cwzBxfxyxn5sJLTyVcj/82IkyQM/rwBufvTmBa4TZEkEeZ6JRj+YeVk9hh3RviL0YE8Q+mhtL4tJOd/Wrvn/+4hCKCkLnrm5YW2Er5MvwGTOWaKWWGO/tCAb8HjR5/w6SloAwFbPQDmAxPuNm2GjwdLyI8dNYfL6U=; 25:nix/FMrA2Bc9KuIOhShngdxly/5ZrMvQFmI7ezSTtnMk8buiey8GAgq1XN6K/gikpXQSxXfDMDV+YihSIwgVgqev3zmUj+mhQcFeBcZ+yxSdwrKJWg03MITc85YM7g5awbP3phlaRf8R0sr9TxD3mQ7z0wu3yWLv3my6SlOla/DVkK2ozHMZaggODTl6UnVhPNxB3uWZ6zhIMRAv/KhgO00pjAexLf3l8qLltSSOX3843y9mTZ2Srud0s/pfqxjdFeoz3v204w0uSvNcF+uGrDIWz8AQVwN8oqtTMlY4+imKvVnYCLXKlPD6zoyK9Ix8gObc4jAmwPqlC9Q5WCBG+WLQiVosI751+BBQebk+Jd0DZ3kuPAT7WziY96y1hpqzk/280nYqHmhfP8XJ90w09jpYI/CRUxoYaYu3HyjliBq3WCk2bWetkgr6l6UwDUBn X-Microsoft-Exchange-Diagnostics: 1; CY4PR12MB1607; 31:WvQkbSpJB7CcfSdOSeCHj5szfrXl5POTkjsyFY7YLIeUFbyuDEmHNqTuhQlwcl0981pxlvRU56odoQQq8yPo5xYEg+iSF+rNvjrRchIaJm9FBQi0JvG8ftsBUFGZdoHk5Vve1bBhO4vpDs1L6TCiyv2w4gwWbOs+xzDJn94cGnc3QmQemwxqyvjPY0xoDexSZ/Cx2rQUHW2E1pKD1ckVirhbA7jYWEJ3V7AB/r/+6HIApFWSMAboUtrrY/TMSUgg; 20:9crmA/mIQ4UfLgxftoGCk34cWEhWNfjNDEVHmCS6WprkQiCemIUoqk9TrRefKQfSvd72juUhFeScHvhBTxAPYyJZmMCi+LnFTuqqbJNDRfbQgLzmUbDN6eN1tkFsxrN78mDGnvbx/bZGe7+PPFesoifKGjgZycy4bcqYHsoYk7RPOvVJ1NhH5BzxREX/mA2wD94YEvHzajQn6cV/K5kX9dLTpLNyPFN9N6vBXLZzyx1zqM4gwMuaTnZduCd63Mo86n+czPcz0LhYTm0VZSrZ98vhIoJtX7Oqn0ieg4pV4/Wl5KMVugt6jSIrRQGs+ClSgMTzjyN6riC+McxYOqUwowEEtpc8J/Ijxqckk4p4m2z/Fv59Rnm5NMi44dBn6rpvojxOvJ80oVJ0UFR8gbrDkGZEtUgZXgPwMcAjldF0euTvD9Z6DKTo0vUG4VNs5Oc6J+l0lt6dVzgMF2NACCthnDh454n5TW1IrU70sGDt9waDn/9AjRU3xI954SRb93o4 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026)(6041248)(20161123562025)(20161123558025)(20161123564025)(20161123560025)(20161123555025)(6072148); SRVR:CY4PR12MB1607; BCL:0; PCL:0; RULEID:; SRVR:CY4PR12MB1607; X-Microsoft-Exchange-Diagnostics: 1; CY4PR12MB1607; 4:RmoFkxn/AnXHylgctffrynMrBk7oJXSmk4GhUT44z2CejmbPZia2fUJIGNgs/29kuR9cQe0kh1LNTEDrk19lHQtFsF3GYaxvpPEyfYhkwxWre+lSSk+mffpwk4ep8dogCq9BFTSt5gH4xYCrlsab9KhgVXy7BwcUyIf3sVZLNfP0BtRBxcqbxwIHOA2r17JNkPAxXvHJm9XgZrZznF7CUDL2Wz1ZslIYJxIRoSPyhiecgRXBk9EASZAO0omqtpEOqXif9uF4Q0fRzDbuCkxbeQcbPjRZANX72z2KoMpH/KobWaamcLidp/PIpUMmR5z37twKVqiPaqfGIJRgv8B6qWmdT35dJbEITDFkgdZLPvq8SdiNYIuCBlT9ac8k3gHcmjJH11F/ghAcXzPb1hxC1EXGCH9QFDLsXQCNy5gX8ukLGFycZlrGRmacLOirac1xofEiQkikaS1A7JgA4Ed1v52MtcNPmCIC5T6b5SobaRNxrhKGUxU7uHqz9OGxaH35sf/PXXrzssJiXV8//1pputQjhyNN1cxJfmPMuWvApOhed8mOq0WZuEmOMvkaHL6udNbdrlbrE57GA9CSb0u6//MQCU5CxPV+aHI+zOyxYzDSWJmVg6Thitl+YWWnBa+2Q6EB0PrzNk0oW/jWkvjwWY9Jcynxp4RxhjI2Fap3o6E= X-Forefront-PRVS: 023495660C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(6049001)(7916002)(39410400002)(39840400002)(39850400002)(39860400002)(39450400003)(33716001)(53936002)(7736002)(38730400002)(6486002)(77096006)(25786008)(90366009)(189998001)(1191002)(305945005)(50986999)(76176999)(54356999)(33646002)(2950100002)(5660300001)(6666003)(83506001)(9686003)(6116002)(3846002)(92566002)(2201001)(230700001)(7406005)(86362001)(7366002)(7416002)(50466002)(2906002)(23676002)(42186005)(103116003)(47776003)(81166006)(66066001)(8676002)(921003)(1121003)(2101003)(83996005)(84006005)(217873001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR12MB1607; H:[127.0.1.1]; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTRQUjEyTUIxNjA3OzIzOlY4bFQ1Z2ZnY0dhRWxZSWYwZisrdVF4Tis5?= =?utf-8?B?WU1BeDBCV3NkclhTbW5yVUhaMjZqQUR3Qm9VMCtEVExoc3Q3emttVFF1bFMz?= =?utf-8?B?MUdMSDNIQ3Y2N3RGaWxPMG9jcHhsR3lEMjlJek5zSGVPR2xEVWZUd2FGMEdR?= =?utf-8?B?eHhtb21hQWp4WXI1UXVzUGVhM0krNlQ5em12K2RrZ1kzenlWbFMzeTVPMzFp?= =?utf-8?B?ek5qUjg4NGE3TFg3dDFzL2JPQzhsanV2NFNvWWV4VkdRTi80OW1xZUJhQlFa?= =?utf-8?B?ZHY3Rjl5Y3ZJT0wzOUhyYzEwVEREenNyaUNvVjRrRk5nemZhdWVQWmZoc2M2?= =?utf-8?B?OWhyN0hzT1l2ZUdudlVpcnFPcGJub2FQZ1Z3TmhWcEpucUc3RUIrLzZBYzBI?= =?utf-8?B?UjZUcXErUStadytpSTYxMGR0UGJORTRWZGFoNFExRTcrRFYrQmgvRTREOFZR?= =?utf-8?B?dGg2cVlIcm5aRVVaMUhDVXh4aHZPa0xSRUtMeXVjVUdVMXQ5bUtnbVowR1NY?= =?utf-8?B?V2ZzQ3dFcy9vVzJOMjUrbXdFWDV3RktkakJFeFVRSlBacDdxM3pKc0N6ejJK?= =?utf-8?B?dUhicmU3alBYUStwTDVINmhBdHJZNmM0aHlybkFMenhudkJoVThWS3h5aVVS?= =?utf-8?B?UVdsMlc3MzNxS1ZXKzlzN1lkSVhKSWYvQlhaNStrYi82OEZxaExZeDFlZXk4?= =?utf-8?B?dUxvVEQzWG1HMGNZOHB6dmUvTmFERnlPYXdmcm1peHJST2gwWXhhYjlXakFl?= =?utf-8?B?NmNEeUJ3czE5alNlTkFiYUpHWjV6SkJjSUdYejYzbXo3SWN5ZFMyVlpLcS9F?= =?utf-8?B?Yk5DbFpZdldZaURKN3BqczhERzZPWFR6STN1a1ZNZTcwczhpU21XeEc1d3Fm?= =?utf-8?B?a2kzQnR4bno5WUZXRTFEWlcrRGJjaHFjZEdsNkF4TlZMNmVndWoyakY3eW5I?= =?utf-8?B?R0x3aGVicWRUQ0V0bzVQUCt3VXo3NXU1bk5VOHIrd1VWOGV2R200d01YWEh0?= =?utf-8?B?OHZ3UDNObERPVUMzMXozSmJOMmQwaHZsWnExUThPVnFFMkVzaml0U292N0RZ?= =?utf-8?B?UVlrVys1Q05FRHhROE5sMGNxdlI2SzFYVytBczVjS2lBd1JFRHo3ZkJSMEJW?= =?utf-8?B?QzgyeDY4OWFlS1grM1A3YjZhZlpHMnlhSDlVdHJxZWtiZWVOSWZwWjBVUk45?= =?utf-8?B?Qkl2R3Jzb0pPRFZPOS9mZFNTZHlqYnZkdnl6Z0dLaEEzd2hOK3h5cE9sekVq?= =?utf-8?B?ZnVOS2RuT2dIVjVQdC9MWksySG9LZzE5NGtWeStSclo4cER1RG15cEllVG8y?= =?utf-8?B?Y0VBejRHY3hmYXdyanVrZ0VvbkVtYVAvcVZXRVRBTmxHWWkyb2FCMmIvc3dY?= =?utf-8?B?STNrc3JjV2VRTjJtWExHSTNaeEN6UWRmcGJQbHhsRjJFOHlxWnhHUHBtL2E3?= =?utf-8?B?L29TRnZBT1NpVERIblVjcHBEUTJ4MFBLOVppemZoUUdYL2R6ZWlxcE1Uclpp?= =?utf-8?B?NFEvTS9VU084cEZTTnhjWDhFcDdVVElYeEVZMFpzcGw5dWNFMHlKQ0ltUW1i?= =?utf-8?B?clFyZ1JxRitKS2JGZXc5b21oc0JLL3FZc25TTzFKeWYyTzN4NnB6NUxZalpr?= =?utf-8?B?QXlMeDk4c2pjZklsekNFcERzNUJsY1ByeHZndzAyMWFvV2pnZTY4cHN0OG8w?= =?utf-8?Q?R1VYyI3Mr7/oEtCn9656oHbWyCTO5C0brR14jKP?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR12MB1607; 6:JK5qlfsau02S6vBJ0G0deLYDkd4CZ1iF2WgPMEh+o/GCC3J9CxSDGRvTl5DglrPl1xWVOAXdQ8G4PexAp09VeZG4CA+FsGDOmVeNgn3CFdKdzmbMvm6QLJykJUMn/ZbpOiziaLq/iCUP9U3HHVy1zeDA16tG9KFP00WmNrQA1vlO+m/47FwUaE/kMEbylIB2Cf4W1mIkGV7HALAc+79VHtEHylbYUA/dN7GGg6ohVU57ky9zyrM3jKMh38B4z8rcvuSFzv5ZWm1+rrUr+wDu/9E42R4otmmwUfjUgu4kRgP9JH9adsIa2Y+Pa9l3XIqArcZY+Ptimk6r5DzKGdHP92NyIxgqbgwOmRgK02UrL2QTLcPxBTlOK135YkhoyrBEft6txhmvDk8EUqLP3p8/dg==; 5:Ial4jwScix+nbeWLlK0dNjja3cb1GBDKEbSFFmwaICgQIlchepi+05cohPhfos0L1e2nKMAOVRbbd5J9xhIWSqxtC/DJ0J5sTUHVIig5NTdE31uWxtiyJHcBR296TdxGnJbSC8sV0+pz6UeJRbWfIg==; 24:bwrzfI9cyPG9Df7DVwRNfm3ed7LUr8MhTpRb+8RKctQnnIxhHvy0GSBAH+FN3ljBnvfMbdJ+izaeAPb+2bJr9Wzl3GWZyDEYRwgXW8KCLs8= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR12MB1607; 7:HF7nAEHSYchmqM5GUYB4uZtvBLg2U166Z9JXjsI/Pf0PvOR8tvslfn6LxkrfSXXCzPnXmAuy/2+9FdfG0r76rquQPVy2oiT/1sO2Se9V9ZJ5zD3idIvUNjcUfURh8e0QT3Q1MohTnjnjhpAhf/AXult/0FRV3DQx17bBF8VCg5ZIz2Oa3WGnioXdOrsbOgRAd9CXqLFItbOfGEss9llB6uMuoJLqEPxRTXKQRQAy81DF2hcy6bF+4/VA49hW/uNTGUEGHfIIGjRtE3tyDw709PJged4NRz7Cgg0yFrnrFZWKAZRHQfoGSisJ29hKVr6HA701+S74UUCvG96JVZwqdQ==; 20:B6SIP8XpQHUKtRR3XwTyz+DYAJ4kWcUoWwXGNt2xPcS+Xj4QlJQ71IZOQ6wcAf4GR6EXiA9T20LWgfKQ5MAB5lmnTok7Vi6c9ijqtlK+JpMbY4RG2Ke/cbl01w7m2Rfpto0tlxkDoNsdxanMk6jOVbBKN+PuFxYgUW9gaQlUNzPMP8CHhUPv4D9uG+mdKJ7iYlfNLc7Fmfbk/g1NkzTDImr7ch5MdyOvm6Oa2Jd+MFpc6J5K8dKEaXwO0foVYNrF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Mar 2017 15:15:18.6364 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1607 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If kernel_maps_pages_in_pgd is called early in boot process to change the memory attributes then it fails to allocate memory when spliting large pages. The patch extends the cpa_data to provide the support to use memblock_alloc when slab allocator is not available. The feature will be used in Secure Encrypted Virtualization (SEV) mode, where we may need to change the memory region attributes in early boot process. Signed-off-by: Brijesh Singh --- arch/x86/mm/pageattr.c | 51 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 46cc89d..9e4ab3b 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -37,6 +38,7 @@ struct cpa_data { int flags; unsigned long pfn; unsigned force_split : 1; + unsigned force_memblock :1; int curpage; struct page **pages; }; @@ -627,9 +629,8 @@ try_preserve_large_page(pte_t *kpte, unsigned long address, static int __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, - struct page *base) + pte_t *pbase, unsigned long new_pfn) { - pte_t *pbase = (pte_t *)page_address(base); unsigned long ref_pfn, pfn, pfninc = 1; unsigned int i, level; pte_t *tmp; @@ -646,7 +647,7 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 1; } - paravirt_alloc_pte(&init_mm, page_to_pfn(base)); + paravirt_alloc_pte(&init_mm, new_pfn); switch (level) { case PG_LEVEL_2M: @@ -707,7 +708,8 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, * pagetable protections, the actual ptes set above control the * primary protection behavior: */ - __set_pmd_pte(kpte, address, mk_pte(base, __pgprot(_KERNPG_TABLE))); + __set_pmd_pte(kpte, address, + native_make_pte((new_pfn << PAGE_SHIFT) + _KERNPG_TABLE)); /* * Intel Atom errata AAH41 workaround. @@ -723,21 +725,50 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address, return 0; } +static pte_t *try_alloc_pte(struct cpa_data *cpa, unsigned long *pfn) +{ + unsigned long phys; + struct page *base; + + if (cpa->force_memblock) { + phys = memblock_alloc(PAGE_SIZE, PAGE_SIZE); + if (!phys) + return NULL; + *pfn = phys >> PAGE_SHIFT; + return (pte_t *)__va(phys); + } + + base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + if (!base) + return NULL; + *pfn = page_to_pfn(base); + return (pte_t *)page_address(base); +} + +static void try_free_pte(struct cpa_data *cpa, pte_t *pte) +{ + if (cpa->force_memblock) + memblock_free(__pa(pte), PAGE_SIZE); + else + __free_page((struct page *)pte); +} + static int split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address) { - struct page *base; + pte_t *new_pte; + unsigned long new_pfn; if (!debug_pagealloc_enabled()) spin_unlock(&cpa_lock); - base = alloc_pages(GFP_KERNEL | __GFP_NOTRACK, 0); + new_pte = try_alloc_pte(cpa, &new_pfn); if (!debug_pagealloc_enabled()) spin_lock(&cpa_lock); - if (!base) + if (!new_pte) return -ENOMEM; - if (__split_large_page(cpa, kpte, address, base)) - __free_page(base); + if (__split_large_page(cpa, kpte, address, new_pte, new_pfn)) + try_free_pte(cpa, new_pte); return 0; } @@ -2035,6 +2066,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { int retval = -EINVAL; + int use_memblock = !slab_is_available(); struct cpa_data cpa = { .vaddr = &address, @@ -2044,6 +2076,7 @@ int kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, .mask_set = __pgprot(0), .mask_clr = __pgprot(0), .flags = 0, + .force_memblock = use_memblock, }; if (!(__supported_pte_mask & _PAGE_NX))