From patchwork Tue Dec 5 01:04:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10091877 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id EC74F60327 for ; Tue, 5 Dec 2017 01:10:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0AA72465D for ; Tue, 5 Dec 2017 01:10:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D47B129452; Tue, 5 Dec 2017 01:10:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1223E2465D for ; Tue, 5 Dec 2017 01:10:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752292AbdLEBJj (ORCPT ); Mon, 4 Dec 2017 20:09:39 -0500 Received: from mail-bn3nam01on0088.outbound.protection.outlook.com ([104.47.33.88]:33708 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752034AbdLEBFz (ORCPT ); Mon, 4 Dec 2017 20:05:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=7odhlEjXUDt/jOMqyFS2BNLCsThZRR2U8qp2NpRk0CE=; b=SMAWpoe9QKKh7m6d05v2tAQV4Q/tzp+LD5CKYm4zet5KTKs9tdJRLQGvhZYicsuPThUd05SB/EySNpmxMY4lA5U2WjSrZB0LATQ74HRkDC+bbvDECkcZxwSPEqY7phOu6pjxDKqJdwN7yhbyGboM785aOCttqMMbZZ8YIMZLWXs= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from wsp141597wss.amd.com (165.204.78.1) by CY1PR12MB0149.namprd12.prod.outlook.com (10.161.173.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.282.5; Tue, 5 Dec 2017 01:05:19 +0000 From: Brijesh Singh To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: bp@alien8.de, Brijesh Singh , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Joerg Roedel , Borislav Petkov , Tom Lendacky Subject: [Part2 PATCH v9 28/38] KVM: SVM: Add support for KVM_SEV_LAUNCH_UPDATE_DATA command Date: Mon, 4 Dec 2017 19:04:28 -0600 Message-Id: <20171205010438.5773-29-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171205010438.5773-1-brijesh.singh@amd.com> References: <20171205010438.5773-1-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: CY4PR0601CA0087.namprd06.prod.outlook.com (52.132.96.156) To CY1PR12MB0149.namprd12.prod.outlook.com (10.161.173.19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 57a682b8-fcf2-41ba-1839-08d53b7c4183 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603286); SRVR:CY1PR12MB0149; X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0149; 3:afqe6LNxmXQwVaLQud6W19aADiKNK1dFErMIPrQqh0ubC/2j578wZNgH8J9/n7HAqCaAzk7ouOzujYTcFobum4nkBUcLU38DAmn0OJoFZxKxTITxTkLGfSHTsh6MtNeMIMiECzWO9SH3vkyYlvFAzXJQZe5BkkAzZWLSDlQdOmSeZLyPbddz4TmEjYLlsVKR0XF3t2xbwExwtufprcLTVlb3I+8PhcGYKHA4GTAXS7e5cX73juH9QGfJWK5x+QtL; 25:cksG4FsmktfQdh9bEwqqS0d76rt9ve6Nb6VGMeNNfEcxdj49hRVXL2QjE2D/uu9/xRbIkzmvCd6WNiyxmVtIra6Ha+wsRgH/3wIotmWq4AsQy1KTmNIdePdwUyLyDADvPFjHGFoq2vHIjzfQ0zhk5DuKIgazI/Zv+NBK7iX+qREpRxWd/nR0oE1wtj0xW8WBLUw2P3g3gynz3JHTo8u5jZglQIP7cPsCrKQ2Y7kToCMFNEOuvzQ+9CVHd5RlOldUcy1Of+fcCWOWVyMXYEebcx4mrPQvhrkK5JnUfdrNXNtA9Re3S4ATXW5Kn4LDHPfrCZCISriZLdMsyrQPJxF37g==; 31:LpbZW89M5AfmQLSpiTHJOkFX8DnYPVRHwmPohgi4htr38LrBc/2qyqt4oMPKGejravoTc/nwGAyX9oS+cHrp/syL4wUn02JgViclDqNINwKYNaxv2hiG4f2j7aKctbb9L8UmVgzM09iGw2JKPN4Rp9yGTrDgR5NdCTB+gfXhwrRTG1MyAREjDnjzdRei8/TrBm8uqA7Nu+ufbMS2GoHvGymGg3uVJC5vEyk5eo8XjK8= X-MS-TrafficTypeDiagnostic: CY1PR12MB0149: X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0149; 20:PdfJzPjAsPA8CzTmKCg9Yij2PqP2JQX4w3zsRsle6VZmjM/vPPagGLxTkgkm5RUG0lvxrnjtTu8D0C1NbuRwCwelztUhVVdtfuUd0LN3S25KYImm9qLlG1dcLl3ZyDg7HxWFFuaIcq3iDqpUxnCgFMRGoramhJcAFyAlE8QZtZJm2Us3wOHx2eIFmvRbcVGgkZLSXxFMeU4GAFMk+ajWI9jvY28uFGYr+z4xmxYn8QMgflZyxiuHRtipbOAfflPUm64b9daw931evVJ63wBsLtcnk+HeAiG2bXgrGE7Wlz998y04E8dDviTW7tdU29YsPmy2CS5cndS8dK0Pn5YrfWM/nFzlHO7aL60rqXs7EWupa99/ehS8ovbvJsCnHH2nxUFtBkGJOrTn52k7RboTk0e+G87OE3LPxKcPfnAgYoZR3pbdTouK6LwNNX0aVktNrCfydfGmch1uhX7mbjpj6iY1UnL4WAInGKFFEKj+MmE9YVTlJUhdNrTWeDMWn5cp; 4:mpYbyBCnCaAgKekhwtmgIeUdsJjRWfHVTaA4XnKbLDs3aN9XLQonIlXuFwAjuXiX5lVwg3STBC4maoVJ+altHv7tEg4rXQ3GU8q87VWGcpNsuOGyaFsWuXBKD63XJjxpyYLt64l/zly1aiWLl43oADNLaaVsOIWXnn3PEoVyyIZHjPJUy/VDfS4zzi2+fYmZ01V655JEgftZJz7tkdDSDtyXpetbdCyZtVoGp0NrzPqimwm1U0pnucKX9jp2pWLQ6kuZdfMRKGiB+SoTWrnwQS0iRFcaMmfKQz2pLeBk5i4PS49F2s0pbiVX8+OoilYShJc3ECO0mU8n2o7fZ6OFGA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3002001)(3231022)(6055026)(6041248)(20161123558100)(20161123555025)(20161123562025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(6072148)(201708071742011); SRVR:CY1PR12MB0149; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:CY1PR12MB0149; X-Forefront-PRVS: 0512CC5201 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(366004)(39860400002)(346002)(376002)(189002)(199003)(16526018)(189998001)(106356001)(478600001)(7416002)(2870700001)(101416001)(23676004)(52116002)(97736004)(86362001)(575784001)(7696005)(33646002)(54906003)(25786009)(105586002)(76176011)(316002)(2950100002)(6666003)(7736002)(2906002)(6486002)(50226002)(81166006)(81156014)(8936002)(53936002)(8676002)(4326008)(1076002)(305945005)(66066001)(6116002)(53416004)(5660300001)(47776003)(68736007)(3846002)(50466002)(36756003); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR12MB0149; H:wsp141597wss.amd.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTFQUjEyTUIwMTQ5OzIzOi9tZmcyeWlPeWVnVnAxZkZoWVljc2VKc0d3?= =?utf-8?B?aHp0UldYMHB5RmxTSU8wNjc1bzdLdksrY2M1blFodnJ6b2thcHhqQVlsL2dt?= =?utf-8?B?NURIN0E3TGttR1N3NHQ2OGdtUVVvR3lHSmowei8rV2xDbTlWWDA4ZVo4ZW55?= =?utf-8?B?aVB0aHJPa2FaZmo0cFRESEFSQ0Fpc1lEN1MzNkttRE4yRjBMVkt2d2lOTmRE?= =?utf-8?B?TTlOajRIMVhuMUsyMUhLMkxEdllOcjVIU28ySWo4eC9LMGYwNDVXeTVwYitX?= =?utf-8?B?dGl2WFRDb1MyQWhZMmUzV2pxSXZnMmV3QlArQU1ycGt3OHFlOUh5VTlEZCt5?= =?utf-8?B?NzhvV3hqWW1XWGk5cGcvSUtXWUdab0tuTXBIUTJucTU1OGJyZXFtaWg4NHhn?= =?utf-8?B?dFkwVHk1QnNZQmlZaDkxMU9IQXFCTVdvdWNWa1l3VDQvZTVWcDlCUjBnWDlH?= =?utf-8?B?SU5tdENhTy9iNCtPS0Y3QytKTUlRUGVoYlg3bUlTVm5CektkMFE0VGxkQ1dZ?= =?utf-8?B?Zms1aFB6Tm5GN3ZKa2VCREszMFhqaGpueDFrUXl0QU41UkkwZm9ZaWIyVVkz?= =?utf-8?B?ZHZCcWFXdmJNYkdXRWUyU2xMQmV3UnBJUm9PWEw5WjkyM3cyV3FyRlYzWmtO?= =?utf-8?B?aGpTUmVRdDBVSmNzVStKWk9qSFRlZEJBMENYVEpZK3VMbFptWnVadURsNkRs?= =?utf-8?B?eEphZC90anpBbVZaWFhUOTVvekIzcHJ5QW5yS1ZXK1oxSTVBZ08rdVlGWnNO?= =?utf-8?B?d0RrWklEUEd2cGw0VWFsUDc2bHNOUG91TGRPcFlnMk1DN1RldkpOMjl6a3VQ?= =?utf-8?B?c1d4UjdvRDBMdUtVYm9HRThxdWpJcUJwcjVEdVM2VkpXNk4rQXlycU56WWdM?= =?utf-8?B?UmhGeEQrcW5qdkJxVXhiVm5TeGRYUDdvYVdtbTRINERXWVRHRiswS2dhRkZ2?= =?utf-8?B?elp1U3lHdWJKaGJLQ2xCVFg2VEF5SDdKVFFMTEFaUjBmNUFDWnZXZW1GTmJj?= =?utf-8?B?d21yd05NWWhUMEZrc0FOc3dvdjFiYlRqamR6Qks0d0NSRHZxaXFTZGwwemwx?= =?utf-8?B?alhoMit5cTExeGpBRkQzZ3J6NUVnQ0lEY0h1THdOU1E1bEtTbWJleGFGbTNO?= =?utf-8?B?MFBINkxxd25GaXRIOVdoajBZeFRJQUE4U1daM0h6WE9rYStTcVVyOVpXc2Ni?= =?utf-8?B?WElaMjltYUEwaXVZMVFYNmFQQ2lkMUw4YlBxSkwrbXNWRXl1TUpveDQ4anZa?= =?utf-8?B?WnByTWRFRS9Ndkl6eFRRN0tsVjRTMEs1NFF6TEpjbGwrejBvY2FONzVXMndZ?= =?utf-8?B?T2VoRExjSDRzVGd1WFM1VW9sZWQ4azJpbHpEN290NW9wM0RDZks0dXYvWUcr?= =?utf-8?B?UGd1MHVxNHlWcmFMbEhRdTFEUkQ2d1hwQndVVHpscGc0OFpkVEhxOGxJVHRS?= =?utf-8?B?V0NzVG5uUmNsRldsQWl3REZqQTVIdWZVSXVtRFYwd1AySUNFam9IelQzUW9G?= =?utf-8?Q?HN+Ynhdh5cMqY2kVL5zOdXCbhuy68lauoRPdBps+XOF7VV?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0149; 6:7lWAUx17aauM3I/9U041rc3L30KCwKOSUGsY6r5yQtlu2VI9pX162+8RbGYYUKh8d9D9Gt9Kn0TlYn1EjpOkKHJV0GOGJCU/7S6xXAHBotcEh8Bc1dDSmi/3HPz8E6szpJMfCduk+opZNS/xIEHkqnUuE/sz2aDLeqEoedBW4zlIUIgvGR+DwY1+RIXFI1iSOiO15DOGxMt2PWL2ruNTWiIrAD1NY3rHNSu15evifOjAdmxnBXHlubBQ0n5K0ojOikuiwKqbs/jyiTV5q46ACCzPjbBcAylI+Mjf+brNPj8ORkVh7AsL5ZYo7SzKJjEf91IAD+T7qSqw1pPZhDnjz5UOrtSCcRextT/XE9B6mbA=; 5:TlKKdp6/I1y8zVCXsK4FhQG0uNUwJfTp63KquGlCdxLK0sTK/blbIn/OWSvQchRStHg2wp0b7/Wrhf4W8Ap5Mye39iZusKc41IcSLbfFjagrPZ9jOUjgEfTwZwLsSCZrDiKhV+39PmHkO38bSR3+JA4jAdbO9B9MC2DUpYrZuWE=; 24:3CD1d8Tn2PcHQ7i+xoxnCN+WPUOu3utQVFUfqGo7F80fv1A5IREorX/rOLsE6OIr0nGKOMs/Fg3f9Q/0yy01uGIf2lbb5Onsi2SozI+kkxE=; 7:vzKinxWQ/0SdaH84fXwkFdL/+tEUMyWnpRcGIyYidckcuFxkHtwnj4zBPNbOV1nAjN2FQIM349S3YOx6kB57fXNjzWmbbfRbE5spw0c8KLuRN93paWIUevOwkCExdxBeSrdR8sF/t83k80FGB04SQkY1vSiK9lPh4lDbnJ+J+7k3Df2/2cjq5mrjBGxxH6esi6w37oCtY/FQJbbKQFvlTHFSXmrF3zWNydIi9eJX4IJl9kwI3e9CX8BRBlqpuKsE SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR12MB0149; 20:cIwgWveGP1VP7mVm0XlHfm8b+ixkpYKeacEwawQAg0m4p7Gf9FkBAEKgxnkaU8I1jb73BbXw/6it8qvJE705QPDE2+nkAlmReZLfRvNqPoDoW9ySEXc/zssSTf6AhmlD0oDVdsRffCnHDaj9wY2Xc9O6WfCzAPooUjUJpXYgcX6H86mRBnOqdtprtoe/YvphmInA6xA+VLGdIsRRbdDh/6SxUaCKxT3PE+UEVmRMKOA45CnwjQnrbChPK8/Wbt0P X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2017 01:05:19.7949 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 57a682b8-fcf2-41ba-1839-08d53b7c4183 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB0149 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The command is used for encrypting the guest memory region using the VM encryption key (VEK) created during KVM_SEV_LAUNCH_START. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Joerg Roedel Cc: Borislav Petkov Cc: Tom Lendacky Cc: x86@kernel.org Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Improvements-by: Borislav Petkov Signed-off-by: Brijesh Singh Reviewed-by: Borislav Petkov --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm.c | 191 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 190 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 0ea890375532..a0b021f1fd05 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -752,6 +752,7 @@ struct kvm_sev_info { unsigned int asid; /* ASID used for this guest */ unsigned int handle; /* SEV firmware handle */ int fd; /* SEV device fd */ + unsigned long pages_locked; /* Number of pages locked */ }; struct kvm_arch { diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index e5b712e55186..88951cbef3ec 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -39,6 +39,8 @@ #include #include #include +#include +#include #include #include @@ -331,6 +333,7 @@ enum { static unsigned int max_sev_asid; static unsigned int min_sev_asid; static unsigned long *sev_asid_bitmap; +#define __sme_page_pa(x) __sme_set(page_to_pfn(x) << PAGE_SHIFT) static inline bool svm_sev_enabled(void) { @@ -1569,6 +1572,83 @@ static void sev_unbind_asid(struct kvm *kvm, unsigned int handle) kfree(decommission); } +static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, + unsigned long ulen, unsigned long *n, + int write) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + unsigned long npages, npinned, size; + unsigned long locked, lock_limit; + struct page **pages; + int first, last; + + /* Calculate number of pages. */ + first = (uaddr & PAGE_MASK) >> PAGE_SHIFT; + last = ((uaddr + ulen - 1) & PAGE_MASK) >> PAGE_SHIFT; + npages = (last - first + 1); + + locked = sev->pages_locked + npages; + lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { + pr_err("SEV: %lu locked pages exceed the lock limit of %lu.\n", locked, lock_limit); + return NULL; + } + + /* Avoid using vmalloc for smaller buffers. */ + size = npages * sizeof(struct page *); + if (size > PAGE_SIZE) + pages = vmalloc(size); + else + pages = kmalloc(size, GFP_KERNEL); + + if (!pages) + return NULL; + + /* Pin the user virtual address. */ + npinned = get_user_pages_fast(uaddr, npages, write ? FOLL_WRITE : 0, pages); + if (npinned != npages) { + pr_err("SEV: Failure locking %lu pages.\n", npages); + goto err; + } + + *n = npages; + sev->pages_locked = locked; + + return pages; + +err: + if (npinned > 0) + release_pages(pages, npinned); + + kvfree(pages); + return NULL; +} + +static void sev_unpin_memory(struct kvm *kvm, struct page **pages, + unsigned long npages) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + + release_pages(pages, npages); + kvfree(pages); + sev->pages_locked -= npages; +} + +static void sev_clflush_pages(struct page *pages[], unsigned long npages) +{ + uint8_t *page_virtual; + unsigned long i; + + if (npages == 0 || pages == NULL) + return; + + for (i = 0; i < npages; i++) { + page_virtual = kmap_atomic(pages[i]); + clflush_cache_range(page_virtual, PAGE_SIZE); + kunmap_atomic(page_virtual); + } +} + static void sev_vm_destroy(struct kvm *kvm) { struct kvm_sev_info *sev = &kvm->arch.sev_info; @@ -5767,7 +5847,7 @@ static int sev_bind_asid(struct kvm *kvm, unsigned int handle, int *error) return ret; } -static int sev_issue_cmd(int fd, int id, void *data, int *error) +static int __sev_issue_cmd(int fd, int id, void *data, int *error) { struct fd f; int ret; @@ -5782,6 +5862,13 @@ static int sev_issue_cmd(int fd, int id, void *data, int *error) return ret; } +static int sev_issue_cmd(struct kvm *kvm, int id, void *data, int *error) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + + return __sev_issue_cmd(sev->fd, id, data, error); +} + static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) { struct kvm_sev_info *sev = &kvm->arch.sev_info; @@ -5829,7 +5916,7 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) start->policy = params.policy; /* create memory encryption context */ - ret = sev_issue_cmd(argp->sev_fd, SEV_CMD_LAUNCH_START, start, error); + ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_LAUNCH_START, start, error); if (ret) goto e_free_session; @@ -5858,6 +5945,103 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) return ret; } +static int get_num_contig_pages(int idx, struct page **inpages, + unsigned long npages) +{ + unsigned long paddr, next_paddr; + int i = idx + 1, pages = 1; + + /* find the number of contiguous pages starting from idx */ + paddr = __sme_page_pa(inpages[idx]); + while (i < npages) { + next_paddr = __sme_page_pa(inpages[i++]); + if ((paddr + PAGE_SIZE) == next_paddr) { + pages++; + paddr = next_paddr; + continue; + } + break; + } + + return pages; +} + +static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) +{ + unsigned long vaddr, vaddr_end, next_vaddr, npages, size; + struct kvm_sev_info *sev = &kvm->arch.sev_info; + struct kvm_sev_launch_update_data params; + struct sev_data_launch_update_data *data; + struct page **inpages; + int i, ret, pages; + + if (!sev_guest(kvm)) + return -ENOTTY; + + if (copy_from_user(¶ms, (void __user *)(uintptr_t)argp->data, sizeof(params))) + return -EFAULT; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + vaddr = params.uaddr; + size = params.len; + vaddr_end = vaddr + size; + + /* Lock the user memory. */ + inpages = sev_pin_memory(kvm, vaddr, size, &npages, 1); + if (!inpages) { + ret = -ENOMEM; + goto e_free; + } + + /* + * The LAUNCH_UPDATE command will perform in-place encryption of the + * memory content (i.e it will write the same memory region with C=1). + * It's possible that the cache may contain the data with C=0, i.e., + * unencrypted so invalidate it first. + */ + sev_clflush_pages(inpages, npages); + + for (i = 0; vaddr < vaddr_end; vaddr = next_vaddr, i += pages) { + int offset, len; + + /* + * If the user buffer is not page-aligned, calculate the offset + * within the page. + */ + offset = vaddr & (PAGE_SIZE - 1); + + /* Calculate the number of pages that can be encrypted in one go. */ + pages = get_num_contig_pages(i, inpages, npages); + + len = min_t(size_t, ((pages * PAGE_SIZE) - offset), size); + + data->handle = sev->handle; + data->len = len; + data->address = __sme_page_pa(inpages[i]) + offset; + ret = sev_issue_cmd(kvm, SEV_CMD_LAUNCH_UPDATE_DATA, data, &argp->error); + if (ret) + goto e_unpin; + + size -= len; + next_vaddr = vaddr + len; + } + +e_unpin: + /* content of memory is updated, mark pages dirty */ + for (i = 0; i < npages; i++) { + set_page_dirty_lock(inpages[i]); + mark_page_accessed(inpages[i]); + } + /* unlock the user pages */ + sev_unpin_memory(kvm, inpages, npages); +e_free: + kfree(data); + return ret; +} + static int svm_mem_enc_op(struct kvm *kvm, void __user *argp) { struct kvm_sev_cmd sev_cmd; @@ -5878,6 +6062,9 @@ static int svm_mem_enc_op(struct kvm *kvm, void __user *argp) case KVM_SEV_LAUNCH_START: r = sev_launch_start(kvm, &sev_cmd); break; + case KVM_SEV_LAUNCH_UPDATE_DATA: + r = sev_launch_update_data(kvm, &sev_cmd); + break; default: r = -EINVAL; goto out;