From patchwork Mon Nov 6 18:11:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10043997 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AA9D2603FF for ; Mon, 6 Nov 2017 18:13:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0F9329BEC for ; Mon, 6 Nov 2017 18:13:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 94FC329F54; Mon, 6 Nov 2017 18:13:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AFCC429BEC for ; Mon, 6 Nov 2017 18:13:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932796AbdKFSMx (ORCPT ); Mon, 6 Nov 2017 13:12:53 -0500 Received: from mail-by2nam03on0078.outbound.protection.outlook.com ([104.47.42.78]:49424 "EHLO NAM03-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932775AbdKFSMr (ORCPT ); Mon, 6 Nov 2017 13:12:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=XzTdvf70v6RqfPeh2xzmexAO7sYoauPnqHv2+Q05XpM=; b=CSrTpBfnTsV+GjocODziXfXil7I5S5ZucvfTKEwcC89I3HMloYXU7I7+kzETv5haYDSqgvVv9dpqGT+pcKN58+OeSPzGkDmP1hAThcpGYEccjOkFA8c1esxpQrPHYOEyqGDlidgF2NqCFLbw+dHk8mcemxoxhcO3gndYBZQ+6+g= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Received: from wsp141597wss.amd.com (165.204.78.1) by SN1PR12MB0159.namprd12.prod.outlook.com (10.162.3.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.197.13; Mon, 6 Nov 2017 18:12:30 +0000 From: Brijesh Singh To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: bp@alien8.de, Brijesh Singh , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= , Joerg Roedel , Borislav Petkov , Tom Lendacky , x86@kernel.org Subject: [Part2 PATCH v8 28/38] KVM: SVM: Add support for KVM_SEV_LAUNCH_UPDATE_DATA command Date: Mon, 6 Nov 2017 12:11:20 -0600 Message-Id: <20171106181130.68491-29-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171106181130.68491-1-brijesh.singh@amd.com> References: <20171106181130.68491-1-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: BN6PR10CA0031.namprd10.prod.outlook.com (10.175.102.145) To SN1PR12MB0159.namprd12.prod.outlook.com (10.162.3.146) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 023c6ef5-1b61-4d0a-7f55-08d52541f2c3 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(2017052603199); SRVR:SN1PR12MB0159; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0159; 3:FmQIbZAJ/PLVN4rCzPaOHHaSZJuMeemagziQ2GTjE8MbE0gGE033vqE0iR9ewWZxCQYjmK1u2dPKcfrVDjWbyYy+kwf4cOzrZrkwn6oNNuto2eND7cod/JXdK0E8zYqDSOpZYs6EPsfWubE5ttCHmie9epZc22odKs7X1XlcTWrEEbiCyMkMdYHMiUKuKmtPD9zgB2AXLTC3jN21yrfK6BsLtNTw9/z3L7ili2j7nZxzFxCoK1m14rraGDXCpM4g; 25:8KxMC816lDBDyKpAS8be6XVRrye8cpCRF5b00EID5C1QU3VYusZxOQZYvpbLz2mBqT01Y3tr/NKD3d3HtvEc7cgAEKjSHWFictBk3e9Z86oLUlrq8Vs/cWRnhXPdlzn1U/3Iamp9aQSjWg1tGytMqWBjL/XphcLHWaZEoC6AiSqqe7UVPFbuLqyF08lSChf/87ZZNazJEVUokRQtkY9PEScY39x2SGY1hiKRw1WSf+CXQKrDCeTJxXWtO9gT8YNo0qHKZ6N4B8VoTHkgWQ290dGA008X5aV0Rw96H8J0fu2n4kHXM7JSbrwztQdaF00WTVSUrCvSE13zKv6ynkkDIA==; 31:zkMYpoyPWUi+AzFR3erkAjGQ4z5zfmCcUcmAo4cDRFRAccl+PVhMYieJMsonbxjKFMUGLCGc4U3prYuYh6DtZHCOq/z+2nYKSrS5uRLH/FaI7Qd6N5s1iq+VWp+F7nPGF3rziEkHSGW+Rye1j4/lC2be6zo/uWbRtx64QknpZv7ysGPS2iP/2hs7fi3oVwysmPPmdUtunMziRbKZLZOCrlasQZ8g+2rYGV1CHpFypa8= X-MS-TrafficTypeDiagnostic: SN1PR12MB0159: X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0159; 20:Lc29ia/xIMIioyZl1ZB3wCoyTlofje1AgDFqdVTYFfVYIoK8gs4xEGPntD9sN++r1Ff3Ndm85nlrT97TcvgYlMKzHiX8xI8EKgdhLQV6nwqOjdMdY9BoLH6DL4KLod/PXd0P57MT+Uqebr1Q6eQrE9MRcUsBcMz76YkJyNfLog1TyYGEAtwdR/gWOVj025tZHiies27G5tc3tNFg7kP7Fk/UZrJG9sqHOGxLOagbc9764vX/3jN4M2HbDJn4+xe4g6IIKN70cCi4oKmSgIUAuPjLkZM8qmkpSVnw/PIk/5GqKskxHhU4hiSY9wubxLJQADELbBdSBnJm55Bf+t2qM0swrZ6aTToq+SFYP12jcXyvFN2mhF9p+iAqRpXM7LTvjWhdYGtGZiIQ5ZOcTvhv6rI+SUOf7UKkJMeU/z0bEmrfDeCRuGVhHQ1rGowUjexWx9Nh9n1cKL20InlCH4M8paPKWgCqIj4urtjoEcS0f7uZ/YTej2r53AKidxkJ4Q0B; 4:BM0brrKc7vVoGKBRlHaHla6H/bEjviLxzoBSA8rSGmklvZiaMxxgMrA2GB3GCFsEDm6sTpJ+S6v6eTaz3FnEq27q4b/iRVS419d0juMXDgKHlJCy0NN4AiF5jrt2bQ7GCzjFhDGE8ngmRm2zjUTkUpPN6dFbq8y52zGcPpm+fsddisLosebs9E0H+loaro9qWV4TrvwtEvw2DNWqBod+v3y5c3Gui6YO5FkUrzI2GzieHSk3aAq434uCUAfI2F1N3ICB4O0n3/1mO0QIxgqz2V3qJO5OnOX2naMFPktYr/zlq8RmmatjcntcLmdX03XCXajOz0Ht3nbOxGSue8d1ow== X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(3231021)(100000703101)(100105400095)(3002001)(93006095)(93001095)(10201501046)(6055026)(6041248)(20161123564025)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:SN1PR12MB0159; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SN1PR12MB0159; X-Forefront-PRVS: 048396AFA0 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(39860400002)(189002)(199003)(68736007)(97736004)(4326008)(316002)(16526018)(54906003)(7736002)(8936002)(2906002)(81166006)(81156014)(25786009)(53936002)(8676002)(2870700001)(1076002)(6116002)(23676003)(50226002)(189998001)(305945005)(86362001)(3846002)(50466002)(33646002)(66066001)(36756003)(53416004)(47776003)(5660300001)(7416002)(76176999)(478600001)(6486002)(105586002)(50986999)(2950100002)(101416001)(106356001)(6666003); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR12MB0159; H:wsp141597wss.amd.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtTTjFQUjEyTUIwMTU5OzIzOkhVQXR0bGpuSzZFUWRqZHhOdUR3U0xjRmFX?= =?utf-8?B?eFVLcWJxcngzTmNqaDhiQzhlR2RnV0Ztb280VlJwSXRydHdaZ2JaOTdIRy84?= =?utf-8?B?K3JENTNvVFdtR3p0NWJzQ2NZaW1uQzk1dVQ1YVQ5MFFOUHlSWXRjN21hcFFU?= =?utf-8?B?a3VqSVhvemxPWXJlYStxMlN6VVAweCtpbDB1Qno5VldMS3BQNDhrMzU0cnhJ?= =?utf-8?B?N1BNd1JZR2RHUzB5ckxPeWZ0TlpZMGplamZETlhqS0pyeVF2U3hMSW1DelJE?= =?utf-8?B?QmxDWHVFWXJJK05BaUtiYi9leHNNU2NNNUNlTllCMzhMWCs3WVR2cHFueTQw?= =?utf-8?B?Smh4Vy96TGlkWTJkSW8yM3JRbXJKMEVrbzRIZjUyQWdzUXE0TDJ5c2JEQStz?= =?utf-8?B?cXo5Z0l1SHNkSjNxb0thRWRIT1RTMGRJaVl6LzRycUpzL2ZQSWhvTVRsMm5o?= =?utf-8?B?dzh5c28veHFDK0VxWmZkTWl4Vnd4ODlaR0tSMmIzOHpxNWpLclAyd0pYeFNO?= =?utf-8?B?TkV3KzBWK1NhRzlpWlc1dDZXKzdPVXRsQ05IUVZ4OUJrU2s4NW1LTXN0WkN0?= =?utf-8?B?TTM0ekdJd0dPbGM4a2M0MmlRZ00xRllrOFFHc1A4ZkM2U3BYMjVYOW84QmEv?= =?utf-8?B?bUZhcTY1U1ZnVnF6eE1GN3BBYXI4dTNycTBlMTZ6VkRFeUs0b1Z0QjFiRHI1?= =?utf-8?B?U2ppV1FpREVSRStkVzRiK1FtUWp6cTVHYnBwTzk2di85eEFpVDJ5QktML3Vw?= =?utf-8?B?dStIZUFrUEJWcVVmakNyMm4wRmRCWnNIMVQvanJSQzE0WjJ3SndycFZmVWYr?= =?utf-8?B?N1hHZVVQMVN6Tng4RXhWeE84RVhXdS9EYnFINkFJZktoa0lIZzFrR2NSNW1J?= =?utf-8?B?RDMyL2NLUld6MWhXS3Z1OU5HNEJzbFdkd1JwV3NuaVFtbjBvckxDaWxOWmNa?= =?utf-8?B?LzdERUhZVlptb0VNczBIWlJOQi9UL0x6bVN1aFo3ZzBpWnpvQTNobEJER1lV?= =?utf-8?B?eGNOb3BtV3ZZWE0zVDNrZHFVL0tIcHMvSzdwUUlYVzlmaGVoeVBPaElLbGhF?= =?utf-8?B?MFl1eDNkMlRFZUtwdGRTeEIrTWNzTGVkemlGVnVrYXdDbzREYzBrVjI2WlU5?= =?utf-8?B?b24wcDRERk5rRDhLRGljVDNSYkNwS0hjbnYyc3JvTzVVSm5weHU5S1BXd0V6?= =?utf-8?B?clB3WHIveEllREdlWXRmc2k5UCs4TEMyRmhuSEZTaHdERjZlemJkRjVMMWhG?= =?utf-8?B?YTJ1UEZoU0N5TWo1dkFzck41ODRha2lXdXdOUHdLdk5sSUxkVlcxSG5KQ1A2?= =?utf-8?B?dDBrN1VoM0FGSmNrd2RzcmZIb3h4Z2dINEw4dEVnK0hXLzhYMEFaclB3VDFK?= =?utf-8?B?OFVQSWtpeExDQVFTUGh5S3hkQnovbFEvQlZhRDZ3b2d0UytSaWhETHFPNDRa?= =?utf-8?Q?4JAdFbKCreWAqLZXYmV/yWtmZL2?= X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0159; 6:xn//+QkaeNZn3c1AIXNC64i+MOCV2cLdSZiA+aiEWoNmwbvUfrCRV3SfKtFHU3oXPMptpfSwH0+t2RlcuCbIga8fpGVKTEutGOInHnzlOkaqex9tty4U4+H44u5+XniBLfxVHxAwZiEdBM0v1MKhB7zRoIpnXZKQejYM1pOKI8UREjPD47yFW8xgToHnEZvC2qYdRZ7IGQoCKsFTRbYM9pXap8S0qZLlJR98MFWEqqIUAMh/kkhXM8VaAxaeUFRCa4olFP3wqF89KqujhkXpAXXdjsFM/miqAbCyTdqCgobsj32aP3/EqGIudZ4XzWMMqHUR1IC1SsoG2Bn/Cc4Y1aD+PBtJYdmjRywC0vPvFJE=; 5:X0ZpVtVq70pd5o2fpO9X3/T0OZedjp13CIbClVK6bl9ovTqv2C3R17apr3eNy1zpAU8Zc6MyNh83Wq54EvAEo3G6Ksi4mx0hQjs6Hy74BuwhTaTwKL/fTWyJLV+MYEZvmkYqI7FrZ/to3LT7hmjDHDHF5k2IHC2EV5t7U9wcUAo=; 24:+e3vZOz2d5EwuDPs/BQUhBFCCMnlDiuRaCHP4t7OEmHxC5ltx4LZ6yMzOAZqxPYmmjOkEQ2Nx4LQkhAOiGAUMz7XyN3FhyIhbFWSPdXIYy4=; 7:YBhpchCTl2SKlAelHpWsYBsK4/Ao7z69jVShRBmNSujRI6okqQa/No0ulkGJO/WhmyU2P7BQti3irO5lfOxwReCv1iwCh6OFeja9pv1JH8QpTgmBnaGCOovhVtvJqSlRPKmwcYql5IynjJP7ezlkYn07K5IcbSd85jK2AAWOFArqiEyrKrKX5JQtbtkWPDGwXEKLh7duRn8c1oOtK2uJXziZ0rgknGEPKsvg3NHw1nyQCBbUC7LQIKetTF9FSvb7 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0159; 20:QMPgGi71jphKvsUyUTkNmcSVVbCLDIp5WBqVL4ZPvYK1ppK1nGk98qpL1oUwY/leAOnvjCABEFv1qlPHZXJ/MHLgjkCIbdaux9EO/TfInO27AHgvHmvLShfCS+skpzx2UvaaCtttEVY4vKHY6jSxlIYoeL+osb1FkXxZQLWkFfUUUhj9dgN1u+bN4tMD7URRbH1IojOFb0RzWrGyK06a5WSoRXoiqWDM9JF+PTFui9A0Is+G0ydkOCWsmM+/t9Fa X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Nov 2017 18:12:30.8637 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 023c6ef5-1b61-4d0a-7f55-08d52541f2c3 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0159 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The command is used for encrypting the guest memory region using the VM encryption key (VEK) created during KVM_SEV_LAUNCH_START. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Joerg Roedel Cc: Borislav Petkov Cc: Tom Lendacky Cc: x86@kernel.org Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Improvements-by: Borislav Petkov Signed-off-by: Brijesh Singh Reviewed-by: Borislav Petkov --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm.c | 191 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 190 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index b58d9ebc2081..85fa530f9dd5 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -752,6 +752,7 @@ struct kvm_sev_info { unsigned int asid; /* ASID used for this guest */ unsigned int handle; /* SEV firmware handle */ int fd; /* SEV device fd */ + unsigned long pages_locked; /* Number of pages locked */ }; struct kvm_arch { diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index aa90a27976f6..8f2a676b5553 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -39,6 +39,8 @@ #include #include #include +#include +#include #include #include @@ -330,6 +332,7 @@ enum { static unsigned int max_sev_asid; static unsigned long *sev_asid_bitmap; +#define __sme_page_pa(x) __sme_set(page_to_pfn(x) << PAGE_SHIFT) static inline bool svm_sev_enabled(void) { @@ -1539,6 +1542,83 @@ static void sev_unbind_asid(struct kvm *kvm, unsigned int handle) kfree(decommission); } +static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, + unsigned long ulen, unsigned long *n, + int write) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + unsigned long npages, npinned, size; + unsigned long locked, lock_limit; + struct page **pages; + int first, last; + + /* Calculate number of pages. */ + first = (uaddr & PAGE_MASK) >> PAGE_SHIFT; + last = ((uaddr + ulen - 1) & PAGE_MASK) >> PAGE_SHIFT; + npages = (last - first + 1); + + locked = sev->pages_locked + npages; + lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { + pr_err("SEV: %lu locked pages exceed the lock limit of %lu.\n", locked, lock_limit); + return NULL; + } + + /* Avoid using vmalloc for smaller buffers. */ + size = npages * sizeof(struct page *); + if (size > PAGE_SIZE) + pages = vmalloc(size); + else + pages = kmalloc(size, GFP_KERNEL); + + if (!pages) + return NULL; + + /* Pin the user virtual address. */ + npinned = get_user_pages_fast(uaddr, npages, write ? FOLL_WRITE : 0, pages); + if (npinned != npages) { + pr_err("SEV: Failure locking %lu pages.\n", npages); + goto err; + } + + *n = npages; + sev->pages_locked = locked; + + return pages; + +err: + if (npinned > 0) + release_pages(pages, npinned, 0); + + kvfree(pages); + return NULL; +} + +static void sev_unpin_memory(struct kvm *kvm, struct page **pages, + unsigned long npages) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + + release_pages(pages, npages, 0); + kvfree(pages); + sev->pages_locked -= npages; +} + +static void sev_clflush_pages(struct page *pages[], unsigned long npages) +{ + uint8_t *page_virtual; + unsigned long i; + + if (npages == 0 || pages == NULL) + return; + + for (i = 0; i < npages; i++) { + page_virtual = kmap_atomic(pages[i]); + clflush_cache_range(page_virtual, PAGE_SIZE); + kunmap_atomic(page_virtual); + } +} + static void sev_vm_destroy(struct kvm *kvm) { struct kvm_sev_info *sev = &kvm->arch.sev_info; @@ -5727,7 +5807,7 @@ static int sev_bind_asid(struct kvm *kvm, unsigned int handle, int *error) return ret; } -static int sev_issue_cmd(int fd, int id, void *data, int *error) +static int __sev_issue_cmd(int fd, int id, void *data, int *error) { struct fd f; int ret; @@ -5742,6 +5822,13 @@ static int sev_issue_cmd(int fd, int id, void *data, int *error) return ret; } +static int sev_issue_cmd(struct kvm *kvm, int id, void *data, int *error) +{ + struct kvm_sev_info *sev = &kvm->arch.sev_info; + + return __sev_issue_cmd(sev->fd, id, data, error); +} + static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) { struct kvm_sev_info *sev = &kvm->arch.sev_info; @@ -5789,7 +5876,7 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) start->policy = params.policy; /* create memory encryption context */ - ret = sev_issue_cmd(argp->sev_fd, SEV_CMD_LAUNCH_START, start, error); + ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_LAUNCH_START, start, error); if (ret) goto e_free_session; @@ -5818,6 +5905,103 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) return ret; } +static int get_num_contig_pages(int idx, struct page **inpages, + unsigned long npages) +{ + unsigned long paddr, next_paddr; + int i = idx + 1, pages = 1; + + /* find the number of contiguous pages starting from idx */ + paddr = __sme_page_pa(inpages[idx]); + while (i < npages) { + next_paddr = __sme_page_pa(inpages[i++]); + if ((paddr + PAGE_SIZE) == next_paddr) { + pages++; + paddr = next_paddr; + continue; + } + break; + } + + return pages; +} + +static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) +{ + unsigned long vaddr, vaddr_end, next_vaddr, npages, size; + struct kvm_sev_info *sev = &kvm->arch.sev_info; + struct kvm_sev_launch_update_data params; + struct sev_data_launch_update_data *data; + struct page **inpages; + int i, ret, pages; + + if (!sev_guest(kvm)) + return -ENOTTY; + + if (copy_from_user(¶ms, (void __user *)(uintptr_t)argp->data, sizeof(params))) + return -EFAULT; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + vaddr = params.uaddr; + size = params.len; + vaddr_end = vaddr + size; + + /* Lock the user memory. */ + inpages = sev_pin_memory(kvm, vaddr, size, &npages, 1); + if (!inpages) { + ret = -ENOMEM; + goto e_free; + } + + /* + * The LAUNCH_UPDATE command will perform in-place encryption of the + * memory content (i.e it will write the same memory region with C=1). + * It's possible that the cache may contain the data with C=0, i.e., + * unencrypted so invalidate it first. + */ + sev_clflush_pages(inpages, npages); + + for (i = 0; vaddr < vaddr_end; vaddr = next_vaddr, i += pages) { + int offset, len; + + /* + * If the user buffer is not page-aligned, calculate the offset + * within the page. + */ + offset = vaddr & (PAGE_SIZE - 1); + + /* Calculate the number of pages that can be encrypted in one go. */ + pages = get_num_contig_pages(i, inpages, npages); + + len = min_t(size_t, ((pages * PAGE_SIZE) - offset), size); + + data->handle = sev->handle; + data->len = len; + data->address = __sme_page_pa(inpages[i]) + offset; + ret = sev_issue_cmd(kvm, SEV_CMD_LAUNCH_UPDATE_DATA, data, &argp->error); + if (ret) + goto e_unpin; + + size -= len; + next_vaddr = vaddr + len; + } + +e_unpin: + /* content of memory is updated, mark pages dirty */ + for (i = 0; i < npages; i++) { + set_page_dirty_lock(inpages[i]); + mark_page_accessed(inpages[i]); + } + /* unlock the user pages */ + sev_unpin_memory(kvm, inpages, npages); +e_free: + kfree(data); + return ret; +} + static int svm_mem_enc_op(struct kvm *kvm, void __user *argp) { struct kvm_sev_cmd sev_cmd; @@ -5838,6 +6022,9 @@ static int svm_mem_enc_op(struct kvm *kvm, void __user *argp) case KVM_SEV_LAUNCH_START: r = sev_launch_start(kvm, &sev_cmd); break; + case KVM_SEV_LAUNCH_UPDATE_DATA: + r = sev_launch_update_data(kvm, &sev_cmd); + break; default: r = -EINVAL; goto out;