From patchwork Thu Jun 20 18:03:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 11007743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D04D96C5 for ; Thu, 20 Jun 2019 18:52:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C39C5286A2 for ; Thu, 20 Jun 2019 18:52:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B7D6A28846; Thu, 20 Jun 2019 18:52:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A06F1286A2 for ; Thu, 20 Jun 2019 18:52:25 +0000 (UTC) Received: from localhost ([::1]:52158 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1he2Aa-0005xi-SC for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Jun 2019 14:52:24 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43151) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1he1cZ-0005Dx-DD for qemu-devel@nongnu.org; Thu, 20 Jun 2019 14:17:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1he1PB-0003vs-HI for qemu-devel@nongnu.org; Thu, 20 Jun 2019 14:03:47 -0400 Received: from mail-eopbgr790073.outbound.protection.outlook.com ([40.107.79.73]:37285 helo=NAM03-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1he1PB-0003mO-9z for qemu-devel@nongnu.org; Thu, 20 Jun 2019 14:03:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=78EaxjDNkcvFzn+iO+Nuva/+qUD2IPxa3CAOogqXDKU=; b=wk6gJf/lKVbEp1cD19u1MROmbPBacE6rf8QlAIpsop0WDRCwMmqAa/5e0A4hXvhYYMi0evqDnGlTc+qSKed+rVf30a+/6Y12l74m8/rrGQZEdSgH2qN+7ps5bW39eeG/2A4/tvpOwhTRFAo446OBbZgn5YoexH5hI9HtIKHDrlM= Received: from DM6PR12MB2682.namprd12.prod.outlook.com (20.176.116.31) by DM6PR12MB2842.namprd12.prod.outlook.com (20.176.116.80) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1987.13; Thu, 20 Jun 2019 18:03:10 +0000 Received: from DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::b9c1:b235:fff3:dba2]) by DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::b9c1:b235:fff3:dba2%6]) with mapi id 15.20.1987.014; Thu, 20 Jun 2019 18:03:10 +0000 From: "Singh, Brijesh" To: "qemu-devel@nongnu.org" Thread-Topic: [RFC PATCH v1 02/12] kvm: introduce high-level API to support encrypted guest migration Thread-Index: AQHVJ5Jp0L4i7VOWuEiaJ/44K3kyUA== Date: Thu, 20 Jun 2019 18:03:06 +0000 Message-ID: <20190620180247.8825-3-brijesh.singh@amd.com> References: <20190620180247.8825-1-brijesh.singh@amd.com> In-Reply-To: <20190620180247.8825-1-brijesh.singh@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DM5PR16CA0025.namprd16.prod.outlook.com (2603:10b6:4:15::11) To DM6PR12MB2682.namprd12.prod.outlook.com (2603:10b6:5:4a::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [165.204.77.1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ca20a3f9-d0c2-4b76-8a92-08d6f5a98c04 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DM6PR12MB2842; x-ms-traffictypediagnostic: DM6PR12MB2842: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1060; x-forefront-prvs: 0074BBE012 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(376002)(396003)(366004)(346002)(39860400002)(189003)(199004)(81166006)(81156014)(386003)(86362001)(256004)(486006)(476003)(50226002)(186003)(2616005)(4326008)(8936002)(25786009)(68736007)(14454004)(5660300002)(26005)(11346002)(66066001)(6666004)(64756008)(5640700003)(73956011)(3846002)(76176011)(66556008)(6512007)(66476007)(53936002)(66946007)(2351001)(446003)(6486002)(66446008)(478600001)(99286004)(71200400001)(305945005)(14444005)(7736002)(6506007)(102836004)(71190400001)(52116002)(6436002)(8676002)(36756003)(54906003)(316002)(2501003)(6916009)(2906002)(1076003)(6116002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR12MB2842; H:DM6PR12MB2682.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: GGS19TzrnYgbEMqhslEFHa/91qVCxKzmmdmmd73uRWMrdxEwaAlkS1dUc9harSMfRp90IPcrA+ERhyVt7NlhDeQKeXsdX7+J/bW19yj/9Jd2pgpKWeZnH/OmvvlKGm3VtPjgGWIU1WTRvejTvm7cZxuBfy4CNReIItA0c3jJCS4oCfArWguFXOCT+SvNAQyrNyFT3ncTL02LJpBYlVMZhygTgGbSh0Hkpu9eSix264Fej8yAsC8aQFrGVVCWFXX1V/chP3/Hn8KU+LkxjDnNq60C36POoZ5Y3JDhWzDxBN6GREeKi1FJ7p1fM3m5CKOe2mlF7CHJxn/1arSEEhK7laNsOHvEfBqbQmlEo4J4n/424dJLMVckUHXRERRuJ3QNAI+lq6Fzm++lzWTAy8/w5rFfexZbB4BLTfLBpP9g4Kc= MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ca20a3f9-d0c2-4b76-8a92-08d6f5a98c04 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jun 2019 18:03:06.9348 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: sbrijesh@amd.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2842 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.79.73 Subject: [Qemu-devel] [RFC PATCH v1 02/12] kvm: introduce high-level API to support encrypted guest migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Lendacky, Thomas" , "Singh, Brijesh" , "kvm@vger.kernel.org" Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP When memory encryption is enabled in VM, the guest pages will be encrypted with the guest-specific key, to protect the confidentiality of data in transit. To support the live migration we need to use platform specific hooks to access the guest memory. The kvm_memcrypt_save_outgoing_page() can be used by the sender to write the encrypted pages and metadata associated with it on the socket. The kvm_memcrypt_load_incoming_page() can be used by receiver to read the incoming encrypted pages from the socket and load into the guest memory. Encrypted VMs have concept of private and shared memory. The private memory is encrypted with the guest-specific key, while shared memory may be encrypted with hyperivosr key. The KVM_{SET,GET}_PAGE_ENC_BITMAP ioctl can be used to get/set the bitmap from/to the hypervisor. The kvm_memcrypt_sync_page_enc_bitmap() can be used by the sender to get the page encryption bitmap. The bitmap is used to determine the page state (private or shared). The kvm_memcrypt_send_outgoing_page_enc_bitmap() can be used by the sender to write the page encryption bitmap on the socket. The kvm_memcrypt_load_incoming_page_enc_bitmap() can be used by the receiver to read the page encryption bitmap from the socket. Signed-off-by: Brijesh Singh <> --- accel/kvm/kvm-all.c | 68 ++++++++++++++++++++++++++++++++++++++++++ accel/kvm/sev-stub.c | 28 +++++++++++++++++ accel/stubs/kvm-stub.c | 30 +++++++++++++++++++ include/sysemu/kvm.h | 33 ++++++++++++++++++++ include/sysemu/sev.h | 9 ++++++ 5 files changed, 168 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index b0c4bed6e3..4d5ff8b9f5 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -109,6 +109,15 @@ struct KVMState /* memory encryption */ void *memcrypt_handle; int (*memcrypt_encrypt_data)(void *handle, uint8_t *ptr, uint64_t len); + int (*memcrypt_save_outgoing_page)(void *ehandle, QEMUFile *f, + uint8_t *ptr, uint32_t sz, uint64_t *bytes_sent); + int (*memcrypt_load_incoming_page)(void *ehandle, QEMUFile *f, + uint8_t *ptr); + int (*memcrypt_load_incoming_page_enc_bitmap)(void *ehandle, QEMUFile *f); + int (*memcrypt_save_outgoing_page_enc_bitmap)(void *ehandle, QEMUFile *f, + uint8_t *host, uint64_t length, unsigned long *bmap); + int (*memcrypt_sync_page_enc_bitmap)(void *ehandle, uint8_t *host, + uint64_t length, unsigned long *bmap); }; KVMState *kvm_state; @@ -164,6 +173,65 @@ int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len) return 1; } +int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t *ptr, + uint32_t size, uint64_t *bytes_sent) +{ + if (kvm_state->memcrypt_handle && + kvm_state->memcrypt_save_outgoing_page) { + return kvm_state->memcrypt_save_outgoing_page(kvm_state->memcrypt_handle, + f, ptr, size, bytes_sent); + } + + return 1; +} + +int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr) +{ + if (kvm_state->memcrypt_handle && + kvm_state->memcrypt_load_incoming_page) { + return kvm_state->memcrypt_load_incoming_page(kvm_state->memcrypt_handle, + f, ptr); + } + + return 1; +} + +int kvm_memcrypt_load_incoming_page_enc_bitmap(QEMUFile *f) +{ + if (kvm_state->memcrypt_handle && + kvm_state->memcrypt_load_incoming_page_enc_bitmap) { + return kvm_state->memcrypt_load_incoming_page_enc_bitmap( + kvm_state->memcrypt_handle, f); + } + + return 1; +} + +int kvm_memcrypt_save_outgoing_page_enc_bitmap(QEMUFile *f, uint8_t *host, + uint64_t length, + unsigned long *bmap) +{ + if (kvm_state->memcrypt_handle && + kvm_state->memcrypt_save_outgoing_page_enc_bitmap) { + return kvm_state->memcrypt_save_outgoing_page_enc_bitmap( + kvm_state->memcrypt_handle, f, host, length, bmap); + } + + return 1; +} + +int kvm_memcrypt_sync_page_enc_bitmap(uint8_t *host, uint64_t length, + unsigned long *bmap) +{ + if (kvm_state->memcrypt_handle && + kvm_state->memcrypt_sync_page_enc_bitmap) { + return kvm_state->memcrypt_sync_page_enc_bitmap( + kvm_state->memcrypt_handle, host, length, bmap); + } + + return 1; +} + static KVMSlot *kvm_get_free_slot(KVMMemoryListener *kml) { KVMState *s = kvm_state; diff --git a/accel/kvm/sev-stub.c b/accel/kvm/sev-stub.c index 4f97452585..5d8c3f2ecd 100644 --- a/accel/kvm/sev-stub.c +++ b/accel/kvm/sev-stub.c @@ -24,3 +24,31 @@ void *sev_guest_init(const char *id) { return NULL; } + +int sev_save_outgoing_page(void *handle, QEMUFile *f, uint8_t *ptr, + uint32_t size, uint64_t *bytes_sent) +{ + return 1; +} + +int sev_load_incoming_page(void *handle, QEMUFile *f, uint8_t *ptr) +{ + return 1; +} + +int sev_load_incoming_page_enc_bitmap(void *handle, QEMUFile *f) +{ + return 1; +} + +int sev_save_outgoing_page_enc_bitmap(void *handle, QEMUFile *f, + unsigned long *bmap) +{ + return 1; +} + +int sev_sync_page_enc_bitmap(void *handle, uint8_t *host, uint64_t size, + unsigned long *bitmap) +{ + return 1; +} diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c index 6feb66ed80..bef7376985 100644 --- a/accel/stubs/kvm-stub.c +++ b/accel/stubs/kvm-stub.c @@ -114,6 +114,36 @@ int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len) return 1; } +int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t *ptr, + uint32_t size, uint64_t *bytes_sent) +{ + return 1; +} + +int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr) +{ + return 1; +} + +int kvm_memcrypt_load_incoming_page_enc_bitmap(QEMUFile *f) +{ + return 1; +} + +int kvm_memcrypt_save_outgoing_page_enc_bitmap(QEMUFile *f, uint8_t *host, + uint64_t length, + unsigned long *bmap) +{ + return 1; +} + +int kvm_memcrypt_sync_page_enc_bitmap(uint8_t *host, uint64_t size, + unsigned long *bitmap) +{ + return 1; +} + + #ifndef CONFIG_USER_ONLY int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev) { diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index a6d1cd190f..f85a60e411 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -246,6 +246,39 @@ bool kvm_memcrypt_enabled(void); */ int kvm_memcrypt_encrypt_data(uint8_t *ptr, uint64_t len); +/** + * kvm_memcrypt_save_outgoing_buffer - encrypt the outgoing buffer + * and write to the wire. + */ +int kvm_memcrypt_save_outgoing_page(QEMUFile *f, uint8_t *ptr, uint32_t size, + uint64_t *bytes_sent); + +/** + * kvm_memcrypt_load_incoming_buffer - read the encrypt incoming buffer and copy + * the buffer into the guest memory space. + */ +int kvm_memcrypt_load_incoming_page(QEMUFile *f, uint8_t *ptr); + +/** + * kvm_memcrypt_load_incoming_page_enc_bitmap: read the page encryption bitmap + * from the socket and pass it to the hypervisor. + */ +int kvm_memcrypt_load_incoming_page_enc_bitmap(QEMUFile *f); + +/** + * kvm_memcrypt_sync_page_enc_bitmap: sync the page encryption bitmap + * The caller is responsible to allocate/free the bitmap. + */ +int kvm_memcrypt_sync_page_enc_bitmap(uint8_t *host, uint64_t size, + unsigned long *bitmap); + +/** + * kvm_memcrypt_save_outgoing_page_enc_bitmap: write the page encryption bitmap + * on socket. + */ +int kvm_memcrypt_save_outgoing_page_enc_bitmap(QEMUFile *f, uint8_t *host, + uint64_t length, + unsigned long *bmap); #ifdef NEED_CPU_H #include "cpu.h" diff --git a/include/sysemu/sev.h b/include/sysemu/sev.h index 98c1ec8d38..009be45230 100644 --- a/include/sysemu/sev.h +++ b/include/sysemu/sev.h @@ -18,4 +18,13 @@ void *sev_guest_init(const char *id); int sev_encrypt_data(void *handle, uint8_t *ptr, uint64_t len); +int sev_save_outgoing_page(void *handle, QEMUFile *f, uint8_t *ptr, + uint32_t size, uint64_t *bytes_sent); +int sev_load_incoming_page(void *handle, QEMUFile *f, uint8_t *ptr); +int sev_load_incoming_page_enc_bitmap(void *handle, QEMUFile *f); +int sev_save_outgoing_page_enc_bitmap(void *handle, QEMUFile *f, + uint8_t *host, uint64_t length, + unsigned long *bmap); +int sev_sync_page_enc_bitmap(void *handle, uint8_t *host, uint64_t size, + unsigned long *bitmap); #endif