From patchwork Wed Jul 10 20:23:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 11038935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1925B13B1 for ; Wed, 10 Jul 2019 20:31:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0B7D5287E3 for ; Wed, 10 Jul 2019 20:31:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F41072882D; Wed, 10 Jul 2019 20:30:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 10ED5287E3 for ; Wed, 10 Jul 2019 20:30:58 +0000 (UTC) Received: from localhost ([::1]:36888 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlJEw-0002f4-8h for patchwork-qemu-devel@patchwork.kernel.org; Wed, 10 Jul 2019 16:30:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57149) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlJ7N-0001a3-T3 for qemu-devel@nongnu.org; Wed, 10 Jul 2019 16:23:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hlJ7L-0002PY-Od for qemu-devel@nongnu.org; Wed, 10 Jul 2019 16:23:09 -0400 Received: from mail-eopbgr790080.outbound.protection.outlook.com ([40.107.79.80]:1472 helo=NAM03-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hlJ7L-0002MZ-C3 for qemu-devel@nongnu.org; Wed, 10 Jul 2019 16:23:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4ZPFbQ5tk01R05nbv01ZCuCdEDptHDOEFGh8dh0fsjc=; b=MhZZknJQU+WD7sCGFMxWpnMPzCNfybv36UO0ljFj0fXUHYg2PBbGlsd+99ThH5Kz8Ht8qrv2h0yaYkz//+IAzCl0uHCJhNqZfIKopAd6pxW9fr28YC4jp21jwrrvx1dlOC0SYtxtlSXluzmmuPMbYubnX6wVYxA27i0qN65Lkjw= Received: from DM6PR12MB2682.namprd12.prod.outlook.com (20.176.116.31) by DM6PR12MB3820.namprd12.prod.outlook.com (10.255.173.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2052.18; Wed, 10 Jul 2019 20:23:05 +0000 Received: from DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::bc1a:a30d:9da2:1cdd]) by DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::bc1a:a30d:9da2:1cdd%6]) with mapi id 15.20.2073.008; Wed, 10 Jul 2019 20:23:05 +0000 From: "Singh, Brijesh" To: "qemu-devel@nongnu.org" Thread-Topic: [PATCH v2 04/13] kvm: add support to sync the page encryption state bitmap Thread-Index: AQHVN11F+Po2QzUWe02pucGkrz+7bA== Date: Wed, 10 Jul 2019 20:23:01 +0000 Message-ID: <20190710202219.25939-5-brijesh.singh@amd.com> References: <20190710202219.25939-1-brijesh.singh@amd.com> In-Reply-To: <20190710202219.25939-1-brijesh.singh@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DM3PR11CA0020.namprd11.prod.outlook.com (2603:10b6:0:54::30) To DM6PR12MB2682.namprd12.prod.outlook.com (2603:10b6:5:4a::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [165.204.77.1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: af942760-5217-48e9-ad6c-08d7057467b8 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DM6PR12MB3820; x-ms-traffictypediagnostic: DM6PR12MB3820: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:428; x-forefront-prvs: 0094E3478A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(4636009)(366004)(346002)(39860400002)(136003)(376002)(396003)(43544003)(189003)(199004)(256004)(8936002)(2616005)(1076003)(316002)(14444005)(476003)(81166006)(478600001)(54906003)(81156014)(2351001)(8676002)(486006)(25786009)(6436002)(6486002)(6666004)(4326008)(66476007)(66946007)(30864003)(446003)(53936002)(50226002)(71190400001)(66446008)(68736007)(6506007)(14454004)(386003)(5660300002)(36756003)(66066001)(186003)(6512007)(99286004)(305945005)(102836004)(26005)(2501003)(11346002)(64756008)(52116002)(5640700003)(66556008)(76176011)(86362001)(6116002)(6916009)(3846002)(7736002)(71200400001)(2906002)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR12MB3820; H:DM6PR12MB2682.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: uTdOn14jHseACF1pEwWxE4Bx3zVIQRxz73uU/7RD+pBGeZou9zpLzVIt3L/HIHQ5bY8eu1xE4EehRCCoM1vHxscEIqbxU02Ree5mAB08bM9iz9oiVPW20ZfW40QmhQ9eKUsUOMSCqR2ZW0IkEN1Ir3pfySmXeB5zxCoiQDfo6I0rWV7h8zm4Vg9EQKcaws6Xe9hGt6RmrycU4bmTDwVoqp71oKtRAmOUKf6iQIMIRhamoqJcEObN5GKSK3zzVquj3z3cX2iNrifICOIVZig/D2oQhYEbgSCwYEiqHOFswleQELCCBUZzLLiSB5R9l693cb+pnfhhPVNjJNbF2iANqWyVA39gMy0twOQ8/8xd43BuJCw2b5bkkCoYqsUj+bw8Hm2otMLLhYr1jnBqw2rdpG9rz6UvtIa8OLqSOTg01bY= MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: af942760-5217-48e9-ad6c-08d7057467b8 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Jul 2019 20:23:01.3164 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: sbrijesh@amd.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3820 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.79.80 Subject: [Qemu-devel] [PATCH v2 04/13] kvm: add support to sync the page encryption state bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "pbonzini@redhat.com" , "Lendacky, Thomas" , "Singh, Brijesh" , "dgilbert@redhat.com" , "ehabkost@redhat.com" Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The SEV VMs have concept of private and shared memory. The private memory is encrypted with guest-specific key, while shared memory may be encrypted with hyperivosr key. The KVM_GET_PAGE_ENC_BITMAP can be used to get a bitmap indicating whether the guest page is private or shared. A private page must be transmitted using the SEV migration commands. Add a cpu_physical_memory_sync_encrypted_bitmap() which can be used to sync the page encryption bitmap for a given memory region. Signed-off-by: Brijesh Singh --- accel/kvm/kvm-all.c | 38 ++++++++++ include/exec/ram_addr.h | 161 ++++++++++++++++++++++++++++++++++++++-- include/exec/ramlist.h | 3 +- migration/ram.c | 28 ++++++- 4 files changed, 222 insertions(+), 8 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 162a2d5085..c935e9366c 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -504,6 +504,37 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section, #define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1)) +/* sync page_enc bitmap */ +static int kvm_sync_page_enc_bitmap(KVMMemoryListener *kml, + MemoryRegionSection *section, + KVMSlot *mem) +{ + unsigned long size; + KVMState *s = kvm_state; + struct kvm_page_enc_bitmap e = {}; + ram_addr_t pages = int128_get64(section->size) / getpagesize(); + ram_addr_t start = section->offset_within_region + + memory_region_get_ram_addr(section->mr); + + size = ALIGN(((mem->memory_size) >> TARGET_PAGE_BITS), + /*HOST_LONG_BITS*/ 64) / 8; + e.enc_bitmap = g_malloc0(size); + e.start_gfn = mem->start_addr >> TARGET_PAGE_BITS; + e.num_pages = pages; + if (kvm_vm_ioctl(s, KVM_GET_PAGE_ENC_BITMAP, &e) == -1) { + DPRINTF("KVM_GET_PAGE_ENC_BITMAP ioctl failed %d\n", errno); + g_free(e.enc_bitmap); + return 1; + } + + cpu_physical_memory_set_encrypted_lebitmap(e.enc_bitmap, + start, pages); + + g_free(e.enc_bitmap); + + return 0; +} + /** * kvm_physical_sync_dirty_bitmap - Grab dirty bitmap from kernel space * This function updates qemu's dirty bitmap using @@ -553,6 +584,13 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml, } kvm_get_dirty_pages_log_range(section, d.dirty_bitmap); + + if (kvm_memcrypt_enabled() && + kvm_sync_page_enc_bitmap(kml, section, mem)) { + g_free(d.dirty_bitmap); + return -1; + } + g_free(d.dirty_bitmap); } diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index f96777bb99..6fc6864194 100644 --- a/include/exec/ram_addr.h +++ b/include/exec/ram_addr.h @@ -51,6 +51,8 @@ struct RAMBlock { unsigned long *unsentmap; /* bitmap of already received pages in postcopy */ unsigned long *receivedmap; + /* bitmap of page encryption state for an encrypted guest */ + unsigned long *encbmap; }; static inline bool offset_in_ramblock(RAMBlock *b, ram_addr_t offset) @@ -314,9 +316,41 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start, } #if !defined(_WIN32) -static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap, + +static inline void cpu_physical_memory_set_encrypted_range(ram_addr_t start, + ram_addr_t length, + unsigned long val) +{ + unsigned long end, page; + unsigned long * const *src; + + if (length == 0) { + return; + } + + end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS; + page = start >> TARGET_PAGE_BITS; + + rcu_read_lock(); + + src = atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_ENCRYPTED])->blocks; + + while (page < end) { + unsigned long idx = page / DIRTY_MEMORY_BLOCK_SIZE; + unsigned long offset = page % DIRTY_MEMORY_BLOCK_SIZE; + unsigned long num = MIN(end - page, DIRTY_MEMORY_BLOCK_SIZE - offset); + + atomic_xchg(&src[idx][BIT_WORD(offset)], val); + page += num; + } + + rcu_read_unlock(); +} + +static inline void cpu_physical_memory_set_dirty_enc_lebitmap(unsigned long *bitmap, ram_addr_t start, - ram_addr_t pages) + ram_addr_t pages, + bool enc_map) { unsigned long i, j; unsigned long page_number, c; @@ -349,10 +383,14 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap, if (bitmap[k]) { unsigned long temp = leul_to_cpu(bitmap[k]); - atomic_or(&blocks[DIRTY_MEMORY_MIGRATION][idx][offset], temp); - atomic_or(&blocks[DIRTY_MEMORY_VGA][idx][offset], temp); - if (tcg_enabled()) { - atomic_or(&blocks[DIRTY_MEMORY_CODE][idx][offset], temp); + if (enc_map) { + atomic_xchg(&blocks[DIRTY_MEMORY_ENCRYPTED][idx][offset], temp); + } else { + atomic_or(&blocks[DIRTY_MEMORY_MIGRATION][idx][offset], temp); + atomic_or(&blocks[DIRTY_MEMORY_VGA][idx][offset], temp); + if (tcg_enabled()) { + atomic_or(&blocks[DIRTY_MEMORY_CODE][idx][offset], temp); + } } } @@ -372,6 +410,17 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap, * especially when most of the memory is not dirty. */ for (i = 0; i < len; i++) { + + /* If its encrypted bitmap update, then we need to copy the bitmap + * value as-is to the destination. + */ + if (enc_map) { + cpu_physical_memory_set_encrypted_range(start + i * TARGET_PAGE_SIZE, + TARGET_PAGE_SIZE * hpratio, + leul_to_cpu(bitmap[i])); + continue; + } + if (bitmap[i] != 0) { c = leul_to_cpu(bitmap[i]); do { @@ -387,6 +436,21 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap, } } } + +static inline void cpu_physical_memory_set_encrypted_lebitmap(unsigned long *bitmap, + ram_addr_t start, + ram_addr_t pages) +{ + return cpu_physical_memory_set_dirty_enc_lebitmap(bitmap, start, pages, true); +} + +static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap, + ram_addr_t start, + ram_addr_t pages) +{ + return cpu_physical_memory_set_dirty_enc_lebitmap(bitmap, start, pages, false); +} + #endif /* not _WIN32 */ bool cpu_physical_memory_test_and_clear_dirty(ram_addr_t start, @@ -406,6 +470,7 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start, cpu_physical_memory_test_and_clear_dirty(start, length, DIRTY_MEMORY_MIGRATION); cpu_physical_memory_test_and_clear_dirty(start, length, DIRTY_MEMORY_VGA); cpu_physical_memory_test_and_clear_dirty(start, length, DIRTY_MEMORY_CODE); + cpu_physical_memory_test_and_clear_dirty(start, length, DIRTY_MEMORY_ENCRYPTED); } @@ -474,5 +539,89 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb, return num_dirty; } + +static inline bool cpu_physical_memory_test_encrypted(ram_addr_t start, + ram_addr_t length) +{ + unsigned long end, page; + bool enc = false; + unsigned long * const *src; + + if (length == 0) { + return enc; + } + + end = TARGET_PAGE_ALIGN(start + length) >> TARGET_PAGE_BITS; + page = start >> TARGET_PAGE_BITS; + + rcu_read_lock(); + + src = atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_ENCRYPTED])->blocks; + + while (page < end) { + unsigned long idx = page / DIRTY_MEMORY_BLOCK_SIZE; + unsigned long offset = page % DIRTY_MEMORY_BLOCK_SIZE; + unsigned long num = MIN(end - page, DIRTY_MEMORY_BLOCK_SIZE - offset); + + enc |= atomic_read(&src[idx][BIT_WORD(offset)]); + page += num; + } + + rcu_read_unlock(); + + return enc; +} + +static inline +void cpu_physical_memory_sync_encrypted_bitmap(RAMBlock *rb, + ram_addr_t start, + ram_addr_t length) +{ + ram_addr_t addr; + unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS); + unsigned long *dest = rb->encbmap; + + /* start address and length is aligned at the start of a word? */ + if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == + (start + rb->offset) && + !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) { + int k; + int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS); + unsigned long * const *src; + unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE; + unsigned long offset = BIT_WORD((word * BITS_PER_LONG) % + DIRTY_MEMORY_BLOCK_SIZE); + unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS); + + rcu_read_lock(); + + src = atomic_rcu_read( + &ram_list.dirty_memory[DIRTY_MEMORY_ENCRYPTED])->blocks; + + for (k = page; k < page + nr; k++) { + unsigned long bits = atomic_read(&src[idx][offset]); + dest[k] = bits; + + if (++offset >= BITS_TO_LONGS(DIRTY_MEMORY_BLOCK_SIZE)) { + offset = 0; + idx++; + } + } + + rcu_read_unlock(); + } else { + ram_addr_t offset = rb->offset; + + for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) { + long k = (start + addr) >> TARGET_PAGE_BITS; + if (cpu_physical_memory_test_encrypted(start + addr + offset, + TARGET_PAGE_SIZE)) { + set_bit(k, dest); + } else { + clear_bit(k, dest); + } + } + } +} #endif #endif diff --git a/include/exec/ramlist.h b/include/exec/ramlist.h index bc4faa1b00..2a5eab8b11 100644 --- a/include/exec/ramlist.h +++ b/include/exec/ramlist.h @@ -11,7 +11,8 @@ typedef struct RAMBlockNotifier RAMBlockNotifier; #define DIRTY_MEMORY_VGA 0 #define DIRTY_MEMORY_CODE 1 #define DIRTY_MEMORY_MIGRATION 2 -#define DIRTY_MEMORY_NUM 3 /* num of dirty bits */ +#define DIRTY_MEMORY_ENCRYPTED 3 +#define DIRTY_MEMORY_NUM 4 /* num of dirty bits */ /* The dirty memory bitmap is split into fixed-size blocks to allow growth * under RCU. The bitmap for a block can be accessed as follows: diff --git a/migration/ram.c b/migration/ram.c index 3c8977d508..d179867e1b 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -1680,6 +1680,9 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb, rs->migration_dirty_pages += cpu_physical_memory_sync_dirty_bitmap(rb, 0, length, &rs->num_dirty_pages_period); + if (kvm_memcrypt_enabled()) { + cpu_physical_memory_sync_encrypted_bitmap(rb, 0, length); + } } /** @@ -2465,6 +2468,22 @@ static bool save_compress_page(RAMState *rs, RAMBlock *block, ram_addr_t offset) return false; } +/** + * encrypted_test_bitmap: check if the page is encrypted + * + * Returns a bool indicating whether the page is encrypted. + */ +static bool encrypted_test_bitmap(RAMState *rs, RAMBlock *block, + unsigned long page) +{ + /* ROM devices contains the unencrypted data */ + if (memory_region_is_rom(block->mr)) { + return false; + } + + return test_bit(page, block->encbmap); +} + /** * ram_save_target_page: save one target page * @@ -2491,7 +2510,8 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, * will take care of accessing the guest memory and re-encrypt it * for the transport purposes. */ - if (kvm_memcrypt_enabled()) { + if (kvm_memcrypt_enabled() && + encrypted_test_bitmap(rs, pss->block, pss->page)) { return ram_save_encrypted_page(rs, pss, last_stage); } @@ -2724,6 +2744,8 @@ static void ram_save_cleanup(void *opaque) block->bmap = NULL; g_free(block->unsentmap); block->unsentmap = NULL; + g_free(block->encbmap); + block->encbmap = NULL; } xbzrle_cleanup(); @@ -3251,6 +3273,10 @@ static void ram_list_init_bitmaps(void) block->unsentmap = bitmap_new(pages); bitmap_set(block->unsentmap, 0, pages); } + if (kvm_memcrypt_enabled()) { + block->encbmap = bitmap_new(pages); + bitmap_set(block->encbmap, 0, pages); + } } } }