From patchwork Tue Jul 9 13:20:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Roy X-Patchwork-Id: 13727951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C0DAC3DA41 for ; Tue, 9 Jul 2024 13:21:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33DD66B00AB; Tue, 9 Jul 2024 09:21:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C5B56B00AD; Tue, 9 Jul 2024 09:21:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13F796B00AE; Tue, 9 Jul 2024 09:21:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DBF946B00AB for ; Tue, 9 Jul 2024 09:21:11 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8AC11A1DD2 for ; Tue, 9 Jul 2024 13:21:11 +0000 (UTC) X-FDA: 82320275142.30.E1784BD Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by imf14.hostedemail.com (Postfix) with ESMTP id 7C9D010000F for ; Tue, 9 Jul 2024 13:21:09 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=amazon.co.uk header.s=amazon201209 header.b=abv3Rx2i; spf=pass (imf14.hostedemail.com: domain of "prvs=913fd7204=roypat@amazon.co.uk" designates 52.95.49.90 as permitted sender) smtp.mailfrom="prvs=913fd7204=roypat@amazon.co.uk"; dmarc=pass (policy=quarantine) header.from=amazon.co.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720531238; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=JvsKDQtPEpQi+6f5PZkphVAQipCdfdQA5rHBjtTGne4=; b=gk3QoZupLgpIR4azicbTLdxoFtJrXR6+kI3P6NwxjEA0saD98pZafWRWIEpbhfVAQUs3AN qkZ/CJO9WJV/jLuPFKdbHOmxJD1vR9Fs9lDZXYezAQxumwJXgbzYvRLAA12ZLyFsW8t7Fm i82GlDlEBwo+L3eRxsc9v1sIwYheMb4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720531238; a=rsa-sha256; cv=none; b=A0e1KYzCRYO5+WB2aRioMppE5oaPJO6XVzUnkTQsJiIW244A1X29olObWeSCKsvXxpKjDB XdOa2Q82kqoEDUS5ipLn8q21G2rflTG+GfbTPBDrS4YJ5FCEOW/wiY+NGkmwHXEhDmiy8T vD6TdO+I9jkYP58Iq53M6taDyvSbmjk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=amazon.co.uk header.s=amazon201209 header.b=abv3Rx2i; spf=pass (imf14.hostedemail.com: domain of "prvs=913fd7204=roypat@amazon.co.uk" designates 52.95.49.90 as permitted sender) smtp.mailfrom="prvs=913fd7204=roypat@amazon.co.uk"; dmarc=pass (policy=quarantine) header.from=amazon.co.uk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.co.uk; i=@amazon.co.uk; q=dns/txt; s=amazon201209; t=1720531270; x=1752067270; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=JvsKDQtPEpQi+6f5PZkphVAQipCdfdQA5rHBjtTGne4=; b=abv3Rx2iG4gaaV9m/1OT0bu5L0Lka6YI06ZkHgN6T/uejpmsBU7YhBQX VUIgb9erWtnDcVloEJMs/NFiMdhI6seMbOFq+Oe1oaJ+jAtKk6PESB8kL JaW9Phyn3edMeTFPa2GcUMSNXUHGP4nHeKu7fiUpnls1l8w1CodOnJpkr o=; X-IronPort-AV: E=Sophos;i="6.09,195,1716249600"; d="scan'208";a="418820814" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jul 2024 13:20:55 +0000 Received: from EX19MTAUEB002.ant.amazon.com [10.0.44.209:19356] by smtpin.naws.us-east-1.prod.farcaster.email.amazon.dev [10.0.40.149:2525] with esmtp (Farcaster) id 26b0b38b-82b5-4c70-a8ce-d2f8b26566eb; Tue, 9 Jul 2024 13:20:54 +0000 (UTC) X-Farcaster-Flow-ID: 26b0b38b-82b5-4c70-a8ce-d2f8b26566eb Received: from EX19D008UEA002.ant.amazon.com (10.252.134.125) by EX19MTAUEB002.ant.amazon.com (10.252.135.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Tue, 9 Jul 2024 13:20:53 +0000 Received: from EX19MTAUEC001.ant.amazon.com (10.252.135.222) by EX19D008UEA002.ant.amazon.com (10.252.134.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Tue, 9 Jul 2024 13:20:53 +0000 Received: from ua2d7e1a6107c5b.ant.amazon.com (172.19.88.180) by mail-relay.amazon.com (10.252.135.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Tue, 9 Jul 2024 13:20:50 +0000 From: Patrick Roy To: , , , , , CC: Patrick Roy , , , , , , , , , , , , , , , , , Subject: [RFC PATCH 0/8] Unmapping guest_memfd from Direct Map Date: Tue, 9 Jul 2024 14:20:28 +0100 Message-ID: <20240709132041.3625501-1-roypat@amazon.co.uk> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 X-Stat-Signature: ehhu9zuymiet7q1u5cc357ctz3ayagjz X-Rspamd-Queue-Id: 7C9D010000F X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1720531269-649919 X-HE-Meta: U2FsdGVkX18ex9LebVUMNRapWApSvzm9d5Skp0sk3/A6S+oMGOH9+xL6WKAtXL1M0neScQ4x5rNJCR9BGHCqfBBIhez7Ua/dvH09vWxwR9U0MHiSeVqaMxaFB3ro6d+b2BYN7EM4rYQaq9oJq5OhihRSxcfhD2uFSjCAauV8CiiLLdV2nVxy3p7Dyy4r/wfsS0uYNVQmG97X4BDWioy0RPuir5vU6WikCyzj+1+WkUAdkBFPSXC8z1kZhi04b42oWVfA5PaxC5fhUQSlvxqesP3V+n69vP02uywPCv4EjzG3kex8848LVC+zTAHdtHoNxFFp0jZHUfS3BeeDLjDz2GH9WezbpphBNPluSX2kbv0q3PmbMyY2dILOHxMDgLTgKH67iw7ztVHDRe6veTXRvTAQz5hPmdFFOlzndxQbVgKm5SN4JQI93IYIZXavl/+f+uTt+j/fJw96+IbyQgtnESkoXuHai+vK6FczrjkpqV4ETtO1kM4iaICbEdr/t6QdIjxWGHdctfr/xYG1W6+ldWzb85Mo2z+oW7UINayJ2Sa6qnbTvNWfeP/HRr3Ld+ri3oCuCsCq6PG+E+WKMtep5x5orNbrbZ9cSuV2bzhCaQ2P3N9wAD7Y7yxYJG+SNimmV0QFBkhIPLJeqjaYZdIKR5R4wvk148jekDXAahZ9EYc2M50qiz5d3VzsLZdo/gVWbLYg+YSWnKxxVU/otukeHjVuXoy4qpazTqAqthe/BA0T4fqTm4e/rQtBQ9+DsqjvcFfnDwyPUxW4j0ymCw3q3QRt0c6hcTBYDXdXSCExkkIe9ETRayYyyNFW2qN5zO22ysJsojDX+Bf/U+RrYXJovTbrqIHc2Vqd9halswuQYAzrBYc6U/kSBMhvQ40IoN4GCSquxLrYkcdj16jOFStTwFJe1N9PHP1oa6L4y9W2I36M9SvaqudEIUj6zQlsS5TSMBjckos3v3+bZYNOL5l WohsFGTO T5A9x3M1FMsQTDY72o0pHvOQaHGS12/RpCXB58qdp8phabee70TA9wxGKj5lTBqU5Rz4XRjDW6E0SIwC9aiW9gMmxF3k5qLVZSn+YXIDihlZgZqDTiUnC6zVQ8USjLC+hFm3TcAeSHW8/2ihsWqGf2vdCFsgfuL4VeBeh53KOWPqkdY3gweQx3NCW4yiSi/qfPS5gyZY9sgDEYvg0V6lNPLh5tA5du1+NYLPU9BQnTOqrsofGEJHObN886E8uexA4ZXQkRVq2HfuslrZgUT2CcfpoVAO0OrZEnYky95qqDcaFX7tQWES2v1xQabAxptqM43Fy/QdlfLuXzlGCzzD6zAXbgmujhiYZFgQPtR6D99oBEQd/S6kRKp6eNcGM+joYVxjlQohOFyEhMHr8pGURQ1pEtypGiBHnHQRG4+W/T021otimCC2HHklj8tNGHo7zMGGwA1+VIjk8iPJeREZd5C9awMLoSrEMhJi/IkSP/k1jOlY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hey all, This RFC series is a rough draft adding support for running non-confidential compute VMs in guest_memfd, based on prior discussions with Sean [1]. Our specific usecase for this is the ability to unmap guest memory from the host kernel's direct map, as a mitigation against a large class of speculative execution issues. === Implementation === This patch series introduces a new flag to the `KVM_CREATE_GUEST_MEMFD` to remove its pages from the direct map when they are allocated. When trying to run a guest from such a VM, we now face the problem that without either userspace or kernelspace mappings of guest_memfd, KVM cannot access guest memory to, for example, do MMIO emulation of access memory used to guest/host communication. We have multiple options for solving this when running non-CoCo VMs: (1) implement a TDX-light solution, where the guest shares memory that KVM needs to access, and relies on paravirtual solutions where this is not possible (e.g. MMIO), (2) have KVM use userspace mappings of guest_memfd (e.g. a memfd_secret-style solution), or (3) dynamically reinsert pages into the direct map whenever KVM wants to access them. This RFC goes for option (3). Option (1) is a lot of overhead for very little gain, since we are not actually constrained by a physical inability to access guest memory (e.g. we are not in a TDX context where accesses to guest memory cause a #MC). Option (2) has previously been rejected [1]. In this patch series, we make sufficient parts of KVM gmem-aware to be able to boot a Linux initrd from private memory on x86. These include KVM's MMIO emulation (including guest page table walking) and kvm-clock. For VM types which do not allow accessing gmem, we return -EFAULT and attempt to prepare a KVM_EXIT_MEMORY_FAULT. Additionally, this patch series adds support for "restricted" userspace mappings of guest_memfd, which work similar to memfd_secret (e.g. disallow get_user_pages), which allows handling I/O and loading the guest kernel in a simple way. Support for this is completely independent of the rest of the functionality introduced in this patch series. However, it is required to build a minimal hypervisor PoC that actually allows booting a VM from a disk. === Performance === We have run some preliminary performance benchmarks to assess the impact of on-the-fly direct map manipulations. We were mainly interested in the impact of manipulating the direct map for MMIO emulation on virtio-mmio. Particularly, we were worried about the impact of the TLB and L1/2/3 Cache flushes that set_memory_[n]p entails. In our setup, we have taken a modified Firecracker VMM, spawned a Linux guest with 1 vCPU, and used fio to stress a virtio_blk device. We found that the cache flushes caused throughput to drop from around 600MB/s to ~50MB/s (~90%) for both reads and writes (on a Intel(R) Xeon(R) Platinum 8375C CPU with 64 cores). We then converted our prototype to use set_direct_map_{invalid,default}_noflush instead of set_memory_[n]p and found that without cache flushes the pure impact of the direct map manipulation is indistinguishable from noise. This is why we use set_direct_map_{invalid,default}_noflush instead of set_memory_[n]p in this RFC. Note that in this comparison, both the baseline, as well as the guest_memfd-supporting version of Firecracker were made to bounce I/O buffers in VMM userspace. As GUP is disabled for the guest_memfd VMAs, the virtio stack cannot directly pass guest buffers to read/write syscalls. === Security === We want to use unmapping guest memory from the host kernel as a security mitigation against transient execution attacks. Temporarily restoring direct map entries whenever KVM requires access to guest memory leaves a gap in this mitigation. We believe this to be acceptable for the above cases, since pages used for paravirtual guest/host communication (e.g. kvm-clock) and guest page tables do not contain sensitive data. MMIO emulation will only end up reading pages containing privileged instructions (e.g. guest kernel code). === Summary === Patches 1-4 are about hot-patching various points inside of KVM that access guest memory to correctly handle the case where memory happens to be guest-private. This means either handling the access as a memory error, or simply accessing the memslot's guest_memfd instead of looking at the userspace provided VMA if the VM type allows these kind of accesses. Patches 5-6 add a flag to KVM_CREATE_GUEST_MEMFD that will make it remove its pages from the kernel's direct map. Whenever KVM wants to access guest-private memory, it will temporarily re-insert the relevant pages. Patches 7-8 allow for restricted userspace mappings (e.g. get_user_pages paths are disabled like for memfd_secret) of guest_memfd, so that userspace has an easy path for loading the guest kernel and handling I/O-buffers. === ToDos / Limitations === There are still a few rough edges that need to be addressed before dropping the "RFC" tag, e.g. * Handle errors of set_direct_map_default_not_flush in kvm_gmem_invalidate_folio instead of calling BUG_ON * Lift the limitation of "at most one gfn_to_pfn_cache for each gfn/pfn" in e1c61f0a7963 ("kvm: gmem: Temporarily restore direct map entries when needed"). It currently means that guests with more than 1 vcpu fail to boot, because multiple vcpus can put their kvm-clock PV structures into the same page (gfn) * Write selftests, particularly around hole punching, direct map removal, and mmap. Lastly, there's the question of nested virtualization which Sean brought up in previous discussions, which runs into similar problems as MMIO. I have looked at it very briefly. On Intel, KVM uses various gfn->uhva caches, which run in similar problems as the gfn_to_hva_caches dealt with in 200834b15dda ("kvm: use slowpath in gfn_to_hva_cache if memory is private"). However, previous attempts at just converting this to gfn_to_pfn_cache (which would make them work with guest_memfd) proved complicated [2]. I suppose initially, we should probably disallow nested virtualization in VMs that have their memory removed from the direct map. Best, Patrick [1]: https://lore.kernel.org/linux-mm/cc1bb8e9bc3e1ab637700a4d3defeec95b55060a.camel@amazon.com/ [2]: https://lore.kernel.org/kvm/ZBEEQtmtNPaEqU1i@google.com/ Patrick Roy (8): kvm: Allow reading/writing gmem using kvm_{read,write}_guest kvm: use slowpath in gfn_to_hva_cache if memory is private kvm: pfncache: enlighten about gmem kvm: x86: support walking guest page tables in gmem kvm: gmem: add option to remove guest private memory from direct map kvm: gmem: Temporarily restore direct map entries when needed mm: secretmem: use AS_INACCESSIBLE to prohibit GUP kvm: gmem: Allow restricted userspace mappings arch/x86/kvm/mmu/paging_tmpl.h | 94 +++++++++++++++++++----- include/linux/kvm_host.h | 5 ++ include/linux/kvm_types.h | 1 + include/linux/secretmem.h | 13 +++- include/uapi/linux/kvm.h | 2 + mm/secretmem.c | 6 +- virt/kvm/guest_memfd.c | 83 +++++++++++++++++++-- virt/kvm/kvm_main.c | 112 +++++++++++++++++++++++++++- virt/kvm/pfncache.c | 130 +++++++++++++++++++++++++++++---- 9 files changed, 399 insertions(+), 47 deletions(-) base-commit: 890a64810d59b1a58ed26efc28cfd821fc068e84