| Message ID | 20250226082549.6034-1-shivankg@amd.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add NUMA mempolicy support for KVM guest-memfd | expand |
On Wed, Feb 26, 2025 at 12:28 AM Shivank Garg <shivankg@amd.com> wrote: > > In this patch-series: > Based on the discussion in the bi-weekly guest_memfd upstream call on > 2025-02-20[4], I have dropped the RFC tag, documented the memory allocation > behavior after policy changes and added selftests. > > > KVM's guest-memfd memory backend currently lacks support for NUMA policy > enforcement, causing guest memory allocations to be distributed arbitrarily > across host NUMA nodes regardless of the policy specified by the VMM. This > occurs because conventional userspace NUMA control mechanisms like mbind() > are ineffective with guest-memfd, as the memory isn't directly mapped to > userspace when allocations occur. > > This patch-series adds NUMA binding capabilities to guest_memfd backend > KVM guests. It has evolved through several approaches based on community > feedback: > > - v1,v2: Extended the KVM_CREATE_GUEST_MEMFD IOCTL to pass mempolicy. > - v3: Introduced fbind() syscall for VMM memory-placement configuration. > - v4-v6: Current approach using shared_policy support and vm_ops (based on > suggestions from David[1] and guest_memfd biweekly upstream call[2]). > > For SEV-SNP guests, which use the guest-memfd memory backend, NUMA-aware > memory placement is essential for optimal performance, particularly for > memory-intensive workloads. > > This series implements proper NUMA policy support for guest-memfd by: > > 1. Adding mempolicy-aware allocation APIs to the filemap layer. I have been thinking more about this after the last guest_memfd upstream call on March 6th. To allow 1G page support with guest_memfd [1] without encountering significant memory overheads, its important to support in-place memory conversion with private hugepages getting split/merged upon conversion. Private pages can be seamlessly split/merged only if the refcounts of complete subpages are frozen, most effective way to achieve and enforce this is to just not have struct pages for private memory. All the guest_memfd private range users (including IOMMU [2] in future) can request pfns for offsets and get notified about invalidation when pfns go away. Not having struct pages for private memory also provide additional benefits: * Significantly lesser memory overhead for handling splitting/merge operations - With struct pages around, every split of 1G page needs struct page allocation for 512 * 512 4K pages in worst case. * Enable roadmap for PFN range allocators in the backend and usecases like KHO [3] that target use of memory without struct page. IIRC, filemap was initially used as a matter of convenience for initial guest memfd implementation. As pointed by David in the call, to get rid of struct page for private memory ranges, filemap/pagecache needs to be replaced by a lightweight mechanism that tracks offsets -> pfns mapping for private memory ranges while still keeping filemap/pagecache for shared memory ranges (it's still needed to allow GUP usecases). I am starting to think that the filemap replacement for private memory ranges should be done sooner rather than later, otherwise it will become more and more difficult with features landing in guest_memfd relying on presence of filemap. This discussion matters more for hugepages and PFN range allocations. I would like to ensure that we have consensus on this direction. [1] https://lpc.events/event/18/contributions/1764/ [2] https://lore.kernel.org/kvm/CAGtprH8C4MQwVTFPBMbFWyW4BrK8-mDqjJn-UUFbFhw4w23f3A@mail.gmail.com/ [3] https://lore.kernel.org/linux-mm/20240805093245.889357-1-jgowans@amazon.com/
On Sat, Mar 8, 2025 at 5:09 PM Vishal Annapurve <vannapurve@google.com> wrote: > > On Wed, Feb 26, 2025 at 12:28 AM Shivank Garg <shivankg@amd.com> wrote: > > > > In this patch-series: > > Based on the discussion in the bi-weekly guest_memfd upstream call on > > 2025-02-20[4], I have dropped the RFC tag, documented the memory allocation > > behavior after policy changes and added selftests. > > > > > > KVM's guest-memfd memory backend currently lacks support for NUMA policy > > enforcement, causing guest memory allocations to be distributed arbitrarily > > across host NUMA nodes regardless of the policy specified by the VMM. This > > occurs because conventional userspace NUMA control mechanisms like mbind() > > are ineffective with guest-memfd, as the memory isn't directly mapped to > > userspace when allocations occur. > > > > This patch-series adds NUMA binding capabilities to guest_memfd backend > > KVM guests. It has evolved through several approaches based on community > > feedback: > > > > - v1,v2: Extended the KVM_CREATE_GUEST_MEMFD IOCTL to pass mempolicy. > > - v3: Introduced fbind() syscall for VMM memory-placement configuration. > > - v4-v6: Current approach using shared_policy support and vm_ops (based on > > suggestions from David[1] and guest_memfd biweekly upstream call[2]). > > > > For SEV-SNP guests, which use the guest-memfd memory backend, NUMA-aware > > memory placement is essential for optimal performance, particularly for > > memory-intensive workloads. > > > > This series implements proper NUMA policy support for guest-memfd by: > > > > 1. Adding mempolicy-aware allocation APIs to the filemap layer. > > I have been thinking more about this after the last guest_memfd > upstream call on March 6th. > > To allow 1G page support with guest_memfd [1] without encountering > significant memory overheads, its important to support in-place memory > conversion with private hugepages getting split/merged upon > conversion. Private pages can be seamlessly split/merged only if the > refcounts of complete subpages are frozen, most effective way to > achieve and enforce this is to just not have struct pages for private > memory. All the guest_memfd private range users (including IOMMU [2] > in future) can request pfns for offsets and get notified about > invalidation when pfns go away. > > Not having struct pages for private memory also provide additional benefits: > * Significantly lesser memory overhead for handling splitting/merge operations > - With struct pages around, every split of 1G page needs struct > page allocation for 512 * 512 4K pages in worst case. > * Enable roadmap for PFN range allocators in the backend and usecases > like KHO [3] that target use of memory without struct page. > > IIRC, filemap was initially used as a matter of convenience for > initial guest memfd implementation. > > As pointed by David in the call, to get rid of struct page for private > memory ranges, filemap/pagecache needs to be replaced by a lightweight > mechanism that tracks offsets -> pfns mapping for private memory > ranges while still keeping filemap/pagecache for shared memory ranges > (it's still needed to allow GUP usecases). I am starting to think that Going one step further, If we support folio->mapping and possibly any other needed bits while still tracking folios corresponding to shared memory ranges along with private memory pfns in a separate "gmem_cache" to keep core-mm interaction compatible, can that allow pursuing the direction of not needing filemap at all? > the filemap replacement for private memory ranges should be done > sooner rather than later, otherwise it will become more and more > difficult with features landing in guest_memfd relying on presence of > filemap. > > This discussion matters more for hugepages and PFN range allocations. > I would like to ensure that we have consensus on this direction. > > [1] https://lpc.events/event/18/contributions/1764/ > [2] https://lore.kernel.org/kvm/CAGtprH8C4MQwVTFPBMbFWyW4BrK8-mDqjJn-UUFbFhw4w23f3A@mail.gmail.com/ > [3] https://lore.kernel.org/linux-mm/20240805093245.889357-1-jgowans@amazon.com/
In this patch-series: Based on the discussion in the bi-weekly guest_memfd upstream call on 2025-02-20[4], I have dropped the RFC tag, documented the memory allocation behavior after policy changes and added selftests. KVM's guest-memfd memory backend currently lacks support for NUMA policy enforcement, causing guest memory allocations to be distributed arbitrarily across host NUMA nodes regardless of the policy specified by the VMM. This occurs because conventional userspace NUMA control mechanisms like mbind() are ineffective with guest-memfd, as the memory isn't directly mapped to userspace when allocations occur. This patch-series adds NUMA binding capabilities to guest_memfd backend KVM guests. It has evolved through several approaches based on community feedback: - v1,v2: Extended the KVM_CREATE_GUEST_MEMFD IOCTL to pass mempolicy. - v3: Introduced fbind() syscall for VMM memory-placement configuration. - v4-v6: Current approach using shared_policy support and vm_ops (based on suggestions from David[1] and guest_memfd biweekly upstream call[2]). For SEV-SNP guests, which use the guest-memfd memory backend, NUMA-aware memory placement is essential for optimal performance, particularly for memory-intensive workloads. This series implements proper NUMA policy support for guest-memfd by: 1. Adding mempolicy-aware allocation APIs to the filemap layer. 2. Implementing get/set_policy vm_ops in guest_memfd to support shared policy. With these changes, VMMs can now control guest memory placement by specifying: - Policy modes: default, bind, interleave, or preferred - Host NUMA nodes: List of target nodes for memory allocation The policy change only affect future allocations and do not migrate existing memory. This matches mbind(2)'s default behavior which affects only new allocations unless overridden with MPOL_MF_MOVE/MPOL_MF_MOVE_ALL flags, which are not supported for guest_memfd as it is unmovable. This series builds on the existing guest-memfd support in KVM and provides a clean integration path for NUMA-aware memory management in confidential computing environments. The work is primarily focused on supporting SEV-SNP requirements, though the benefits extend to any VMM using the guest-memfd backend that needs control over guest memory placement. == Example usage with QEMU (requires patched QEMU from [3]) == Snippet of the QEMU changes[3] needed to support this feature: /* Create and map guest-memfd region */ new_block->guest_memfd = kvm_create_guest_memfd( new_block->max_length, 0, errp); ... void *ptr_memfd = mmap(NULL, new_block->max_length, PROT_READ | PROT_WRITE, MAP_SHARED, new_block->guest_memfd, 0); ... /* Apply NUMA policy */ int ret = mbind(ptr_memfd, new_block->max_length, backend->policy, backend->host_nodes, maxnode+1, 0); ... QEMU Command to run SEV-SNP guest with interleaved memory across nodes 0 and 1 of the host: $ qemu-system-x86_64 \ -enable-kvm \ ... -machine memory-encryption=sev0,vmport=off \ -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 \ -numa node,nodeid=0,memdev=ram0,cpus=0-15 \ -object memory-backend-memfd,id=ram0,host-nodes=0-1,policy=interleave,size=1024M,share=true,prealloc=false == Experiment and Analysis == SEV-SNP enabled host, AMD Zen 3, 2 socket 2 NUMA node system NUMA for Policy Guest Node 0: policy=interleave, host-node=0-1 Test: Allocate and touch 50GB inside guest on node=0. * Generic Kernel (without NUMA supported guest-memfd): Node 0 Node 1 Total Before running Test: MemUsed 9981.60 3312.00 13293.60 After running Test: MemUsed 61451.72 3201.62 64653.34 Arbitrary allocations: all ~50GB allocated on node 0. * With NUMA supported guest-memfd: Node 0 Node 1 Total Before running Test: MemUsed 5003.88 3963.07 8966.94 After running Test: MemUsed 30607.55 29670.00 60277.55 Balanced memory distribution: Equal increase (~25GB) on both nodes. == Conclusion == Adding the NUMA-aware memory management to guest_memfd will make a lot of sense. Improving performance of memory-intensive and locality-sensitive workloads with fine-grained control over guest memory allocations, as pointed out in the analysis. [1] https://lore.kernel.org/linux-mm/6fbef654-36e2-4be5-906e-2a648a845278@redhat.com [2] https://lore.kernel.org/linux-mm/82c53460-a550-4236-a65a-78f292814edb@redhat.com [3] https://github.com/shivankgarg98/qemu/tree/guest_memfd_mbind_NUMA [4] https://lore.kernel.org/linux-mm/2b77e055-98ac-43a1-a7ad-9f9065d7f38f@amd.com == Earlier postings and changelogs == v6 (current): - Rebase to linux mainline - Drop RFC tag - Add selftests to ensure NUMA support for guest_memfd works correctly. v5: - https://lore.kernel.org/linux-mm/20250219101559.414878-1-shivankg@amd.com - Fix documentation and style issues. - Use EXPORT_SYMBOL_GPL - Split preparatory change in separate patch v4: - https://lore.kernel.org/linux-mm/20250210063227.41125-1-shivankg@amd.com - Dropped fbind() approach in favor of shared policy support. v3: - https://lore.kernel.org/linux-mm/20241105164549.154700-1-shivankg@amd.com - Introduce fbind() syscall and drop the IOCTL-based approach. v2: - https://lore.kernel.org/linux-mm/20240919094438.10987-1-shivankg@amd.com - Add fixes suggested by Matthew Wilcox. v1: - https://lore.kernel.org/linux-mm/20240916165743.201087-1-shivankg@amd.com - Proposed IOCTL based approach to pass NUMA mempolicy. Shivank Garg (4): mm/mempolicy: export memory policy symbols KVM: guest_memfd: Pass file pointer instead of inode pointer KVM: guest_memfd: Enforce NUMA mempolicy using shared policy KVM: guest_memfd: selftests: add tests for mmap and NUMA policy support Shivansh Dhiman (1): mm/filemap: add mempolicy support to the filemap layer include/linux/pagemap.h | 39 +++++++++ mm/filemap.c | 30 +++++-- mm/mempolicy.c | 6 ++ .../testing/selftests/kvm/guest_memfd_test.c | 86 +++++++++++++++++- virt/kvm/guest_memfd.c | 87 +++++++++++++++++-- 5 files changed, 233 insertions(+), 15 deletions(-)