From patchwork Mon Dec 7 11:30:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: yulei zhang X-Patchwork-Id: 11955403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8053CC4361B for ; Mon, 7 Dec 2020 11:31:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C15DE2339F for ; Mon, 7 Dec 2020 11:31:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C15DE2339F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BD0D08D0002; Mon, 7 Dec 2020 06:31:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B817B8D0001; Mon, 7 Dec 2020 06:31:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D42B8D0002; Mon, 7 Dec 2020 06:31:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 8029B8D0001 for ; Mon, 7 Dec 2020 06:31:39 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3A8738249980 for ; Mon, 7 Dec 2020 11:31:39 +0000 (UTC) X-FDA: 77566271118.30.shop75_3918393273de Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 15AE6180B3C85 for ; Mon, 7 Dec 2020 11:31:39 +0000 (UTC) X-HE-Tag: shop75_3918393273de X-Filterd-Recvd-Size: 11615 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Mon, 7 Dec 2020 11:31:38 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id g18so8666050pgk.1 for ; Mon, 07 Dec 2020 03:31:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ILnSBWwyABm/ViEoQp4lFyaqVJLrOFveysOKfZkRhag=; b=rOqpLC970kODnReOtTNxrnnMNPzauP6Lc115KldbryjjQRCZwd2tgMSQz36C/kXt9Z 56dN0/oxHJyldy4W9HdAVs4m9hHTgiYq6XORi2+17t9QxzLq9CSyGMoY6sU9WsUCUJmq hFSae+7CYWJuWnmH/utLWNjoeEGfdF6L92cc0b3mNBllb5TrS0zuwZQIrwEPB3RlN99I AL9gnTSq4awwyxv0ReEbuh0yGvL6GN3JwGQN39IHk9PEYfw2jBETrr4s+uq3N4+1Qm17 takM/sOTbb34RuK6uOEAKMGSZYeKzXJt0NLJiHOg0sAG9T948DCs8tOXnElF9Bk+1YeY e2WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ILnSBWwyABm/ViEoQp4lFyaqVJLrOFveysOKfZkRhag=; b=bOriLE87sOEyo6ek5iAQAZNvNa6ESF5TTsY3+M/F7FzMtSLmknDJIUv/jQZVJGp6Zf cBKqoh+ccwrFFeFd9bYUvjUfjtop/u4J2oo5A+gcLP5H+edF95Miy+Tr7zyL+1Xz32bm 4w7dRJGNCb7VdU0IpHWIWiOsyuIwiBVy4CXOa4QINrVy2ddBuT2ZT16PFg6dQ07M7jRI d4ZONqtCS2ZJm8GWrT38rZJZwcZ7Dn9cdaf5oxRP2f6Jl4qoNoKSwADS8TDRKlGPWT13 sj4WlhKf5I3osorFmlq8p8nEgh9IHDsFrlKguxsMVojhv6S7g97Vs5v8kwKE5k04Cnbn oxqQ== X-Gm-Message-State: AOAM531R6e1o+YWyA6RHxlZLKagFiVtxHJ+XAux8ofvxIJMQ0SXV6Kjd oAHkcSDHLbT9IQYmc7nsDLh+GnePDAc= X-Google-Smtp-Source: ABdhPJymSUXlaRjzXEzU2ngDriybhzR1Pzab6/cvJ6uKqjlNaxk3d/92EIH+NhH+um1mO4rml2NgOw== X-Received: by 2002:a17:902:6103:b029:da:c46c:b3d6 with SMTP id t3-20020a1709026103b02900dac46cb3d6mr15400411plj.46.1607340696714; Mon, 07 Dec 2020 03:31:36 -0800 (PST) Received: from localhost.localdomain ([203.205.141.39]) by smtp.gmail.com with ESMTPSA id d4sm14219822pfo.127.2020.12.07.03.31.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 03:31:36 -0800 (PST) From: yulei.kernel@gmail.com X-Google-Original-From: yuleixzhang@tencent.com To: linux-mm@kvack.org, akpm@linux-foundation.org, linux-fsdevel@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, naoya.horiguchi@nec.com, viro@zeniv.linux.org.uk, pbonzini@redhat.com Cc: joao.m.martins@oracle.com, rdunlap@infradead.org, sean.j.christopherson@intel.com, xiaoguangrong.eric@gmail.com, kernellwp@gmail.com, lihaiwei.kernel@gmail.com, Yulei Zhang Subject: [RFC V2 00/37] Enhance memory utilization with DMEMFS Date: Mon, 7 Dec 2020 19:30:53 +0800 Message-Id: X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yulei Zhang In current system each physical memory page is assocaited with a page structure which is used to track the usage of this page. But due to the memory usage rapidly growing in cloud environment, we find the resource consuming for page structure storage becomes more and more remarkable. So is it possible that we could reclaim such memory and make it reusable? This patchset introduces an idea about how to save the extra memory through a new virtual filesystem -- dmemfs. Dmemfs (Direct Memory filesystem) is device memory or reserved memory based filesystem. This kind of memory is special as it is not managed by kernel and most important it is without 'struct page'. Therefore we can leverage the extra memory from the host system to support more tenants in our cloud service. As the belowing figure shows, we uses a kernel boot parameter 'dmem=' to reserve the system memory when the host system boots up, the remaining system memory is still managed by system memory management which is associated with "struct page", the reserved memory will be managed by dmem and assigned to guest system, the details can be checked in /Documentation/admin-guide/kernel-parameters.txt. +------------------+--------------------------------------+ | system memory | memory for guest system | +------------------+--------------------------------------+ | | v | struct page | | | v v system mem management dmem And during the usage, the dmemfs will handle the memory request to allocate and free the reserved memory on each NUMA node, the user space application could leverage the mmap interface to access the memory, and kernel module such as kvm and vfio would be able to pin the memory thongh follow_pfn() and get_user_page() in different given page size granularities. +-----------+ +-----------+ | QEMU | | dpdk etc.| user +-----+-----+ +-----------+ +-----------|------\------------------------------+ | | v kernel | | | +-------+ +-------+ | | | | KVM | | vfio | | | | +-------+ +-------+ | | | | | | | +----v---------v----------v------+ | | | | | | | Dmemfs | | | | | | | +--------------------------------+ | +-----------/-----------------------\-------------+ / \ +------v-----+ +----v-------+ | node 0 | | node 1 | +------------+ +------------+ Theoretically for each 4k physical page it can save 64 bytes if we drop the 'struct page', so for guest memory with 320G it can save about 5G physical memory totally. Detailed usage of dmemfs is included in /Documentation/filesystem/dmemfs.rst. V1->V2: * Rebase the code the kernel version 5.10.0-rc3. * Introudce dregion->memmap for dmem to add _refcount for each dmem page. * Enable record_steal_time for dmem before entering guest system. * Adjust page walking for dmem. Yulei Zhang (37): fs: introduce dmemfs module mm: support direct memory reservation dmem: implement dmem memory management dmem: let pat recognize dmem dmemfs: support mmap for dmemfs dmemfs: support truncating inode down dmem: trace core functions dmem: show some statistic in debugfs dmemfs: support remote access dmemfs: introduce max_alloc_try_dpages parameter mm: export mempolicy interfaces to serve dmem allocator dmem: introduce mempolicy support mm, dmem: introduce PFN_DMEM and pfn_t_dmem mm, dmem: differentiate dmem-pmd and thp-pmd mm: add pmd_special() check for pmd_trans_huge_lock() dmemfs: introduce ->split() to dmemfs_vm_ops mm, dmemfs: support unmap_page_range() for dmemfs pmd mm: follow_pmd_mask() for dmem huge pmd mm: gup_huge_pmd() for dmem huge pmd mm: support dmem huge pmd for vmf_insert_pfn_pmd() mm: support dmem huge pmd for follow_pfn() kvm, x86: Distinguish dmemfs page from mmio page kvm, x86: introduce VM_DMEM for syscall support usage dmemfs: support hugepage for dmemfs mm, x86, dmem: fix estimation of reserved page for vaddr_get_pfn() mm, dmem: introduce pud_special() for dmem huge pud support mm: add pud_special() check to support dmem huge pud mm, dmemfs: support huge_fault() for dmemfs mm: add follow_pte_pud() to support huge pud look up dmem: introduce dmem_bitmap_alloc() and dmem_bitmap_free() dmem: introduce mce handler mm, dmemfs: register and handle the dmem mce kvm, x86: enable record_steal_time for dmem dmem: add dmem unit tests mm, dmem: introduce dregion->memmap for dmem vfio: support dmempage refcount for vfio Add documentation for dmemfs Documentation/admin-guide/kernel-parameters.txt | 38 + Documentation/filesystems/dmemfs.rst | 58 ++ Documentation/filesystems/index.rst | 1 + arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 32 +- arch/x86/include/asm/pgtable_types.h | 13 +- arch/x86/kernel/setup.c | 3 + arch/x86/kvm/mmu/mmu.c | 1 + arch/x86/mm/pat/memtype.c | 21 + drivers/vfio/vfio_iommu_type1.c | 13 +- fs/Kconfig | 1 + fs/Makefile | 1 + fs/dmemfs/Kconfig | 16 + fs/dmemfs/Makefile | 8 + fs/dmemfs/inode.c | 1060 ++++++++++++++++++++ fs/dmemfs/trace.h | 54 + fs/inode.c | 6 + include/linux/dmem.h | 54 + include/linux/fs.h | 1 + include/linux/huge_mm.h | 5 +- include/linux/mempolicy.h | 3 + include/linux/mm.h | 9 + include/linux/pfn_t.h | 17 +- include/linux/pgtable.h | 22 + include/trace/events/dmem.h | 85 ++ include/uapi/linux/magic.h | 1 + mm/Kconfig | 19 + mm/Makefile | 1 + mm/dmem.c | 1196 +++++++++++++++++++++++ mm/dmem_reserve.c | 303 ++++++ mm/gup.c | 101 +- mm/huge_memory.c | 19 +- mm/memory-failure.c | 70 +- mm/memory.c | 74 +- mm/mempolicy.c | 4 +- mm/mincore.c | 8 +- mm/mprotect.c | 7 +- mm/mremap.c | 3 + mm/pagewalk.c | 4 +- tools/testing/dmem/Kbuild | 1 + tools/testing/dmem/Makefile | 10 + tools/testing/dmem/dmem-test.c | 184 ++++ virt/kvm/kvm_main.c | 13 +- 43 files changed, 3483 insertions(+), 58 deletions(-) create mode 100644 Documentation/filesystems/dmemfs.rst create mode 100644 fs/dmemfs/Kconfig create mode 100644 fs/dmemfs/Makefile create mode 100644 fs/dmemfs/inode.c create mode 100644 fs/dmemfs/trace.h create mode 100644 include/linux/dmem.h create mode 100644 include/trace/events/dmem.h create mode 100644 mm/dmem.c create mode 100644 mm/dmem_reserve.c create mode 100644 tools/testing/dmem/Kbuild create mode 100644 tools/testing/dmem/Makefile create mode 100644 tools/testing/dmem/dmem-test.c