From patchwork Tue Feb 11 00:40:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nico Pache X-Patchwork-Id: 13968649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8038C0219D for ; Tue, 11 Feb 2025 00:42:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5663A280005; Mon, 10 Feb 2025 19:42:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5146728000C; Mon, 10 Feb 2025 19:42:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38D39280005; Mon, 10 Feb 2025 19:42:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 19A5B280003 for ; Mon, 10 Feb 2025 19:42:31 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C0F858085C for ; Tue, 11 Feb 2025 00:42:30 +0000 (UTC) X-FDA: 83105812860.14.F5FAC35 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 1121A1A0008 for ; Tue, 11 Feb 2025 00:42:28 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RgTCgvgd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of npache@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=npache@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739234549; a=rsa-sha256; cv=none; b=IbJUNa5kJj6G8vvRPFLbQr+6A6jnxtPCN3fx4O+dhvKVRKaq9zvbzavdNoW7w+zO/CTXBI EEFe+0SaTzA4g1COcWAWKfyFewDddnooz09Bp6uANoyoU4rsMuORJOLmJt8z6RPleg+QPL BLTKEdmCU7AhsPQJgXxJLA2JAQSfl+s= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RgTCgvgd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of npache@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=npache@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739234549; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9t+x6jSPF/Kaax2beF3oSfdLLrK+SQWfb/7DMQGVOLs=; b=n3HJA+zH8+D9IhBopwlDlIg3I/W5ec2RUp9FDh/lscBsoal9Rle7aXABxXMulObxUO2LC0 o1FKt1m+KxJb6xi5uUS2L1LwnLKTXzTjmtBy1wC2H7+nOJJ4AnidwAMkud/Vzed5K3nCKk 3ywBZmviMMLUHeakqrmQ0/fCa/O8xfk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739234548; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9t+x6jSPF/Kaax2beF3oSfdLLrK+SQWfb/7DMQGVOLs=; b=RgTCgvgdNbM1NXNc6eu7LZQpDrSAFe3WBELCiTxaiPXHkFzCnddJifIPT3wmGIdYxUkzmo 5H0boztPg5oORYMx78l2cBs80km6pCHkWRnyeO4tIy5S+V5/JU0uo/YO+3aebMnZCsK7VZ hqz7YF9nm7vwVv+8+7qj7DLSpcg+jGE= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-Ku2qpg2aMryfCR2GurdvVQ-1; Mon, 10 Feb 2025 19:42:23 -0500 X-MC-Unique: Ku2qpg2aMryfCR2GurdvVQ-1 X-Mimecast-MFC-AGG-ID: Ku2qpg2aMryfCR2GurdvVQ Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5936B180087A; Tue, 11 Feb 2025 00:42:19 +0000 (UTC) Received: from h1.redhat.com (unknown [10.22.88.129]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 602901800873; Tue, 11 Feb 2025 00:42:10 +0000 (UTC) From: Nico Pache To: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org Cc: ryan.roberts@arm.com, anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, jhubbard@nvidia.com, 21cnbao@gmail.com, willy@infradead.org, kirill.shutemov@linux.intel.com, david@redhat.com, aarcange@redhat.com, raquini@redhat.com, dev.jain@arm.com, sunnanyong@huawei.com, usamaarif642@gmail.com, audra@redhat.com, akpm@linux-foundation.org, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, tiwai@suse.de, baolin.wang@linux.alibaba.com, corbet@lwn.net, shuah@kernel.org Subject: [RFC v2 5/5] mm: document mTHP defer setting Date: Mon, 10 Feb 2025 17:40:54 -0700 Message-ID: <20250211004054.222931-6-npache@redhat.com> In-Reply-To: <20250211004054.222931-1-npache@redhat.com> References: <20250211004054.222931-1-npache@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Rspam-User: X-Rspamd-Queue-Id: 1121A1A0008 X-Rspamd-Server: rspam12 X-Stat-Signature: 439bz7n78qprxzpxyoj7nati9y5ituh7 X-HE-Tag: 1739234548-648622 X-HE-Meta: U2FsdGVkX1+IBFuYBsWPjIRZyjBWhRu0A12hdYYCD183LIYiJndpyNRkVsy8cVr3w6I2no3aPuX0jHPMhape3xgnHoribqGXnSSD6EP+aXhR3aChnZIe/SPH3bLxxK4AWCc2JKf/vnKquK4M93nvt2qJgZT2HLby9iH/j8Ql8v8K5GkuwykwYev0tu0FOcgcUln5mDnr34nov0sFER/Z46E9GZ0KjMeW1o83Rj93+TrxsRdlzj70Xpyk93YXEVUg+NmyzD27VYdgqTHKcmfJpj/IEwm09uRoS+pIg0YYWBabXrGqr0yjUOJd0YlI0Sd45RvcxSx2qhcxGp3eqPPclIng58vWmjm59ZPvjN7Spp2q1CM2XkcGYkV7TnOZl0Rw1SSv6T/pfeFfp4JkADyZ2sC4oHa98uQjpJ9lbh/mRnZS9wjIJU2VmekY7zwqnvHIS7H/sYkQiBgoIzMP84KwPuEYNjukMWXcS7p4ityGZrO9rVuS901zkCECIAJr9WcT1azx835eOdZ2dc6uo26ghNzSPJRyafckw636NTSXu3w7l61kdTK3lhfifhag3igHBYc5fDx0zaqKGm2ghHeeq7nRHuve/Kp4cOiGRwdW/Cy+qtY1ULmd9TfjH9FGgMLT1X3pE0Zj+W+pfRcKMtVWw7K4yBFY9gKtkkQXrreIQyEjnTdV4+EtFkooLBeSXXxJVtJukEhk6SQt8y7/AYFRacykJZs055U7w+IwuHOk4ZQ9umInjOdi2pZ7fksePr+QQxvkBvNwj07NrvwVggBO8J8I4Bn5Xd2/6nFwOodgVWlKPulGbxGvmfocE431nCtiGtYXAUKy0nw7nzp5ddnHSfYcCty640YePAvpM1fpCgN9FlXfbfUkk0qLB82NuUr365l6ZrGD4TeOY5o3FFZkmx0j3e6mSdRO0NoviRMTBtTuCXMZlqH6n33M21RE9lyp+SjFD6VpN80LYVUwMay cGS2qrQK xOHGL0RN4JJyNgeRSxKrVfA0RC1H5BeVXxLWhTkLarNkDXfIIgvCjTo4Eahl9utiEMvGkkK+gEodyor9Kid59RQfLOl97lLUlkDv85GPtlQTBcaz8DheCXA8INC659QQ/sa4ZnCBmcnilfvvKx/hb4zelnFoA5U2GoKOyOMnlJbiOnrdvn6uwZKmByVl5AGXY1agxxFwcwlaXcm+jHrApG/+6d560sSb7kNdhwITR6jBbUOgJ5quRlP4Y+kIMI/3wZOXDWOgj83nb8/5a7LjfbIox7qV06UBv4tPA0A8eF0mF3fnKl0Au9eQP+q9zWusApTMmphMwc0py2GmIs0cZj0iYapQJCfTSEoI4v8FYV4J5Y5L3IttZZlaa5QsISSF7OpLZABq5gHH0RXnVAw/mYWkTM5mILXQ1R+TDzQa5oK3eTdKfamwwcc1mLA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that we have mTHP support in khugepaged, lets add it to the transhuge admin guide to provide proper guidance. Signed-off-by: Nico Pache --- Documentation/admin-guide/mm/transhuge.rst | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index b3b18573bbb4..99ba3763c1c4 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -63,7 +63,7 @@ often. THP can be enabled system wide or restricted to certain tasks or even memory ranges inside task's address space. Unless THP is completely disabled, there is ``khugepaged`` daemon that scans memory and -collapses sequences of basic pages into PMD-sized huge pages. +collapses sequences of basic pages into huge pages. The THP behaviour is controlled via :ref:`sysfs ` interface and using madvise(2) and prctl(2) system calls. @@ -103,8 +103,8 @@ madvise(MADV_HUGEPAGE) on their critical mmapped regions. Applications that would like to benefit from THPs but would still like a more memory conservative approach can choose 'defer'. This avoids inserting THPs at the page fault handler unless they are MADV_HUGEPAGE. -Khugepaged will then scan the mappings for potential collapses into PMD -sized pages. Admins using this the 'defer' setting should consider +Khugepaged will then scan the mappings for potential collapses into (m)THP +pages. Admins using this the 'defer' setting should consider tweaking khugepaged/max_ptes_none. The current default of 511 may aggressively collapse your PTEs into PMDs. Lower this value to conserve more memory (ie. max_ptes_none=64). @@ -119,11 +119,14 @@ Global THP controls Transparent Hugepage Support for anonymous memory can be entirely disabled (mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE -regions (to avoid the risk of consuming more memory resources) or enabled -system wide. This can be achieved per-supported-THP-size with one of:: +regions (to avoid the risk of consuming more memory resources), defered to +khugepaged, or enabled system wide. + +This can be achieved per-supported-THP-size with one of:: echo always >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled echo madvise >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled + echo defer >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled echo never >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled where is the hugepage size being addressed, the available sizes @@ -155,6 +158,13 @@ hugepage sizes have enabled="never". If enabling multiple hugepage sizes, the kernel will select the most appropriate enabled size for a given allocation. +khugepaged use max_ptes_none scaled to the order of the enabled mTHP size to +determine collapses. When using mTHPs its recommended to set max_ptes_none low. +Ideally less than HPAGE_PMD_NR / 2 (255 on 4k page size). This will prevent +undesired "creep" behavior that leads to continously collapsing to a larger +mTHP size. max_ptes_shared and max_ptes_swap have no effect when collapsing to a +mTHP, and mTHP collapse will fail on shared or swapped out pages. + It's also possible to limit defrag efforts in the VM to generate anonymous hugepages in case they're not immediately free to madvise regions or to never try to defrag memory and simply fallback to regular @@ -318,7 +328,7 @@ Alternatively, each supported anonymous THP size can be controlled by passing ``thp_anon=[KMG],[KMG]:;[KMG]-[KMG]:``, where ```` is the THP size (must be a power of 2 of PAGE_SIZE and supported anonymous THP) and ```` is one of ``always``, ``madvise``, -``never`` or ``inherit``. +``defer``, ``never`` or ``inherit``. For example, the following will set 16K, 32K, 64K THP to ``always``, set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M