From patchwork Tue Oct 18 23:50:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13011207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E28B7C433FE for ; Tue, 18 Oct 2022 23:51:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58B516B0072; Tue, 18 Oct 2022 19:51:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 53B426B0073; Tue, 18 Oct 2022 19:51:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 429EA6B0074; Tue, 18 Oct 2022 19:51:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 335A76B0072 for ; Tue, 18 Oct 2022 19:51:06 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 04041A08EA for ; Tue, 18 Oct 2022 23:51:05 +0000 (UTC) X-FDA: 80035718532.26.2BF4A99 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf20.hostedemail.com (Postfix) with ESMTP id 903FE1C0012 for ; Tue, 18 Oct 2022 23:51:05 +0000 (UTC) Received: by mail-pj1-f73.google.com with SMTP id b1-20020a17090a10c100b0020da29fa5e5so7547297pje.2 for ; Tue, 18 Oct 2022 16:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mwwrA+AKJG2rv4I4FBclmb9mZ0XqPEz5+xc96NXfVbo=; b=TnlJdkmWH28tthLcGelqLd0I/jGxMhGO75+K7nEELFCKTI+s/3/mHiZC3wijcMamyG Vdr221Gocs4t41NOolPIFGBvT8qU79G1GxpyHPnKezD9IRtTKlM8E1BW9TDQZIywBHGi xlDJ2acHFF3P/iQ1AggZTpUbiI1YZ5ChvcwfIR9h4lOAw5tCblDQ817R2IpJ/9GqxIrD DAMF6GlMs1RYg1UOPp0QlzgXnoENnUbjRWWehQY1vEQ/Cr/wkXRL4Wmllp9GkzvKPRue U9uQANY0Z8FVyJzncKJBypYDjKIK4pxHbLmF3zLGB46I2HlHXskig+zShpaJEqy81uBb ubcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mwwrA+AKJG2rv4I4FBclmb9mZ0XqPEz5+xc96NXfVbo=; b=LKqafK8XqijR+GLrkGqhB19oj7AdZEHT40Fiu1tyamKB1XwIWlWOXecOxAWcz0e6Sf LT2Ou2B43kEMamD7a328NIleRK77Id+1g4ZPwRkfmtJDmlwYJIK0ZElG2cL2/6aEGscC wIZiarIEUqZ5aR8hCp7KPxIk+k9fUO2X2cqdGdD3O0hFh8Edknm340Ge7Ol5pmLEPVR7 y9MWa8Zn0rj5jR1kgRAtzZwiNP/jHHkdqqh/sbeZ2UEnTUU8aOtVpjrpkk3folcnoNHO xq/u6rn6YDlkaH4LEg0fQKax0RUzgh5Z3XxbN5ftE3GADSWdvAEDlaH/251xcAAFnBDX rupQ== X-Gm-Message-State: ACrzQf00vNhRlz1TShP0nm744XGOBQKZfMBMTRq65ri0G4bYF/PvDbWB eAD9yGyOClu/mLy632ZAeej7okDXRoH2 X-Google-Smtp-Source: AMsMyM7x5IWjOuzuBkp4TLq/73g6nfs6biMwjKxfPOkF+RuhdxgsQgpU+G1TxT9q5LT270Wsv4OORvzB4H42 X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a05:6a00:1a92:b0:565:d5c0:f627 with SMTP id e18-20020a056a001a9200b00565d5c0f627mr5787283pfv.10.1666137064421; Tue, 18 Oct 2022 16:51:04 -0700 (PDT) Date: Tue, 18 Oct 2022 16:50:48 -0700 In-Reply-To: <20221018235051.152548-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221018235051.152548-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221018235051.152548-2-zokeefe@google.com> Subject: [PATCH man-pages v2 1/4] madvise.2: update THP file/shmem documentation for +5.4 From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666137065; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mwwrA+AKJG2rv4I4FBclmb9mZ0XqPEz5+xc96NXfVbo=; b=X5xCGJypexAQ/EWYVRNKeqf2mPIlgqTT4dv9qgbjjbLp7b434dOUKXfgwomHMHmGWYprNa 4khhAIa7Y7UMK/fyhQH732FpLVlZ9K2ba/OX27uGx0cbnaiWYznFJmwxPJzjb2G8NrxYLW eaCh/trkzOWp/gx9tmuj4GccUnp9iSg= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TnlJdkmW; spf=pass (imf20.hostedemail.com: domain of 36DtPYwcKCL02rnhhihjrrjoh.frpolqx0-ppnydfn.ruj@flex--zokeefe.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=36DtPYwcKCL02rnhhihjrrjoh.frpolqx0-ppnydfn.ruj@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666137065; a=rsa-sha256; cv=none; b=JVH1bXmsENqCgUkWJ9s1Edqi73bcvfWBxDeo71yBVA3Vc5eGpSrQAK0XzzPbRx6VWy8rxh DfYKORyvceNNBPa8kN27dr2yw0ZsN9ct+MJnb880cAvWUp90a8MBls6mohUUjUoIqzm4Wr 347pNU0ugQjX687y3S4v3wn2WsRGGt0= Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TnlJdkmW; spf=pass (imf20.hostedemail.com: domain of 36DtPYwcKCL02rnhhihjrrjoh.frpolqx0-ppnydfn.ruj@flex--zokeefe.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=36DtPYwcKCL02rnhhihjrrjoh.frpolqx0-ppnydfn.ruj@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 903FE1C0012 X-Stat-Signature: w6bug1csahyjwxiq3qsffxnkweefok7i X-Rspam-User: X-HE-Tag: 1666137065-170047 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe Since Linux 5.4, Transparent Huge Pages now support both file-backed memory and shmem memory. Update MADV_HUGEPAGE advice description to reflect this. Additionally, expand the description of requirements for memory to be considered eligible for THP: alignment / mapping requirements, VMA flags, prctl(2) settings, inode status, etc. Signed-off-by: Zach O'Keefe --- man2/madvise.2 | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/man2/madvise.2 b/man2/madvise.2 index 81cce56af..64f788ace 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -320,8 +320,6 @@ Enable Transparent Huge Pages (THP) for pages in the range specified by .I addr and .IR length . -Currently, Transparent Huge Pages work only with private anonymous pages (see -.BR mmap (2)). The kernel will regularly scan the areas marked as huge page candidates to replace them with huge pages. The kernel will also allocate huge pages directly when the region is @@ -354,12 +352,46 @@ an access pattern that the developer knows in advance won't risk to increase the memory footprint of the application when transparent hugepages are enabled. .IP +.\" commit 99cb0dbd47a15d395bf3faa78dc122bc5efe3fc0 +Since Linux 5.4, +automatic scan of eligible areas and replacement by huge pages works with +private anonymous pages (see +.BR mmap (2)), +shmem pages, +and file-backed pages. +For all memory types, +memory may only be replaced by huge pages on hugepage-aligned boundaries. +For file-mapped memory \(em including tmpfs (see +.BR tmpfs (2)) +\(em the mapping must also be naturally hugepage-aligned within the file. +Additionally, +for file-backed, +non-tmpfs memory, +the file must not be open for write and the mapping must be executable. +.IP +The VMA must not be marked +.BR VM_NOHUGEPAGE , +.BR VM_HUGETLB , +.BR VM_IO , +.BR VM_DONTEXPAND , +.BR VM_MIXEDMAP , +or +.BR VM_PFNMAP , +nor can it be stack memory or backed by a DAX-enabled device +(unless the DAX device is hot-plugged as System RAM). +The process must also not have +.B PR_SET_THP_DISABLE +set (see +.BR prctl (2) ). +.IP The .B MADV_HUGEPAGE and .B MADV_NOHUGEPAGE operations are available only if the kernel was configured with -.BR CONFIG_TRANSPARENT_HUGEPAGE . +.B CONFIG_TRANSPARENT_HUGEPAGE +and file/shmem memory is only supported if the kernel was configured with +.BR CONFIG_READ_ONLY_THP_FOR_FS . .TP .BR MADV_NOHUGEPAGE " (since Linux 2.6.38)" Ensures that memory in the address range specified by From patchwork Tue Oct 18 23:50:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13011208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E023EC4332F for ; Tue, 18 Oct 2022 23:51:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D57C6B0073; Tue, 18 Oct 2022 19:51:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 785956B0074; Tue, 18 Oct 2022 19:51:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64D226B0075; Tue, 18 Oct 2022 19:51:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 54A476B0073 for ; Tue, 18 Oct 2022 19:51:08 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 22D98C0167 for ; Tue, 18 Oct 2022 23:51:08 +0000 (UTC) X-FDA: 80035718616.06.5145928 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf30.hostedemail.com (Postfix) with ESMTP id B512B80024 for ; Tue, 18 Oct 2022 23:51:07 +0000 (UTC) Received: by mail-pl1-f201.google.com with SMTP id c12-20020a170903234c00b0017f695bf8f0so10584627plh.6 for ; Tue, 18 Oct 2022 16:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=wd/fY6QKSavm7D1EQgjukQCfw7sIII/V1ACtOSRxwG8=; b=aHetoDSIQJVnq8NUhnzmf11U/kL09k6c1YTQSkzohD/Z0UO0iLChwZCJGBfY4RQvdG EZepcBLhTRbaHHrlNGfg7amGPhNd9pL4e7jrdVABKpz41BeLajfQ9v+ujhM4xEbeW/fm JOnNrCfU1JviAtDdUsq24AHLNcEELrMBGU4JxugsMOovDH0pOriXVvXkvO1pvGJ9mFm2 9LqzOZDrZRe5bf7RxxouQ7QQZ81kLpmastbmmOu4gpGzQ8srLbDkYnzgn8DASqo787bU zeVM/h9VsHOJK62K1ltU4N9M7megtQQ6av68Xr52yP+vZhTT7hRFXOjWVW4mdscZiwJ8 rTSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wd/fY6QKSavm7D1EQgjukQCfw7sIII/V1ACtOSRxwG8=; b=WF2IUm0Fmq5mR4nEOI05oZGEW0UFM8jZsRxN7iKsbYZSO8t1wfV5X2T7suVrutE2a4 /xXsMnVUcg/jy4uDWb2VhCgLpbS0sdi2n22Hy/CfaAJPfL6DEEPIuapdIMXkXwbYPdyg sn/t8wmeH6qiz7A3kbFJTHVom2i2wsv0j1srAoJjX4oK2gAyC35jtG9OkoHM6rj1v5QB EPSEX92IYWXD6KEutxoBtNQL0mjV1iMKFuSWIZJD0XwDL0i8/5ZcS1qG0Qm+I40BSS8V jb7TFJvLrfPGRmuM2cF1lOsqwvl5VKEBKcf3ZdrkofaWsWg0LollB//UUYG6oR3EaFjv bFIg== X-Gm-Message-State: ACrzQf3vXaSZbSk18bBPtnPr2k1SxnlO8QDfAz1iyA0EBc8l0I124vri Q6hQXK8D//y2WsJW5uCdUNiNhdyXN2Hp X-Google-Smtp-Source: AMsMyM6F3VrdnWK11gjs2ukDG4Yeylh6kcOkNOnMEXcElOKlidoqn4RHNF5QmFfW5WRXBoMTki3TK2Kdf3QI X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:90a:4811:b0:20a:fee1:8f69 with SMTP id a17-20020a17090a481100b0020afee18f69mr1905851pjh.0.1666137066188; Tue, 18 Oct 2022 16:51:06 -0700 (PDT) Date: Tue, 18 Oct 2022 16:50:49 -0700 In-Reply-To: <20221018235051.152548-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221018235051.152548-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221018235051.152548-3-zokeefe@google.com> Subject: [PATCH man-pages v2 2/4] madvise.2: document reliable probe for advice support From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=aHetoDSI; spf=pass (imf30.hostedemail.com: domain of 36jtPYwcKCL84tpjjkjlttlqj.htrqnsz2-rrp0fhp.twl@flex--zokeefe.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=36jtPYwcKCL84tpjjkjlttlqj.htrqnsz2-rrp0fhp.twl@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666137067; a=rsa-sha256; cv=none; b=a+ROFnkFb5fI+FK6cOcWVaz0lbaqXjADfza9lu1B0gvdp/looOvL+RYwcc155n1PufU571 +rWIIYRL+605Y1HhG43Z9wDpkbAzgq262U84Vn1h8cgce1svZZjwZl0X/RyNA6Zyv6q14p o29QMj1sWTIH2gA7WJaOXUN6683KHSo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666137067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wd/fY6QKSavm7D1EQgjukQCfw7sIII/V1ACtOSRxwG8=; b=X3Y6pNH4AdyTTl5iJHjJPHqpwlyEMKmVCGZQltsejxHSjyIZmpQgAs3ihZ/L4pcjP+rKWI kvSdENUv1VN65UZ2AjltM3wUjKFf49yh6IuoUWVUKguEydUFyiRsUCxXdaAJVElAfcCavR l+IcUqTOt9uYCZTeAussqOom4oqPbDY= X-Stat-Signature: bnzsrzsidhawqfzbb8atj5jcyz94xpab X-Rspamd-Queue-Id: B512B80024 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=aHetoDSI; spf=pass (imf30.hostedemail.com: domain of 36jtPYwcKCL84tpjjkjlttlqj.htrqnsz2-rrp0fhp.twl@flex--zokeefe.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=36jtPYwcKCL84tpjjkjlttlqj.htrqnsz2-rrp0fhp.twl@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1666137067-389469 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe EINVAL is an overloaded error code for madvise(2) and it's not clear under what context it means "advice is not valid" vs another error. Explicitly document that madvise(0, 0, advice) can reliably be used to probe for kernel support for "advice", returning zero iff "advice" is supported by the kernel. Signed-off-by: Zach O'Keefe --- man2/madvise.2 | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/man2/madvise.2 b/man2/madvise.2 index 64f788ace..df3413cc8 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -790,6 +790,11 @@ that are not mapped, the Linux version of ignores them and applies the call to the rest (but returns .B ENOMEM from the system call, as it should). +.PP +.IR madvise(0,\ 0,\ advice) +will return zero iff +.I advice +is supported by the kernel and can be relied on to probe for support. .\" .SH HISTORY .\" The .\" .BR madvise () From patchwork Tue Oct 18 23:50:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13011209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE760C43219 for ; Tue, 18 Oct 2022 23:51:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B8B06B0074; Tue, 18 Oct 2022 19:51:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 368736B0075; Tue, 18 Oct 2022 19:51:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E2516B0078; Tue, 18 Oct 2022 19:51:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0E6AC6B0074 for ; Tue, 18 Oct 2022 19:51:10 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BE6BA1A01B2 for ; Tue, 18 Oct 2022 23:51:09 +0000 (UTC) X-FDA: 80035718658.16.20C6DEA Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf20.hostedemail.com (Postfix) with ESMTP id 4FFDA1C0006 for ; Tue, 18 Oct 2022 23:51:09 +0000 (UTC) Received: by mail-pg1-f202.google.com with SMTP id e187-20020a6369c4000000b0041c8dfb8447so8884833pgc.23 for ; Tue, 18 Oct 2022 16:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xX7SZCGfmXZ9MOjmP6qyYqMKH5CuY/6TOQvAYYPIYYw=; b=sHdWx7G9/fM/c7r6+WsofMB7vJ9sI1JAAObHcdZNH616Gp1lQbsJRPvhDu7Xd2B194 +vrmMs9tTJgglNdY2WqJa1ngBneV0HuUCNHVuoq+iOyLEQkE78VzhkYZi32RqLibB0js W9b4vUtBXVLSptnxcRa+kvdmIJaN2+INYv44xU0bNSVdncAFkPMj+uMsvnNo4UwDhmjR EKVIWSf/nvBrLN3qUyte5Yiz/4eeM1UaBCbYAHVpFTOKkvarQfDMewOZ/QS+3D5kUbWJ 8bcTTLX3ddwOE/GYLMhpJKEyuq/NgARqn/nbwEsEzpNUEjV4SPwopcpHjpDcJ6srJ5Vs Y7iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xX7SZCGfmXZ9MOjmP6qyYqMKH5CuY/6TOQvAYYPIYYw=; b=iGddGoJENkWL/8GRWbIDxUu5iZRGH2SYLFFy1xApBimNJmI71PglviBs+oB1iktO1u cGh74/7MgXkrX/7nt4L3CA1JKu4JTajiQB4EjSWQxk2fsBZHIeFOyHTE527DbSdU4REp RCTTsHlOqVh5+wB/rs3osQkxNfNzNFxe9rqrJrSu3T+rHN2rXNB/lk5F8wYyQdMrVHxe L06XkdmOeY46tUYMYe2CnrL5VKzzw+NjtdFc8e4w/n7tZ0f2g1NTRNaa7RFj5XSxuSsL qJJpcH/QZsaHu0ESTYeOHYLHl386yMyOHeGLcF3o/jXy7eqNnde6LeYSi1kpGZMkqwWh Dwyw== X-Gm-Message-State: ACrzQf3pU8IC+YhMTIsIW7/8QWcG0wvmbt6bQ0/+XKkgunAkNuj3t9d9 0lSn9C9O4v+VMhCS57M9GrqSRqt80VF+ X-Google-Smtp-Source: AMsMyM4v6d/D/srODISveiVroAryRJvrZrOw7rGp4Zd4Yrdrmph7oorymu4glSMyQYuMTHgI23j+Iqrvgr0P X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:902:d58a:b0:17f:7ed0:233a with SMTP id k10-20020a170902d58a00b0017f7ed0233amr5290016plh.23.1666137068275; Tue, 18 Oct 2022 16:51:08 -0700 (PDT) Date: Tue, 18 Oct 2022 16:50:50 -0700 In-Reply-To: <20221018235051.152548-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221018235051.152548-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221018235051.152548-4-zokeefe@google.com> Subject: [PATCH man-pages v2 3/4] process_madvise.2: CAP_SYS_ADMIN cleanup From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" , Suren Baghdasaryan , Minchan Kim ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666137069; a=rsa-sha256; cv=none; b=drFt5TnCsO6gPOj/0AcSVQzX1aVdIAOnbsIXDGOfOQIQb0Bw4WxkknvcTq9ys7kEPM+0i+ 9JawHhpsuhApjTeMuF3WbXMAaFXqnyc8UvL9JO2SYIX3oTFSicj16IdUzff/km+NehZuTO 7Jkdc7fkjr8j2hzj37b3pJLXG5A5ex4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sHdWx7G9; spf=pass (imf20.hostedemail.com: domain of 37DtPYwcKCME6vrllmlnvvnsl.jvtspu14-ttr2hjr.vyn@flex--zokeefe.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=37DtPYwcKCME6vrllmlnvvnsl.jvtspu14-ttr2hjr.vyn@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666137069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xX7SZCGfmXZ9MOjmP6qyYqMKH5CuY/6TOQvAYYPIYYw=; b=pv9mnBfJIxTW8l5ra0jbJr/bFOCDruyEUz4zxCCkpNM55MV/oGa0gcAYaNaOOT8+g0cSt3 ZFRsoDgbS3FINg6sXXEBFR3TWanQvJuQRw835jNtS6R1nNOTUS2XiP11MAH/ZMv1a02JAG pda6eVY8r2W/kBxAuKATcr+aSNiW+38= X-Rspam-User: Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sHdWx7G9; spf=pass (imf20.hostedemail.com: domain of 37DtPYwcKCME6vrllmlnvvnsl.jvtspu14-ttr2hjr.vyn@flex--zokeefe.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=37DtPYwcKCME6vrllmlnvvnsl.jvtspu14-ttr2hjr.vyn@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: yhqrnknzqq66ntx8qg5xdixuqnmccd14 X-Rspamd-Queue-Id: 4FFDA1C0006 X-Rspamd-Server: rspam10 X-HE-Tag: 1666137069-26790 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe The initial commit of process_madvise(2) to man-pages project included an error, indicating that CAP_SYS_ADMIN capability was required when, in fact, CAP_SYS_NICE was the required capability. The initial commit of process_madvise(2) to Linux, commit ecb8ac8b1f14 ("mm/madvise: introduce process_madvise() syscall: an external memory hinting API"), relied on PTRACE_MODE_ATTACH_FSCREDS (see ptrace(2)), but was amended by commit 96cfe2c0fd23 ("mm/madvise: replace ptrace attach requirement for process_madvise") which replaced this with a combination of PTRACE_MODE_READ and CAP_SYS_NICE (PTRACE_MODE_READ to prevent leaking ASLR metadata and CAP_SYS_NICE for influencing process performance). Correct this in the man-page for process_madvise(2). Fixes: a144f458b ("process_madvise.2: Document process_madvise(2)") Cc: Suren Baghdasaryan Cc: Minchan Kim Signed-off-by: Zach O'Keefe Reviewed-by: Suren Baghdasaryan --- man2/process_madvise.2 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/man2/process_madvise.2 b/man2/process_madvise.2 index 6208206e4..7bee1a098 100644 --- a/man2/process_madvise.2 +++ b/man2/process_madvise.2 @@ -113,7 +113,8 @@ check (see in addition, because of the performance implications of applying the advice, the caller must have the -.B CAP_SYS_ADMIN +.\" commit 96cfe2c0fd23ea7c2368d14f769d287e7ae1082e +.B CAP_SYS_NICE capability. .SH RETURN VALUE On success, From patchwork Tue Oct 18 23:50:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13011210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 331E9C433FE for ; Tue, 18 Oct 2022 23:51:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B328D6B0075; Tue, 18 Oct 2022 19:51:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE2426B0078; Tue, 18 Oct 2022 19:51:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 90CD56B007B; Tue, 18 Oct 2022 19:51:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 82CA66B0075 for ; Tue, 18 Oct 2022 19:51:11 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 564C6C04AF for ; Tue, 18 Oct 2022 23:51:11 +0000 (UTC) X-FDA: 80035718742.04.29CDFFE Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf24.hostedemail.com (Postfix) with ESMTP id EFA9D180039 for ; Tue, 18 Oct 2022 23:51:10 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id ls4-20020a17090b350400b0020d63c0efb9so12654291pjb.0 for ; Tue, 18 Oct 2022 16:51:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=Wa2g3vVRf9EsfQ4YJX+6nAuXP5rzS4A1qWB6UcnBf+U=; b=cahjfRHFbQveHoBBGFhmb9H7O6gdo1yNeIEY7sByptjQK8btNk2zV99SWYIi8+N7WC NUEG5L3MbI2MrIydRkT/evDo+uxczlj0rKW6kfRIhSM05xVfrdDeqQsllbSACG1HXqQZ /RKXTJrT0BntcQ8HHL5VucJYJTWsiEH2eoSo9RX66v4K3IzbZMmtFQvr2dP+qufvV4PD gEocUnJJcrN91bhRDYHGlQGH/7g2NPptyVBjzTSRBLIA1RVBByV1fhGywGOeDDEY/O43 dT5iQHWbqy4TkkSQQ2pevngYdp1enoJ0RncXwCPEZUabBM7vy/qYOFASZjTw9/z8kRiE DhBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=Wa2g3vVRf9EsfQ4YJX+6nAuXP5rzS4A1qWB6UcnBf+U=; b=E6xg3JcmeY5kIilVba1YV3NrnmjvM2j1k4H0VX0m1lHkMAkcPIfKWCt22xTFb3qA26 bBQ8ecr2m4GL3mvW8KONWEd0/afzngMA/FWxWtZ/RN80rZBo9rHU04hQ1drto4ZcBi4D 4TMidtpqC0LpvrlWgLOIhAH9XS5KL7awIUMybrgjmHGj2YU98yUcVtcULDhEsWROIJrv bubqOdNnsORapqGRfmwLdtstAx6PkWP4JwWUsg6q9cHjAMP2cUWQqP3O/TaiQMW76Bj0 qWaYNSBJeWeNcJ6zIJjvyHWJHB+Y7WBG0iA3alcToXmjDP6kkGph58t9uJ7ykn1txytY McuA== X-Gm-Message-State: ACrzQf3YrNR6XaJm0q/eWF/phfR4nRNeLNzc6Yq/eKbWAfabSGSQVxqp J4oFksl+XIM/MsHalcxe8sexq5FW/Fgz X-Google-Smtp-Source: AMsMyM5xVpx73tkBDwDIZ3grYpYv2TxLSc3VjInsqis+R6Uih/QybkKKk5f4nUNXrHpKSyTf1ldljkjjhmTj X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:90a:f284:b0:20d:8287:9342 with SMTP id fs4-20020a17090af28400b0020d82879342mr6438846pjb.25.1666137070059; Tue, 18 Oct 2022 16:51:10 -0700 (PDT) Date: Tue, 18 Oct 2022 16:50:51 -0700 In-Reply-To: <20221018235051.152548-1-zokeefe@google.com> Mime-Version: 1.0 References: <20221018235051.152548-1-zokeefe@google.com> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221018235051.152548-5-zokeefe@google.com> Subject: [PATCH man-pages v2 4/4] madvise.2: add documentation for MADV_COLLAPSE From: Zach OKeefe To: Alejandro Colomar , Michael Kerrisk Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org, "Zach O'Keefe" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666137071; a=rsa-sha256; cv=none; b=bbRmTPP6C+5gjta36yoZtGZP8tPe6zZQW3sH2MurLVd5qeA2CihcxqXROJQwYVaTO6W2tF muytHQ4tpAysulfPSDlHG7hlKqjQdVapWtLU63gnNV4BbbGXoxeqF5m0brCzTVy0oY9E4y 7yMH94+TOwiyuRnQid1mi9x5rHCofY4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=cahjfRHF; spf=pass (imf24.hostedemail.com: domain of 37jtPYwcKCMM8xtnnonpxxpun.lxvurw36-vvt4jlt.x0p@flex--zokeefe.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=37jtPYwcKCMM8xtnnonpxxpun.lxvurw36-vvt4jlt.x0p@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666137071; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wa2g3vVRf9EsfQ4YJX+6nAuXP5rzS4A1qWB6UcnBf+U=; b=uW382lNeSV5y0ozndfL0d0mdv9ERX13jbFyEI7agMvWxPMSFwHAs3y1sWaxEN3HQ2V/t7L OlQ5LeP0gzshUhpAfw6hoG6XEr/u5hFvklpEoIsLPwnQAO21WjHXpiz0plSo6jLVVdfi7H 8KnompyaJMp1fTzH4eT473jW1OZghHE= X-Stat-Signature: bufxrxbwktdu1s3dbmsfnh883twin9qh X-Rspamd-Queue-Id: EFA9D180039 X-Rspam-User: X-Rspamd-Server: rspam03 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=cahjfRHF; spf=pass (imf24.hostedemail.com: domain of 37jtPYwcKCMM8xtnnonpxxpun.lxvurw36-vvt4jlt.x0p@flex--zokeefe.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=37jtPYwcKCMM8xtnnonpxxpun.lxvurw36-vvt4jlt.x0p@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1666137070-803256 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zach O'Keefe Linux 6.1 introduced MADV_COLLAPSE in upstream commit 7d8faaf15545 ("mm/madvise: introduce MADV_COLLAPSE sync hugepage collapse") and upstream commit 34488399fa08 ("mm/madvise: add file and shmem support to MADV_COLLAPSE"). Update the man-pages for madvise(2) and process_madvise(2). Link: https://lore.kernel.org/linux-mm/20220922224046.1143204-1-zokeefe@google.com/ Link: https://lore.kernel.org/linux-mm/20220706235936.2197195-1-zokeefe@google.com/ Signed-off-by: Zach O'Keefe --- man2/madvise.2 | 90 +++++++++++++++++++++++++++++++++++++++++- man2/process_madvise.2 | 10 +++++ 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/man2/madvise.2 b/man2/madvise.2 index df3413cc8..b03fc731d 100644 --- a/man2/madvise.2 +++ b/man2/madvise.2 @@ -385,9 +385,10 @@ set (see .BR prctl (2) ). .IP The -.B MADV_HUGEPAGE +.BR MADV_HUGEPAGE , +.BR MADV_NOHUGEPAGE , and -.B MADV_NOHUGEPAGE +.B MADV_COLLAPSE operations are available only if the kernel was configured with .B CONFIG_TRANSPARENT_HUGEPAGE and file/shmem memory is only supported if the kernel was configured with @@ -400,6 +401,81 @@ and .I length will not be backed by transparent hugepages. .TP +.BR MADV_COLLAPSE " (since Linux 6.1)" +.\" commit 7d8faaf155454f8798ec56404faca29a82689c77 +.\" commit 34488399fa08faaf664743fa54b271eb6f9e1321 +Perform a best-effort synchronous collapse of the native pages mapped by the +memory range into Transparent Huge Pages (THPs). +.B MADV_COLLAPSE +operates on the current state of memory of the calling process and makes no +persistent changes or guarantees on how pages will be mapped, +constructed, +or faulted in the future. +.IP +.B MADV_COLLAPSE +supports private anonymous pages (see +.BR mmap (2)), +shmem pages, +and file-backed pages. +See +.B MADV_HUGEPAGE +for general information on memory requirements for THP. +If the range provided spans multiple VMAs, +the semantics of the collapse over each VMA is independent from the others. +If collapse of a given huge page-aligned/sized region fails, +the operation may continue to attempt collapsing the remainder of the +specified memory. +.B MADV_COLLAPSE +will automatically clamp the provided range to be hugepage-aligned. +.IP +All non-resident pages covered by the range will first be +swapped/faulted-in, +before being copied onto a freshly allocated hugepage. +If the native pages compose the same PTE-mapped hugepage, +and are suitably aligned, +allocation of a new hugepage may be elided and collapse may happen +in-place. +Unmapped pages will have their data directly initialized to 0 in the new +hugepage. +However, +for every eligible hugepage-aligned/sized region to be collapsed, +at least one page must currently be backed by physical memory. +.IP +.BR MADV_COLLAPSE +is independent of any sysfs +(see +.BR sysfs (5)) +setting under +.IR /sys/kernel/mm/transparent_hugepage , +both in terms of determining THP eligibility, +and allocation semantics. +See Linux kernel source file +.I Documentation/admin\-guide/mm/transhuge.rst +for more information. +.BR MADV_COLLAPSE +also ignores +.B huge= +tmpfs mount when operating on tmpfs files. +Allocation for the new hugepage may enter direct reclaim and/or compaction, +regardless of VMA flags +(though +.BR VM_NOHUGEPAGE +is still respected). +.IP +When the system has multiple NUMA nodes, +the hugepage will be allocated from the node providing the most native +pages. +.IP +If all hugepage-sized/aligned regions covered by the provided range were +either successfully collapsed, +or were already PMD-mapped THPs, +this operation will be deemed successful. +Note that this doesn’t guarantee anything about other possible mappings of +the memory. +Also note that many failures might have occurred since the operation may +continue to collapse in the event collapse of a single hugepage-sized/aligned +region fails. +.TP .BR MADV_DONTDUMP " (since Linux 3.4)" .\" commit 909af768e88867016f427264ae39d27a57b6a8ed .\" commit accb61fe7bb0f5c2a4102239e4981650f9048519 @@ -619,6 +695,11 @@ A kernel resource was temporarily unavailable. .B EBADF The map exists, but the area maps something that isn't a file. .TP +.B EBUSY +(for +.BR MADV_COLLAPSE ) +Could not charge hugepage to cgroup: cgroup limit exceeded. +.TP .B EFAULT .I advice is @@ -716,6 +797,11 @@ maximum resident set size. Not enough memory: paging in failed. .TP .B ENOMEM +(for +.BR MADV_COLLAPSE ) +Not enough memory: could not allocate hugepage. +.TP +.B ENOMEM Addresses in the specified range are not currently mapped, or are outside the address space of the process. .TP diff --git a/man2/process_madvise.2 b/man2/process_madvise.2 index 7bee1a098..900210106 100644 --- a/man2/process_madvise.2 +++ b/man2/process_madvise.2 @@ -73,6 +73,10 @@ argument is one of the following values: See .BR madvise (2). .TP +.B MADV_COLLAPSE +See +.BR madvise (2). +.TP .B MADV_PAGEOUT See .BR madvise (2). @@ -170,6 +174,12 @@ The caller does not have permission to access the address space of the process .TP .B ESRCH The target process does not exist (i.e., it has terminated and been waited on). +.PP +See +.BR madvise (2) +for +.IR advice -specific +errors. .SH VERSIONS This system call first appeared in Linux 5.10. .\" commit ecb8ac8b1f146915aa6b96449b66dd48984caacc