From patchwork Mon Apr 4 20:02:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F690C433F5 for ; Mon, 4 Apr 2022 20:03:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E07E36B0073; Mon, 4 Apr 2022 16:03:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB67B8D0001; Mon, 4 Apr 2022 16:03:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C57DA6B0075; Mon, 4 Apr 2022 16:03:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id B7F996B0073 for ; Mon, 4 Apr 2022 16:03:15 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7B40817B8 for ; Mon, 4 Apr 2022 20:03:05 +0000 (UTC) X-FDA: 79320270330.12.8798101 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf08.hostedemail.com (Postfix) with ESMTP id 0382316001D for ; Mon, 4 Apr 2022 20:03:04 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id s11so4635346pla.8 for ; Mon, 04 Apr 2022 13:03:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NCkHW7qdKf6bzLGj3j7v6pAZ5FXathduMO1XmoapD2w=; b=NCYcMbdBRvRn+XRjiGfIxN2tcqtf3m28j6Ss9IzYKdTVClDjf3RZa3dGeoRDcblFz7 lpG/BEu5k3wkLaxvKPQeQaLes4MrBITFco3y2M+Kp347fIot9hIiXvKKmF+dJigCPYG4 B/q/jRSOjGQXfPhlJmn3+5voL7xHCn5jGOPR8dpHxjDGbtAlGbfMNyDprgwmJKVJm4zv PHkwsPH645GSCAosRwop6qgP6Nb5WtrIINCwtfEALkz+5PaTyaRoMErryMtRF30s22oU EDxOpJKcFLVvu9HhhIjSytflYUAOLG73XjN8C5QEmVPurGV9Incx69Dw9oqg2WXRtzQE Rjuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NCkHW7qdKf6bzLGj3j7v6pAZ5FXathduMO1XmoapD2w=; b=D4RHPsFnupWe4Cnb7ITecesd48f1qrYHOq6s5Dl3TzHxRZ2PE/0/6M7o+RJvUv402F PGh2wNewe/ztkE7LPLMkR2JPqJS81yePJFTxnM/6oJNPEncGDNxylyhCbRiLdof5Oj6w RD4pZoO7bXfUy7fDPIHv7DC3zwKsrz6d6nfpDtSvozZqaotceyafeCC37ML18VXdRZPJ ACtyMIBHBvpgy9Ho6F3axC2jmo7IgqT6v5Hndrav9py30chszmjAHWJ5mWM3ABg4l41h CUj/0Ma+Mr6xnk5KYwqIuprPlDMhLMb7y17V4adA2gbvUutm3tuGzICIH0pQdsYfBSBg RfPQ== X-Gm-Message-State: AOAM532iwv3AisKVGZatK41DR67FbeVnnR30X5foLBYOA6ivU1yWqH/S N+uBaxdYpdR4dXhmFbiaKFI= X-Google-Smtp-Source: ABdhPJwwyRU/9G37xXjcgkqkjgM4U7jgrGc/4YgXWVgLfx/3lCnWccRDoLxMFlr3jYl/E2J/XSzSHw== X-Received: by 2002:a17:902:bf07:b0:150:9b8a:a14f with SMTP id bi7-20020a170902bf0700b001509b8aa14fmr1450757plb.127.1649102584016; Mon, 04 Apr 2022 13:03:04 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:03 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 1/8] sched: coredump.h: clarify the use of MMF_VM_HUGEPAGE Date: Mon, 4 Apr 2022 13:02:43 -0700 Message-Id: <20220404200250.321455-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: ftc8kiub6w1kf6qxnpy95gx8im8hdjc8 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NCYcMbdB; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0382316001D X-HE-Tag: 1649102584-292343 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: MMF_VM_HUGEPAGE is set as long as the mm is available for khugepaged by khugepaged_enter(), not only when VM_HUGEPAGE is set on vma. Correct the comment to avoid confusion. Reviewed-by: Miaohe Lin Acked-by: Song Liu Signed-off-by: Yang Shi Acked-by: Vlastmil Babka --- include/linux/sched/coredump.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/coredump.h b/include/linux/sched/coredump.h index 4d9e3a656875..4d0a5be28b70 100644 --- a/include/linux/sched/coredump.h +++ b/include/linux/sched/coredump.h @@ -57,7 +57,8 @@ static inline int get_dumpable(struct mm_struct *mm) #endif /* leave room for more dump flags */ #define MMF_VM_MERGEABLE 16 /* KSM may merge identical pages */ -#define MMF_VM_HUGEPAGE 17 /* set when VM_HUGEPAGE is set on vma */ +#define MMF_VM_HUGEPAGE 17 /* set when mm is available for + khugepaged */ /* * This one-shot flag is dropped due to necessity of changing exe once again * on NFS restore From patchwork Mon Apr 4 20:02:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0D9BC433F5 for ; Mon, 4 Apr 2022 20:04:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 813BA8D0002; Mon, 4 Apr 2022 16:03:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C2D18D0001; Mon, 4 Apr 2022 16:03:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68A948D0002; Mon, 4 Apr 2022 16:03:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id 5A8918D0001 for ; Mon, 4 Apr 2022 16:03:17 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0DE82AD6DA for ; Mon, 4 Apr 2022 20:03:07 +0000 (UTC) X-FDA: 79320270414.25.EE62DA9 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf05.hostedemail.com (Postfix) with ESMTP id 84F4B100036 for ; Mon, 4 Apr 2022 20:03:06 +0000 (UTC) Received: by mail-pl1-f177.google.com with SMTP id p17so9028021plo.9 for ; Mon, 04 Apr 2022 13:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pjgpRMK0NYimSGrt6nJpOa6PTGJdKr7+fAKLKjoWSDA=; b=IECkqfJQKLLWFdbYM3okONxP8OCi2B/qhAroDXHSWfm5bYnfJL/3ruGjY7jjefY0Gl L5WjDq/N9jwBTvcrJCsT4J0fzr2xjvuh8bWlexy6BVuINn3TpgJJBr3tFnwRqKaSbXQk UYJi5I759jd55kj8vaDvPYhYuuRuxJrwRLhlVhmuNcZgQQxWNw03wVSIYlfZopIore1n Tt8mE/Gd9OPxT4FZm62VVyFVklJsp0cXJdXN7OoL2bdskeBCGMfODwfLgV3uYuYcubAa Q/M/j+t5uN2t9sdXHQticavAavTRY2vjlqVVwxtzg9HpDw8PcXGrXbh7M9fdlwtfra5I AU1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pjgpRMK0NYimSGrt6nJpOa6PTGJdKr7+fAKLKjoWSDA=; b=sS8n6ApLaZ8Z8jXBuIExNMLhGpBLCPZmCSFY+PEwTm5Tp5OH+AtUeccyxLjhz6dH+K hGS8YvtV4zsvYY0d2agzVul9+FTXINNMH9OJKW8MrCfrbG5kAlhSGQ/32VHncNCeANnj 5nCPD/+BNUcXURDkofFaVI/TzWjbgPo1nCyJQIQzKKVctrsn0TYUQqPw/XkJEqcbnkUN 0rKemCB0+izWwEGCeWFH4RxPU5X98deS8NzYSZDVsCbpbr2kizNX1Kl23+zr5z5+N8Ed SYvSU0ReypcbiQU8iQg1aOq2Sgmp4hQ0row77JKEGtpKbtVrfWze8AqIQ37RvemLhO4P 1kSw== X-Gm-Message-State: AOAM532zG/KxtfsGgKBrn8D1BGMvCkaX4SdwZnlczmhqQ1aSEz/la9MP ElqLTGQ4nM0nuqYOh5ll39g= X-Google-Smtp-Source: ABdhPJwmqyYOoY4I6R15zAUVTjRG2/sPdiZ1ExIttI9HW/6wIEkHQfrDPIEksQO0dehGXzHTVVqHfA== X-Received: by 2002:a17:90b:1b4f:b0:1c6:d91b:9d0 with SMTP id nv15-20020a17090b1b4f00b001c6d91b09d0mr920843pjb.72.1649102585569; Mon, 04 Apr 2022 13:03:05 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:05 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 2/8] mm: khugepaged: remove redundant check for VM_NO_KHUGEPAGED Date: Mon, 4 Apr 2022 13:02:44 -0700 Message-Id: <20220404200250.321455-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: b3eihwj53wn7urji7p534xt8xcdxqphk X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 84F4B100036 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IECkqfJQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1649102586-57932 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_check() called by khugepaged_enter_vma_merge() does check VM_NO_KHUGEPAGED. Remove the check from caller and move the check in hugepage_vma_check() up. More checks may be run for VM_NO_KHUGEPAGED vmas, but MADV_HUGEPAGE is definitely not a hot path, so cleaner code does outweigh. Reviewed-by: Miaohe Lin Acked-by: Song Liu Signed-off-by: Yang Shi Acked-by: Vlastimil Babka --- mm/khugepaged.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a4e5eaf3eb01..7d197d9e3258 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -365,8 +365,7 @@ int hugepage_madvise(struct vm_area_struct *vma, * register it here without waiting a page fault that * may not happen any time soon. */ - if (!(*vm_flags & VM_NO_KHUGEPAGED) && - khugepaged_enter_vma_merge(vma, *vm_flags)) + if (khugepaged_enter_vma_merge(vma, *vm_flags)) return -ENOMEM; break; case MADV_NOHUGEPAGE: @@ -445,6 +444,9 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, if (!transhuge_vma_enabled(vma, vm_flags)) return false; + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) return false; @@ -470,7 +472,8 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, return false; if (vma_is_temporary_stack(vma)) return false; - return !(vm_flags & VM_NO_KHUGEPAGED); + + return true; } int __khugepaged_enter(struct mm_struct *mm) From patchwork Mon Apr 4 20:02:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DFF7C433F5 for ; Mon, 4 Apr 2022 20:04:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26EE18D0003; Mon, 4 Apr 2022 16:03:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 21D3D8D0001; Mon, 4 Apr 2022 16:03:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E5588D0003; Mon, 4 Apr 2022 16:03:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 01AF48D0001 for ; Mon, 4 Apr 2022 16:03:18 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id B4B7312040A for ; Mon, 4 Apr 2022 20:03:08 +0000 (UTC) X-FDA: 79320270456.15.C31938D Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf11.hostedemail.com (Postfix) with ESMTP id 2717240020 for ; Mon, 4 Apr 2022 20:03:07 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id ku13-20020a17090b218d00b001ca8fcd3adeso358588pjb.2 for ; Mon, 04 Apr 2022 13:03:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5Byfy9VbIfRqB/lMq9E6coezN6yPt+vkfM62rVKg8Ao=; b=AxElP8EkjsWu9uFPszBvkLOkxsuJWaGma6j/kzg1J45uxCQnKl53MKPLXM3bzLcB+I UwsfPob2VOt2V0XPQHOnu7TAxvVCTNSE6KuScOlC36TWg6HJUDLUHa3UKvMJ9tgVEMxM 9mCufQX3uNL8r407B9+wD7q4K36WEcTb94BSQY1zv4p3JQmfLZ+CfjlQOg0dLcMIL1xq u/37RX8brDJ70ogZGKbqWohxUI4JqurrzTqjcOhQQuCP0Ts7lmMczbesU4zwZch+8Nfe YJib5ZcU28w7VIwuyZk4D/d6e8f3uSR1N1molGoJqNfbFIIpiMh3azZJOiAIyidZztZT MxGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5Byfy9VbIfRqB/lMq9E6coezN6yPt+vkfM62rVKg8Ao=; b=mswXftcYvzedKeSYNsPcubCKwS/OKkcoWV9hhANBMt4eFrPgcwrt/Fyx6HbptoN0Ou aRapgc6rlPHLbbJsFjN8PwxLUTY/UfKntJUj42OXavkjwUjz9GD/eT8zki+vHBBAoHKo dqJq86khlk2RdtwRr4GM+llVNs/mNyzpupwtMEfEF1WpqA7PTkmWFWpcKMPisdoHsS2M MUhomn0xxALYLI0IuMFxBuCV5qATEC5aITWSBiG91MsVoPJBGkFqhJeQhkoczinDuBQy ms1QM1hrvj0jPASom5+Zx0c6ayHouRByl2pxc7zFtdYfOcyIwobn1m9rlMWjk2LWFupq /LBA== X-Gm-Message-State: AOAM533uYyVX2yqTlKgjBf7UCvgwHawukgyhSyaNDR1Zz9mcHSaz6W5N CxdLPIpXq39GlmM2CKldMFw= X-Google-Smtp-Source: ABdhPJwDGZ1eLAnRqgsHcj4iCzKkBHl8n8iAS//8VpbDXoJE+obiPqKFFGMKejpUxqvN5w0R05aYZA== X-Received: by 2002:a17:90a:888:b0:1ca:a9ac:c866 with SMTP id v8-20020a17090a088800b001caa9acc866mr857483pjc.203.1649102587116; Mon, 04 Apr 2022 13:03:07 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:06 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 3/8] mm: khugepaged: skip DAX vma Date: Mon, 4 Apr 2022 13:02:45 -0700 Message-Id: <20220404200250.321455-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AxElP8Ek; spf=pass (imf11.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.41 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: 13dk5e464u3mghfwa435nnwnujqfng5n X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2717240020 X-HE-Tag: 1649102587-680747 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The DAX vma may be seen by khugepaged when the mm has other khugepaged suitable vmas. So khugepaged may try to collapse THP for DAX vma, but it will fail due to page sanity check, for example, page is not on LRU. So it is not harmful, but it is definitely pointless to run khugepaged against DAX vma, so skip it in early check. Reviewed-by: Miaohe Lin Acked-by: Song Liu Signed-off-by: Yang Shi Acked-by: Vlastimil Babka --- mm/khugepaged.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 7d197d9e3258..964a4d2c942a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -447,6 +447,10 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, if (vm_flags & VM_NO_KHUGEPAGED) return false; + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) return false; From patchwork Mon Apr 4 20:02:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C86EC433F5 for ; Mon, 4 Apr 2022 20:05:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7ECA68D0005; Mon, 4 Apr 2022 16:03:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 79D308D0001; Mon, 4 Apr 2022 16:03:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63F288D0005; Mon, 4 Apr 2022 16:03:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 573EB8D0001 for ; Mon, 4 Apr 2022 16:03:20 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 25647120F81 for ; Mon, 4 Apr 2022 20:03:10 +0000 (UTC) X-FDA: 79320270540.03.16173FA Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf07.hostedemail.com (Postfix) with ESMTP id A413740017 for ; Mon, 4 Apr 2022 20:03:09 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id z16so10016563pfh.3 for ; Mon, 04 Apr 2022 13:03:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BXLHEFwxWdm5e2nqHirZ7xmXS6hF3uNKov0CWo+0WO8=; b=YHcz6zZf1ucCYQxs+BMo39i9t7OYROE0FpXZIgsWDaCsFsP9IDsqC4+3466mFNuIoG XKIxMX/EeLaGaDSkKXYDFXgr8HlDPlMuePpgvGfBZZzWdJZVhx0N+ih7KKqf5szrrHnG U2wo/VL4ygXkXvcglMVPrPHRDkifd17EqbT23AB19ccgs8/Bx2WixuZmCXKcJXjImMCi 9pRUoe3dQlpums5on1sx94iJXwq3mFhiR9L5ujIOE0fwiTJFVJbx9Nq/TPSCPK9/I2yE laPfy1lMXdXzUfgdglhd6cCi38CCqOWi0XfaDResSt6TIo2znuOjhuUfNN/4SJFyrfBA f7AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BXLHEFwxWdm5e2nqHirZ7xmXS6hF3uNKov0CWo+0WO8=; b=hB32OgnwLrhU9G6/jNRZ420bbCuRzyuzauuhW7YB/R/yo22PlSQGgQF9slaxETDihz rdilzDCMdNNG+o54S9DAuInlx63SmByA+c726chd8rQ4hoaVmUU65ybeLkAMlQR9RLgJ 52LNiQA5D4M4QII7uI6lcNI2CriHnbJXrusQVNntLZzKNJO19ADU4H5xcBYc7SeIZzYn s2uJMzFq4JojhxWVXqwXTYN9Wm/t6QOgeAsODMwD5yXDLcIuZrcLYYXXfYD6JTI5I/Bb daEWCmLJPEd5edl8PRpVefY/8e4rsCuaEIXBtw0xCpx45xzlA8MuTAJEASyqDF1C+ktD pzpQ== X-Gm-Message-State: AOAM532o/RoTQvakIbPA4xj/9diAnexhVhHKpb2HprkxJph3Pk56pFQV rx8iIb6B/IEhR2499dP7blk= X-Google-Smtp-Source: ABdhPJyagGXZMLfLFwAcYGG46fuX5fs53R3Ia/YY5p1GvsQ1lu5ZQE6+/01O8gqaGg7avP+BtQr8dA== X-Received: by 2002:a63:2ad0:0:b0:398:31d7:9955 with SMTP id q199-20020a632ad0000000b0039831d79955mr1273444pgq.198.1649102588645; Mon, 04 Apr 2022 13:03:08 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:08 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 4/8] mm: thp: only regular file could be THP eligible Date: Mon, 4 Apr 2022 13:02:46 -0700 Message-Id: <20220404200250.321455-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A413740017 X-Stat-Signature: 8db4xsng3y95tsmu4zs4gxrqybcao4fc Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YHcz6zZf; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1649102589-61433 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since commit a4aeaa06d45e ("mm: khugepaged: skip huge page collapse for special files"), khugepaged just collapses THP for regular file which is the intended usecase for readonly fs THP. Only show regular file as THP eligible accordingly. And make file_thp_enabled() available for khugepaged too in order to remove duplicate code. Acked-by: Song Liu Signed-off-by: Yang Shi Acked-by: Vlastimil Babka --- include/linux/huge_mm.h | 14 ++++++++++++++ mm/huge_memory.c | 11 ++--------- mm/khugepaged.c | 9 ++------- 3 files changed, 18 insertions(+), 16 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2999190adc22..62a6f667850d 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -172,6 +172,20 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } +static inline bool file_thp_enabled(struct vm_area_struct *vma) +{ + struct inode *inode; + + if (!vma->vm_file) + return false; + + inode = vma->vm_file->f_inode; + + return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && + (vma->vm_flags & VM_EXEC) && + !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); +} + bool transparent_hugepage_active(struct vm_area_struct *vma); #define transparent_hugepage_use_zero_page() \ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2fe38212e07c..183b793fd28e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -68,13 +68,6 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -static inline bool file_thp_enabled(struct vm_area_struct *vma) -{ - return transhuge_vma_enabled(vma, vma->vm_flags) && vma->vm_file && - !inode_is_open_for_write(vma->vm_file->f_inode) && - (vma->vm_flags & VM_EXEC); -} - bool transparent_hugepage_active(struct vm_area_struct *vma) { /* The addr is used to check if the vma size fits */ @@ -86,8 +79,8 @@ bool transparent_hugepage_active(struct vm_area_struct *vma) return __transparent_hugepage_enabled(vma); if (vma_is_shmem(vma)) return shmem_huge_enabled(vma); - if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) - return file_thp_enabled(vma); + if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + return true; return false; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 964a4d2c942a..609c1bc0a027 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -464,13 +464,8 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Only regular file is valid */ - if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file && - (vm_flags & VM_EXEC)) { - struct inode *inode = vma->vm_file->f_inode; - - return !inode_is_open_for_write(inode) && - S_ISREG(inode->i_mode); - } + if (file_thp_enabled(vma)) + return true; if (!vma->anon_vma || vma->vm_ops) return false; From patchwork Mon Apr 4 20:02:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2468CC433F5 for ; Mon, 4 Apr 2022 20:05:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 295C98D0006; Mon, 4 Apr 2022 16:03:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 245AB8D0001; Mon, 4 Apr 2022 16:03:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10C348D0006; Mon, 4 Apr 2022 16:03:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 042848D0001 for ; Mon, 4 Apr 2022 16:03:22 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id C0AE0611B5 for ; Mon, 4 Apr 2022 20:03:11 +0000 (UTC) X-FDA: 79320270582.07.6FF1909 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf02.hostedemail.com (Postfix) with ESMTP id 3568080047 for ; Mon, 4 Apr 2022 20:03:11 +0000 (UTC) Received: by mail-pl1-f175.google.com with SMTP id y6so9059239plg.2 for ; Mon, 04 Apr 2022 13:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s/lBuFlDBU1GRGFU9YRcAqT+yGPT0++0CD70PJQOVug=; b=jV2+yjJwu1XGCS/IRhjcd4DZs/IJskOYfgOFxua1OeyC9E1XiBQ1wmird8bypPrcZd MoGBN5NQmHruIHbraHzI416gKP9u1Rk/wCKqGnIARi5LArpYMbo9qrGE7m+wZIrMYgB7 sQCOXxRoKp6iGqfMZBH/dbXVaI7kIT84aRYJK1SkaC2+vsBgI3iSqVcH8CfF7R8x9LgO WnRl0woQcCgIkuorfpvG3+oj8JjHG/unaFUIs1HTA9jTUoUOhNUCfrzFgi9TP0Lbt06K Ut85ccYdHygANYMcQX+k3Qz+za4rSK9z6yduraH5Uf2d5HYFHkA6SkVP4GD+n5IbvCPJ UC2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s/lBuFlDBU1GRGFU9YRcAqT+yGPT0++0CD70PJQOVug=; b=NGAX16m74F6PSXievTjcZDylFY+39ZLtAedyG5zySHaBnmeUQtxeoVtAWtFwfdZuKn P7pQfOBPm97pUVNnqPGmRPzcjiJtUCF8AUPi+RWmdsTfvDTptfxMr4CDk1RJQ7SLY9IV 4vMfbrUXZqmK2Tzru6Og94LFEHwKFp1wBFDq225Z1uKl5IM9kW0ibQlfLb8WyZHmYIgd L61JAz44BmJSq1AgibP+s7kJir8he+63TCLs5SfKO83W8POYrvhQM8P5oEIAbkq38oZ2 KQ5yXM0abmAK9mwZA7QVsDgfAX743aTBmE/peCD3i/sWHWGnlAOAUbQ5ztFhqM71XK6s 6Cjw== X-Gm-Message-State: AOAM530O0HPKoVII2PtuhlYWXcamy9PxVq3m98s7I22kzQsaI0jTmBb0 GI152lcBFPMnvPvfyOVNs6k= X-Google-Smtp-Source: ABdhPJzg9Vfz5QEnjinv2EOkeMGgWfDWMGTZSI+KOlHEC65AKTLMoaowQLItcjuPWMR/DDhthX79vA== X-Received: by 2002:a17:902:7245:b0:156:20aa:1534 with SMTP id c5-20020a170902724500b0015620aa1534mr1673619pll.94.1649102590218; Mon, 04 Apr 2022 13:03:10 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:09 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 5/8] mm: khugepaged: make khugepaged_enter() void function Date: Mon, 4 Apr 2022 13:02:47 -0700 Message-Id: <20220404200250.321455-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: xy3w4yat4obai4nbnm1asofw3qu61kek X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 3568080047 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=jV2+yjJw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1649102591-850864 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The most callers of khugepaged_enter() don't care about the return value. Only dup_mmap(), anonymous THP page fault and MADV_HUGEPAGE handle the error by returning -ENOMEM. Actually it is not harmful for them to ignore the error case either. It also sounds overkilling to fail fork() and page fault early due to khugepaged_enter() error, and MADV_HUGEPAGE does set VM_HUGEPAGE flag regardless of the error. Acked-by: Song Liu Signed-off-by: Yang Shi Acked-by: Vlastimil Babka --- include/linux/khugepaged.h | 30 ++++++++++++------------------ kernel/fork.c | 4 +--- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++----------- 4 files changed, 22 insertions(+), 34 deletions(-) diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 2fcc01891b47..0423d3619f26 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -12,10 +12,10 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern int __khugepaged_enter(struct mm_struct *mm); +extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); -extern int khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags); +extern void khugepaged_enter_vma_merge(struct vm_area_struct *vma, + unsigned long vm_flags); extern void khugepaged_min_free_kbytes_update(void); #ifdef CONFIG_SHMEM extern void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr); @@ -40,11 +40,10 @@ static inline void collapse_pte_mapped_thp(struct mm_struct *mm, (transparent_hugepage_flags & \ (1<flags)) - return __khugepaged_enter(mm); - return 0; + __khugepaged_enter(mm); } static inline void khugepaged_exit(struct mm_struct *mm) @@ -53,7 +52,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) __khugepaged_exit(mm); } -static inline int khugepaged_enter(struct vm_area_struct *vma, +static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) @@ -62,27 +61,22 @@ static inline int khugepaged_enter(struct vm_area_struct *vma, (khugepaged_req_madv() && (vm_flags & VM_HUGEPAGE))) && !(vm_flags & VM_NOHUGEPAGE) && !test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - if (__khugepaged_enter(vma->vm_mm)) - return -ENOMEM; - return 0; + __khugepaged_enter(vma->vm_mm); } #else /* CONFIG_TRANSPARENT_HUGEPAGE */ -static inline int khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) +static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { - return 0; } static inline void khugepaged_exit(struct mm_struct *mm) { } -static inline int khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) +static inline void khugepaged_enter(struct vm_area_struct *vma, + unsigned long vm_flags) { - return 0; } -static inline int khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags) +static inline void khugepaged_enter_vma_merge(struct vm_area_struct *vma, + unsigned long vm_flags) { - return 0; } static inline void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) diff --git a/kernel/fork.c b/kernel/fork.c index 9796897560ab..0d13baf86650 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -612,9 +612,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm, retval = ksm_fork(mm, oldmm); if (retval) goto out; - retval = khugepaged_fork(mm, oldmm); - if (retval) - goto out; + khugepaged_fork(mm, oldmm); prev = NULL; for (mpnt = oldmm->mmap; mpnt; mpnt = mpnt->vm_next) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 183b793fd28e..4fd5a6a79d44 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -725,8 +725,8 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return VM_FAULT_FALLBACK; if (unlikely(anon_vma_prepare(vma))) return VM_FAULT_OOM; - if (unlikely(khugepaged_enter(vma, vma->vm_flags))) - return VM_FAULT_OOM; + khugepaged_enter(vma, vma->vm_flags); + if (!(vmf->flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(vma->vm_mm) && transparent_hugepage_use_zero_page()) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 609c1bc0a027..b69eda934d70 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -365,8 +365,7 @@ int hugepage_madvise(struct vm_area_struct *vma, * register it here without waiting a page fault that * may not happen any time soon. */ - if (khugepaged_enter_vma_merge(vma, *vm_flags)) - return -ENOMEM; + khugepaged_enter_vma_merge(vma, *vm_flags); break; case MADV_NOHUGEPAGE: *vm_flags &= ~VM_HUGEPAGE; @@ -475,20 +474,20 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, return true; } -int __khugepaged_enter(struct mm_struct *mm) +void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; int wakeup; mm_slot = alloc_mm_slot(); if (!mm_slot) - return -ENOMEM; + return; /* __khugepaged_exit() must not run from under us */ VM_BUG_ON_MM(khugepaged_test_exit(mm), mm); if (unlikely(test_and_set_bit(MMF_VM_HUGEPAGE, &mm->flags))) { free_mm_slot(mm_slot); - return 0; + return; } spin_lock(&khugepaged_mm_lock); @@ -504,11 +503,9 @@ int __khugepaged_enter(struct mm_struct *mm) mmgrab(mm); if (wakeup) wake_up_interruptible(&khugepaged_wait); - - return 0; } -int khugepaged_enter_vma_merge(struct vm_area_struct *vma, +void khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags) { unsigned long hstart, hend; @@ -519,13 +516,12 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma, * file-private shmem THP is not supported. */ if (!hugepage_vma_check(vma, vm_flags)) - return 0; + return; hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; hend = vma->vm_end & HPAGE_PMD_MASK; if (hstart < hend) - return khugepaged_enter(vma, vm_flags); - return 0; + khugepaged_enter(vma, vm_flags); } void __khugepaged_exit(struct mm_struct *mm) From patchwork Mon Apr 4 20:02:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A08DDC433F5 for ; Mon, 4 Apr 2022 20:06:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C1E18D0007; Mon, 4 Apr 2022 16:03:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 971638D0001; Mon, 4 Apr 2022 16:03:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 811558D0007; Mon, 4 Apr 2022 16:03:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 74ADF8D0001 for ; Mon, 4 Apr 2022 16:03:23 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 53E8F24529 for ; Mon, 4 Apr 2022 20:03:13 +0000 (UTC) X-FDA: 79320270666.10.7A8B711 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf25.hostedemail.com (Postfix) with ESMTP id C0FEFA0006 for ; Mon, 4 Apr 2022 20:03:12 +0000 (UTC) Received: by mail-pf1-f174.google.com with SMTP id x31so9969069pfh.9 for ; Mon, 04 Apr 2022 13:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=299dhLN8fZQaqsT9OGmphjyBEYT6w3EQIbst3RvXrxI=; b=pNHjfPkGU3nk4TyLoHEURVpFplmDQJP/QKBA1MTf9iPA+5O96M5KmGbjPNHgH3kGdo PB8MP/4BFQMW8iZUT0rXAvH6ICgHnflXm0yZiIGUM7ObvsZjlYZ1kOzxdSYTQ8oKUlcD cyxjX9o+hs4GHpbvU7N2kQhZtmPKCBsWlLx0CuJIf+mTLae52Gt2eRcr7K83q0NUB28Y 5ztyPyaLXwKf5AcFHDFb/wXTRVMhRIA5p7kNnRPfO85xZkwHO/SKyosZrLqhwLQsDT8W hNrKMR2Ofm3nfe4E6BIpLMjFYbX0PQryTsa8GkPDukq569Dkfw3xpXh7ltx8WWQNiRp9 ND+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=299dhLN8fZQaqsT9OGmphjyBEYT6w3EQIbst3RvXrxI=; b=OntTUST6SeyzD1lSNzBfOliar2Z7sb+22pTDoOsvl/t22h3haD/n0BOyuLTeRGmZHR TnGh5+IFGapQGK78LhUyGSgjmr+ZCoR+AYi2B37i29wNUL/KsSTSznYGbftMKkW0vnLj BRIklQbxpS6mEmUd63i9QO+24x5hEKeiipkqBL9vUn8uzVAv2Htvc7tyPhhOubhrWkXT DT9eX8dzRL6iLg+IHgfy2IMCAx9ULeZJiSJAzcjyr/UHNvFuoW/rnZYjso3iDYGhNWkK dCwSA5HzAZ2MuH21bV5/+ixlkCImq9Unlo+KYIK1D1NLMAX6rMZM/bI/PBmZOgeFMOTe 5o5A== X-Gm-Message-State: AOAM533KH9nDY/RppDYolsOMD4NKxvwuri62BrY8jGk3BPg+0diP4KVl c9yEzJmwtaeJ5+qIjsfI71Y= X-Google-Smtp-Source: ABdhPJznJU9yE4ofpoTqB4D8WgSLsYOq7fqnscvbG/DJjVa+BvOJgo1PB6ZgI+8pMnraeasjO3kvtg== X-Received: by 2002:a63:4a5d:0:b0:399:5cd:1589 with SMTP id j29-20020a634a5d000000b0039905cd1589mr1293015pgl.22.1649102591800; Mon, 04 Apr 2022 13:03:11 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:11 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 6/8] mm: khugepaged: move some khugepaged_* functions to khugepaged.c Date: Mon, 4 Apr 2022 13:02:48 -0700 Message-Id: <20220404200250.321455-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: oo87c3z4apz5fncg5iu5od9ga3erwgtd Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=pNHjfPkG; spf=pass (imf25.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C0FEFA0006 X-HE-Tag: 1649102592-546110 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To reuse hugepage_vma_check() for khugepaged_enter() so that we could remove some duplicate code. But moving hugepage_vma_check() to khugepaged.h needs to include huge_mm.h in it, it seems not optimal to bloat khugepaged.h. And the khugepaged_* functions actually are wrappers for some non-inline functions, so it seems the benefits are not too much to keep them inline. So move the khugepaged_* functions to khugepaged.c, any callers just need to include khugepaged.h which is quite small. For example, the following patches will call khugepaged_enter() in filemap page fault path for regular filesystems to make readonly FS THP collapse more consistent. The filemap.c just needs to include khugepaged.h. Acked-by: Song Liu Signed-off-by: Yang Shi --- include/linux/khugepaged.h | 37 ++++++------------------------------- mm/khugepaged.c | 20 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 31 deletions(-) diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 0423d3619f26..6acf9701151e 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -2,10 +2,6 @@ #ifndef _LINUX_KHUGEPAGED_H #define _LINUX_KHUGEPAGED_H -#include /* MMF_VM_HUGEPAGE */ -#include - - #ifdef CONFIG_TRANSPARENT_HUGEPAGE extern struct attribute_group khugepaged_attr_group; @@ -16,6 +12,12 @@ extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags); +extern void khugepaged_fork(struct mm_struct *mm, + struct mm_struct *oldmm); +extern void khugepaged_exit(struct mm_struct *mm); +extern void khugepaged_enter(struct vm_area_struct *vma, + unsigned long vm_flags); + extern void khugepaged_min_free_kbytes_update(void); #ifdef CONFIG_SHMEM extern void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr); @@ -33,36 +35,9 @@ static inline void collapse_pte_mapped_thp(struct mm_struct *mm, #define khugepaged_always() \ (transparent_hugepage_flags & \ (1<flags)) - __khugepaged_enter(mm); -} - -static inline void khugepaged_exit(struct mm_struct *mm) -{ - if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) - __khugepaged_exit(mm); -} - -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) - if ((khugepaged_always() || - (shmem_file(vma->vm_file) && shmem_huge_enabled(vma)) || - (khugepaged_req_madv() && (vm_flags & VM_HUGEPAGE))) && - !(vm_flags & VM_NOHUGEPAGE) && - !test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - __khugepaged_enter(vma->vm_mm); -} #else /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b69eda934d70..ec5b0a691d87 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -556,6 +556,26 @@ void __khugepaged_exit(struct mm_struct *mm) } } +void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) +{ + if (test_bit(MMF_VM_HUGEPAGE, &oldmm->flags)) + __khugepaged_enter(mm); +} + +void khugepaged_exit(struct mm_struct *mm) +{ + if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) + __khugepaged_exit(mm); +} + +void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) +{ + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && + khugepaged_enabled()) + if (hugepage_vma_check(vma, vm_flags)) + __khugepaged_enter(vma->vm_mm); +} + static void release_pte_page(struct page *page) { mod_node_page_state(page_pgdat(page), From patchwork Mon Apr 4 20:02:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FD2FC433EF for ; Mon, 4 Apr 2022 20:06:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 446AF8D0008; Mon, 4 Apr 2022 16:03:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F2108D0001; Mon, 4 Apr 2022 16:03:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 294828D0008; Mon, 4 Apr 2022 16:03:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 1CC108D0001 for ; Mon, 4 Apr 2022 16:03:25 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E6E65F7A for ; Mon, 4 Apr 2022 20:03:14 +0000 (UTC) X-FDA: 79320270708.01.4D18EEF Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf06.hostedemail.com (Postfix) with ESMTP id 7631418001F for ; Mon, 4 Apr 2022 20:03:14 +0000 (UTC) Received: by mail-pf1-f179.google.com with SMTP id w7so9961022pfu.11 for ; Mon, 04 Apr 2022 13:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FgKTSjMxp3AdwYz23ZAB1EJPIeK5nseej60E5OuWwX0=; b=Q1A3YlfgEbe8lLn0skK/++u9FqKcXEy32OnnDrIyJYpqURGO1hpoUykNLgw4agvXSP w7vqfhvAjNEbyICxsSt9wuLLGSKFFs3irFnCLN/eHA4nlKtz0V5Dx6+p0BEi2BgLwidF NcWLIMZ/iWj2y4M5XC7JxDZH8spQXT8QJxCnpJqOf/4O8O6tMAzWhbuCZUpVpZOb2AGM cyW4y0XBUDwtngu5qmWA1Dw5YE0H1fYD0uDhRHtnxT3mzcYt83eU9v7v5TUzahx3OeHw ejbXEJ3J5u3MzCi/ia0iLBqINzXhAiPh0SJk6/sg5j+I0pETLHMgbcd7VVpAzhwAuCFX pvvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FgKTSjMxp3AdwYz23ZAB1EJPIeK5nseej60E5OuWwX0=; b=b+m618jMpCz6sh/6Vw2TmNYlo+xWrodFHY1PNZJUZUEfXGpUZSiOypDGFhsRuVel0h /rS3DAx6p307O6lQLvh85bJa+YYhn9nkC/hMOw9d04fOyf7aIzn3YbrFIDyNSDmIkOsU pLcNZDRz54WYxPoGpE3FhgAsvS9In05LSTVGXu7CAk5aIQZHrLU2GhxY/oZvqV+ztVUd NGjhqy8ykeeQGOC6nwUxM9O9d78Ik59CRof2vXrS7Z7cBEZEfH5XJsnkmVvcCSRoJjjs V9cb6Tx/jM7YHwtvhXB1xj3ezZhqL0/IqfqlHa+lWdVpXrEDUDrE+nU2uIxp95fBMvqm 5vVg== X-Gm-Message-State: AOAM5330NCQ9md4AfGHBcJ1Xs0cWNyfAS/SAl9HSUG3WH1oWSClxh2KS 3AZ5Jlgfhkih41lf+Fw+z58= X-Google-Smtp-Source: ABdhPJznqnP9rpTwiXQKLYVOrJeLRb7oT1LQF5qT/hkPokfJzaSck749CHWu+2vv9VToyaEKUJAX/w== X-Received: by 2002:a63:8c2:0:b0:380:bfd8:9e10 with SMTP id 185-20020a6308c2000000b00380bfd89e10mr1275436pgi.422.1649102593484; Mon, 04 Apr 2022 13:03:13 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:13 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 7/8] mm: khugepaged: introduce khugepaged_enter_vma() helper Date: Mon, 4 Apr 2022 13:02:49 -0700 Message-Id: <20220404200250.321455-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7631418001F X-Stat-Signature: u6y4n7qqpmo7aagopio8fo3nepf3oecm Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Q1A3Ylfg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1649102594-834476 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The khugepaged_enter_vma_merge() actually does as the same thing as the khugepaged_enter() section called by shmem_mmap(), so consolidate them into one helper and rename it to khugepaged_enter_vma(). Signed-off-by: Yang Shi Acked-by: Vlastimil Babka --- include/linux/khugepaged.h | 8 ++++---- mm/khugepaged.c | 26 +++++++++----------------- mm/mmap.c | 8 ++++---- mm/shmem.c | 12 ++---------- 4 files changed, 19 insertions(+), 35 deletions(-) diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 6acf9701151e..f4b12be155ab 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,8 @@ extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); -extern void khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags); +extern void khugepaged_enter_vma(struct vm_area_struct *vma, + unsigned long vm_flags); extern void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm); extern void khugepaged_exit(struct mm_struct *mm); @@ -49,8 +49,8 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { } -static inline void khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags) +static inline void khugepaged_enter_vma(struct vm_area_struct *vma, + unsigned long vm_flags) { } static inline void collapse_pte_mapped_thp(struct mm_struct *mm, diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ec5b0a691d87..c5c3202d7401 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -365,7 +365,7 @@ int hugepage_madvise(struct vm_area_struct *vma, * register it here without waiting a page fault that * may not happen any time soon. */ - khugepaged_enter_vma_merge(vma, *vm_flags); + khugepaged_enter_vma(vma, *vm_flags); break; case MADV_NOHUGEPAGE: *vm_flags &= ~VM_HUGEPAGE; @@ -505,23 +505,15 @@ void __khugepaged_enter(struct mm_struct *mm) wake_up_interruptible(&khugepaged_wait); } -void khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags) +void khugepaged_enter_vma(struct vm_area_struct *vma, + unsigned long vm_flags) { - unsigned long hstart, hend; - - /* - * khugepaged only supports read-only files for non-shmem files. - * khugepaged does not yet work on special mappings. And - * file-private shmem THP is not supported. - */ - if (!hugepage_vma_check(vma, vm_flags)) - return; - - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart < hend) - khugepaged_enter(vma, vm_flags); + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && + khugepaged_enabled() && + (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < + (vma->vm_end & HPAGE_PMD_MASK))) + if (hugepage_vma_check(vma, vm_flags)) + __khugepaged_enter(vma->vm_mm); } void __khugepaged_exit(struct mm_struct *mm) diff --git a/mm/mmap.c b/mm/mmap.c index 3aa839f81e63..604c8dece5dd 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1218,7 +1218,7 @@ struct vm_area_struct *vma_merge(struct mm_struct *mm, end, prev->vm_pgoff, NULL, prev); if (err) return NULL; - khugepaged_enter_vma_merge(prev, vm_flags); + khugepaged_enter_vma(prev, vm_flags); return prev; } @@ -1245,7 +1245,7 @@ struct vm_area_struct *vma_merge(struct mm_struct *mm, } if (err) return NULL; - khugepaged_enter_vma_merge(area, vm_flags); + khugepaged_enter_vma(area, vm_flags); return area; } @@ -2460,7 +2460,7 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address) } } anon_vma_unlock_write(vma->anon_vma); - khugepaged_enter_vma_merge(vma, vma->vm_flags); + khugepaged_enter_vma(vma, vma->vm_flags); validate_mm(mm); return error; } @@ -2538,7 +2538,7 @@ int expand_downwards(struct vm_area_struct *vma, } } anon_vma_unlock_write(vma->anon_vma); - khugepaged_enter_vma_merge(vma, vma->vm_flags); + khugepaged_enter_vma(vma, vma->vm_flags); validate_mm(mm); return error; } diff --git a/mm/shmem.c b/mm/shmem.c index 529c9ad3e926..92eca974771d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2239,11 +2239,7 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma) file_accessed(file); vma->vm_ops = &shmem_vm_ops; - if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - ((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK)) { - khugepaged_enter(vma, vma->vm_flags); - } + khugepaged_enter_vma(vma, vma->vm_flags); return 0; } @@ -4136,11 +4132,7 @@ int shmem_zero_setup(struct vm_area_struct *vma) vma->vm_file = file; vma->vm_ops = &shmem_vm_ops; - if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - ((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK)) { - khugepaged_enter(vma, vma->vm_flags); - } + khugepaged_enter_vma(vma, vma->vm_flags); return 0; } From patchwork Mon Apr 4 20:02:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12800669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BF74C433F5 for ; Mon, 4 Apr 2022 20:07:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BEBF8D0009; Mon, 4 Apr 2022 16:03:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2449C8D0001; Mon, 4 Apr 2022 16:03:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E5B88D0009; Mon, 4 Apr 2022 16:03:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id F42148D0001 for ; Mon, 4 Apr 2022 16:03:26 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id AB843812484D for ; Mon, 4 Apr 2022 20:03:16 +0000 (UTC) X-FDA: 79320270792.24.425AAC7 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf26.hostedemail.com (Postfix) with ESMTP id 1C92214000C for ; Mon, 4 Apr 2022 20:03:15 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id h24so5658435pfo.6 for ; Mon, 04 Apr 2022 13:03:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UG1sy1iNDRLB0uN0cB6h3KgifvWMYW6S1PnONIaP3Qs=; b=bnyhwFA2JenCWTOWzQqWYVxa9U6bgYLgMEzDgfNWHGyZsV/BIcrzi093QQp94ckdhr Uw8gXhPnm8EuKl/5kb5GqXeC3lGRqiOLu9IvMiBPkwR4H0Ox20Wgtkdm7WK4E0FFoW7p wBwJ4LVJJbbzN5VkYE7c1wBpW+agLiIyWjn4uBQfG9gXmt4xRAypdwfzxiB5ZaojQqk2 miEKZA17rtUekHkwxJf4j57ZKUr/cvVqaMRt1mi5xuwpogTpPjzRWKaLpFWJfAafCj5H izyLOdxXzfD3wtidP2aNkn5eO5R5SVaFkrQE9Hxs7MyYzSHQ6jXu89Vin4glfydzoCMV 9u6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UG1sy1iNDRLB0uN0cB6h3KgifvWMYW6S1PnONIaP3Qs=; b=aqGjYltwPrOLPXhbODrFbiZFoOecgAeAm6IJEdIm1To3Z+fyv8KZO4oxsFhsytJZrR ORWbRK4zjwdlGcSnWihJ4SrDxdXDW5QQikufVaQSEFzf5ZVNNfY1W0v6J9S1T4i6fxEl RQYpNbFlvdSHpyvxA2aieoleZgbKW6dymVrPeJiJVSErq/yb08D69OUuh+ggIFCu//rs 0I0I3nZH0GyqoGWWVoSH/+3xEHpSzfnTspExvSYfZYdwdfuKWRbtp1rx0H+hi7B30+9a FvYduI5D43rPZF2+Z6Id3Fwgm8i7WMk+PWHhxBDuEbYVH7AQIH+xHWcZOSyII0DI+S96 SBMA== X-Gm-Message-State: AOAM532ApDFUV6qs7C6pa3hddUJ728SnsxdWMX26piYus26/P8hdfjqD NWCWFqQ7hKqY7LynxEjdeXs= X-Google-Smtp-Source: ABdhPJyDNRgC+2cRPyN77hk18FFO+ZRAYgMHHVdxa0JbddzqzK8q+SgHKF/+4FYmXT2hQ1FHod/anw== X-Received: by 2002:a63:4862:0:b0:385:fb1d:fc54 with SMTP id x34-20020a634862000000b00385fb1dfc54mr1284620pgk.57.1649102595118; Mon, 04 Apr 2022 13:03:15 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id bw17-20020a056a00409100b004fadad3b93esm12779295pfb.142.2022.04.04.13.03.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 13:03:14 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, tytso@mit.edu, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 8/8] mm: mmap: register suitable readonly file vmas for khugepaged Date: Mon, 4 Apr 2022 13:02:50 -0700 Message-Id: <20220404200250.321455-9-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220404200250.321455-1-shy828301@gmail.com> References: <20220404200250.321455-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: 8brpb47zb8hhixmti9ejwsob4miiq5xk X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1C92214000C Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=bnyhwFA2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1649102595-366520 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The readonly FS THP relies on khugepaged to collapse THP for suitable vmas. But it is kind of "random luck" for khugepaged to see the readonly FS vmas (https://lore.kernel.org/linux-mm/00f195d4-d039-3cf2-d3a1-a2c88de397a0@suse.cz/) since currently the vmas are registered to khugepaged when: - Anon huge pmd page fault - VMA merge - MADV_HUGEPAGE - Shmem mmap If the above conditions are not met, even though khugepaged is enabled it won't see readonly FS vmas at all. MADV_HUGEPAGE could be specified explicitly to tell khugepaged to collapse this area, but when khugepaged mode is "always" it should scan suitable vmas as long as VM_NOHUGEPAGE is not set. So make sure readonly FS vmas are registered to khugepaged to make the behavior more consistent. Registering suitable vmas in common mmap path, that could cover both readonly FS vmas and shmem vmas, so removed the khugepaged calls in shmem.c. Still need to keep the khugepaged call in vma_merge() since vma_merge() is called in a lot of places, for example, madvise, mprotect, etc. Reported-by: Vlastimil Babka Signed-off-by: Yang Shi Acked-by: Vlastimil Babka --- mm/mmap.c | 6 ++++++ mm/shmem.c | 4 ---- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 604c8dece5dd..616ebbc2d052 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1842,6 +1842,12 @@ unsigned long mmap_region(struct file *file, unsigned long addr, } vma_link(mm, vma, prev, rb_link, rb_parent); + + /* + * vma_merge() calls khugepaged_enter_vma() either, the below + * call covers the non-merge case. + */ + khugepaged_enter_vma(vma, vma->vm_flags); /* Once vma denies write, undo our temporary denial count */ unmap_writable: if (file && vm_flags & VM_SHARED) diff --git a/mm/shmem.c b/mm/shmem.c index 92eca974771d..0c448080d210 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -34,7 +34,6 @@ #include #include #include -#include #include #include #include @@ -2239,7 +2238,6 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma) file_accessed(file); vma->vm_ops = &shmem_vm_ops; - khugepaged_enter_vma(vma, vma->vm_flags); return 0; } @@ -4132,8 +4130,6 @@ int shmem_zero_setup(struct vm_area_struct *vma) vma->vm_file = file; vma->vm_ops = &shmem_vm_ops; - khugepaged_enter_vma(vma, vma->vm_flags); - return 0; }