From patchwork Thu Mar 17 23:48:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82D17C43217 for ; Thu, 17 Mar 2022 23:48:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229679AbiCQXt5 (ORCPT ); Thu, 17 Mar 2022 19:49:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230196AbiCQXty (ORCPT ); Thu, 17 Mar 2022 19:49:54 -0400 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB7BF1A844F; Thu, 17 Mar 2022 16:48:37 -0700 (PDT) Received: by mail-pf1-x42f.google.com with SMTP id p5so2001637pfo.5; Thu, 17 Mar 2022 16:48:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iUyPpt9yITb+v7E1k0NmB1H1X1O1cjuxJvXjp7qmzqs=; b=KGAXD1pdDKssYLDVJnzInpkYLVCjld8aQdSBPEtJyRYQAiZw78n32N/TO102SQyHmf yOjOvBMg/wJhYADen1q6xkUT8Kar/VUyxw5Gau+O7qYozSjj73pdubTi8sBu+AXtmQ9w xHDtlPvykFut8SJqHAJvJG8KZXZEgPAfadKVFTIJbw8dVHx1yzTemBdDvujvug/x2qut eix5xqdQfVaM/rl64UUQMBtdNSpAx+pya3QxiaWXDfnBGfWayz7Mc6b1ZgeEJOtJBr9L vkOMQEbHQ2bn1ECcRnsEg3MzTu+Log1ScTC8rww7WRae0dOAlzhO2LwjprdtM/HKhCXP sqrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iUyPpt9yITb+v7E1k0NmB1H1X1O1cjuxJvXjp7qmzqs=; b=GuyWLhm0W5J1lEq7dOiKdIuVCFNP+u1tI5WeJRpPO478sQbsuA8lhWPz4mjOavkeHa JiW0Estc1OOVqYsM4VtWZqRJ2pvfYxnFj/81+LIzcTWI72UbsdLH63g+8S5qK4193YoW hLcg+u4clV9MsVmcSbJpSVEqMExZXwDRQHhgY3KlIBLQagXwnvpJ5C/l5uLcj3ibCG6a 0o1KuYQ6qp8ViGh/qfeQdST1ofOt4NN9+ifMcwfmv0Nt8uL3GoUg4eIh8DBZUTjDiFI5 UHjA+g22y2QViloi2zAGVBo27/jlS/sXcdnhu20J6+7Nb0a9pWj7IjNje4XZYC5MflKA lPNQ== X-Gm-Message-State: AOAM5336Nwhc+TS+lKiRIpcuf9kav0LX1y3pMqY9YkrV89gCksvKh//e KsaNvCQCo2QSN9khD6NGjCc= X-Google-Smtp-Source: ABdhPJy/71kNE1QNAXF397FsY5nOp4oXeRKHByVUgbhgNMHcH+U1wY50LwEcOseYcTKRNJDA4ZMvIg== X-Received: by 2002:a63:4e41:0:b0:381:7d61:ed68 with SMTP id o1-20020a634e41000000b003817d61ed68mr5651161pgl.132.1647560917456; Thu, 17 Mar 2022 16:48:37 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:37 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 1/8] sched: coredump.h: clarify the use of MMF_VM_HUGEPAGE Date: Thu, 17 Mar 2022 16:48:20 -0700 Message-Id: <20220317234827.447799-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org MMF_VM_HUGEPAGE is set as long as the mm is available for khugepaged by khugepaged_enter(), not only when VM_HUGEPAGE is set on vma. Correct the comment to avoid confusion. Reviewed-by: Miaohe Lin Signed-off-by: Yang Shi --- include/linux/sched/coredump.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/coredump.h b/include/linux/sched/coredump.h index 4d9e3a656875..4d0a5be28b70 100644 --- a/include/linux/sched/coredump.h +++ b/include/linux/sched/coredump.h @@ -57,7 +57,8 @@ static inline int get_dumpable(struct mm_struct *mm) #endif /* leave room for more dump flags */ #define MMF_VM_MERGEABLE 16 /* KSM may merge identical pages */ -#define MMF_VM_HUGEPAGE 17 /* set when VM_HUGEPAGE is set on vma */ +#define MMF_VM_HUGEPAGE 17 /* set when mm is available for + khugepaged */ /* * This one-shot flag is dropped due to necessity of changing exe once again * on NFS restore From patchwork Thu Mar 17 23:48:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AB80C43219 for ; Thu, 17 Mar 2022 23:48:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230242AbiCQXuG (ORCPT ); Thu, 17 Mar 2022 19:50:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230212AbiCQXt6 (ORCPT ); Thu, 17 Mar 2022 19:49:58 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD58D1A844F; Thu, 17 Mar 2022 16:48:39 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id q13so5721192plk.12; Thu, 17 Mar 2022 16:48:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XKQspk4LAoVq0Cn8x51eh1PH1jIoAPLYMSrh0f750IY=; b=jFpvMyu8Lmr30lvoJVeQyn6l2I3hQx7c4LidmLWMeHE4pc9vJJjEZWCIvkxKg+angN IKSwRoWtxS2l+HwIpyduOicwl2cPw8BId1H8G1v857wETnyNwbCcD8rxkYN7IHnbM2jv 2yh1znyUjEMD/yp7wB5tzUGvQ9qmp4T31S+grwxSZv1lsoZwjngjtovgzZVDcZokHo3g LyHy8HqznKKYKNlcp7xXBV9YLMUT+CNXT7N/eUh+CGOJJPl1dryMvjJ7YiSeiYh+gS7/ IzlJm8QRKIDVl5EKLTbQIgRQyqa0RoX0GyQyIGZWYNQWnC1C1SUsLE+Vii5kQf4cSfrj vv9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XKQspk4LAoVq0Cn8x51eh1PH1jIoAPLYMSrh0f750IY=; b=nIgUqY4YDCcRa+87glr/ySl8kTjMsEHjd5w2CEnCA7qWiMHZUjY6uG5jauvb+kd+Dk gseKnTLy10QfTQDuC7Hh+HSGZLY1t857Zm/AXV1YKv4jS/dQsBlDyDHDV0MbwA2xf1ni RSJ1HWXIyB4182vUsp2ceDMe7lMf45bS4SS8YZkPe7w5fS48jLzdrkunrdgdVhocyywT tz/91FErl3GmnbI9Hmkz0yswIGXOFKc+3x7qnORYSUhWuuGQM9XHQ7i8uIaK9SpWSNLi 8zQgnAbbuaVL6nwIdRjva80a86ed8u40DWmJem4tBH0mr/7nq0WaIryvclsZZeOih+Sp p+EA== X-Gm-Message-State: AOAM5339b8rnt1RV6sqIvV2l+Lsae9Ki9Ko8V3KdTL+uM8Dm/fKWJg35 E3U4A5PdX2yinmnjvL4uEXGj8h409zc= X-Google-Smtp-Source: ABdhPJydZul27UYLlzwCg5wysvPkxxFiprMv7eHU1Yr/csXpk/Erca9BMe2mrozIbghvD/AKSU+PgQ== X-Received: by 2002:a17:90a:aa83:b0:1b9:7c62:61e5 with SMTP id l3-20020a17090aaa8300b001b97c6261e5mr8263237pjq.118.1647560919231; Thu, 17 Mar 2022 16:48:39 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:38 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 2/8] mm: khugepaged: remove redundant check for VM_NO_KHUGEPAGED Date: Thu, 17 Mar 2022 16:48:21 -0700 Message-Id: <20220317234827.447799-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org The hugepage_vma_check() called by khugepaged_enter_vma_merge() does check VM_NO_KHUGEPAGED. Remove the check from caller and move the check in hugepage_vma_check() up. More checks may be run for VM_NO_KHUGEPAGED vmas, but MADV_HUGEPAGE is definitely not a hot path, so cleaner code does outweigh. Reviewed-by: Miaohe Lin Signed-off-by: Yang Shi --- mm/khugepaged.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 131492fd1148..82c71c6da9ce 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -366,8 +366,7 @@ int hugepage_madvise(struct vm_area_struct *vma, * register it here without waiting a page fault that * may not happen any time soon. */ - if (!(*vm_flags & VM_NO_KHUGEPAGED) && - khugepaged_enter_vma_merge(vma, *vm_flags)) + if (khugepaged_enter_vma_merge(vma, *vm_flags)) return -ENOMEM; break; case MADV_NOHUGEPAGE: @@ -446,6 +445,9 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, if (!transhuge_vma_enabled(vma, vm_flags)) return false; + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) return false; @@ -471,7 +473,8 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, return false; if (vma_is_temporary_stack(vma)) return false; - return !(vm_flags & VM_NO_KHUGEPAGED); + + return true; } int __khugepaged_enter(struct mm_struct *mm) From patchwork Thu Mar 17 23:48:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08872C433F5 for ; Thu, 17 Mar 2022 23:48:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230254AbiCQXuG (ORCPT ); Thu, 17 Mar 2022 19:50:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230222AbiCQXt7 (ORCPT ); Thu, 17 Mar 2022 19:49:59 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7598B2AA86E; Thu, 17 Mar 2022 16:48:41 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id q1-20020a17090a4f8100b001c6575ae105so4587516pjh.0; Thu, 17 Mar 2022 16:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=30LWLJQV8dYXosutckVn4XhId3NQkyArrNjeupviuTU=; b=QAVrMWYYLggwaur7/Slf6NucuIwOCNCO6CoeQBthKAHyKZvkgx+a7sm2sHBPqQ0opF 4Ki5b6Z0eXLr3PGCm9qtgaAY/ODWDKMdlFmdYoeEq0N1GBRlIz2z2tIICaXP45MUsac/ f69JzjhxHzv8Bsf0K8UnOYG82Z8vsMetYba3YirBpAPXzyPHEbiwKSpahVFE7gDZoOY5 FvLg3msNOHnXsZsKaVFyPHJUUWR7ZAq2C7JZIPlQMklTIZjjvHOCDAnKEEpMlUeDhOW2 3TEalbREksSCv5UTseNYULzPHRoNQN60aTvVm/fgh8TzEUmBAhlMjIDjE5EqZkvYj82d J7uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=30LWLJQV8dYXosutckVn4XhId3NQkyArrNjeupviuTU=; b=fS4eJo7A3VI0IYEO+178w7GRqfZLwmcGP1CAvoRSCPTAtdglFgQTNllHMc7hc7+p3o 2NG/nEtWR1d1CrO3KuL5xEawBTBxyERa85Vfl83qe0czdHqYt0IYYpH5v3j0TCfkeJ/M 0kUEW3ZFVrYJysl4oy0zbqpS6Wt93W7fmBnDvCwVr0aAB7K1FS+6vWvte6RV1wPXn8uX AUGX6NPK4qYNB9pFzE5DctfgZp+4N/58d2lmdnWSDc+6GlUvWPrf9kBaF6+F9yrHS7x4 LICHsf95ZH5NJzz+IRuHZJta6J5fx+mpk++L0JmczT0I1bczVD9x9mn8BPt9p3ieFGC6 f/NA== X-Gm-Message-State: AOAM531I1KkrTYy3kq6n+NtQnrrCT0efIOJnGY4xbmv9Yy0CvsDsfpLs FhMq6mH0tmlxs5fYW3EdgW0= X-Google-Smtp-Source: ABdhPJyiE6aP96mN2Z6JWDNGGQUqnYvg1jk7rBq+HLDBH4ccEO+al+eqvAHhq16+7YAPwjWxn2X9Ng== X-Received: by 2002:a17:90a:ba07:b0:1bc:a0fd:faf with SMTP id s7-20020a17090aba0700b001bca0fd0fafmr8254608pjr.194.1647560921021; Thu, 17 Mar 2022 16:48:41 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:40 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 3/8] mm: khugepaged: skip DAX vma Date: Thu, 17 Mar 2022 16:48:22 -0700 Message-Id: <20220317234827.447799-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org The DAX vma may be seen by khugepaged when the mm has other khugepaged suitable vmas. So khugepaged may try to collapse THP for DAX vma, but it will fail due to page sanity check, for example, page is not on LRU. So it is not harmful, but it is definitely pointless to run khugepaged against DAX vma, so skip it in early check. Reviewed-by: Miaohe Lin Signed-off-by: Yang Shi --- mm/khugepaged.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 82c71c6da9ce..a0e4fa33660e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -448,6 +448,10 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, if (vm_flags & VM_NO_KHUGEPAGED) return false; + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) return false; From patchwork Thu Mar 17 23:48:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BE5EC433FE for ; Thu, 17 Mar 2022 23:48:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbiCQXuH (ORCPT ); Thu, 17 Mar 2022 19:50:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230196AbiCQXuC (ORCPT ); Thu, 17 Mar 2022 19:50:02 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31A672AA871; Thu, 17 Mar 2022 16:48:43 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id v4so6195794pjh.2; Thu, 17 Mar 2022 16:48:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lh1UGdWCnbI+ssw0qidScDsXf7KLfyH1hGff66irmcE=; b=R1QJxrwiBU4YS9L9DkNURkUR9lfvb/Cy/T8EIWRnh6gbYSCC+76DVh7/Jg9jXSf55g K6fpUi/kIkvkwh+o4gfxQRQ5zFay3g2W0HPBHBD6F0DwymIdlYggztiNO9NHEThRgSbr TtnUpg1e0ovlzbqghftQc0RWPR74EJoBCVaUoW93aADPRfCh7a47k3yhbTmOiWYCBy1I 9EotNKrzw5joQz0eceyxb0qCftsl2fDAIaB+5eS4UgleMXpNCi4e479tODCu9TAGkrNj 06Ao5McZwciCNSoBQ3XEaZmXgisinxI7sJzX9BY53KfeZsQezL79/RTAj83B6BCpzaJs BQ5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lh1UGdWCnbI+ssw0qidScDsXf7KLfyH1hGff66irmcE=; b=B9HGystSF1SvrQjwBr79rV8ZBeYH+eQ7fB7ZWosJ0GHASxv4qrRPe52PscGYCVn8rz quOrohOABwRnZiJPvPtwCI6tNEkj+YdjGNeBRQGrpsFpr/3ceQfRIewvWkocj3tw11Qx syDVCe++44SP2o3pCCbPNn3R3BPCesA0YnqyjXnxaNgWHwGzQ5gfM86h+Cssn6URYn3e SjB/f+YMiV14GiczFm+RmuSJDifYqba/qqb0ceXPMIFSTRv3y8TT8sG3wYIYF48//2P9 zUGv4FpZgYUbIWWTZsl70vezO7TIbegx4QN1Dp52erGOcmVMSnmOAgb6MgEwBSJU6OD8 y2Xg== X-Gm-Message-State: AOAM5323lV7EP/jczdf22+tYehrQ9PKc/kmdKvaJcx6XKeoCaYtbxLwk zD+XgRg16X6Y0wouky+tFEM= X-Google-Smtp-Source: ABdhPJxH1oWUw7HG5U383xf4tuloVnFsIXbUZKg15Usu8FWbDYGloqS9CKbXc3kH1w7IZ+w1PBStrQ== X-Received: by 2002:a17:90a:e541:b0:1c6:55e5:ae4b with SMTP id ei1-20020a17090ae54100b001c655e5ae4bmr13968964pjb.62.1647560922758; Thu, 17 Mar 2022 16:48:42 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:42 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 4/8] mm: thp: only regular file could be THP eligible Date: Thu, 17 Mar 2022 16:48:23 -0700 Message-Id: <20220317234827.447799-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Since commit a4aeaa06d45e ("mm: khugepaged: skip huge page collapse for special files"), khugepaged just collapses THP for regular file which is the intended usecase for readonly fs THP. Only show regular file as THP eligible accordingly. And make file_thp_enabled() available for khugepaged too in order to remove duplicate code. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 14 ++++++++++++++ mm/huge_memory.c | 11 ++--------- mm/khugepaged.c | 9 ++------- 3 files changed, 18 insertions(+), 16 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e4c18ba8d3bf..3cfa79732112 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -172,6 +172,20 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } +static inline bool file_thp_enabled(struct vm_area_struct *vma) +{ + struct inode *inode; + + if (!vma->vm_file) + return false; + + inode = vma->vm_file->f_inode; + + return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && + (vma->vm_flags & VM_EXEC) && + !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); +} + bool transparent_hugepage_active(struct vm_area_struct *vma); #define transparent_hugepage_use_zero_page() \ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 406a3c28c026..a87b3df63209 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -64,13 +64,6 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -static inline bool file_thp_enabled(struct vm_area_struct *vma) -{ - return transhuge_vma_enabled(vma, vma->vm_flags) && vma->vm_file && - !inode_is_open_for_write(vma->vm_file->f_inode) && - (vma->vm_flags & VM_EXEC); -} - bool transparent_hugepage_active(struct vm_area_struct *vma) { /* The addr is used to check if the vma size fits */ @@ -82,8 +75,8 @@ bool transparent_hugepage_active(struct vm_area_struct *vma) return __transparent_hugepage_enabled(vma); if (vma_is_shmem(vma)) return shmem_huge_enabled(vma); - if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) - return file_thp_enabled(vma); + if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + return true; return false; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a0e4fa33660e..3dbac3e23f43 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -465,13 +465,8 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Only regular file is valid */ - if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file && - (vm_flags & VM_EXEC)) { - struct inode *inode = vma->vm_file->f_inode; - - return !inode_is_open_for_write(inode) && - S_ISREG(inode->i_mode); - } + if (file_thp_enabled(vma)) + return true; if (!vma->anon_vma || vma->vm_ops) return false; From patchwork Thu Mar 17 23:48:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24EC8C43217 for ; Thu, 17 Mar 2022 23:48:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230274AbiCQXuI (ORCPT ); Thu, 17 Mar 2022 19:50:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230229AbiCQXuF (ORCPT ); Thu, 17 Mar 2022 19:50:05 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18B0B2B4A58; Thu, 17 Mar 2022 16:48:45 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id mm4-20020a17090b358400b001c68e836fa6so2552906pjb.3; Thu, 17 Mar 2022 16:48:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PYzHG/r/zQJfX7Jqdvp+ttkJo6GETokQfZcD3GwLWs8=; b=Z4F+yQTk2He8L/86bsJLnVVNlsXypelqhrfvzSqV9C9uS+amGuZCeVp1JeAn5V/7yh Fe0KGQJIietgygywMef0Co4w8KuNoDKDjd1d0qYVUicTjBQws1GaxyXqp26STEkA2ADW dVZFMSeR3f5c5FQSOTsq0rRzAuBhst7Miwm3JV7jr/r8SN6eg5uLINUQbRmJlzxw50bD 15E/yagk5ULvlb9js2dCx4CpZiWQg0bLfhyRNBm1oWE78+7vL/YAmu59kzczJ7BGLVll q3S7WykzeYzB4HOyqDIXRZlBRFdquSXC6Bk57NC4Q6HWcqBoRpUwF+yZncsx2TKCSXkX CCBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PYzHG/r/zQJfX7Jqdvp+ttkJo6GETokQfZcD3GwLWs8=; b=OG75b3UV16Mnxs0S18ezm2dLAkVjAlvKuJ1M37t+qEvp0npGl3SbzUkHRwDy1aMWU0 GsBSiupknCW0NU8VuuSQDdyzUC77BLA3aIDodrhIjcFRh6RVFCr6CFzE4dRJHHYBW3ko Se4iM8QT8r+9T34ksgffd/+EvnpSuuQSHiRpxPoSiTN0zkUfK3VQax+bPRVksFV0bI91 HSQEKSN6IfF3cvnjlEUpBGHxwdSLHDFEjvmk1/Q3cLXZClu+Y/OaqlUIefDGCUxMAYcb 2rc+50GXaI3EhRiAxhA5Eyekk70eZrYY8FqsaJhYnvhMLbDErWompaX2ibOnVb9d65x5 VeCw== X-Gm-Message-State: AOAM532UBezN/mq+RLSVBVQQ65LteMVwJPdejIexXaIIjq8fuGq+GV1f ZcJAo5HNAuf4jep4rXv9bKkGxIRSthI= X-Google-Smtp-Source: ABdhPJx97F5tIfVkFzBbXPTT78rOrWAxbgyRshn1PHWj1ladodq9la1CylBGIpmZgN62nrCLrqglmQ== X-Received: by 2002:a17:902:6944:b0:153:9866:7fea with SMTP id k4-20020a170902694400b0015398667feamr7104342plt.6.1647560924573; Thu, 17 Mar 2022 16:48:44 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:44 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 5/8] mm: khugepaged: make khugepaged_enter() void function Date: Thu, 17 Mar 2022 16:48:24 -0700 Message-Id: <20220317234827.447799-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org The most callers of khugepaged_enter() don't care about the return value. Only dup_mmap(), anonymous THP page fault and MADV_HUGEPAGE handle the error by returning -ENOMEM. Actually it is not harmful for them to ignore the error case either. It also sounds overkilling to fail fork() and page fault early due to khugepaged_enter() error, and MADV_HUGEPAGE does set VM_HUGEPAGE flag regardless of the error. Signed-off-by: Yang Shi --- include/linux/khugepaged.h | 30 ++++++++++++------------------ kernel/fork.c | 4 +--- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++----------- 4 files changed, 22 insertions(+), 34 deletions(-) diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 2fcc01891b47..0423d3619f26 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -12,10 +12,10 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern int __khugepaged_enter(struct mm_struct *mm); +extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); -extern int khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags); +extern void khugepaged_enter_vma_merge(struct vm_area_struct *vma, + unsigned long vm_flags); extern void khugepaged_min_free_kbytes_update(void); #ifdef CONFIG_SHMEM extern void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr); @@ -40,11 +40,10 @@ static inline void collapse_pte_mapped_thp(struct mm_struct *mm, (transparent_hugepage_flags & \ (1<flags)) - return __khugepaged_enter(mm); - return 0; + __khugepaged_enter(mm); } static inline void khugepaged_exit(struct mm_struct *mm) @@ -53,7 +52,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) __khugepaged_exit(mm); } -static inline int khugepaged_enter(struct vm_area_struct *vma, +static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) @@ -62,27 +61,22 @@ static inline int khugepaged_enter(struct vm_area_struct *vma, (khugepaged_req_madv() && (vm_flags & VM_HUGEPAGE))) && !(vm_flags & VM_NOHUGEPAGE) && !test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - if (__khugepaged_enter(vma->vm_mm)) - return -ENOMEM; - return 0; + __khugepaged_enter(vma->vm_mm); } #else /* CONFIG_TRANSPARENT_HUGEPAGE */ -static inline int khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) +static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { - return 0; } static inline void khugepaged_exit(struct mm_struct *mm) { } -static inline int khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) +static inline void khugepaged_enter(struct vm_area_struct *vma, + unsigned long vm_flags) { - return 0; } -static inline int khugepaged_enter_vma_merge(struct vm_area_struct *vma, - unsigned long vm_flags) +static inline void khugepaged_enter_vma_merge(struct vm_area_struct *vma, + unsigned long vm_flags) { - return 0; } static inline void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) diff --git a/kernel/fork.c b/kernel/fork.c index a024bf6254df..dc85418c426a 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -523,9 +523,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm, retval = ksm_fork(mm, oldmm); if (retval) goto out; - retval = khugepaged_fork(mm, oldmm); - if (retval) - goto out; + khugepaged_fork(mm, oldmm); prev = NULL; for (mpnt = oldmm->mmap; mpnt; mpnt = mpnt->vm_next) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a87b3df63209..ec2490d6af09 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -725,8 +725,8 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return VM_FAULT_FALLBACK; if (unlikely(anon_vma_prepare(vma))) return VM_FAULT_OOM; - if (unlikely(khugepaged_enter(vma, vma->vm_flags))) - return VM_FAULT_OOM; + khugepaged_enter(vma, vma->vm_flags); + if (!(vmf->flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(vma->vm_mm) && transparent_hugepage_use_zero_page()) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 3dbac3e23f43..b87af297e652 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -366,8 +366,7 @@ int hugepage_madvise(struct vm_area_struct *vma, * register it here without waiting a page fault that * may not happen any time soon. */ - if (khugepaged_enter_vma_merge(vma, *vm_flags)) - return -ENOMEM; + khugepaged_enter_vma_merge(vma, *vm_flags); break; case MADV_NOHUGEPAGE: *vm_flags &= ~VM_HUGEPAGE; @@ -476,20 +475,20 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, return true; } -int __khugepaged_enter(struct mm_struct *mm) +void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; int wakeup; mm_slot = alloc_mm_slot(); if (!mm_slot) - return -ENOMEM; + return; /* __khugepaged_exit() must not run from under us */ VM_BUG_ON_MM(khugepaged_test_exit(mm), mm); if (unlikely(test_and_set_bit(MMF_VM_HUGEPAGE, &mm->flags))) { free_mm_slot(mm_slot); - return 0; + return; } spin_lock(&khugepaged_mm_lock); @@ -505,11 +504,9 @@ int __khugepaged_enter(struct mm_struct *mm) mmgrab(mm); if (wakeup) wake_up_interruptible(&khugepaged_wait); - - return 0; } -int khugepaged_enter_vma_merge(struct vm_area_struct *vma, +void khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags) { unsigned long hstart, hend; @@ -520,13 +517,12 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma, * file-private shmem THP is not supported. */ if (!hugepage_vma_check(vma, vm_flags)) - return 0; + return; hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; hend = vma->vm_end & HPAGE_PMD_MASK; if (hstart < hend) - return khugepaged_enter(vma, vm_flags); - return 0; + khugepaged_enter(vma, vm_flags); } void __khugepaged_exit(struct mm_struct *mm) From patchwork Thu Mar 17 23:48:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06640C433EF for ; Thu, 17 Mar 2022 23:49:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230236AbiCQXuP (ORCPT ); Thu, 17 Mar 2022 19:50:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230233AbiCQXuF (ORCPT ); Thu, 17 Mar 2022 19:50:05 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE6C32B5EE5; Thu, 17 Mar 2022 16:48:46 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id m22so6234506pja.0; Thu, 17 Mar 2022 16:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g+Z88nKMaMfnoSMNb28nSkaCjqMBUKAv77UuVBn2dOA=; b=NOASi9NgJH/JhidTvkHtQlNha/nZlQouiv+VMSrwVtvNEwYlfFWCG+n2/7jjdRpBSm MY/X+xNVwdSloqJFf3LBUjzMllxCucZpqM9XZaVh9SCVlL5YIc8l3Ya6LRBD22kEwjuu dgnuWVgjO6iSnyFi4Z2IEO1E7EFhe1HtApOYrT/S2d7FQ0u2s5zwp53tgjQCg57iqBz/ ID4ucyePPmJOTJcwhxzDjbN8SHnPxdRgOfWtE20nncuBfKLaF294ZhS+bxp4vKMj0Lmo uvQQJDiDTM8uxNZa62mz3YJEKqwfFEoCSqHkOSUvQA6r1I3W01MGxHi+uXPu8UrECc+v IaXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g+Z88nKMaMfnoSMNb28nSkaCjqMBUKAv77UuVBn2dOA=; b=uTqKgdvXkient94bJ96n4XkbvBxNyJ/aVecLaayco1IRoGgiybI8kL8jPSMQpoDxLK AepxbXBtysS5y63dT+MCYByuNx5ydv0WX/A23o0GlhoEZJ4y6OR/fD7nPJuIMi5kGZW8 nMzuRlDl2u0isnsH3yWPMiTyM4VLGLQhlNm3Zhs6rYHEOgzuZccjq/M4iXoMRc8qt0X1 kYmav5MLn53YRwykuPf4wZK9GuImxVtA9f6RtSpRJQ/i2hJq5xnr7zxLdGlJ62aYkBsk C4DTnuWEMM58kfsvoYuBsigvUE36z6rjPecQ3wFVRswWyaNjUi/iJuZDWBj/qK32YqG2 XpuQ== X-Gm-Message-State: AOAM532OYS7RrwXa0do54NsuQoA5XbLUEbQFekC3sXOFWjLcOrlpXdgj Wl4pJy9GZaSfgPoQbhMOuWA= X-Google-Smtp-Source: ABdhPJxDC2A0C1V8B2FdGbOhYK6a94gMe52gtKBZcHpjue2Sw4OdOaK/2/qo2FvvMWbiqB7Gu8VYLw== X-Received: by 2002:a17:902:f684:b0:151:93ab:3483 with SMTP id l4-20020a170902f68400b0015193ab3483mr7397253plg.4.1647560926315; Thu, 17 Mar 2022 16:48:46 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:45 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 6/8] mm: khugepaged: move some khugepaged_* functions to khugepaged.c Date: Thu, 17 Mar 2022 16:48:25 -0700 Message-Id: <20220317234827.447799-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org This move also makes the following patches easier. The following patches will call khugepaged_enter() for regular filesystems to make readonly FS THP collapse more consistent. They need to use some macros defined in huge_mm.h, for example, HPAGE_PMD_*, but it seems not preferred to polute filesystems code with including unnecessary header files. With this move the filesystems code just need include khugepaged.h, which is quite small and the use is quite specific, to call khugepaged_enter() to hook mm with khugepaged. And the khugepaged_* functions actually are wrappers for some non-inline functions, so it seems the benefits are not too much to keep them inline. This also helps to reuse hugepage_vma_check() for khugepaged_enter() so that we could remove some duplicate checks. Signed-off-by: Yang Shi --- include/linux/khugepaged.h | 33 ++++++--------------------------- mm/khugepaged.c | 20 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 27 deletions(-) diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 0423d3619f26..54e169116d49 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -16,6 +16,12 @@ extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags); +extern void khugepaged_fork(struct mm_struct *mm, + struct mm_struct *oldmm); +extern void khugepaged_exit(struct mm_struct *mm); +extern void khugepaged_enter(struct vm_area_struct *vma, + unsigned long vm_flags); + extern void khugepaged_min_free_kbytes_update(void); #ifdef CONFIG_SHMEM extern void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr); @@ -33,36 +39,9 @@ static inline void collapse_pte_mapped_thp(struct mm_struct *mm, #define khugepaged_always() \ (transparent_hugepage_flags & \ (1<flags)) - __khugepaged_enter(mm); -} - -static inline void khugepaged_exit(struct mm_struct *mm) -{ - if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) - __khugepaged_exit(mm); -} - -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) - if ((khugepaged_always() || - (shmem_file(vma->vm_file) && shmem_huge_enabled(vma)) || - (khugepaged_req_madv() && (vm_flags & VM_HUGEPAGE))) && - !(vm_flags & VM_NOHUGEPAGE) && - !test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - __khugepaged_enter(vma->vm_mm); -} #else /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b87af297e652..4cb4379ecf25 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -557,6 +557,26 @@ void __khugepaged_exit(struct mm_struct *mm) } } +void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) +{ + if (test_bit(MMF_VM_HUGEPAGE, &oldmm->flags)) + __khugepaged_enter(mm); +} + +void khugepaged_exit(struct mm_struct *mm) +{ + if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) + __khugepaged_exit(mm); +} + +void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) +{ + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && + khugepaged_enabled()) + if (hugepage_vma_check(vma, vm_flags)) + __khugepaged_enter(vma->vm_mm); +} + static void release_pte_page(struct page *page) { mod_node_page_state(page_pgdat(page), From patchwork Thu Mar 17 23:48:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE7A2C4332F for ; Thu, 17 Mar 2022 23:48:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230211AbiCQXuK (ORCPT ); Thu, 17 Mar 2022 19:50:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230266AbiCQXuH (ORCPT ); Thu, 17 Mar 2022 19:50:07 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D01D02A5A85; Thu, 17 Mar 2022 16:48:48 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id n15so5765283plh.2; Thu, 17 Mar 2022 16:48:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e8qThGmR6qWWYZNQEqh4yIj3Ne/feg2xUmJaASesT2M=; b=Q8u/tgbEZte1l+GJAkZCMz3Ct5k4Nu5o8GV+SvDdYg7FPjhtRQ4S6BthMj0tx6333x CNQuCEPSL3r1/gAm7gZ3BvPAHQu/VIQhP548H716jpthOj1bvkeHftkCgd+iir4u5Wb/ 7QWlt6efJHKZJatCDSMMLkYyCk9YWhoclsbWlJP04Q93Yvxe/B9AG86U35LbVlxuTZE8 JutmKy2dyY9KA51e5m6N3o9gH6/XfzVlA1lI7LRdNn53bxGlRH/u9FhrTAbji6rxuDGq mZTT1gAywfZuHm247vqURRsIUcEa6NOKt+fnFQgEGI4oP4xEGbtL/qCbZSEg+FiUurEh Ei7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e8qThGmR6qWWYZNQEqh4yIj3Ne/feg2xUmJaASesT2M=; b=ES0AxqUtvDIMJ/YJ4DcNTSuxxqcb/+xWGj+UjeiRvKdCAGMjAMMZjJrml8jkKsx6au mmYelxOCJwUdHyVymL/ce+5EpqVlZEC2ouzZ1smOIgvLDd5nbpDmoRYw8XwefJZwo0nG pWLNFK9LHOgnC/KHKHQYCa6P964vJulAVxeb2EiKKwjXePAvEGiYapRic2b9zyPUO7pk VxxX04emjotgDViDOPrvfddcXhAIphBvK8lzLyzRXTDuy7yvmWE4Yd1seEGvopz5YSoH snm7LTiop6GIzB340JgkF345lbYseD0tIkuxUP1nDFVkawqpgeb+bs/FABpNPB/bHxKb TZig== X-Gm-Message-State: AOAM532noN0eAswGCwLA1XaGkf1DOSZ1UYEBaIUUE6iT863bOoPQowGM Gy2H86qSV1oyjt6wqIQlK/k= X-Google-Smtp-Source: ABdhPJyp8fZ/A0uESg9v+zKNEb0Mpr9NlB5MF0RbJ0fvxFGMYc0TTPAaDF54pFiTc/DyCZMv8jCtSg== X-Received: by 2002:a17:902:dad2:b0:151:f895:9c31 with SMTP id q18-20020a170902dad200b00151f8959c31mr7666608plx.93.1647560928254; Thu, 17 Mar 2022 16:48:48 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:47 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 7/8] mm: khugepaged: introduce khugepaged_enter_file() helper Date: Thu, 17 Mar 2022 16:48:26 -0700 Message-Id: <20220317234827.447799-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org The following patch will have filesystems code call khugepaged_enter() to make readonly FS THP collapse more consistent. Extract the current implementation used by shmem in khugepaged_enter_file() helper so that it could be reused by other filesystems and export the symbol for modules. Signed-off-by: Yang Shi --- include/linux/khugepaged.h | 6 ++++++ mm/khugepaged.c | 11 +++++++++++ mm/shmem.c | 14 ++++---------- 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 54e169116d49..06464e9a1f91 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -21,6 +21,8 @@ extern void khugepaged_fork(struct mm_struct *mm, extern void khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags); +extern void khugepaged_enter_file(struct vm_area_struct *vma, + unsigned long vm_flags); extern void khugepaged_min_free_kbytes_update(void); #ifdef CONFIG_SHMEM @@ -53,6 +55,10 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { } +static inline void khugepaged_enter_file(struct vm_area_struct *vma, + unsigned long vm_flags) +{ +} static inline void khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4cb4379ecf25..93c9072983e2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -577,6 +577,17 @@ void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) __khugepaged_enter(vma->vm_mm); } +void khugepaged_enter_file(struct vm_area_struct *vma, unsigned long vm_flags) +{ + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && + khugepaged_enabled() && + (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < + (vma->vm_end & HPAGE_PMD_MASK))) + if (hugepage_vma_check(vma, vm_flags)) + __khugepaged_enter(vma->vm_mm); +} +EXPORT_SYMBOL_GPL(khugepaged_enter_file); + static void release_pte_page(struct page *page) { mod_node_page_state(page_pgdat(page), diff --git a/mm/shmem.c b/mm/shmem.c index a09b29ec2b45..c2346e5d2b24 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2233,11 +2233,9 @@ static int shmem_mmap(struct file *file, struct vm_area_struct *vma) file_accessed(file); vma->vm_ops = &shmem_vm_ops; - if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - ((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK)) { - khugepaged_enter(vma, vma->vm_flags); - } + + khugepaged_enter_file(vma, vma->vm_flags); + return 0; } @@ -4132,11 +4130,7 @@ int shmem_zero_setup(struct vm_area_struct *vma) vma->vm_file = file; vma->vm_ops = &shmem_vm_ops; - if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - ((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK)) { - khugepaged_enter(vma, vma->vm_flags); - } + khugepaged_enter_file(vma, vma->vm_flags); return 0; } From patchwork Thu Mar 17 23:48:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12784690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FD6BC433FE for ; Thu, 17 Mar 2022 23:48:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230313AbiCQXuM (ORCPT ); Thu, 17 Mar 2022 19:50:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230293AbiCQXuJ (ORCPT ); Thu, 17 Mar 2022 19:50:09 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 958272A5AA8; Thu, 17 Mar 2022 16:48:50 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id o68-20020a17090a0a4a00b001c686a48263so1899098pjo.1; Thu, 17 Mar 2022 16:48:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lpJtQjEfn/J2FKfZSO5FEN5nDVRpWWfEH9M+CcSvJbU=; b=RW+a9Wh2STaMT2/ByGK2Md1L8l3qKvZPNAaJhGfAVuHce7kmLxJdX9c9n536/G9P9y rvKTyft8Ed8eVecVBtxNJ2AhZb5BHkZZRQ6tIYAz7LjMedc55NDEtR/KaNFKvJJDd+P7 dy6mETgUQ3B6OTZJRO7Su7WKmvdgNXwy+9EwnZOz9NXxEQTMUvnGftE1KdHEXVH19Unr 03ZuudCFe8ibtoQLSzHvkHZddzxVkQNTFTI+HDVGugx8wRTOOgdYd3gwkGBPGeF2R8/z YER8jENcx9I+w/nKNDbx91mtq4iJfWJC/8eIMf2hu7fjepJma0HgRYdRXFf3QEkXmcBs OFnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lpJtQjEfn/J2FKfZSO5FEN5nDVRpWWfEH9M+CcSvJbU=; b=Uz9cXlJMhNaitTQyoxcUP9wwa54uEhBtI6mpqH4ZbaPgPLnJ0M6meLZVxKupqbcgr8 X1KRIxoKE6uV37w0Az5JRdSi4zr0OOTzik+32QxTtJZwUvR+NcE0aNzeTMOa07mHi+oP mI7gJBqxKZ/uEgT5gUZCiAcqRozt6JB4/uTgYa5RAN0mDh1WZU55Q41ppwiGkIQXpzuJ W8O/QrUrYK8KEm8PJLM94/jggvP9KR+LlB/8HLHqrZuPf9cRMxUOtn6QVMIPiA+hERV7 n6xq4Q4nhTL7TSI7EgQ9ZXhM0ZH+s7mLehmub/De2sN33ZOP4sLN4TCSWlOWaX80uQTW cOFA== X-Gm-Message-State: AOAM531HjpWBEnUOfDer8rJh1q7sGmKNg0R/hvWqM/7ysEFZDf2xIKuF I6xwtYzlxLTHNxeu6FcFMgQ= X-Google-Smtp-Source: ABdhPJzOSd1Yo9VoDx5vkTBlM+flQ+uJvq5hb6bD1MASZybdj0vNcol2PHZgGMYbr9bl0KDm+fi/4g== X-Received: by 2002:a17:902:aa01:b0:153:3a40:1097 with SMTP id be1-20020a170902aa0100b001533a401097mr7196864plb.107.1647560930108; Thu, 17 Mar 2022 16:48:50 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id o7-20020aa79787000000b004f8e44a02e2sm8581329pfp.45.2022.03.17.16.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Mar 2022 16:48:49 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, songliubraving@fb.com, riel@surriel.com, willy@infradead.org, ziy@nvidia.com, akpm@linux-foundation.org, tytso@mit.edu, adilger.kernel@dilger.ca, darrick.wong@oracle.com Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 8/8] fs: register suitable readonly vmas for khugepaged Date: Thu, 17 Mar 2022 16:48:27 -0700 Message-Id: <20220317234827.447799-9-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220317234827.447799-1-shy828301@gmail.com> References: <20220317234827.447799-1-shy828301@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org The readonly FS THP relies on khugepaged to collapse THP for suitable vmas. But it is kind of "random luck" for khugepaged to see the readonly FS vmas (https://lore.kernel.org/linux-mm/00f195d4-d039-3cf2-d3a1-a2c88de397a0@suse.cz/) since currently the vmas are registered to khugepaged when: - Anon huge pmd page fault - VMA merge - MADV_HUGEPAGE - Shmem mmap If the above conditions are not met, even though khugepaged is enabled it won't see readonly FS vmas at all. MADV_HUGEPAGE could be specified explicitly to tell khugepaged to collapse this area, but when khugepaged mode is "always" it should scan suitable vmas as long as VM_NOHUGEPAGE is not set. So make sure readonly FS vmas are registered to khugepaged to make the behavior more consistent. Registering the vmas in mmap path seems more preferred from performance point of view since page fault path is definitely hot path. Reported-by: Vlastimil Babka Signed-off-by: Yang Shi --- fs/ext4/file.c | 4 ++++ fs/xfs/xfs_file.c | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 8cc11715518a..b894cd5aff44 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "ext4.h" #include "ext4_jbd2.h" #include "xattr.h" @@ -782,6 +783,9 @@ static int ext4_file_mmap(struct file *file, struct vm_area_struct *vma) } else { vma->vm_ops = &ext4_file_vm_ops; } + + khugepaged_enter_file(vma, vma->vm_flags); + return 0; } diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 5bddb1e9e0b3..d94144b1fb0f 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -30,6 +30,7 @@ #include #include #include +#include static const struct vm_operations_struct xfs_file_vm_ops; @@ -1407,6 +1408,9 @@ xfs_file_mmap( vma->vm_ops = &xfs_file_vm_ops; if (IS_DAX(inode)) vma->vm_flags |= VM_HUGEPAGE; + + khugepaged_enter_file(vma, vma->vm_flags); + return 0; }