From patchwork Mon Aug 21 23:48:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 13359941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77E05EE49AA for ; Mon, 21 Aug 2023 23:48:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F24AF280006; Mon, 21 Aug 2023 19:48:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED5F694000B; Mon, 21 Aug 2023 19:48:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9C51280006; Mon, 21 Aug 2023 19:48:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C69E394000B for ; Mon, 21 Aug 2023 19:48:52 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8C8BF160152 for ; Mon, 21 Aug 2023 23:48:52 +0000 (UTC) X-FDA: 81149754504.27.BCAC5B6 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf01.hostedemail.com (Postfix) with ESMTP id E38954000B for ; Mon, 21 Aug 2023 23:48:50 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=1vRA8NFD; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 34ffjZAcKCI4H62wwxwy66y3w.u64305CF-442Dsu2.69y@flex--zokeefe.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=34ffjZAcKCI4H62wwxwy66y3w.u64305CF-442Dsu2.69y@flex--zokeefe.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692661730; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=YCOyIsFUlLjTnVIlQjRMIQSr9BmEp4/1j6SRmVYuUQQ=; b=3cNH9yJgtqoKlmNx/wiTiiZ3SptzmOdaaFgCG6cIh1nQZeXRt66wX7C1+X2dPCYQv58iGv OOzqJZwGnCONG+2ZG0juGMNYz06rW1XwLj8gAsdAakAiVs4Ta8+HDn4wbdXAKbykeITCmx WM/17fjzvhivP4wg8gCk34Lcttanigg= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=1vRA8NFD; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 34ffjZAcKCI4H62wwxwy66y3w.u64305CF-442Dsu2.69y@flex--zokeefe.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=34ffjZAcKCI4H62wwxwy66y3w.u64305CF-442Dsu2.69y@flex--zokeefe.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692661730; a=rsa-sha256; cv=none; b=QPZPnOgwEo3Ar6Q+pwulwG10axxvA/dCxJBCdT2vrUJLnm8P2AiPAiXeyYQ98qz9tJ4fK3 OEHZX85jJnx6a4lamHxDCIHhca8Yicj0Wd3elcygS7uZDsyDlqVlVplvtAo6leXrs812r8 /OE0y7fYTg+RbbQ8i0UCZtFC/4jWIH8= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-58c583f885cso53336407b3.1 for ; Mon, 21 Aug 2023 16:48:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692661730; x=1693266530; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=YCOyIsFUlLjTnVIlQjRMIQSr9BmEp4/1j6SRmVYuUQQ=; b=1vRA8NFDehhZixvfUqi0M0SQpQ2MFDSs44seZB949NKKzBZzpVSxGDWwO/LNrXjrHy Ada9q6DbnJx8rN+sOeayQJ4TFmInwlfhnYp0unQOmBy2T+vbbSdOyVnplEv2mJvZ+Dff SWMYWuIB0eBOo1HwE8yLGk6cfIC2ia61Kt6fyO4lPPzrlwqb072OXA+mpUn3Wz61zk70 yvfG0y1E/5HgQJxDdxwmvUU40pkNMUL4jVvCsxGbvVM+JZAZ6YkxUZ7RnqYUA7xg5wq+ Pn/vFjpRIqpYBlKiPfoBO6l4+ZHwhY2Abs59vVQqDFyNamEoftAu3ozdwKw/sjtLR59T lppw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692661730; x=1693266530; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=YCOyIsFUlLjTnVIlQjRMIQSr9BmEp4/1j6SRmVYuUQQ=; b=bfRFGEAY2T3U2SogcblhCy5q0ynXwLrDsToPDTvByhuEZ7J7SpXvxmhIn8NutRgnit 0Z7P/c7Opx8Y95cGAwm8oNnTCXbdqiJChUpIGmjRmgv01TBGXSGobZg7nzQoqNfFHbrD gjqPxfAtwS3NiS8NeWtlZFjbI+uhOKX2xJgRGLdQ1bL11GhaIqgA1/hRVMct6+X5FPDc oL4KvhYLXxxa5fucunjoX920OkusY2B0TAYAUr4tadIOID5vv+cz7ptCvkOWYHQ5bfkN WOYRPrdphqZQrzdBN9Jf0oJ4uuYRGgMRo8uD9RmCn19/+QGClMyRBsUkoaOma986j8Md Wrtg== X-Gm-Message-State: AOJu0Yxc+bmW98GCAIzce9CoJQPjL+fLKgIfR02cofQRrHJr79MKbG22 x1S5ic/3wjrn0NcKb+HxzJ2Ipe4dHsnVHilhZqJvLICqszlPubzaeLP2b2L0WcZE27Zqi7tdZOM p6qD0GmnnyA5OOCDH8xkdwOHVhKxqa9SieFfG4TvnqJNj2WRjzR+1uLC6KII= X-Google-Smtp-Source: AGHT+IF/HIypG9hKuqBUy9shzQU2f2lrrmetedwPUcFRd7NrkxJpYSdhCJsQY85/dvUbv4G15qj30HF917UZ X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a81:4304:0:b0:577:619e:d3c9 with SMTP id q4-20020a814304000000b00577619ed3c9mr79027ywa.10.1692661729731; Mon, 21 Aug 2023 16:48:49 -0700 (PDT) Date: Mon, 21 Aug 2023 16:48:44 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog Message-ID: <20230821234844.699818-1-zokeefe@google.com> Subject: [PATCH v3] mm/thp: fix "mm: thp: kill __transhuge_page_enabled()" From: "Zach O'Keefe" To: linux-mm@kvack.org, Yang Shi Cc: linux-kernel@vger.kernel.org, "Zach O'Keefe" , Saurabh Singh Sengar X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E38954000B X-Stat-Signature: b1mfyyqs5yu48ajw114ah45d4wgtx4yn X-HE-Tag: 1692661730-547943 X-HE-Meta: U2FsdGVkX1/4nSVWoNWAvW8K3a/JLzLLZ3hIt8wf/Nj4NJ9uhIu8X7AKkAiV1XVXRqz1WkbWdXLDI8s8+7Fre6KgzeJlFqC6o2QnJuxZgYdrm1eAMF2U140gB1zSOTj9MwRGhCoWRgrKUXRpFZHhx1eofU0NymR3FYLIWQk20tyP8pKH7zL6YA9m5VSXWMbCw5aLOmTDwQtNuERs09aHx2slkAY3DZFYxBvTCpTynT5+3ZvcLXi6qr/sGXLYfZEjWnuUWAnNksjR5CZ5+A9k3NUBbV1SIGI44DZYj7NtyKlbQx8kCU4LnZ2s4tct75z2GAnl7xJErOnpvEglUbzzafe6xNwNvtyYZU1V+QFcJ/5sPjAeZRQkBEpkm5UZOGWTXMCT8C4VHEklud1T5HuGW4Y5OeRSOWnamt3pK+BZXejdWAvr2BQObl4EFeMWi//o4XL74IZ4sLPqGf5PJYGe6CRfbxdNPpcGImRPuJk0frn/BkJ3E4TGJsgTES9VODZaEOSw9orz7NKA1tci0VQ1RiygtZeSn4EpxqQVWZ3+FfUX3Ye/POqURPDxX1GUtjmkdn8Q2vntZ8jj986U69Nb1b3dQtIyG/ge1oIexHSC91eDQapTRZUqUCFVP2nEX97wh5lnnbF+BwltO090hTMOCsPl3ExFTZSoEZKu/Nd5/+6QLfLJ/xeF4FwQCBnDtIZqzsbrg7wj+oX0fMDKThhtLhH0S7zTQxx7IfSUp6h70jw17TFpV81c80h7MNOJtpDD9FPy4NXq2xtxsXRqx8S8Db0sjq9XX6Ta2eRolTHQh4kAZwj0sd7OrJ1AJwLWvdXNfbq49PV/vZS6BBICenI0OIp7EneM9iJSrONDKN2P4a1Y8AKY6DBUS9BwqrheGjQCC7vfiNJ3YhhC8JdcDl7rf+ejPukghaw1Ckynn1Lu1teNSjk7cyCp2WeLkwWnJYM6Q3YqL6ZtIXn8I13nrBe lbiPEJLB T0qKMtcCdfrH5P1egy8QTKg0F29kMx4A5hb5YRlFdpZfLrmx2XwzzQpAaN10y1eClKNJ4Gs/KOr1Kx/+v+mA7z41mxs1q47Qbfy/lTwZBHX6MwBBfX8UiX3Zu9WtKtRXTYyKNKsEOHunTwxq01VhFfcEE4RcV/UL47iXrWwrhEJoLc5F7dxyWFapiF05P6KdL72JySDMABqRVWQGZigaYPt7jIuDWkg7u6HRv+RmdWSRjATw0IwVkzWS0MXjW//sk4dC2Z+Az3tf7LB+y8/YbSKbIFDFElL4sok4i2yldec18FmmJdN8ETiIuvkl4Nlo8yqtMbDyzl5BADmgJIG+MesAUQ47KQyA2IwXVeij4slgdUIx8yoKmCkQxziTkEeHvb4RTcua8P5ontjYkfS9lbzJD7jn2A5cLw74HcHMgwXv9uNDkiF0U9OhT+7jDkqNuHbq4FftdOGGNBuwP/UAPoXpnIoattnqfUXjty1nNW5HPjihmd/ZL/kOadYXrUHUFTYY4BgQefRoeyAX2kTuYOF8nxMgSl8gS7PNLjy+3t+a+QlV8JatEs+no3UO5qfGHa88UdE1Tb6GMm3Bu8vItZBTbaMcJq/SBj9j/bJg7hgXF+wuHF5SZD5d+ws9oUMF7922pukJWzZZha6EBHZk7kbnaFbALoo83CZqyFvGYmigzBQCVD0szpEBBrx74Mh6INJ1NBspMmLOQSZgDxSnF/KYGcYHWQe1yP7Cu/JQacBTBHGpZn03UsNNiZUeOVbuIxvO+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The 6.0 commits: commit 9fec51689ff6 ("mm: thp: kill transparent_hugepage_active()") commit 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()") merged "can we have THPs in this VMA?" logic that was previously done separately by fault-path, khugepaged, and smaps "THPeligible" checks. During the process, the semantics of the fault path check changed in two ways: 1) A VM_NO_KHUGEPAGED check was introduced (also added to smaps path). 2) We no longer checked if non-anonymous memory had a vm_ops->huge_fault handler that could satisfy the fault. Previously, this check had been done in create_huge_pud() and create_huge_pmd() routines, but after the changes, we never reach those routines. During the review of the above commits, it was determined that in-tree users weren't affected by the change; most notably, since the only relevant user (in terms of THP) of VM_MIXEDMAP or ->huge_fault is DAX, which is explicitly approved early in approval logic. However, there is at least one occurrence where an out-of-tree driver that used VM_HUGEPAGE|VM_MIXEDMAP with a vm_ops->huge_fault handler, was broken. Remove the VM_NO_KHUGEPAGED check when not in collapse path and give any ->huge_fault handler a chance to handle the fault. Note that we don't validate the file mode or mapping alignment, which is consistent with the behavior before the aforementioned commits. Fixes: 7da4e2cb8b1f ("mm: thp: kill __transhuge_page_enabled()") Reported-by: Saurabh Singh Sengar Signed-off-by: Zach O'Keefe Cc: Yang Shi --- Changed from v2[1]: - Fixed false negative in smaps check when !dax && ->huge_fault Changed from v1[2]: - [Saurabhi] Allow ->huge_fault handler to handle fault, if it exists There are some logical holes in smaps' THPeligible checks here, but those are best dealt with in follow-up patches. For now, just make sure the fault path is dealt with. [1] https://lore.kernel.org/linux-mm/20230818211533.2523697-1-zokeefe@google.com/ [2] https://lore.kernel.org/linux-mm/CAAa6QmQw+F=o6htOn=6ADD6mwvMO=Ow_67f3ifBv3GpXx9Xg_g@mail.gmail.com/ --- mm/huge_memory.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index eb3678360b97..901dcf8db8d2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -96,11 +96,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, return in_pf; /* - * Special VMA and hugetlb VMA. + * khugepaged special VMA and hugetlb VMA. * Must be checked after dax since some dax mappings may have * VM_MIXEDMAP set. */ - if (vm_flags & VM_NO_KHUGEPAGED) + if (!in_pf && !smaps && (vm_flags & VM_NO_KHUGEPAGED)) return false; /* @@ -128,12 +128,18 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, !hugepage_flags_always()))) return false; - /* Only regular file is valid */ - if (!in_pf && file_thp_enabled(vma)) - return true; - - if (!vma_is_anonymous(vma)) + if (!vma_is_anonymous(vma)) { + /* + * Trust that ->huge_fault() handlers know what they are doing + * in fault path. + */ + if (((in_pf || smaps)) && vma->vm_ops->huge_fault) + return true; + /* Only regular file is valid in collapse path */ + if (((!in_pf || smaps)) && file_thp_enabled(vma)) + return true; return false; + } if (vma_is_temporary_stack(vma)) return false;