From patchwork Wed Apr 3 01:32:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13614861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35C6FC6FD1F for ; Wed, 3 Apr 2024 01:32:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7CC16B0083; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2C9E6B0085; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F3FD6B0087; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6F8DB6B0083 for ; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E1F6CA0AA2 for ; Wed, 3 Apr 2024 01:32:57 +0000 (UTC) X-FDA: 81966496794.09.8AC0A95 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id C0B1D18000E for ; Wed, 3 Apr 2024 01:32:55 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OgBGW3lP; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712107975; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2ynux/F/qmq0vm7Pu7L1906YdTD8uR3G8JQyOTFf45Q=; b=6noBeyHsK6fRYS9Xg1N4ySyqbAjqmF51a/pFYrme5akn0PcLpAM2FIAEUnzLtAlaBu1EQ2 Qqjlw75XSG+ntkQzaXN9DpgEMOSI0lX3z4bdslFphRK6nL51L/S3AfrzbeRxW3BNIxuAs9 Qv7/zGBbPPpenEEFi9dYSeKEvU812Tc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OgBGW3lP; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712107975; a=rsa-sha256; cv=none; b=0A8b3wP1PmVC5Nrqm3gw7g7Zobxkel07nceAj03BVL5gWHk4+hSoJgddJHvRV0dK5Xz/Nk dVivyWxyf0zunGDgFHzTbTC0Q9D98ct8RM8wcj4BKMYGbxfRSufVNdpeyoeUXt84twWDAc 8FuUqcEgWmhVC48TbDiqNLK69iL+118= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712107975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2ynux/F/qmq0vm7Pu7L1906YdTD8uR3G8JQyOTFf45Q=; b=OgBGW3lPUzZxA4UCJn0IGnixIoKY8UHM/6x+I5I7D9oPk55bk03wJDd8bwNSVTRMOXvKzY CIWgdle8FwegMiKK4N6UtYzw150/vl9jz+iUi1YbUnscvy7WY+PwmDYoPa8LZ9SDDndrQW 2GI5C06gTyeFBxTg8pK/8yR2zSK44js= Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com [209.85.221.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-269-cLYm1ItgPPGjxYOItuB7CQ-1; Tue, 02 Apr 2024 21:32:54 -0400 X-MC-Unique: cLYm1ItgPPGjxYOItuB7CQ-1 Received: by mail-vk1-f200.google.com with SMTP id 71dfb90a1353d-4d87c150d6cso934682e0c.1 for ; Tue, 02 Apr 2024 18:32:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712107973; x=1712712773; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2ynux/F/qmq0vm7Pu7L1906YdTD8uR3G8JQyOTFf45Q=; b=dr81j49Z4qyX8I0gdPrvn/IEnoW00PGkq0XCAnVyC9Yd5U7/XpiVRxyquuT2vMcgCt Uf0Ij5i139SL+wLEylaTWRc3Sw/vH074n/MPqXGpg3jtodUoVyUh69U10QTUhiF0m4Vg 3H750XWYB6XUuLRmNTpG5VhRDJDARe2iIjb52qvEL5KLj7/f0O8PfOqrf/+mcLg+B+jS qJFoHbaEtSdK455GhtOoQgCcmAtVzD+GMzawjB5y62idpOzYrmSEVzQ1/LbItGSb7Npt zv5S3A07wjNsNCB2Mxz/brYKYLhOT2Fbwb6ndWtBqGsEX4x0ZnTUjfM/c1H5JECvNf3C Pz3g== X-Gm-Message-State: AOJu0Ywj38syQP5MqwRaxfeoYHdNRQgPFMW2PDZHpPfUN79Zdnddbon2 A1M9NPmGFCr/hV4LyvsnihyBxeVuk64jPdULFJBh10ZTbecHnAeuIo5qgLpplERczpJjH4d5FS1 USY4ZvlZWZDxoz6KxD/PtJqcPTeIaY0hxcNuvZRT38hbeVmqmqHdu2ss9/f8NtKXpx4NdKH7Knq VkCyZaWPbnQiw0B+hUDa1vRNQBdi94sA== X-Received: by 2002:a05:6102:588c:b0:478:9533:a75a with SMTP id ju12-20020a056102588c00b004789533a75amr2155422vsb.1.1712107972946; Tue, 02 Apr 2024 18:32:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE9+hYqxzCimkmspOuHcmCsRdwFLI47CvtrgBTMbK0H1Qo/OVIYE/CpCTqBmVGSH1xwK+vwHQ== X-Received: by 2002:a05:6102:588c:b0:478:9533:a75a with SMTP id ju12-20020a056102588c00b004789533a75amr2155389vsb.1.1712107972182; Tue, 02 Apr 2024 18:32:52 -0700 (PDT) Received: from x1n.redhat.com ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id qm18-20020a056214569200b0068ff8bda6c7sm6031687qvb.92.2024.04.02.18.32.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 18:32:51 -0700 (PDT) From: peterx@redhat.com To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Huacai Chen , peterx@redhat.com, David Hildenbrand , Jason Gunthorpe , Nathan Chancellor , Andrew Morton , Matthew Wilcox , WANG Xuerui , Ryan Roberts , loongarch@lists.linux.dev Subject: [PATCH 1/3] mm: Allow anon exclusive check over hugetlb tail pages Date: Tue, 2 Apr 2024 21:32:47 -0400 Message-ID: <20240403013249.1418299-2-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240403013249.1418299-1-peterx@redhat.com> References: <20240403013249.1418299-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: C0B1D18000E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: q9r3ypjj1y1ge3gmcec7tkhtthns1dey X-HE-Tag: 1712107975-720968 X-HE-Meta: U2FsdGVkX1/jTwRfhSU0SI8AJc5VavYcEQEL07jhpIm2QBIeqeqbjviyFKUbOa3SJhZocP1EyBS32U+b38bbhZgqIqTtIBdJ+5LU2uUBlBk8Iqj1ENXtL50QwLy+TNfP6jDinQLxdENrp8pOS54A+M0m4owDyroummeook3S/sebzKAvpzz9bkJxzMH0tL+sAcF4YcW+2XiyD2N92j8N09FNtSH8luXX3FvsdW0hf0pFD+n3sOwHGLbBkIWvxETqoYolNPPQaoSmLEjGBfzaDFfylyVBVA40onDcGJuU1wy1U44f8Bc1bLyd/1lh2DGVg+30lCIGiBS2SH+MYYX49Wmia9JSutbYHbiZ87LGNq4LPT2zqRlmU35M21Bj3hJ99JSXmBZ+XWN9B5zxJhbTfXYLmcUy/YYm2HzO3FP5NS4Z9iu4bowGQIO8MnPiEyTJJfo/108bGcm0/bhxlNFgtSoXrHyy4P84Pxlf/RZEeQ4YoQToHG9mkXjEpBkwnaHh9TgV7CDS7/EQkVMBPSWhpDbO/FK5W5SzamJUAu+Q9BO9wrYPjNZB0hVEURCejLyMNBjWd/EJ81MBlcj2lBWKECet5dJjoLayDWLagF3VIM9U5otGKqJFHs31x4pZozAPwoFrHutnfRlkz3++4978XA/aA9SzJltaP+n90CagnuXFdP0Vgu+JCxOPK0DZN83tV8cpfS4UtMUXGiwCcZBc0DvoBEvdGBQgU3PJtvHOZcxuL+LxbP83DBJEypYwDT6yP7dJkAKrkwpNeIFIUZyqS0Gkjm1++DfApBMUuX4wCQKm3Bm1x43LpwsCNiDzTCje9Ljej/W+L/sPteDY87AXAmknXVV49+VSnQyO8aSnwIHfMDCrfXZp5Jw7+/UV5cni8NtZvKfejeQGMvCVA/Kt9Ek8ZjuqGz+ljSIn2o770jG2h//IOr/OThXOytwpPYEiWJNg+6FCtRd1RaSnsi2 bZbvAXSK /SlT96JlMfgUoKJiIWipoyzFUoqaGsq0AYmcrHHRMCkuF0L1m95PXElWW941YewP/DJENIw/SI50h8WA5znVFd8xjJh9SMQsMRb4oDoSn9N6jsg32Tp62iS/xDhHptTELQouLnQuwbTBPE8Rcsz3NxMcHpoxkB9UFxyU9qf10Q3AyXotBZrMqQvKdJ4vWqd1T82djUmAU+WYd9xgPpJZv8yFLkFVhnJos+U5qtbm4ikYvdVUkcGIlY9uNG7K9Ixp1ehDL+UMxWDLywVRryPJhLliY+rObd0VoQDiSqjq/dnRk698KwYKc9O2MmgGM2tQ1Ga0Lx+dD1iFuKhKWwuMrB13BWXLLnM7u3gqvEHP1ft+bRzAcmEcrrKWmGQixDgtdiBopaxJM8xB5Z8Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu PageAnonExclusive() used to forbid tail pages for hugetlbfs, as that used to be called mostly in hugetlb specific paths and the head page was guaranteed. As we move forward towards merging hugetlb paths into generic mm, we may start to pass in tail hugetlb pages (when with cont-pte/cont-pmd huge pages) for such check. Allow it to properly fetch the head, in which case the anon-exclusiveness of the head will always represents the tail page. There's already a sign of it when we look at the fast-gup which already contain the hugetlb processing altogether: we used to have a specific commit 5805192c7b72 ("mm/gup: handle cont-PTE hugetlb pages correctly in gup_must_unshare() via GUP-fast") covering that area. Now with this more generic change, that can also go away. Signed-off-by: Peter Xu Acked-by: David Hildenbrand Signed-off-by: Andrew Morton --- include/linux/page-flags.h | 8 +++++++- mm/internal.h | 10 ---------- 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 888353c209c0..225357f48a79 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -1095,7 +1095,13 @@ PAGEFLAG(Isolated, isolated, PF_ANY); static __always_inline int PageAnonExclusive(const struct page *page) { VM_BUG_ON_PGFLAGS(!PageAnon(page), page); - VM_BUG_ON_PGFLAGS(PageHuge(page) && !PageHead(page), page); + /* + * Allow the anon-exclusive check to work on hugetlb tail pages. + * Here hugetlb pages will always guarantee the anon-exclusiveness + * of the head page represents the tail pages. + */ + if (PageHuge(page) && !PageHead(page)) + page = compound_head(page); return test_bit(PG_anon_exclusive, &PF_ANY(page, 1)->flags); } diff --git a/mm/internal.h b/mm/internal.h index 9512de7398d5..87f6e4fd56a5 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1259,16 +1259,6 @@ static inline bool gup_must_unshare(struct vm_area_struct *vma, if (IS_ENABLED(CONFIG_HAVE_FAST_GUP)) smp_rmb(); - /* - * During GUP-fast we might not get called on the head page for a - * hugetlb page that is mapped using cont-PTE, because GUP-fast does - * not work with the abstracted hugetlb PTEs that always point at the - * head page. For hugetlb, PageAnonExclusive only applies on the head - * page (as it cannot be partially COW-shared), so lookup the head page. - */ - if (unlikely(!PageHead(page) && PageHuge(page))) - page = compound_head(page); - /* * Note that PageKsm() pages cannot be exclusive, and consequently, * cannot get pinned.