From patchwork Wed Apr 3 01:32:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13614861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35C6FC6FD1F for ; Wed, 3 Apr 2024 01:32:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A7CC16B0083; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2C9E6B0085; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F3FD6B0087; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6F8DB6B0083 for ; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E1F6CA0AA2 for ; Wed, 3 Apr 2024 01:32:57 +0000 (UTC) X-FDA: 81966496794.09.8AC0A95 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id C0B1D18000E for ; Wed, 3 Apr 2024 01:32:55 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OgBGW3lP; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712107975; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2ynux/F/qmq0vm7Pu7L1906YdTD8uR3G8JQyOTFf45Q=; b=6noBeyHsK6fRYS9Xg1N4ySyqbAjqmF51a/pFYrme5akn0PcLpAM2FIAEUnzLtAlaBu1EQ2 Qqjlw75XSG+ntkQzaXN9DpgEMOSI0lX3z4bdslFphRK6nL51L/S3AfrzbeRxW3BNIxuAs9 Qv7/zGBbPPpenEEFi9dYSeKEvU812Tc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OgBGW3lP; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712107975; a=rsa-sha256; cv=none; b=0A8b3wP1PmVC5Nrqm3gw7g7Zobxkel07nceAj03BVL5gWHk4+hSoJgddJHvRV0dK5Xz/Nk dVivyWxyf0zunGDgFHzTbTC0Q9D98ct8RM8wcj4BKMYGbxfRSufVNdpeyoeUXt84twWDAc 8FuUqcEgWmhVC48TbDiqNLK69iL+118= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712107975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2ynux/F/qmq0vm7Pu7L1906YdTD8uR3G8JQyOTFf45Q=; b=OgBGW3lPUzZxA4UCJn0IGnixIoKY8UHM/6x+I5I7D9oPk55bk03wJDd8bwNSVTRMOXvKzY CIWgdle8FwegMiKK4N6UtYzw150/vl9jz+iUi1YbUnscvy7WY+PwmDYoPa8LZ9SDDndrQW 2GI5C06gTyeFBxTg8pK/8yR2zSK44js= Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com [209.85.221.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-269-cLYm1ItgPPGjxYOItuB7CQ-1; Tue, 02 Apr 2024 21:32:54 -0400 X-MC-Unique: cLYm1ItgPPGjxYOItuB7CQ-1 Received: by mail-vk1-f200.google.com with SMTP id 71dfb90a1353d-4d87c150d6cso934682e0c.1 for ; Tue, 02 Apr 2024 18:32:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712107973; x=1712712773; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2ynux/F/qmq0vm7Pu7L1906YdTD8uR3G8JQyOTFf45Q=; b=dr81j49Z4qyX8I0gdPrvn/IEnoW00PGkq0XCAnVyC9Yd5U7/XpiVRxyquuT2vMcgCt Uf0Ij5i139SL+wLEylaTWRc3Sw/vH074n/MPqXGpg3jtodUoVyUh69U10QTUhiF0m4Vg 3H750XWYB6XUuLRmNTpG5VhRDJDARe2iIjb52qvEL5KLj7/f0O8PfOqrf/+mcLg+B+jS qJFoHbaEtSdK455GhtOoQgCcmAtVzD+GMzawjB5y62idpOzYrmSEVzQ1/LbItGSb7Npt zv5S3A07wjNsNCB2Mxz/brYKYLhOT2Fbwb6ndWtBqGsEX4x0ZnTUjfM/c1H5JECvNf3C Pz3g== X-Gm-Message-State: AOJu0Ywj38syQP5MqwRaxfeoYHdNRQgPFMW2PDZHpPfUN79Zdnddbon2 A1M9NPmGFCr/hV4LyvsnihyBxeVuk64jPdULFJBh10ZTbecHnAeuIo5qgLpplERczpJjH4d5FS1 USY4ZvlZWZDxoz6KxD/PtJqcPTeIaY0hxcNuvZRT38hbeVmqmqHdu2ss9/f8NtKXpx4NdKH7Knq VkCyZaWPbnQiw0B+hUDa1vRNQBdi94sA== X-Received: by 2002:a05:6102:588c:b0:478:9533:a75a with SMTP id ju12-20020a056102588c00b004789533a75amr2155422vsb.1.1712107972946; Tue, 02 Apr 2024 18:32:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE9+hYqxzCimkmspOuHcmCsRdwFLI47CvtrgBTMbK0H1Qo/OVIYE/CpCTqBmVGSH1xwK+vwHQ== X-Received: by 2002:a05:6102:588c:b0:478:9533:a75a with SMTP id ju12-20020a056102588c00b004789533a75amr2155389vsb.1.1712107972182; Tue, 02 Apr 2024 18:32:52 -0700 (PDT) Received: from x1n.redhat.com ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id qm18-20020a056214569200b0068ff8bda6c7sm6031687qvb.92.2024.04.02.18.32.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 18:32:51 -0700 (PDT) From: peterx@redhat.com To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Huacai Chen , peterx@redhat.com, David Hildenbrand , Jason Gunthorpe , Nathan Chancellor , Andrew Morton , Matthew Wilcox , WANG Xuerui , Ryan Roberts , loongarch@lists.linux.dev Subject: [PATCH 1/3] mm: Allow anon exclusive check over hugetlb tail pages Date: Tue, 2 Apr 2024 21:32:47 -0400 Message-ID: <20240403013249.1418299-2-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240403013249.1418299-1-peterx@redhat.com> References: <20240403013249.1418299-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: C0B1D18000E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: q9r3ypjj1y1ge3gmcec7tkhtthns1dey X-HE-Tag: 1712107975-720968 X-HE-Meta: U2FsdGVkX1/jTwRfhSU0SI8AJc5VavYcEQEL07jhpIm2QBIeqeqbjviyFKUbOa3SJhZocP1EyBS32U+b38bbhZgqIqTtIBdJ+5LU2uUBlBk8Iqj1ENXtL50QwLy+TNfP6jDinQLxdENrp8pOS54A+M0m4owDyroummeook3S/sebzKAvpzz9bkJxzMH0tL+sAcF4YcW+2XiyD2N92j8N09FNtSH8luXX3FvsdW0hf0pFD+n3sOwHGLbBkIWvxETqoYolNPPQaoSmLEjGBfzaDFfylyVBVA40onDcGJuU1wy1U44f8Bc1bLyd/1lh2DGVg+30lCIGiBS2SH+MYYX49Wmia9JSutbYHbiZ87LGNq4LPT2zqRlmU35M21Bj3hJ99JSXmBZ+XWN9B5zxJhbTfXYLmcUy/YYm2HzO3FP5NS4Z9iu4bowGQIO8MnPiEyTJJfo/108bGcm0/bhxlNFgtSoXrHyy4P84Pxlf/RZEeQ4YoQToHG9mkXjEpBkwnaHh9TgV7CDS7/EQkVMBPSWhpDbO/FK5W5SzamJUAu+Q9BO9wrYPjNZB0hVEURCejLyMNBjWd/EJ81MBlcj2lBWKECet5dJjoLayDWLagF3VIM9U5otGKqJFHs31x4pZozAPwoFrHutnfRlkz3++4978XA/aA9SzJltaP+n90CagnuXFdP0Vgu+JCxOPK0DZN83tV8cpfS4UtMUXGiwCcZBc0DvoBEvdGBQgU3PJtvHOZcxuL+LxbP83DBJEypYwDT6yP7dJkAKrkwpNeIFIUZyqS0Gkjm1++DfApBMUuX4wCQKm3Bm1x43LpwsCNiDzTCje9Ljej/W+L/sPteDY87AXAmknXVV49+VSnQyO8aSnwIHfMDCrfXZp5Jw7+/UV5cni8NtZvKfejeQGMvCVA/Kt9Ek8ZjuqGz+ljSIn2o770jG2h//IOr/OThXOytwpPYEiWJNg+6FCtRd1RaSnsi2 bZbvAXSK /SlT96JlMfgUoKJiIWipoyzFUoqaGsq0AYmcrHHRMCkuF0L1m95PXElWW941YewP/DJENIw/SI50h8WA5znVFd8xjJh9SMQsMRb4oDoSn9N6jsg32Tp62iS/xDhHptTELQouLnQuwbTBPE8Rcsz3NxMcHpoxkB9UFxyU9qf10Q3AyXotBZrMqQvKdJ4vWqd1T82djUmAU+WYd9xgPpJZv8yFLkFVhnJos+U5qtbm4ikYvdVUkcGIlY9uNG7K9Ixp1ehDL+UMxWDLywVRryPJhLliY+rObd0VoQDiSqjq/dnRk698KwYKc9O2MmgGM2tQ1Ga0Lx+dD1iFuKhKWwuMrB13BWXLLnM7u3gqvEHP1ft+bRzAcmEcrrKWmGQixDgtdiBopaxJM8xB5Z8Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu PageAnonExclusive() used to forbid tail pages for hugetlbfs, as that used to be called mostly in hugetlb specific paths and the head page was guaranteed. As we move forward towards merging hugetlb paths into generic mm, we may start to pass in tail hugetlb pages (when with cont-pte/cont-pmd huge pages) for such check. Allow it to properly fetch the head, in which case the anon-exclusiveness of the head will always represents the tail page. There's already a sign of it when we look at the fast-gup which already contain the hugetlb processing altogether: we used to have a specific commit 5805192c7b72 ("mm/gup: handle cont-PTE hugetlb pages correctly in gup_must_unshare() via GUP-fast") covering that area. Now with this more generic change, that can also go away. Signed-off-by: Peter Xu Acked-by: David Hildenbrand Signed-off-by: Andrew Morton --- include/linux/page-flags.h | 8 +++++++- mm/internal.h | 10 ---------- 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 888353c209c0..225357f48a79 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -1095,7 +1095,13 @@ PAGEFLAG(Isolated, isolated, PF_ANY); static __always_inline int PageAnonExclusive(const struct page *page) { VM_BUG_ON_PGFLAGS(!PageAnon(page), page); - VM_BUG_ON_PGFLAGS(PageHuge(page) && !PageHead(page), page); + /* + * Allow the anon-exclusive check to work on hugetlb tail pages. + * Here hugetlb pages will always guarantee the anon-exclusiveness + * of the head page represents the tail pages. + */ + if (PageHuge(page) && !PageHead(page)) + page = compound_head(page); return test_bit(PG_anon_exclusive, &PF_ANY(page, 1)->flags); } diff --git a/mm/internal.h b/mm/internal.h index 9512de7398d5..87f6e4fd56a5 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1259,16 +1259,6 @@ static inline bool gup_must_unshare(struct vm_area_struct *vma, if (IS_ENABLED(CONFIG_HAVE_FAST_GUP)) smp_rmb(); - /* - * During GUP-fast we might not get called on the head page for a - * hugetlb page that is mapped using cont-PTE, because GUP-fast does - * not work with the abstracted hugetlb PTEs that always point at the - * head page. For hugetlb, PageAnonExclusive only applies on the head - * page (as it cannot be partially COW-shared), so lookup the head page. - */ - if (unlikely(!PageHead(page) && PageHuge(page))) - page = compound_head(page); - /* * Note that PageKsm() pages cannot be exclusive, and consequently, * cannot get pinned. From patchwork Wed Apr 3 01:32:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13614862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62F2CCD1284 for ; Wed, 3 Apr 2024 01:33:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F10966B0087; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC0326B0088; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3AD96B0089; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AE9026B0087 for ; Tue, 2 Apr 2024 21:32:58 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7ACEC1C0D98 for ; Wed, 3 Apr 2024 01:32:58 +0000 (UTC) X-FDA: 81966496836.30.44CB373 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 79D48C000F for ; Wed, 3 Apr 2024 01:32:56 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SY6AX4Ij; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712107976; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UqFcRORApCbbLwHiV/NvWvcvg5AC7kP7mUGUslgtJS8=; b=p5G7W6N1mZsHZBdeeSjUuzsgqSqcKcTg9r7LzD+979WXVWpxrLx9vPLpOxkrGi8vFS3KJW msWA6OgOGPemMV5qejbCUjP3F4wyMlQaptvSy662rUPSHaUSE7aHJSuDD355J3VGistY/S oDSdMGCbR0HrqjWykz1FNxYS0RR/+dc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712107976; a=rsa-sha256; cv=none; b=dIkgmIQ8oqtYLCRnJecECHWkg6nQNOmO0UBJ31+uLqGq7ja3BZlQssf1sMRXZBbLMRWnLx VVVhdVzKF2JEEKbZXTvdU8x2LyY8zWINiP4ucd7MjwBrhi06adnS+oKIwVqy6PjdqoNXwX fiI64b0/+xAamGlzI8YEpPWc84rLFuU= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SY6AX4Ij; spf=pass (imf10.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712107975; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UqFcRORApCbbLwHiV/NvWvcvg5AC7kP7mUGUslgtJS8=; b=SY6AX4IjOjJimQe8gDAgQI5wTphgDRV6VTtHnamBJKu55d/xBMUL3f86YAmBjijftq+/Ow b6iYKcflMjBzILLVet5LkaWENNNcbJyV5eMK2K8AE9L3siYWfExHDVhMaKSFZgvaXKrGpz nteXs9Jrdyam3rIYHNWW5yozWQ4ZZWY= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-311-ke7RIBtFNcme-ZzQXkXC0Q-1; Tue, 02 Apr 2024 21:32:54 -0400 X-MC-Unique: ke7RIBtFNcme-ZzQXkXC0Q-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-69925f44915so937346d6.1 for ; Tue, 02 Apr 2024 18:32:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712107974; x=1712712774; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UqFcRORApCbbLwHiV/NvWvcvg5AC7kP7mUGUslgtJS8=; b=DtP8bmYdGzZVx4a/2TaJYuhqty4pdjnsHaU16gZDAQsNOWUCFH3kZp9RdFa/50nRTa nd5VGhcqm0EQD11PGTgIxteAiZhbCqXF2wsnYo6FMGdFT6keINQZ1XU0BXYser8ENVoN 9JxxtNtjEOdVyC5fw2PCTSlchu5B335eQaJamB3q9Ct1Vi8dNNTwXAHMTO/IHyTJ0FSm e1dqsFOtZkbiK5uW+0+7ay2gmZiXSqZlrO0RnBbbqu9b3ngp8VdVSyjshEfBppeU6GE/ WPoIjUj/seVu4A4jLYxE8+5oyAolPcgXSNFmolsWcVYSahXtUJpvJbZynuwC/flBdlTL G1yg== X-Gm-Message-State: AOJu0Yxut96bFWd2sXJWGNHRWKVK2jj+GsP1t5CDij5YuBqDE6lqOunF +u1WcBOBfLqJ7ucQWdhpcB0UP0TLy329gj/vLaKWAB5lmymSW1TBnjJeq6Ps3UvMXaKoxr8bwiM IvuHzWpgyX8DGuFg/7Fkl6g8DCa4k+rDEHstGG+A4mjn4IdkdwzT/nxsfbKsAnOwnmECoyJ94Fq ryFRBA/1jZ2XPhhqr+UXE3NcBPZX2Hew== X-Received: by 2002:a05:6214:946:b0:698:ff21:99f6 with SMTP id dn6-20020a056214094600b00698ff2199f6mr9007664qvb.3.1712107973906; Tue, 02 Apr 2024 18:32:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFdeiJvxhuNcK+1vYTZU0OZ0cczYLrBalyCYqNCwtKHKX2kkX+lMyBxEcKCdgdQrvrdhUD2Gg== X-Received: by 2002:a05:6214:946:b0:698:ff21:99f6 with SMTP id dn6-20020a056214094600b00698ff2199f6mr9007637qvb.3.1712107973351; Tue, 02 Apr 2024 18:32:53 -0700 (PDT) Received: from x1n.redhat.com ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id qm18-20020a056214569200b0068ff8bda6c7sm6031687qvb.92.2024.04.02.18.32.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 18:32:53 -0700 (PDT) From: peterx@redhat.com To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Huacai Chen , peterx@redhat.com, David Hildenbrand , Jason Gunthorpe , Nathan Chancellor , Andrew Morton , Matthew Wilcox , WANG Xuerui , Ryan Roberts , loongarch@lists.linux.dev Subject: [PATCH 2/3] fixup! mm/gup: handle huge pmd for follow_pmd_mask() Date: Tue, 2 Apr 2024 21:32:48 -0400 Message-ID: <20240403013249.1418299-3-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240403013249.1418299-1-peterx@redhat.com> References: <20240403013249.1418299-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Stat-Signature: k83szdtkfgym5xwtq7dd91fygjnib43r X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 79D48C000F X-Rspam-User: X-HE-Tag: 1712107976-400815 X-HE-Meta: U2FsdGVkX18zf3gNGHONMOTlPOFrBLXNO96/3VRkVMJhRHyxOj+81BYp4RaP5cqrg90xYw7ZcttFefcaKR70+Zf+IC3cJDQ8TwrrHx/5ksl2M1djFQZeHlwYCe4WujVhjcJ9CN/yGIPniqTDBcsiMZFBpvbqF2QIvRmpAVvj6myX42DsaPNNXcWkJ6Zl2QRMgu91Z9a/2JpTzvOPVwWldMTJ2GTPvuZ+20aDWq05WAfnkHQt+LlTajheILXqCrpmVaFLqD+lv6J0spp4jIPEUAg4Zm/A2+dh9dfWHKo9nNW9hf2z4qdFjQW2FI4FDp0kLjZEZ98I3RPcRIf7xK7OyfEVxrul4U+9x3QPK3bhAsMdKV+buE20Qc2mZm8s9IVnu2endwu4MQP1uZQWgRh8EFTHszpdeIo/mzFWCOB95UVMXOy7WRAMF8xRoTovgSsW4eeGmPFXrgmrYTcuwtH+9p4IChCOdn+owXhCMTf87TNM16GvI8ONbI1Q8y2D5M7zbFcX6ShOSCyE20EP/YhOwSpEc1UWGWWQTc+F7nM7rzzB98AMe/otlFlPvlQzPFILI9LGJIYoOu/jABDutoga9Qoy1iSzNxqAhIJdHB5gADiDP7V2LDNso6QfOIJhyrIeUhqejPhMAly79xdaW9uJ92X952MK1x4v3OHoWiNk5yLj0Tqpydqe7hUPCWcas1jgsNhR5kBL8f01xvu3FCMTK+ORXAv4uZgpofVDrqQPE0l2fVrAE1jZSglyP4GLoXDZiNBFbla6JS/6WU0UOOhF8jQwA6NkqcnDptJi4kZ54XUUSS37TCkFUvaaRzyfI8ECwNbBG28KvwlHYzRI4QKZ5zj7r0XWZj9xtZrR7D2p6Vo4WV9vzyIlQkKm0ERF7pTIDP8IFlQH2Te35zwXYZA+0xnYqAqkQIWNdJ3gMeidoES6UWlxtg/iiiM0gnaykyEGGIAKKj/sQrxPbDVR+62 qxbTBgQH ZDl3xysKsq+KI+BSj6/btrw3gzCqoHZ9h3nTv7vV9ZLv/4c5DIQXkHEFqH/L4yqCWGnLJC9rfSpjMVnOwQMtFdWaTHG7jdDbaGAlTK5+NU5lWfSVQjnH/gpPivlR26ILcL9tpwloMTGgzle68pFOb5ytywxQsG3pq2NqVQ/IaiMdVwUnyY+AkGu8HWRvgmqckz64JDkLYoxFQMMi8q1EhGGjelaXe9fterapMtvQkMawzqCxyHLGbEvdTpB4xbZPpoGW43qT026MEnFQ/2zzXni+gj6klEfGRT94Vxz6DwMwNpOt7sbFu2y3Qkaz8Gw2kazlRQA8aNHc1Uxuyd9VkV+HGMG/JpMP7+9aKG1C7pVKa0q6PqJ6OfCUnXHqaNdtE0d2ObmLJKaAFoZCisqhg9BbnSLA3+mEV9oYeC5OS0/M/joXokjVIPWokq7seW8zn5B0iAIZ/+HSMzAjVyfJPEWkXB2Nu9s5nCtIPaCH1+qCUbwDKQ3dPP3RWalANLUufJwC2LkX/9a9aHtsdAAwxmBaIPg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Allow follow_pmd_mask() to take hugetlb tail pages. The old warnings do not help now as hugetlb now allows it to happen, so drop them. Reported-by: Ryan Roberts Signed-off-by: Peter Xu --- mm/gup.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 91d70057aea0..d60b63fcfc82 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -775,8 +775,6 @@ static struct page *follow_huge_pmd(struct vm_area_struct *vma, assert_spin_locked(pmd_lockptr(mm, pmd)); page = pmd_page(pmdval); - VM_BUG_ON_PAGE(!PageHead(page) && !is_zone_device_page(page), page); - if ((flags & FOLL_WRITE) && !can_follow_write_pmd(pmdval, page, vma, flags)) return NULL; @@ -805,7 +803,6 @@ static struct page *follow_huge_pmd(struct vm_area_struct *vma, page += (addr & ~HPAGE_PMD_MASK) >> PAGE_SHIFT; ctx->page_mask = HPAGE_PMD_NR - 1; - VM_BUG_ON_PAGE(!PageCompound(page) && !is_zone_device_page(page), page); return page; } From patchwork Wed Apr 3 01:32:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13614864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 974C7CD11C2 for ; Wed, 3 Apr 2024 01:33:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D7BB6B0088; Tue, 2 Apr 2024 21:33:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 460EB6B0089; Tue, 2 Apr 2024 21:33:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D9DD6B008A; Tue, 2 Apr 2024 21:33:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0472F6B0088 for ; Tue, 2 Apr 2024 21:32:59 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B06FDA1033 for ; Wed, 3 Apr 2024 01:32:59 +0000 (UTC) X-FDA: 81966496878.21.6B1D61A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 99352120017 for ; Wed, 3 Apr 2024 01:32:57 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FdxrEUo6; spf=pass (imf29.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712107977; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PR/NBCt+5qJ4X31foyrecLlAlU9UbBCiiBiwAD7jaH8=; b=EDukoSVwT9Zm87qt3sLG86372z0mOAN1bYk6vvDIIl75THchIfTap70bp+ZdJ/L7kgXt8S vxTgAQrIERJpeCxBf5Sh5ZjAEEmQMby+sm1vtrFlHWwWu6/pUfPVOM/vTeQyty0Hh2fUTn rutBGUTkjOVxebNOWpT/TmYe7y8I94E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712107977; a=rsa-sha256; cv=none; b=bMzBAqVIMJl/cLjbYE61c+tGrjcEdPQ/ZFx9M2TLALID+fBYNtsvIwUGNtAAxN3J3XmvtJ 3CUD+c69PFkwpTZv3uw+ogPLN0tJoRSMsR5xEM4syL73MPPJrWcwsCCFIKzFfmmjosOVo5 r9BZBBMukZhtXFF3MhHcS1wM5ogVH9o= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FdxrEUo6; spf=pass (imf29.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712107977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PR/NBCt+5qJ4X31foyrecLlAlU9UbBCiiBiwAD7jaH8=; b=FdxrEUo6wFSgOOFz7oF5QcO+6f5q47MHFDEAYapSATTxnembeBln8wXXAuxo85BrJY2kU5 ylmEUeGvGuCMmu2/tM6OxHQ+djz0/xE50pPskDl2hCM49Tm2+0oXTLN4TcCnyAzRcQZQA9 AmkBzrCTRxSUYpuR5u4By93PNHzlNcw= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-274-6w4bW0YGMwWaNa4oxPjaSQ-1; Tue, 02 Apr 2024 21:32:55 -0400 X-MC-Unique: 6w4bW0YGMwWaNa4oxPjaSQ-1 Received: by mail-qk1-f199.google.com with SMTP id af79cd13be357-78a547efbaeso48384685a.0 for ; Tue, 02 Apr 2024 18:32:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712107975; x=1712712775; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PR/NBCt+5qJ4X31foyrecLlAlU9UbBCiiBiwAD7jaH8=; b=NyB1Um+lIpiCF7bpfE3RN4lY9cnOFS7b4YypQ/ppFnxLdEueeASAYXOSeXVJDk02zp obNa9FhUJcdujvu//odB6H53jo7XKbBNnSftAmBHihKE/I8WQ1uw8x/41n2qomqNZ3CZ V5dLTZTurH7VlgVwOl9wkrFI/SLXNXWGy0ag+BERDvkQlnGf/XNPUL4WX1KpYYjbEWlB pXgSnd7MemqG3p6Bibh5yyaw9pzEQzHzIw5lszAbY18mmqzyun7DUMUWI9dAKl4FkC55 kjmvvsRJkbJpR1DovTsSJC4/jIaDNMZIBBdZGGCatZ0Rj4egpWLvJooF+XGxCKXtqgjX L8Vw== X-Gm-Message-State: AOJu0YwvAY3wRI6CYHx6LMMX1MNGemA5y4NG3PBOnv1CE/G2aXExVXE3 wW30LAK9OdnwOmhz9xEv+SqhssUDpHqorZi4SK3JGqqcavadoVleayIzpRTWNkLCBIJepx3vi27 PqG4bhdWAwymLLvKDcrdH1Aap6RUTruyuVAhPfgjJR0Q7gSPzJ0HHvFmdkMrKy0g2RBF4pxb2wI ai1ouQjHPSJIlaa0Fhbpff4WtKuQGboQ== X-Received: by 2002:ad4:46d3:0:b0:698:fc78:21ae with SMTP id pm19-20020ad446d3000000b00698fc7821aemr9828997qvb.6.1712107974990; Tue, 02 Apr 2024 18:32:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEDjHRAOIt7pAcFSSzP58lew6Exi/iOo8/2ozkB/HO9HAPtSJzX+RWcE9OqEaW9cSxSEbXVlA== X-Received: by 2002:ad4:46d3:0:b0:698:fc78:21ae with SMTP id pm19-20020ad446d3000000b00698fc7821aemr9828970qvb.6.1712107974456; Tue, 02 Apr 2024 18:32:54 -0700 (PDT) Received: from x1n.redhat.com ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id qm18-20020a056214569200b0068ff8bda6c7sm6031687qvb.92.2024.04.02.18.32.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Apr 2024 18:32:54 -0700 (PDT) From: peterx@redhat.com To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Huacai Chen , peterx@redhat.com, David Hildenbrand , Jason Gunthorpe , Nathan Chancellor , Andrew Morton , Matthew Wilcox , WANG Xuerui , Ryan Roberts , loongarch@lists.linux.dev Subject: [PATCH 3/3] fixup! mm/arch: provide pud_pfn() fallback Date: Tue, 2 Apr 2024 21:32:49 -0400 Message-ID: <20240403013249.1418299-4-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240403013249.1418299-1-peterx@redhat.com> References: <20240403013249.1418299-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: 99352120017 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: nyxgfyghs3a4z6s95b6848frk1b7h6j3 X-HE-Tag: 1712107977-824637 X-HE-Meta: U2FsdGVkX18Ljfjj15fDv1YhSWJr00766ojOek4X+RzreQ3ooNzE7WTfEfW2s4N7R8SUMyn0+j278gpF2UMuIKO1nJx5u0l6ZSBUNPTGBfoBGvx6d5xgZimRns7gCtS14lWM0wRfa/iU6SbO5uJbh97w7JZH7mp9IRWyrceML30gYZgcfKe1mmq+kHIuGeLQE2oY37uG/Cycowu1h54EhxoF/XJ4QLFFOBHjiuhRFVOPws3/VVpnnnULuFaCA3qe2SId8/ong6Mbv9IkypDlNpT+2yoru80/Y1g1+GvOcM2Ee0qf/rtHAldWNpo5liVKy5P3Y6u0/3nZ41y5gmn/mKVm/JBtuBghe9y3QjlzkwrzPxi92f9xAjl5I6akJPdHULPnRqu3ORJH6p6rlHAFOWiEuSVAn7FWhmgENnJr2RqrhfDvxcXEijDdYwHodCSVCLIJOvyV6j/0PvuryGux73tPUa2f3SF4roke/9OHLDPBUXBNWDYeleK8m/SEFnlJX6D2V1M99QR8bwignqNe0yCde8WBVJh73MalsDBZIqKQc98yhzB7i+QtJef1QakyjXU/Y06Kjb9uV3hZoYYFxHyDdmrsIEtzw1b9KRndWpxMsN15RNHq/B0J6rqORlR3GoNvEUmPrDaFAcM/DrR/mwrRMkSGZa+znAEISW/+cFA/CToOzsrMhN4GGbPNz1BBXtFedzQJVAnYILsk8gBdXy+r/DITVToIRpdwTNK2ooaZAIYYuZPJcYmS9AoIeeoisvoMzxQBT5cBs9hKl+m4dR2FBXx1RafQWiCxkxZyfwWNOpmf8TF7C+NDfsW7q2srj2FFr3uEVY9LOidlLmyGJxEvwyIRqOVfsQNpdaZvjNDKmMBOkaocm55DHHsQuiDQTnnlUqwGx8Dxa4oCvF02dIfgZkf9eMNnmKyA3CctOIBq+mjgoyJwBejN89Xq+vzXjQvVhRC/9mmfPbWs9mG 0nmp7zbK A9TmS07pkQWw88GtfIoDDnLgLuJKf85mWg8czoz3UEuY+x29oNLYrSU4sUx7oM+8bN3wWkLdmfvxp6b+1tpHveAognZ/A2fHu+XUHdmBb7SvtC4ShBOow8TNlyiSVpa8GUV2mh2nkYZN3SCL2IYfwmeNfhxxyUnZIunR2gxVaZYIUsn7xN3FE6lHWCSFGZTSW5TJ00cbbQJJZzYHKC452Wfi1Y0jqV65SFIqWtR/ML5y6lM67RRPaJn0Sn59XnH9ObBwslKv84NFkuvqJTf4sqBvWdzFUvvtDnhDvH3mnd/yJrPCS9S7/5pO7POH1Tv3mi+R+tJvj/5cLY5N2OTpbBI78NFKsATH5KWWlDScmMAB5qrgy4Z2JbG07FsoNaDFUiNiknrumAXPgm7jk0tJKWuK48dWUbJq50J8CPvm+bB/vUUvKgGDY31QAjrgNtFsFJNJtcTKAqlc7FMcTxqR1eXWjgjXVDGOR9/BG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Signed-off-by: Peter Xu --- include/linux/pgtable.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index fa8f92f6e2d7..0f4b2faa1d71 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1882,9 +1882,13 @@ typedef unsigned int pgtbl_mod_mask; * code. Now it happens too for pud_pfn (and can happen for larger * mappings too in the future; we're not there yet). Instead of defining * it for all archs (like pmd_pfn), provide a fallback. + * + * Note that returning 0 here means any arch that didn't define this can + * get severely wrong when it hits a real pud leaf. It's arch's + * responsibility to properly define it when a huge pud is possible. */ #ifndef pud_pfn -#define pud_pfn(x) ({ BUILD_BUG(); 0; }) +#define pud_pfn(x) 0 #endif /*