From patchwork Mon Mar 18 20:03:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595762 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3650C54E69 for ; Mon, 18 Mar 2024 20:04:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C5766B0087; Mon, 18 Mar 2024 16:04:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 526876B0085; Mon, 18 Mar 2024 16:04:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C77D6B0087; Mon, 18 Mar 2024 16:04:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2FFA56B0083 for ; Mon, 18 Mar 2024 16:04:15 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id F3100A082B for ; Mon, 18 Mar 2024 20:04:14 +0000 (UTC) X-FDA: 81911236428.28.63715FC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id C7A6B1A0008 for ; Mon, 18 Mar 2024 20:04:12 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=L6kRHjYt; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792252; a=rsa-sha256; cv=none; b=t6VHP1WEZQp3XX+jgj7P8eAq87FWZLCOtr5Uhx7tUf+ytOnRqpqrsIrYRCClj6xJqY8irz smqnRGWlPw5Tz9jzaz9QfSpCir0GvHDJO6wSNGw5XoTCCvjQ9GhxofBM1I/L8w513Xt+L/ ORuHWhSeCMW4M7uLGLgviLzlwtBppkA= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=L6kRHjYt; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792252; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o8evz9V3UX8qsNtRrNT6H/1m1GM34KP8FDwkI2vjQ2A=; b=goMmOda+oO8dgsMgzeWgbVnKDwVQyT8l/E8XSvPpMsLG6qEWOTpfwIu6Cqy1XC5c7ERK5R DgvKElfEOReq3g8fH+d+dNBnbtrnoCN/95NOqQ81WOKaq9/Uspw3NfADmDsshIqIGXz3hL SDNtkcfsq6Ub3pTiDVUKa2E9zH96pdk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o8evz9V3UX8qsNtRrNT6H/1m1GM34KP8FDwkI2vjQ2A=; b=L6kRHjYt1gpwvn4RI9SIlBuglh8YQDFaX/4TipjDQ502EOAsCyGVBwod1Wo4CfsJ4+y86n mznCoeklXNoRaQk8eI54YIfaaPpVFIfffI7ZsGxJYD2xhj45shvual+hUWYFd9ulmaE3Gd fDIg7J+ygm/E0HlOOUAeablVHouplT0= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-491--hOKYtLoPW2rtxNnBd_y0g-1; Mon, 18 Mar 2024 16:04:08 -0400 X-MC-Unique: -hOKYtLoPW2rtxNnBd_y0g-1 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-430d73c0492so3449591cf.1 for ; Mon, 18 Mar 2024 13:04:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792248; x=1711397048; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o8evz9V3UX8qsNtRrNT6H/1m1GM34KP8FDwkI2vjQ2A=; b=E+u+NouJG7MnrX7fdDh3B2j3C8fRiqmCUJuCcV4Ai2GUj6SxnVBZv1vNfYeP6lKziK Z5qqkadiK1XZFFT9Vhv3JGu8/Ps1bKQYDMGyLSVQhUqWxGJTd5P88Xy58y8VF52hNojW mOIC7ZE063Cc/lHA5KExKFy4awb+MSaOuviyIPJwFTd1VhGSWscxvqqMIrq5VH85+FYR 3zYC9frWWYl2jY5vqdVeOe+TssZyxY/+mSEtqV7ymRNh69I+Y3jzeXCiD1E/F6q6H3PC 6f8Ug41bwHYq9vCMZsFj7/tgcW1eKZJpFy9TRM/P8osA3M8+3Kb7rDIOEpfna9WXPm0X niZw== X-Forwarded-Encrypted: i=1; AJvYcCWXYuMgpaYIwHh8QZKbkd4s/WFm56iJLWyMOWd+Nb9NYUQcdkd8k/aXsMDe2etFnMJK8U0vNY1kz5fZDjC6uhh9aHA= X-Gm-Message-State: AOJu0YzI+tYO4L8QMXwxGgEg6Sma2+A5m7+b7pMwNQOwvbgXQB/tNg6M 96qPo495jwrhfBspmz5wCi59zZvkwnVop5igF3v6B6+R6PqQ+X5kCcodbgDhPirCi8ZHot5GTAY kGOR7FofcH9yOucGloqQ2IlBxC7gunNyh1D6uNSPv1CKeDWfA X-Received: by 2002:ac8:4e2c:0:b0:430:d071:2b0 with SMTP id d12-20020ac84e2c000000b00430d07102b0mr368389qtw.0.1710792248063; Mon, 18 Mar 2024 13:04:08 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH6cgX9VgTY+xZMMhFw6lqDmni31WpaCJ4ySK//jo7z1XpjzLPLjahfNZgReizXXgubZZVEzA== X-Received: by 2002:ac8:4e2c:0:b0:430:d071:2b0 with SMTP id d12-20020ac84e2c000000b00430d07102b0mr368368qtw.0.1710792247694; Mon, 18 Mar 2024 13:04:07 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:07 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com, Alistair Popple Subject: [PATCH v2 01/14] mm/hmm: Process pud swap entry without pud_huge() Date: Mon, 18 Mar 2024 16:03:51 -0400 Message-ID: <20240318200404.448346-2-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C7A6B1A0008 X-Stat-Signature: hxneu8rybekxggdc995cq68zksguk46x X-Rspam-User: X-HE-Tag: 1710792252-823209 X-HE-Meta: U2FsdGVkX18yoFxWLuh1hChVdlnowB/MlbEJJ/ky6JWkfDqrIzZWOSEXk85U+OgFVSywkblCVWXb7FmVrrIE89I1HhAAcV3OrLtXc3fTOoG7Erpr8Ydi0RRGcb6SNiSRF1s1vZtXMYqczgDhbSH3AZ9CJV0g85fTUwWC8/ltdcYgCmqAUCWTQrXGkIOkIc94jKIlSitVV2eZKkBPVKhrz8wbfKskAHtlQNgWHO/e1QD87W2hyEHkbwkYwhSVenSr6VthKbIWRzD9n6vYZ3YjOTzzc1LOteSm47LWYdG4yuWZyrU++QTc6o2KXmluNiWmCIFg1PeGGtyZaNc1Zmolsp7GiCjh8woIporZ20GQqRBG0/Ca4vVjsoquVe5PQujGg9zNicpTO/35JJ/P4sTU6RHwT8AaSBoFSp8nMhemdQpypR8ECXs00g7nccRBXlSpCg9WP6QmWOYS0KUATfh+B9H6EW9zDUvE4iHQNmwxAcpakJg0T6AR0g6TPRUGbjMtteZCPqYol8Ej4ymtsM5yr/mkUKxpPTAV22OvjSEl/Eel/fKQw9gpN70/t2XlsbwIwt9IW65Sxdc8iayi3zwStnQGR3ub8euCQMWpwFYqT6aQaezz6Z09k2WpRO1ral7tuQHvwaWtwJM2X/KU5RGqLyQRpvlMFc2r4HNtpzZRdXUI1UzyxV62d8AhdDoK2LBQffcn+RJ9hHrq00kivndt6MONKYyzLmM/tG9WrFkrY8Vw/OX099s8zPw1gg4Ov0TrI85DPr7h7t+xgjIOuIGFuvQLWL7dMf354/egC+HQr/VC/jflv/RJWTK6iZKWoO14wBAtSBNreBcIXSNl7dBk/smLKg2G1hwG+nyr/0/BUg4feUqH7mYe3CWXasxNs2yKVhbv+YieuUv6zzTTv55NqffxjlSs1A8G7Udu30i304hlexR3VwNkRFsrPtw7hwuc+jtvZVp4EfS5mUERE0a VLmprMDb QwZhE59PHpNJl1iTticXx9UnVqdohlpOGkOJYJSIMNyz1t9DrfR8OLrK7enS2pgxVVwO1TYllV5Bir/Fc4GaM5f5C/BSane6EawKg+ly3DWptVTLu7TBCpklbDjTU2M+2S1sBhpk38mARazucxDSEXNIZisvYBRl7zwOEBa1OJXbsBv6yw9iXwqzbxosuZXGcQRtvD9LC3MA+PQ5sF30Td7I+J58IKcK3gkq9lNtR7VjvKIKYYEH+hlHI0o44/arzncXuMKBjpi7JTSd65puseu+DjyBdVJFT4+SSTk6+QoNDwFPjt7zGSxNUO2K/xImlbGiYtu+24zb7xmIXLLb03t9oARntfqU6mZx2PPr5TPLguEU8ZhZ45Etnj8bKTQIYiMxF2xTsFaUOELzFaP9i2VcPKPFRiELpnDP1dCIN965QhJuc0PoZX/E86d0DvL7IHwZUs8otdKMun78= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Swap pud entries do not always return true for pud_huge() for all archs. x86 and sparc (so far) allow it, but all the rest do not accept a swap entry to be reported as pud_huge(). So it's not safe to check swap entries within pud_huge(). Check swap entries before pud_huge(), so it should be always safe. This is the only place in the kernel that (IMHO, wrongly) relies on pud_huge() to return true on pud swap entries. The plan is to cleanup pXd_huge() to only report non-swap mappings for all archs. Cc: Alistair Popple Reviewed-by: Jason Gunthorpe Signed-off-by: Peter Xu --- mm/hmm.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index 277ddcab4947..c95b9ec5d95f 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -424,7 +424,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, walk->action = ACTION_CONTINUE; pud = READ_ONCE(*pudp); - if (pud_none(pud)) { + if (!pud_present(pud)) { spin_unlock(ptl); return hmm_vma_walk_hole(start, end, -1, walk); } @@ -435,11 +435,6 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, unsigned long *hmm_pfns; unsigned long cpu_flags; - if (!pud_present(pud)) { - spin_unlock(ptl); - return hmm_vma_walk_hole(start, end, -1, walk); - } - i = (addr - range->start) >> PAGE_SHIFT; npages = (end - addr) >> PAGE_SHIFT; hmm_pfns = &range->hmm_pfns[i]; From patchwork Mon Mar 18 20:03:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3A3BC54E58 for ; Mon, 18 Mar 2024 20:04:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6605A6B0082; Mon, 18 Mar 2024 16:04:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 610966B0083; Mon, 18 Mar 2024 16:04:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B0DA6B0085; Mon, 18 Mar 2024 16:04:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3A7DB6B0082 for ; Mon, 18 Mar 2024 16:04:14 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 19347402C5 for ; Mon, 18 Mar 2024 20:04:14 +0000 (UTC) X-FDA: 81911236428.08.1F6D884 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 01FEC20018 for ; Mon, 18 Mar 2024 20:04:11 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H9WSofPm; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792252; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3D9LYre9es5Pr/ZmF0ksBqg/v+REJVkPqwZxeVyu2sc=; b=CUUeAuDaTFIA3zXKyqwMehsvYBbjjfUVCM4zDxLbH/G1+medsUQOuLCcuaEZG4DyLdwwtp v+P3UiHMCcOe9mu5QpQcrAswbekWhk192qIRIFwm37ntoXc1JOeSF3k/3Z/cgKDznOYE4W KGrA4m3ZqTVtaIsASOlIlZXlioosngA= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=H9WSofPm; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792252; a=rsa-sha256; cv=none; b=6WvFVFNCZ/zlFcqjmw2mO9VIbJwvbton0tw1xkZr2Fri38FBdjcXnt3QmcYmzHZBWoqGRM RH4T7QpuzU6yUfoNOVWTeRlnh4/dC8w03ry2ZSX5iAZos1Zf2p2KMbiFIkDLz1ijBpQ+Ad vOCG9djI+rycpYDePUa+mu1PbqHIs3Q= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792251; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3D9LYre9es5Pr/ZmF0ksBqg/v+REJVkPqwZxeVyu2sc=; b=H9WSofPmRGkqYfpr0CIOZPhF0qPkczJXvKMCEsoSlwPZT8Q+uaLj33Cyu8T4ZBMeUKZbul 3Qqe3gwJc450yt0zNvT9hZhjRPA2R27pvFvGlh4nHw/v7irUVNnjyEbhpIO5sleNtREiAi Z2hfegWHL3c+hyFdaiVghPAjjw603CE= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-404-ssIyHP6_NfCiD4p5zsxggw-1; Mon, 18 Mar 2024 16:04:10 -0400 X-MC-Unique: ssIyHP6_NfCiD4p5zsxggw-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-430c76df729so6672251cf.0 for ; Mon, 18 Mar 2024 13:04:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792249; x=1711397049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3D9LYre9es5Pr/ZmF0ksBqg/v+REJVkPqwZxeVyu2sc=; b=OKNDFZTniqlX1nIva4UWs3JAu/4GmdRPwFcQAe0DqhQ5IgD/VvbcBxCIcWM+V/Cu+W SvkgnRhmK81J44y3aFrpVWeRUN8GH4DDZrmeoNYYiFjjBZeSMF0v6NW2azzDFdP++ky2 RQ3MFpqr8HB28doQYJcqdYxpNK6yvzFJzRWRwcpqis6p9qKIVvlUlxAuC1ob0brKXbmC BMnNexlTxGRJzPV5oA9OmODuu7hPIVz8r2EMEFvsBio3tzSj6/rncNWWKUAkoybHyOO9 3ePmDplO0q4dE9aBrbHPFes3ya25M7qphkLjysJ3jsVPRcIkhan/ytaQ/3xZXlFR5BPE ASAA== X-Forwarded-Encrypted: i=1; AJvYcCVnaxJhaCzkAGuEOPbFrgFy7LZGE3My/td9FqHNauk+3saMXyVAI9WCuS0grlhzxrjs51bTt77aBpi/asGUtI2FZr8= X-Gm-Message-State: AOJu0YyFnM2HfaineZY7vRXwJEfJVbEhnrggmyzg0Mjlcj0GrvBYqWnZ Gyr6uSyIEjKrQC1lOGpf1NebXLUzXBpqADt2fTGJ2nL5NDC2uFqJ17ovteuifkOlDv5qDvRcZC8 ANBFHUmXZh7X8BiqL0SA5l6p7TldUBPTGYq+gfZFub0PJ9ULs X-Received: by 2002:ac8:5809:0:b0:430:9ec1:77f9 with SMTP id g9-20020ac85809000000b004309ec177f9mr12704570qtg.3.1710792249342; Mon, 18 Mar 2024 13:04:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFbpoPvKLD658FrZMZUfa21VFbJZXAX/nOy5o3FQUuNePIa1lkPrmXkrLKWAEePy+S7Jq7UDQ== X-Received: by 2002:ac8:5809:0:b0:430:9ec1:77f9 with SMTP id g9-20020ac85809000000b004309ec177f9mr12704524qtg.3.1710792248947; Mon, 18 Mar 2024 13:04:08 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:08 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com Subject: [PATCH v2 02/14] mm/gup: Cache p4d in follow_p4d_mask() Date: Mon, 18 Mar 2024 16:03:52 -0400 Message-ID: <20240318200404.448346-3-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 01FEC20018 X-Stat-Signature: 4p3eyoozxfbjb86w3a6yo6h1tognqxqb X-Rspam-User: X-HE-Tag: 1710792251-425627 X-HE-Meta: U2FsdGVkX18Vhq3GY37XPkfN6mYyuRwo3hVVLmi8el+z6ij70GNOnqBlDWoeHRUBEz58A0XdYPbUbZP2NVd898mlLimvuvTC/gwRKX0k9rYQ1PyCHDlUfVANMNdE5YWtY9LMyIscxw6N5jSjwAiWRi7E5oOdM4flCiXCaz3HJrp22gY3VybOzkCBPk/IQWND9Lmr33vTO6l0fiWQrUgwCPlSaFQhVTZI9l8UfYw0H+33GBZtPezRgZdVl6RQIi7AIDFbZnQ53v03lVdkay3BLkz+xx1snIGR0u3+PVDbsqSFPJoKb6Vu057lcrZHfLmLUb/xOKdk8EOPaFzZNpc2Ay1BlUKL53PxSPHTheNlbuKpz/XmKdNj11oRvXE4t+FZpRPymCcUt2AcYFVsiM4r7dFa7T1ZW4SZZwoYO52nqoQXsUR5+MXUJ1ooTLsRp+linytg9gK52cbXSp4b6XTRs3ryfzQlVjw+wLWfq1RZef2xpV1onUOs8OOXCMJXa1KfJrDI2f1K0E7cstf1CGgH+To5vX93/id1YmNe3yd2OL1OWjuHZpROm+DLluMpk4+fqUmME+j6pahAnnafUbsVOh6Kh35sStUKT0pdNUFsd4Dy3HBvHcwaA78uZaasPQdcKyFCAhWac0V9L3vLH2EJBF84gx234civTn8FiWl6wdFdHIb+QHQvE6uiwbNM9/qjw7mxcVXxEJEy9dhF4lX0bGuxFULzPvaDC1UG4QOKSTV7BRWErmyDlhS0POvfxMwI8NNqz74yNrzTCYBMcMoE+2tAwVREo5tkkWqtM7gNkI4DL4J2qvmE0VTorLgNRuwHJSJ4nK68dSJ6xW1cvZXi4jK/4hUsIJMunU+uxDGvcWcneR+D1Ggernn+3qpyxTlhe3/ApOTHDFuJ4a0DmNZ0Qmh+cmMKrtz9InHXA5NMVnrsOweDPturzOFHX2HPDjdmnvWDu+oAToJ2Zv4VH1r w2HDrZ5N KUr9fJnDFZH1yWSDLiQV4VuJkQspeHcGG+HqqXyl60MybNi/gVCOISXvjHhfzgFZynS7K6QgHlTKQYiisM4W9xjYDD99YZLJKNsr+Z25Gcm7uZ2h6x0ccK7QedKZeYxDMzsWnyYgcAQy8gwst8pzCzKIOeuPGOTmtMNx/tSTo6/xumbLr5Y+CiKu2QGLv9jIQlyNYlR9hhUe3Koc/4MRO62CTHmAU5OCETz4bpRlhkoO4ResT4FXZdA0zFY1JaWsS9phqqwCalb32DSRLLbRMMnuGImAjc9WroTyWVmdihoowYmjAv9B0S0jy2Gk7OPKBg2MxGd43dWIgi/5wS24t1FrFwD2aU8tk5uNxcSaehfXgXKyxfftHYbucb9cpBheAHb88IzV+Gf6cH5JDZACE/XL+CquhV7E50ZkyIjYsjXk87tjDoFdCCXd4PQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Add a variable to cache p4d in follow_p4d_mask(). It's a good practise to make sure all the following checks will have a consistent view of the entry. Signed-off-by: Peter Xu --- mm/gup.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index df83182ec72d..69a777f4fc5c 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -772,16 +772,17 @@ static struct page *follow_p4d_mask(struct vm_area_struct *vma, unsigned int flags, struct follow_page_context *ctx) { - p4d_t *p4d; + p4d_t *p4dp, p4d; - p4d = p4d_offset(pgdp, address); - if (p4d_none(*p4d)) + p4dp = p4d_offset(pgdp, address); + p4d = READ_ONCE(*p4dp); + if (p4d_none(p4d)) return no_page_table(vma, flags); - BUILD_BUG_ON(p4d_huge(*p4d)); - if (unlikely(p4d_bad(*p4d))) + BUILD_BUG_ON(p4d_huge(p4d)); + if (unlikely(p4d_bad(p4d))) return no_page_table(vma, flags); - return follow_pud_mask(vma, address, p4d, flags, ctx); + return follow_pud_mask(vma, address, p4dp, flags, ctx); } /** From patchwork Mon Mar 18 20:03:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1582C54E5D for ; Mon, 18 Mar 2024 20:04:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 110656B0083; Mon, 18 Mar 2024 16:04:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 046866B0085; Mon, 18 Mar 2024 16:04:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3DAE6B0088; Mon, 18 Mar 2024 16:04:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C42D86B0083 for ; Mon, 18 Mar 2024 16:04:15 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 90CEE14068F for ; Mon, 18 Mar 2024 20:04:15 +0000 (UTC) X-FDA: 81911236470.24.FE5BAF9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 7B8F54001A for ; Mon, 18 Mar 2024 20:04:13 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hXBlPnLc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792253; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OThRyXZFMjugEtcpgf//pX/PpGcoxbD3wNFnQts0Kco=; b=sRtJMn0oz+hP4LK32mA48Pzw1km5JgebRrN5FY+POEzdGZZBrrl+sMevZmol9nMAIS2i9E NOoygWL6uwR4aRH98caDmhnoePbsY/VfjtGuH7trOyXYGH9QJXmTjNyoMrTp6OfIdbRcya Rn+Pj7bizZhQbbmswAzb+/vrivqLU9g= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hXBlPnLc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792253; a=rsa-sha256; cv=none; b=UiUDLi0FoeUYwccZ8ep1iHmdvy55W1eVtCOXQwVKidggDi7MJaT772MqyuQ9dpZBuaScyl V9+4ZiIT0gU3kFV1BSc2ZbumUN69eHwIB4ODATwAc8yqkAZ3eYNcM/ifUjb87IQN8S238V SPYol4qkhGZ7JRnvd7H9GDdAjMUvyAU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OThRyXZFMjugEtcpgf//pX/PpGcoxbD3wNFnQts0Kco=; b=hXBlPnLcptxIoQrB4kSJW1/+u3eLQlV3Gt5Ez4iTGDi25xjF2oXhpEtd4DU/McbM/yv4Ze qB9M76bya54d4Si2iiVwyfPooHsaZQIPeK9Xr4TMI8QWtGOkuLAyybXFLW5trULmU4MoWU dXnfEm/3fE437vajDLqlDuBP4QxO61I= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-277-pbQb-BGjNliSRaoLNnfwCw-1; Mon, 18 Mar 2024 16:04:11 -0400 X-MC-Unique: pbQb-BGjNliSRaoLNnfwCw-1 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-430b4572a97so17522711cf.1 for ; Mon, 18 Mar 2024 13:04:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792251; x=1711397051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OThRyXZFMjugEtcpgf//pX/PpGcoxbD3wNFnQts0Kco=; b=fgdmL3IMeqT7une3yaj0LN3BFoCavwY34SztG3B3/3q8zsH3HY20JC67jz7bD8cHqn g5oqNvVNZA6X1Lc3HbmpZTe92WlOl59ytArMV1QAbdUtX1S0EKHpTF6A4rs9BKUcpjBx ooTWyez2Pdg9K68qMtwas4yy3/CQl22esXffUlPPgz5npG/2QjdpxiC5ErECOfrdq1ol mEcz9re0/1dWj1VuEbtq28N6od1MIoQs5sknMgkpkmemcj1GhmRam6G6Hn/ugE21xxWI FV8HG9p6ZwSFfLbtQ3BboXKamWOXF98YkqbcAYTa42DAUBHCsXauJ9eBbSJylB1POKA5 Jtkg== X-Forwarded-Encrypted: i=1; AJvYcCVQfmObpUbumvr17TkQChs0EoeuysWHEhnuQlZEkB3M9lP22/mL3Ic75a2kBuG/sfYNnaLQL4Lq8rd7ocPhUZyAd2Y= X-Gm-Message-State: AOJu0YyVE+Xxo/HBIi7ieodlDF+FxHQdA1XLGRHIEfsKzcEwpspD2vvr //7/Vq7WIKezvC4KiBJoLtuYHFNv4dgCHfnDlOQqmWOz3ay6JHpwl+VWV74jRbhR7DfZivgUVqw zAmkMFJje/PFKJ4O2YtrbIAaxYsGicP4yAtgunyHau5AqtFqO X-Received: by 2002:a05:622a:100b:b0:430:ace8:9801 with SMTP id d11-20020a05622a100b00b00430ace89801mr13041013qte.1.1710792250872; Mon, 18 Mar 2024 13:04:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE3lB75/Xyi/Yr3FGp4iv8P5rE4DSC0aUJAJLEw6DejnKFtl7A4ZV0qXSchssNp4WRDw4IDoQ== X-Received: by 2002:a05:622a:100b:b0:430:ace8:9801 with SMTP id d11-20020a05622a100b00b00430ace89801mr13040968qte.1.1710792250407; Mon, 18 Mar 2024 13:04:10 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:09 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com Subject: [PATCH v2 03/14] mm/gup: Check p4d presence before going on Date: Mon, 18 Mar 2024 16:03:53 -0400 Message-ID: <20240318200404.448346-4-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 7B8F54001A X-Stat-Signature: qff76hkfgho877zurkcka7b7oeorga9z X-HE-Tag: 1710792253-939640 X-HE-Meta: U2FsdGVkX18y5TYjQt8MWiObORbS2q3GEpY9nrOG7Yr9h1QLavlGBdsStMDFm7/h6+TRl4IqPaV4WAFurTkcXorSJLvACg7O4Wakcej5AXw3JaFdiUgtu5A2NcUEfdvfA6BzWr6J0BakitaIMhXbZGs9hvlavOyqGpmDUa4uhD2Vq4qWj5po7QpWRrdQMKKrVgV1HITjyzxaOdHOruLCsp/pVNnja6UkTNxUwMbcR/VJlybvOgM8cQ5sMdPcUsKdcep0jjkN1UG9bznEHVRjBl/TxuMvtFD2MZn9Rt3HOX6o03a1E5xDOMxcdlRQIpjBbnzrqn7+9K+pw982DiFZWQz9v2/gw4BODRwfbR6l3e6zNy+aObXOAoKPgluZN86WRquprUtPsKkve4nksFDrV91Ihh/yNyc+RliMLNQ8rrpBuXe+Nhj4dhnztduAj+Qg/iY3bXfX0v60hMYiFEshDh3VNqN0NzLGqQ3F5jIgBgGsYPXTt11l6TUcYrNpk+dLKbtTM+I1V49MM3zayZGGPuz0xi9fbDxcS6EZsvUCBYHm2stxLEZAtVrWwtpYKlwF4ipkpYPJm/E5K2U/1w5iOsJ5kmysbTHjxi8ruJ23JGRLxpEeAVVm6sQ/yILkr+TmKXYwuInS0R1ppe/enUguIgR4XsSLIROPhPe4tSc9p6Q0vomTwX9lNvM/iYeVdNf8RZF+lB4Cm5hksJLtJ5re2Ui0GyKh42cZdBNKtt5Z+pjgLqNtEV1pmEXTCAK6sMEBiMh8wn62bnTYdCCmdGLMZk0RyILATXlG4g8P8qNVeGLjRn5y1mpC6mD+kZ0T/KC0Bs1PSdVH71tFM+xzyAyRlEJYxeIFr+gt2xfO2vpSJ/tzBi8z+39ELFsjD1eFmQQSaSxsVedg8FmkPwh1zmaIq9hKDHWQx8CnuP49ycnxJBpZ/ju/DLZUDVB7dOEITkIh2TDgrwc8N+Q0NlnT0jE XOYgMEGw jGu3C9RC4PKFGwIGUqxBKNAct9g6HyMPUkhXWVrFTjf0+Ym5z6fzpe6ViVzwfopzBvPzn1m+cg/wULTnmOXDYjRApiJFzZJ9QqjjETQExvwjvLzlAsN2DywTZkHE/B+ZbjQB8bcaRnGX1rZ0yRgoSmLMbVbCP2h+/yT12XbBHz0MUF+QUblAX+2JqsdXdcCawCtqB+z0fjxc1xqvJvfPMy6iS1rMVIpsLoAONxqu2k/DsukhQOolks79ivCyIZvSMRuDRAxToUK2bnK+2hSJfziq6FktKmCqj8RBnrowbidTlqvqScat0W7CVnKlCdMi3cgn6bx9lmIohTa8o/4awXCA1fuBvvbgmPA2Csw7k1V4XA1g+vY8+Q7rXGjAGbtxpEns3KZAQkwIbJzs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Currently there should have no p4d swap entries so it may not matter much, however this may help us to rule out swap entries in pXd_huge() API, which will include p4d_huge(). The p4d_present() checks make it 100% clear that we won't rely on p4d_huge() for swap entries. Signed-off-by: Peter Xu --- mm/gup.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 69a777f4fc5c..802987281b2f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -776,7 +776,7 @@ static struct page *follow_p4d_mask(struct vm_area_struct *vma, p4dp = p4d_offset(pgdp, address); p4d = READ_ONCE(*p4dp); - if (p4d_none(p4d)) + if (!p4d_present(p4d)) return no_page_table(vma, flags); BUILD_BUG_ON(p4d_huge(p4d)); if (unlikely(p4d_bad(p4d))) @@ -3069,7 +3069,7 @@ static int gup_p4d_range(pgd_t *pgdp, pgd_t pgd, unsigned long addr, unsigned lo p4d_t p4d = READ_ONCE(*p4dp); next = p4d_addr_end(addr, end); - if (p4d_none(p4d)) + if (!p4d_present(p4d)) return 0; BUILD_BUG_ON(p4d_huge(p4d)); if (unlikely(is_hugepd(__hugepd(p4d_val(p4d))))) { From patchwork Mon Mar 18 20:03:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595764 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04498C54E58 for ; Mon, 18 Mar 2024 20:04:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C4466B0085; Mon, 18 Mar 2024 16:04:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 770D56B0089; Mon, 18 Mar 2024 16:04:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5ECEF6B008C; Mon, 18 Mar 2024 16:04:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 485BA6B0085 for ; Mon, 18 Mar 2024 16:04:17 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 10C811608D4 for ; Mon, 18 Mar 2024 20:04:17 +0000 (UTC) X-FDA: 81911236554.03.CA92D11 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id E21134001F for ; Mon, 18 Mar 2024 20:04:14 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RghpMXmc; spf=pass (imf12.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792255; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vtZJyICZEGrBJHpYWSu5beimQ7UmPGzVa4S5UCB/Dcs=; b=cJ3XgVwQ597FG7hKZyEy/wzN/cuL0W0peSyCeZcRWr8ihH1DXiEa92t+fZewVHsoiI8LD/ HUcoWvLAX7uji7Uw6rE7K4seRcSdClMCDyC68yTyAC3u187y6pcbXpGP+5YtYcdYCWWIsy nFQDmYvVzcEBMDDTHDi96bhkjoEcfLY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RghpMXmc; spf=pass (imf12.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792255; a=rsa-sha256; cv=none; b=S5nMHpSjcAAHQk8Qh32jqyr0L9TV4HT4J7S/SIDjUhpbirCN5XmfCJSkR4Mym+0eQnf5T/ I1GtRzMeY34S6tyFNg5h4Q32aKUxew3LdZCpD+Sb1D9iW4mBER0QrHFrDcxDiWpZ5SVFcJ gvcGDVYaclTwd0yNl1AdcqMAXy7rCz8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vtZJyICZEGrBJHpYWSu5beimQ7UmPGzVa4S5UCB/Dcs=; b=RghpMXmcmYmGS250nfwprZ5nJJJW+A0lma9Zlm8Wkgxz67MnsHnDEQA5DCMVZAJOSn/5ru 2RWU/aoearGtLy1uLhnWHbO1FOgdYkAsU0ke1dTbn1a2Efh2uHpKd7BNiInMq7HVYaLRwp jzcZz1e0Db7llGdZGFFMxEI5T0x2sdk= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-395-evHMpCAfNjOd-eu3Vw8pZA-1; Mon, 18 Mar 2024 16:04:13 -0400 X-MC-Unique: evHMpCAfNjOd-eu3Vw8pZA-1 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-430c76df729so6672541cf.0 for ; Mon, 18 Mar 2024 13:04:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792252; x=1711397052; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vtZJyICZEGrBJHpYWSu5beimQ7UmPGzVa4S5UCB/Dcs=; b=vmJ9FHzDawyx1wMp2rSZ7Rq1QgJpHeeKsazZlu3CelVAJqv7IDPL+1VWa694va91yu DSOXUyfAt5So02p7nPmmhDa6EMA32gstnqxmxaQ3YyhsAiHRPDC3XBtQEn/uGT6wtPB1 9eqoAsrum6OyKE48Fj8q4NxC82RRXwdwj24Etu2ZaORd8sGO6HWPFsaTBdsguB8qzXW7 5D91RYl4m1dodWGyJo38JMFbA5hgWtztJh8INSwChFJrR636jhIUnQ0gL+r0/b5hNZa9 UN/7FwnMbA0oE9mvNmNmztCbXKGB8kh88SpBHN9k87IrY0rfvAAciJs+1YEG/xDCVYPq zxWQ== X-Forwarded-Encrypted: i=1; AJvYcCUQvE0cprbrzz3L1U/4k6fJ7MoI7jOEZfE31ymaI51ITJb3kbILNNWgjaIeVoCN02vvWWzJtbWaPGYnUdEjtUpdXoA= X-Gm-Message-State: AOJu0YwMHYBlbgO4zg5RH3Lx/xJIbJL+mrL3FVYnrCceK/4aFdvaRXQN swDlsxd/qezpNJJwkogKaIBnmGuCoVt6X04I98NEgekakCUp5xslvVtcZnNBxD3eWG2juGknXSu u808UdqJqpjTaIHFPk0PX2m5DPPPZbTpVoiU8VdXOzMhdY1tj X-Received: by 2002:a05:622a:134e:b0:430:c82f:b9ab with SMTP id w14-20020a05622a134e00b00430c82fb9abmr6625413qtk.0.1710792252328; Mon, 18 Mar 2024 13:04:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFopBSFWwBK3v15ombZ7ZQDo2UFCgzW4nradhEvm8gSdWhrm9dq3oWG6x2VBEnxvd6M1ShIXA== X-Received: by 2002:a05:622a:134e:b0:430:c82f:b9ab with SMTP id w14-20020a05622a134e00b00430c82fb9abmr6625362qtk.0.1710792251793; Mon, 18 Mar 2024 13:04:11 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:11 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com, Naoya Horiguchi , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen Subject: [PATCH v2 04/14] mm/x86: Change pXd_huge() behavior to exclude swap entries Date: Mon, 18 Mar 2024 16:03:54 -0400 Message-ID: <20240318200404.448346-5-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: E21134001F X-Rspam-User: X-Stat-Signature: cnjykqf3u9dmeyr4nbseq7ocaqgszdsd X-Rspamd-Server: rspam01 X-HE-Tag: 1710792254-743348 X-HE-Meta: U2FsdGVkX1+u1kkcL4Bwx007Rz2GoKEJO+6b3tlsusB7Z2g8o3JWkhFfxtj5WuyJ1lMAGRROQbw/m52XbP0EqwXY5Gh2EOGVQNqhApzCAK7RREPrrNbcjquaPRzE+vFOxCQODF6TFpcEogsAwZzdSAaai1+LYysyKaU8vPTrK6Cs4BnQinzKFhAot5YZwz5uz5HK7syf2r9ZUBwI837SG5VXdzIjkiULoS+YlXx6x/7Wa/Yti+pBUuFByffnGE+IB1+FF5mObm0ArvOmCYRW/IdzFFu4zD9r7cD+kSnxUfFiC/TCDc5P03Oj41eVhcAo/PchAQeHawGg34BMg9y4RANQT359+LMXi5+nCD79s8qr3fmY2NPfVe4YVzh1druk48fT7Wt6sS5R4D1fCDP2Lj7RTRnX4stA+qFRLeYaC5TfQmHovarYr7PyyO0rex/6gLS0maeFco0tXUq6fPI8y24TXS51T3uiMo/cSSk3doZ0zJD8cLE+n9AjvWflqC9tdotYI5h6IyKI6pi4gX0uFTwuKkDpWrt5cyT1s/kpj61Ttzd38l0f1vKz3b8uze3hKqs/IJ0+TZz4c2mObfAO+Ww/aiQTaNBSJzfztyDn5u37XlMMfmHZd5nXT10tm5TM4T8mHwSi+I39RytwFjKMhnFaExbBsN5qi2ZPeYwWubT1utNwLfsuusbiHcC1biie95C24bb4pafyRyaAgAEezU3AbIIKUE4+7R5uVdWs/cZx2B32zvsjJW9fvRR27hFhhTBY8tPHw2rSchATFMOxpwmCRxJ98dse6wrGEKe4WyIqspZaFtqPWby0O+wKH4jWJhHRUD/W3Mbzo3heyCZ+wF05JCVj1trA+kspES2npdqz6dAf9qlB2Vj5XOBkd2oghE1H4vHxW3ozV7zrYp8cgIFNDctIWe+s1Ec9OHz2dxyUYnBWy7AIhwUuQC5lbI6b+lkXd4K/AE8mDpdmNAL cU4Gq0yB jLfZuhD9F/sg4KiL21i5rypdQCZOOBG5bruDmq2fQ5nbmk05atW0yZw8cT7TIZhjtzA8BacYyEbh1jgVShgjndtu8p1rwYzw7Sv3blS20ljnsq0y9bFA9E3frjSXLl8ZkTz+tw7tGcMt1GdXil3e6wAxa+N6YzHyB1t08KWBoJUGMAWm9006V1Fx3SXMJB+8v+3qaU8itAutLc1+k9R3jNCA5Q3OP9zLW9EonbUJ9zOC8pTOT4MMO7vomyVPfvXB6dC1To32GECm1tH1BQEILDbtfjbY65Z8XTAdpDn5czhxeX4mdfYjrQnmipg2q1r2Ne2dcbjOyXfczxYdEl8Shc6tp8kIAQMd/oq/GIJqDkMmca0Tktaqe955B3q91Y/7gwqdNF9TI8PF0SpjYQ3qZUb+GZX6bGGQE+OiRc+/x3J1vMoGuDjoPcb3Sm0/IQq+1zFLxY/Q5Fm2lfLIL/a7+X/VQDIQ+4qBoXJPdija5z2Kbm5DwifOXvQp3kMiiUXXcmUPkUq62sCSN00ATVRIsu68lMBqMz69LhIaOKCsPZFjkE+WsKqIt3ICHl5Id8kLYcf8h+Bh2Cp6h9r4vtIafScKufA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu This patch partly reverts below commits: 3a194f3f8ad0 ("mm/hugetlb: make pud_huge() and follow_huge_pud() aware of non-present pud entry") cbef8478bee5 ("mm/hugetlb: pmd_huge() returns true for non-present hugepage") Right now, pXd_huge() definition across kernel is unclear. We have two groups that think differently on swap entries: - x86/sparc: Allow pXd_huge() to accept swap entries - all the rest: Doesn't allow pXd_huge() to accept swap entries This is so confusing. Since the sparc helpers seem to be added in 2016, which is after x86's (2015), so sparc could have followed a trend. x86 proposed such swap handling in 2015 to resolve hugetlb swap entries hit in GUP, but now GUP guards swap entries with !pXd_present() in all layers so we should be safe. We should define this API properly, one way or another, rather than keep them defined differently across archs. Gut feeling tells me that pXd_huge() shouldn't include swap entries, and it turns out that I am not the only one thinking so, the question was raised when the current pmd_huge() for x86 was proposed by Ville Syrjälä: https://lore.kernel.org/all/Y2WQ7I4LXh8iUIRd@intel.com/ I might also be missing something obvious, but why is it even necessary to treat PRESENT==0+PSE==0 as a huge entry? It is also questioned when Jason Gunthorpe reviewed the other patchset on swap entry handlings: https://lore.kernel.org/all/20240221125753.GQ13330@nvidia.com/ Revert its meaning back to original. It shouldn't have any functional change as we should be ready with guards on !pXd_present() explicitly everywhere. Note that I also dropped the "#if CONFIG_PGTABLE_LEVELS > 2", it was there probably because it was breaking things when 3a194f3f8ad0 was proposed, according to the report here: https://lore.kernel.org/all/Y2LYXItKQyaJTv8j@intel.com/ Now we shouldn't need that. Instead of reverting to _PAGE_PSE raw check, leverage pXd_leaf(). Cc: Naoya Horiguchi Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: x86@kernel.org Signed-off-by: Peter Xu --- arch/x86/mm/hugetlbpage.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 5804bbae4f01..8362953a24ce 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -20,29 +20,19 @@ #include /* - * pmd_huge() returns 1 if @pmd is hugetlb related entry, that is normal - * hugetlb entry or non-present (migration or hwpoisoned) hugetlb entry. - * Otherwise, returns 0. + * pmd_huge() returns 1 if @pmd is hugetlb related entry. */ int pmd_huge(pmd_t pmd) { - return !pmd_none(pmd) && - (pmd_val(pmd) & (_PAGE_PRESENT|_PAGE_PSE)) != _PAGE_PRESENT; + return pmd_leaf(pmd); } /* - * pud_huge() returns 1 if @pud is hugetlb related entry, that is normal - * hugetlb entry or non-present (migration or hwpoisoned) hugetlb entry. - * Otherwise, returns 0. + * pud_huge() returns 1 if @pud is hugetlb related entry. */ int pud_huge(pud_t pud) { -#if CONFIG_PGTABLE_LEVELS > 2 - return !pud_none(pud) && - (pud_val(pud) & (_PAGE_PRESENT|_PAGE_PSE)) != _PAGE_PRESENT; -#else - return 0; -#endif + return pud_leaf(pud); } #ifdef CONFIG_HUGETLB_PAGE From patchwork Mon Mar 18 20:03:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4555EC54E69 for ; Mon, 18 Mar 2024 20:04:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96F026B008C; Mon, 18 Mar 2024 16:04:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 91E1B6B0092; Mon, 18 Mar 2024 16:04:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 770F46B0093; Mon, 18 Mar 2024 16:04:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 64CF36B008C for ; Mon, 18 Mar 2024 16:04:21 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1A6461201B3 for ; Mon, 18 Mar 2024 20:04:21 +0000 (UTC) X-FDA: 81911236722.24.216A215 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id E83001A0005 for ; Mon, 18 Mar 2024 20:04:17 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UvAC3mlJ; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792258; a=rsa-sha256; cv=none; b=dM6l9wRICpHTT3WCcvu0puvC0EMzIM4ccY0Mlf8BYK0UR7L4cJiqlso/OtT4DDu1HLP7u0 d0IBNftuSSfty7VKSBEAF8jRGhARnJ4x2pob83uvXYhu3277+nVokn5OLZWGPLbpy7kf6v DGkRAX7eren5OIRMhWsXvKlHSbWnNeQ= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UvAC3mlJ; spf=pass (imf19.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792258; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PXv9WADGZnpW9LG755UO1HRR+yBh/xYRmZG+iRDILn4=; b=u9rxdbkLRngfyNzf9O/2DsytT6/6qO6lxvKEQ92hcTSjMbtSPKX1ZuEmotu2wm62C8FwEj b75Ixf32jWbAs9HbGG64TZEWTizxJzvt8lUEsoHbjWCtqLRfcT/tTb53SPolQEd28D03P9 1jMevd389oXYEi2CBxwoTcjwX8ZYRqs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792257; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PXv9WADGZnpW9LG755UO1HRR+yBh/xYRmZG+iRDILn4=; b=UvAC3mlJJCSOHNU5MD1NjSKMPmJAP7m+zAHZMj4VIQ7DR16FAGGCcRNf8Ky8MK1wlAZepj EkolWP9ryJTHkDDwvA3O+94HW0nLPdiqpZpV8YRInFUxA5rii2guOKPiSJkPq7s+mmRN+G 1Y/CoYQNOCL/cc98X8ADKQ08KNQtAok= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-394-wmR2qvzBNkyBdtT_qCYA1A-1; Mon, 18 Mar 2024 16:04:16 -0400 X-MC-Unique: wmR2qvzBNkyBdtT_qCYA1A-1 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-430c76df729so6672811cf.0 for ; Mon, 18 Mar 2024 13:04:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792255; x=1711397055; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PXv9WADGZnpW9LG755UO1HRR+yBh/xYRmZG+iRDILn4=; b=bUfhwdvhZz5W5EO++JjoaR881HumI3mXXKcY0X/EzSgStH0yULL72GiVur7KNx6T2c 3Mbx/oHrkmwDAQQFgEujQlT4Ope96rnvGKLOF9w2WHSS8+xT3GQC19B42vvSV7Omg/pG dytyoe6shtobtRgh1E6/He8lb0A3zy1b1pWpgG5An/aTmWXXxTnkXwsLj0swWAB/UPJK uA1aC0qExT+StBo6E6p0Cl8Euo3L3xZdFATXxiob9fapwDGPVKQm96aBeeLPb1cpMJ87 ppXIly/neRzjttwcB0YqMJiyac3UNd5ykvBTDsMmLHh1vZj0uBZrOqn6lwmbgycaMu61 LX5g== X-Forwarded-Encrypted: i=1; AJvYcCXSNP5zh7EUXwtxHwZojteTdk03tkfsHNrCrJ+Jp/LGyiN2tCsIeYXoMxHMNdup15ZecGFCemMp4Ber+2BMb8Sh0y4= X-Gm-Message-State: AOJu0YyO0Ql7lGY28YHw5/oA/y1KD2gjgd3+AnFge7euD7Jqz9xBb2eJ EXBSJFj0PUpQswa+i/HkaVY2HqPi2xn+rIVsAWvlXVg9INP1DKNb3XmTN9QYPJKI+iC9Kp5Xs44 0xD6Kyp3vzKskfbn/ieg2iQvdaX34dI3BUfiGxsGqIjpQZrk8 X-Received: by 2002:a05:6214:ab2:b0:695:c55d:fdf8 with SMTP id ew18-20020a0562140ab200b00695c55dfdf8mr6345896qvb.1.1710792255543; Mon, 18 Mar 2024 13:04:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHM5if+BJ4z/wKMKgmRKGlctu1Y5Zn0gs2J/D1UPDtxcNZL4bpQ3QnCrk82omp9xwt5ITq/IA== X-Received: by 2002:a05:6214:ab2:b0:695:c55d:fdf8 with SMTP id ew18-20020a0562140ab200b00695c55dfdf8mr6345849qvb.1.1710792254868; Mon, 18 Mar 2024 13:04:14 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:14 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com, Russell King , Shawn Guo , Krzysztof Kozlowski , Bjorn Andersson , Arnd Bergmann , Konrad Dybcio , Fabio Estevam Subject: [PATCH v2 06/14] mm/arm: Use macros to define pmd/pud helpers Date: Mon, 18 Mar 2024 16:03:56 -0400 Message-ID: <20240318200404.448346-7-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E83001A0005 X-Stat-Signature: tkmj44k5crjp6nxrcth91wgm3kp7z5bw X-Rspam-User: X-HE-Tag: 1710792257-521660 X-HE-Meta: U2FsdGVkX18P/SJx7rJLGg4k9MYbkcBNSN8ITlr4b/2i1ueVLAQJqbQByUAmkKA/qfJvd3vQh6qit0gKgo4zxvvjyDVZ8ybV+ID4h29tImZCwDewQyggpGmvQyF8fw0uJZl7GmDwq3eGZVu3jHPjJSlDqEOkO4OHwcYAr4sS90rvAhamR9bZRMMNlrVCJYWxHtmUqi4ExZmlOaAE1rGklE/7mo4WNCtbb5W3O4wqyVb8ftnW3iUXhmR/wpsdeVGOWwe47Jraow++EudaJ/pCwnyambiWQOJLTIXwE9DJXuef9lw8/0Nyw0qNiIaLzfdiqTyvoqcENC9UUdp1agbjtybeesx1Wq8JHPJvEsOoTWVeyL66AzrO+aNQmIDTsrvRdaOI98KOn7ZYW3WfCvVlePIRt91E1PeFIw/XVi8J64JSZ/pupTe8i9ANeQQo6VrhS98DStmF4Cn4pZKbqIxwjtR2XHOk9hYXwTfVnxpVJB198h7L5KQ+z3cbGcech3p3YxDBnqW4lg4lPZUp+xRI+G6yu9urw72lKIdCerEqV3dFjtdV0d8/ZFSMyvWE6+fWKB+xmBqnmCQ3HtT6d3tkRkiSh0ivbzJ47fBzcnQjPy9MwctSMIdHJVd/h8arPw/D9USrcd4B5QZFud1T9/hWPM1t09H57G5K4JMSUZppwBd4EJT4q4D7ewJI1k4rNnY0omJNY49PyU/xZHhR1AUNUl3Yp3kYHtH5xCDMeqnBATgzk0aVn46OHokL2oRmIwyRYg8CLSGKwDaHZPDU4LR3SRX26l2DQgouLg53HIWRoH/X6+5KEqsZNiNbpJ5R/pcETHFNwQt44ojj94ByPANk9sLcjCbYyZmrADO7mJQnsNqeu/u4w+oIscVxNQQtxoZR4Kq/0Riij4DFbHJCLDvG8e+DDnpLTNKr+xOb12JLjSQ7uhGwzxcSksQ7q1tP8zrmntRHxT0ZOTiSoenFzHb sw510xNj ko6U5I9P9mvZNzOgIyxuLMsRbm22MrYrW08q0hi610Hw+drFphdWbMlX55rNP1lqWY+KRmGaDQnOPgdcIhFAWpV9CQjT1pqZQJjwyyvjuupzKTTq1CJ7pemQtkUxq7eXqkbVDPuGt9lpP1LDmfbe4cLmxyv7tzgx4TNohE9Qy7R7fGnriOcFF1K0Ds/5vkI1lvVh/DDo4rCQnAcaqrKOAuFy36q1ozzppW9vwknAsQpdgiR9HjRvAnykDcnYV1Zh/3wAq4AyjAWZmTfWrC2B8lpQcp/0lqLQYao4xtvKlrE3XPzapglkjkA9utO31K1mougH/qC4oN9yOGcS3L0uwIiqJoGT92kotnzUyv/YxltSZUxTQbUMGJzOwvP6zSCl7wztByA9Nn6GKC2vi6sNN+6rZgogiZDIf8oYgyvdEYFgr5HLM17tm2mjcMKZS7uZnY6+iKniHZ953P4MMx5H5XPLFU5OamJpMXvtaXNHyjZabr4fHsqJXyxam5/SC7oyCdS7mqMPtGoDEFJc5YYf0Jc4dL8qfvshNH6Hp/JQeHcACEzSQ+UIx83aMwOeIPji9QBh0qSToDxYOWPFUaB/Aak5VkUGCJ/1mMsEzDAiH7ALGrQ8XrQn06qeWQA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu It's already confusing that ARM 2-level v.s. 3-level defines SECT bit differently on pmd/puds. Always use a macro which is much clearer. Cc: Russell King Cc: Shawn Guo Cc: Krzysztof Kozlowski Cc: Bjorn Andersson Cc: Arnd Bergmann Cc: Konrad Dybcio Cc: Fabio Estevam Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Peter Xu --- arch/arm/include/asm/pgtable-2level.h | 4 ++-- arch/arm/include/asm/pgtable-3level-hwdef.h | 1 + arch/arm/include/asm/pgtable-3level.h | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/arm/include/asm/pgtable-2level.h b/arch/arm/include/asm/pgtable-2level.h index b0a262566eb9..4245c2e74720 100644 --- a/arch/arm/include/asm/pgtable-2level.h +++ b/arch/arm/include/asm/pgtable-2level.h @@ -213,8 +213,8 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr) #define pmd_pfn(pmd) (__phys_to_pfn(pmd_val(pmd) & PHYS_MASK)) -#define pmd_leaf(pmd) (pmd_val(pmd) & 2) -#define pmd_bad(pmd) (pmd_val(pmd) & 2) +#define pmd_leaf(pmd) (pmd_val(pmd) & PMD_TYPE_SECT) +#define pmd_bad(pmd) pmd_leaf(pmd) #define pmd_present(pmd) (pmd_val(pmd)) #define copy_pmd(pmdpd,pmdps) \ diff --git a/arch/arm/include/asm/pgtable-3level-hwdef.h b/arch/arm/include/asm/pgtable-3level-hwdef.h index 2f35b4eddaa8..e7b666cf0060 100644 --- a/arch/arm/include/asm/pgtable-3level-hwdef.h +++ b/arch/arm/include/asm/pgtable-3level-hwdef.h @@ -14,6 +14,7 @@ * + Level 1/2 descriptor * - common */ +#define PUD_TABLE_BIT (_AT(pmdval_t, 1) << 1) #define PMD_TYPE_MASK (_AT(pmdval_t, 3) << 0) #define PMD_TYPE_FAULT (_AT(pmdval_t, 0) << 0) #define PMD_TYPE_TABLE (_AT(pmdval_t, 3) << 0) diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h index 4b1d9eb3908a..e7aecbef75c9 100644 --- a/arch/arm/include/asm/pgtable-3level.h +++ b/arch/arm/include/asm/pgtable-3level.h @@ -112,7 +112,7 @@ #ifndef __ASSEMBLY__ #define pud_none(pud) (!pud_val(pud)) -#define pud_bad(pud) (!(pud_val(pud) & 2)) +#define pud_bad(pud) (!(pud_val(pud) & PUD_TABLE_BIT)) #define pud_present(pud) (pud_val(pud)) #define pmd_table(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \ PMD_TYPE_TABLE) @@ -137,7 +137,7 @@ static inline pmd_t *pud_pgtable(pud_t pud) return __va(pud_val(pud) & PHYS_MASK & (s32)PAGE_MASK); } -#define pmd_bad(pmd) (!(pmd_val(pmd) & 2)) +#define pmd_bad(pmd) (!(pmd_val(pmd) & PMD_TABLE_BIT)) #define copy_pmd(pmdpd,pmdps) \ do { \ From patchwork Mon Mar 18 20:03:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595766 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29A97C54E58 for ; Mon, 18 Mar 2024 20:04:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96EE76B0093; Mon, 18 Mar 2024 16:04:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FB5D6B0095; Mon, 18 Mar 2024 16:04:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74AAF6B0096; Mon, 18 Mar 2024 16:04:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5E4036B0093 for ; Mon, 18 Mar 2024 16:04:24 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2A17C14068F for ; Mon, 18 Mar 2024 20:04:24 +0000 (UTC) X-FDA: 81911236848.13.3AE5DD9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 001D6C0020 for ; Mon, 18 Mar 2024 20:04:21 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=O3WmJ+n7; spf=pass (imf22.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792262; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xZurg04juSYF/uW9HtnhgkfMM1C/gArxmrpoVNEgvCQ=; b=K+MDgz2cx34t2kArbIZKuqd3RjW2n58jF/5Gfed+zzW3nt/YrIkLV074/ZId9FbsyZi/sW 33yYhfRvxYDcadC2Fgvd9rAf0VmWGP2BDJggfFJz5x+L9w4yOYDSltltU4FZV0Jt/HOjXd fTSAnzMdiBNtfXlZmJRl2WwAh4tgFMI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792262; a=rsa-sha256; cv=none; b=uNav+fPtE0V7A+UIvF/bPBChlYtO4rMTqtOceFBzuuQptD4bJZ+c8Py6EOAIgSbHMC9HSW ee/IMFJqUbzZDJyJO2F5kpKrdYz0gywerZDDVNKD4zuK7enlC3hlX5xDDTkxVmMqZFtGyY 3iN7sZ9gzGd/ItOVE/nTV4W9BD3Jk2g= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=O3WmJ+n7; spf=pass (imf22.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xZurg04juSYF/uW9HtnhgkfMM1C/gArxmrpoVNEgvCQ=; b=O3WmJ+n7XNNnxgnQ25kBqfo+fbeEk/1Y1UiHc/I+r1s+Jnm+U+jSTPJ08K6OeOSQz9nJQ2 FWBtcUPwIt5fVkYDpa+8pXRHJkFZDyV+JIRXsdW+4lR6GGV4VMdIwY6LuJNVvNUKn3eYrP RpTRI2839yVkqxy7+KfOsholIY7pkxg= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-590-rDlcGFdKOOaHCVeu3v0YUw-1; Mon, 18 Mar 2024 16:04:20 -0400 X-MC-Unique: rDlcGFdKOOaHCVeu3v0YUw-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-690c19ee50bso14534016d6.0 for ; Mon, 18 Mar 2024 13:04:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792258; x=1711397058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xZurg04juSYF/uW9HtnhgkfMM1C/gArxmrpoVNEgvCQ=; b=N861VHLtcctv+PP0JzztCTSOaa4zBdwW8ukkpEVyumy1moYs7YSwLoABc/aMx1mRFO XRlofHevJZPM5eHhaNPRqJI2/VmpQvER9TdYUObIxL6aoRrw3hoqtXKRI0exDZrAifaf kd52Ub6IRL7D7jnnPlMB7i0GtWtUpDM/csazZtr03C3iX9boagMy8W7rmkmCgZkce/ch iyTo463m9ePM+W5gaPtR9gsnI2YoMu+jpyJzBuLBl+3eYteGzxPuu8ixeXHsqbExozlJ qAW0Zsvwyi8HAL8448aVPw/QLxx72S3bQSqP7sztvpMu+r19eZ/rXdG+d89SpH0dVkFf 0dlA== X-Forwarded-Encrypted: i=1; AJvYcCVpRfRRBUl4EFM8Sg06fttr+6cYuVaQu3uYUBXbRoJDe1Yl2vXQpAY8aI4bZgjkDG4Xv9pcGrNYEK7K9ueuMppqzwM= X-Gm-Message-State: AOJu0Yx0Ri9EsbL23JSUzYNoiovp0mGvSUMA2kUzq76WvdRw8I2waATv GnTbtOfd/cH5qyyuuG+X8ZXtYGboRVpErqEQQpP0se2OxRZTjlY5DME6sLczLOW2+G1ccNo6um5 ubupBDVe9ue3xAYFE6h47bGnqmvU3/G3KC6e1jJWERdwuLFbd X-Received: by 2002:a05:6214:3f85:b0:690:9db6:f410 with SMTP id ow5-20020a0562143f8500b006909db6f410mr462674qvb.3.1710792258412; Mon, 18 Mar 2024 13:04:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGPMjFRvOQnrQWtPInqAdggXMmLflMtETkazIbCBO67t5p2Mbipn5U9tbb0PWyioZzi1AzvbQ== X-Received: by 2002:a05:6214:3f85:b0:690:9db6:f410 with SMTP id ow5-20020a0562143f8500b006909db6f410mr462644qvb.3.1710792258055; Mon, 18 Mar 2024 13:04:18 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:17 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com, Mark Salter , Catalin Marinas , Will Deacon Subject: [PATCH v2 08/14] mm/arm64: Merge pXd_huge() and pXd_leaf() definitions Date: Mon, 18 Mar 2024 16:03:58 -0400 Message-ID: <20240318200404.448346-9-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: 001D6C0020 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: f4keokearhys9ejas9g8b49eky8mxrgn X-HE-Tag: 1710792261-565842 X-HE-Meta: U2FsdGVkX1/Iw1ww/C5C1/D0nNxAJj7s/I8ScpB9mEKipW6AEfv2qL5hSnJkvRlevvzoTx4kcD+JY25Sgewo3CQl2m0Yx2xql9O5wBvnwlwlIvGqbTupeX1jCUKvmrLQsVJYnizECt5aF7t0gxnXVD8+9B/eQ6yHpGHToaEJDWkjZHXAsrhKPFvxes+F9La/LYJrkvmTcIkr7V7QrzkU2wu/yM4Yv9vgvjVcsHIWTUxMXo14beHOhgk6H61QaGe2VtiboQ+0Evc+lvYaa03rvleBsJdvI2XLU9ukISrpJi4YmJAvMQbrstOf+sx87riJMTOpi9pnb3kDNCVNQzO963xTqSvNZEzYo9ezEZ7eon9XyatwiRrUGe0+dZHb9vWUXw3eS21GT3764H9EX62t93UyBBuRzvuhVRHoSQuKKUaUobnI9Hfu4eACrF2juqjUZH/vJ5mjuIuiiUOYi9BFlJg5xEiqy4sqiHXope5W9EDSwIVMkXSwHLhc/hiLCUUsiCZYMORMRmYtaI09nAUShsnybUZ8qVgfQo+tQhlB04ODtxHSC8Q1ESrAZWGIHG4UFCv8LPMNVKndZ2fIeJs6Hl0y7bzh4y7pl5Y1G0K1HovfiqkYWGRTtMc/du4nBToY+M7mFIN5qSH8JqLAlBTm7SqmOlUCkrC61EXRk95SmWuNUW+GGAtG0eFhr5P5fWphwNOGBHbth2w1IOElJQ7RGcIVTWoMn5fkQOjqNx3taSNFX8KZao7cE0SM59eS2e4IMalIM3H8hrjFahcDv0Am2A/DYjNGh0o3dkVw7WSmD2Gws76l+zWLA0sIALmnDhA6BjTqZU8+4e+BMA4VP0YUzHWr90+hyuL8FPRiTxvaoh/+lz9xeeIl74h8fXEvGnbbeb0IpnGZKUuH6L36IE8e8q197S/dpJ/orxKQRAGJDpXvlfSHtKsngh/AuBlQoYTy9DekC68Vt/ZSSKnhCCX 0F8hOvsh jDO8QJ79Snl9+2iWcaD9AEmZ7whi7V+/3GcdbCOadJIYyQd4HrKGwjWNdoRfdWVe4NtvgZCyCuYGPzoeKGw24cMWdqqtiwK3TSkEfVwk/kP1/T+KRw9+VAhBImx8GTpDt+vY4bGcCLamdpSFXDciLCqyakYHlvgt27pl6rd9naxdYsd3QpVhSpBBzIxHxK8dwAndznAwvJ7QNt/Syh+G9MCN948S1ZSPrjiyn66LjEXUdas9BJT8lDeDDSSjgaPIDexfWvzNt6LGrx/xrCfk8V1ns7BzY8dgfspN7WXcIRo1UkS7AuZIyouYkQn7FAl1DppmhtbO13ibw1YWCDSY0HcMCzIrfDBsow6fjhg2mAyp/dEyZMpo5gSLma8uzIHQk6yHvsAhM9Qhz+fLR+NjQZJFK8kS+nfaMhn3ak38l8X3925sxYwzSfouxBQ4NuzaiPff4UPJHDzsMI6YPUZQmEVQI66GNMMYB+ZmxNT/be/eeB6FJF4/6zshwnUis7yPCh9Fkm6K14IgfS4lgMOS0ahutCkDIzI5Phh79G4icxICcIYF+W/96P5DsaOmr8B+PdZo1R70yg3cPzRRt5FP9VkQMpFzPLllh31xh78A8EdTUvPY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Unlike most archs, aarch64 defines pXd_huge() and pXd_leaf() slightly differently. Redefine the pXd_huge() with pXd_leaf(). There used to be two traps for old aarch64 definitions over these APIs that I found when reading the code around, they're: (1) 4797ec2dc83a ("arm64: fix pud_huge() for 2-level pagetables") (2) 23bc8f69f0ec ("arm64: mm: fix p?d_leaf()") Define pXd_huge() with the current pXd_leaf() will make sure (2) isn't a problem (on PROT_NONE checks). To make sure it also works for (1), we move over the __PAGETABLE_PMD_FOLDED check to pud_leaf(), allowing it to constantly returning "false" for 2-level pgtables, which looks even safer to cover both now. Cc: Muchun Song Cc: Mark Salter Cc: Catalin Marinas Cc: Will Deacon Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Peter Xu --- arch/arm64/include/asm/pgtable.h | 4 ++++ arch/arm64/mm/hugetlbpage.c | 8 ++------ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 401087e8a43d..14d24c357c7a 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -704,7 +704,11 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) #define pud_none(pud) (!pud_val(pud)) #define pud_bad(pud) (!pud_table(pud)) #define pud_present(pud) pte_present(pud_pte(pud)) +#ifndef __PAGETABLE_PMD_FOLDED #define pud_leaf(pud) (pud_present(pud) && !pud_table(pud)) +#else +#define pud_leaf(pud) false +#endif #define pud_valid(pud) pte_valid(pud_pte(pud)) #define pud_user(pud) pte_user(pud_pte(pud)) #define pud_user_exec(pud) pte_user_exec(pud_pte(pud)) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 0f0e10bb0a95..1234bbaef5bf 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -81,16 +81,12 @@ bool arch_hugetlb_migration_supported(struct hstate *h) int pmd_huge(pmd_t pmd) { - return pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT); + return pmd_leaf(pmd); } int pud_huge(pud_t pud) { -#ifndef __PAGETABLE_PMD_FOLDED - return pud_val(pud) && !(pud_val(pud) & PUD_TABLE_BIT); -#else - return 0; -#endif + return pud_leaf(pud); } static int find_num_contig(struct mm_struct *mm, unsigned long addr, From patchwork Mon Mar 18 20:03:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C3C8C54E58 for ; Mon, 18 Mar 2024 20:04:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 293966B0096; Mon, 18 Mar 2024 16:04:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 242EE8D0002; Mon, 18 Mar 2024 16:04:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BC8D6B0099; Mon, 18 Mar 2024 16:04:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E4F656B0096 for ; Mon, 18 Mar 2024 16:04:25 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B613F1A0811 for ; Mon, 18 Mar 2024 20:04:25 +0000 (UTC) X-FDA: 81911236890.23.50AD79B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf15.hostedemail.com (Postfix) with ESMTP id 8AFE8A002F for ; Mon, 18 Mar 2024 20:04:22 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hsHeQL8E; spf=pass (imf15.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792262; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vy04FBLzPG5Zls0F5g74nprsAX71rsrg1jdch17dleE=; b=GlnIy1wSsROukMSksD0DECvWsg0oFtTcoi9bCM+0YG/dLKGSkkx/rmfEREFVUh+845FfZ7 OWMGy8pFGZvMihHeQXNl21pqQPA6Ch9Our8km0F/UnN6zCaJihwDhbuk99JyrW01QVwoTA TJvDyUh7YAkz8tfICUmiG1+eDdCX2Kk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hsHeQL8E; spf=pass (imf15.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792262; a=rsa-sha256; cv=none; b=BVstcjRmfBIULELnOuQ0oYCimZlY0V/h9RI+vNboY/zZiQHQExOI/xO77TK3YbD+oKdDrW ung81Dd6rXb1ndYfYLEjrJLnSDxIoxOCVB+jrL7nVpwF3kWfMl5JOu0YpFPrlQVSI96BMl tidf8XuW+PafKb3csciygTVADMMVlXc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vy04FBLzPG5Zls0F5g74nprsAX71rsrg1jdch17dleE=; b=hsHeQL8EiI13epBDM2TKEQdbEJoX1HHnwiP3kwKLfl5LMQKoNGrcREF6RUz1nM4bjURhw9 0Fwj2OTdaHRzyOb+LBn6QmE5HwUamMcOilt2H2ZC6+TKoIe+Ezfl3c8QgcAdCT5VMdCOnw uee5hvLwHmVYwoB0kJ+sPShfSCWcRJg= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-591-AJl-T9yUP6q1E2U-ty6z5g-1; Mon, 18 Mar 2024 16:04:20 -0400 X-MC-Unique: AJl-T9yUP6q1E2U-ty6z5g-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-42f30752eb4so15656711cf.1 for ; Mon, 18 Mar 2024 13:04:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792260; x=1711397060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vy04FBLzPG5Zls0F5g74nprsAX71rsrg1jdch17dleE=; b=AL90Un7pIavKxvdKeGaPKBWVDaLBKxUFTTE+VdHEeDmdzkaC8bqVjaWysjf0GMfrXw gMHWV4X0Qad1/5wzwOyGrzEGkyWxGoXUBx0vNbEDptOYjG7jBmeqPFmAFvU5nKPS3gY+ FKsMqv38Wk2r4gxd3jNsDXa529tgk0ZDC/2/buMz4f0otWRqvDGwGWt9Y+KhcLKvAjyh 0h4Upan3nBGeinNCF+uG/uIWtGyI7hJiM4p2oGGwjBzsMrAJtUpHpDOeFNxXVK3J4mQY EElNpfbWLNA++lWthhUNk/iUbqJa8OyBwDwHiOJnfS3003/pwUNCr7OJ6Pt1VLI+DCue NwTg== X-Forwarded-Encrypted: i=1; AJvYcCXCfGmLbEXGw1piHwhTVzqad1Zrzeu0qEQcCNq3m5E9tBOqTdsSai2VljoFV34c1t1Xp30Fy1dwTnJXibnBYW/2NUQ= X-Gm-Message-State: AOJu0Yyfgef0cD/Yp7l5M1bGBSe6nDksMSn7IJFHzvaKl2VuAogWH1aM ArayP8yoIJrHFUkBe2JRMxrLxAchV1HPu5fRpRR1xu+0MH9rwQto2LO6Fz9sYFYwSMZJya2F+QG QTzc2CaPw1AzhINgiEPvNAOuQYaDvoffGhm0IUoXH5+oHeJ1V X-Received: by 2002:ac8:4c86:0:b0:430:c314:d150 with SMTP id j6-20020ac84c86000000b00430c314d150mr275846qtv.6.1710792259908; Mon, 18 Mar 2024 13:04:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGpXWUncergFjfMSFDTE8/2jUGQ81NOYysiHRDsucc9G0cV8OOvdubCssw9QQ67SSZupi5Ulw== X-Received: by 2002:ac8:4c86:0:b0:430:c314:d150 with SMTP id j6-20020ac84c86000000b00430c314d150mr275803qtv.6.1710792259416; Mon, 18 Mar 2024 13:04:19 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:19 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com, Michael Ellerman , Nicholas Piggin , "Aneesh Kumar K.V" , "Naveen N. Rao" Subject: [PATCH v2 09/14] mm/powerpc: Redefine pXd_huge() with pXd_leaf() Date: Mon, 18 Mar 2024 16:03:59 -0400 Message-ID: <20240318200404.448346-10-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: 8AFE8A002F X-Rspam-User: X-Stat-Signature: cu7zqstmzp9a1uxayyajx7uoiix3uqgn X-Rspamd-Server: rspam01 X-HE-Tag: 1710792262-357815 X-HE-Meta: U2FsdGVkX1++yL4TdjEHYZTUalZQtMmiLJkqmzv8oPfydGDDdaLSvlt8eCxSo6fNrC/2tElEmrIlKICtCNmvSEKZiS/6PhiXRdbNj6WK3y0L3RqW442xLzR7NGHkW14OYJDfsJNfoWw3ukUx5D5fN1iCgpo08lcSzrUUJ3NrQLISFDe5/Z4MkZI/OQCW+M18sMe9EV1DDLp6T/Ogc75QB0gxkm8CXp7ep8mt12kc3Ms538yY4wWhlEkvPEjVh+n6SAwOBisvdUQNLglMduXLe3PwIDOawSn/3zPhPfaWTloxg77Yc3EBKbWLXERwRpRsU81QsTCw1tDkKW55Dk1sHiF6r/pyKyKaCFk+QhbvO6w9zs5lw84aLNES7uD/wmgy2rFc5sdk8z06asPB4P1t734FfqlNU3tRJ85rviJPQjOosrJi63ks24VPRFvJdvBuoEbjpI31RD5c6cwEzHX7XRS3Cq653dDWBPJ4wwYSacHUuF660WOWTX+qY6N50XflMdKG+G/B7I9RLhL3yUOV3ip6cnLO+NhbSVBfM53HesrIZfbk5j6KKiPfRy+Z0XVovK1GmNE2uBj+iluMXZPNO4Ud7AsYlO2y2mSn4EWSPTm+d/L20f/vO/B7jOnfr9i/5zvIvSmrn4FmhICEwwCHtIvs7ABbg9nF+fGbiZbc2Een18JCu+BE0QtgkX0mRIs2sLTt+LESfwGIqBPsNxoHLFyFvo4O1kgwIrz29ryuuLCRdV1IJo/HAv5f/C+Yg+L3E0Gf+ADVqjWcYbAYeXBVOHxhKDPprlZDN7RS6Ta1/82eeWmVISshwESI5S2USiZFxg9FbcwnFI7VWXKlAVu/Z6UP2LIqd9N611Tjm01kjessErBAWsCxnDVHG7DT5HWp+oUFcxNal+6YWZWX/DLWwIK7HBv8rbAe8DjsAEFbsfelVG3kj9gKc1UcIH4GEMNTVg1VCV/Wb8WXkXdq+Wt 49QkbTFL KCgR5vihmCKn469iG7vOdC2vIrvGqxhbWWenSqlKjbFbpWAe3CUqMTPoyGyrOvfmOGWcR24FLBtkfSQBPiEYYrt5Ap0YujAlQGBhT5wMfu92QAZAjih9w8pzghQMqCPGPhZ+HOuIfb4GXHEFJcN7kalFthKVlM7JtRuAt58gza1EoF5tS//OiW7hu7i8gfp5swcK/Fik7UuQaTrKDW/bh8XVj6JbtX/VXXfK3B9Eh+BbZj6QLamaiBUnkU+5fr76lLXDurRZ1UDvK41EUEFIRHMsX46JvHd+X5o4wAFnbGGSgzu0PRqfqoZByDjj0x2ZKT3doVvLE5O4xYY/HYx94mcjC7VWrPIRoUM//NVR6ILWQan7T8y+wxhrfDF7c++chmjSS3LiTV5J3EFUJVn5Al/XrGxHhkOmjPRcG9vonU9058ZEAwPl6n+9yJtMuUN4HlRvR2esjWTr03z0x3TK6c4YGW/5+rINXF9Hc5xxLwUaoYkf93GwjbhHt9A0G+90QCyUksOVKsOv/+ZGKdmF8/GCazTx4xyp1PBTrYuwPw3iBroTjxxeafZMjtg85afmorekzfmuuiasNuzmyz07cv19NcJtxVNtuJDXRo/QJJJxRtX9lp88Ps4hStSko3ulO5P5g/UdTTYYpE0C1UDb9Odwj6A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu PowerPC book3s 4K mostly has the same definition on both, except pXd_huge() constantly returns 0 for hash MMUs. As Michael Ellerman pointed out [1], it is safe to check _PAGE_PTE on hash MMUs, as the bit will never be set so it will keep returning false. As a reference, __p[mu]d_mkhuge() will trigger a BUG_ON trying to create such huge mappings for 4K hash MMUs. Meanwhile, the major powerpc hugetlb pgtable walker __find_linux_pte() already used pXd_leaf() to check leaf hugetlb mappings. The goal should be that we will have one API pXd_leaf() to detect all kinds of huge mappings (hugepd is still special in this case, though). AFAICT we need to use the pXd_leaf() impl (rather than pXd_huge()'s) to make sure ie. THPs on hash MMU will also return true. This helps to simplify a follow up patch to drop pXd_huge() treewide. NOTE: *_leaf() definition need to be moved before the inclusion of asm/book3s/64/pgtable-4k.h, which defines pXd_huge() with it. [1] https://lore.kernel.org/r/87v85zo6w7.fsf@mail.lhotse Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Christophe Leroy Cc: "Aneesh Kumar K.V" Cc: "Naveen N. Rao" Cc: linuxppc-dev@lists.ozlabs.org Signed-off-by: Peter Xu --- .../include/asm/book3s/64/pgtable-4k.h | 14 ++-------- arch/powerpc/include/asm/book3s/64/pgtable.h | 27 +++++++++---------- 2 files changed, 14 insertions(+), 27 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable-4k.h b/arch/powerpc/include/asm/book3s/64/pgtable-4k.h index 48f21820afe2..92545981bb49 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable-4k.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable-4k.h @@ -8,22 +8,12 @@ #ifdef CONFIG_HUGETLB_PAGE static inline int pmd_huge(pmd_t pmd) { - /* - * leaf pte for huge page - */ - if (radix_enabled()) - return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); - return 0; + return pmd_leaf(pmd); } static inline int pud_huge(pud_t pud) { - /* - * leaf pte for huge page - */ - if (radix_enabled()) - return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE)); - return 0; + return pud_leaf(pud); } /* diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index df66dce8306f..fd7180fded75 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -262,6 +262,18 @@ extern unsigned long __kernel_io_end; extern struct page *vmemmap; extern unsigned long pci_io_base; + +#define pmd_leaf pmd_leaf +static inline bool pmd_leaf(pmd_t pmd) +{ + return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); +} + +#define pud_leaf pud_leaf +static inline bool pud_leaf(pud_t pud) +{ + return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE)); +} #endif /* __ASSEMBLY__ */ #include @@ -1436,20 +1448,5 @@ static inline bool is_pte_rw_upgrade(unsigned long old_val, unsigned long new_va return false; } -/* - * Like pmd_huge(), but works regardless of config options - */ -#define pmd_leaf pmd_leaf -static inline bool pmd_leaf(pmd_t pmd) -{ - return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); -} - -#define pud_leaf pud_leaf -static inline bool pud_leaf(pud_t pud) -{ - return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE)); -} - #endif /* __ASSEMBLY__ */ #endif /* _ASM_POWERPC_BOOK3S_64_PGTABLE_H_ */ From patchwork Mon Mar 18 20:04:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18847C54E58 for ; Mon, 18 Mar 2024 20:04:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9A8E6B009B; Mon, 18 Mar 2024 16:04:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B1AB66B009C; Mon, 18 Mar 2024 16:04:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96ED66B009D; Mon, 18 Mar 2024 16:04:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 84B596B009B for ; Mon, 18 Mar 2024 16:04:33 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5692916089F for ; Mon, 18 Mar 2024 20:04:33 +0000 (UTC) X-FDA: 81911237226.18.34F97A6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 32A301C0012 for ; Mon, 18 Mar 2024 20:04:30 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eNPwpQYv; spf=pass (imf20.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792270; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z0Y+ceGC2u4mzi2lycxKAqHeA0qAqllKwMSRaV+2/Lw=; b=krojSJrgubPlwN6AMmfv5bhmPyjUTEQsxebncouawPvmKpUyNpAFWVRNZ0ZQFYGch9NUne 94uw81vsTW9wUfaKZrlWVMgE8XgdpxUnYUaJZbKMHZbWLpmbaG+pstB8yvJD1vr2ECWkCV 1xgSIRCdZk1hPIVMST7LLN6UzCi6kac= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792270; a=rsa-sha256; cv=none; b=l6tXgYWiV7hZ9vRR39vbRfBHYMKZSmWLpLuXC6lRE7Hrxh+id42QJrT1enOuVRwHd7uuoP Ng85RLDLIfV9i581wx4deDyx5AebTPteC3VlAzWg84Kwi8YhpvQX+LNAuACOaxCXVodgtr +yeZR8pq/Yjwyr7dt1uygbx4qe3pIrs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eNPwpQYv; spf=pass (imf20.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z0Y+ceGC2u4mzi2lycxKAqHeA0qAqllKwMSRaV+2/Lw=; b=eNPwpQYvKPW0aVZZj0vPZQdUeNUhxCH1bh/qJ4jPfNagfQTyIzkoR9060uTSBfvjy07Gcp nt6Ow8x100yuJ4UC1jhJN/1vf71rjAcqK0XW5srTqrkWG95wWhRjqzgUvFSuzKTvVpNQqK rmtQ6CVfDKFwiP8k9Y9Gn0wuYbJ+PEg= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-622-6ZpROsw_OJCoygo6HV3mSA-1; Mon, 18 Mar 2024 16:04:22 -0400 X-MC-Unique: 6ZpROsw_OJCoygo6HV3mSA-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-430b4572a97so17524001cf.1 for ; Mon, 18 Mar 2024 13:04:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792261; x=1711397061; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z0Y+ceGC2u4mzi2lycxKAqHeA0qAqllKwMSRaV+2/Lw=; b=G3sAv/cOLAGxp/XSxREH9kU/s24h8Z/+GhfvVVhEM8T56BPd/3fWKabgOwwBcrx5Ku 456aXLS39SD+j1mJQERoLQf622AdI0mOJeSXmP0CmTdplyg3pkIUjKre5hmb8jTrTXVa utaYWbRgMdvdxNncHDQJNtsyNZ9eohfaFxTaU2f6TP2Sbhee3HHsJrZBWtx0/Lea+305 atEh3EEeHHEjE9YB0c0sCBilyusyoYLeV36TByhrzPcYayjmINgmMkE909kcTDPkQM2V oMwIFNHOYDwdO8WmVxSGhCH4j0ybrChiLS7G2mqhj5JdYon9tVP7DsJ+tWz3xN3p+6B5 h8Mg== X-Forwarded-Encrypted: i=1; AJvYcCXZXu7xYpHQY4AoKOHcN7x82M1hfyQj0RPl1vwm5bcv6yU5Y2IBQHBz5PD6uo69+RC5QJxAzwv5URI5clS3qmOqxgo= X-Gm-Message-State: AOJu0YyO0YktcWy/OCdqwJcrf78EkHOs8O4W9l3Fd0MLvatB6kxEaJx/ DoSkIygo75oXDvmDcf1T6TzUaFaUi8QG3rzGzOMadClxUydCty4TqkC4t4ZEWKGdpK1goaP39jI FdnWhnEaoIyy03PjTRUr7CpHdwTwDRuy2sjuhRmsl6FbwIntN X-Received: by 2002:a05:622a:15c8:b0:430:ae0a:c08f with SMTP id d8-20020a05622a15c800b00430ae0ac08fmr12238068qty.6.1710792261458; Mon, 18 Mar 2024 13:04:21 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGQhMJRmPtSMLbLztL+aVkPHyOJfqD1JG0yGIJ+aGiheaz6nxeexYpcIdXvqwuavYhzTjiHrw== X-Received: by 2002:a05:622a:15c8:b0:430:ae0a:c08f with SMTP id d8-20020a05622a15c800b00430ae0ac08fmr12238049qty.6.1710792261015; Mon, 18 Mar 2024 13:04:21 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:20 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com Subject: [PATCH v2 10/14] mm/gup: Merge pXd huge mapping checks Date: Mon, 18 Mar 2024 16:04:00 -0400 Message-ID: <20240318200404.448346-11-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: 32A301C0012 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5ji5c57r61etmus5ame3gef54w6gremc X-HE-Tag: 1710792270-30636 X-HE-Meta: U2FsdGVkX1+Tszt/KZ8vAzn1QMl7DIUUp/79wKGOHXTlMywfbKOM29HDhNaN2TOx6Hovq2wBrtDudCS0lNo6Nqn1+12YKkaq3M9nxVqFCY80duzfonETMfJdaui5Q20y7BTy444SmSv3u1sZgkOuDI2zDzDrdfpcW1f5v9sWUeFEu/jZDh00h0ihPETMuXlH3Q199Hp2PEOQnLe2iTv1TSDeYP+V2Yh7zT/xoobEpvX5Elt7QEJTEl6RUJZAr+GCpjpZ6A0DyAOW7KgAzeGI+aN2kkpZNoVdlOV3AX8dCnW2fvNbbLiSbuajGF1DiD7DMvOLJ3Ax2N48ggRNXohTAPt8bqbpAQRujpueECO2bMGQQpAdEugkvZj0zS8wVCA2AtUILnSWt7krqBEKXDyP0FQMhWTDx3rcNiqhpyKVXtjmHxdUY1inVaBZLeG7u/LUsgwVYIyIUih5wMKQHE1Ijpfiwh2Ri7vkDwrnQ4W9n8yNnhUrVsPSDwYUHm3O9jdfmNcL3iO88uOel35G0Rw9QnGE4ogwVmOhqvv0tVKUzatFA4egyKAmBfHpyHIqoJ95fvun6q9XaWkvHkmKYFgJeAfKOIs0LyrRgmkfACyFEhw/K0i0kgQyra7KkSavEilwfzaKcjt91hBxmtFkXx3Usks9KKwx0X0PgaNeytOyGz4sIBB4g+4gojsJ82W/dRSK4iygHnPKHomj8fhTAXUhZb1LZepwoZuU4Ziau8Nfq6qKHYx58hM9KdtxQc+KHhltDmMI67xL73Iy3/y0ald37yiI9z+S6KJiCnDkJSO5dY38wTy/8xTrqAd5+PHueLmCTQzMGZsUo/xs8TryMB5IZ2aAcSNof7TxLv/poZfOsxtq7DVbVerUG3iHFcS5ks1EfoXHrnrdy7pwDtHJgdZL2+bylGGpAxIV9Vd+RoaglC7Otayme0cDvdm+iJUh//KevzWe9uDYQ3kGIdoFKAH 3vDZ51W/ IyPjjAK2jHLsxMLWVjzzYi4mGW9/D21Nyvn6u7MW35rkHWPTH7X0W5BKUjHN/hLiFKmeGEYArAEY2J8utSdOkGsQrYTT8iHwlcri/DsNzdEh/eD2z5hr6Th1gUoa/Aa0nY9elhymmkVgHrJWZe90a9oe/6Ts/BByOVyiFaWELf8s7diPunik1kMUcrH8TEHAry3q6Ua7JUFMgFCliuQzJeImj34WLRHXFx2DQg5SdPvCnrWNJ1Tp68oIlltAd313C8BOUkWMnyzDu5IMuG6djuJ3vrRoVH5ABoGLc5plP42UPJrxk05D3EGidwEs52cjTl7tQTKPbl9Ald9TT6maeMoJJmZw26v3FtlKCm/5ENQ8ewxheyzgA2wL7k5uN70sHjtykLN7j0Q2tF5jlHa08MpDJkX8fAMtjy9/K X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Huge mapping checks in GUP are slightly redundant and can be simplified. pXd_huge() now is the same as pXd_leaf(). pmd_trans_huge() and pXd_devmap() should both imply pXd_leaf(). Time to merge them into one. Reviewed-by: Jason Gunthorpe Signed-off-by: Peter Xu --- mm/gup.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 802987281b2f..e2415e9789bc 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3005,8 +3005,7 @@ static int gup_pmd_range(pud_t *pudp, pud_t pud, unsigned long addr, unsigned lo if (!pmd_present(pmd)) return 0; - if (unlikely(pmd_trans_huge(pmd) || pmd_huge(pmd) || - pmd_devmap(pmd))) { + if (unlikely(pmd_leaf(pmd))) { /* See gup_pte_range() */ if (pmd_protnone(pmd)) return 0; @@ -3043,7 +3042,7 @@ static int gup_pud_range(p4d_t *p4dp, p4d_t p4d, unsigned long addr, unsigned lo next = pud_addr_end(addr, end); if (unlikely(!pud_present(pud))) return 0; - if (unlikely(pud_huge(pud) || pud_devmap(pud))) { + if (unlikely(pud_leaf(pud))) { if (!gup_huge_pud(pud, pudp, addr, next, flags, pages, nr)) return 0; @@ -3096,7 +3095,7 @@ static void gup_pgd_range(unsigned long addr, unsigned long end, next = pgd_addr_end(addr, end); if (pgd_none(pgd)) return; - if (unlikely(pgd_huge(pgd))) { + if (unlikely(pgd_leaf(pgd))) { if (!gup_huge_pgd(pgd, pgdp, addr, next, flags, pages, nr)) return; From patchwork Mon Mar 18 20:04:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595768 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C928BC54E58 for ; Mon, 18 Mar 2024 20:04:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D37B6B0098; Mon, 18 Mar 2024 16:04:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 882EC6B009A; Mon, 18 Mar 2024 16:04:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 723FB6B009B; Mon, 18 Mar 2024 16:04:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5C2276B0098 for ; Mon, 18 Mar 2024 16:04:28 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3DC5DA0E52 for ; Mon, 18 Mar 2024 20:04:28 +0000 (UTC) X-FDA: 81911237016.21.B3666B4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf21.hostedemail.com (Postfix) with ESMTP id E81981C001D for ; Mon, 18 Mar 2024 20:04:25 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Z498adPF; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792266; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C5IYGWVxUA2LCxblG8GUdWUManYweqkS1nMwnfIC++w=; b=ExsDZtM6AyN/lsf8IwSKXj/zpIucFUxPmLtxkfZrYbX6h6fFavYBls5yS74ISm3R5Up36k BYKQ/x+V/5iBEz8XaTTDAaarbjbyfuVoL+gy1Ma7cASq7U/LoGc+hlIwBTUV45a1mUxDVS eDmQhiCiS1c3Kt89Jlw0qmLI+sgWXK4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Z498adPF; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792266; a=rsa-sha256; cv=none; b=cPr+4BoX7P2a/NCqIDYWXDBZ+pYubuJ1NOi4wN7vM0/q+QPCmQpj7Ig6H8Axdum8IebPkQ rrF0jeX+vgBMQ57LuRB/lv+7+x13sI8fUpivogFt45jTdtIMnm1JdK12Vt2HIaI0ueczzA mUVesc/ws2m2srjtobgej6WIwJ+4b5o= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C5IYGWVxUA2LCxblG8GUdWUManYweqkS1nMwnfIC++w=; b=Z498adPF2UKiyFZAlIFmjrZG2Wpmhzpldr9g3qI77nrFrrQ43wqLLIvfVC55nIP3xThKqP L2htOrlfbz1s0fr29Turz39a23GsSDgCF5g6SDTe1pWAlg2a5/hjwyhd7pO/sGqnDa43hP k1tO4zlXjzj5NItW8d0VYdzDNMOMfoc= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-635-2DJazxPKP8KvZXJ0AxHMXw-1; Mon, 18 Mar 2024 16:04:23 -0400 X-MC-Unique: 2DJazxPKP8KvZXJ0AxHMXw-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-430d73c0492so3450541cf.1 for ; Mon, 18 Mar 2024 13:04:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792263; x=1711397063; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5IYGWVxUA2LCxblG8GUdWUManYweqkS1nMwnfIC++w=; b=ek24LZnnF0Z73EbRhiO7NK0j3rp68r2E1ic0HzORMQp8yHT/KW6tPzpbYesQKNsn6C QBd+pVxS84xqoOzdA8v27WF7rNNgHJM8Acmy+b3SoLSTdt2uwU8JKxmdgaNCrPbe4pYt BFDN82iso1UsU5zO5e+BheGPuGtyXb+I5qodFpHKHGhq1yWlEJ49oiJ65N1/jTTTHDTV oluuV4SembvUYBGqAML7MdLHo/QYyfxsCZ6cugicp6LBFOpavKB/dCaGcPz8N2jqgHfb cgTR2+o3TVnt5gF08n8etx3I2aDXat/AlimytiU6bMROIT5BH/e0FpEcgwtLrPXhsxov Ku2Q== X-Forwarded-Encrypted: i=1; AJvYcCXH/YQb7HT5MEfDFtDpUMfrGCCTnuwv6hrBj3ZAEnXvnzxtaaObLgctW5Jpxvn2Y949jvOhF2SixOnbo0dO2epwLcQ= X-Gm-Message-State: AOJu0YzvKZ0xu62uP6ECnG2BVYpofoJf6pHJyj1uNgmHGzT/62GqzPoj 6HfqFsfgxqkuC6q4GtFzo9qL6aI/UF3h42Etp1zgAs/2471lSAod/tpElnIfOuXka4N4g0MgP5+ 8FqU9Vj7u8XwuM+sR4E5b/sBt6yvgZS2N7/5VMTWUkXqEXEYR X-Received: by 2002:ac8:5f94:0:b0:430:a3bd:178b with SMTP id j20-20020ac85f94000000b00430a3bd178bmr450225qta.2.1710792263111; Mon, 18 Mar 2024 13:04:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHp9pbq1yp/IEiDWtUuy/HeuR42qObaCDKOoRIIvKhBVTuD6LDz0uhx2Vxyh/zWi2fCOK+MTg== X-Received: by 2002:ac8:5f94:0:b0:430:a3bd:178b with SMTP id j20-20020ac85f94000000b00430a3bd178bmr450185qta.2.1710792262668; Mon, 18 Mar 2024 13:04:22 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:22 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com Subject: [PATCH v2 11/14] mm/treewide: Replace pXd_huge() with pXd_leaf() Date: Mon, 18 Mar 2024 16:04:01 -0400 Message-ID: <20240318200404.448346-12-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E81981C001D X-Stat-Signature: yt8gwcgijwiap1f985dd9qzba88i8jsh X-HE-Tag: 1710792265-744719 X-HE-Meta: U2FsdGVkX196avUdUFtUFF9h9laKnTdMOCZyAHoX/eqiNUbimhw+rOTF3e55YXqf1b5q6VhVwHnOLm6yYFq6CfZiCIDjKvJYvF8db36cT0qF35VR7U9KW+3xSphtqk/z8K4TrhrU8pWq+3jdD642zpohuICNFLXPcSsMAvAzH464d8iI/TZNSZUcbqKlCVMaldOhK2RzdX1YE6OumqXuKdSUx1DGHlFCSpW1K3Lw3hFPDEQO8LIAguTKNPpl+9O9+t6AH8mqtLw5kB5obpN+7C+Y1dgBBh037Kqyrz7/NMtipF244FIdoszSTvvCtKO0Yk0kuzTRFoPcIqxfl/49p/EdKNl/TSXPfXyljSRgGHtFU+TC90AO3RsGpZc1Ky3bza/uCNqYX5TGE0MbOdhAqm3lfAOqmIk29w801665AH15Bz1/46zy3DxVPDw8ZDn+wc/H2kmIfvwlFhlm0uyS8Vu+jrrHehMQAXyRjPMsWI4XQ3BEluNfNOEDoGaolzBvp6NER60onXxOjzwux/3EkCxJqTYyt+lcrAmq9bAfBrABgXuaJa7GhmmP1a0yNWxa6ZtKc9kzrHn77JULzq6lN4YckM9oZlHMla9rKo004CjRae0RFVo0B95CKIuvTJAgdNHIVxeJ2W9YX9pYOI0P8NJg29EJaDN6KC8pnoLYDrtcLNKjn/PQ7eZ5ZAy4B3XFA96zN5BSkmuQQ8P8HVeWVTttNtPmV0XnuH1D3W8/61FMOjKbW3zdYZTlVKNvpJzoVbedZHi/RN/JkVp5RrPyYaBUZPwct+TkBEu86J8UMxF1cRdmWmFK9tdkkbUqWSn88VxYCJiF9DIwxxWZHPiX0qlLHqJo8R0RN0iinCZl2k1qnti9nlnsUEtbmHuBOPmMRnWlozy4e+M7mDIlhozLaebSRewsVqYFeWsyojhqjGy9Mw8f6j3/99pMDOHohgweIIhak9V81BCfvVc39zc VSp5lleW uHnqAaculRn423ttf5dq+h6RTVvcBHxq3GT/2NOAlC+bYvoIKN2MyjbOFhqgp8LetOff80q7t9KYxjYa4Z1o6feB83tyW+uaG8zHB0d2fi1llGy7qVwrVk2nDyeORtj0maDd+LvbBqoF4eQJVPEv6kbFrPadKso3GxQ+QW2y/hxNN6UuvPKsdDNPvZu3u5QfjzehSsg7325GptqAofJjWLZIhXRQEVzdz7MFDmP+dlitXor19EjoYtpkqgWxrrM3BGqmJ1tOsoAMiEflLtpXKisae5aQuaeCjUlBWT+Uar153q5eQ7Abh8fkxsWCX48M1HBx3tPlTurFw9SIs/ikvTJBDOxpF2h7oOc31F4XCrlwk07jKMYueLQtIMoybUrzbPDS0M+AGhMiiecrpNU/bKTy4fdE0Vxm8ZfFtUgP00XIcB2AzFT3LliRwYA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu Now after we're sure all pXd_huge() definitions are the same as pXd_leaf(), reuse it. Luckily, pXd_huge() isn't widely used. Signed-off-by: Peter Xu --- arch/arm/include/asm/pgtable-3level.h | 2 +- arch/arm64/include/asm/pgtable.h | 2 +- arch/arm64/mm/hugetlbpage.c | 4 ++-- arch/loongarch/mm/hugetlbpage.c | 2 +- arch/mips/mm/tlb-r4k.c | 2 +- arch/powerpc/mm/pgtable_64.c | 6 +++--- arch/x86/mm/pgtable.c | 4 ++-- mm/gup.c | 4 ++-- mm/hmm.c | 2 +- mm/memory.c | 2 +- 10 files changed, 15 insertions(+), 15 deletions(-) diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h index e7aecbef75c9..9e3c44f0aea2 100644 --- a/arch/arm/include/asm/pgtable-3level.h +++ b/arch/arm/include/asm/pgtable-3level.h @@ -190,7 +190,7 @@ static inline pte_t pte_mkspecial(pte_t pte) #define pmd_dirty(pmd) (pmd_isset((pmd), L_PMD_SECT_DIRTY)) #define pmd_hugewillfault(pmd) (!pmd_young(pmd) || !pmd_write(pmd)) -#define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd)) +#define pmd_thp_or_huge(pmd) (pmd_leaf(pmd) || pmd_trans_huge(pmd)) #ifdef CONFIG_TRANSPARENT_HUGEPAGE #define pmd_trans_huge(pmd) (pmd_val(pmd) && !pmd_table(pmd)) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 14d24c357c7a..c4efa47fed5f 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -512,7 +512,7 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) return pmd; } -#define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd)) +#define pmd_thp_or_huge(pmd) (pmd_leaf(pmd) || pmd_trans_huge(pmd)) #define pmd_write(pmd) pte_write(pmd_pte(pmd)) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 1234bbaef5bf..f494fc31201f 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -321,7 +321,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, if (sz != PUD_SIZE && pud_none(pud)) return NULL; /* hugepage or swap? */ - if (pud_huge(pud) || !pud_present(pud)) + if (pud_leaf(pud) || !pud_present(pud)) return (pte_t *)pudp; /* table; check the next level */ @@ -333,7 +333,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, if (!(sz == PMD_SIZE || sz == CONT_PMD_SIZE) && pmd_none(pmd)) return NULL; - if (pmd_huge(pmd) || !pmd_present(pmd)) + if (pmd_leaf(pmd) || !pmd_present(pmd)) return (pte_t *)pmdp; if (sz == CONT_PTE_SIZE) diff --git a/arch/loongarch/mm/hugetlbpage.c b/arch/loongarch/mm/hugetlbpage.c index 1e76fcb83093..a4e78e74aa21 100644 --- a/arch/loongarch/mm/hugetlbpage.c +++ b/arch/loongarch/mm/hugetlbpage.c @@ -64,7 +64,7 @@ uint64_t pmd_to_entrylo(unsigned long pmd_val) { uint64_t val; /* PMD as PTE. Must be huge page */ - if (!pmd_huge(__pmd(pmd_val))) + if (!pmd_leaf(__pmd(pmd_val))) panic("%s", __func__); val = pmd_val ^ _PAGE_HUGE; diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c index 4106084e57d7..76f3b9c0a9f0 100644 --- a/arch/mips/mm/tlb-r4k.c +++ b/arch/mips/mm/tlb-r4k.c @@ -326,7 +326,7 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte) idx = read_c0_index(); #ifdef CONFIG_MIPS_HUGE_TLB_SUPPORT /* this could be a huge page */ - if (pmd_huge(*pmdp)) { + if (pmd_leaf(*pmdp)) { unsigned long lo; write_c0_pagemask(PM_HUGE_MASK); ptep = (pte_t *)pmdp; diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c index 9b99113cb51a..6621cfc3baf8 100644 --- a/arch/powerpc/mm/pgtable_64.c +++ b/arch/powerpc/mm/pgtable_64.c @@ -102,7 +102,7 @@ struct page *p4d_page(p4d_t p4d) { if (p4d_leaf(p4d)) { if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP)) - VM_WARN_ON(!p4d_huge(p4d)); + VM_WARN_ON(!p4d_leaf(p4d)); return pte_page(p4d_pte(p4d)); } return virt_to_page(p4d_pgtable(p4d)); @@ -113,7 +113,7 @@ struct page *pud_page(pud_t pud) { if (pud_leaf(pud)) { if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP)) - VM_WARN_ON(!pud_huge(pud)); + VM_WARN_ON(!pud_leaf(pud)); return pte_page(pud_pte(pud)); } return virt_to_page(pud_pgtable(pud)); @@ -132,7 +132,7 @@ struct page *pmd_page(pmd_t pmd) * enabled so these checks can't be used. */ if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP)) - VM_WARN_ON(!(pmd_leaf(pmd) || pmd_huge(pmd))); + VM_WARN_ON(!pmd_leaf(pmd)); return pte_page(pmd_pte(pmd)); } return virt_to_page(pmd_page_vaddr(pmd)); diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index ff690ddc2334..d74f0814e086 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -731,7 +731,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) return 0; /* Bail out if we are we on a populated non-leaf entry: */ - if (pud_present(*pud) && !pud_huge(*pud)) + if (pud_present(*pud) && !pud_leaf(*pud)) return 0; set_pte((pte_t *)pud, pfn_pte( @@ -760,7 +760,7 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot) } /* Bail out if we are we on a populated non-leaf entry: */ - if (pmd_present(*pmd) && !pmd_huge(*pmd)) + if (pmd_present(*pmd) && !pmd_leaf(*pmd)) return 0; set_pte((pte_t *)pmd, pfn_pte( diff --git a/mm/gup.c b/mm/gup.c index e2415e9789bc..8e04a04ef138 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -778,7 +778,7 @@ static struct page *follow_p4d_mask(struct vm_area_struct *vma, p4d = READ_ONCE(*p4dp); if (!p4d_present(p4d)) return no_page_table(vma, flags); - BUILD_BUG_ON(p4d_huge(p4d)); + BUILD_BUG_ON(p4d_leaf(p4d)); if (unlikely(p4d_bad(p4d))) return no_page_table(vma, flags); @@ -3070,7 +3070,7 @@ static int gup_p4d_range(pgd_t *pgdp, pgd_t pgd, unsigned long addr, unsigned lo next = p4d_addr_end(addr, end); if (!p4d_present(p4d)) return 0; - BUILD_BUG_ON(p4d_huge(p4d)); + BUILD_BUG_ON(p4d_leaf(p4d)); if (unlikely(is_hugepd(__hugepd(p4d_val(p4d))))) { if (!gup_huge_pd(__hugepd(p4d_val(p4d)), addr, P4D_SHIFT, next, flags, pages, nr)) diff --git a/mm/hmm.c b/mm/hmm.c index c95b9ec5d95f..93aebd9cc130 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -429,7 +429,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end, return hmm_vma_walk_hole(start, end, -1, walk); } - if (pud_huge(pud) && pud_devmap(pud)) { + if (pud_leaf(pud) && pud_devmap(pud)) { unsigned long i, npages, pfn; unsigned int required_fault; unsigned long *hmm_pfns; diff --git a/mm/memory.c b/mm/memory.c index 904f70b99498..baee777dcd2d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2765,7 +2765,7 @@ static int apply_to_pmd_range(struct mm_struct *mm, pud_t *pud, unsigned long next; int err = 0; - BUG_ON(pud_huge(*pud)); + BUG_ON(pud_leaf(*pud)); if (create) { pmd = pmd_alloc_track(mm, pud, addr, mask); From patchwork Mon Mar 18 20:04:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13595769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23F48C54E69 for ; Mon, 18 Mar 2024 20:04:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCAB76B009A; Mon, 18 Mar 2024 16:04:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2DEE6B009B; Mon, 18 Mar 2024 16:04:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A57346B009C; Mon, 18 Mar 2024 16:04:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9216B6B009A for ; Mon, 18 Mar 2024 16:04:32 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 71985C09C0 for ; Mon, 18 Mar 2024 20:04:32 +0000 (UTC) X-FDA: 81911237184.30.9E7160B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 55FAB1C001F for ; Mon, 18 Mar 2024 20:04:30 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=B6o8mPGZ; spf=pass (imf20.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710792270; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JXDMCLhq+SpXi29RlzCNqEUhIAFpzfV1ImIXAFsvuS0=; b=Eo3ah1h3AfWQLxy00/QIfgL0452GidBfgL5mbR0C6bU3jC7Pusa0//8LXCWyLDgHtvz9Ne yRUvvax5YK9Qv49qh/6KYGPWerlDUKZ6Zc6lMiSxoXqCgf6QG0FmmwZJRtJ0VhKDzvuJh2 ooY1F0REa4tGU/DcIc+bJ3tZB/j4mVo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710792270; a=rsa-sha256; cv=none; b=8PrKXKJ6caqd3tKanQ92viHPgDaNi/r1cosCAk9WssCWQsnFZ5pQll+ttqvCgXDo3luIaJ JHDWqNp6POyXaeu6bF5fnJdU3+7y84Z06qKp++cIOL0TtLsIusHN5CQYPEMOKYW0rSISr4 cwUxP6BxdWgIBAlEZgnvgSj5zam+UKE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=B6o8mPGZ; spf=pass (imf20.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710792269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JXDMCLhq+SpXi29RlzCNqEUhIAFpzfV1ImIXAFsvuS0=; b=B6o8mPGZAywSLNnVx1wf8uB8J2Qi0hm7+sL992HCsi3nMSIwlIUeHOWhN0YSA3kEe6TMOy GWeguzk/pUcFIcy3GpY/HCQ0Qrx480cTznfd7Qs63cXajGuZiHp0+leOpVwEFNchEFinke 0aYrB9bQSDalyEhMbmjuDRqM0VU3b8Q= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-9-_LNssHFlPHCPfwzXwaVaSg-1; Mon, 18 Mar 2024 16:04:27 -0400 X-MC-Unique: _LNssHFlPHCPfwzXwaVaSg-1 Received: by mail-qt1-f198.google.com with SMTP id d75a77b69052e-42f30752eb4so15657051cf.1 for ; Mon, 18 Mar 2024 13:04:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710792266; x=1711397066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JXDMCLhq+SpXi29RlzCNqEUhIAFpzfV1ImIXAFsvuS0=; b=EiDr9gYwE27uhsIGaYg8lO5qbuy6s65Dz96oU5s3MJ61IGY1laRHB58X/niHyR7CGN 99NHINFEOsSPmiFveaH640EhiKquBS8awktWM2zxPuac1X/DU+wj1DjwwIEbGIUt8q79 bhps8biaIj0EvVWVwsdtHlHWxhSEDWL2IQj7WdAQuULzSxyyD0fnPAhAhh8gBQNKt+L1 DxoeOVY+FyIorxTaBhSqGhN66jMKrBnxBDzFPk0iqlYRsNHeCHY0xb4ciUQx5NXAWsJZ rJeVK6RHKYOXLiIWZcX/Z0yCMoMfjnUvfpHnaNskd5wwSLpf2ll0XdCub5UckTJqxq8o MuGA== X-Forwarded-Encrypted: i=1; AJvYcCWJHLSzQZ2EJh0BKRVM2ir4QCKCaFmsyvFShlEOVG4vVDDeVTXGsEnnd42HYZJq/8dL/s72bdDkUrpG0xlDECNYJKM= X-Gm-Message-State: AOJu0YwmEo7PV4vlZlJebfcPDpkNmbt4WRb1aHljCgPyLzR4MkB8fPvq hptrlsV7Ca2WD5uukCgjzoE5vwCjnwCOW++KggWU1WZa+XbkRKpvGTnSSW9Mqsv21JTBDgWVxqN JnwTmRDJlQv2D2XAz0f6YRutcrM9t6G6iiEnaZ1DjfN3G8pz7 X-Received: by 2002:ac8:5f91:0:b0:430:c2dc:a5ef with SMTP id j17-20020ac85f91000000b00430c2dca5efmr447851qta.2.1710792266526; Mon, 18 Mar 2024 13:04:26 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHBYHTFmkqNiU+ToXou6wXh4FVDhWI9gJPxxfj+3GqwNDwAz7q/kbYXHDobvjaQIuNomZFyOQ== X-Received: by 2002:ac8:5f91:0:b0:430:c2dc:a5ef with SMTP id j17-20020ac85f91000000b00430c2dca5efmr447812qta.2.1710792266091; Mon, 18 Mar 2024 13:04:26 -0700 (PDT) Received: from x1n.. ([99.254.121.117]) by smtp.gmail.com with ESMTPSA id hj10-20020a05622a620a00b0042ebbc1196fsm3484491qtb.87.2024.03.18.13.04.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Mar 2024 13:04:25 -0700 (PDT) From: peterx@redhat.com To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Andrew Morton , x86@kernel.org, Muchun Song , Mike Rapoport , Matthew Wilcox , sparclinux@vger.kernel.org, Jason Gunthorpe , linuxppc-dev@lists.ozlabs.org, Christophe Leroy , linux-arm-kernel@lists.infradead.org, peterx@redhat.com, Mark Salter , Catalin Marinas , Will Deacon , Russell King , Shawn Guo , Krzysztof Kozlowski , Bjorn Andersson , Arnd Bergmann , Konrad Dybcio , Fabio Estevam Subject: [PATCH v2 13/14] mm/arm: Remove pmd_thp_or_huge() Date: Mon, 18 Mar 2024 16:04:03 -0400 Message-ID: <20240318200404.448346-14-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240318200404.448346-1-peterx@redhat.com> References: <20240318200404.448346-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Stat-Signature: n3f35htrcopm8gki7wbpo83zsqrg7zu4 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 55FAB1C001F X-Rspam-User: X-HE-Tag: 1710792270-956082 X-HE-Meta: U2FsdGVkX1/vp+ppWwg6M+bpC48kIVAq8MJ/PECJD7pCCEsrAfAHsyLc/c1BNSzzzVZPbVFUho+OZCuApdLbEnTXiZtyptoUwe6NA3Wbwz5i2U0LQ7bKBKuwLiW35sRztizalvsUAWMxlV1/VUvSQZ3N5nS3COFfdPRDjloMv5GaGmLkDDOIJ5niBLr8f8rftgKRDGojWlIGMOEKa0njDieyhI5iv+3bS0zG0KWwueOEmScrffBDtx0sDfhBDnZQmEuOf2JBv9AZ76deLE+qfWkj/JulYiYEBrSSD6cO/slk9IqNsWMUVSy7e+4A5+/OQxCWAhF7TKHBsirqPbkOzuVXOXpRO+gph7s+2kE+bEfpg7zjSiOd/7rgFYDh0p+P9l8TRSByZOgvQSsWq7FKhWedL8uLTrVSXgEJfQJXpWjkxz/o8+9vXIyMyhJ2XcBs48q2jVfbf6P4mocOXXf88/N+0FSkR+4SG4TzFumXASoisAKhGyFN4yUwd5yUrWc9djmFiBjXsfAMFuPk+WXSEsGDoWi/TTnj19/GtU9pR1hur1iGTomHmTM0u5ykS3Li5978lJ3VDuzphO8vrrHClQi8XLTfjC8j7YE1GG+45JG011Umx5PcDBhM6B7x/LSjwt0wC6FP87hV3WqJkbBvcyDTImdzh90nPBMmdwKxufaxexV9T4zneurLrslwakMQHq+QTQ77L2T8oOQ6U3Xvsc/URirSGLqJGfBjt+lldQ/oAhLZPMsSfls/P4zny20tJpvqE8hTAPbQgJaTpffkMqlk8PONOpQ5+mJYUmOfD6lqkfV/hRemS6yEdtMIESsiWdFBu6HQNPtfansKmAUg3psblsiB5YcHg6gDdBSyZLCByiMGiPxEUOvwXct50nh16cLjaDGnPWG/023cH2gkV7HEsPjQDKYRZHE3BW7tyBqfi4SagPcDLKVaHqK87O21JvyDvg+6QQpXDjLeecw FRy0FVj5 gtgiXabBUKedoxW8ikSrV+s5tbBEnF0XfJCGeGSJgt5C3omF4L8oxNJMZ1y7oycrsoSR1B68+qL71wji2eSGnYmvfr0Lu/b3+OuK6ng4mxO8GU/xPlPddd7qdHObydX6uCyDAvy4p9E1wpV82bpseX6J+fpmsUxGbILqEyQ832m0urCuAGP1s0da7lk/kun6xwLe7jXPCbrupbVsm8t8L7lySgY1PboGmKPsO0cS8a9i1tPP1Rcy7KCHxVdqJhl2/5B/IQIkCjK9hfaqd/6qwIxcOsPsh5+mqw92ztphluhS2Veddn9vrVQ5FinhSPDbW2Yp5jbIyo22JxkT7RIw6VJjZgIs2DCrEIEjfRbvAFsgWleDHlWaqY4jYSqF5eAxzjgcTslMe1odvuQw1lbtGiLjyp2YhuP1D238l/OyyFtOlhEIHztU2wBAvzaVjHvpg4bpiHBs8DpLVcZ0XF85TIs3EiMFxMK83n6KV2MPa20ZBVS0GEYK5bo+5wtMwklz/ElfLtNjtRdAWkcin1BJH6CUQSEEtLFVszT6BVXIiCOc+K+rdg7ZH08ITb4hFEa6y/MVv8ARNtIu6phJ4fckbHUPKZ9cH24P+UWGDlkqomPa+EkgIKNmgPSO4WI+hrMxBQLQlT1ouRSNVz9NzAsyKgNpAtw6vM6iLgXi8u79/u81YW2DHPOEum6dSqO1eih7axX5UD0bHcGFYTUEc9B+ozFRLLtfeGwSsxRB2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Peter Xu ARM/ARM64 used to define pmd_thp_or_huge(). Now this macro is completely redundant. Remove it and use pmd_leaf(). Cc: Mark Salter Cc: Catalin Marinas Cc: Will Deacon Cc: Russell King Cc: Shawn Guo Cc: Krzysztof Kozlowski Cc: Bjorn Andersson Cc: Arnd Bergmann Cc: Konrad Dybcio Cc: Fabio Estevam Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Peter Xu --- arch/arm/include/asm/pgtable-2level.h | 1 - arch/arm/include/asm/pgtable-3level.h | 1 - arch/arm/lib/uaccess_with_memcpy.c | 4 ++-- arch/arm64/include/asm/pgtable.h | 2 -- 4 files changed, 2 insertions(+), 6 deletions(-) diff --git a/arch/arm/include/asm/pgtable-2level.h b/arch/arm/include/asm/pgtable-2level.h index 4245c2e74720..6b5392e20f41 100644 --- a/arch/arm/include/asm/pgtable-2level.h +++ b/arch/arm/include/asm/pgtable-2level.h @@ -241,7 +241,6 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long addr) * define empty stubs for use by pin_page_for_write. */ #define pmd_hugewillfault(pmd) (0) -#define pmd_thp_or_huge(pmd) (0) #endif /* __ASSEMBLY__ */ diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h index 9e3c44f0aea2..fa5939eb9864 100644 --- a/arch/arm/include/asm/pgtable-3level.h +++ b/arch/arm/include/asm/pgtable-3level.h @@ -190,7 +190,6 @@ static inline pte_t pte_mkspecial(pte_t pte) #define pmd_dirty(pmd) (pmd_isset((pmd), L_PMD_SECT_DIRTY)) #define pmd_hugewillfault(pmd) (!pmd_young(pmd) || !pmd_write(pmd)) -#define pmd_thp_or_huge(pmd) (pmd_leaf(pmd) || pmd_trans_huge(pmd)) #ifdef CONFIG_TRANSPARENT_HUGEPAGE #define pmd_trans_huge(pmd) (pmd_val(pmd) && !pmd_table(pmd)) diff --git a/arch/arm/lib/uaccess_with_memcpy.c b/arch/arm/lib/uaccess_with_memcpy.c index 2f6163f05e93..c0ac7796d775 100644 --- a/arch/arm/lib/uaccess_with_memcpy.c +++ b/arch/arm/lib/uaccess_with_memcpy.c @@ -56,10 +56,10 @@ pin_page_for_write(const void __user *_addr, pte_t **ptep, spinlock_t **ptlp) * to see that it's still huge and whether or not we will * need to fault on write. */ - if (unlikely(pmd_thp_or_huge(*pmd))) { + if (unlikely(pmd_leaf(*pmd))) { ptl = ¤t->mm->page_table_lock; spin_lock(ptl); - if (unlikely(!pmd_thp_or_huge(*pmd) + if (unlikely(!pmd_leaf(*pmd) || pmd_hugewillfault(*pmd))) { spin_unlock(ptl); return 0; diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index c4efa47fed5f..a580068a6564 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -512,8 +512,6 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) return pmd; } -#define pmd_thp_or_huge(pmd) (pmd_leaf(pmd) || pmd_trans_huge(pmd)) - #define pmd_write(pmd) pte_write(pmd_pte(pmd)) #define pmd_mkhuge(pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT))