From patchwork Fri Apr 4 13:14:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECD2CC36010 for ; Fri, 4 Apr 2025 13:16:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE07428000D; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8E7228000C; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D07BD28000D; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AE1B328000C for ; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DAB3314017B for ; Fri, 4 Apr 2025 13:16:20 +0000 (UTC) X-FDA: 83296410120.30.1EEB913 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf22.hostedemail.com (Postfix) with ESMTP id BD51CC0005 for ; Fri, 4 Apr 2025 13:16:18 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ny2OzY9C; spf=pass (imf22.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772579; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KdeR7X0VD5TGF1E859GfE1E2NKkX8Ya0qtNSI/kkmbg=; b=r4PUXPYXMUio5ZJgRbbgCz/CsZeEVhTod+3G7k1RBTMtP3d0pH72NFJsq4AHkaptzLwrAY LTmiEnJt+eOiBWqtqhJhB7PptIpxqXwaO8Ba3y/wr7l0n9NyKdNNN8aL/yXvriVml5sCVB I+hFzcbVttpLSKWxYvx3dt/e4m+wsCQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772579; a=rsa-sha256; cv=none; b=2h0C1WsLrs6HjEJAgtKuvG94yh+lCD5KKRSxcUra8O7KoQct8QJD7uar038RzOLOtfkiKP S+D7vdfj0mYp/E5r6eWBdeGZGUPk38y62tCB58O2I+WpckYYrNH6DEk6liKECRPqivhmDH 6WJJq8QZFj5j7YBjbB8bQz/VXbll9KU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ny2OzY9C; spf=pass (imf22.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772579; x=1775308579; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Xxt+3lL+Fu5dj056IVRqfGZK76TUDUg49twetCywmu0=; b=ny2OzY9CoPgDsyqhySeQ7S7NoVtKzOZJ682Gqw+rTjoi9QPSAcwLYYCA 9IzPGDG0kzWlJqvYVowcsSVS9UEBOL81p9PpvISMFo8S/m5faaE5ipz5U WLyiHXDVxSMRK2n29vNuxdmY8yNutXQVEVhY9Zzu4FJkdvSsDj7ToYq5k JY+hdJoVGdFRbD7LR6sEKQ1PBLoUwOMmJ8aBT5QQryPVSR5+wKuljAz3v Zt9uMGHxjNGzKoL4VXeaUZzbadS0E2LoiVQONnmPv7uIE7oeSFA9zxBQ6 lWIB+z0LRM+7ywWWJrKOKqx5D4PG/npJt4B546l3qCE7MLt87A/RXBPgx g==; X-CSE-ConnectionGUID: n0bmJrwHQfm5aPpz+UitRw== X-CSE-MsgGUID: f9Vblp/mRz6BkG2fkoGWTw== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401770" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401770" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:18 -0700 X-CSE-ConnectionGUID: 8CO0HA7DQ/SZXENmyMAkLw== X-CSE-MsgGUID: PBphbyn9T9ueR/8k6hXkMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157194" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:02 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 06/14] x86: Physical address comparisons in fill_p*d/pte Date: Fri, 4 Apr 2025 15:14:10 +0200 Message-ID: <926742095b7e55099cc48d70848ca3c1eff4b5eb.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: r3cp7c49i5aw5fio11ugx7kxqwgq84tj X-Rspam-User: X-Rspamd-Queue-Id: BD51CC0005 X-Rspamd-Server: rspam08 X-HE-Tag: 1743772578-239654 X-HE-Meta: U2FsdGVkX19mhHXCTZkpP1peEWUrUeoCTcwzftATdFInE0PDXiy4M/caFNgZYooF9bkAututmub0P1dc0vrJ6Y7kLw2e5IM+wn7oiueAkGnjBBG6ZHg7252SQVkDY9q8zRtGJTpe00+it+FhU4syP5ITp/May/R7SeaJc2pXFUuR/jI25pNcArkpsrKeXWs3TAjg4Puqe4u/it3N89j4DJzoxiIEpT0FlmaNPfMzidvlfHyG7UVFGjn/LGlvlN9avtmZJ2o2fnhBptRWZ0UJvv7cJz9oGtzh8GfYI7ccobADC7XH6x9LUwTUuYaR8XLnx5nqnILrROhfFpBQbcs7h7wpUw46kt5RKej1qbhdqD5kQMgkMYTAZZ3gTAsOArEcSj1Mxk9pmvlMsrGDv+1kQStmwf1ALhpg8YT4ap21bp4hNLa4zREBXkRVE25ny2rkXbqzmGllYOcPevL9yiZGqenlhjO+BRJ/RPJ9c7x25JUDNY9dmg89HGrt3OnUchBp/qdPa2+DG6IdH1Lr51r0kQAJ8KBVBKZzz15YNicwJtXBMblT4Mf3uHvh2YJde4KrjOjnxOKKGqyRHL/WClVWkvUOUpknQp/1WUnpZCo97F+YSqnwj3Nqa3pdFZBGWARDo0f0HOE+n4BD4kP15FChfjAs78ySlfglco+xNziVx8Y0SvrRBA6bx0a0TIA6g6l68Qk0Tk6PkEAxqnFhdN1MAwlDpexxwpSBJD65CZWCy1CrSHX1ffmmUMqVHKjtDdYS2HE/MaG1DAv35FmEiltKxXvD2Z+ZKswRL67ZHfShhswsfU2nk/ELvPGMsULnCuwzWYTozLgoIYYjRvs3T7BvJGG3QfeHyJQuifiA0p+a+IED2nUUEZljk9dbDtBt+GB0YgjDGMCHePTSe+UiCYXmK3GwkPRhbTFvhldpxuFQC1C3uNSW/dtxlWnNZQ7g5QxOn3u1EUfRYYQQohL5kEa Is4Knn7d MDorTqllAjEf1r5d1rfMTf9NLBaBjDmY5cLOiwQVE+F/SSrx4dP2MI1tG544Wij+u6cyVg/Mc5iyQGyn3v3Ehc8RzmiZWU4BEz2j7uS3mF06T5tRWjZGsVS10hJ6rxD3Ho+7cikxc+8hUi41gqHJhOv3R4hFXBOHqdQ6hDwM/YaAwVvWyzdIh6DC37nlJj25Tzfu1iLPsSFBb4krclypMzBWEJdihLScOpyUsLvv3cWdNgYCi4Sr7g4lzGqxGR8C4dE8flp5xTbfpXlgTvnhTgvourQ5KS+4SUCXNYhvTnC53rcM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Calculating page offset returns a pointer without a tag. When comparing the calculated offset to a tagged page pointer an error is raised because they are not equal. Change pointer comparisons to physical address comparisons as to avoid issues with tagged pointers that pointer arithmetic would create. Open code pte_offset_kernel(), pmd_offset(), pud_offset() and p4d_offset(). Because one parameter is always zero and the rest of the function insides are enclosed inside __va(), removing that layer lowers the complexity of final assembly. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Open code *_offset() to avoid it's internal __va(). arch/x86/mm/init_64.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 519aa53114fa..d40699c16f14 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -251,7 +251,10 @@ static p4d_t *fill_p4d(pgd_t *pgd, unsigned long vaddr) if (pgd_none(*pgd)) { p4d_t *p4d = (p4d_t *)spp_getpage(); pgd_populate(&init_mm, pgd, p4d); - if (p4d != p4d_offset(pgd, 0)) + + if (__pa(p4d) != (pgtable_l5_enabled() ? + (unsigned long)pgd_val(*pgd) & PTE_PFN_MASK : + __pa(pgd))) printk(KERN_ERR "PAGETABLE BUG #00! %p <-> %p\n", p4d, p4d_offset(pgd, 0)); } @@ -263,7 +266,7 @@ static pud_t *fill_pud(p4d_t *p4d, unsigned long vaddr) if (p4d_none(*p4d)) { pud_t *pud = (pud_t *)spp_getpage(); p4d_populate(&init_mm, p4d, pud); - if (pud != pud_offset(p4d, 0)) + if (__pa(pud) != (p4d_val(*p4d) & p4d_pfn_mask(*p4d))) printk(KERN_ERR "PAGETABLE BUG #01! %p <-> %p\n", pud, pud_offset(p4d, 0)); } @@ -275,7 +278,7 @@ static pmd_t *fill_pmd(pud_t *pud, unsigned long vaddr) if (pud_none(*pud)) { pmd_t *pmd = (pmd_t *) spp_getpage(); pud_populate(&init_mm, pud, pmd); - if (pmd != pmd_offset(pud, 0)) + if (__pa(pmd) != (pud_val(*pud) & pud_pfn_mask(*pud))) printk(KERN_ERR "PAGETABLE BUG #02! %p <-> %p\n", pmd, pmd_offset(pud, 0)); } @@ -287,7 +290,7 @@ static pte_t *fill_pte(pmd_t *pmd, unsigned long vaddr) if (pmd_none(*pmd)) { pte_t *pte = (pte_t *) spp_getpage(); pmd_populate_kernel(&init_mm, pmd, pte); - if (pte != pte_offset_kernel(pmd, 0)) + if (__pa(pte) != (pmd_val(*pmd) & pmd_pfn_mask(*pmd))) printk(KERN_ERR "PAGETABLE BUG #03!\n"); } return pte_offset_kernel(pmd, vaddr);