From patchwork Tue Feb 18 08:15:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13979248 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFAC7C02198 for ; Tue, 18 Feb 2025 08:20:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B1B32800FA; Tue, 18 Feb 2025 03:20:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 589532800F9; Tue, 18 Feb 2025 03:20:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 429B42800FA; Tue, 18 Feb 2025 03:20:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 259392800F9 for ; Tue, 18 Feb 2025 03:20:29 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DEED8A0D64 for ; Tue, 18 Feb 2025 08:20:28 +0000 (UTC) X-FDA: 83132368536.08.751C4CF Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) by imf30.hostedemail.com (Postfix) with ESMTP id A5F1880008 for ; Tue, 18 Feb 2025 08:20:26 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="DNrLW0/0"; spf=pass (imf30.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 192.198.163.16 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739866826; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LvxOhHcvAG8oEwurNxE+t8tG/ggtZ5vdhW46eAhAGP0=; b=O6dLl2PaAA+aqBVJdky7eGPlM0/+xyaTaC3ASe9fIhg3Xuv4RwgrpH8Kx7ispOWDARZ8NG YhgkAz3lPR2IRI4w+38W8d+ICYHBm0BRo7SM6PnJ05TJWzy0eQfsZG+tf0YoBqnyV52gy6 Bp0Rq3PWdGnS+7PDAChsT9dzA4eUPKI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="DNrLW0/0"; spf=pass (imf30.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 192.198.163.16 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739866826; a=rsa-sha256; cv=none; b=DuHv/f+emMqVf4BNvBEr5Lhu14H8FFODFBtk8n3gdCvxzikz1WIqzXAHpQ0fP5R1ge2X8k GfmhTQICpEaU4kZKRG8k93cKSSymHizbDPok0W4JweZ9/9nBqhlw/dd2iVsV0Z8w29uP/F TXeq58q61AkxP3pA7NvrXfhIoNyLPL0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1739866827; x=1771402827; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tLFNi6QiCcH4XS9SXJrys5meLjBaYY7ZaQ2eE4lAVKI=; b=DNrLW0/07rJSZ/FyoNHS3WszzlClUjIQFX0f8oFsCOsQH+37OB6orYs6 hkUu4xbGQ1yER/9/kx4sFi7XD6MMPuxlMkNGSlg3onKWlnGzYA399aSYn PEm+HgvYW0uGSt7b+chA16HBIKg+Dx+a11Y9gnp59jRneLA620TpsLZLS BXx+FvjypyNUcIuMCVtmYsWUJ0fNlGXOiiz6IK44gWCbF/e2m/AdU200o b4TK5S7MsKQR3o2tJm8a8A/WqRSIsypWw5jpmElypksi9X64SVwAvcrk/ EtCDNoJbHGMRggnp4SIl7bGHhBB0DHOxxaYstCChq+/wRZOs8kH/+zYYI A==; X-CSE-ConnectionGUID: QUbwTnKpRcarI87bzb26Qw== X-CSE-MsgGUID: t8EOB518RYOQb9xZfXxFWA== X-IronPort-AV: E=McAfee;i="6700,10204,11348"; a="28150558" X-IronPort-AV: E=Sophos;i="6.13,295,1732608000"; d="scan'208";a="28150558" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2025 00:20:25 -0800 X-CSE-ConnectionGUID: ZBHrRLDGSHmtBzW5XHTLlA== X-CSE-MsgGUID: DOPqudHOR6e9SmDmLLK9pQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="119248027" Received: from ijarvine-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.49]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2025 00:20:07 -0800 From: Maciej Wieczor-Retman To: kees@kernel.org, julian.stecklina@cyberus-technology.de, kevinloughlin@google.com, peterz@infradead.org, tglx@linutronix.de, justinstitt@google.com, catalin.marinas@arm.com, wangkefeng.wang@huawei.com, bhe@redhat.com, ryabinin.a.a@gmail.com, kirill.shutemov@linux.intel.com, will@kernel.org, ardb@kernel.org, jason.andryuk@amd.com, dave.hansen@linux.intel.com, pasha.tatashin@soleen.com, ndesaulniers@google.com, guoweikang.kernel@gmail.com, dwmw@amazon.co.uk, mark.rutland@arm.com, broonie@kernel.org, apopple@nvidia.com, bp@alien8.de, rppt@kernel.org, kaleshsingh@google.com, richard.weiyang@gmail.com, luto@kernel.org, glider@google.com, pankaj.gupta@amd.com, andreyknvl@gmail.com, pawan.kumar.gupta@linux.intel.com, kuan-ying.lee@canonical.com, tony.luck@intel.com, tj@kernel.org, jgross@suse.com, dvyukov@google.com, baohua@kernel.org, samuel.holland@sifive.com, dennis@kernel.org, akpm@linux-foundation.org, thomas.weissschuh@linutronix.de, surenb@google.com, kbingham@kernel.org, ankita@nvidia.com, nathan@kernel.org, maciej.wieczor-retman@intel.com, ziy@nvidia.com, xin@zytor.com, rafael.j.wysocki@intel.com, andriy.shevchenko@linux.intel.com, cl@linux.com, jhubbard@nvidia.com, hpa@zytor.com, scott@os.amperecomputing.com, david@redhat.com, jan.kiszka@siemens.com, vincenzo.frascino@arm.com, corbet@lwn.net, maz@kernel.org, mingo@redhat.com, arnd@arndb.de, ytcoode@gmail.com, xur@google.com, morbo@google.com, thiago.bauermann@linaro.org Cc: linux-doc@vger.kernel.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [PATCH v2 13/14] x86: runtime_const used for KASAN_SHADOW_END Date: Tue, 18 Feb 2025 09:15:29 +0100 Message-ID: <2a2f08bc8118b369610d34e4d190a879d44f76b8.1739866028.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: A5F1880008 X-Stat-Signature: makd6s37xg5ue5o5uh3sb658p45qu13o X-HE-Tag: 1739866826-470028 X-HE-Meta: U2FsdGVkX1+8XihDtfYkTU1Ql1b/obkSBp350n6yoT0SKmbqcE+7c+Z1VLxXhzLK2Z+QkljZ5zYLPKz5f1dZs1lLatizJYUQ2hPGf3eOJ5GmkvXd68ulUp3biuyZuHnlaE9GKKU+dbZmJsQof4MEYag6BJq8rRls+bMpNwueAfHdL3RUbAPeaaDE+s6CygUzkzJx64W+3HLuJ05qmntCQRe0eHjzhNmgjt07Xb7pfHcTtHPU7JKr92M2Z6QDmEn+TeIEi0mm3ThTt1R93C+BKEdYOOWXAE/mC1+bO/ux6nhEHJhl1vv92OjIPlBOtYUogPxXsbGnwcPXHqnKPbaE48ZXiqD20Jxt70+4lGrHH3ZmxWkR5r5EyEUEBXl18+Q7bjVFbzm7MG9Tz1hy/9v6um2NXSdtJQ+Tc5OPxwGrwSzi+tKTvwlUnfj73HAGP3XQ/Fau8HFIVjuzjs6MF7DwTNlANWjUJ2ZZzE/UoTBGjHHToSeJX+rt8BScvRoQlmWwmNj76ztw5LA7YwygDicl8FKN59SbiZRyA50OGa4+467n/EfSFf+vnqufTu7+Qll6Rlb5M51f9o0E4aHY932vuEzbJV3ZxOlQCU2oudCcWBkBnET36DX2b9h9s+j/pqNZwHSJ3badEqYX7J6pY9i77bQgrK2CLb8grFql4UlnUEhOhawlG5yBEqcjUiPFX2Tqfph7MV7W1cBBbIYNaxiyEEU7a1nCOr3QdEivLBgn5G3LR+E4gKGlUBKkElnsRBfePdBDHRlrsrf8WcI7o7H2orn1A+FMlmros4k+0Ojv5AuH9baU2MPJasr5AZx8dOXlxECqkna8He2wgK0/rDbAHSaoWH09HrgtvQS5HVz9XjoDypt9+MJ/LRU0oegmP/44k+wkMoREPbgEVrs5/psYOd5UCd7vv8Bi4dFGldq5SPLuDU3sgZMHFOqxdi8vicoqrYkAOfD82ERMSodM+2g xzH5J3MN aSVYzdSGPMYhZUcENPR13ZMglQ7w641hj4bJ9MwpOEXQCqtGoTfwxLRmIMy845HyQpZ6mUt4Ri9SybUb4OIWrL9Dt0EoG4TxV6XBV25iGTAdUOFeaDBIcUxJBZmea+HlloSmikL0GZO1IXjL/lPBmiDj74MMRNu7gLCVs8judzcNRAFAs9IW+jRM7ZgU6UmBjTstPll/K9vnt8UmbuhxARoznGdKIRLhfgvxw3+Y+BNFNiWyEkbsujS8QwD8c0nAxwE6B4L2g4IlBdKM2cYSLvwojJ1EaVpzA8UYPOjiiAbcyLuc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On x86, generic KASAN is setup in a way that needs a single KASAN_SHADOW_OFFSET value for both 4 and 5 level paging. It's required to facilitate boot time switching and it's a compiler ABI so it can't be changed during runtime. Software tag-based mode doesn't tie shadow start and end to any linear addresses as part of the compiler ABI so it can be changed during runtime. This notion, for KASAN purposes, allows to optimize out macros such us pgtable_l5_enabled() which would otherwise be used in every single KASAN related function. Use runtime_const infrastructure with pgtable_l5_enabled() to initialize the end address of KASAN's shadow address space. It's a good choice since in software tag based mode KASAN_SHADOW_OFFSET and KASAN_SHADOW_END refer to the same value and the offset in kasan_mem_to_shadow() is a signed negative value. Setup KASAN_SHADOW_END values so that they're aligned to 4TB in 4-level paging mode and to 2PB in 5-level paging mode. Also update x86 memory map documentation. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Change documentation kasan start address to non-dense values. Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 3 +-- arch/x86/include/asm/kasan.h | 14 +++++++++++++- arch/x86/kernel/vmlinux.lds.S | 1 + arch/x86/mm/kasan_init_64.c | 5 ++++- 5 files changed, 23 insertions(+), 6 deletions(-) diff --git a/Documentation/arch/x86/x86_64/mm.rst b/Documentation/arch/x86/x86_64/mm.rst index f2db178b353f..5014ec322e19 100644 --- a/Documentation/arch/x86/x86_64/mm.rst +++ b/Documentation/arch/x86/x86_64/mm.rst @@ -60,7 +60,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory (generic mode) + fffff40000000000 | -8 TB | fffffc0000000000 | 8 TB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 56-bit one from here on: @@ -130,7 +131,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory (generic mode) + ffe0000000000000 | -6 PB | fff0000000000000 | 4 PB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on: diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 6df7779ed6da..f4ef64bf824a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -400,8 +400,7 @@ config AUDIT_ARCH config KASAN_SHADOW_OFFSET hex - depends on KASAN - default 0xdffffc0000000000 + default 0xdffffc0000000000 if KASAN_GENERIC config HAVE_INTEL_TXT def_bool y diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index a75f0748a4b6..4bfd3641af84 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -5,7 +5,7 @@ #include #include #include -#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) + #define KASAN_SHADOW_SCALE_SHIFT 3 /* @@ -14,6 +14,8 @@ * for kernel really starts from compiler's shadow offset + * 'kernel address space start' >> KASAN_SHADOW_SCALE_SHIFT */ +#ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_START (KASAN_SHADOW_OFFSET + \ ((-1UL << __VIRTUAL_MASK_SHIFT) >> \ KASAN_SHADOW_SCALE_SHIFT)) @@ -24,12 +26,22 @@ #define KASAN_SHADOW_END (KASAN_SHADOW_START + \ (1ULL << (__VIRTUAL_MASK_SHIFT - \ KASAN_SHADOW_SCALE_SHIFT))) +#endif + #ifndef __ASSEMBLY__ +#include #include #include #include +#ifdef CONFIG_KASAN_SW_TAGS +extern unsigned long KASAN_SHADOW_END_RC; +#define KASAN_SHADOW_END runtime_const_ptr(KASAN_SHADOW_END_RC) +#define KASAN_SHADOW_OFFSET KASAN_SHADOW_END +#define KASAN_SHADOW_START (KASAN_SHADOW_END - ((UL(1)) << (__VIRTUAL_MASK_SHIFT - KASAN_SHADOW_SCALE_SHIFT))) +#endif + #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) #define arch_kasan_get_tag(addr) __tag_get(addr) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 0deb4887d6e9..df6c85f8f48f 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -353,6 +353,7 @@ SECTIONS RUNTIME_CONST_VARIABLES RUNTIME_CONST(ptr, USER_PTR_MAX) + RUNTIME_CONST(ptr, KASAN_SHADOW_END_RC) . = ALIGN(PAGE_SIZE); diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 299a2144dac4..5ca5862a5cd6 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -358,6 +358,9 @@ void __init kasan_init(void) int i; memcpy(early_top_pgt, init_top_pgt, sizeof(early_top_pgt)); + unsigned long KASAN_SHADOW_END_RC = pgtable_l5_enabled() ? 0xfff0000000000000 : 0xfffffc0000000000; + + runtime_const_init(ptr, KASAN_SHADOW_END_RC); /* * We use the same shadow offset for 4- and 5-level paging to @@ -372,7 +375,7 @@ void __init kasan_init(void) * bunch of things like kernel code, modules, EFI mapping, etc. * We need to take extra steps to not overwrite them. */ - if (pgtable_l5_enabled()) { + if (pgtable_l5_enabled() && !IS_ENABLED(CONFIG_KASAN_SW_TAGS)) { void *ptr; ptr = (void *)pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_END));