From patchwork Tue Feb 4 17:33:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8F17C02194 for ; Tue, 4 Feb 2025 17:34:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33C9E6B0083; Tue, 4 Feb 2025 12:34:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C5CC6B0085; Tue, 4 Feb 2025 12:34:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 117946B0088; Tue, 4 Feb 2025 12:34:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E7A0F6B0083 for ; Tue, 4 Feb 2025 12:34:53 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 41ECE140BE3 for ; Tue, 4 Feb 2025 17:34:53 +0000 (UTC) X-FDA: 83082962466.04.F081685 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf17.hostedemail.com (Postfix) with ESMTP id 4A04040005 for ; Tue, 4 Feb 2025 17:34:49 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=EJoqJGwT; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf17.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690491; a=rsa-sha256; cv=none; b=t+oJWu+HZhHS3yfXGM7B1BR3nJ2iNf+7wbHijFJHNLDvSkSr3tJDKYl0pe3SKEL5hBbt0e djUcJVn/Wm6ti+Lxna9AJvA5rgIWVIPvnZsvdGs8DdcZRly+t3dWp2OwktareqXrGSGxVb c4TlfTqdKIbFwtvOn0tUdMlPGWg8qoI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=EJoqJGwT; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf17.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ygTPRb3ywRhSzYmq1+08ZzVsKFhrG8jkildWvsOZQlE=; b=FzNL/g9n9YJJely/RRBDO7AZJwVnEINxFouOWrXeleHGv7RYZw07QzLglER1zmMENU3rO+ svM82S2CVaTk3+xgnH1QwY9K8Ab5E+xJGpk55g/dQCrWGyAOR8BI0i7LhxtBGgRbeobtVG xLKt60+iRBaHJ5pTwZMvLvb9w8zSRv4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690490; x=1770226490; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LrFP/S58o6ICJkrRaOG0zWteTvtIiSjG9jGn/c23LW4=; b=EJoqJGwT9FMoCC8dOPHB6nrD8yHtR2s7+ZIDRvt0yhJkbgvmRgqHUEse Hfl4X+FtAzcYgclC9NgoFjrNcU+gv/ZINoAkq4mTyd/3ZVvr3WStfXhZY 3uHhsOhUq1q5w+OigEPq5YhGrs+tvUZ1Sjeb98NWy0P8hJkYUFU+bgPGW ita6k6S5djivWRc3bg26KQJojs9vc73A794eaWmm3PGbirVrxNm4qpFtu q8CoDz6GIlVIg5LnHw73BEafy5bWbQmnLkAAu21WpyUTRNOBWSlQY3wIi BISbnTTPi9Z4Ec5REEz10cynSyKONcDUQUDX+VLfa5sotkZS0ry34M/up g==; X-CSE-ConnectionGUID: uDMrwu+YThySp570a5zRog== X-CSE-MsgGUID: ahGjqB/QQwuQa9GxmXy74A== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930340" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930340" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:34:49 -0800 X-CSE-ConnectionGUID: hsBgrPAEQzS1k8EGLwQyaA== X-CSE-MsgGUID: MO+fRiNSTeubDAn41XyOaA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866143" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:34:37 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 01/15] kasan: Allocation enhancement for dense tag-based mode Date: Tue, 4 Feb 2025 18:33:42 +0100 Message-ID: <808cc6516f47d5f5e811d2c237983767952f3743.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4A04040005 X-Stat-Signature: heas577ptbzcop9c5e7yog34ir8gp585 X-Rspam-User: X-HE-Tag: 1738690489-547114 X-HE-Meta: U2FsdGVkX1+SLKYwNf9tAlyzUHxWOu3qiea7shzLr6bUqEBKPXXif8sdsHlIFC4ee/GQVRBLy7rXfIpSMn/CXbQODdcKXk/ptT6dOP8WisHUd7ujEPeArNp0C6dsKrnAs5mJdNEpILwFySxfJ9cxQDL63YNT7LIjeQeLlyiuaVGaNOx/JlSK0JzXtOFeU83sQL4o2E5O1frm7JUgg1j0rQr+bNmfMvYbgliU2G+YoJuS5XgyR1RUBBM+qk1lB3BDfRYM55Dqy2rwl6+DTnbhrJQi3UYgH4LyteOtsgULWeeLqQFmuz5/1P6wRLXUw0yyD8hg8OPcXPyW4gWTzzz1nztSqxow/HsRVcQWrmMUNA2/t8r7yWl6DYmtSy2auMlc5D0OSTFBjgLspG4urgShx4K7su7E/p/4w3KW5JwhT/QHZnqBItlOsNPRP6ei8Tb0Tdaqadh/qeyq27TG4IsT4CAd2hW8+SlEgDMHkVyJLVVWoeJjw8DpYtISFpBeplr1DTQmx9au2bZA8be+fiHdBwXHB6rc0cLRs4lCpABbWumaH8YVeTGcA3Nwf1lGPEYHz1fp++k3WyQhXRo3UYxfdSJRKUp0aXMHkPryWzgWdiBtyuXEbokFyK4kJxlsbG1ocOQSUPBSKhSBibECIyrku1QrJ4CS6J0cHpAbM8arcTQ/32hDZQdhYksOJRq+qr4YByLqBDfhwN+dBkiy9VKfR/1N3ZIWX9wFVbcCI1qIagOYuyBZoFPzliQgp/Sv7yKnGd+kfnSD5n6dqSFLqTXftgz0YegRX7hZ/wCihbCMBm5ndpJKlWYd+fnCbDVVYkQzBDU9ipP6cVK5Fr+yHZyKl4SlWg2OUeQb6VGVe9RytN3g5955I+ZGpLQiAcWEGyxY2/fMryhQH6DzWYZzTL4o3RBvyA0mPkyyWLSu7WL36Hjqqlj9rt7IGGtnL+hnT5JxMNlX/G95ugk+km6CsGS OnfTibYz i0rEa6k3zM99ur+gXM26FZUstaYPh8NKJ6Z/h7tpLiWtNP2j2IqFjW4WdEoLlzRjV8S0uln9y9EIj00F6JPVnYMAuB+1QD4XI1hTx/oHgw2AvE547rJp88+xspg8q+to+va07P+SWwXmxZWg45HA7Of4s4CEJIIdvHyDTt3cxqd1MFCtAPvtmSm32JLdxtOMOJ4GMk01nvdnwJBwThKpYc+g5My9b0GGMGiXKGJLIRVJpGCUN1K45thf8ljgKUfXl4CP8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Tag-based KASAN (on arm64) works by generating a random 8-bit tag and putting it in both the top byte of the pointer (that points to the allocated memory) and into all bytes of shadow memory that correspond to the chunk of allocated regular memory. Each byte of shadow memory covers a 16 byte chunk of allocated memory - a value called KASAN granularity. This means that out-of-bounds memory accesses that happen inside the 16 bytes can't be caught. The dense mode offers reducing the tag width from 8 to 4 bits and storing two tags in one byte of shadow memory - one in the upper 4 bits of the byte and one in the lower 4. This way one byte of shadow memory can cover 32 bytes of allocated memory while still keeping the "16 bytes per one tag" granularity. The lower 4 bits of each shadow byte map bytes of memory with offsets 0-15 and the upper 4 bits map offsets 16-31. Example: The example below shows how the shadow memory looks like after allocating 48 bytes of memory in both normal tag-based mode and the dense mode. The contents of shadow memory are overlaid onto address offsets that they relate to in the allocated kernel memory. Each cell | | symbolizes one byte of shadow memory. = The regular tag based mode: - Randomly generated 8-bit tag equals 0xAB. - 0xFE is the tag that symbolizes unallocated memory. Shadow memory contents: | 0xAB | 0xAB | 0xAB | 0xFE | Shadow memory address offsets: 0 1 2 3 4 Allocated memory address offsets: 0 16 32 48 64 = The dense tag based mode: - Randomly generated 4-bit tag equals 0xC. - 0xE is the tag that symbolizes unallocated memory. Shadow memory contents: |0xC 0xC |0xC 0xE |0xE 0xE |0xE 0xE | Shadow memory address offsets: 0 1 2 3 4 Allocated memory address offsets: 0 32 64 96 128 Add a new config option and defines that can override the standard system of one tag per one shadow byte. Add alternative version of the kasan_poison() that deals with tags not being aligned to byte size in shadow memory. Signed-off-by: Maciej Wieczor-Retman --- include/linux/kasan.h | 18 ++++++++++++++++++ lib/Kconfig.kasan | 21 +++++++++++++++++++++ mm/kasan/kasan.h | 4 +--- mm/kasan/shadow.c | 33 ++++++++++++++++++++++++++++++--- 4 files changed, 70 insertions(+), 6 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 03b440658817..ea0f5acd875b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -35,6 +35,24 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t; /* Software KASAN implementations use shadow memory. */ +#ifdef CONFIG_KASAN_SW_TAGS_DENSE +#define KASAN_GRANULE_SHIFT (KASAN_SHADOW_SCALE_SHIFT - 1) +#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) +static inline u8 kasan_dense_tag(u8 tag) +{ + return (tag << KASAN_TAG_WIDTH | tag); +} +#else +#define KASAN_GRANULE_SHIFT KASAN_SHADOW_SCALE_SHIFT +#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_GRANULE_SHIFT) +static inline u8 kasan_dense_tag(u8 tag) +{ + return tag; +} +#endif + +#define KASAN_GRANULE_SIZE (1UL << KASAN_GRANULE_SHIFT) + #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ #define KASAN_SHADOW_INIT 0xFE diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index 98016e137b7f..d08b4e9bf477 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -19,6 +19,13 @@ config ARCH_DISABLE_KASAN_INLINE Disables both inline and stack instrumentation. Selected by architectures that do not support these instrumentation types. +config ARCH_HAS_KASAN_SW_TAGS_DENSE + bool + help + Enables option to compile tag-based KASAN with densely packed tags - + two 4-bit tags per one byte of shadow memory. Set on architectures + that have 4-bit tag macros. + config CC_HAS_KASAN_GENERIC def_bool $(cc-option, -fsanitize=kernel-address) @@ -223,4 +230,18 @@ config KASAN_EXTRA_INFO boot parameter, it will add 8 * stack_ring_size bytes of additional memory consumption. +config KASAN_SW_TAGS_DENSE + bool "Two 4-bit tags in one shadow memory byte" + depends on KASAN_SW_TAGS + depends on ARCH_HAS_KASAN_SW_TAGS_DENSE + help + Enables packing two tags into one shadow byte to half the memory usage + compared to normal tag-based mode. + + After setting this option, tag width macro is set to 4 and size macros + are adjusted based on used KASAN_SHADOW_SCALE_SHIFT. + + ARCH_HAS_KASAN_SW_TAGS_DENSE is needed for this option since the + special tag macros need to be properly set for 4-bit wide tags. + endif # KASAN diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 72da5ddcceaa..0e04c5e2c405 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -128,9 +128,7 @@ static inline bool kasan_requires_meta(void) #endif /* CONFIG_KASAN_GENERIC */ -#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) -#define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) -#else +#ifdef CONFIG_KASAN_HW_TAGS #include #define KASAN_GRANULE_SIZE MTE_GRANULE_SIZE #endif diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d6210ca48dda..368503f54b87 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -123,7 +123,8 @@ EXPORT_SYMBOL(__hwasan_memcpy); void kasan_poison(const void *addr, size_t size, u8 value, bool init) { - void *shadow_start, *shadow_end; + u8 *shadow_start, *shadow_end, *shadow_start_aligned, *shadow_end_aligned, tag; + u64 addr64, addr_start_aligned, addr_end_aligned; if (!kasan_arch_is_ready()) return; @@ -134,16 +135,42 @@ void kasan_poison(const void *addr, size_t size, u8 value, bool init) * addresses to this function. */ addr = kasan_reset_tag(addr); + addr64 = (u64)addr; - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) + if (WARN_ON(addr64 & KASAN_GRANULE_MASK)) return; if (WARN_ON(size & KASAN_GRANULE_MASK)) return; shadow_start = kasan_mem_to_shadow(addr); shadow_end = kasan_mem_to_shadow(addr + size); + addr_start_aligned = round_up(addr64, KASAN_SHADOW_SCALE_SIZE); + addr_end_aligned = round_down(addr64 + size, KASAN_SHADOW_SCALE_SIZE); + shadow_start_aligned = kasan_mem_to_shadow((void *)addr_start_aligned); + shadow_end_aligned = kasan_mem_to_shadow((void *)addr_end_aligned); + + /* If size is empty just return. */ + if (!size) + return; - __memset(shadow_start, value, shadow_end - shadow_start); + /* Memset the first unaligned tag in shadow memory. */ + if (addr64 % KASAN_SHADOW_SCALE_SIZE) { + tag = *shadow_start & KASAN_TAG_MASK; + tag |= value << KASAN_TAG_WIDTH; + *shadow_start = tag; + } + + /* Memset the middle aligned part in shadow memory. */ + tag = kasan_dense_tag(value); + __memset(shadow_start_aligned, tag, shadow_end_aligned - shadow_start_aligned); + + /* Memset the last unaligned tag in shadow memory. */ + if ((addr64 + size) % KASAN_SHADOW_SCALE_SIZE) { + tag = KASAN_TAG_MASK << KASAN_TAG_WIDTH; + tag &= *shadow_end; + tag |= value; + *shadow_end = tag; + } } EXPORT_SYMBOL_GPL(kasan_poison); From patchwork Tue Feb 4 17:33:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB0F1C02193 for ; Tue, 4 Feb 2025 17:35:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 426546B0088; Tue, 4 Feb 2025 12:35:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D6726B0089; Tue, 4 Feb 2025 12:35:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 277796B008A; Tue, 4 Feb 2025 12:35:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0904B6B0088 for ; Tue, 4 Feb 2025 12:35:13 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 729C3491C8 for ; Tue, 4 Feb 2025 17:35:12 +0000 (UTC) X-FDA: 83082963264.26.0CDF7F0 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf15.hostedemail.com (Postfix) with ESMTP id A344BA0009 for ; Tue, 4 Feb 2025 17:35:08 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LNfmSMbz; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690510; a=rsa-sha256; cv=none; b=e/gGYZwSTljGggi9sgkiHa0IUyeMpEOsVnbo+DJ/olDsxshNkWdtgfdjW2FAhJnwodUoVp H1qapFTBje2KZIwC9LjnaJITjF16rYXw52Bw5eicFbmWKrQKfz5F3UwoivPCYgApeQBLxs DB5bE17lz2f45pXS3z720DtDFEGUkhU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=LNfmSMbz; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pvxPfnrd9zxeTvTnimcFo2KrbNr0G4HQhkaUqyV3B7o=; b=R+dOb9emuLPRzsVaPKh8cnSXh0EUvNdoko+K+NSFcSC/wVvWXYmY4ChXqbKzpJD8qLHmoK qT52CLjHghUiWI838EICRTPf8XKCBXUVGaxtDHO97E1bpw/z2V1cOGY1pOcS/mLE08+CqU Tcmse2RPhvbz9zvl3LH1Ts1ORiY4IJE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690508; x=1770226508; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aHGhpjph+eDn968NL6XwKtEp1WbZxR8Rwtm1kIwOPTc=; b=LNfmSMbzlRtgfRc5CTza7dx8iVSXRj48s56MKvewoQdRnG0tWUrh2luM 3JK8HfUEk0ALgz+Cynl1XUdWuBdLd/K5Io8XQbUfpeGdwjYcctqAtD94N fEf88ZTJME/ys5Suq5RDTwxVQzP8JLJffwQNrJlFMcYCGrZOJttKVpPW0 MrAlFDbqpQNQX6f6sA1UNRqIx0rFgRWraGDi0Yj6hf3kWEItNJ+JhCemV 78TEVK+9FK5smqKIJu/8UYqJ8LLnt4uL2N1UENUkIsUlZ1Fh9FMLfBYFc ocG3I2ua8wd2bZgVqhSGUWfpRD2c9R4n9PZJztHOT+fv70h9VW/Y+cWhO Q==; X-CSE-ConnectionGUID: Te6UbGN5Qauswnr65Xr9Iw== X-CSE-MsgGUID: CKEODZbHTU+EW8o2YKlqZA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930403" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930403" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:04 -0800 X-CSE-ConnectionGUID: eBbq5MWaSZONJhS+dGPi8A== X-CSE-MsgGUID: mjasgO5WTcSBmxRL24BE0w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866217" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:34:49 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 02/15] kasan: Tag checking with dense tag-based mode Date: Tue, 4 Feb 2025 18:33:43 +0100 Message-ID: <8f790bb7e166c1ea2e5003318149eb1d7aba3596.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A344BA0009 X-Stat-Signature: 8oadpn6xgedh4qaekt7os5535u6s3pbn X-Rspam-User: X-HE-Tag: 1738690508-857608 X-HE-Meta: U2FsdGVkX19F/ODg049NzOlaBI5MIpwue4sMns7QcEZpM8jbxR3fEZFQGXTpbx4oCb5gBCKVMiYO1xoEYvZn3z9ZOGjMIfaIdBR1jbvTgPTH9oJgcx/kF7mT4kfiFw2KfC4R6ha0fBHRdRZz5siqlfV30tt/8Q4ypJMMQdDRKROorL3IVU82qn/eQy3wNMPU8P3//WoapROjahdnIg39Hqru9WBOH6ntztg0lhJEo985cQo+kO1vyX6RVf6agykRp8KKGP7osjmIRYRjQ/7cbThd8Z8NrGxrGdCHqi2QVi/WRUR+B6V6UWa16Umipv57f8Nxy697KfBa8mGd6FQz4ToR/0FNb+RKkg8j6IvAMCElx3uAY7f+e/Ae2XhUNE+hdB7DV/2Tv0CpR4ylvNdqUwgYjxeEpDN9GyHf8wYGAuKiZH5MJ3ddJH05DQpHexu34N8BxCYhN42Ay4HCkzHh7Ojl3uBShUWs0bXITDaaRf2AL3zLkaM+4ADGLfjeNpFyQ9GGJLZ/sp6UAhhZgFS1jWuH9RBirHydhQC2Fg2/TDk6Bk7TVZURjRzz3TvFNG06sUWJb7WlAglOXdprxgimZXzpAMkyBdJLQQxuUCMMTxDx9kp+JqQxhSCcmiIO5Sgtc2xLxCSnOyKA7Gy5ttUJ14lqR+tSBQ10ns68wvSvwZS89EfA+yhwIHlI++WYwOC0ieWO7ojMpMOa/hggWQqtRj7Io9IpvTcR6L7CsxApdVSYuunzm+qSmEvm+OasmO+7EiT401ZRIu8ko3Lwk8OI3x0jM4mBSqMv6l1O7717jsL4iprKtDdx3RFlMeqXJMklREwlEGqkzkVb8ahJp9NPkJb9mFza445C7gxw0+LMS9Kjz7ht8B+K5lNVQ1vB9DYy2YSZLAtsgxLOS39YC8xMyPnIyfMC8MH00v+uJihrNvr4Pgbp4743iqPEAs5hxx0blNx6hS3v1fxXaQhcKDj vn1OM9in 0X9WnN7MamW0Q5oJXEAzwpx/EV+oqhY8bXgmC92Vc+2K7PQ2KOiGwcCRe8LPcSyxOHkxvMQ+5h5Uh9MI0zZNW1sUpsXToUnB+2884F5TXxUI4DRGTB/ENiLhxBJkrUCDkOZVbqskDzE8Rwdm8lYWBI5aTS0RsORMiH3ihiis/H7MUpzWjONniHm4W6XPc3ZgpMgQZlKKYpHdCnDYrhbn9OHNhU63tFxgsX64hKGcnqoUE29RFBYfUhzbfmfTsOAEu9CfY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In KASAN's tag-based mode (arm64) when a memory access occurs, the tag stored in the top 8 bits of the pointer is compared with tags saved in the region of the shadow memory that maps to memory the pointer points to. If any of the tags in the shadow memory region do not match the one stored in the pointer an error report is generated. With the introduction of the dense mode, tags won't necessarily occupy whole bytes of shadow memory if the previously allocated memory wasn't aligned to 32 bytes - which is the coverage of one shadow byte. Add an alternative implementation of kasan_check_range() that performs special checks on first and last bytes of shadow memory ranges if the originally allocated memory wasn't aligned to 32 bytes. Signed-off-by: Maciej Wieczor-Retman --- include/linux/kasan.h | 47 +++++++++++++++------- mm/kasan/Makefile | 3 ++ mm/kasan/dense.c | 83 +++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 2 +- mm/kasan/report.c | 2 +- mm/kasan/report_sw_tags.c | 12 ++---- mm/kasan/sw_tags.c | 8 ++++ 7 files changed, 133 insertions(+), 24 deletions(-) create mode 100644 mm/kasan/dense.c diff --git a/include/linux/kasan.h b/include/linux/kasan.h index ea0f5acd875b..5a3e9bec21c2 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -33,6 +33,20 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t; #include +#ifndef kasan_mem_to_shadow +static inline void *kasan_mem_to_shadow(const void *addr) +{ + void *scaled; + + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + scaled = (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT); + else + scaled = (void *)((long)addr >> KASAN_SHADOW_SCALE_SHIFT); + + return KASAN_SHADOW_OFFSET + scaled; +} +#endif + /* Software KASAN implementations use shadow memory. */ #ifdef CONFIG_KASAN_SW_TAGS_DENSE @@ -53,6 +67,25 @@ static inline u8 kasan_dense_tag(u8 tag) #define KASAN_GRANULE_SIZE (1UL << KASAN_GRANULE_SHIFT) +#ifdef CONFIG_KASAN_SW_TAGS_DENSE +static inline u8 kasan_get_shadow_tag(const void *ptr) +{ + u8 shadow_byte = *(u8 *)kasan_mem_to_shadow(ptr); + unsigned long addr = (unsigned long)ptr; + int shift; + + shift = !!(addr & KASAN_GRANULE_SIZE) * KASAN_TAG_WIDTH; + shadow_byte >>= shift; + + return shadow_byte & KASAN_TAG_KERNEL; +} +#else +static inline u8 kasan_get_shadow_tag(const void *addr) +{ + return (*(u8 *)kasan_mem_to_shadow(addr)); +} +#endif + #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ #define KASAN_SHADOW_INIT 0xFE @@ -73,20 +106,6 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); -#ifndef kasan_mem_to_shadow -static inline void *kasan_mem_to_shadow(const void *addr) -{ - void *scaled; - - if (IS_ENABLED(CONFIG_KASAN_GENERIC)) - scaled = (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT); - else - scaled = (void *)((long)addr >> KASAN_SHADOW_SCALE_SHIFT); - - return KASAN_SHADOW_OFFSET + scaled; -} -#endif - int kasan_add_zero_shadow(void *start, unsigned long size); void kasan_remove_zero_shadow(void *start, unsigned long size); diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index b88543e5c0cc..3a460abd4c18 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -5,6 +5,7 @@ KCOV_INSTRUMENT := n # Disable ftrace to avoid recursion. CFLAGS_REMOVE_common.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_dense.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_generic.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE) @@ -24,6 +25,7 @@ CC_FLAGS_KASAN_RUNTIME += -fno-stack-protector CC_FLAGS_KASAN_RUNTIME += -DDISABLE_BRANCH_PROFILING CFLAGS_common.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_dense.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_generic.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME) @@ -49,6 +51,7 @@ RUSTFLAGS_kasan_test_rust.o := $(RUSTFLAGS_KASAN) CFLAGS_kasan_test_module.o := $(CFLAGS_KASAN_TEST) obj-y := common.o report.o +obj-$(CONFIG_KASAN_SW_TAGS_DENSE) += dense.o obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o obj-$(CONFIG_KASAN_HW_TAGS) += hw_tags.o report_hw_tags.o tags.o report_tags.o obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o tags.o report_tags.o diff --git a/mm/kasan/dense.c b/mm/kasan/dense.c new file mode 100644 index 000000000000..306bbbfdce29 --- /dev/null +++ b/mm/kasan/dense.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "kasan.h" + +static __always_inline bool kasan_check_range_inline(const void *addr, + size_t size, bool write, + unsigned long ret_ip) +{ + u8 *shadow_first, *shadow_last, *shadow, *shadow_first_aligned, *shadow_last_aligned; + u64 addr_start_aligned, addr_end_aligned; + u8 tag, kasan_granule_offset; + size_t aligned_size; + void *untagged_addr; + + if (unlikely(size == 0)) + return true; + + if (unlikely(addr + size < addr)) + return !kasan_report(addr, size, write, ret_ip); + + tag = get_tag((const void *)addr); + + /* + * Ignore accesses for pointers tagged with native kernel + * pointer tag to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KASAN_TAG_KERNEL). + */ + if (tag == KASAN_TAG_KERNEL) + return true; + + untagged_addr = kasan_reset_tag((void *)round_down((u64)addr, KASAN_GRANULE_SIZE)); + if (unlikely(!addr_has_metadata(untagged_addr))) + return !kasan_report(addr, size, write, ret_ip); + + kasan_granule_offset = ((u64)addr & KASAN_GRANULE_MASK); + aligned_size = round_up(size + kasan_granule_offset, KASAN_GRANULE_SIZE); + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + aligned_size); + addr_start_aligned = round_up((u64)untagged_addr, KASAN_SHADOW_SCALE_SIZE); + addr_end_aligned = round_down((u64)untagged_addr + aligned_size, KASAN_SHADOW_SCALE_SIZE); + shadow_first_aligned = kasan_mem_to_shadow((void *)addr_start_aligned); + shadow_last_aligned = kasan_mem_to_shadow((void *)addr_end_aligned); + + /* Check the first unaligned tag in shadow memory. */ + if ((u64)untagged_addr % KASAN_SHADOW_SCALE_SIZE) { + if (unlikely((*shadow_first >> KASAN_TAG_WIDTH) != tag)) + return !kasan_report(addr, size, write, ret_ip); + } + + /* Check the middle aligned part in shadow memory. */ + for (shadow = shadow_first_aligned; shadow < shadow_last_aligned; shadow++) { + if (unlikely(*shadow != ((tag << KASAN_TAG_WIDTH) | tag))) + return !kasan_report(addr, size, write, ret_ip); + } + + /* Check the last unaligned tag in shadow memory. */ + if (((u64)untagged_addr + aligned_size) % KASAN_SHADOW_SCALE_SIZE) { + if (unlikely((*shadow_last & KASAN_TAG_MASK) != tag)) + return !kasan_report(addr, size, write, ret_ip); + } + + return true; +} + +#if IS_ENABLED(CONFIG_KASAN_SW_TAGS_DENSE) +bool kasan_check_range(const void *addr, size_t size, bool write, + unsigned long ret_ip) +{ + return kasan_check_range_inline(addr, size, write, ret_ip); +} +#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 0e04c5e2c405..d29bd0e65020 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -183,7 +183,7 @@ static inline bool kasan_requires_meta(void) #define META_BYTES_PER_BLOCK 1 #define META_BLOCKS_PER_ROW 16 #define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK) -#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_GRANULE_SIZE) +#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_SHADOW_SCALE_SIZE) #define META_ROWS_AROUND_ADDR 2 #define KASAN_STACK_DEPTH 64 diff --git a/mm/kasan/report.c b/mm/kasan/report.c index c08097715686..ee9e406b0cdb 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -436,7 +436,7 @@ static int meta_pointer_offset(const void *row, const void *addr) * plus 1 byte for space. */ return 3 + (BITS_PER_LONG / 8) * 2 + - (addr - row) / KASAN_GRANULE_SIZE * 3 + 1; + (addr - row) / KASAN_SHADOW_SCALE_SIZE * 3 + 1; } static void print_memory_metadata(const void *addr) diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 689e94f9fe3c..1ac5c7a9011d 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -39,7 +39,7 @@ const void *kasan_find_first_bad_addr(const void *addr, size_t size) if (!addr_has_metadata(p)) return p; - while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) + while (p < end && tag == kasan_get_shadow_tag(p)) p += KASAN_GRANULE_SIZE; return p; @@ -48,7 +48,6 @@ const void *kasan_find_first_bad_addr(const void *addr, size_t size) size_t kasan_get_alloc_size(void *object, struct kmem_cache *cache) { size_t size = 0; - u8 *shadow; /* * Skip the addr_has_metadata check, as this function only operates on @@ -59,13 +58,11 @@ size_t kasan_get_alloc_size(void *object, struct kmem_cache *cache) * The loop below returns 0 for freed objects, for which KASAN cannot * calculate the allocation size based on the metadata. */ - shadow = (u8 *)kasan_mem_to_shadow(object); while (size < cache->object_size) { - if (*shadow != KASAN_TAG_INVALID) + if (kasan_get_shadow_tag(object + size) != KASAN_TAG_INVALID) size += KASAN_GRANULE_SIZE; else return size; - shadow++; } return cache->object_size; @@ -78,9 +75,8 @@ void kasan_metadata_fetch_row(char *buffer, void *row) void kasan_print_tags(u8 addr_tag, const void *addr) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); - - pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, + kasan_get_shadow_tag(addr)); } #ifdef CONFIG_KASAN_STACK diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 32435d33583a..7a6b8ea9bf78 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -79,6 +79,7 @@ u8 __hwasan_generate_tag(void) } EXPORT_SYMBOL(__hwasan_generate_tag); +#if !IS_ENABLED(CONFIG_KASAN_SW_TAGS_DENSE) bool kasan_check_range(const void *addr, size_t size, bool write, unsigned long ret_ip) { @@ -127,17 +128,24 @@ bool kasan_check_range(const void *addr, size_t size, bool write, return true; } +#endif bool kasan_byte_accessible(const void *addr) { u8 tag = get_tag(addr); void *untagged_addr = kasan_reset_tag(addr); u8 shadow_byte; + int shift; if (!addr_has_metadata(untagged_addr)) return false; shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(untagged_addr)); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS_DENSE)) { + shift = !!((u64)addr & BIT(KASAN_TAG_WIDTH)) * KASAN_TAG_WIDTH; + shadow_byte = (shadow_byte >> shift) & KASAN_TAG_KERNEL; + } + return tag == KASAN_TAG_KERNEL || tag == shadow_byte; } From patchwork Tue Feb 4 17:33:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4B98C02193 for ; Tue, 4 Feb 2025 17:35:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BF1C6B008A; Tue, 4 Feb 2025 12:35:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6955F6B008C; Tue, 4 Feb 2025 12:35:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 536DD6B0092; Tue, 4 Feb 2025 12:35:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 336306B008A for ; Tue, 4 Feb 2025 12:35:19 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F1D3214094F for ; Tue, 4 Feb 2025 17:35:18 +0000 (UTC) X-FDA: 83082963516.02.51573CC Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf15.hostedemail.com (Postfix) with ESMTP id 670C9A001B for ; Tue, 4 Feb 2025 17:35:15 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=lNICcpBG; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690517; a=rsa-sha256; cv=none; b=Xf9NYKYRoOm7pTdSbfejXFQ9eHlUxiUVOUHqJ8IwPxGbE/sBw100gfoH6MDtY8uzQngGUO Xk+KJoz3Urr6tCeUTrohQ7ancnl5nzwsGpZRX3NG140cdsAJecZvai+JJmd1FJHArMyfL1 XJfG0CzjJo6yG3xX0CZ+A8J5UY8elQY= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=lNICcpBG; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf15.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WFW8QfccSsjvCfxPySJ9vuy2PIzxbcJNxEP0RlFEUe8=; b=ETjiy8E2LdY8gvXlL9iCJLxNYmsMHm+tw8m7aZFf0Izo6nf1hobENNcPLbOYxUzLyVMvHb s7jS1a8D9/3MYUHeoiY6gZa9u+HdArmuvwO7407pC+immhQzsGt7WDJ3uxK9nIZVOqtI7O rL6ePRhqN6PkRrDpHcfyWuDCScqZecs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690516; x=1770226516; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=edAJD92hFjQk8fjbvzxioatc9um2IkZNk+/tzcDM8p4=; b=lNICcpBGl4Cgen8CLZjEAN6xxA51yANkn8sU7HxAPbyq3ON4dmDDzymu kRa+OqGj7yDGsFi8ZVLahpvUbZhGYx6EPIqGAxtfRsdfzC9DujaBWvHJ3 VcaBZtEuGYZ2vOaeESOWR9s7XwswCTB5/m83/vdXZOzSpQu5qHT4uu6gS X+jarWKn8v0CGDc7R90Ur6+ShsZkHCNUx/SYM3KEFbDeiYOVHpiGIM0E1 vbhkN9vP8cuonWH3Cm/uQsGPAcVdMSqWBnuTCwlCVBZsH/d6e0KbficYW BWlOu4Xt6iSpVpxX5Osdb1zW3NK9K2znKiDvv6N4rYEkPQJbrr4wcf9yF g==; X-CSE-ConnectionGUID: 9q3Y+Et6RoGcQZWWw6NYaQ== X-CSE-MsgGUID: uQiBUb9ZRpOvbrAYLrJA4w== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930461" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930461" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:14 -0800 X-CSE-ConnectionGUID: GXavZLsESKizsbio/E/Srw== X-CSE-MsgGUID: e0b52Ax9SOSm68zuxVJ9gQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866342" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:02 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 03/15] kasan: Vmalloc dense tag-based mode support Date: Tue, 4 Feb 2025 18:33:44 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 670C9A001B X-Stat-Signature: ex4yd5kkasctosubw1pamo846pjz8nen X-Rspam-User: X-HE-Tag: 1738690515-863660 X-HE-Meta: U2FsdGVkX192ukhQ5IhoJohGv0ulbW7DsjnBNt6KZZ7TBaOxgtyNWG6/Z6VGWrV2XoPz/c7zimvCxQVs2KKxcZvFMBaOZXvjTcixmLNfg/tKYkb/wnIBI+O94qaHsTQm0qerJQbYFow+7ay1b3RrYUrc7KuGGK1FeoxYKDlCl3/3OHX1F02yvV8Prom9npZyiRM7Q9pi2PcwKXa5qOc5OrXv5P7SHtC2fx/Bk3ykPurM5V74doiaCavVaP9i7jURTALCllBwJa28B5Ds3b1CS7kIdaliSfUznKVmPMdWe7GR5zKY7e/dk2JcMG9oZZehy7b5co7ZalSHaagvZ2mggqp14yjIrlFzxk/Zr2qL1k6liNuWVtTKkZKqjoLo2Geuo6Vgh3ODhnlZqZD8/DRT3bh8Xe++uIRmB4sxQXFKsj0s43H4HE1ewGWFDdB8F8HbfFYrfKDIfKbOwGT/aFVNfxW0q5GoyDHAjkTLFIEvl4evCWtGsYMc74L98VVHVa1tILlelnNQiUoUcSWB8PT+7CGlau1b086ps79wm3jr174cvGlLCOHAyKQ1UvjOOFhQtDleJxs6g6hmolI6CmssoIa4jGgj0sQnd0s80pGISV7IblwtNoYFh5Kg0oUFOVR34alserIO5+O2HEW/Uq8/7lYP9K53tHdHfY/ZV/zN6fAHcbLDUYcbBEwgCeakyeRw7eaNYCSNdkL0haUv2/qKmvqKJAf+/sDUMH42sX21sexHOBHoCdIY0rVcYRYjht1+dAnzMpKYvI1twXwZfPjdDl7HdRVo4IBpuMsRC2tPB91MRUDvny4Feg3R584/TIvuuSyzp4j2L5gdQ1rfj557fhlGyyBvPDrmETTzgoD92lLxCZUCgd82PVwW0a43kaw9hQOihjhVKQFgfL9Sbin2QskFR29Xi8KgMolhppmWqzpMCgJ06Z0T8whQ5psIvz5TM+zaC7UBxYSeaYy3ZWV TmirhWlS 9lVelsWhPxPmcBMU8/k+SHCTIKgf7OU0CrPt7cWa+xEZMnrxoxiwhdhILxUNtz5HNJ0e+ccovohUs43He93Y/7CM811pbR6d/kyIjCM22I1qjt09aBlW8nsSGlCtSO5vHbKe8p8UvUKRefyh3wNCAomti1elmx2J0gyvgeIr/MLyBE/5DrFemzbwVlW9z0oPM6qzqNFLLJFn2U/AJdKgJpkayNkO+hK1cjetOIS4RScGwvR3BvB2fLzvSr63UJ9A2KGwf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To use KASAN with the vmalloc allocator multiple functions are implemented that deal with full pages of memory. Many of these functions are hardcoded to deal with byte aligned shadow memory regions by using __memset(). With the introduction of the dense mode, tags won't necessarily occupy whole bytes of shadow memory if the previously allocated memory wasn't aligned to 32 bytes - which is the coverage of one shadow byte. Change __memset() calls to kasan_poison(). With dense tag-based mode enabled that will take care of any unaligned tags in shadow memory. Signed-off-by: Maciej Wieczor-Retman --- mm/kasan/kasan.h | 2 +- mm/kasan/shadow.c | 14 ++++++-------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d29bd0e65020..a56aadd51485 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -135,7 +135,7 @@ static inline bool kasan_requires_meta(void) #define KASAN_GRANULE_MASK (KASAN_GRANULE_SIZE - 1) -#define KASAN_MEMORY_PER_SHADOW_PAGE (KASAN_GRANULE_SIZE << PAGE_SHIFT) +#define KASAN_MEMORY_PER_SHADOW_PAGE (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT) #ifdef CONFIG_KASAN_GENERIC #define KASAN_PAGE_FREE 0xFF /* freed page */ diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 368503f54b87..94f51046e6ae 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -332,7 +332,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, if (!page) return -ENOMEM; - __memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE); + kasan_poison((void *)page, PAGE_SIZE, KASAN_VMALLOC_INVALID, false); pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL); spin_lock(&init_mm.page_table_lock); @@ -357,9 +357,6 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size) if (!is_vmalloc_or_module_addr((void *)addr)) return 0; - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr); - shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size); - /* * User Mode Linux maps enough shadow memory for all of virtual memory * at boot, so doesn't need to allocate more on vmalloc, just clear it. @@ -368,12 +365,12 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size) * reason. */ if (IS_ENABLED(CONFIG_UML)) { - __memset((void *)shadow_start, KASAN_VMALLOC_INVALID, shadow_end - shadow_start); + kasan_poison((void *)addr, size, KASAN_VMALLOC_INVALID, false); return 0; } - shadow_start = PAGE_ALIGN_DOWN(shadow_start); - shadow_end = PAGE_ALIGN(shadow_end); + shadow_start = PAGE_ALIGN_DOWN((unsigned long)kasan_mem_to_shadow((void *)addr)); + shadow_end = PAGE_ALIGN((unsigned long)kasan_mem_to_shadow((void *)addr + size)); ret = apply_to_page_range(&init_mm, shadow_start, shadow_end - shadow_start, @@ -546,7 +543,8 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, if (shadow_end > shadow_start) { size = shadow_end - shadow_start; if (IS_ENABLED(CONFIG_UML)) { - __memset(shadow_start, KASAN_SHADOW_INIT, shadow_end - shadow_start); + kasan_poison((void *)region_start, region_start - region_end, + KASAN_VMALLOC_INVALID, false); return; } apply_to_existing_page_range(&init_mm, From patchwork Tue Feb 4 17:33:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 253C6C02193 for ; Tue, 4 Feb 2025 17:35:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B2B426B0092; Tue, 4 Feb 2025 12:35:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B0D726B0095; Tue, 4 Feb 2025 12:35:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A4006B0096; Tue, 4 Feb 2025 12:35:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7CFEF6B0092 for ; Tue, 4 Feb 2025 12:35:31 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 40F6848CB6 for ; Tue, 4 Feb 2025 17:35:31 +0000 (UTC) X-FDA: 83082964062.19.685213A Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf20.hostedemail.com (Postfix) with ESMTP id 279281C0003 for ; Tue, 4 Feb 2025 17:35:27 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JlXjeOEb; spf=pass (imf20.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690529; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NvaRO4UHPd/bOrfuOO4NoEzfTdAiR2iNdaeHPrh09jo=; b=vRpspjB/oR6Z9+w2r7T3hVQF0Wf8UGtdmgnemB7mBB+yrGvR+J9okIXRRmoaNG9ZM/WK0s /sJrqlfGGD5GnqXo+uurL/4AZ7riGPMaX7aihtTPBCvEL3VxcXk7SPglMyZbw81gTzoo+N 5poEscMUhN5rrjtdHYnVWabwLKx5i2g= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=JlXjeOEb; spf=pass (imf20.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690529; a=rsa-sha256; cv=none; b=4TQxbXs4Lyv46lmPn4AoMCAj4ICpI0oIC4yhyvt3mSG1eVPZA2TdKxLLlxM28Wksue4ass CMmpX6OqOWH6P3CUtnWPfmHBIj6hqtmCsvqFZoZycq88Pi0BhaTiphKdtZ8voxaLtHgg07 sxiB1kGVg2ws9aPol484L2Mx5Le3KRs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690528; x=1770226528; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7SPlp9QEiWP+nB12GdV91yF1dqiTepZdICL7iZuq4h4=; b=JlXjeOEbVvPMC3hhHt5yK9rjz5KPSuv2F9HBzIZvCkiL8dghqrlkSoLU emlfyS3R8R8ER41sQd+Y1uduO8pIyB2CgNSpkFWEHeOUtg9spfx1dVdZF pABfWJ97iIdw5um+JyjM3sbqg60hYeYU+fA+JCQ+xCily1xYkxReS8v7S 7Rw5GQRgSC2yj9foJjxBqcH4hcFng4cpx2Jq1y8qQxajNpq4xo1hraYJw Togy6PmSLPEqC01TWQTRL4MawcXP+4GabFXJFR2K0IX8xN9Xt2Z4WPtwC 7VM3qwWoxjhI5hiV0lhXofeMc3EkdVUqxKgzWKmccJR/DmZ/19FmNSH19 w==; X-CSE-ConnectionGUID: GQH758VjQ+6eRaJA74dRtg== X-CSE-MsgGUID: XavGOm2cQqC6AQLmTwJEcA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930518" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930518" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:25 -0800 X-CSE-ConnectionGUID: /sEOqI4kSuqYrX4FgQuozA== X-CSE-MsgGUID: zBO7dwN1RyyrRSQp4bH21Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866447" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:14 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 04/15] kasan: arm64: x86: risc-v: Make special tags arch specific Date: Tue, 4 Feb 2025 18:33:45 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 279281C0003 X-Stat-Signature: negifzhqoozzmm6zxmw1an9up3tipmn8 X-Rspam-User: X-HE-Tag: 1738690527-738467 X-HE-Meta: U2FsdGVkX1/Y4NJPizrDGzbwkZTIYk0/TyGFNUP/mvnKVPFkvd/XxLkTpFciO2Shp5HH6QxYk/FhOyeAgAH1D2xewR6F4qNBWC4S0eCIS3uAYbq4QrYLPgAL74dZN/+WqmfWuS0hn15V+caktfTV3UvPuXpS2hBEVVZ2v1aRU1d5GVeDsfEAOhGp52CMmB5Zt5ylJXFjbbIXS10thUDQiTpVICa7Lb0kA5CCpXgxDDi3+yLzxU7XZENhuvJGosqORmrzyWQfdd7M4aBo/5xYIfhHQRGZF4E+RUdOs0tmRU3OmhzWhSyr3SDaTvvDhaSDhZNdhSBfF/gRfRYVSbz7kbd7RV2DC2hpOGcOAJJXWmprs51bW0Xl51QiwOcpXEJa744feNUb+vDhs599Pkc7WNgcOVJe2RRgwLO5sZ4swokKp6IzRQQhH3EvJk7YAEJRboB6W+l4qxMe4+r6sYa0VY3hNa6Z9LZSjYRk7mmS75ubmI0MG3SZny5lWFfhc8VHDO2K3xoNFqxSkYinzd9LmgdQfrTPddA31vnjzi9LblEBK3Qw5pDSnxcGZYiyrIlPmTikOrSqA1qGrTGN9UDdzn7rS7lxvGU6Fxg5cI8fry4eHoe9s0tnRYQpdhKMrIhuJMX7XWpL8l4ldBuw97zmv/A1F60GdknOa68MZzwwpGza0oj7FxncCUmbTzQxJJohb4IthCZx83qAv+lSYZTcPHiKOa3+H00jNjjxzHxIrsIqqdPbX2pnTUkabcPa+wLkIRTzC5RnwfeMPZ2x7uN30h00yL1278qkye8kL0RlrJUguB+E3kJg/XlwnUfFb/c3FbibHHpXVRgANSubxXIPImS1oQufGbdVf+WKiN36bvQOg6ckO0eFCTyMXS/qp8s7GSgL3cjRQLrdboRp1l3yQWuCjWA3QKeRL3PNVOjxQeQz59badQE8JPKVZtIF8zQWinA3E6cS3Ct8toE+uwG 91yWyIRc 1IG4+0uiTd6cgJaE4hcl4uxHAN3NlpKiHf4p7wNyJM3bQkM+82JzQeLYiOiX6GVVkwm08MVYqNOOr5lneYDKBqAjxFGK4oUgz5VYKbzZ1o9xwMmEc/JAT7I4gV+NASqr8enS6rtYt29pPvn2j0j0b7l+/ZjEG5lAeJ7G5dy7x0CGqGuNh0iahIt/zKpgL9MD3Rm7WvvPwCidwhgOdxOiGni/38kXmK1r/9KeW7XpJzPDJL3TDtp4+NT4RuS0EcI4VKBgEF6ftbWg7dbpemc52B7zPM0rPXq9QrTU9IlGJ2ZAKCguOBXIdJ1+DVkYTrRHzZZw2xtSFe6pYKkw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: KASAN's tag-based mode defines multiple special tag values. They're reserved for: - Native kernel value. On arm64 it's 0xFF and it causes an early return in the tag checking function. - Invalid value. 0xFE marks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory. - Max value. 0xFD is the highest value that can be randomly generated for a new tag. Metadata macro is also defined: - Tag width equal to 8. Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly. Make tags arch specific for x86, risc-v and arm64. On x86 the values just lose the top 4 bits. Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code. Signed-off-by: Maciej Wieczor-Retman --- MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 9 +++++++++ arch/riscv/include/asm/kasan-tags.h | 12 ++++++++++++ arch/riscv/include/asm/kasan.h | 4 ---- arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 12 +++++++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/page-flags-layout.h | 7 +------ 9 files changed, 49 insertions(+), 16 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/riscv/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.h diff --git a/MAINTAINERS b/MAINTAINERS index b878ddc99f94..45671faa3b6f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12227,7 +12227,7 @@ L: kasan-dev@googlegroups.com S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasan diff --git a/arch/arm64/include/asm/kasan-tags.h b/arch/arm64/include/asm/kasan-tags.h new file mode 100644 index 000000000000..9e835da95f6b --- /dev/null +++ b/arch/arm64/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/arch/riscv/include/asm/kasan-tags.h b/arch/riscv/include/asm/kasan-tags.h new file mode 100644 index 000000000000..83d7dcc8af74 --- /dev/null +++ b/arch/riscv/include/asm/kasan-tags.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_KERNEL 0x7f /* native kernel pointers tag */ +#endif + +#define KASAN_TAG_WIDTH 8 + +#endif /* ASM_KASAN_TAGS_H */ + diff --git a/arch/riscv/include/asm/kasan.h b/arch/riscv/include/asm/kasan.h index f6b378ba936d..27938e0d5233 100644 --- a/arch/riscv/include/asm/kasan.h +++ b/arch/riscv/include/asm/kasan.h @@ -41,10 +41,6 @@ #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) -#ifdef CONFIG_KASAN_SW_TAGS -#define KASAN_TAG_KERNEL 0x7f /* native kernel pointers tag */ -#endif - #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) #define arch_kasan_get_tag(addr) __tag_get(addr) diff --git a/arch/x86/include/asm/kasan-tags.h b/arch/x86/include/asm/kasan-tags.h new file mode 100644 index 000000000000..68ba385bc75c --- /dev/null +++ b/arch/x86/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/include/linux/kasan-tags.h b/include/linux/kasan-tags.h index e07c896f95d3..b4aacfa8709b 100644 --- a/include/linux/kasan-tags.h +++ b/include/linux/kasan-tags.h @@ -2,7 +2,17 @@ #ifndef _LINUX_KASAN_TAGS_H #define _LINUX_KASAN_TAGS_H -#include +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +#include +#endif + +#ifdef CONFIG_KASAN_SW_TAGS_DENSE +#define KASAN_TAG_WIDTH 4 +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif #ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 5a3e9bec21c2..83146367170a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -88,7 +88,9 @@ static inline u8 kasan_get_shadow_tag(const void *addr) #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT +#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0 #endif diff --git a/include/linux/mm.h b/include/linux/mm.h index 61fff5d34ed5..ddca2f63a5f6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1813,7 +1813,7 @@ static inline u8 page_kasan_tag(const struct page *page) if (kasan_enabled()) { tag = (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; - tag ^= 0xff; + tag ^= KASAN_TAG_KERNEL; } return tag; @@ -1826,7 +1826,7 @@ static inline void page_kasan_tag_set(struct page *page, u8 tag) if (!kasan_enabled()) return; - tag ^= 0xff; + tag ^= KASAN_TAG_KERNEL; old_flags = READ_ONCE(page->flags); do { flags = old_flags; @@ -1845,7 +1845,7 @@ static inline void page_kasan_tag_reset(struct page *page) static inline u8 page_kasan_tag(const struct page *page) { - return 0xff; + return KASAN_TAG_KERNEL; } static inline void page_kasan_tag_set(struct page *page, u8 tag) { } diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7d79818dc065..ac3576f409ad 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -3,6 +3,7 @@ #define PAGE_FLAGS_LAYOUT_H #include +#include #include /* @@ -72,12 +73,6 @@ #define NODE_NOT_IN_PAGE_FLAGS 1 #endif -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -#define KASAN_TAG_WIDTH 8 -#else -#define KASAN_TAG_WIDTH 0 -#endif - #ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8 #define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1) From patchwork Tue Feb 4 17:33:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2759FC02194 for ; Tue, 4 Feb 2025 17:35:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7A086B0096; Tue, 4 Feb 2025 12:35:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B2A776B0098; Tue, 4 Feb 2025 12:35:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CA536B0099; Tue, 4 Feb 2025 12:35:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7D2266B0096 for ; Tue, 4 Feb 2025 12:35:43 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0C7DA80BDE for ; Tue, 4 Feb 2025 17:35:43 +0000 (UTC) X-FDA: 83082964566.16.5956B7B Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf18.hostedemail.com (Postfix) with ESMTP id 24E501C0011 for ; Tue, 4 Feb 2025 17:35:39 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=KsgZ3uc6; spf=pass (imf18.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690540; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hd/rmwwNUxEWJaueMUiOYmy433JWOqfiDFpi9akupUI=; b=Yfqt8qBHuI0y0y/l6hble+vL7VNYdTUJWFrmDxoZbF2uYAtZWAfOlZaOLMkA/SuJSdx+aZ QZmTbFuesnMOaz593tmF3xNjvT/c8xmHmV3OyVVpm7o+ESb795frP0Wign5bK39IR4KWDD C/lVCcVp2elN7gm0k3pkUSHb7XiUaFs= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=KsgZ3uc6; spf=pass (imf18.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690540; a=rsa-sha256; cv=none; b=rUGCT0Z7/vqzNBqwI5aAiCg03Gbaa/FIZUP7Ghm+rqqBuznjmsVfd33kvIhM8Bw22cTFaO MDfs3XnykdV0tu8PS+KZ+rHn5E/aS9cMIaAarhTlXz/sb7ZvsMHLTEdfKifdLf3+9YA8G6 f/Ri1AuFzYqQnwpYdiPplk0I2H4V7yE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690540; x=1770226540; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1nix+N5DUro8RwKo5+pglZtqohhVbR1ohGEV2E0y1dM=; b=KsgZ3uc64dLCDQwUfoj4msNCM1jw+SUtj/QrWhpnaODvC+nvmyznnj95 /ptkPZIHk8RCycaJ7RUSDW8s6w0jBpt5ycAkREdTC1dSxNJxqQitVXnaL g3KHCiiMVHCaKtAS/O8qO3ahb2CsSqjLvkzl8QZVURVXylk24rZvrccot mGMiNemu9iqhluhAEeecHzVbs1Udp4sneS+h8+58Ef/qR+BBsn/+8gLdD 2CjV52urcUCjNJodrJncmbwMFFdAc6B1VCJlx0P/pVnGsQ+Ed418agXdU 0Pkf5hLiaiM/fgXB0/fCG4mFvAjXc7tl7KkbXukct2XRD+nO9jBrs/Mbe Q==; X-CSE-ConnectionGUID: 2Ffk+VUsQ2CGQc65GXLpyQ== X-CSE-MsgGUID: dTrUJBK0T6ecvrTDsZjUIA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930598" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930598" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:38 -0800 X-CSE-ConnectionGUID: BjNSJDU0RVuIyvdxJ63vGQ== X-CSE-MsgGUID: INVQujqZTM+d6XKry25vsA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866530" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:26 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 05/15] x86: Add arch specific kasan functions Date: Tue, 4 Feb 2025 18:33:46 +0100 Message-ID: <911ad4b9f001bca4c274b60144b1db80eab2015f.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 24E501C0011 X-Stat-Signature: ezo7jn9y7578ttgnbyjgxsijh1mpny87 X-Rspam-User: X-HE-Tag: 1738690539-572665 X-HE-Meta: U2FsdGVkX1+v7jmSXDv/jpR7QXIFs4cMImQHGE/ZXP5f7TIb+qCRxZp0ZKsHYIst8E1DfJDhQPONoenvsLetP5cCIYYOhr/JB4o5Lu7mRw+cL9r092hxCGODoW48KZoPjPMbEZ1NKAsGYJXY5c4a3A6/T+T7Vk/rq4bT9tsIb2RuQ/G0gvYqKblD3sJJz7iaD5yonYaUufwCQaikpQhx8KPPyxyIocLUz3yBhfmatgug7deBs3AbrEMOuP4NdBz6iSNeomUV+SEunywtAL2/Alw0nlgrLyWMY+ylkOjymxIB1l0RR1uMJGPeQNa3xVOqKFY8iw1hyFH3vIRYGMIuZG7rsxuiASqjqA5jdLLw5Y9sd0FrQogMMY7OFe08GEROklSdF9RAGChB5+8IC69u/535Qo0yVNt4LljyygijxoBewnweQtGSrlo+ALsHRhUB4HA/o8jTGma1hRErDF/LAe7aqMXPAf2ODhvfoqFvLlnd3jUaQ/NST2UTxrN126v7cWCKm/aVSdlZQMzYjj1oZT6v3Tlxaa6SfX+rzdXhfzIJrQ/4YHJ1FJdmOfeS+hW4U+xYOCwao6L6eWGaehBK0oqs3/0KLnSqSfSpeUiv43VFRtPQRugGnxe8uaOielhQPu9LMzi8V4XN0UB816Gs0GicJBXKZRDZlb5DDHU2gu662Ph0GZx3FCKvWjPjyKK+h6mx4o8ylxQdFfrscRVnt2Kueb/BiRmzvoqnhD/2Zls61IuLR68Y0d1dOTmz7i1Lrc7Ue/i6ujVCFN79qpB1ss4um44Ag/y/h/njvXR1sGXY5o+FZ6dcIifAMNGsERUhY/NPwP/xfbITBUHlJqGk0P90d2vL0Xmdx+5ZPzcxdQ/o5WtM9sUHzPwdn+vydrp4rUQyXE09o9HVQThVbJg4xw+isYlVbckGrVpxoLYhwhTSyQNFldCQsWb/5XL59JdVUKEBrro5h/tEeRwMSkz vy0IKdMW S4HBT0TkKUxumQ/k/v8qaLblqgEIgxkoyHgaYJgxyOtDYrj9GyGLLnBSmg99pd3nZ/ZlfVOE2ZhJ+NgP1WuPe3SUCGkv/cI6gzkZ9b1RunRKcF/PliFCblDzHpv838bzTKXPWIarcfYdAq5iwYxt6i0n+twmbdhpXplj1HdQxp/TGmx+MBiOiWQiqtm/zlOhu5kZOuUPQRiCpFTQ7vPjuxDvJwhsSEKH6uLyvUGPcTmjszvw7m/GHaSwQnFfP+mhYlkUVF+5j15SBWQaBT6wumxVZOBzfRqNXvLgH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: KASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - mainly to set and retrieve tags from the top bits of a pointer. Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/kasan.h | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index de75306b932e..8829337a75fa 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -3,6 +3,8 @@ #define _ASM_X86_KASAN_H #include +#include +#include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_SCALE_SHIFT 3 @@ -24,8 +26,33 @@ KASAN_SHADOW_SCALE_SHIFT))) #ifndef __ASSEMBLY__ +#include +#include +#include + +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) + +#ifdef CONFIG_KASAN_SW_TAGS + +#define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted(tag) 0UL +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ #ifdef CONFIG_KASAN + +static inline const void *__tag_set(const void *addr, u8 tag) +{ + u64 __addr = (u64)addr & ~__tag_shifted(KASAN_TAG_KERNEL); + return (const void *)(__addr | __tag_shifted(tag)); +} + void __init kasan_early_init(void); void __init kasan_init(void); void __init kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid); @@ -34,8 +61,9 @@ static inline void kasan_early_init(void) { } static inline void kasan_init(void) { } static inline void kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid) { } -#endif -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLY__ */ #endif From patchwork Tue Feb 4 17:33:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E8BCC02197 for ; Tue, 4 Feb 2025 17:35:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8B506B0099; Tue, 4 Feb 2025 12:35:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C3B8E6B009A; Tue, 4 Feb 2025 12:35:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2A2F6B009B; Tue, 4 Feb 2025 12:35:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 949F56B0099 for ; Tue, 4 Feb 2025 12:35:57 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4CB12120C35 for ; Tue, 4 Feb 2025 17:35:57 +0000 (UTC) X-FDA: 83082965154.15.08A54E1 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf06.hostedemail.com (Postfix) with ESMTP id E9ABE180011 for ; Tue, 4 Feb 2025 17:35:52 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Zl0vXEOV; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf06.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690555; a=rsa-sha256; cv=none; b=WQwBbp8xbvpRAKUHBNLjIkVe1OvRSLBKw08fIrDFZLNiPmw5XdpCnCPOHK7A7Hjb6icoVF r0C9b4kE/GXAqCJLKA25R0UdcQSE+wLDlcc8tfJJjGHgiG6fAKMLDgwNhBAUjYWKw/Pppi dXPjSJY9WbpjlekzFE3eEAGhR7xqn0k= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Zl0vXEOV; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf06.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690555; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4BMCwGV6PWVYHkQSqSGJPvpDzUETn/4+hKDV9iJhNHI=; b=Ay3vLw0oFzmqwHVBKTSBEL0gWY3yW1ON+Pt2gy6gB/f86FnVo7prkESXVpFg6q0Y9XvR1c nBSuDORZyWk6NigJBa2jXvgjVQq4Wg+3ZCN3EeuIeZUhGv1E5evRuHKXF5nSAx4H42v+/C hob5yZSesJk2OXB8xlQpaCeb9QSq9Fg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690554; x=1770226554; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GgxfVN861pKW86Az0//GU2BTNPe6s6rSZNyJJoV8Pn4=; b=Zl0vXEOVbshg7hr4RQ1e9NES6OLhflTlJ0R09i/x1d1hKhUhgto/XAsB wexmaS4nmkWFLK6q1em8863mgALU+w2vsrRuSTKRXjdu/v1MERPwmPr0D sLibIkigUVJbUUGbLDq5i12D3dn9/FMrBWZWPRc8UqSUq8/b+u0wwhePv qir6dlBU9ns1tbnoS6XOV/0UFtb3bOblFrRHjbm/O2q78Ph9tQ2vPRYug evLIsfLPjjAoUqcj+y/XYuKFTK1dYOvU2cEqennG3iuEfFWk7eHvhm4rA KJOLlkV/58ZlTyaWti62j1zJ3yAWpyi7q9nIPeMn2QE4bQWUsMWb4EZ5b w==; X-CSE-ConnectionGUID: NL+lQQpvQ5CAUsBj19KN1w== X-CSE-MsgGUID: lUIr5eAhR5G/KUkyl8cMRg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930651" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930651" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:50 -0800 X-CSE-ConnectionGUID: EjCbNRnNQ/eG0RktDBjJIQ== X-CSE-MsgGUID: zLXQup0VQWiJeXH9lbUyzw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866602" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:38 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 06/15] x86: Reset tag for virtual to physical address conversions Date: Tue, 4 Feb 2025 18:33:47 +0100 Message-ID: <80aa9a4c633502b5330c40f8b2d4da705dca92e7.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: E9ABE180011 X-Rspamd-Server: rspam10 X-Stat-Signature: epkkm4wqimup1167ea6mb4ymds96i6zm X-HE-Tag: 1738690552-397514 X-HE-Meta: U2FsdGVkX18nxMg37xBDHzpy8c5dzJ2WN9IeSoSeE9Gvidlq9J5AHOxV5sAej63CnaGnyrSPVEk4C7aeIClRTq/EN16o1+wbcUyoNu1kp6I724dA3kBMo+G4QXAPvzrAoqhlBvLHlS/Mtbyo//K7bedevw4DhhT5YGL1/WGMvJ9CrGWvm6vGBQmJWVu1x5PJ1YhfE2DUgzsVxrjUAF7ogLSEK97C4Ow4UlgRwSzEiHsBKJTakfQ+atdLtcX46RmIkWjJyx5+0r6hTPwf46pP8N8xk4jFBmI3qMqHgUp/HmSBDNKvOnKd5v8aEIDiTSKW4QxB2yhgNbP3AdX+k0M+qU9KB8g3eetggXniB/SwfIXC4GCQuHAO1/dr9ut10c9I5+a/Syg2dNEuoLievgygHAeZkj+qgVvD2FaKlz7rv1n7Xb6r5IEcKzlKq1XKy61n9Z+/T3wtdn0OkyBwOM+hBOR9bUUxtLTaUfS4eepuokg4opV4KiZnsQGa0NGSeMGdswnEf9yNpHl86Qv/aPxb0JwIilMb3gxMW6f7vdVkPflMQOebPwmhkP51P+g6yYldnCpefXbenslWviS3omNb93SJnUTSSbFHnZUR3HIH5ZTOcNVX/u31VV8n4E8J53fZdwPXRGjI5F4CXss+GAoN2M1WnpUBf3MvIxIr15VZiYy90MWjc0+xaMTVMIugZIAiUNG6NWBfgS/JwU7hf9WBKd1Oj1S728+nIUkxptBDpXHQLZGwX+Eo+ONUJIs5hH35LeEmCbo6IwAZDn/qXH6SddvfcciztyOVHcjD76dU7EznsrdPhZSssmdi0/rAVoT0NTOpv6VNhlrKPlGfjZfcMZiiZ17ItJcqhyAW2G/rIkOePj/bkeCCBeZ1n5bcBxmhiBR8LqdDB1RgYsMlq7yQJy+YYjvu2Tvo4P21NK8rPhZatGY1O9wuzgdHegG/EJ+BXzYesHeSak7JYqQsnCa kpGD/WtA 5vFgNxAI4xxy+7kbgM4SC45lW2l8kZbQyyEXC8iLr/nE22MZkVMbm8fOD+yFrHQr+ZtOCgK5FjMlO2lxFvhGz8qUe0vvWoQJDpxsneSFJkUC18uNaFH82huq1PDhjM/vKrWVa7pw2Lq3FghZwiuYKiQuvGduh5TJzaq6Rcoid9B19mYC6X6CUkIGyFHynzKWA2zhQVLA1dDEYMs32E7UDFf0PXbUIAOpiKHB1j/pLTW3dRvE7BJ4ErFpMhg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Any place where pointer arithmetic is used to convert a virtual address into a physical one can raise errors if the virtual address is tagged. Reset the pointer's tag by sign extending the tag bits in macros that do pointer arithmetic in address conversions. There will be no change in compiled code with KASAN disabled since the compiler will optimize the __tag_reset() out. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/page.h | 17 +++++++++++++---- arch/x86/include/asm/page_64.h | 2 +- arch/x86/mm/physaddr.c | 1 + 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 1b93ff80b43b..09c3914d8ce4 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -7,6 +7,7 @@ #ifdef __KERNEL__ #include +#include #ifdef CONFIG_X86_64 #include @@ -41,7 +42,7 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr, #define __pa(x) __phys_addr((unsigned long)(x)) #endif -#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(x)) +#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(__tag_reset(x))) /* __pa_symbol should be used for C visible symbols. This seems to be the official gcc blessed way to do such arithmetic. */ /* @@ -65,9 +66,17 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr, * virt_to_page(kaddr) returns a valid pointer if and only if * virt_addr_valid(kaddr) returns true. */ -#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) + +#ifdef CONFIG_KASAN_SW_TAGS +#define page_to_virt(x) ({ \ + __typeof__(x) __page = x; \ + void *__addr = __va(page_to_pfn((__typeof__(x))__tag_reset(__page)) << PAGE_SHIFT); \ + (void *)__tag_set((const void *)__addr, page_kasan_tag(__page)); \ +}) +#endif +#define virt_to_page(kaddr) pfn_to_page(__pa((void *)__tag_reset(kaddr)) >> PAGE_SHIFT) extern bool __virt_addr_valid(unsigned long kaddr); -#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long) (kaddr)) +#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long)(__tag_reset(kaddr))) static __always_inline void *pfn_to_kaddr(unsigned long pfn) { @@ -81,7 +90,7 @@ static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) { - return __canonical_address(vaddr, vaddr_bits) == vaddr; + return __canonical_address(vaddr, vaddr_bits) == __tag_reset(vaddr); } #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index f3d257c45225..6e24aeff36eb 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -33,7 +33,7 @@ static __always_inline unsigned long __phys_addr_nodebug(unsigned long x) extern unsigned long __phys_addr(unsigned long); extern unsigned long __phys_addr_symbol(unsigned long); #else -#define __phys_addr(x) __phys_addr_nodebug(x) +#define __phys_addr(x) __phys_addr_nodebug(__tag_reset(x)) #define __phys_addr_symbol(x) \ ((unsigned long)(x) - __START_KERNEL_map + phys_base) #endif diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c index fc3f3d3e2ef2..7f2b11308245 100644 --- a/arch/x86/mm/physaddr.c +++ b/arch/x86/mm/physaddr.c @@ -14,6 +14,7 @@ #ifdef CONFIG_DEBUG_VIRTUAL unsigned long __phys_addr(unsigned long x) { + x = __tag_reset(x); unsigned long y = x - __START_KERNEL_map; /* use the carry flag to determine if x was < __START_KERNEL_map */ From patchwork Tue Feb 4 17:33:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37D3BC02193 for ; Tue, 4 Feb 2025 17:36:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C65C8280002; Tue, 4 Feb 2025 12:36:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C1444280001; Tue, 4 Feb 2025 12:36:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB551280002; Tue, 4 Feb 2025 12:36:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8B649280001 for ; Tue, 4 Feb 2025 12:36:06 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 45596A0C26 for ; Tue, 4 Feb 2025 17:36:06 +0000 (UTC) X-FDA: 83082965532.27.39BCAE9 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf02.hostedemail.com (Postfix) with ESMTP id 2095F8001D for ; Tue, 4 Feb 2025 17:36:03 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=msPdVFfQ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf02.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690564; a=rsa-sha256; cv=none; b=gSd8+3PVsdUctm5m5orOBiF1kXI3eL+99SPfDAkZZTwp3uCafic18ffZY/+PnIUcj4L/Am QTjA8wXYCBm1C3NPnnCLi5q3iWvqpF33Tf7Iuyoth2SPLPer89rf9/29sHROKXunThqufz k+JntVwbNzZj/V+aRHTgFZwLgb7gSSo= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=msPdVFfQ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf02.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o0QxH8z6cj5lUaoAwjX8vI7JCJobGz53CoJ6O2LJ4cM=; b=pRQsawjBAjeniUIFrfExuUl55SKzObwl3UViVKlNnSlxVcTOkDKWXFvrKfewcxiOF/L2P9 xPditnM28TB5MZXS++6PstZlu0UFEpGiclOODFM080l8SozhdY11pNdF6DP2EobDVRnhdQ WHRxqIWgzuSdi+rc/y7+wz3CtbT22wg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690564; x=1770226564; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PewXt3mq2ZzovNzL7I8Fp3xpOP5/pryHgDbcA5gr/7A=; b=msPdVFfQtW92VN7JAebPejE9J72+qlIHdn5s1vFQYMacfbZ0KKYGxKRR lakeXB13BZw/g5V1wcY/S6vawTwL2Wtwv0UdnUvR/Igz3ZhJGEDD/LLZq afcC3ZlSAqyvGHlf0vOaqXnTKc1AIHnIg1sTFrwoFM0B2kSXKOtKwiAMn Mu7yeCmr8OEcZoDLDyOBZ3DC0UrAi0qtQJ2jY/qlFCk4ry2rNC+qqnHSi lWLn7jwMSwuc2FQmTIMGH1ydPwo6YW7XXS1ug9LhXLYADQjXEU2u63XsE renwk1D8t1tMX1dAaqt2RFgjj712ggwPAhwkpubC3j8r2KnUm68rirBUt Q==; X-CSE-ConnectionGUID: efCCrDe4Q7CdNs69jy6Qnw== X-CSE-MsgGUID: q8lCZeEiQLa4+Z4COwU3fg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930687" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930687" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:02 -0800 X-CSE-ConnectionGUID: PpX6VWM3RaGBO1JoNfSfGA== X-CSE-MsgGUID: vYDFTil1TqeMDIrSmNQ1tw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866647" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:50 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 07/15] mm: Pcpu chunk address tag reset Date: Tue, 4 Feb 2025 18:33:48 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 2095F8001D X-Stat-Signature: 5wznwhoskkgdb548koieiek6zn1tbfrc X-HE-Tag: 1738690563-384779 X-HE-Meta: U2FsdGVkX19emdXPLqrdGPzgjJpEdxNsgOHvlLB4yd7yN4lHT55a306x0T6uvLib7X9ra7BlYFbblU13HraIcAFqreCwBMTWExqH7eRWd+9V6008tdnt5VrZG6IYBjiFBrh2ZVUOfKXVmg7BBMhir/8tP+1TP8OVyIKTMP4vNERtH6fyNk1GITkEUeCLq7UnYGr4FNf4/7Jz4AMpEPV7rAa036qw+ne2ffd+sJvXpEeVsk9ACmet4awPLWhU7D6CkZLRiotXD+3huJGJVTBjWI9+1CnSMWPPBlrC2zhhNkxtnHPvh8aJLZx8NMnwFu/ZBzYM3xHxUvg4evouhYvlMXMnYj+VuiwGpqaxWbjWwikevirhgZf6VVuvuOz66NcaTY46or5/55ll/AZ+1Y/Ta1g6s4NfyIHsS9LPU6PEDKh1xNn/hQGvyoDPk1ZRyI+W2h2MK9gxP0EQuD8mXPYr3sJUGMNR7xKgKTwvfeKyQQqlt44KylJJdiJaYgL1tlJlQzdQEL6nTOlpp+vl7Bjr4ybZpVygZcIDX0WiriGYzk3om23SrXOo4mbLK1MwO2EUv6tlBhS2eB8wRWZ3YYOYnid3f74IhIE7YjlVcF7XXj5LF4P4BrYZAOnwyjQLITxqwGU7dG4cs6rRZFKA/ceQijAY0mQ3uhUJTu0csp0ppl38/78vxHYtYOtgRT1wCvgSmzXxCUPWQNj7YtsCFyQOKJAvY/OdqAHrd6Odycq7jHus3fT774/+7Kum9uHPe2p5C9kpD9ocaRb/M1k8dzCfQMGTtfFcg/ECHbPMsG39B4JlBamDKpy2UT27QGJPsSeaHHPsSNWlwH+eVU/9vFM5rG7oWzMa+Nbr1LkqghmAtL+hr2S0FDuikR52nW2Ox4RFqAyUqcE6sAHGw0EW9l1sUg2P2CN32d7/1cGlmecQ+T8F6um0GiCZdI1ndLL7+YU1e7W2xbhsjwiQVQmusv5 aPvkHaIf uizx3APofSR+Jjvi9ljEvp9NeXOpeu80xLunnFfYq3B8RIAQU8Fuiq2igfYPFh6yo8tC7zo2GT4Ruht6PRcdkz6r8UmyJvpx0/RhFxfvucTbt3+hGF2Ar2s1egkCAELLqWTX9v4TlK0gDZhvUqI8VuzOeXPzVAT//jEWpagCyNt/ORtwgMY0UWMHRToCfaH/s/CMLlEzVadfWi/EG2pkNzjLPWoc5CXkwnVxARkwyZ5qara2HSUS8brW3JQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The problem presented here is related to NUMA systems and tag-based KASAN mode. Getting to it can be explained in the following points: 1. A new chunk is created with pcpu_create_chunk() and vm_structs are allocated. On systems with one NUMA node only one is allocated, but with more NUMA nodes at least a second one will be allocated too. 2. chunk->base_addr is assigned the modified value of vms[0]->addr and thus inherits the tag of this allocated structure. 3. In pcpu_alloc() for each possible cpu pcpu_chunk_addr() is executed which calculates per cpu pointers that correspond to the vms structure addresses. The calculations are based on adding an offset from a table to chunk->base_addr. Here the problem presents itself since for addresses based on vms[1] and up, the tag will be different than the ones based on vms[0] (base_addr). The tag mismatch happens and an error is reported. Reset the base_addr tag, since it will disable tag checks for pointers derived arithmetically from base_addr that would inherit its tag. Signed-off-by: Maciej Wieczor-Retman --- mm/percpu-vm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c index cd69caf6aa8d..e13750d804f7 100644 --- a/mm/percpu-vm.c +++ b/mm/percpu-vm.c @@ -347,7 +347,7 @@ static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp) } chunk->data = vms; - chunk->base_addr = vms[0]->addr - pcpu_group_offsets[0]; + chunk->base_addr = kasan_reset_tag(vms[0]->addr) - pcpu_group_offsets[0]; pcpu_stats_chunk_alloc(); trace_percpu_create_chunk(chunk->base_addr); From patchwork Tue Feb 4 17:33:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43972C02194 for ; Tue, 4 Feb 2025 17:36:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D00EC280003; Tue, 4 Feb 2025 12:36:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB1C3280001; Tue, 4 Feb 2025 12:36:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B51F1280003; Tue, 4 Feb 2025 12:36:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7A7B0280001 for ; Tue, 4 Feb 2025 12:36:20 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id ED1E5120B5C for ; Tue, 4 Feb 2025 17:36:19 +0000 (UTC) X-FDA: 83082966078.28.1C5AFA4 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf04.hostedemail.com (Postfix) with ESMTP id 51AB240006 for ; Tue, 4 Feb 2025 17:36:17 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GxkhQful; spf=pass (imf04.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690578; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lQ5v1NnA6R05ztWWI7B+8ia+Rd5WZ3/fmwVsOhBY4uI=; b=Smx0HUW4zCCqFAar26dfcAKJohXuNs8KFqaD9PXvj6mfFsHbxkTWfFpT/uKH+66D+6dqrq 6V2chbUpKCKktJ/DYZL1abCPWRoBWRrY+VNtB1tZPR9NlvbThQLl02jwwxfb2T+KJTiH/Z Y5ERnSdASvnYRJRArFIR0TscxtVAWt8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GxkhQful; spf=pass (imf04.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690578; a=rsa-sha256; cv=none; b=3sOj5hYcg3v+D3iOCTWdpTm+PRnikJR8q3kp9DNcCEubgnvyRAfdgAO8nUYyaJc8bH2oHE CSjIrhgUB8iisNDiXfDS9RlPpd+NVli7UnucVlQ08iIZZRFERBZlIeQB7fcmyLLaVe3S0S u8wC+SpKGI5Va2enaCJhj1Oncc4k22E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690577; x=1770226577; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/9iE/zrVNGKs+zQWoSO6YvnUY6c4YJpTItZxJ7K+vOY=; b=GxkhQfulD/0iU4cGPGTGSdLKpPmOKHJW7U1SjUmMrvJGYu+R1v935cCL 0/uqriI/+72lOdcOpQZMzCmd9iI7XR/yMrDsKM50JUHBwvBB8xSaf7kGc EoT4QrvwzyNszJ4evZcNR0P2HOoZWvRQKIZCIxnAaRREjb9wjK7iKWV1B SpNScrmmueFpAv2cvBTX8R+QAZZc9SDsq6Vf5OCUwRn8czuAk5k/av4S/ gttzQ/UnN6037RLDJeFFYsKsJJvkzi+3ttv0GiZu/GsFzTP7IX3m6UbVh FjQSG6zVEOhRqX1vgaXY69fY7GqwmiPDCo3MvJEeXTJm37j95FFvraDGR g==; X-CSE-ConnectionGUID: dWVxfCbISSa4/s1JebvhoQ== X-CSE-MsgGUID: ndcAsNdhQmGTey+SpuRpYg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930755" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930755" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:14 -0800 X-CSE-ConnectionGUID: +qvwGRPBT+u0hTrJ0ps4gQ== X-CSE-MsgGUID: gKQBPywxQQq2MfdmEZxhgA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866742" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:02 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 08/15] x86: Physical address comparisons in fill_p*d/pte Date: Tue, 4 Feb 2025 18:33:49 +0100 Message-ID: <2c2a71ec844db597f30754dd79faf87c9de0b21f.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 51AB240006 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: rrgx84hq7tm3odeftk7jam8oo9xkdxnw X-HE-Tag: 1738690577-869045 X-HE-Meta: U2FsdGVkX18/2DgtcK2n5RIh0MhC6DmjnuF/GsodIVBTmxt+Rp8KmprJ2DXPuDwW8SKBXkZh9G7wPiLWTVmHemMek11sU77Q5B5ChIwfrqLPI71DBLUnAIfIyncDnJOvTEJCM0lFQX3p8KhZObYXlp7zzsx/snD5YmIaA+0coBVIT/1khF1YieTCuI09sqKjzpzX8GU5DWC7wl3UBkJzIhWXskwwpd1geesTp4SjtpADJnL7l6x4j4SzaRajXt1JCKrVZlBPyVOiapu0Rg/J6w6ZR93BOYBMPMWdiS9GChIPJQiWXxGyTV67UDA2EEoZay5sX/HYED+LtwnaPL8D7/oaT8DwWvU8d2iq3p7J5TBOYfjV0ANQZ3aQwz5FOeEY+eSCKlf7l3mnq9oyfS5hUNKOC8Hf1Q58znCart4elb9cJEjShgnYZkVRu5uWaRkXvCBp6ca/mrtfsSRC0TFEIOCh86svGcxrHjoaIfLBlkTlOg0aDInAi9n3Suk13nMXI6PNQ8vQRmdAj9oYogUiP+uz9zvM8WhUfgYmyncBwKk3kV8ee+IgbF//LLs7xRphTAPFjxvOlcbyJzz/wFTb8lbxnRRyV55f2ZcZAhLTXO6sk8TtnAchq1281buXS5aLshzrJKpHhZHG+3QO7NnhPL9kOfsydwEJhdPTtBCNaa5BbApYI5i3MS9Mn09ig083vCO3CWQz3r7cVxyfo77bDGCHYN0nd86AimN4QzjklVdEV6RmeRI2nhibRz7dLa8/vvHZwZRDBZ5JgzzuTqRrl0L5Bnz8eb5cgqzDa4FEaSc6WwJauLwVEbfY4rqEKHq0Wtm+NMeauaeZRl+SeQFZg65DTqrE3LXDzSSyh0sh0GGg/gVJ392h6a+K/mtVwJyHJpuRehsBSo+D1OVMcUR4VZ/x5jh8RZIrU9Q+1/ZvdDc66V+yVmKqK8pd3bsqGDFYfn4BEDeUH9ByimgJu7A PueULIu+ WWvOCtfWi3I/Cxky7GWOL+r7jbUQ2q8G90rEASNdpciRimpKtKc1zAFcpS+C2xVTrsqEFmxO38z51hAoleoCybOOyZVXInrE2sdPW4bv/t3V1C3TJBxJZ36CcpdFLRo767lVoa3MQyuoWpCdLHdMD9OvPVEThkMkv9e6lYJIwZY70nyVYY3KwzbNm6JiM6TU76dRscH3x9P52nmT2vU2xCASQHyaYA4nI+NVHMF6DCB5Y9s3PbDPRFbY4Wx07HYYOLOWkcp7KbPM89nkQQaQ/cdTe57x/FjsP6S4rI8UrQh4FR7Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Calculating page offset returns a pointer without a tag. When comparing the calculated offset to a tagged page pointer an error is raised because they are not equal. Change pointer comparisons to physical address comparisons as to avoid issues in KASAN that pointer arithmetic would create. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/mm/init_64.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index ff253648706f..bb101412424a 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -251,7 +251,7 @@ static p4d_t *fill_p4d(pgd_t *pgd, unsigned long vaddr) if (pgd_none(*pgd)) { p4d_t *p4d = (p4d_t *)spp_getpage(); pgd_populate(&init_mm, pgd, p4d); - if (p4d != p4d_offset(pgd, 0)) + if (__pa(p4d) != __pa(p4d_offset(pgd, 0))) printk(KERN_ERR "PAGETABLE BUG #00! %p <-> %p\n", p4d, p4d_offset(pgd, 0)); } @@ -263,7 +263,7 @@ static pud_t *fill_pud(p4d_t *p4d, unsigned long vaddr) if (p4d_none(*p4d)) { pud_t *pud = (pud_t *)spp_getpage(); p4d_populate(&init_mm, p4d, pud); - if (pud != pud_offset(p4d, 0)) + if (__pa(pud) != __pa(pud_offset(p4d, 0))) printk(KERN_ERR "PAGETABLE BUG #01! %p <-> %p\n", pud, pud_offset(p4d, 0)); } @@ -275,7 +275,7 @@ static pmd_t *fill_pmd(pud_t *pud, unsigned long vaddr) if (pud_none(*pud)) { pmd_t *pmd = (pmd_t *) spp_getpage(); pud_populate(&init_mm, pud, pmd); - if (pmd != pmd_offset(pud, 0)) + if (__pa(pmd) != __pa(pmd_offset(pud, 0))) printk(KERN_ERR "PAGETABLE BUG #02! %p <-> %p\n", pmd, pmd_offset(pud, 0)); } @@ -287,7 +287,7 @@ static pte_t *fill_pte(pmd_t *pmd, unsigned long vaddr) if (pmd_none(*pmd)) { pte_t *pte = (pte_t *) spp_getpage(); pmd_populate_kernel(&init_mm, pmd, pte); - if (pte != pte_offset_kernel(pmd, 0)) + if (__pa(pte) != __pa(pte_offset_kernel(pmd, 0))) printk(KERN_ERR "PAGETABLE BUG #03!\n"); } return pte_offset_kernel(pmd, vaddr); From patchwork Tue Feb 4 17:33:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C0C5C02194 for ; Tue, 4 Feb 2025 17:36:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 140EB280004; Tue, 4 Feb 2025 12:36:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F1A3280001; Tue, 4 Feb 2025 12:36:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB229280004; Tue, 4 Feb 2025 12:36:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CB40D280001 for ; Tue, 4 Feb 2025 12:36:42 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C8CEBA0C7E for ; Tue, 4 Feb 2025 17:36:30 +0000 (UTC) X-FDA: 83082966540.10.2D9C30C Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf10.hostedemail.com (Postfix) with ESMTP id 6BD59C0019 for ; Tue, 4 Feb 2025 17:36:28 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=QJJdt2Pa; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690588; a=rsa-sha256; cv=none; b=rxE3jSPmGSSCqr2NNUQ7GGcWednAf5egqPb9U9Oc4CJiM0In7QWi97J7kd3ra4tg9SBUou b7yNI1R1gyeLDWeSomEgITo/gTnGdpb9NOYZ9gXwUUbaToSOgF5+a+0mKMtY0zYFGFmj+U mSzCiPZRB3/4ELo05DLEKfIOtH0jR8A= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=QJJdt2Pa; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OjOreTL+Ot1aK4xRTqvixmr6X+5Hdo5QfDrtzsBAFBk=; b=n+HFg7sY4kJ+WL4xOG5uiaZjtp2Nqt7c4AYG6yCYLnHRpDmJ6sy0NjHAVr5vthv+Vt5SC9 eG1Yc+e3CuDXLrt92JeRWPAWuBVzsFfF6+7bqIHuvL8niq8JflQbZSMN6l6tojmRhujDXO 7FVOkmXhS41epYXgr+oUYPafPCi1x7c= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690588; x=1770226588; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZPDKrg9roFM67mEQ2fI48/uwWPYPBpbE1ZWp7rfkzQw=; b=QJJdt2Pa5SkVFcYXoKE9T8aJKEjwDs3ohZukkt8mZCqeMpk2OQtVBxwF HYyiQy1lHeez6Bs5WKdQA/qKl+mwaloknEDAj4LeXjeLQk75SJu7rh1rg oKB1KIMIZEEFQ+up/yCX7kTqkFpFcHZ66Rv0dt2FHfVOLqHKwxETCK9Ds wv/RsnQA4O3PwckiTyPaUeXQ9WPnPK0rV35hnhpCT9J5W1lebE/qQxsm/ 8ExrFYMmggdnNgMdRHb2WgbU8XH4ugsYcpIJiFyh4cXhOf90h9tDbIj/0 X2ic/PYhxrcD01oinVTK8xpp6RGkezs8h3zkb7ktzZTlySuQmbzOkBzOf w==; X-CSE-ConnectionGUID: O59URw+GSReF5jOJU8kj+g== X-CSE-MsgGUID: ZRUd77SSTfa10iykTY0YYg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930858" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930858" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:26 -0800 X-CSE-ConnectionGUID: jXIRAGRVSi6AV3rBRiAebw== X-CSE-MsgGUID: yAFfDOxmTeCrIA01iFfVXA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866806" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:14 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 09/15] x86: Physical address comparison in current_mm pgd check Date: Tue, 4 Feb 2025 18:33:50 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6BD59C0019 X-Stat-Signature: 1wy46edaeis1dtgfsywpfw5en8meewzu X-HE-Tag: 1738690588-436473 X-HE-Meta: U2FsdGVkX1+9gVpB8LHTW0oUHumb+GFx3vUTJhLdKPC4PFSmbjmU7z1gxv/g9Ic49B5crsXOZWbBG0VGfwUP8htIdeGFgrPm2AFeOKHx5y1JlXqdffF97Mj+zp9Nvsy0FL7ATD3jWQWgHf7AmhQiX6eonOEndZKdF1LiUOHqteIBicqRmiFnBZeacCv4Jm1qznIR9t02ZcltovA05t+K01btiw7qlriEVhdnflDlasA9D0Cz8O8Rd07Lp2X3StkwuxMEmCa43jCU25GBNuPu0PvRba9Ei4jblgVAfSBtL7vNFW3oM74lklO1HbhWxgr6WDhkxD6mFBuqKcVX8GEGcFLeBFIvDsYLrCpSRPg0M7L6MZb6Mtrt/+EB6zP1TCHngANoFkGDeQvG5uNnsoZhdmzptxoPgtOz1D+x/i7qLeUKtTw75JP7tVctTZzGTY8y0LQXF9nNB/iSVnwHfd+ahRlGNss8Q7sOlg/i2MHe/9q6aXboXwUI75YeBC/URQIJeSLhvu2ePs0ePVBG1yTY5OQoURzUkRffrrC+xi06pK1XaQ/m0jQiwfus4Zc5M7Rub3NmRG7/xUZ24GlxdDZ3pu87gedZMjHFk1Lb2LrlDTvv9gSuA4TYbA90IN5uQ4HeRYD3HDT4/Ty7yRi2aLHGNIJ0CdpzniWeWpoiWjztNEm7QPauIwscXdVuAx0PEVbXGPan58Ppez2TUVH4bu1/5MnYe6NoWj9sAts9a2yTWK+7vOe2mviwjZz/R4qzpo/u6DFIpj2wtg5zDxd0BHIypKeqE+d1mPt9fkKGnvRwwvHVEs8EBVIwaiI7FPFlsWIy3xXPmda62atNUI4myT8vuZnJlC6ZssK9Np7xRBNit+GyzSqHRBKOaD+43VWrTbN/TWGaba+tWZMdjwiW1AAiEIIF9qqr0s1qL+He6fICSucFF2eOJ5GJJjPSlYGl/CEU0lXpxwr2ZIUbTUddnTI gQ3cVTw5 zl7z9Rmg1xTReTy4cGyyRdZx2AczlbR18LTz2PQHVK5cPiylI8HYZo2bX1pRN8K5DA4X4XLy7QoigBDJdoLw30LlxYAf3kxp1m0ompTqYZtRJISR0jBPotmP/TP1CzTTB2rbzfoQi5HMUxErjhLPhGDG1jaqV2O4XiIBwqchNKatv3TDINMKo/eWeO5k0SahBqdEFZudaCh19gxtopGtQg3hXvMIFGjMOkUxYx2+zUXqMwPK3Zp2z/vF/O9YEN3jJOSwELMJ8gbI8DodU3e0le2muXgT+F1wMEoSHGJCR0lPZ0hs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With KASAN software tag-based mode enabled PGD pointer stored in current_mm structure is tagged while the same pointer computed through __va(read_cr3_pa()) ends up with the tag space filled with ones. Use current_mm->pgd' physical address and drop the __va() so the VM_WARN_ON_ONCE can work properly and not report false positives while KASAN is enabled. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/mm/tlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 86593d1b787d..95e3dc1fb766 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1295,7 +1295,7 @@ bool nmi_uaccess_okay(void) if (loaded_mm != current_mm) return false; - VM_WARN_ON_ONCE(current_mm->pgd != __va(read_cr3_pa())); + VM_WARN_ON_ONCE(__pa(current_mm->pgd) != read_cr3_pa()); return true; } From patchwork Tue Feb 4 17:33:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7BF0C02193 for ; Tue, 4 Feb 2025 17:36:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DA5B280005; Tue, 4 Feb 2025 12:36:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 58683280001; Tue, 4 Feb 2025 12:36:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D818280005; Tue, 4 Feb 2025 12:36:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1E261280001 for ; Tue, 4 Feb 2025 12:36:46 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CA95D1602C6 for ; Tue, 4 Feb 2025 17:36:45 +0000 (UTC) X-FDA: 83082967170.17.10FA3A6 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf23.hostedemail.com (Postfix) with ESMTP id DCFC514000F for ; Tue, 4 Feb 2025 17:36:42 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=RTOCnDVB; spf=pass (imf23.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690603; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pLFVRgBwiXf+T1hr1v6y3oXYvLLcOkmu5U12KRQmKlw=; b=MoSb6nAR+rmPPnMgkqTiK2oq292xiDZxLnkZL6aOrlcUE9YQXyLypo+zf2HhNJsIDGrsjy +h6wzcSCGFOtLWptqau0oWSLfCFuDld469G9gUkyEI3S4DMJ4D6ea4D/8SwPVz18ghuDQW aBNHv0ImGLY/xHIxg2OF1O5/IFI2Sgw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690603; a=rsa-sha256; cv=none; b=DO+CwDUwmjlbAP+8IEOF8eJcI9CWfLZ7a+aUCQG/0kcEUiX0HNuNaW1MVfx2bSkZSQJ2Su hOhtOlfncKKQBYxuHhY5d0N9NXYa5gKI23q2+4CCouAm5G26wGIm6b01Ij7N55/bODpMEc pHHBosSqcdmqorg0Ba8TM04iIsqGlxM= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=RTOCnDVB; spf=pass (imf23.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690603; x=1770226603; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J5wZ/mcDbil10hn8JDYxev0M9sT2GI5gsQFk6+vTupM=; b=RTOCnDVBaTkOxHrD/hTF0bIaVQU93uqwF5cKCEi5nMYe5PAR4K1uofBA izbauIEidJkdUueFKcL+ypqQ1I4aHchBcLdoVGncCFFoYNKSJiy/rbBTT Iryz4Ai6uiZyqoGJb3sNBZwF70xI6yKrjH4RSVsGyVd3p2vWXi2sy0imQ 1mfyJVVmJ1liivZ+jEouACQRMZY4WxR8gQizGRiCl17DbUM8U0yp1fpBt sQDtT06JmZKOq+Br3dOL7AjlpSs6wOprY5FgIlt4bSg1WaEsZCekTAPy7 PkmxmoodcCg0vMSbEmGzD1KFyjW80Gh0VmVmXEGCh74gvls0CUxMt7Khc A==; X-CSE-ConnectionGUID: LBoxu+ctSaeyjCoyesVrCg== X-CSE-MsgGUID: 81niKgaOSxid0oKm/cdeFQ== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930943" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930943" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:39 -0800 X-CSE-ConnectionGUID: euT5Fx0UQBaZT6g1J2Ba6Q== X-CSE-MsgGUID: VDPCZKL7T1GLAmeyS4orWQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866863" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:27 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 10/15] x86: KASAN raw shadow memory PTE init Date: Tue, 4 Feb 2025 18:33:51 +0100 Message-ID: <28ddfb1694b19278405b4934f37d398794409749.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: xcqneaf4fcseg6iimayby3b14o1rzjy5 X-Rspamd-Queue-Id: DCFC514000F X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1738690602-74393 X-HE-Meta: U2FsdGVkX18rQg9Gbx0vyk+hRi8b1B3PxoTb5Ud+60hUHu6wFeIHhH+RZMGn1kRAmr4naQqqhg1pdFK2Y/kyZoxfhPSHWqpLqZTKm/Fu0PUKsJKdgJmAmNYf2E2DtwlXFPVtWl4bJLCSYu71eYamCeCKmt72rcNpQzVAaYLdnxYKojmrHSlc/rB2eu8Zkn7Kvwu9Rv7zFN4/1mY2Yovyd60DnYYBNSdXqpRd95wGt7gwEr+wSZJHWIRglskxLWbuELh1klaDGPOmDy0XI+mUpFAHz7NzcmVKUyH8GglNGo39d+QE96zHHNWltOWxuQrLIAtA93C/x9c+fuUJrFZdQ5CW4BaFAD2dFDvSV+cisvAAV3DPpjWes+uWSD7FWtDsU0uDjk9MBCe8/uBoQEkzQC5GVYYknE1xHpFM4doTnIYVahPOfKfGddPU7lKSLgVkB27u1ngF5V3b4yvnsZOjpkHlJU+SMM59hpe7LSUmLOoFryLRXbI2D+YwwCP8mE5vxpAwC2/yEFvzf8H8/r5bjkpT9eU4Q30J9hFV6aAmA3B/UKiGyWtXh1P+ez58Gm9bF0CNlIml/b6ulvP2DBn2tElm0m5JnhzJuXWkzHiOFOZwdp0L9LhNl3vCBb9MVw2YBybWlG64178W4jKaoChQV4jwAKu12Y2nvhrCHGGHeL4lh7BvSxAe7A4sqgQ0kVE8uwFnwqgpHPsRmWZOacJEFcASgQVdkSV7aLIk6zP4vJHecJqozlZ2YNxl+Il/Rb4GaqUhtHfNS29IcUZ66LEbW9xijF651mMl4UzlTgRIwSHdR6WNUcJDV9VzBfSugjjl07378VPyY3yX7hgh+5iFqoaWNLNw8sP1vssHcmLnBfH0w72mUyhjVGuoayUkHTW10VBEWbPWUUBql+vrMxVKqXr6iMgI2Sqzfhbb1IfTih+ryP+b96pvf44Inv3VpeAVrlRQP4KDZ45hTUwiCOO 0dxIkdJN I7hgFVWaLp7MDekw/de1I3py0gYDlB4GIvvmQVCsUc+1VlKAciIkZhBjohS2fORLGA36ooClaB4bZRo/RnZQ9wIOMaWumMN4i38vIpJgQ8Hu4hix4U/VB/1pEFTBVzYcUBjWQKiAabP12RENQA9ldmtx4ur4c211joEg20lqwtbhjg+scx7lTLVdQLCUA0DHNaBnDzlbSKDBpe5vsr6zDkihGNOTvRu07vYGXHPqCSC1x4hSt9qWdY1jObw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In KASAN's generic mode the default value in shadow memory is zero. During initialization of shadow memory pages they are allocated and zeroed. In KASAN's tag-based mode the default tag for the arm64 architecture is 0xFE which corresponds to any memory that should not be accessed. On x86 (where tags are 4-bit wide instead of 8-bit wide) that tag is 0xE so during the initializations all the bytes in shadow memory pages should be filled with 0xE or 0xEE if two tags should be packed in one shadow byte. Use memblock_alloc_try_nid_raw() instead of memblock_alloc_try_nid() to avoid zeroing out the memory so it can be set with the KASAN invalid tag. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/mm/kasan_init_64.c | 19 ++++++++++++++++--- include/linux/kasan.h | 25 +++++++++++++++++++++++++ mm/kasan/kasan.h | 19 ------------------- 3 files changed, 41 insertions(+), 22 deletions(-) diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 9dddf19a5571..55d468d83682 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -35,6 +35,18 @@ static __init void *early_alloc(size_t size, int nid, bool should_panic) return ptr; } +static __init void *early_raw_alloc(size_t size, int nid, bool should_panic) +{ + void *ptr = memblock_alloc_try_nid_raw(size, size, + __pa(MAX_DMA_ADDRESS), MEMBLOCK_ALLOC_ACCESSIBLE, nid); + + if (!ptr && should_panic) + panic("%pS: Failed to allocate page, nid=%d from=%lx\n", + (void *)_RET_IP_, nid, __pa(MAX_DMA_ADDRESS)); + + return ptr; +} + static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr, unsigned long end, int nid) { @@ -64,8 +76,9 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr, if (!pte_none(*pte)) continue; - p = early_alloc(PAGE_SIZE, nid, true); - entry = pfn_pte(PFN_DOWN(__pa(p)), PAGE_KERNEL); + p = early_raw_alloc(PAGE_SIZE, nid, true); + memset(p, PAGE_SIZE, kasan_dense_tag(KASAN_SHADOW_INIT)); + entry = pfn_pte(PFN_DOWN(__pa_nodebug(p)), PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); } while (pte++, addr += PAGE_SIZE, addr != end); } @@ -437,7 +450,7 @@ void __init kasan_init(void) * it may contain some garbage. Now we can clear and write protect it, * since after the TLB flush no one should write to it. */ - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + kasan_poison(kasan_early_shadow_page, PAGE_SIZE, KASAN_SHADOW_INIT, false); for (i = 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 83146367170a..af8272c74409 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -151,6 +151,31 @@ static __always_inline void kasan_unpoison_range(const void *addr, size_t size) __kasan_unpoison_range(addr, size); } +#ifdef CONFIG_KASAN_HW_TAGS + +static inline void kasan_poison(const void *addr, size_t size, u8 value, bool init) +{ + if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) + return; + if (WARN_ON(size & KASAN_GRANULE_MASK)) + return; + + hw_set_mem_tag_range(kasan_reset_tag(addr), size, value, init); +} + +#else /* CONFIG_KASAN_HW_TAGS */ + +/** + * kasan_poison - mark the memory range as inaccessible + * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE + * @size - range size, must be aligned to KASAN_GRANULE_SIZE + * @value - value that's written to metadata for the range + * @init - whether to initialize the memory range (only for hardware tag-based) + */ +void kasan_poison(const void *addr, size_t size, u8 value, bool init); + +#endif /* CONFIG_KASAN_HW_TAGS */ + void __kasan_poison_pages(struct page *page, unsigned int order, bool init); static __always_inline void kasan_poison_pages(struct page *page, unsigned int order, bool init) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a56aadd51485..2405477c5899 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -466,16 +466,6 @@ static inline u8 kasan_random_tag(void) { return 0; } #ifdef CONFIG_KASAN_HW_TAGS -static inline void kasan_poison(const void *addr, size_t size, u8 value, bool init) -{ - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) - return; - if (WARN_ON(size & KASAN_GRANULE_MASK)) - return; - - hw_set_mem_tag_range(kasan_reset_tag(addr), size, value, init); -} - static inline void kasan_unpoison(const void *addr, size_t size, bool init) { u8 tag = get_tag(addr); @@ -497,15 +487,6 @@ static inline bool kasan_byte_accessible(const void *addr) #else /* CONFIG_KASAN_HW_TAGS */ -/** - * kasan_poison - mark the memory range as inaccessible - * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE - * @size - range size, must be aligned to KASAN_GRANULE_SIZE - * @value - value that's written to metadata for the range - * @init - whether to initialize the memory range (only for hardware tag-based) - */ -void kasan_poison(const void *addr, size_t size, u8 value, bool init); - /** * kasan_unpoison - mark the memory range as accessible * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE From patchwork Tue Feb 4 17:33:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959494 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6632C02193 for ; Tue, 4 Feb 2025 17:37:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 400A3280007; Tue, 4 Feb 2025 12:37:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B19E280006; Tue, 4 Feb 2025 12:37:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2037E280007; Tue, 4 Feb 2025 12:37:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F329B280006 for ; Tue, 4 Feb 2025 12:37:02 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DA610120CC1 for ; Tue, 4 Feb 2025 17:36:56 +0000 (UTC) X-FDA: 83082967632.06.1F0C2CB Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf07.hostedemail.com (Postfix) with ESMTP id CF4B040012 for ; Tue, 4 Feb 2025 17:36:52 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=lTyGKWCp; spf=pass (imf07.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690614; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wcThim39EYPHxXLHIVD/RA5qpLaRNquzUdH4zXdFJ+w=; b=5cd3RpsBS3nDv58Jn4H0nQJmk252veQ1H9TXnXO8PCFB92pkT3CaEdGv7KTnScXxqrDgyt v4yFZTkrNP2/4ZiGGMDWG31u1d1/nfC8RZ8pi9F4T+dH2BUtvgnU8os3/rJQHroSUgg5Pv B9X76D8mr6CCjBowK92jvyOQm4kETEo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690614; a=rsa-sha256; cv=none; b=POwomZ59uWT3gFbDTLaifkWX3x2qcjJ8LtXCdaWCrdykNADASS23EnOIcLKGdy1w2F5XT4 894veMXVdp74lHDYYdby7acinVvRx54jUbT/xYvfhlOP15G/FapnHXLbqOtRr0+z9XGia5 vWCia9237MbJJor9dO5BMVA1LaklRuQ= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=lTyGKWCp; spf=pass (imf07.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690614; x=1770226614; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EUgvSoD6ZXvZMdQY8EXK092WIVXF++/3+F17U+GxOWU=; b=lTyGKWCps+Gv8uBBObQvGrzpGKpend5mnXBqP/YOlCATUeIX6E/yXR8e iO7Jw2WaD8NLcFP2suUr888jgSK/5VSQ1dImzo7WyeTsb/l5yxAX7jzHY afHL0DgFVh3ZFvryBE3Q88iUqbw+AyaWwOn8v2sFutsPc4v17A/NcxzDV Y4ADOeiSTvCcbnhp2j8QUR4cPScpBnJ+FC8GZ0ACjf5YTPN8lh+D9EUp0 is7gk9N6H9zccswE1RbRdlXsQOSxy+zpM8kvEwFn7D6k/UxWrPxz/SjbD xG12PKuWrRUrdT+SkDk2t8yY32epiAp2dhknEy3cTpzn3A3qErVjwGrxZ w==; X-CSE-ConnectionGUID: 23dPVRDMRBu51+hzBR3J3A== X-CSE-MsgGUID: CHITLcQ0T4O3Gyxg2PICYw== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930987" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930987" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:51 -0800 X-CSE-ConnectionGUID: P6du7V6VScafjuMIXvnbWA== X-CSE-MsgGUID: W7s9nIAMS+GBPujRL+zY2g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866889" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:39 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 11/15] x86: LAM initialization Date: Tue, 4 Feb 2025 18:33:52 +0100 Message-ID: <01104816cdd0d430ac843847a8056d07b8770be0.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: CF4B040012 X-Stat-Signature: zp8wn961788jx8cz5h7q9gn6uamcf3nb X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1738690612-971968 X-HE-Meta: U2FsdGVkX1/lTo6UCJP+D73rImn746X90PR/GOyuww2NONd4VovImhAg8xDtbEatiM0XyIDs8oX2RwRkcd0/F3T3F1Axku8SW/1HVyifG7wH9K8bjCN4GfH83steihMEPBASk5UBAqqvXExQQYPLqYeyC7D+p7x+64IPdixvWTceyHu+NNuP6vrMpIt7TMQFUloRdeZtu9A1hgobstT4p0+8eQ//xqKJDQkWAaZmiNnMmccgmAMq1wI23fzMoEfpf2cdvIOEOAdG3hyoBJ+vO3OSMoIiisQ3J7UERy+GHV2V5iISBLOASkOEUIMc41Vqczf4P/RxSlECzGleAbOMV/TZOaTc2MEp6PibdOaPj6itRCMgp2NQ6UbNAH6HP9s9mE++ODT1YiywaznYciYUONuqAC5tS6indCz0fZ+S9e+f/st9Ovj7StAmfbO5yry5BG62SG7+nVreQDWvSnivySELpc7lP/XHA9Ft5EsEql4/kPv5rgx+BUrZotoaQT7TWdYfdls2U5VG+UjXeheKIix5vhU1vR58oJknbSI/pU8CgISEu1iHrSYbuJqgZ4urcYMe5MywAPdF/CUWXdsBenFpaHygRcE7aOZtocN37S8RFv+DLNDOdqQE3UHaAVLwvpFBo/p5T7zGpvHq7459Njf6oHglirwF5KADUaDTsMQjWp0HKYEgM1/rY0Z5pjHeUM53Df6Awbq3bfHvbn/664wsi/zBliJ/qEF+PXzJIHB/Rky++hxfm9/uAn+W0IHnlZKo73lNI08HXrSJcn70fVRuXsmHInPUVsR4+bR2brlzMkWa+uizlS2LEAo/1sy7RWSGq4YosQWVUnmtHp7G9Qir5HfREKAH4/9QEvMKZWwqCkmmIzCAv3JJsIYIE5gqS+zQj9W1IbQLuRXvk1LQ7Fc4llBvh/J42v54sbmIiO1Yx/f02uOIUoxw4mSPJIyq9kPnDIgSeZ5GRsh2Qf3 8vPgMO16 tehymF/sxmbGHjvXVdbQmR+T7+CWuH+aTQKXdkZmr+THmPb7Z7ymE4eHEu1Y1aQEglWWw/zhYzUux/zK4nGRv2Vt5PBoDyTznV1A4fr4DTMsePQi4gcQ/Fd6p3kIn1h4e9Rvx3Rhi4+aj6C9gdA/exqVbsWCfSZMdy0RYSVl9IXWsVIpN7H3HD8lDdRClMX4iF9UCgijxo0xmgHpjvo+k+StRT6qMcLKGUo13vEpt/w2quJbKdgoyxPKxhQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To make use of KASAN's tag based mode on x86 Linear Address Masking (LAM) needs to be enabled. To do that the 28th bit in CR4 needs to be set. Set the bit in early memory initialization. When launching secondary CPUs the LAM bit gets lost. To avoid this it needs to get added in a mask in head_64.S. The bit mask permits some bits of CR4 to pass from the primary CPU to the secondary CPUs without being cleared. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/kernel/head_64.S | 3 +++ arch/x86/mm/init.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 16752b8dfa89..7cdafcedbc70 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -199,6 +199,9 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) * there will be no global TLB entries after the execution." */ movl $(X86_CR4_PAE | X86_CR4_LA57), %edx +#ifdef CONFIG_ADDRESS_MASKING + orl $X86_CR4_LAM_SUP, %edx +#endif #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index eb503f53c319..4dc3679fedd1 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -756,6 +756,9 @@ void __init init_mem_mapping(void) probe_page_size_mask(); setup_pcid(); + if (boot_cpu_has(X86_FEATURE_LAM) && IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + cr4_set_bits_and_update_boot(X86_CR4_LAM_SUP); + #ifdef CONFIG_X86_64 end = max_pfn << PAGE_SHIFT; #else From patchwork Tue Feb 4 17:33:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CF2DC02193 for ; Tue, 4 Feb 2025 17:37:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E4B2280009; Tue, 4 Feb 2025 12:37:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 86C1E280008; Tue, 4 Feb 2025 12:37:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 697BB280009; Tue, 4 Feb 2025 12:37:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 42046280008 for ; Tue, 4 Feb 2025 12:37:09 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0C28B1C8369 for ; Tue, 4 Feb 2025 17:37:09 +0000 (UTC) X-FDA: 83082968178.20.CB79B82 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf10.hostedemail.com (Postfix) with ESMTP id CA563C0013 for ; Tue, 4 Feb 2025 17:37:06 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TEFLVNMc; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690627; a=rsa-sha256; cv=none; b=rqKlkMRhNKaoWA+jrg8StBGXb321JCL46G1zSyzhguXIz1Bpe83PddbOiU0MmvtOowDQbe QdXb4bHzoW3SgTvFOEKhAqr4xHl3v0NipZTJ3qhq3Qa/oJl9nndvigBJacwabcqQIHu9Nb QLqlYSXpV005zfwqTgxF3ZvIHN4Q37M= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TEFLVNMc; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690627; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9cP48FhwOS/LMk0h6akqBPktUCXlskOxYu2pQkbZsrI=; b=fGNJHsI2qrdEm+Vc9L9cTf+7aCjBP6g6XBceYLxYzcp8o7BPwr45RZndUboJrC5X8L2FAZ /m4etVH2ye9UDFm8dfCtL7JCSwRKKl6DjcObfK0UImTHpOSlBdjTVRBxENtCXZSmKIuvkF x0YEZWJV7ffQpUNYgFtAdp6h/flaWOo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690627; x=1770226627; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r2iiDiPZOS4/QANoocz7hbq1LBuYUYYysQaiQVX4JUg=; b=TEFLVNMce6gHlOLoCH5u1GAwdzzHg4AHFn7Cgdsr4pBzjh/GQELY+j4q hrb6BbSNhhyoX1Anh8mrbVuF2GBjJdjXoGqmShDAfXSn76IiOU1P4q6dw ZIy86sb70aWNv9dl4kR5DQfqkNKtjxtPG92owVH8P6eNJ86sWvojMBBK6 ONZ4EBnu1lK4b2NUjv0NKQSgSUyTWVngrunster0nthoPqGMHzEkWM+Ms W/HYyI8ZYJsjXpkHrh5CqnilC1LTN/AIhIgWUUUxli/YVirjmxTV3IOfQ ZT6S/ffTtuKocyDQDgEzhAAmfYhdGuEiP6PvRVkey9f+wx5x/9uR77TMh Q==; X-CSE-ConnectionGUID: t5yW6EUDQYGlO0UZ/DJCkg== X-CSE-MsgGUID: iQbMghXqRFqgHXaT/yZYww== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931055" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931055" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:04 -0800 X-CSE-ConnectionGUID: F7WrZE34TJC30qp8xAUvrQ== X-CSE-MsgGUID: 1Ko21FfoQgSOsBU65eanYQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866985" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:51 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 12/15] x86: Minimal SLAB alignment Date: Tue, 4 Feb 2025 18:33:53 +0100 Message-ID: <162610a0af3e04e2f42872401461b1d62ec78fbd.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: CA563C0013 X-Rspamd-Server: rspam10 X-Stat-Signature: bukm3w51sopcq19axwm66qmpqonzx66x X-HE-Tag: 1738690626-37851 X-HE-Meta: U2FsdGVkX1+ER1f+GAMIljpE2k7rAqvVsoD1EgWR/K7Y9L35JpRN/ZMvEKbcqejgp9a15Y1fiENp5lWYN8RcdPvSsrHpBEWEUpyr+4ul8STc2e/Sza4gzPh1Tu5EYAyvZf6h8NwKrMxs0BS1YijS/CBPkjj34Pgxp0CLPYoErk6qVfhBN3rq4PyjjOuo8p9fB8vGMENsz2YIE37gYhvpYI9fduU/IXX1orIwlSc8hxCiIVmHm7k9lJ3wB/rpVjPzhpRaxAJvL80tn7d7RMZ1RfjH6u009zqNGlv5pDp1lbShilhVZrEtv+GOXliG+PLSLf4QLCF0B1u5aJUDMp1hqXdQGr7a1DJYYH5sl6VU+v6x3cX3Lb6/xn0gD9Uy49xq8urXJHAfSP6ad6EPaIrwGRSwmgZ34M6qX2pQpKdkI5ltn3p2CSAr3QhLHnL94m831e9F/ym3Na6IKYmHZIGe//UFUzf0Vr5a/JDf5i7K1rzn5EbVmefibktSCX67L0n7sxRmSl+gr0m1yrWMJyyJqKIQlFNkmH9YVuJqIFUOy6lgEih3Abe16HeP2FsGaCvHlIhgXbqPrGSkh6g/THgwwTrw3Xuc1q14TByGyv3hdRnUdbBI9aQWOWKFu7urZznJ3sJNZrz4NYO+0cFbq25yi6HflMtfYO38ighT9dGsYHs64ETQTULVXetXsgq8GBDe04hUNppP9ssDPHpuVIxk6cWXYhnOm0mMu65dP+cgXn3LuSQa01Hm+xfdzVAAIbS7H8fP8+wiFpdkTF34eC7FHR62VKjbAQ7KWhH6RiKcRI/6Qr3qNst+WHD/2IJLxRSgVeeQ62yrQ+2u06W8imF3Z3y3BAOO58bCiVFPg6jzLzqAI8jfTju9cWjfI1KEqxY8ZPwyq5ZovlztkgPinZnsrtaFytf4bpFhOa7CKOzjJlLvQZhFmP+CMyMtvDPMQ+S/lowKCjz6xvp8bjZrlp3 5/86PfKr PzExLqfRn3vjOHn9D9YS59i3AbxbN1ihoxG9EXPmSyXc/quzwVxXznWpyXBkjidvkgtJYeq1SL/UM0hg/KZ9Bi2/cRb2ZDE5ZI91vBnEw17p37eC5oNXQtG4pksfkIAxYcdzqXcap/luNNrxv1L249qp6YgWvN7TOkDioynoGr/XDP9VoSwJ46sdUs1kr6PVj2Tdmc9IN1aVm+Q8/DHtKWnR7E4F9nsZbt8kVpSMXuXKWAIuo/S3CGt+rFg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Adjust x86 minimal SLAB alignment to match KASAN granularity size. In tag-based mode the size changes to 16 bytes so the value needs to be 4. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/kasan.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 8829337a75fa..f7a8d3763615 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -36,6 +36,8 @@ #ifdef CONFIG_KASAN_SW_TAGS +#define ARCH_SLAB_MINALIGN (1ULL << KASAN_GRANULE_SHIFT) + #define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) #define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) #define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) From patchwork Tue Feb 4 17:33:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959496 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2770DC02193 for ; Tue, 4 Feb 2025 17:37:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B545F28000A; Tue, 4 Feb 2025 12:37:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ADDEE280008; Tue, 4 Feb 2025 12:37:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92FFB28000A; Tue, 4 Feb 2025 12:37:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6F133280008 for ; Tue, 4 Feb 2025 12:37:27 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D038EA0C30 for ; Tue, 4 Feb 2025 17:37:25 +0000 (UTC) X-FDA: 83082968850.22.3A59517 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf14.hostedemail.com (Postfix) with ESMTP id 392A2100013 for ; Tue, 4 Feb 2025 17:37:21 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=mFESbTU9; spf=pass (imf14.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AJ+KNrmQGw87u/Lv3sW/ixn2Pc83/zgfJdm0b3G7QSs=; b=X7eHMLrny0/xPRMz0mWkbl0xe7YNcghK7r5K/etsqqWY9VlZRlx5pfhDKy+SkJezEzZmLx I2N4qTkqkGvSFd+8+EFM4dLUU7FYAdXkT/MxCVz2omkqoOe0HFXuDlpwM2pi0Xmj4DQlnJ XrXdC7/ytvweK24ybl9x4pqi1PdMZlk= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=mFESbTU9; spf=pass (imf14.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690643; a=rsa-sha256; cv=none; b=oZTwZaXrsTM0Osj41rs34GJcyLAP1/ZduYEmDJBaFppj0GQTYPfxqaOQ5yqEFAXF92bIYE yVG6ogMTWWLQX5y9fpuM7BKVY9F/jvq6JUL0ASIxUCWRUccptNHeVtmNrELkzCH3aATw1E l544sPkhe8hYM3wFzshu03FDAxg19kY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690642; x=1770226642; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tgQnPnMq4WHkStd3vJbMqoHpzLn5Ygn4fGXf7vNlMB4=; b=mFESbTU9nXeiVEzQwpZ0873UAhu6UFcSPd8YK8UqGu4wfncKoanfqlkN pudmvobTB4jR11zXTVUS6w+hoJmZTPnAkF6ofXPDY+SWTiDTGI7Ik+Ani qphC/4zUT7lbeY6I37t27/uc0/rkCOIL+bCgcSc9RgBg/z9zpxwjjkqqw +3+V746PjlR8qn0y2mz7zeNK4LbD5qJeoD1fZq6oPZs7EXx/lgwfz+g8l w881eXWJ2zdFe1HSJkds0lHgbnuISFhJA7s7HHWKMRXTsy5BNvzt1Fk9u 96GV8y2FByJO+I9P/X6VbkVpO4CHPSrNeS43bANE60rrZ3+f5qSluqKsR g==; X-CSE-ConnectionGUID: +YDXhTd0SyKm/pYpBo5gAw== X-CSE-MsgGUID: pkf+qc3MR02TRAS8+i7eiA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931130" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931130" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:19 -0800 X-CSE-ConnectionGUID: OH9xu6cPRWustG/bFhP1dw== X-CSE-MsgGUID: vAuLaG4TRpmkRprgXY7/Og== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147867096" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:04 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 13/15] x86: runtime_const used for KASAN_SHADOW_END Date: Tue, 4 Feb 2025 18:33:54 +0100 Message-ID: <5d0f9dbd0f7c2326229f2a1f3dcedd46842a9615.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 392A2100013 X-Stat-Signature: gbwm8myuo4qexhjyoeotz8tcbrggikfr X-HE-Tag: 1738690641-484196 X-HE-Meta: U2FsdGVkX19w7Hi0B/NpV/RAycCBipSqTfVujaE7WiBCEhwtrMTDvzp5WNTtyLPs04erzM9eqO7A4fEp+sSjCnpK3D2JXjGwPCB0TXILDD9VzPQDfE+nvQ2Qqp5wB1un8WT6vKbd0aDyy0E/C9BLAmWlOoFR5tHjrPe25L5bdWMFv7nNocCCy31pF5KxjGv6fBiVWBUbehJmxrY+9NnQlvTb02PEg0e/7OARpl3+/TiQxFLHMU0Z6VzlN2Iux6qXWaWEsRgVNioSgBIPkmWU/CVdPEFK7nQq5vxGSNEAKVEVgApE27Ub/6TL3DrfA47sNMxbSRopy86chesh0/FUxKEGcmHx5xrZwIF/aumGFXwyLsuboz4axH4dBpigfGAMFOkHzyeglfmpIGP7kNz25LOZwD6Bf1aPwkQLWwF6sovbEFGhI/XPvoI6GbuFGDbGJ6Pzm9kmMiVhFSCQyhpnd/HaknGh4EPFFAHHMVkQmNg9EMZH9NR9JhOJFYsnwoXjeo/9wmBnRp0lc9W5P1cb6hNc6un5afxNNk0IyrvQoeRS18XvQKL3RBgGTDBO6F4xriNeDmoEMq1UtWmmY17gFuXvXvfd60LSZYEFUx4f7nLEtvRyzM7otUI68DxqyL+EEf/a8HjXiNKTVNeaRxTOdHCyIvlq3EHlAI/y+q9X3dLEhicMxXmgippZLikMdmwAw6L6OMNrmYWmjBNlqPN0oOrWqLg6dPdQYnGhUeCssARDsuuUQqlI1XMZB1wC6tbPVA7ShUVjl0ksj6/WuyBvaPfkfam9OIu5O1QPpAdC4MbOhNplOTW6XlUEF8nMrrQzFFY80O+XD970W7GwGL2AIz2ecusAb278L9gNd1rX43By+jMugv2is1Jqmn44PGPAIzf7DuylkXuxBkU7efZy+i8qY/ypz1q4FOGr7rulTyiA4/Y8dY05J1pWnUH1nK523OnY2UdzXrT3Q9dv5aY MhCZ2jrf s12QKHzEC23unZ47uOM4pqKP3DUim6c3E/YD6rQfIn32ZHnl0KqIs/UUQdoQ880ty4PSTKbbPpKDkm7+VmvIWpz10OdH6iqNBfaQKoMCxnLEJJ7DMLmzHJwDlLeSxbT7GJxL5HB+r/bzhEJyroBLrsc1XLvImTeR4MMpXLLNHnguzzTUuReCyb6Be42DYfNWBy0gblvEKk4JqVsDfXtbKF21bPpgVmn0t2o2muxKzQfKCiRq5/7ZsHwJWLp918bMD5LcB2inEtFMBA7MAOoqUUeK7v/bor4PYsnhEEnONrlVb1jo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On x86, generic KASAN is setup in a way that needs a single KASAN_SHADOW_OFFSET value for both 4 and 5 level paging. It's required to facilitate boot time switching and it's a compiler ABI so it can't be changed during runtime. Software tag-based mode doesn't tie shadow start and end to any linear addresses as part of the compiler ABI so it can be changed during runtime. This notion, for KASAN purposes, allows to optimize out macros such us pgtable_l5_enabled() which would otherwise be used in every single KASAN related function. Use runtime_const infrastructure with pgtable_l5_enabled() to initialize the end address of KASAN's shadow address space. It's a good choice since in software tag based mode KASAN_SHADOW_OFFSET and KASAN_SHADOW_END refer to the same value and the offset in kasan_mem_to_shadow() is a signed negative value. Setup KASAN_SHADOW_END values so that they're aligned to 4TB in 4-level paging mode and to 2PB in 5-level paging mode. Also update x86 memory map documentation. Signed-off-by: Maciej Wieczor-Retman --- Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 3 +-- arch/x86/include/asm/kasan.h | 14 +++++++++++++- arch/x86/kernel/vmlinux.lds.S | 1 + arch/x86/mm/kasan_init_64.c | 5 ++++- 5 files changed, 23 insertions(+), 6 deletions(-) diff --git a/Documentation/arch/x86/x86_64/mm.rst b/Documentation/arch/x86/x86_64/mm.rst index 35e5e18c83d0..4e8c04d71a13 100644 --- a/Documentation/arch/x86/x86_64/mm.rst +++ b/Documentation/arch/x86/x86_64/mm.rst @@ -48,7 +48,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory (generic mode) + fffff80000000000 | -8 TB | fffffc0000000000 | 4 TB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 56-bit one from here on: @@ -107,7 +108,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory (generic mode) + ffe8000000000000 | -6 PB | fff0000000000000 | 2 PB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on: diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7b9a7e8f39ac..dfec7bc692d4 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -392,8 +392,7 @@ config AUDIT_ARCH config KASAN_SHADOW_OFFSET hex - depends on KASAN - default 0xdffffc0000000000 + default 0xdffffc0000000000 if KASAN_GENERIC config HAVE_INTEL_TXT def_bool y diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index f7a8d3763615..79151356d5f2 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -5,7 +5,7 @@ #include #include #include -#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) + #define KASAN_SHADOW_SCALE_SHIFT 3 /* @@ -14,6 +14,8 @@ * for kernel really starts from compiler's shadow offset + * 'kernel address space start' >> KASAN_SHADOW_SCALE_SHIFT */ +#ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_START (KASAN_SHADOW_OFFSET + \ ((-1UL << __VIRTUAL_MASK_SHIFT) >> \ KASAN_SHADOW_SCALE_SHIFT)) @@ -24,12 +26,22 @@ #define KASAN_SHADOW_END (KASAN_SHADOW_START + \ (1ULL << (__VIRTUAL_MASK_SHIFT - \ KASAN_SHADOW_SCALE_SHIFT))) +#endif + #ifndef __ASSEMBLY__ +#include #include #include #include +#ifdef CONFIG_KASAN_SW_TAGS +extern unsigned long KASAN_SHADOW_END_RC; +#define KASAN_SHADOW_END runtime_const_ptr(KASAN_SHADOW_END_RC) +#define KASAN_SHADOW_OFFSET KASAN_SHADOW_END +#define KASAN_SHADOW_START (KASAN_SHADOW_END - ((UL(1)) << (__VIRTUAL_MASK_SHIFT - KASAN_SHADOW_SCALE_SHIFT))) +#endif + #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) #define arch_kasan_get_tag(addr) __tag_get(addr) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index feb8102a9ca7..46183f7439c9 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -359,6 +359,7 @@ SECTIONS RUNTIME_CONST_VARIABLES RUNTIME_CONST(ptr, USER_PTR_MAX) + RUNTIME_CONST(ptr, KASAN_SHADOW_END_RC) . = ALIGN(PAGE_SIZE); diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 55d468d83682..0f8190e0e5f6 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -358,6 +358,9 @@ void __init kasan_init(void) int i; memcpy(early_top_pgt, init_top_pgt, sizeof(early_top_pgt)); + unsigned long KASAN_SHADOW_END_RC = pgtable_l5_enabled() ? 0xfff0000000000000 : 0xfffffc0000000000; + + runtime_const_init(ptr, KASAN_SHADOW_END_RC); /* * We use the same shadow offset for 4- and 5-level paging to @@ -372,7 +375,7 @@ void __init kasan_init(void) * bunch of things like kernel code, modules, EFI mapping, etc. * We need to take extra steps to not overwrite them. */ - if (pgtable_l5_enabled()) { + if (pgtable_l5_enabled() && !IS_ENABLED(CONFIG_KASAN_SW_TAGS)) { void *ptr; ptr = (void *)pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_END)); From patchwork Tue Feb 4 17:33:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45DFAC02193 for ; Tue, 4 Feb 2025 17:37:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C66B228000B; Tue, 4 Feb 2025 12:37:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BEE79280008; Tue, 4 Feb 2025 12:37:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1A2928000B; Tue, 4 Feb 2025 12:37:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7ABB4280008 for ; Tue, 4 Feb 2025 12:37:35 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0EF634A58B for ; Tue, 4 Feb 2025 17:37:32 +0000 (UTC) X-FDA: 83082969144.11.A503322 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf14.hostedemail.com (Postfix) with ESMTP id 7B923100015 for ; Tue, 4 Feb 2025 17:37:29 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AT+emMBs; spf=pass (imf14.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690650; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PCQPfnDRBxmJXeqGE9pZTyAVVotiYe5di+G+aFzDagA=; b=LCPDdqyjBwPUKVNB4zWCFM9dM9pwuKN2jV0zjmp8bkFNPnMhTs9njnecDvp95iMtGCJdbe Px7biLUrOrDlK2V7BZ6zYaSn/G/bOuywFoxY30R3FT0w0GL+7/NN3QQgzWG4NOAyIEhQ45 /6BAsRjEe8TWjIn0elUisgqGGqtYJM4= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AT+emMBs; spf=pass (imf14.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690650; a=rsa-sha256; cv=none; b=F/Cje172/WBgL/XHJJvzurHxrQ+gCazJHzB0M5ofW7AadSwOQfnVpNxKs3E0qvtjoEMOAa qSeUVuMfyumLN4v9AKQsW8xYhRa5hasDfTSs1nMc1tuq7wQKEDWNyaHCyeoawEkvEpNYuc sd+eZbf6msfcp4WwUGYFfFWWdOw4YNg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690649; x=1770226649; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oiRBgvokfmZMzIN+jtwGJEePgjLFxs7ade5tHmSo4go=; b=AT+emMBscQCyn5fPQ0oRIf8s1+/d9RwkDxYMHXX17mDxoicd+LJWOuql geskYX11c+AAzWrHvldMohSNmfnEGiQdy+kv/pJnxUrlpFswzzsAzI0Cq jy13QgoMsIBxhruW2ZqAMoFWabJM8BDv5EC5BuIGZzwJZ4HBMe6fP2g62 R+eV2bJlg52AwWy1Ou8c5BEZMWqD6jJkWdrgGEeZjoibPrWWGT3fPxdS0 R0QO1iDNLI7pnezuNXF7v3EB5fBt3Q7HFszVJd2HgnP+JGLDm6cStpnN8 S94DXFk0kg2w/nXh7IFPyousj4GdP0EiGRz8RkhxDl8vSLWJIIjrjjjW3 A==; X-CSE-ConnectionGUID: 9vb9AemGSM2fr1SMsOY0SA== X-CSE-MsgGUID: exZOcm2gR+i/fvLhJukydQ== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931168" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931168" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:29 -0800 X-CSE-ConnectionGUID: VKIBhH62QgyxuyV1BEgDvg== X-CSE-MsgGUID: /T7gEKWsTDWNscbEm9C8IA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147867163" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:17 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 14/15] x86: Make software tag-based kasan available Date: Tue, 4 Feb 2025 18:33:55 +0100 Message-ID: <794a931acfb8e73e28c02932ef08bed9254f164e.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 7B923100015 X-Stat-Signature: wdmk4ja4jgapwfd5a6whgqnu4pkb6pwy X-HE-Tag: 1738690649-215513 X-HE-Meta: U2FsdGVkX1/x8Hi+6pvSnvIgH0T+XPkeLOFmIHlxli8cysqV6efBfB8d7C734xfVmmrRmEZUzWHIQVXcFUWajre8kpADBwqKEHgzhoVJHHNMv9bGzMZHMlaG5bDaJA8KHmrMFMW3CElmBSSOBCCKUFyVtvDj6Qn0M4w9goKwvL70OuWcF2PCEmf2WUR89BmnrJvMgwFAU6kCSM7dQVGVLR51Wwy2aRqnqyeBDP1VgYG1TBJgskgiNdIIkixiy/eAPrZCulwstiLPdGOD0I5YNCksY5aqDGrqN0tb252rgbY+0Z+g4mrWyESTSVarRn4qeff4ypC9ctiAkEqo4leaSmBC04O4w6rPkEzQczRMIwKUSJxLRivHFQVAK7xd38m/ws7LJBUXw7yxlGFtyjg5Zvx1KWiKfTohpLDjhkcn8hwzhVaD9jS+q5gPpPwMI1zOBYqC1L6vS8IhHDhtNyhWj5EG75tafDR/BlbZ1pahYhXe7Pdujk7G4HikYv4jA8wggVjDE+3ePj0sINNfLHafmdRzrmpwNxca9jID+xzUfm5423uekSsNxT/wJdrSHw2gsM0evjDcu0hFwL6ek4vxftO37MBXzKrBjQJY9h9HkvQgmnFuKLMrfOFZH6UoSVnkU4IkvEMg2s2BV0Z1oszmACrY6X7ZnBJiOpyUQKmxc+gg5G8rBarob3ue3SQZB9HVH3OMfNr2k5tLNttIt5BcLRKJP8f9SGlNEXNl6+Lp5H17rrm6a6e4T5NKm/Z9nueERo+2pE4KXMEp9yUxfoyKm8ieKNNq5pM2o9YVFhdz4rvjXt9gvocKHCqBQJTyFoIcLdg52arFnQ8lcYFzFBfqf/pLPLcdrrZj+wx8nwQbfFxQUkD+gd7PJRcL5UKKw9lbXhqjDo+7cyw57V2yVyUBZlu+dXMtgM8uFH9HBZedrYkISmA+6xU9bwBfeMQGHzeBwoCylTU0+b0Sd7z/2ff RFa/ndy+ zjsMBKkIRHnCOuNKqzd3DZEwp0tC7AR5UOYuiTd7GQdpzQwjllEJ4gxN4JlUYJuKLdCmXTCRgmg9c+Gri4zAm/ksI8bQRMgJREYlwaNggTlrc2re9K8sf1lwjXlNBSAMF9C9Du47w6CmApfU5phXr1Mzk5h7E5SUEo9Ch3E0ESjwSRZwl2oUzyXMEDa1/rTHXKalgmrJMlY3NNUmzUVJl1dIhbqt0sBDye+ON5L60JIfNRjcY0JolWppTyA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make CONFIG_KASAN_SW_TAGS available for x86 machines if they have ADDRESS_MASKING enabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform. Set scale macro based on KASAN mode: in software tag-based mode 32 bytes of memory map to one shadow byte and 16 in generic mode. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/Kconfig | 8 ++++++++ arch/x86/boot/compressed/misc.h | 2 ++ arch/x86/include/asm/kasan.h | 2 +- arch/x86/kernel/setup.c | 2 ++ 4 files changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index dfec7bc692d4..afbcf27ad278 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -36,6 +36,7 @@ config X86_64 select ARCH_HAS_ELFCORE_COMPAT select ZONE_DMA32 select EXECMEM if DYNAMIC_FTRACE + select ARCH_HAS_KASAN_SW_TAGS_DENSE config FORCE_DYNAMIC_FTRACE def_bool y @@ -190,6 +191,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB @@ -394,6 +396,12 @@ config KASAN_SHADOW_OFFSET hex default 0xdffffc0000000000 if KASAN_GENERIC +config KASAN_SHADOW_SCALE_SHIFT + int + default 5 if KASAN_SW_TAGS_DENSE + default 4 if KASAN_SW_TAGS + default 3 + config HAVE_INTEL_TXT def_bool y depends on INTEL_IOMMU && ACPI diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index dd8d1a85f671..397a70558ffa 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -13,6 +13,8 @@ #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN #undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS +#undef CONFIG_KASAN_SW_TAGS_DENSE #define __NO_FORTIFY diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 79151356d5f2..99ff4ae83bf7 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -6,7 +6,7 @@ #include #include -#define KASAN_SHADOW_SCALE_SHIFT 3 +#define KASAN_SHADOW_SCALE_SHIFT CONFIG_KASAN_SHADOW_SCALE_SHIFT /* * Compiler uses shadow offset assuming that addresses start diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index f1fea506e20f..c300274e205a 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1121,6 +1121,8 @@ void __init setup_arch(char **cmdline_p) kasan_init(); + kasan_init_sw_tags(); + /* * Sync back kernel address range. * From patchwork Tue Feb 4 17:33:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 13959510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4F18C02193 for ; Tue, 4 Feb 2025 17:41:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A11B28000C; Tue, 4 Feb 2025 12:41:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 450BD280008; Tue, 4 Feb 2025 12:41:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CA3728000C; Tue, 4 Feb 2025 12:41:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0BA70280008 for ; Tue, 4 Feb 2025 12:41:24 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9A6881A0C50 for ; Tue, 4 Feb 2025 17:41:07 +0000 (UTC) X-FDA: 83082978174.01.F2E9BE4 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf25.hostedemail.com (Postfix) with ESMTP id 7847BA000D for ; Tue, 4 Feb 2025 17:41:04 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=KF+aN2p3; spf=pass (imf25.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738690865; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qK1chVyWc3x+ocpxBgge4/JeZPmwM7T3xKM6CHzx0Ts=; b=Jr8OCtkNW/ltQIlOWMgMz5hK1TyRUIFhDgNx4Wy6EqDYTulXAfDbU6NLIpefmkm0IzQy8h kNRtD0GIpuEx5qfmbYiwMuvTpImg8BGgfJ7Yo85CcFsEECJdEkQLbq9cBdQEa8UiqRLY4/ SpmwMaJb1hR4meRtpts6j5zJjhNJcHA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738690865; a=rsa-sha256; cv=none; b=O3OkCaSPpUXQjZAV475bPR2SiCQlOszLONbQMgkPmbxh/ZhZpt7DGUTo9rEwya01YoTe0A 1cMwar60Wl9VPY7/dxleAIXtbAfrQeaiD2XyUOBhMAXtMs1ASspvTF8GV1Lnli861vHbQX qu2n1MpgUr0Q3HhO2P99UTS0f7PMat8= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=KF+aN2p3; spf=pass (imf25.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690864; x=1770226864; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ht7/TOmkT+iwiNKb8KTAnbXFoYcUy6oM0kb2gCQ20Wc=; b=KF+aN2p3BafUwQF34qoMnP2kCxSBayftKBhQ8H5DA8tToodOUgjeZpEt Ru6nfgKtzfpE1x35LJpTO58+QCRqrlKs02TTc8PP4xrfUw05J8eYm5ynl zlyDoF5kvVzwCkC2f7b38xVDFaCwVC3CC2eie4Tzl7DWPp/Nqrtx9sRvr r72d3plevm4J1q2uEDyO93QeEloFoHVixRihwzJNWULOC63OnXXIxkR7Z 0sh694aVR1A9zoxmOf4vbaP0JhpqPkmZSh4+wFNnZFpKkMfI3Yaluifio nKzTv7VZV7381e1itNk9ZBCwh//ZrP3U2wqtuRyUTxVsPoCMLYfBRbOzR w==; X-CSE-ConnectionGUID: 6udrRXXjThW7ggvP2VFuqw== X-CSE-MsgGUID: 52PacGU2So+wOUkZTWBAOg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931228" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931228" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:41 -0800 X-CSE-ConnectionGUID: UAr5/zW7Rha56MjtNiYK0Q== X-CSE-MsgGUID: 2nPTC7p0TeC/cQLyC4Ywqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147867266" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:29 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 15/15] kasan: Add mititgation and debug modes Date: Tue, 4 Feb 2025 18:33:56 +0100 Message-ID: <450a1fe078b0e07bf2e4f3098c9110c9959c6524.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 7847BA000D X-Stat-Signature: ox5wue9gomx393ww7y5ffmp33zo3kdw4 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1738690864-716018 X-HE-Meta: U2FsdGVkX19nHAvUyb6WSnI2z1IHsBb5YznpH43gyq1riNuj3lt+76nyoJAdc0bl+yjs6/Kfe7OCQUQnJ+uUjucT6eKtplWvUUlWEj8Ktbnhfl9Ht5xif1lRKuBuOGL57KiNHpCAQG9dDZnwLr9Ii6HmTUgsw3/M1yroUWEp3k84FseM8QPbiydQgN7BnIhFyLYr1xHFM6M6JExwTK7J+DJsyoTaluvc8j6jwWl/AezHkQH2icZj56g037cDzk9N2hDHvgb+VtMVOidmTnipzcDNtOjP+EJAjO4IIqRy6HMwMzQ9oRZ8bEfxeahhDZCwahmukPAt3pFETpcYRwOmnJMgMjuluM7If1Jp8IRCI0/BUqZTaQdgKYpSScdudgyLVH9sQURgrJr6VxvM/6+qBCDEM1TMks9VQhHWR3wLIWq6KQempk9V64NZHpTbR/BwS3KuumeS6qQxG3HXEfC1oukAvnFt7QAirkxJXAEIX9fCWvOH8B58thTiR5gsN0Z6QyrfPFUgk14otziZjc+09EVl964hlezdkO+IVksvQasMsVSGxiyEF8AlGmSFD0LIqZSChfsAVDP2Xyk5gbl2WE6idG4p/Dyrj/LIyeNlv95DMKBf3Kv8eeG+dKyYibs/b1cCLPypipGRd/h5ZWo5hfuuQhOht89Zh/ubFmk96KnU1aoqLdZGQAYeTjhG3I01IX1ZTSifXbdHO2y8i23702TcvZxxHFhOvZ84G9FDS1FszhNx1OdtajR0lKzW+YVsJXszI1WBQQ7C7OENXvMMbz5X9fPu8bqzBt/gomcK0iwyerkXWBpT0v0ecFfMJmiNHDAymGzwDfTg2uLqs7U17xFYRn/UGBmsRrG9DxxV7qupODQBarhgOdoq8K8fNF2dF7yNVoVpq/wNeIPP3SW5o7+L59IdLk+AayaN60Zl+4/AWQFUYPJgizLhAWvBhcz3MWYIoP+augVUYFYlxfJ mhsSixqK ievfIO7CRQ3snmid8REjzkP91T+CTXp742UalYIx9jmvUfRgiZOAfA+WOxVnaOLasUDOg+ya9cIJYUHWtsi35MWu2hPP3EXsZ1pTmBsro+0WrRv63vErOaBrCUpkKNMx7jeR6lW3LRZeQ2jzIE+/F7IGIlPHOqFsyH8u9aqq7QSuyZ421NQWYQ10ShhiCOrXSga8v18fZSp0QQO/SCQTzRcAaw18Zp2yQFtUoWk8K4Q+hFcAYRGNBkDROGQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With smaller memory footprint KASAN could be used in production systems. One problem is that saving stacktraces slowes memory allocation substantially - with KASAN enabled up to 90% of time spent on kmalloc() is spent on saving the stacktrace. Add mitigation mode to allow the option for running KASAN focused on performance and security. In mitigation mode disable saving stacktraces and set fault mode to always panic on KASAN error as a security mechanism. Signed-off-by: Maciej Wieczor-Retman --- lib/Kconfig.kasan | 28 ++++++++++++++++++++++++++++ mm/kasan/report.c | 4 ++++ mm/kasan/tags.c | 5 +++++ 3 files changed, 37 insertions(+) diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index d08b4e9bf477..6daa62b40dea 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -244,4 +244,32 @@ config KASAN_SW_TAGS_DENSE ARCH_HAS_KASAN_SW_TAGS_DENSE is needed for this option since the special tag macros need to be properly set for 4-bit wide tags. +choice + prompt "KASAN operation mode" + default KASAN_OPERATION_DEBUG + help + Choose between the mitigation or debug operation modes. + + The first one disables stacktrace saving and enables panic on error. + Faster memory allocation but less information. The second one is the + default where KASAN operates with full functionality. + +config KASAN_OPERATION_DEBUG + bool "Debug operation mode" + depends on KASAN + help + The default mode. Full functionality and all boot parameters + available. + +config KASAN_OPERATION_MITIGATION + bool "Mitigation operation mode" + depends on KASAN + help + Operation mode dedicated at faster operation at the cost of less + information collection. Disables stacktrace saving for faster + allocations and forces panic on KASAN error to mitigate malicious + attacks. + +endchoice + endif # KASAN diff --git a/mm/kasan/report.c b/mm/kasan/report.c index ee9e406b0cdb..ae989d3bd919 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -47,7 +47,11 @@ enum kasan_arg_fault { KASAN_ARG_FAULT_PANIC_ON_WRITE, }; +#ifdef CONFIG_KASAN_OPERATION_MITIGATION +static enum kasan_arg_fault kasan_arg_fault __ro_after_init = KASAN_ARG_FAULT_PANIC; +#else static enum kasan_arg_fault kasan_arg_fault __ro_after_init = KASAN_ARG_FAULT_DEFAULT; +#endif /* kasan.fault=report/panic */ static int __init early_kasan_fault(char *arg) diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index c111d98961ed..2414cddeaaf3 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -78,6 +78,11 @@ early_param("kasan.stack_ring_size", early_kasan_flag_stack_ring_size); void __init kasan_init_tags(void) { + if (IS_ENABLED(CONFIG_KASAN_OPERATION_MITIGATION)) { + static_branch_disable(&kasan_flag_stacktrace); + return; + } + switch (kasan_arg_stacktrace) { case KASAN_ARG_STACKTRACE_DEFAULT: /* Default is specified by kasan_flag_stacktrace definition. */