From patchwork Fri Apr 4 13:14:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99D6AC3601A for ; Fri, 4 Apr 2025 13:15:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A08A1280007; Fri, 4 Apr 2025 09:15:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98BE3280004; Fri, 4 Apr 2025 09:15:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 769C6280007; Fri, 4 Apr 2025 09:15:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 47CFC280004 for ; Fri, 4 Apr 2025 09:15:05 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7A9171C5EBF for ; Fri, 4 Apr 2025 13:15:06 +0000 (UTC) X-FDA: 83296407012.27.359A597 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf12.hostedemail.com (Postfix) with ESMTP id 0E5CF40016 for ; Fri, 4 Apr 2025 13:15:03 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CYxHg1CJ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf12.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772504; a=rsa-sha256; cv=none; b=dNw0LjiatdqyWO3fEVPv0/HR4U6AoMKgb/WsJzW1LoGNOo7X6BEw2K3qKUMNm5hXnXi5ts /rFkVP7Lg5Z+PYTBFZXSwqNTMbUjUta7W0ttliZTeaQOyM6k5SJdV+BmNCQNbmaXol7Yi4 z+D0xaf3siQ96GOwFbKfqU4Gr4gPibY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=CYxHg1CJ; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf12.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772504; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ihdcaXotD4FXvEuiFCQbYKqP3O7bZOZ0FEbpCWYNaAI=; b=GRvunW9fVGlPl6AmUSVKBP/ZT8cRU0fICNJ1yE1taU7UBJTMien69IeqgBZAQ3d6Exr1XJ gms4SkjV39YwZ/MeYD2BGIhXSBKpwvGQY3RVtZy+H+MSfly0KmWld6sNxT9KPpqTt78Sq+ X4rcmmofDYdtD5OWzBVrvvVYFF6xeeY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772504; x=1775308504; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tBO/XjD0zDCynnlPysqAzSK47/nr04nr8LsuOwuwuzk=; b=CYxHg1CJzCSrn9ZwMY9fXEEsfOFKkeZNAC2PWerjtOlfTJHiW85wQMWX Kbbi4T8XPq4zdEHUqmsH9/7VqRLp9XYAklSYVrKqz7ims7v4b583bn4Lb LIKwR867k3hewqp26pY+ti4dVU3Djvt8avfwJX35vTCfUM5jCYBk0hXud 2GAYBvMacF7Kg2woYPJd/GDmUlQxKy3ZoNQyu1DahiyyqIcEygHarg9kX qcWJAwnaCa+D+8JbCIyOQBIv9ki3EwegGa7f2BoU8drxMT4U6Z/xRm7kw iN3obK5AMDQ9sKY2V59d8DT4QOBTazVZ8mq4saEAdBjM/uV02HSu/btnI g==; X-CSE-ConnectionGUID: XctJVuMLTFGb6WoTc7CJtA== X-CSE-MsgGUID: lTgD2j69SrWNcsZF7LrAgQ== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401537" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401537" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:02 -0700 X-CSE-ConnectionGUID: B+Uv8UewR2C2T90dooccjw== X-CSE-MsgGUID: un9CLlJUT4SPzaTveK3NUg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128156919" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:14:47 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 01/14] kasan: sw_tags: Use arithmetic shift for shadow computation Date: Fri, 4 Apr 2025 15:14:05 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0E5CF40016 X-Stat-Signature: qmgecerktmsi7pr5m1qczbn6o1yxmg4e X-Rspam-User: X-HE-Tag: 1743772503-860338 X-HE-Meta: U2FsdGVkX18E0qScRcbQiqZDDxdbvFr/np/6Y5MUB6D0zyZt7cVPvAqwy8KRuKaXS87RTbhfIQPE8kPy6Empei2ErjzJUjKowEbHDtr81Nh5lasKsF04ufo3lb2NixtSMfIBmqdRqZfS+Nzg3h/eFXdY2LbtssEFo18YLkYJqzaMpkyxStmWpx/W6zPcLUE7iumfv8j+xuJtlA22d8NmEmMBeFWctDrXrTU6w4rNEMV6M+C62lXpzicB18nUcYEUhG0I2CF1y7YaB20BU3k4n7d1344BHnplkqJu/qBC88nrKX/pSf0OS/FwxpDac23QkEapsfA18D64CkeMTg3yJUQl+XnDdel2nHUlyunYuzPtrqk94hGRvgafACECO7QtXnhOSz1RcmXGg12KG5nrtrBxmTU+wZvIvSmz3jjMhSmEZR5haOgfCsv73ZIWzThYNh2uX10xG1gRwiR+f80c9YGURB3dZt6gkjevcwf/LtuPL4GttBomZYUy3rTwxhG8wHM49J60j9b8fbFvjeHe3aBEg9BcBCtHw6xbJc/qoG5GTi8PCfqB1DpI36S+nCeiYPHLuZaoxP/Upk+BMZkmT6GMVcaaBEC0rZqAoyY/w5GyJI1Zf6WMFj12EyJ2lGf8cClFxb1ri3K4eWeeUFeZLM/dOtBInAg8r3s7GhrpenKVoj6NUkfOSuS8pVbqmyOHtrdK0j3WenIX7E31WtgsSliNNALzwfyeVeze5BuTUEOVRe3D2ng4+elCzEhW5zleGm8zePXo5nZYywQeaFHdZ2jdvFUWLVW+19R2mah2/yPrnjmjVcKcoosIbu1ugD3/8XC3S1lBOPtsMpNrBS/6tgpW1xAi2waPRBhDs+WVzeEv5rwofHlXwNoIoT1KsYwew20LIxJ1VFbwhp/O+ZHbhqvQDeDZUW6eH5C8sKO+FOERiwDMP31gyuYsjlZP4oXxMAOXq65y34KXwKS4BY8 7kZoT6Uj MEpGBDr+tOWlePuXhvmvEnWeBhp12kjXipyCNXwIDs9AeP0owLcohH6yVbQy8cJynv2cp3EF4jiqX3OgV6NzUy23zVu16NfBB/Xs7dQpKBhx2+vElnRzWzfVJOHEwrgb3X+dH3SHVJkpEAlSeoGTl998b9BmxCfye5l1kH6OQ5d4I9DB2RVRYHqJsqTfDEhnrgfNcYGnF0kcm0Dz6z0OvfC6SaxpSCz1DeEcmYrwjS1500/0zeOer6Z1DDrSW8YfQr0vDL50Q8l6IgKQWqnr09u8UAr/BV6SqtCFUxOsLPzIZzd3p6y4qGeED1YgUFeMVweZwbI5mXMMKsjZOb26BCTaiyEL8jVJWeTUJz6VBj641FZnXT8OHSuzIcKtQQVBemTHd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Samuel Holland Currently, kasan_mem_to_shadow() uses a logical right shift, which turns canonical kernel addresses into non-canonical addresses by clearing the high KASAN_SHADOW_SCALE_SHIFT bits. The value of KASAN_SHADOW_OFFSET is then chosen so that the addition results in a canonical address for the shadow memory. For KASAN_GENERIC, this shift/add combination is ABI with the compiler, because KASAN_SHADOW_OFFSET is used in compiler-generated inline tag checks[1], which must only attempt to dereference canonical addresses. However, for KASAN_SW_TAGS we have some freedom to change the algorithm without breaking the ABI. Because TBI is enabled for kernel addresses, the top bits of shadow memory addresses computed during tag checks are irrelevant, and so likewise are the top bits of KASAN_SHADOW_OFFSET. This is demonstrated by the fact that LLVM uses a logical right shift in the tag check fast path[2] but a sbfx (signed bitfield extract) instruction in the slow path[3] without causing any issues. Using an arithmetic shift in kasan_mem_to_shadow() provides a number of benefits: 1) The memory layout doesn't change but is easier to understand. KASAN_SHADOW_OFFSET becomes a canonical memory address, and the shifted pointer becomes a negative offset, so KASAN_SHADOW_OFFSET == KASAN_SHADOW_END regardless of the shift amount or the size of the virtual address space. 2) KASAN_SHADOW_OFFSET becomes a simpler constant, requiring only one instruction to load instead of two. Since it must be loaded in each function with a tag check, this decreases kernel text size by 0.5%. 3) This shift and the sign extension from kasan_reset_tag() can be combined into a single sbfx instruction. When this same algorithm change is applied to the compiler, it removes an instruction from each inline tag check, further reducing kernel text size by an additional 4.6%. These benefits extend to other architectures as well. On RISC-V, where the baseline ISA does not shifted addition or have an equivalent to the sbfx instruction, loading KASAN_SHADOW_OFFSET is reduced from 3 to 2 instructions, and kasan_mem_to_shadow(kasan_reset_tag(addr)) similarly combines two consecutive right shifts. Make changes to kasan_non_canonical_hook() - specifically the first part that tries to deduce if a faulty address came from kasan_mem_to_shadow(). Link: https://github.com/llvm/llvm-project/blob/llvmorg-20-init/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp#L1316 [1] Link: https://github.com/llvm/llvm-project/blob/llvmorg-20-init/llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp#L895 [2] Link: https://github.com/llvm/llvm-project/blob/llvmorg-20-init/llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp#L669 [3] Signed-off-by: Samuel Holland Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: (Maciej) - Fix scripts/gdb/linux/kasan.py so the new signed mem_to_shadow() is reflected there. - Fix Documentation/arch/arm64/kasan-offsets.sh to take new offsets into account. - Made changes to the kasan_non_canonical_hook() according to upstream discussion. Moved x86 part to separate patch later. Changelog v2: (Maciej) - Correct address range that's checked in kasan_non_canonical_hook(). Adjust the comment inside. - Remove part of comment from arch/arm64/include/asm/memory.h. - Append patch message paragraph about the overflow in kasan_non_canonical_hook(). Documentation/arch/arm64/kasan-offsets.sh | 8 +++-- arch/arm64/Kconfig | 10 +++---- arch/arm64/include/asm/memory.h | 14 ++++++++- arch/arm64/mm/kasan_init.c | 7 +++-- include/linux/kasan.h | 10 +++++-- mm/kasan/report.c | 36 ++++++++++++++++++++--- scripts/gdb/linux/kasan.py | 3 ++ scripts/gdb/linux/mm.py | 5 ++-- 8 files changed, 75 insertions(+), 18 deletions(-) mode change 100644 => 100755 Documentation/arch/arm64/kasan-offsets.sh diff --git a/Documentation/arch/arm64/kasan-offsets.sh b/Documentation/arch/arm64/kasan-offsets.sh old mode 100644 new mode 100755 index 2dc5f9e18039..ce777c7c7804 --- a/Documentation/arch/arm64/kasan-offsets.sh +++ b/Documentation/arch/arm64/kasan-offsets.sh @@ -5,8 +5,12 @@ print_kasan_offset () { printf "%02d\t" $1 - printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \ - - (1 << (64 - 32 - $2)) )) + if [[ $2 -ne 4 ]] then + printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \ + - (1 << (64 - 32 - $2)) )) + else + printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) )) + fi } echo KASAN_SHADOW_SCALE_SHIFT = 3 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 62dc903ecc7f..10e215d6e435 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -428,11 +428,11 @@ config KASAN_SHADOW_OFFSET default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS default 0xdfffffc000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS default 0xdffffff800000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS - default 0xefff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS - default 0xefffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS - default 0xeffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS - default 0xefffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS - default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS + default 0xffff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS + default 0xffffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS + default 0xfffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS + default 0xffffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS + default 0xfffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS default 0xffffffffffffffff config UNWIND_TABLES diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 717829df294e..e71cdf036287 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -89,7 +89,15 @@ * * KASAN_SHADOW_END is defined first as the shadow address that corresponds to * the upper bound of possible virtual kernel memory addresses UL(1) << 64 - * according to the mapping formula. + * according to the mapping formula. For Generic KASAN, the address in the + * mapping formula is treated as unsigned (part of the compiler's ABI), so the + * end of the shadow memory region is at a large positive offset from + * KASAN_SHADOW_OFFSET. For Software Tag-Based KASAN, the address in the + * formula is treated as signed. Since all kernel addresses are negative, they + * map to shadow memory below KASAN_SHADOW_OFFSET, making KASAN_SHADOW_OFFSET + * itself the end of the shadow memory region. (User pointers are positive and + * would map to shadow memory above KASAN_SHADOW_OFFSET, but shadow memory is + * not allocated for them.) * * KASAN_SHADOW_START is defined second based on KASAN_SHADOW_END. The shadow * memory start must map to the lowest possible kernel virtual memory address @@ -100,7 +108,11 @@ */ #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) + KASAN_SHADOW_OFFSET) +#else +#define KASAN_SHADOW_END KASAN_SHADOW_OFFSET +#endif #define _KASAN_SHADOW_START(va) (KASAN_SHADOW_END - (UL(1) << ((va) - KASAN_SHADOW_SCALE_SHIFT))) #define KASAN_SHADOW_START _KASAN_SHADOW_START(vabits_actual) #define PAGE_END KASAN_SHADOW_START diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index d541ce45daeb..dc2de12c4f26 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -198,8 +198,11 @@ static bool __init root_level_aligned(u64 addr) /* The early shadow maps everything to a single page of zeroes */ asmlinkage void __init kasan_early_init(void) { - BUILD_BUG_ON(KASAN_SHADOW_OFFSET != - KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + BUILD_BUG_ON(KASAN_SHADOW_OFFSET != + KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); + else + BUILD_BUG_ON(KASAN_SHADOW_OFFSET != KASAN_SHADOW_END); BUILD_BUG_ON(!IS_ALIGNED(_KASAN_SHADOW_START(VA_BITS), SHADOW_ALIGN)); BUILD_BUG_ON(!IS_ALIGNED(_KASAN_SHADOW_START(VA_BITS_MIN), SHADOW_ALIGN)); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, SHADOW_ALIGN)); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 890011071f2b..b396feca714f 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -61,8 +61,14 @@ int kasan_populate_early_shadow(const void *shadow_start, #ifndef kasan_mem_to_shadow static inline void *kasan_mem_to_shadow(const void *addr) { - return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) - + KASAN_SHADOW_OFFSET; + void *scaled; + + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + scaled = (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT); + else + scaled = (void *)((long)addr >> KASAN_SHADOW_SCALE_SHIFT); + + return KASAN_SHADOW_OFFSET + scaled; } #endif diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 8357e1a33699..f24f11cc644a 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -681,11 +681,39 @@ void kasan_non_canonical_hook(unsigned long addr) const char *bug_type; /* - * All addresses that came as a result of the memory-to-shadow mapping - * (even for bogus pointers) must be >= KASAN_SHADOW_OFFSET. + * For Generic KASAN, kasan_mem_to_shadow() uses the logical right shift + * and never overflows with the chosen KASAN_SHADOW_OFFSET values (on + * both x86 and arm64). Thus, the possible shadow addresses (even for + * bogus pointers) belong to a single contiguous region that is the + * result of kasan_mem_to_shadow() applied to the whole address space. */ - if (addr < KASAN_SHADOW_OFFSET) - return; + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) { + if (addr < (u64)kasan_mem_to_shadow((void *)(0UL)) || + addr > (u64)kasan_mem_to_shadow((void *)(~0UL))) + return; + } + + /* + * For Software Tag-Based KASAN, kasan_mem_to_shadow() uses the + * arithmetic shift. Normally, this would make checking for a possible + * shadow address complicated, as the shadow address computation + * operation would overflow only for some memory addresses. However, due + * to the chosen KASAN_SHADOW_OFFSET values and the fact the + * kasan_mem_to_shadow() only operates on pointers with the tag reset, + * the overflow always happens. + * + * For arm64, the top byte of the pointer gets reset to 0xFF. Thus, the + * possible shadow addresses belong to a region that is the result of + * kasan_mem_to_shadow() applied to the memory range + * [0xFF000000000000, 0xFFFFFFFFFFFFFFFF]. Despite the overflow, the + * resulting possible shadow region is contiguous, as the overflow + * happens for both 0xFF000000000000 and 0xFFFFFFFFFFFFFFFF. + */ + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) && IS_ENABLED(CONFIG_ARM64)) { + if (addr < (u64)kasan_mem_to_shadow((void *)(0xFFUL << 56)) || + addr > (u64)kasan_mem_to_shadow((void *)(~0UL))) + return; + } orig_addr = (unsigned long)kasan_shadow_to_mem((void *)addr); diff --git a/scripts/gdb/linux/kasan.py b/scripts/gdb/linux/kasan.py index 56730b3fde0b..fca39968d308 100644 --- a/scripts/gdb/linux/kasan.py +++ b/scripts/gdb/linux/kasan.py @@ -8,6 +8,7 @@ import gdb from linux import constants, mm +from ctypes import c_int64 as s64 def help(): t = """Usage: lx-kasan_mem_to_shadow [Hex memory addr] @@ -39,6 +40,8 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): + if constants.CONFIG_KASAN_SW_TAGS: + addr = s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.KASAN_SHADOW_OFFSET KasanMemToShadow() diff --git a/scripts/gdb/linux/mm.py b/scripts/gdb/linux/mm.py index 7571aebbe650..2e63f3dedd53 100644 --- a/scripts/gdb/linux/mm.py +++ b/scripts/gdb/linux/mm.py @@ -110,12 +110,13 @@ class aarch64_page_ops(): self.KERNEL_END = gdb.parse_and_eval("_end") if constants.LX_CONFIG_KASAN_GENERIC or constants.LX_CONFIG_KASAN_SW_TAGS: + self.KASAN_SHADOW_OFFSET = constants.LX_CONFIG_KASAN_SHADOW_OFFSET if constants.LX_CONFIG_KASAN_GENERIC: self.KASAN_SHADOW_SCALE_SHIFT = 3 + self.KASAN_SHADOW_END = (1 << (64 - self.KASAN_SHADOW_SCALE_SHIFT)) + self.KASAN_SHADOW_OFFSET else: self.KASAN_SHADOW_SCALE_SHIFT = 4 - self.KASAN_SHADOW_OFFSET = constants.LX_CONFIG_KASAN_SHADOW_OFFSET - self.KASAN_SHADOW_END = (1 << (64 - self.KASAN_SHADOW_SCALE_SHIFT)) + self.KASAN_SHADOW_OFFSET + self.KASAN_SHADOW_END = self.KASAN_SHADOW_OFFSET self.PAGE_END = self.KASAN_SHADOW_END - (1 << (self.vabits_actual - self.KASAN_SHADOW_SCALE_SHIFT)) else: self.PAGE_END = self._PAGE_END(self.VA_BITS_MIN) From patchwork Fri Apr 4 13:14:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DA8CC36014 for ; Fri, 4 Apr 2025 13:15:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F246280008; Fri, 4 Apr 2025 09:15:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07B66280004; Fri, 4 Apr 2025 09:15:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E100E280008; Fri, 4 Apr 2025 09:15:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B57F5280004 for ; Fri, 4 Apr 2025 09:15:29 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DBF5DA8712 for ; Fri, 4 Apr 2025 13:15:30 +0000 (UTC) X-FDA: 83296408020.29.970C034 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf02.hostedemail.com (Postfix) with ESMTP id C1DD680014 for ; Fri, 4 Apr 2025 13:15:28 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aaXtFhL1; spf=pass (imf02.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772529; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eJOHrb/uxZmkYwfE/IAvy2u0mkBJZjoP2KYSbzFPajw=; b=QbN+dRSGBuM/UtGIXcH31JHVcf8SzPihK8xmrQyroGcJUA5Ib/qdk0ShuOLDyPzEo+JH6p WbmBNqJfXWtOTJ87Kyq06mYlz4lSdn14FKnDktLy8lJqSny+wDGF3UVV93Z/OgBteOaKLs VYpK8hS4uF1+f3ww/MfeUsYlscMtPaU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aaXtFhL1; spf=pass (imf02.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772529; a=rsa-sha256; cv=none; b=t6qx9D+AFsZURsCkh/Z5d/zvel/DDm2t+zjqW9TuRqRx8Xsd+ii0SfqroDII7IiryI70bK zbf+RzckU/v4rmpeW3oHqi7ArTBT7JVendQn2llouEW7M01mMGUClqIXftaVYYal0uC1KN 1jKDYosb2pnC9bPV1eCvMaJ1mXLWg5w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772529; x=1775308529; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f3FgrRcCKHKaOTPwP4m+j1DajRC6X+0zKUQ4YI0yMSo=; b=aaXtFhL1w1AynugP7X03BLh2KcIJkalcAidfVHciDtr5hjpCcAzoRH7j QDu5K3XD2K+dd6t0QVSPqu87OIufxWKAdVLIVFfrRhMMBCHc5f6KTyaut 3gtZco4WyWN2sXNMAgpVFVKFKzp1XMDvd+DD2eHR4htsraGJpS9a+IjhQ BV0Zdi27hQ+zXnNob3nBId+8aFzjYvh1SPkvqviMHbb0lCrTw2xa5IufZ ElH6KaHSO/Znp8Jue4R5mTllscvfXllUebOrlHwXJbl8/ORBszPon0Ljr LzMCFQtsDavHXQqUxA4hN5oHdu0Jt837l59hZJxK5dDUmX7izlcYQbkyV g==; X-CSE-ConnectionGUID: bgKks/zwRb2k9rVuehDJJA== X-CSE-MsgGUID: fi8grp1XQMeCOHeV7SYkFg== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401605" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401605" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:17 -0700 X-CSE-ConnectionGUID: 6W/+ARP9QJe63qKWe4pGFQ== X-CSE-MsgGUID: mcoWCZX1T/6lCeGBRJQLmA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128156968" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:02 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 02/14] kasan: sw_tags: Support tag widths less than 8 bits Date: Fri, 4 Apr 2025 15:14:06 +0200 Message-ID: <6db4abef7daff8cdb07ad1cfd4fdd30a4893d5c3.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C1DD680014 X-Stat-Signature: 5ntau6a5uexae679d9q1ojmap58gxi3n X-HE-Tag: 1743772528-571554 X-HE-Meta: U2FsdGVkX18FXv/K8dtj5YDkUIrakdYkJmebiZrjOWJwAl8aa/9GZ/Tv+5SUQ8IDaF8rcDey0esQ+UURWoJd4UnIukTyBddX4Xt2gNKvW88UCDvC7trsiQ4Fe8Y5pg1ZVr4JP6+PdCGXlnuNhhKbQYTa3DRftt/g3ombeTrluJjOQGwuCYZLcbTXxq8WV+MKq+rFIuOd6DL1BuiV1dkuv7wfXWQz1IzokxTYtQ4O1/BpHvJPN7sYIMWPP0C5Z6fkIvZCBdyDXIZxsmQqlvsZzg/sQvEmzo2ePAuDqeY8JFeXavE99dDjAPIIv/8f5+/zv454OjfYAaxX9ZFVEIn6M/XJM9S06bzSQz0hb9+DN8xNUNkyZh05gErvydmK6zY7XULRQ3+7ceuQ7PS1m2g9zUYewWzz1jQlyUhZJ0KOfpcgGUZqjX8p5CeNEIv851edHB43R5OcZ7nMlGL8OsxzLnMg69LOp+vtEJ7cy3fhi2AL5z/BJWZGr1nhXnsyvUX17wBLJImYrmOoqkEqhTEbblWGjxg/eb+G25y0LjnCiXsvMW9napq6FDCTIkTpg+zDOeOr5xRRo+68l+H/X+NOtoHDThDhXob1VIaH+rZtCuS5wi06rb2hMzRgFh0xA89U0+FZaukXeVxMZO/Ce8B9SAi65pMyyfMou1fYIcdGIZBxf852icDP88z2NWITe9uLjS/YyMjHNNRp/gGCKbzSdqlwdJgyzrXjR3ucnuU3tjk2AujDbOP8ms1+PG5D9oEazJ38NrgPazVn0iLqFMW9h1DuKfswVG9l0tsJvc/bNH5b2XURApniMJojCRUfVaQHN5vwn5sIP9uDz0uJi8/AKGiCHFUhva82nbZJswcC9msmhPkCNIUYIv0ahIZ7qK1oRb3nzbrkzvSuZZXu3hi9T4wxJ8im/XC9770IbcSYUesAZw2iJJ964opOAZeacgPobzHYD9W1Ulgixy26tV2 eNtfMaZR JsEtGLILEDxW6CBPn9NAtq+tQvdS1S+6coTGql4JXvNzC2d5qx8Xri9KdrAQCSkpVQjGVoCW6Ux7tjuYJodgyULuKF3jHwVEi8hp5gOBlzG1XZBIf2XDY2BLaZZgYL80zFl851Jmq92VKSPd2tRTtSGHhJTbutQnXAez/SkNcq8XDaQB/fQngsLL8GJoPMsce18TaACOVIsHvHmI6vCw7ct9lmujvSS8/BkK79Glr/FxGyhOd9++2i1qWhxr9y/P+56C2pVikO4SQcMLoKZtYBiip2CCy5gMwPSeGOV1loxeSfdZ3khhDPrk5To9vjnkTOLb8COngqL+G0Q5o1AIIY6qaE4hn2X9wkVExJA7oFV0i6T/80ymKPqWgmLCZVjBx5B90 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Samuel Holland Allow architectures to override KASAN_TAG_KERNEL in asm/kasan.h. This is needed on x86 and RISC-V, which support different tag widths. For consistency, move the arm64 MTE definition of KASAN_TAG_MIN to asm/kasan.h, since it is also architecture-dependent. Reviewed-by: Andrey Konovalov Signed-off-by: Samuel Holland Signed-off-by: Maciej Wieczor-Retman --- arch/arm64/include/asm/kasan.h | 6 ++++-- arch/arm64/include/asm/uaccess.h | 1 + include/linux/kasan-tags.h | 13 ++++++++----- 3 files changed, 13 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h index e1b57c13f8a4..4ab419df8b93 100644 --- a/arch/arm64/include/asm/kasan.h +++ b/arch/arm64/include/asm/kasan.h @@ -6,8 +6,10 @@ #include #include -#include -#include + +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0 /* minimum value for random tags */ +#endif #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 5b91803201ef..f890dadc7b4e 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include diff --git a/include/linux/kasan-tags.h b/include/linux/kasan-tags.h index 4f85f562512c..e07c896f95d3 100644 --- a/include/linux/kasan-tags.h +++ b/include/linux/kasan-tags.h @@ -2,13 +2,16 @@ #ifndef _LINUX_KASAN_TAGS_H #define _LINUX_KASAN_TAGS_H +#include + +#ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ -#define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ -#define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#endif + +#define KASAN_TAG_INVALID (KASAN_TAG_KERNEL - 1) /* inaccessible memory tag */ +#define KASAN_TAG_MAX (KASAN_TAG_KERNEL - 2) /* maximum value for random tags */ -#ifdef CONFIG_KASAN_HW_TAGS -#define KASAN_TAG_MIN 0xF0 /* minimum value for random tags */ -#else +#ifndef KASAN_TAG_MIN #define KASAN_TAG_MIN 0x00 /* minimum value for random tags */ #endif From patchwork Fri Apr 4 13:14:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DC2DC36010 for ; Fri, 4 Apr 2025 13:15:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C02D280009; Fri, 4 Apr 2025 09:15:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 14A81280004; Fri, 4 Apr 2025 09:15:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBAC5280009; Fri, 4 Apr 2025 09:15:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C5559280004 for ; Fri, 4 Apr 2025 09:15:33 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 13BFB8013F for ; Fri, 4 Apr 2025 13:15:35 +0000 (UTC) X-FDA: 83296408230.07.1D12212 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf02.hostedemail.com (Postfix) with ESMTP id BB5D78000F for ; Fri, 4 Apr 2025 13:15:32 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=b+Wmrqc3; spf=pass (imf02.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772533; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RNRm/NWTNCpFCplFUFXOJkHxOM9uo6rxFDglg0b3zbs=; b=rp5KcJd6/CB9+lu3WCzQdj3X9MAsBzytGfSvvufGG93zBRvjbxmHIv5A2uGM8z38ZEcHbW bYmU8kfWs+ZKCC43Ufv+hnJGl/M/ZQaVjYUd8UC12+HB4IRTvlvDOBBP3/ZVR5DjBDTWEa JVgpsG68TMTN1klmUvaD70n52d6C7fo= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=b+Wmrqc3; spf=pass (imf02.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772533; a=rsa-sha256; cv=none; b=22NHHGYO28C5HUkmIgJo99qUCffjQnIw9oVwBWlahVl4zZs/P3RYtmvEGsn0Hdxg20tvWa F+QV5aGAvbeJlNQZQO4N8P6+dlVaGzEAoamYn/wuZa+fvXoHfyGOX+QKhblrc7htF9z80F Ga+n2tzRpNLiccCCKsmgFhFwmTDW5g4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772533; x=1775308533; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AJLSjKGgb59TRhfxq8kMEvmnCKnMy4310E2Hut68m+c=; b=b+Wmrqc3adqt7PJ8e++LpYm4q/BB+Dsl/qsn81I1zeiXVPvmYKeP2X/K Db3sVDsRQJ/y3zhgLGtAib8k1sAYAI2I4C8shzXLUIm0u8hyENH6mrPFE u04AbuxZH/yPnZE+di2C0z4KD5WAlC6+cv8SUTAexAdYH/sIprAe/8reh YFDcyssH9QwpahIQUB1BPTLMiYhX4tZLNYyZE0oqsLYGxs2W7XCxYVZmp GWATnMHoOFS0cr+tTfpVw2jjHB3sRXb/M261/yS0EA2pziWjQPNsEBYRW +EgkwvxQEv2L2Ly2ryUoUZvwf9CR7zUmf3RAlBKelhTMJpZFLlbvSTn7x A==; X-CSE-ConnectionGUID: Yqp4qr+xRG6mfCZ9axKX3A== X-CSE-MsgGUID: w0tZfHe+SCyN/LJX9HzwlA== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401638" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401638" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:32 -0700 X-CSE-ConnectionGUID: tH8/P6IgSuuHLhHzruVqJg== X-CSE-MsgGUID: 0efeALDRQGCxEo3GUCuhZQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157007" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:17 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 03/14] x86: Add arch specific kasan functions Date: Fri, 4 Apr 2025 15:14:07 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: BB5D78000F X-Stat-Signature: 38m9cupfz7jn3ya38ro7nzakq684fs1f X-HE-Tag: 1743772532-869474 X-HE-Meta: U2FsdGVkX18m8UeICtjF4fUwQXY0Q/QvJZxwcd1IqcTT+crnMWnIvYVIzvG2vccQ9IiENEZMY9enUrm45WKoSzUht56vWdfY1CETZjktj51dzKSA4O6h9iEL2klWm/T4Up1MOfOlLldoP9SvTknHJwm/Pm7IMr0FjBGexA6VG8rfBPJcw663n5Rpzwdm+eh7ZCUX5PBvi0EwWQwJ3yrHAZHihoZKvcIRTCBP3Ris1591EMHtSgKiDDVt3xTdmOXOsOzKk1DGl7//r8G6awcnXhPYPFSCXKEkfuQEN4bTBuMNuYVtye2UpVDY71a91YohpNp42fd11SDs61h8fkZLs8nOSd7HVMcZX4IPoC+pC8Vk5YCGLqXz6OQlN9kJuSV7MyJBnHK9UUH01E/IpoEpK9nOyrmSFayenYSs6kobsoPrxthWV9fJ2QPJrOJvCUIiYsfDXh8etee+rT7sX89+hIWdbZsJfgIhkKUhXVhyG/EINj9DMveXyim3Ms1Y6X+/oUJP96CV5eNLdFpx3tdSj5hqgvpza496P7eOIfvkXXNGZi0tgHzND97p3FhVVBtKQkY40XR9Jz+A0JFA3+E538z9L6d8PcWi4Z0vIaNke+3F7o8agod4M3n3+rpWqP32QGT6l5vNV8UFGNWDp/3KSgxKho+NeMrq7AsYKuCsPk3RD5mf9GI3JiY14HQ4H61iWLl7sQFwZfjet/uJgt0emh9DToiwVpvuPV1nfMm8vipgbU0RXqQUbwLXz6OyHXxo863id+zixZrFaPWG1h5hrMHc56oZx0KALcmTwv+72Ye9zshgrYatmSnuOQR3MMuitJrRj47hN/p9Q86KS3Oa8mn69c8ApTkhYHxdbA7o6YRz/CEOBQMQGX2QgbWgbV7IqwqL0ih9v8M2/cXJ5Xoo1UTQn6aUr+k8rX0g7JlAdWlDmBhiTvu0AKJE19OcCLIJpfJvSG7sRrie4D3GJzc EkSygtfN t/GOUOD053VsWwUTRkuNmgRstaNIDJzyzDYcsPRf/HNTV0r+CAwYDql/FkPRM7fxIOLF3Nmx+4TuR/cXrj9alWlKHc048xr7lF+we/kJ6SPUYq9E+/7WZ6lKN+pxZ05sb5bnjnPW6GxoS25+onMbzMHeCbV4EYZyDgMzQ2A+MZyBfDyTbFjEwIUPdYjKZd000dYW4cVF/hAmZtFIOSEDzoc/y1Kq/XNUnRq6zoOuWSocKjTLN+nRcEZF+iO2tuCusuYat8077THTD9ajXxerRjeUSL4cmMCUmWHH8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: KASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - mainly to set and retrieve tags from the top bits of a pointer. Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Reorder functions so that __tag_*() etc are above the arch_kasan_*() ones. - Remove CONFIG_KASAN condition from __tag_set() arch/x86/include/asm/kasan.h | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index d7e33c7f096b..212218622963 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -3,6 +3,8 @@ #define _ASM_X86_KASAN_H #include +#include +#include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_SCALE_SHIFT 3 @@ -24,8 +26,33 @@ KASAN_SHADOW_SCALE_SHIFT))) #ifndef __ASSEMBLER__ +#include +#include +#include + +#ifdef CONFIG_KASAN_SW_TAGS + +#define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted(tag) 0UL +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ + +static inline const void *__tag_set(const void *addr, u8 tag) +{ + u64 __addr = (u64)addr & ~__tag_shifted(KASAN_TAG_KERNEL); + return (const void *)(__addr | __tag_shifted(tag)); +} + +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) #ifdef CONFIG_KASAN + void __init kasan_early_init(void); void __init kasan_init(void); void __init kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid); @@ -34,8 +61,9 @@ static inline void kasan_early_init(void) { } static inline void kasan_init(void) { } static inline void kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid) { } -#endif -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLER__ */ #endif From patchwork Fri Apr 4 13:14:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4804BC36014 for ; Fri, 4 Apr 2025 13:15:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56A4928000A; Fri, 4 Apr 2025 09:15:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F3C9280004; Fri, 4 Apr 2025 09:15:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36E5128000A; Fri, 4 Apr 2025 09:15:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1C3A6280004 for ; Fri, 4 Apr 2025 09:15:50 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 44914120161 for ; Fri, 4 Apr 2025 13:15:51 +0000 (UTC) X-FDA: 83296408902.25.C439D1F Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf06.hostedemail.com (Postfix) with ESMTP id 0D56E18000C for ; Fri, 4 Apr 2025 13:15:48 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GmbHFPxc; spf=pass (imf06.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772549; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1M+o1cDr/cejiJebPvpehZ/hYd9tAcNgHmEsRZx9DNY=; b=esooHGhboRPydvm+f434zShPUJj2iqtj0l9zD0f9WIT9yw+ZxmHlysVt3EuMMiVvVcmvIO NjmNq5zt60wSZZLNTczgw9QM3LJXkwl2b3yL2f47S6U/HSiG9n1PFBCttF8m0msJJtLo+p 4yx35CfEWHzI6cPn0czSMCc16n3s1vs= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GmbHFPxc; spf=pass (imf06.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772549; a=rsa-sha256; cv=none; b=g3BBJz4doHB3Jc/xsgWPkBeegQdSO1isvuLSrFe3b18gLpFG+v9tyx3acHFklFkBrU3kEi Bt55BySt65gK70UGFo57a79ZMWCOjCCH1Qc0er5RKyX2l7GLJYBjxmt+M4BuvfQjy5XPnx +rzPoZH5EbDVOH2sgF4E+z8DU5p/reI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772549; x=1775308549; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=D4i7JF8uDEtWP2nuK8uzBgD8AmJabmAUNQ+XaZEoObg=; b=GmbHFPxc7JFhak6WdGKtDu//ibWAJ780IaJDmfY6hLYh8ATa5pd6t3dm dvyDVPeYWdLxbN64XZH6K1+MeeU10y2CfUReOkTC0cjYE2VSIUCLUco83 fzgDrbNADsC0Jpo+91VCVxC0xLbBA3tSto5ABt0AId0Alebngrgs/8b3a udhGJZlkEIml2PTlsnlRy0ougOwCEuIKGWJpSUrJAusLpcwlEwgkHfxcz SWo1iryJYfGPG2gxtR2Jd0wU6mS4D4IVKO6+HsWcrRBvm9q23DB1nlPeq e5XJPHaXESgwU92ZO65glJljAmPzPe2lOaazn/Y96gcDmqfrXH8Ob+uiG A==; X-CSE-ConnectionGUID: VCmfLO5XRaOoWVNA6tW2Zw== X-CSE-MsgGUID: r0jzIGpCQ5+Sa2HHMXU5BQ== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401682" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401682" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:47 -0700 X-CSE-ConnectionGUID: pzanETZmQhSdtpOzW63ogA== X-CSE-MsgGUID: Fj7gxPnqRE6y35rxD0gliw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157039" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:32 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 04/14] kasan: arm64: x86: Make special tags arch specific Date: Fri, 4 Apr 2025 15:14:08 +0200 Message-ID: <716de282b80fe47895ebc876885e31e344c676cc.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 0D56E18000C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: ebxyd6tqadrien5pwew7x4df1jxb8mtn X-HE-Tag: 1743772548-924587 X-HE-Meta: U2FsdGVkX19XeasqWyTLGPeMgPvPxmy0zxUULfqPU7ViqkpKpM69RGqiWRhCh1X2OIqS7pYPnjNE1M+LKQqe7F9cJ4bNjFBtcKwprsXlTbgWbuNh7wccyvNgE941wM3xTPSwsIlA/M813pX8ret2hnhlR5MW7yjPxcw2Hxsw/kBfkNAAFq8ny8uBSsduNx6GozlkC/yULYNCoynU4ItRL9kUJfsxorcdobyB+pIdil+v3dx/IeBjSa+CEXROusfqEZWOSaJeruG4CdDv/DTHd5U1Mp0l5HHWA13EXj3g1x7k1odnqeSubXj23YDpgZdgfpTLeHmIOIJZsLOCAVdA6c6kzc2irIOwOuUcVp0PA20jXW+HDc8CizjXl55PL+d2kfxGtHb/CyuHe4LKCD3Kts6UdejU3P/u8t/VIj375dBG0yIJWNMJ8WpfPNGRq1DVni2kGiLFvSWH3Q1na3MDEdmgGfmWDKd2uT6e+1TpjwQvVTXZ9M41P5Lp1q4fnR7Ybmbic1LWlGsYlzKwK0InNVxGyLMdC5lcd9DnL+sEtF3iNzE+20zRCEPPUh3r6g4TqoAM50KBEYrOtFnbRnbHcEVu7Ki+Mb3XJgdkslZWBluFEdIMkhT0kOFc/A6Eac1+W9A6FBRzZSdGb3nHi79UD/aCJ0T8OxnkvM6lHd4o8dHjvlnRPnTbXm8RAIvLLpO2A+7LeNuGoGsGJ0HXyj/+/JdGCUBAEiLPPrGnxyNg4ewxoR60v5+MBrejWX6YsSX3HZvDDuy2zI1t7lJK86qxnH54OhNiRwscU5/sOnwZAxrxxd9SxrDZtQXalMmEw28RugGzX98sW1/g7FpukLEGXZ0DlWnwEia2eSnqo2/WS0PnVxuP+xg2yXL8IpWlR8O3QDNNIE2clcKs9/slUegfPFXOGBTimd3TcR5JUJRgHx19rC7zvOrfB2CnA8roZKI9WqnZ2RVdCLxqUA6p3K2 zs0VL0Ee ty8z7RcrVH93gbWv7VBUWpeUlXoW34vP7iKxJBvj5UTCSp2FWP304yhwJEgW/zYRICOkFelLSewBQxeIB9sh41sqNjChKFUiImWF6tXTsiGf62yZ0nq4P90nz8joISMvnMMY+jncJLTulQyjvJlPaD4GxGTdjp3zIuPTHgmN12qOe/qyyaqld6smgJFP+rVidK3YrgEPv2Z2Q2xNMF0bkYmtSk6nin66+Q/hIxTvNuF4TGM1xPo6J7D2KFtLblKOAnVFXFblt/nF5ceP8y8UQPpDfmKqaNWtubfPHnLFAYPWF+/927BaQzZ4coM83w1MnjIHLQmifKfowoN4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: KASAN's tag-based mode defines multiple special tag values. They're reserved for: - Native kernel value. On arm64 it's 0xFF and it causes an early return in the tag checking function. - Invalid value. 0xFE marks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory. - Max value. 0xFD is the highest value that can be randomly generated for a new tag. Metadata macro is also defined: - Tag width equal to 8. Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly. Make native kernel tag arch specific for x86 and arm64. Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Remove risc-v from the patch. MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 9 +++++++++ arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 8 +++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/page-flags-layout.h | 7 +------ 7 files changed, 33 insertions(+), 12 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.h diff --git a/MAINTAINERS b/MAINTAINERS index d5dfb9186962..e6c0a6fff9f9 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12728,7 +12728,7 @@ L: kasan-dev@googlegroups.com S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasan diff --git a/arch/arm64/include/asm/kasan-tags.h b/arch/arm64/include/asm/kasan-tags.h new file mode 100644 index 000000000000..8cb12ebae57f --- /dev/null +++ b/arch/arm64/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/arch/x86/include/asm/kasan-tags.h b/arch/x86/include/asm/kasan-tags.h new file mode 100644 index 000000000000..68ba385bc75c --- /dev/null +++ b/arch/x86/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/include/linux/kasan-tags.h b/include/linux/kasan-tags.h index e07c896f95d3..ad5c11950233 100644 --- a/include/linux/kasan-tags.h +++ b/include/linux/kasan-tags.h @@ -2,7 +2,13 @@ #ifndef _LINUX_KASAN_TAGS_H #define _LINUX_KASAN_TAGS_H -#include +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +#include +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif #ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index b396feca714f..54481f8c30c5 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -40,7 +40,9 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t; #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT +#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0 #endif diff --git a/include/linux/mm.h b/include/linux/mm.h index beba5ba0fd97..610f6af6daf4 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1815,7 +1815,7 @@ static inline u8 page_kasan_tag(const struct page *page) if (kasan_enabled()) { tag = (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; - tag ^= 0xff; + tag ^= KASAN_TAG_KERNEL; } return tag; @@ -1828,7 +1828,7 @@ static inline void page_kasan_tag_set(struct page *page, u8 tag) if (!kasan_enabled()) return; - tag ^= 0xff; + tag ^= KASAN_TAG_KERNEL; old_flags = READ_ONCE(page->flags); do { flags = old_flags; @@ -1847,7 +1847,7 @@ static inline void page_kasan_tag_reset(struct page *page) static inline u8 page_kasan_tag(const struct page *page) { - return 0xff; + return KASAN_TAG_KERNEL; } static inline void page_kasan_tag_set(struct page *page, u8 tag) { } diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 4f5c9e979bb9..b2cc4cb870e0 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -3,6 +3,7 @@ #define PAGE_FLAGS_LAYOUT_H #include +#include #include /* @@ -72,12 +73,6 @@ #define NODE_NOT_IN_PAGE_FLAGS 1 #endif -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -#define KASAN_TAG_WIDTH 8 -#else -#define KASAN_TAG_WIDTH 0 -#endif - #ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8 #define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1) From patchwork Fri Apr 4 13:14:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F27BFC36010 for ; Fri, 4 Apr 2025 13:16:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F063428000B; Fri, 4 Apr 2025 09:16:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EB68D280004; Fri, 4 Apr 2025 09:16:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2F4728000B; Fri, 4 Apr 2025 09:16:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B4ABD280004 for ; Fri, 4 Apr 2025 09:16:04 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EEF55A84D9 for ; Fri, 4 Apr 2025 13:16:05 +0000 (UTC) X-FDA: 83296409490.01.0C3DAFA Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf21.hostedemail.com (Postfix) with ESMTP id C241E1C0019 for ; Fri, 4 Apr 2025 13:16:03 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DKVGvWxT; spf=pass (imf21.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BzPu5bL1P82hpoNCxPuVzaAFLr4FB1kFlHIDeW2Ujs8=; b=Rgb0dHFCylrMZJTjATOnVf/xrijMeR7xiTssUbfRE3Z8nPQk4/NxYEdKKDySN64D5AU7wf cDi6Fwf241UrMQ38D3R6BvPtAbzPcy2cgHZEW+EZnqm3aZ3JbvgX0cbmTmw/ly7WUz0jzU UyNlxYkAZbsK2yx3ZIzn2DzqLMxaenw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=DKVGvWxT; spf=pass (imf21.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772564; a=rsa-sha256; cv=none; b=l/aV7TCVfQZOkzwGF2vEr3BF/3v/0eCoZ9PIWB1JXnIXazZ7t2Rex3xdifj8jsZwURxSuk PvPAzP5kOPhKtZGjRKuonH2lCPY1sF6rY+bwSzvPaAvvRXBbmus+gTki8uW4QEfALIAsNY F4wchXfC8CmgRdpPGM8Chay3qcHQn+I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772564; x=1775308564; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ikWUalPb1GKrXc7zL0s3gnRneZOpXlSFYbJqx1DBgFU=; b=DKVGvWxTLrNus89J3kKj2YWU8qc/hWr2mw6yADf8EbBbdd/zrM0LJHt+ bY9L2f9MvuiXh3iLlAxSOY14LYIzW5lniuGj6kiUoG86KO5tFxLTnL5cN qoutTJP7mpJCJz7aE19esgEmXRnN9M39MknZO1Y2pJ6W5d0ZUxVXRB4KC PpBeRW1ytsm+15VNn8cb3MHCG7Nixb+3kXOUNalSjiqy1WrKqc4VpBkmK s1Kz2naRjwJwcbdazflddIqVnxeda3imthTXlBGbCAvAYnmn4M8E5UygY 9f4MuTIKri9VUdJPg2sflDdsXLxRd6R8p8fxtKMuDHhQ7kkSqmaK1eyl0 Q==; X-CSE-ConnectionGUID: WmLqjwiKTd6YrK8ntyvMbQ== X-CSE-MsgGUID: b9qJm0erT2uH9dpvdwYc3g== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401741" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401741" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:03 -0700 X-CSE-ConnectionGUID: /v4yemxXRNCf9yxykbmk+A== X-CSE-MsgGUID: DvwOzQD2QMSbRSno4tJp0A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157141" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:15:47 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 05/14] x86: Reset tag for virtual to physical address conversions Date: Fri, 4 Apr 2025 15:14:09 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: C241E1C0019 X-Stat-Signature: s5bpqt8bog3myh4mrgop97jjw8eyom1a X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1743772563-527260 X-HE-Meta: U2FsdGVkX1/Mm8j+Bf7+Ro3TCESYAkzqE5HXhlVjcPjHe92r4aDeL1wvZIFO6IMK1s5aolKBytcs7BYdzfd/8riwIzo3GxDdU8KLnBQkoB3UGYhjcx3kETt9QYuD6xat903GspZptEExoWbmSJEgzVb3TejxSGll4VgvQKudwGASoRmDmGKECX5cbHL8K5d2Eyqe/pD7fUu3ynA45O9sysSDWUKOTC8vcvd67WPZvbz2wCMhx2oP9pnZsgae98h3RCY0NDjQyN5LE/lyUl9BciVtSmqIyhdBGaRgSpfUxL1UQd6F9kULPmWtp45sH2fY2qoXS82AF4J+IOwdvVxF9WcCXS4TgteG8jNFQEdyymncwQmvRTfFMAt4VUPgQ1XYNb0CgzoWu0Om0uAZr/xdEeF2DNDpqMVn2Zdbp8w1utB4yjRrNnTltwLb7H2hZYf9BAIBPwciGArN0byucQY6rWH+FsdmwZWnFoMt6/sPZpyvpY8xIJEx3yS1cIxMv3J+2KdBv1KVV6yZFLBCFijuBvfdm6lyvrem2f6Z/6G0TOMdCsLo/C4S5FvCaF2y51z1YovI7KZ09MX7Q60sB/z1u7om5UbsNCiQCYvfKEBN88G79vYgmVOaFXeeWq+363ukTEElYJxpW3SBcfqyZwpVp5t0URordTjbarCCXH9DLL32+oCxilUuMfyJbrDlmFUMqa/ndoQn6zwl+RG0BSmrbMzywzzpuJWFSNjoKor68fsGZShL2RiMRcrjqyyrVx62HH6MHKJY0A5sZXWTNuCaaRZ021q/jUqHEdWBHpRG4UeX67AJ13jRdFEVgwWw4sAmxtNnBEeIUKRbwENiq6V47pKjvfNhe1x6CK2qDh9L5xLuSnEtbhfOS9F1gwA30vfqzGehUVJ+00kRMRLTj2h9RUyr6n6wkm3ZVZTvLQlu4etJS3MIPZkVSd5KB2cxB1RFf4x63moCI/d9zeZLx2V tD/Wm+If TI93EtliT3nNG7IsZPNyqlQdUqyRf+6v7O6BOsWDf0QSpBU8eaFzB7e04CHIFvSLbtZoYQkOY2wWkBOdB8pPM5SKNuUZiVHIF3ps9pMY7JxYP9vsi8n5CDVnIBPg1AMYkJKywD0vy3g9zsNQUCR4NV7UNncQP0WzL1sWzCWyqDgVaTz2dUH9Vuq4uUoF3C3/s8th3oQ521XxZ8zvz+OTqPwdgGkAubkq4Ti9HuAwRJJ4aaYGJXtZzKSv3pZcxsTbuhcwj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Any place where pointer arithmetic is used to convert a virtual address into a physical one can raise errors if the virtual address is tagged. Reset the pointer's tag by sign extending the tag bits in macros that do pointer arithmetic in address conversions. There will be no change in compiled code with KASAN disabled since the compiler will optimize the __tag_reset() out. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/page.h | 17 +++++++++++++---- arch/x86/include/asm/page_64.h | 2 +- arch/x86/mm/physaddr.c | 1 + 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 9265f2fca99a..e37f63b50728 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -7,6 +7,7 @@ #ifdef __KERNEL__ #include +#include #ifdef CONFIG_X86_64 #include @@ -41,7 +42,7 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr, #define __pa(x) __phys_addr((unsigned long)(x)) #endif -#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(x)) +#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(__tag_reset(x))) /* __pa_symbol should be used for C visible symbols. This seems to be the official gcc blessed way to do such arithmetic. */ /* @@ -65,9 +66,17 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr, * virt_to_page(kaddr) returns a valid pointer if and only if * virt_addr_valid(kaddr) returns true. */ -#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) + +#ifdef CONFIG_KASAN_SW_TAGS +#define page_to_virt(x) ({ \ + __typeof__(x) __page = x; \ + void *__addr = __va(page_to_pfn((__typeof__(x))__tag_reset(__page)) << PAGE_SHIFT); \ + (void *)__tag_set((const void *)__addr, page_kasan_tag(__page)); \ +}) +#endif +#define virt_to_page(kaddr) pfn_to_page(__pa((void *)__tag_reset(kaddr)) >> PAGE_SHIFT) extern bool __virt_addr_valid(unsigned long kaddr); -#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long) (kaddr)) +#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long)(__tag_reset(kaddr))) static __always_inline void *pfn_to_kaddr(unsigned long pfn) { @@ -81,7 +90,7 @@ static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) { - return __canonical_address(vaddr, vaddr_bits) == vaddr; + return __canonical_address(vaddr, vaddr_bits) == __tag_reset(vaddr); } #endif /* __ASSEMBLER__ */ diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index d3aab6f4e59a..44975a8ae665 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -33,7 +33,7 @@ static __always_inline unsigned long __phys_addr_nodebug(unsigned long x) extern unsigned long __phys_addr(unsigned long); extern unsigned long __phys_addr_symbol(unsigned long); #else -#define __phys_addr(x) __phys_addr_nodebug(x) +#define __phys_addr(x) __phys_addr_nodebug(__tag_reset(x)) #define __phys_addr_symbol(x) \ ((unsigned long)(x) - __START_KERNEL_map + phys_base) #endif diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c index fc3f3d3e2ef2..7f2b11308245 100644 --- a/arch/x86/mm/physaddr.c +++ b/arch/x86/mm/physaddr.c @@ -14,6 +14,7 @@ #ifdef CONFIG_DEBUG_VIRTUAL unsigned long __phys_addr(unsigned long x) { + x = __tag_reset(x); unsigned long y = x - __START_KERNEL_map; /* use the carry flag to determine if x was < __START_KERNEL_map */ From patchwork Fri Apr 4 13:14:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECD2CC36010 for ; Fri, 4 Apr 2025 13:16:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE07428000D; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8E7228000C; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D07BD28000D; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AE1B328000C for ; Fri, 4 Apr 2025 09:16:19 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DAB3314017B for ; Fri, 4 Apr 2025 13:16:20 +0000 (UTC) X-FDA: 83296410120.30.1EEB913 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf22.hostedemail.com (Postfix) with ESMTP id BD51CC0005 for ; Fri, 4 Apr 2025 13:16:18 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ny2OzY9C; spf=pass (imf22.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772579; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KdeR7X0VD5TGF1E859GfE1E2NKkX8Ya0qtNSI/kkmbg=; b=r4PUXPYXMUio5ZJgRbbgCz/CsZeEVhTod+3G7k1RBTMtP3d0pH72NFJsq4AHkaptzLwrAY LTmiEnJt+eOiBWqtqhJhB7PptIpxqXwaO8Ba3y/wr7l0n9NyKdNNN8aL/yXvriVml5sCVB I+hFzcbVttpLSKWxYvx3dt/e4m+wsCQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772579; a=rsa-sha256; cv=none; b=2h0C1WsLrs6HjEJAgtKuvG94yh+lCD5KKRSxcUra8O7KoQct8QJD7uar038RzOLOtfkiKP S+D7vdfj0mYp/E5r6eWBdeGZGUPk38y62tCB58O2I+WpckYYrNH6DEk6liKECRPqivhmDH 6WJJq8QZFj5j7YBjbB8bQz/VXbll9KU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ny2OzY9C; spf=pass (imf22.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772579; x=1775308579; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Xxt+3lL+Fu5dj056IVRqfGZK76TUDUg49twetCywmu0=; b=ny2OzY9CoPgDsyqhySeQ7S7NoVtKzOZJ682Gqw+rTjoi9QPSAcwLYYCA 9IzPGDG0kzWlJqvYVowcsSVS9UEBOL81p9PpvISMFo8S/m5faaE5ipz5U WLyiHXDVxSMRK2n29vNuxdmY8yNutXQVEVhY9Zzu4FJkdvSsDj7ToYq5k JY+hdJoVGdFRbD7LR6sEKQ1PBLoUwOMmJ8aBT5QQryPVSR5+wKuljAz3v Zt9uMGHxjNGzKoL4VXeaUZzbadS0E2LoiVQONnmPv7uIE7oeSFA9zxBQ6 lWIB+z0LRM+7ywWWJrKOKqx5D4PG/npJt4B546l3qCE7MLt87A/RXBPgx g==; X-CSE-ConnectionGUID: n0bmJrwHQfm5aPpz+UitRw== X-CSE-MsgGUID: f9Vblp/mRz6BkG2fkoGWTw== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401770" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401770" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:18 -0700 X-CSE-ConnectionGUID: 8CO0HA7DQ/SZXENmyMAkLw== X-CSE-MsgGUID: PBphbyn9T9ueR/8k6hXkMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157194" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:02 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 06/14] x86: Physical address comparisons in fill_p*d/pte Date: Fri, 4 Apr 2025 15:14:10 +0200 Message-ID: <926742095b7e55099cc48d70848ca3c1eff4b5eb.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: r3cp7c49i5aw5fio11ugx7kxqwgq84tj X-Rspam-User: X-Rspamd-Queue-Id: BD51CC0005 X-Rspamd-Server: rspam08 X-HE-Tag: 1743772578-239654 X-HE-Meta: U2FsdGVkX19mhHXCTZkpP1peEWUrUeoCTcwzftATdFInE0PDXiy4M/caFNgZYooF9bkAututmub0P1dc0vrJ6Y7kLw2e5IM+wn7oiueAkGnjBBG6ZHg7252SQVkDY9q8zRtGJTpe00+it+FhU4syP5ITp/May/R7SeaJc2pXFUuR/jI25pNcArkpsrKeXWs3TAjg4Puqe4u/it3N89j4DJzoxiIEpT0FlmaNPfMzidvlfHyG7UVFGjn/LGlvlN9avtmZJ2o2fnhBptRWZ0UJvv7cJz9oGtzh8GfYI7ccobADC7XH6x9LUwTUuYaR8XLnx5nqnILrROhfFpBQbcs7h7wpUw46kt5RKej1qbhdqD5kQMgkMYTAZZ3gTAsOArEcSj1Mxk9pmvlMsrGDv+1kQStmwf1ALhpg8YT4ap21bp4hNLa4zREBXkRVE25ny2rkXbqzmGllYOcPevL9yiZGqenlhjO+BRJ/RPJ9c7x25JUDNY9dmg89HGrt3OnUchBp/qdPa2+DG6IdH1Lr51r0kQAJ8KBVBKZzz15YNicwJtXBMblT4Mf3uHvh2YJde4KrjOjnxOKKGqyRHL/WClVWkvUOUpknQp/1WUnpZCo97F+YSqnwj3Nqa3pdFZBGWARDo0f0HOE+n4BD4kP15FChfjAs78ySlfglco+xNziVx8Y0SvrRBA6bx0a0TIA6g6l68Qk0Tk6PkEAxqnFhdN1MAwlDpexxwpSBJD65CZWCy1CrSHX1ffmmUMqVHKjtDdYS2HE/MaG1DAv35FmEiltKxXvD2Z+ZKswRL67ZHfShhswsfU2nk/ELvPGMsULnCuwzWYTozLgoIYYjRvs3T7BvJGG3QfeHyJQuifiA0p+a+IED2nUUEZljk9dbDtBt+GB0YgjDGMCHePTSe+UiCYXmK3GwkPRhbTFvhldpxuFQC1C3uNSW/dtxlWnNZQ7g5QxOn3u1EUfRYYQQohL5kEa Is4Knn7d MDorTqllAjEf1r5d1rfMTf9NLBaBjDmY5cLOiwQVE+F/SSrx4dP2MI1tG544Wij+u6cyVg/Mc5iyQGyn3v3Ehc8RzmiZWU4BEz2j7uS3mF06T5tRWjZGsVS10hJ6rxD3Ho+7cikxc+8hUi41gqHJhOv3R4hFXBOHqdQ6hDwM/YaAwVvWyzdIh6DC37nlJj25Tzfu1iLPsSFBb4krclypMzBWEJdihLScOpyUsLvv3cWdNgYCi4Sr7g4lzGqxGR8C4dE8flp5xTbfpXlgTvnhTgvourQ5KS+4SUCXNYhvTnC53rcM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Calculating page offset returns a pointer without a tag. When comparing the calculated offset to a tagged page pointer an error is raised because they are not equal. Change pointer comparisons to physical address comparisons as to avoid issues with tagged pointers that pointer arithmetic would create. Open code pte_offset_kernel(), pmd_offset(), pud_offset() and p4d_offset(). Because one parameter is always zero and the rest of the function insides are enclosed inside __va(), removing that layer lowers the complexity of final assembly. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Open code *_offset() to avoid it's internal __va(). arch/x86/mm/init_64.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 519aa53114fa..d40699c16f14 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -251,7 +251,10 @@ static p4d_t *fill_p4d(pgd_t *pgd, unsigned long vaddr) if (pgd_none(*pgd)) { p4d_t *p4d = (p4d_t *)spp_getpage(); pgd_populate(&init_mm, pgd, p4d); - if (p4d != p4d_offset(pgd, 0)) + + if (__pa(p4d) != (pgtable_l5_enabled() ? + (unsigned long)pgd_val(*pgd) & PTE_PFN_MASK : + __pa(pgd))) printk(KERN_ERR "PAGETABLE BUG #00! %p <-> %p\n", p4d, p4d_offset(pgd, 0)); } @@ -263,7 +266,7 @@ static pud_t *fill_pud(p4d_t *p4d, unsigned long vaddr) if (p4d_none(*p4d)) { pud_t *pud = (pud_t *)spp_getpage(); p4d_populate(&init_mm, p4d, pud); - if (pud != pud_offset(p4d, 0)) + if (__pa(pud) != (p4d_val(*p4d) & p4d_pfn_mask(*p4d))) printk(KERN_ERR "PAGETABLE BUG #01! %p <-> %p\n", pud, pud_offset(p4d, 0)); } @@ -275,7 +278,7 @@ static pmd_t *fill_pmd(pud_t *pud, unsigned long vaddr) if (pud_none(*pud)) { pmd_t *pmd = (pmd_t *) spp_getpage(); pud_populate(&init_mm, pud, pmd); - if (pmd != pmd_offset(pud, 0)) + if (__pa(pmd) != (pud_val(*pud) & pud_pfn_mask(*pud))) printk(KERN_ERR "PAGETABLE BUG #02! %p <-> %p\n", pmd, pmd_offset(pud, 0)); } @@ -287,7 +290,7 @@ static pte_t *fill_pte(pmd_t *pmd, unsigned long vaddr) if (pmd_none(*pmd)) { pte_t *pte = (pte_t *) spp_getpage(); pmd_populate_kernel(&init_mm, pmd, pte); - if (pte != pte_offset_kernel(pmd, 0)) + if (__pa(pte) != (pmd_val(*pmd) & pmd_pfn_mask(*pmd))) printk(KERN_ERR "PAGETABLE BUG #03!\n"); } return pte_offset_kernel(pmd, vaddr); From patchwork Fri Apr 4 13:14:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24FBFC36010 for ; Fri, 4 Apr 2025 13:16:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C00B6B000D; Fri, 4 Apr 2025 09:16:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 170846B000E; Fri, 4 Apr 2025 09:16:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F296C6B0010; Fri, 4 Apr 2025 09:16:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D16846B000D for ; Fri, 4 Apr 2025 09:16:34 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2E69A80158 for ; Fri, 4 Apr 2025 13:16:36 +0000 (UTC) X-FDA: 83296410792.23.D66C4D7 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf04.hostedemail.com (Postfix) with ESMTP id 0C4A140007 for ; Fri, 4 Apr 2025 13:16:33 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aBFiFOle; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf04.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772594; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u3tIUvdwJ1dXcDSmm2Wnk8OG4iSBjdCOlpfnuayIjeM=; b=FeHh5uOqQNUNckIazQt44dXXgGq/Y0+tqp9MiaWNjpuS4M+drWucbpChUsuSg8p+nYU91O zQt/KKdl5GPLuJ0EaTt9jGYQ7BQYmRZQFb6xzoYe05IfOaIVW6C3D3q9/XkiRn0DUEk/TO VLXX6/ONP9599YMRObHWXhlRDK/JrpM= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aBFiFOle; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf04.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772594; a=rsa-sha256; cv=none; b=PKXdxshytHVC/pVNHs4OINR6EhXT/cCHoeGuw9C04QqiNgnGlvi0YSm3rw4JzzqZZeUhs0 +BC94vEA2flZAfu2UVA8slRag9Orvh/NpjK2/smEsaXmPsoYQkLkd7HYfJbiDZkkejxXmI Fv/1Y6j8ygJB221MEpWSyl87x+97JaQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772594; x=1775308594; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=C6B0G55t9QifqETFe0ucXP9cc/PDbPCg2V7XC9fn1Yw=; b=aBFiFOle7LNCg7RPhbqzjoDdVflmnRQ8pQ0658sNB1mGM4VPerAOpTBB nYpgZmmPWqVMu09r9O05vvkozFG9z46U0Rcb+3We2/qlLnPtBHxe8aZTM FXBQJz/HcW8uNIXhaC4IqX3Qo/2Zr3ctKmdwNOgAc1/36lMvCOUMUZNdC 7FagmW2qkNszn+lKki+7gC2/FJ437Hr78CliRIgRSrrE6AFbdBV6BdSRn g/tYnaO89/qUwlytZnxC/SqNdBBHY393hISbEXLWYUdF1zFkLlw9AQLaQ OjEQmdWztdCgjmv8ShZr2KqI3eNcbcM5uSURwkNWY1lw9nbc5b7oqUoEB g==; X-CSE-ConnectionGUID: lPSZMBohTV2trlAKQ/wX1Q== X-CSE-MsgGUID: 8MhK/5u4TNa2PRiqruv2Mg== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401809" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401809" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:33 -0700 X-CSE-ConnectionGUID: TzYslCmYQFCBxN0Q9bAEvg== X-CSE-MsgGUID: AIqBIgH2RQGX8Ny4prod5A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157216" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:17 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 07/14] x86: KASAN raw shadow memory PTE init Date: Fri, 4 Apr 2025 15:14:11 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 0C4A140007 X-Stat-Signature: 4msq8zzp4jjri36qi3pp83gh5jrsecnz X-Rspam-User: X-HE-Tag: 1743772593-940598 X-HE-Meta: U2FsdGVkX18T5xrtmSCVrMF/oWJ5McMM8GGhczaiZ91i1rvHLtxkFGz/g8yLan2P3xa/BWVY7NZCT59Vtyr0Ti0cqK8RpNJkmxsnxbuPq/MVS4BHRY6mxeHtQrU2guDUG8Rxjq8UVbAQC4hiVqdcczu7YUd1mJzNdwsWc/7xUSTxunoR3Sj/5lzoBaCV3BvgvFVAUyC99BbCS/k2OM8wl41VJk0wHbeu2R11o/TtHTuA3TVa1d05vPCprcd8z148MQJ5FO7rYUPXlOdi/lu+H+sRWuARBTUK/hW9SkMTVCBERyZHYYv7xq7pDb2Io54FMirFC07RIgAGUXyaY4NeHBIe1ql2EchVLkeXUAUplRiC1yG9drNHxlDLOAJhte09nvSIxLtOJfWHDfqUPAr2KD9EjYU0Fge2FW0XktDqmBneylcuiQht1HuvqzJbBwhFASGUnuRHABJ7I0LuAj8y7FYWiP7q9nOnPSuTtJYMaNmgZahatAQF86QBGD/zOlaGVmIR1R6iyZUx5aSohCyijrJ3dC5bC58T4wT6klV4dSMiyVGSivqwpZ8MPibGGtzUcz7GDXy4NApe5H2VNc6OnJefOsUPkE5b7aDLSOybkTzXfVuBM4oEqfnOvO5ZyfxwnKqOVcVgtZh1Nhvz6sVzHaqNvWSA3TOvPj9USITNWe6W6Q79Bk8PGTjAXeTcNeJ/l+QPCnD59O2+Y68LWc97ysYwCsWU7HtmTMcOO4kjIG2fY8luJ5+BvQSqJ5wtyoxjEjRy2fwG9g0c4u4A6m2zj+XXsgHMDq3Y5i7KGxLDtiFv1d+qHB3qLJVRx1Qndr3YzASqns6JOjOnKSKqtIDXNWJtBlsHC5mblDp3wJaJfBUDaqTylJGJVLTnD72uIGbhKW5d0iCEFc3+G6YUNNW/mi2bu2SUgiUdprL1iSxHI1VRwHSAvaTaTT2sLDWs+ebe6DyD7pEurqGrR0PAQjw w8dqBogm hHIVkjeFTDTF76z9pxKvKQ6Art38tKsfD9DSsvHfIoQHtTgBJd5QvR0a9ZlQfeaJsTc31dNZAdgQFEw8HVcXOvrKVBzQi82neWEm/AAlgsyVmfVkK1FcEit6j22BrEuJ262Bwv8FPhaeQ13GPBkw39ZlmjB73eFkDqEG+arcRPeI8muWiLegLvYGRFgAGXBJcUmTQHjRynxmxjj222gued7X/yDDKc+/u2B+UMh1Ergls2/rzb4ftz6boYBUh1OHJB/t+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In KASAN's generic mode the default value in shadow memory is zero. During initialization of shadow memory pages they are allocated and zeroed. In KASAN's tag-based mode the default tag for the arm64 architecture is 0xFE which corresponds to any memory that should not be accessed. On x86 (where tags are 4-bit wide instead of 8-bit wide) that tag is 0xE so during the initializations all the bytes in shadow memory pages should be filled with 0xE. Use memblock_alloc_try_nid_raw() instead of memblock_alloc_try_nid() to avoid zeroing out the memory so it can be set with the KASAN invalid tag. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Remove dense mode references, use memset() instead of kasan_poison(). arch/x86/mm/kasan_init_64.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 0539efd0d216..e8a451cafc8c 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -34,6 +34,18 @@ static __init void *early_alloc(size_t size, int nid, bool should_panic) return ptr; } +static __init void *early_raw_alloc(size_t size, int nid, bool should_panic) +{ + void *ptr = memblock_alloc_try_nid_raw(size, size, + __pa(MAX_DMA_ADDRESS), MEMBLOCK_ALLOC_ACCESSIBLE, nid); + + if (!ptr && should_panic) + panic("%pS: Failed to allocate page, nid=%d from=%lx\n", + (void *)_RET_IP_, nid, __pa(MAX_DMA_ADDRESS)); + + return ptr; +} + static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr, unsigned long end, int nid) { @@ -63,8 +75,9 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr, if (!pte_none(*pte)) continue; - p = early_alloc(PAGE_SIZE, nid, true); - entry = pfn_pte(PFN_DOWN(__pa(p)), PAGE_KERNEL); + p = early_raw_alloc(PAGE_SIZE, nid, true); + memset(p, PAGE_SIZE, KASAN_SHADOW_INIT); + entry = pfn_pte(PFN_DOWN(__pa_nodebug(p)), PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); } while (pte++, addr += PAGE_SIZE, addr != end); } @@ -436,7 +449,7 @@ void __init kasan_init(void) * it may contain some garbage. Now we can clear and write protect it, * since after the TLB flush no one should write to it. */ - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); for (i = 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; From patchwork Fri Apr 4 13:14:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A960C36010 for ; Fri, 4 Apr 2025 13:16:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2C686B0022; Fri, 4 Apr 2025 09:16:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EDB7E6B0023; Fri, 4 Apr 2025 09:16:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D53FB6B0024; Fri, 4 Apr 2025 09:16:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B3C746B0022 for ; Fri, 4 Apr 2025 09:16:49 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 18B8E1A0155 for ; Fri, 4 Apr 2025 13:16:51 +0000 (UTC) X-FDA: 83296411422.12.F1C9E2B Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf14.hostedemail.com (Postfix) with ESMTP id DC50310000A for ; Fri, 4 Apr 2025 13:16:48 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aRl9al2i; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf14.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772609; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wVLQ4BnG0OXK3p1MFinu0qIASsM8XSU4urWEXhy+z2k=; b=EPCLjhLIZvpbtkC6Jf0i7uAqNbddL5aG49G9P+f33sQcrp8npBCbxS8+mDpNVLVtqv8BYM 9nMiUAoJIlfyMWVnVQ5ELu9qg4fFFXcslg7l5ss2XyD9Fa5E9lRztzbZ0hnv2Q+m2QaDZV QYyxuvKteQk7On27m/VuNPpJSMWo524= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772609; a=rsa-sha256; cv=none; b=IfaVMiOJa9ysQT8f27JUK5UIQeYqi3jGMdHYc8xqISofvqEFv5tQ0zKZSDdo26S0DurvAh 27EqFB+iTkQ7pSN60QCQC0ttgmhuwfNb49ojzxkfNd4jtEH9ILD2vkvKJ53CFU3oiZHQr9 BZoxk9kQ1lnfftt86SS1Ai7tMaXiFns= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aRl9al2i; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf14.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772609; x=1775308609; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FDp9HRqNQ0oDD3Qx3RyJgwftrUUBuyFmC51wrtuHP28=; b=aRl9al2iWJ/7qlN+A7u9QuYiehNDpZdBG8h9+UeplmH1zY+MS6o1jeTX x7Cghe32icZuGiSNN/GnRB/pwIWAKw6zgZmUcACkYkJp7pBfdyVuirlPH lr1fWkWc9hfwNAOY0vm6cvedFQbswqm8bZT1fae4g+9+QD7AUfYvi3K04 /24SpDvqNrhXZlgbpWk/2s7tMDDF5tyBlZPSHrGdExv4HMfOZcUMUsyhN 2cJ+LLJYGTRMfBB72CzCfadIm2fsxDVbn3aWqaH+fr2LZCAm89Xy+VdpL yHPKmzv5bXvf4YjI4O3egdzr03WTiuQC/Tf+kSWj+q6D89urzfSYsoFob A==; X-CSE-ConnectionGUID: CFFx2j2PS4W/ty18zhM+NA== X-CSE-MsgGUID: 20u+8RtTRfieTXvTnuT/mg== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401862" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401862" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:48 -0700 X-CSE-ConnectionGUID: KRVPdSrrQwCsFMKOJCYkCg== X-CSE-MsgGUID: BkRczVO7Rcyr+7Hf/uQO/g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157245" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:32 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 08/14] x86: LAM initialization Date: Fri, 4 Apr 2025 15:14:12 +0200 Message-ID: <43b4db5c872bfaa4881cf45103b8fd3a826c1a9e.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DC50310000A X-Stat-Signature: cpgi8qaz7z9zswpxn3zo6z6nwxi5j3co X-Rspam-User: X-HE-Tag: 1743772608-882564 X-HE-Meta: U2FsdGVkX19dEaEy7QvqxBXHv8ocYM9tOdz0nmRyA5WiEVs2NZ/+WTf2Zb10i+MH9VOMEOqtgdpHJ16f0vEeGdApvWjZWZmZ87/p4+OOwaSV6hKqwuAHYMEfajLXpITsW+B5xXAI/NULPWhP/SruYcYLfcx1w3xrJMewTFEiSALt839dq7XfKPO12gluliTDAUS7GjTItKcVA0aoeSYrPgq9niV/WjYu+UnC7nMu1Z3FYEO801Wz540Osxq/rnq25uI/V7mnMvXlVwr0ExqNlxUp7XYeIxshfGxBlHeJ12K5UD1olMShjnNuUeDBbkNqjJbf8nQQdtnnsYMa+ur2KOc7Mwh8YMRgkfDsRV8qreGL/2jrhmlO6th9wmYlo/8dL1F3vk1fZAkZL7N2M30AA+oO/W/s/WGWHV98kv0XAzHb/0nDBAkINrS69cIl45+ST55ojfFjRURNcjNDjzfqdJ8y6/0J8eqdYOQnrmb9En3EZe9GLOHqs149NA9qQxioN8kVhxme3Z4D7qWdO8exWUUka/iApXQIO80ZBr+o6Ir8Uch6czqvPi6A1vT4fJL5ISTkF8m78y+x55ZrRvfJetRGcIWGLafYYVZvxs2wIr2iVhZ8OwimKhuGPEJUkoWKzsRYClAljKFgcCP8ai0RmPo2YZCGSkePcEiXuBJSfe7V10O48s/JFOHTh7Gd9OROGcUHK7yCBoBz/pB4smnYr48zupfUUkyqZDTB0IA0ii9ja/CNLgwi1eFoAtCOGsUr7WO8LNdqciL2aGJqWcaHoF0s2vXHOjPyrfKK/0wHuVvjfcw6aMFgTYcnZVhG5LjIVgGhYlgY7NpmAD99aJ90DwD3pVaoRLPf74e0+5bHLH59vmnTGa4Xp70XWSHchoDK9YTG9e8RLWm0Yww5smKPibpCWr1/bXkVKqhGLvyz7Qwo0ToIfxQ3Za22bv7EldtdJzq3x2opDaRVp6p6Vqo uGVtAJmL ZKYzffkj6nBJI5I1M/GuhC7CVJq8TJxw4wt364ZUp6vlkfD+2hyF0vtUjEb/90llSnuQzCg+zqt9eYajv9OMpDQzr0vOcd6Cyteon44aJgPWzszZ3SvZNbgbOgY4L5hAnlG4D1JGX69OnetxxkhLVTtLFJWhpAHc8aZn/2d29dvT3CIYpCnW60w2KNO8nRf2G9xGlnVcDdt3rng2UkjEzgmrPVUoJNNSf2PABKMA+bTuuWbOfsZQ9tSBmpbVAGB9cr3b1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To make use of KASAN's tag based mode on x86 Linear Address Masking (LAM) needs to be enabled. To do that the 28th bit in CR4 needs to be set. Set the bit in early memory initialization. When launching secondary CPUs the LAM bit gets lost. To avoid this it needs to get added in a mask in head_64.S. The bit mask permits some bits of CR4 to pass from the primary CPU to the secondary CPUs without being cleared. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/kernel/head_64.S | 3 +++ arch/x86/mm/init.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index fefe2a25cf02..95b897b8bbd2 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -209,6 +209,9 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) * there will be no global TLB entries after the execution." */ movl $(X86_CR4_PAE | X86_CR4_LA57), %edx +#ifdef CONFIG_ADDRESS_MASKING + orl $X86_CR4_LAM_SUP, %edx +#endif #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index bfa444a7dbb0..84cefc5dd69b 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -766,6 +766,9 @@ void __init init_mem_mapping(void) probe_page_size_mask(); setup_pcid(); + if (boot_cpu_has(X86_FEATURE_LAM) && IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + cr4_set_bits_and_update_boot(X86_CR4_LAM_SUP); + #ifdef CONFIG_X86_64 end = max_pfn << PAGE_SHIFT; #else From patchwork Fri Apr 4 13:14:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7E9CC36014 for ; Fri, 4 Apr 2025 13:17:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA2CA28000C; Fri, 4 Apr 2025 09:17:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7A3D280001; Fri, 4 Apr 2025 09:17:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF56A28000C; Fri, 4 Apr 2025 09:17:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9FA5E280001 for ; Fri, 4 Apr 2025 09:17:04 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E6A9114017F for ; Fri, 4 Apr 2025 13:17:05 +0000 (UTC) X-FDA: 83296412010.07.AA71226 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf13.hostedemail.com (Postfix) with ESMTP id A995E2000E for ; Fri, 4 Apr 2025 13:17:03 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=D4aiAFjt; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf13.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772623; a=rsa-sha256; cv=none; b=kHrxMW8zTPMARAegCd0uTRzdHLSCpW0K14JKnzmJ6hfptW63mWHhE8jiDlUOawZL8bgtDs AyWD1q++XuFqPEUml2xY9NNYdtdpAAyd2LjhLnXodpU2lcBhw8FVDfiELO87uScxaxE2oz yG7XYBsIyzI0YorBw8lNYEtJEslecAI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=D4aiAFjt; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf13.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772623; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lnf6pC/5n4SCy1ffOsH42Rbn/6DISiknxHlO/AP+4As=; b=1CQI9pDZYO7CX2H+h8ttx3Y2ry2X4cjP+5M5vgZsumGUtzNsF86nThiLxPI1Ub05YBb4Dn jH/HrRof5H7iE0vmH0a4yvnTHAIL2+rB33x8ZCxsbHT7VKmEbPofOWdnpIrJCuiVsUGu5T cMx+xaBhrPc7c8XV3iBgDXbVyhYZk6E= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772624; x=1775308624; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WLJPoxzJ+9icJR/wXoKCRX4w/xjfsl8E7YkLYgEJBpY=; b=D4aiAFjtyX8Oqn64T3OsF5asBVkylOYhdmZQhllIwfRHkcYlcvxw2Ikr UQYh2k7dEnKGQ3Q+oiV9GmTUTY7PyXPWR+axnTe4M4IXpRy9nQXzonBpc hJbXfn/2jKtBLjJ85zdYqA8P0rdd136J19rZv1XIffdFiI967xGWBl7cB mN7t1ihWIUXdWdRDiCLhbADmvCI0qfYLwdIRwKZvMlOegff4E4ciT8XKh 5Lp9Sbu+2x/tZdw3kfYEgs74IZj1T891ErQlxQ7xsdEwEZa9Bj1Jze+TX Xbm14Pv2MaXtHHE/rKwzc0yWDSk5wgqnwQUquEARc+VME1imxjOh7V61H A==; X-CSE-ConnectionGUID: iaHI72a5RLmf4xPFHsJDJw== X-CSE-MsgGUID: z0qvs65SSW6uT+hJDe/JYQ== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401894" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401894" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:03 -0700 X-CSE-ConnectionGUID: 3+mynkr+SXqFVcdE3tJihA== X-CSE-MsgGUID: ZzPu2O3cTHCCuwduFbu/7A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157271" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:16:47 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 09/14] x86: Minimal SLAB alignment Date: Fri, 4 Apr 2025 15:14:13 +0200 Message-ID: <173d99afea37321e76e9380b49bd5966be8db849.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: A995E2000E X-Stat-Signature: ogdq6kih3brrzsjkexrusgkwb71kpdx1 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1743772623-375349 X-HE-Meta: U2FsdGVkX1/IdW4zPdKQomViMYwd1vqBrSELGy7bI24K4cese7g1XTiqAeBWcpnwUCMbfoaAVPsqb17e6KfeJOA4qxmTF618EbR090iXftrYpvSD741ZbAyEtYw974duzFKmbjzeuQIgN3MXKptM9eH3ipBcCCmOfdBT0eiEe4tdMbgRr2wXcwml4PP9K7cu+nDvmRIqUXO8ZIDBfsiZCFeCdWPL8PNAFd7wL7OAMOhFsClRQ9ebJf4/fp/Bmx3uktCRhGC7ldbTelqELyS9WM1fHKQZMLLVIiBekzaJA7/9UGDCta/6UyI1PEpaFA50hxI0guE9Bqr8ZUMIhwcKz5GCaxv8Csz9nEzkfJ45KfV+d+lFNtmFv5JdmdZLVmkHC0kbzFOU9gzQU8aFNSljbOhBxk2L1D0xfTnAZqmX9m4U/IAyvb+ncPlzk8k8LmqBcgrWSbNw/cgXHA25VBCmLz/kcpSpDdA4/V2BcX1B7KgJO044XBYkCBDCEqd3mHr9uiMt+V2Vhs+X+BBfIjB49k+Tkv2LpB0e3cFpE0P0yAXZtTPpiMQsNg6NsRXHStvDKpcQ3h80De+vBm7mEp7ZvZUS4X/HB7ue1rlfA3fK+Zw/DZ3Mm9c1s8f7AIruN4dEx+/7MEeU3iulaIKSVhy2seN12Yk07t/SZhXievZ4NIx8WlRAgp1RTxiI+P962904hjg+T7sxyh4ChmZEYN8EVuBPjogved/vl9vEjD60ODNi1e9PWNQR4KeM/dEOHH8+RDc7MvAW1EuprPj/rf2FWsRngCsPXLT5VDmX/7yOQv1MgJIlFp0rduRGdETG/FSvPUgAMZEU+e1qe0TkgmeA0gblCRpyI58S8e39r1p7qQX4rC/jB55yrUJPdMuL41zSUMylHloLXGRtqFpkCxsF7ZcrnHbRn+Upc1mX1sMSzpVxQsYsOvrv2+shC/XEfg1IWsPc2xKmLJZit3cW8d8 8u8H0hf+ IZWg11jl/8LZ2Z+ywHsKHdx+p6WGEVAP1cDW7wJdfumqSWH8Y9eolj1Tk1QNmL0j0m0qHQ7bIgP3VK14JW4qFfEOtSmyFZ+P47LikIw8prOsZl0m4egV9tfzh/EgZtK0seg9pQXXLlT+tQGHcflwVWDH1jXh7X9rAtws9lkjog+wy+zJQhdkNpxfqhDXNOy1TvpakWyPwcdchxy6Dmtr9m3rpLGxNLEhHV1ZlACFQUCrnCUlyuaA4BfapWbdC+YuhUwsu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Adjust x86 minimal SLAB alignment to match KASAN granularity size. In tag-based mode the size changes to 16 bytes so the value needs to be 16. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Fix typo in patch message 4 -> 16. - Change define location to arch/x86/include/asm/cache.c. arch/x86/include/asm/cache.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/include/asm/cache.h b/arch/x86/include/asm/cache.h index 69404eae9983..3232583b5487 100644 --- a/arch/x86/include/asm/cache.h +++ b/arch/x86/include/asm/cache.h @@ -21,4 +21,8 @@ #endif #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define ARCH_SLAB_MINALIGN (1ULL << KASAN_SHADOW_SCALE_SHIFT) +#endif + #endif /* _ASM_X86_CACHE_H */ From patchwork Fri Apr 4 13:14:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E491C36010 for ; Fri, 4 Apr 2025 13:17:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6523028000E; Fri, 4 Apr 2025 09:17:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 60182280001; Fri, 4 Apr 2025 09:17:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 47A6828000E; Fri, 4 Apr 2025 09:17:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2462E280001 for ; Fri, 4 Apr 2025 09:17:20 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 86EB91601AB for ; Fri, 4 Apr 2025 13:17:21 +0000 (UTC) X-FDA: 83296412682.14.969AD85 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf23.hostedemail.com (Postfix) with ESMTP id 4CE0C140010 for ; Fri, 4 Apr 2025 13:17:19 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=K6xPdlWR; spf=pass (imf23.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772639; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hdkFszdtBdvZ7spFw9JqGKHX1CI6+ij0yexdS7qNDVs=; b=ZqT9nsZw0kb44VwfyVdiff4/eRTdLv9pcmkKnJ1C+APrmZG06dmPHFU1m832kiGpnwBNT4 WGcBZemidVX32X3DXXFLC+FfUu1Cw8A3ZfF8/1yQFTub3zSaLyPWleLQbU+21YkHGmYm6s LmRWe2mMmM0mvRV4KayA6s/iuzk9rps= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=K6xPdlWR; spf=pass (imf23.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772639; a=rsa-sha256; cv=none; b=eTjBu75uCMYCS0VAk/99+MTXrhlR2hIF4xfJ3ostoY60kxxjJOXn99XD0i7elKatyJSxnq k0IQ6dv1y1RrDDLfFzKQ1urjahvOTWYGtEEoyl8laTeH9x6JY/u7YC8spP+1gUCvzn3q8+ Ye0eKlIeDYhVLQz3BaA850Ius8dWSzI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772640; x=1775308640; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SGBIaGE4wssEPedJ31N8z8fzvg1X3QBHbqhKaFUlYMU=; b=K6xPdlWRSZA1RziIDrHcGYD49I7Y58sHI19QgfqFATTKcsGFzmOQ5PTN BDvogcZew3/OzYK9moGFtgU0TPhLw6lvDi5JM0D4kDMjBLs4wIz/VMpVX x6sJAUHLaWGksG/x+Uz7Sqj0QhXVLh+tcaUekbZ1W6G5o4+PBB+Tq204E Oj6UDWmRoMbPxHu6QrN3HjeW9NfR2jcBsZtzXULswQSM63gTRg2Yhqzb3 8vViWNHTDLmEz9g0BM1PY0Db91GqAZC03nCJGBfZ4My7/GwetlJVtPZlV EYJCaEC/FhkI7MJ24E6ZtMZxt/gV85JEicb3bOVq7PLNxF643Sv9UGv6A g==; X-CSE-ConnectionGUID: UyQFa2kMRAWuoj1Pz9M9Wg== X-CSE-MsgGUID: ygGGm9XSS4+xlTi6AnJGgQ== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401958" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401958" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:18 -0700 X-CSE-ConnectionGUID: fsVEwllhSRmFPFm7kCxT5g== X-CSE-MsgGUID: rCQHxZP4R6qYhcSTuBxnqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157312" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:02 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 10/14] x86: Update the KASAN non-canonical hook Date: Fri, 4 Apr 2025 15:14:14 +0200 Message-ID: X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4CE0C140010 X-Stat-Signature: qr5yntf5xinkcxd74sci3d5un4kn9x8k X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1743772639-948121 X-HE-Meta: U2FsdGVkX1+duuBYRgaAWFF2MC/dRGAfMXdUvm2StrVhGir5EUJmtxTIl9R1IZllt+qkJSggXLW4bRVaP4SrntyEbiN5D1y0zb1VwIl1Sq2kDEjnLHIASOEuqcBy6qvh23ECvkdvIOoR8Xkezq0Jj96svAfoOC4PXkdYWnKy6bLhY/FZrsZWxOIadKNXnu41ThsPKiGzqZgptPDn0H/gQdIrDr47ZvuDvJOSK+PDkLtGe9EbNduVrAstMRGkXEeqKcONN5YiUqqUsQi8z8gQbnzCeypiu+s6jrr4mJYQNXRU2RRDvUv4PENAgmuZvl9dVTpEHdxIFrOX9L9h4la4svtzYsqZ4HQAw7913jaUPaQWhVAUA+oo7Mvz6HhbbNl4dozMEKW606+H0u1K4GoQABIdt1fWjeAm5huft0+8G5kQxUnLdYOZ+3+jnkEG2pSYKWBmklyFxdB0/OKtZw/DH1vNUROEcTAASR8uXJ/p6bWH34BA6eV7quxGB3r4pui0+q8tA0fPHVElF0Ki4LK1TcLxCiT67op1LXYBXyr48DXtohpUdqsmqycqCAyjYojGpRrVIjy+J4deTl5SSHogKx/8YErrVXHOUHWw6HO62SCYvNBGuwBliIkKL7iisTamboZoGVx55Y/5g8P2j15v+7LyeAY1b/12UdkOE4GSh8ZDEc/AgI3xLLdHahTKIbgPwkEQea2Vh4SuQkkwCYBeb+dAAsIiuCI9uiCAlE4juoiRn2AGobDTlBCsbMWLl4kcwrq12eKLBGBjrY5A2PDe+jfDQ6wMbaft1HN0+ehzdiG01s+1dLQYUusU4g9YoGjBaVBSBKNR3ITfyHus+T52TTgFkZ48CpH7gWGl0jzEzZeyFUZtjYPUTe+BcVJ6daLJgbDhnSEKD0FOYYVbm9dX/WyA+gyC/nAeJvK0In1Oewlr2p7yzx+x6qKKzf3EdFwgYXO/5zXjQEjFWHg7Deu RZ6+dK/F mS/SBNPUE8e83GctQ7naGpLlhwGSSKgNZ5rDJrWMBlCuj/y00M9Uq7v/dBTSn4hR/WaVlftfAWOLo6lSKI9Q0KRo3S7NZEvFve3BRuuqINGwf3sy/uZ6dK1lNnt+X7XkHoyTzwXDalGvP3EDBhm7m8tbfHxIsu052nbgUe/SqGWw8z8FMVRIVSLMWClHcA7Yqhkl0o25Oi4VxCZQCWkMhMvnRRcqZlt/F9zPBWHU4whTN3hOU9tL3ydS2f0Lxz8/xNQsE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The kasan_non_canonical_hook() is useful in pointing out that an address which caused some kind of error could be the result of kasan_mem_to_shadow() mapping. Currently it's called only in the general protection handler code path but can give helpful information also in page fault oops reports. For example consider a page fault for address 0xffdefc0000000000 on a 5-level paging system. It could have been accessed from KASAN's kasan_mem_to_shadow() called on 0xfef0000000000000 address. Without the kasan_non_canonical_hook() in the page fault case it might be hard to figure out why an error occurred. Add kasan_non_canonical_hook() to the beginning of show_fault_oops(). Update kasan_non_canonical_hook() to take into account the possible memory to shadow mappings in the software tag-based mode of x86. Patch was tested with positive results by accessing the following addresses, causing #GPs and #PFs. Valid mappings (showing kasan_non_canonical_hook() message): 0xFFFFFFFF8FFFFFFF 0xFEF0000000000000 0x7FFFFF4FFFFFFFFF 0x7EF0000000000000 Invalid mappings (not showing kasan_non_canonical_hook() message): 0xFFFFFFFFF8FFFFFF 0xFFBFFC0000000000 0x07EFFC0000000000 0x000E000000000000 Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Move the report.c part from first patch in the series to this new patch to have x86 changes in one place. - Add the call in fault oops. - Extend the comment in report.c with a graphical representation of what addresses are valid and invalid in memory to shadow mapping. arch/x86/mm/fault.c | 2 ++ mm/kasan/report.c | 36 +++++++++++++++++++++++++++++++++++- 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 697432f63c59..16366af60ae5 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -511,6 +511,8 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad if (!oops_may_print()) return; + kasan_non_canonical_hook(address); + if (error_code & X86_PF_INSTR) { unsigned int level; bool nx, rw; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index f24f11cc644a..135307c93c2c 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -700,7 +700,7 @@ void kasan_non_canonical_hook(unsigned long addr) * operation would overflow only for some memory addresses. However, due * to the chosen KASAN_SHADOW_OFFSET values and the fact the * kasan_mem_to_shadow() only operates on pointers with the tag reset, - * the overflow always happens. + * the overflow always happens (for both x86 and arm64). * * For arm64, the top byte of the pointer gets reset to 0xFF. Thus, the * possible shadow addresses belong to a region that is the result of @@ -715,6 +715,40 @@ void kasan_non_canonical_hook(unsigned long addr) return; } + /* + * For x86-64, only the pointer bits [62:57] get reset, and bits #63 + * and #56 can be 0 or 1. Thus, kasan_mem_to_shadow() can be possibly + * applied to two regions of memory: + * [0x7E00000000000000, 0x7FFFFFFFFFFFFFFF] and + * [0xFE00000000000000, 0xFFFFFFFFFFFFFFFF]. As the overflow happens + * for both ends of both memory ranges, both possible shadow regions + * are contiguous. + * + * Given the KASAN_SHADOW_OFFSET equal to 0xffeffc0000000000, the + * following ranges are valid mem-to-shadow mappings: + * + * 0xFFFFFFFFFFFFFFFF + * INVALID + * 0xFFEFFBFFFFFFFFFF - kasan_mem_to_shadow(~0UL) + * VALID - kasan shadow mem + * VALID - non-canonical kernel virtual address + * 0xFFCFFC0000000000 - kasan_mem_to_shadow(0xFEUL << 56) + * INVALID + * 0x07EFFBFFFFFFFFFF - kasan_mem_to_shadow(~0UL >> 1) + * VALID - non-canonical user virtual addresses + * VALID - user addresses + * 0x07CFFC0000000000 - kasan_mem_to_shadow(0x7EUL << 56) + * INVALID + * 0x0000000000000000 + */ + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) && IS_ENABLED(CONFIG_X86_64)) { + if ((addr < (u64)kasan_mem_to_shadow((void *)(0x7EUL << 56)) || + addr > (u64)kasan_mem_to_shadow((void *)(~0UL >> 1))) && + (addr < (u64)kasan_mem_to_shadow((void *)(0xFEUL << 56)) || + addr > (u64)kasan_mem_to_shadow((void *)(~0UL)))) + return; + } + orig_addr = (unsigned long)kasan_shadow_to_mem((void *)addr); /* From patchwork Fri Apr 4 13:14:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB8E9C36010 for ; Fri, 4 Apr 2025 13:17:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1D8228000F; Fri, 4 Apr 2025 09:17:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BCF7E280001; Fri, 4 Apr 2025 09:17:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A470028000F; Fri, 4 Apr 2025 09:17:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 82096280001 for ; Fri, 4 Apr 2025 09:17:35 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 779B01A0198 for ; Fri, 4 Apr 2025 13:17:36 +0000 (UTC) X-FDA: 83296413312.11.89171D0 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf11.hostedemail.com (Postfix) with ESMTP id 518BE40002 for ; Fri, 4 Apr 2025 13:17:34 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NNCjA02e; spf=pass (imf11.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772654; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YmTEjrWSa8coBPFUOPSr23Xq/XeudCw3/a4+DRkDMns=; b=y4AMIw4z2vij46zJcZMl8wLn9v2uIRun6aPz8V5eqcKBSeDsg0ku70xzDh6h7iIWe0OTY/ UkwlDfNqSyGuWmkchgPdYylVT752WcvQit81+raNNRwRcJJVCa5Rnz89Qy8kvieiR/ggbL L3hUZ0sEsiHO7uz49vCdRc84ECmfEBs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772654; a=rsa-sha256; cv=none; b=14eHq2Q4U45DEAFvTRTmHRhFMVGP0fkr6ZZZig1snJ9Pgq183pkO7fw3MD0Ysll6jp9cz2 rCU3f0Ewh7HoLI/XpZptHjOiH5NT3TewjOiSm1VJBeMhwNHxZo+GqnlNy6LUsXgXvgZy2v RWeeWFGhfu6H01JhZXU4L14DeHBLge8= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NNCjA02e; spf=pass (imf11.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772655; x=1775308655; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RSzN8akHhL4pnE/JPwS3fyp5Znr/igOnfk7xe5eymj4=; b=NNCjA02eRrMVs8xD7Ozar4vJjXnqpjST/sbGjA5J5sVG35Ld93ZWDdYh j0/RetAdwFHlQmMZEWi8bj80Y6PQxI8T5D32p93wlO1ugl/8j6rA2qn87 IJZosMw5++vmHdsUoHVEehXLigHfNdGtTJLZlREgonfvOhWEkwsvvsdjP xnNYRvTNEiuU3Ejp1zC7uYfX/vdWS79Ucz3vfTSr9jmlx+K11rkvIDTby hNlHDR3nVWzR1rm1LFMf512PLcCvKk2lOFEoRzfHOi75cmiEtTyemOO4Q ue+iDFOupaipgqjOoiywc/argDFQrJRNC6S1nvQbb8iAnkevJ7oObLiM9 g==; X-CSE-ConnectionGUID: M2yjEtCIS36Ch0D0WKotPw== X-CSE-MsgGUID: gklvbINnQceyKU82DJi3+g== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55401999" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55401999" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:33 -0700 X-CSE-ConnectionGUID: FCqpUyFdSE+x8MZMxgr8BA== X-CSE-MsgGUID: 2YHQdmbJQX2GJD9jL/iewQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157352" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:18 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 11/14] x86: Handle int3 for inline KASAN reports Date: Fri, 4 Apr 2025 15:14:15 +0200 Message-ID: <012c84049b853d6853a7d6c887ce0c2323bcd80a.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: du75s98wzy3sztsyoq4zrita8iorcenb X-Rspam-User: X-Rspamd-Queue-Id: 518BE40002 X-Rspamd-Server: rspam08 X-HE-Tag: 1743772654-534062 X-HE-Meta: U2FsdGVkX1+JmmrPcpUjrGJ6npYUv+LhbOOpys6XQKetFyYlWpRQxQCZMBIZUMfHzIqjMWoVliMvaZP4giJTxa/9h1qB/v09FNW0JOiZigbro3Si7GqsF4fbRJK6Or6N6S52F5pYrs1Qwd/J+Jpvfool9ECUo/0rdj2y6fILUmHtpYKZ1mQ/Ya/awH3W4UB/+akkPb4scujDYY2/kPSgVvRBIEGmTPvBxwHbBhV3ewic0lLJihXUXEOyFpWonxBMf2pe/CX234VPtLNCuLNq3QusLDyjODQzylmMh3zj3YIFIXJR882LBlfJb4lQVVX6dmB0a6lfVsH0QYW9jG0+R4J+wQI9mjYffreRW7mSrpGqy6sV1/SREbcXG1D6Y9wDiwrkDgqWk94438vaXTj+bM0/jXyutD4/F5e91uB0VWdsqXlM+aopFqndDP1KjF/Tp1c7PNU4VoAMDfSB9/C9wd2QXCPsUKMJc6bF9ypQ3Wyb0aOF9LQLqRsrhg2PzziOs894cy4cEDZGG4dDgav05BTA3FcIiJXgtw2W6AgwtE644iMtC6u5daASw+nMLEzxMZCjdAECCAWaxv5JkQNlpZBIeJM+9FcXIDmIdj60eNI9XXFlTDNjaZ5Eel33GX9JwlaUok9iUOm1NZ9ugsZ4N7m5yUNK0vxHbVF29QFtSsHhWt91Jdof7TENiPl0DLTPDeAdTn32j8V+/N6AE9TiXkwxNoytSwwFkVDqDDZLiwNy3xLBf6G+3YoJwlq0USgYaPBbWxkNVvNW3fFpQOyJUcuKry0wZ98wvNsp/BwtTbxSP2H0ace+nCiXN47PsIXeTZ7+b7ih8onKxLa2dF3i8vKpz67TVZcbOV0B2EOceqSQPlwsNkSSBgqXqhBcrdCSor5y7dH8c8r0nZS5/YFJIQnrU2RI2asYicdVPpHyjyQodAGSzL2vHLsXmfCiBiqtsjVCSd0i+7bNJsslhGu kbvwPcQt KLPXBjacDdN2FhS2w40mc+TTMNmabfK/G3Uege+QSKeYRyDqzpVkWOcsskN8RppH3NOWmMDaXe7H2hx6t8CDFaA1JXa/Of/1jnzXpr6ihohRWA2sU81IIs2/WILxuo6WCAFM9n4GH5/ensgWGB0x9AUkXmTa8PudjL4fhurEMLNWjoZCN7JBWm6Sw7yd+SwPeDkqtWsviMGxuEaSjs+dPuUNxw14+ihgxMKEuVgC+pzfUhfeE8XuhVVj0AGhPrtaz/s7b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When a tag mismatch happens in inline software tag-based KASAN on x86 an int3 instruction is executed and needs proper handling. Call kasan_report() from the int3 handler and pass down the proper information from registers - RDI should contain the problematic address and RAX other metadata. Also early return from the int3 selftest if inline KASAN is enabled since it will cause a kernel panic otherwise. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Add this patch to the series. arch/x86/kernel/alternative.c | 3 ++ arch/x86/kernel/traps.c | 52 +++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index bf82c6f7d690..ba277a25b57f 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -1979,6 +1979,9 @@ static noinline void __init int3_selftest(void) }; unsigned int val = 0; + if (IS_ENABLED(CONFIG_KASAN_INLINE)) + return; + BUG_ON(register_die_notifier(&int3_exception_nb)); /* diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 9f88b8a78e50..32c81fc2d439 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -849,6 +850,51 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) cond_local_irq_disable(regs); } +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_RAX_RECOVER 0x20 +#define KASAN_RAX_WRITE 0x10 +#define KASAN_RAX_SIZE_MASK 0x0f +#define KASAN_RAX_SIZE(rax) (1 << ((rax) & KASAN_RAX_SIZE_MASK)) + +static bool kasan_handler(struct pt_regs *regs) +{ + int metadata = regs->ax; + u64 addr = regs->di; + u64 pc = regs->ip; + bool recover = metadata & KASAN_RAX_RECOVER; + bool write = metadata & KASAN_RAX_WRITE; + size_t size = KASAN_RAX_SIZE(metadata); + + if (!IS_ENABLED(CONFIG_KASAN_INLINE)) + return false; + + if (user_mode(regs)) + return false; + + kasan_report((void *)addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; this is controlled by + * current->kasan_depth). All these accesses are detected by the tool, + * even though the reports for them are not printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KASAN", regs, 0); + return true; +} + +#endif + static bool do_int3(struct pt_regs *regs) { int res; @@ -863,6 +909,12 @@ static bool do_int3(struct pt_regs *regs) if (kprobe_int3_handler(regs)) return true; #endif + +#ifdef CONFIG_KASAN_SW_TAGS + if (kasan_handler(regs)) + return true; +#endif + res = notify_die(DIE_INT3, "int3", regs, 0, X86_TRAP_BP, SIGTRAP); return res == NOTIFY_STOP; From patchwork Fri Apr 4 13:14:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FC62C36010 for ; Fri, 4 Apr 2025 13:17:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B308280010; Fri, 4 Apr 2025 09:17:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 588C1280001; Fri, 4 Apr 2025 09:17:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4020F280010; Fri, 4 Apr 2025 09:17:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 201FE280001 for ; Fri, 4 Apr 2025 09:17:50 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8A65BC0175 for ; Fri, 4 Apr 2025 13:17:51 +0000 (UTC) X-FDA: 83296413942.30.B6BEF5D Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf11.hostedemail.com (Postfix) with ESMTP id 4EDD24000C for ; Fri, 4 Apr 2025 13:17:49 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=g7MFaeJ6; spf=pass (imf11.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772669; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VZk6z45/TieMCWLPI8iFY06hsd50xsb22xpBUNnECFQ=; b=f7Se5Ss63KHDPuYO3zFq2gZDK4gjLTPS4aDyNN7a90yQ3Fp3lYTq6nQsXr6PV4Rqv4tiGy AAPP2M3qoVranqCH+EG2fBLqQ4lMuKD+qdEK1y3BW66TAHkdskMMjTfV6y4xmJFpx7sb8K pS7O/EFvzWthkbaeKIoWkFFgRNbpvI0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=g7MFaeJ6; spf=pass (imf11.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772669; a=rsa-sha256; cv=none; b=5daEwXeMb8UObxm0lIQ2jv99HstV/vgLx8o/tbtyTKAJhCivTmtMtd4C7lD4WoZ/b8+PrQ nRPxcN66BFr3cm6lwYfRbqHAAwlKzFw/0jNGBxzSlyTFyV1mYjTa3ZHLk5q6mRck8zNIKu xpJ4PazbKtKSjyNF1+afgIWen78LH+8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772670; x=1775308670; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kLMkp6bZDxIMJX1kOLifwD1KQicNBp9kwQEWVdEeDAs=; b=g7MFaeJ6mRnmNEk22CUmDGasQ6luFYetfs5qEUlkRVoZqUfCbhC9hfzi +ikENt0V+gGQwl6kAqGnL4rHouOQpSgb0L7SfpqJg399+zaUHjlpZQafL LDPI4tupgkGfrs3PW58zO48N6er+NmXhItO6/2G75+nJcK4zUaPGu163O AZbstDbvxH3Q1EXu5Cy+II7h+4LWhefq0TbHA/5Sj4DAVO778MWCtDfoU wpAjm681y9gRuSuU4PiyB3k91+84SLPSPf9o2r2ic3dzxU+3NIZdyTqDx Jnd8dNa2MCRccSdvu8y/V0VHzePKaM+O//ZvlFBHcXfwnzbHX/hhDVpSm g==; X-CSE-ConnectionGUID: gkGHLUDMQPK7FeMcf2xfRg== X-CSE-MsgGUID: fzkMA0v+R+etPVpGzQED8A== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55402028" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55402028" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:48 -0700 X-CSE-ConnectionGUID: /KZBOILGRPaVrrMli4Itpg== X-CSE-MsgGUID: T+1IKqSwRheE1ok0gd7jhQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157374" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:33 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 12/14] kasan: Fix inline mode for x86 tag-based mode Date: Fri, 4 Apr 2025 15:14:16 +0200 Message-ID: <9a8862c380805ac6c2fc137e8edb1d2e70ee2812.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4EDD24000C X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: w6zzmzid5pkid4itkitxufayaifen1pw X-HE-Tag: 1743772669-26513 X-HE-Meta: U2FsdGVkX1+lyw0/SAib2iVJqSR6hApSNMo9zYjYTTu4e8Pax+8LzuZ8RCf6x74SJdHA3+rYVEkFcbGonK9lp2aaQjMPrGuKLwdgt1myr4qKG4swAN3a4W1fi3IvJ8NvwTxY53FjO12UMaQWn59aj1NptItkUGFls8i40RpGJF20rSzqRjppnIETZU5DpGm9KCC7L3mhIPvu3XszJ7q+y9BkKA2YjlgreFidVaLL6FQQc/xR7ihkp/bIFfrANAzSaP4ZVfHasY26hCx155b2yDG78TUgUo/RMF9vqdBk+P0LHGMDewh/m1dbZnljNZmUTbf1iDlevgKuCF2BHYjJqbmuYoleotfTFBGUdXeXa+8zijrehpFmLT/JyNWL1cIje2yvzlCWsW+CDzDqrK7Ssr5b/H1Ec1nOSC32KuAnB1MPYKFxIyynVlGO/QyvepPRJGYO3iCqsuIBY5NJ9xzQUwVfH8FQnrA2QiUnYbkaZfYkr3g94akfeZAWU2e15/kunOjm3wLuFMSkJ2E2XCiXO23IchWeay8HzZGVNDcIzxMjhJbyyADz/6QIGX9LQmTi9EQ8XWbwhnW2bnmutfVQpXLqqFzZ2VhWnOVsqagovbgy1meqN0mnbNGKlRYtB1b/Dx7bgmf4WgNhwRwwu+gc7yWKpEe//hCf1fF8Vn/bHUBPsp5DLrcxJCUuBqFyruXqVpqDr16S1IYIMVL+eikGj88HudVn1bwND/I1Q2mV/ERJj/f/VhE/GuK5EZ59CPB6I2JJ7++mcLNy4RiYAm99IocqBQG2+Rergkcgq2awWJW2bSxhVRCvJGRGyehNqRLihRWK3Vt+eivAR27yWb9Yqtkyz3h+KAu28w4PBUrmQlE5amIQb4asiJ7v7O1Op9dLNoUbIup2mR3n8Cz4wcjPnYsnwqtGnllnOMQsAzz1NSQ8yoMQEP6Qa01hAHCzfY9aLFpDjMhI+undlwCGQub JhCY4em8 k+JYBn/4k/cJKuXcWn1Ly6pUhXwQOEt3YWSeMwFPIJAqTf333Uq/Fg3rut8rN6spCfHWcbf9akCWgeHCSEwIpRZxv8cfAwbp0QIvNbXMKeWJ99tFxSX1LfiCNYXLQXzHZxjCUdZVMHoXdTGK7n/5u9PabZq4UrLEvx/OOThLqqR/kICgEV7kaA6XfNHCihmLx90VjWTvUhR63NiTHezBokU94G6c3rvuCMzeEx+Yy6iQoVjD4BfWfObI95uZucnJVAm0+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The LLVM compiler uses hwasan-instrument-with-calls parameter to setup inline or outline mode in tag-based KASAN. If zeroed, it means the instrumentation implementation will be copied into each relevant location along with appropriate constants during compilation. If set to one, all function instrumentation will be done with function calls instead. The default hwasan-instrument-with-calls value for the x86 architecture in the compiler is "1", which is not true for other architectures. Because of this enabling inline mode in software tag-based KASAN doesn't work on x86 as the kernel script doesn't zero out the parameter. Explicitly zero out hwasan-instrument-with-calls when enabling inline mode in tag-based KASAN. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Add this patch to the series. scripts/Makefile.kasan | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 693dbbebebba..2c7be96727ac 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -76,8 +76,11 @@ CFLAGS_KASAN := -fsanitize=kernel-hwaddress RUSTFLAGS_KASAN := -Zsanitizer=kernel-hwaddress \ -Zsanitizer-recover=kernel-hwaddress +# LLVM sets hwasan-instrument-with-calls to 1 on x86 by default. Set it to 0 +# when inline mode is enabled. ifdef CONFIG_KASAN_INLINE kasan_params += hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) + kasan_params += hwasan-instrument-with-calls=0 else kasan_params += hwasan-instrument-with-calls=1 endif From patchwork Fri Apr 4 13:14:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC059C36014 for ; Fri, 4 Apr 2025 13:18:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 73A8D280011; Fri, 4 Apr 2025 09:18:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E869280001; Fri, 4 Apr 2025 09:18:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5618C280011; Fri, 4 Apr 2025 09:18:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 31421280001 for ; Fri, 4 Apr 2025 09:18:05 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A96AF1201CE for ; Fri, 4 Apr 2025 13:18:06 +0000 (UTC) X-FDA: 83296414572.09.B6F64FA Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf18.hostedemail.com (Postfix) with ESMTP id 729C81C0005 for ; Fri, 4 Apr 2025 13:18:04 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="SF/dxn65"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf18.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772684; a=rsa-sha256; cv=none; b=MfB5esml/dZJBYhruNY7CmSmOLUpSRDB95JMahGuKMXOdjxP3kkcOvsmLGaavXgtcDTNgR mQec6Ztugn1KXQ1TzR17urncpftsHC7s7RhnAEPQSDUGMuGBnUM83AZzr+O9jWFv7d2fG5 a5NbAwc924ipFduAKD7kyarG++Jxcgw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="SF/dxn65"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf18.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772684; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hZzhASaz3Qj/h/GRsL/4I1VLPg+/CfxJsMQCXOGKIVU=; b=7e7wXtyFsleUhGHfqQ3xEtf0Mf6HU4z0DHlRhO2vZZhFs3Kahyj7PZXdL/Eopuzilu6c7K myGAJ13jf/XSppcrbXlZH2Mabhp3iqsIND1/Afg91W7cDnLLFp7V6Q/0j3TjpjKOH8BoLe F6dhFVIztakilI/L07+UhKHnOGUNFiw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772685; x=1775308685; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZsxDv6qG66vEKuwTwHaEtrjd9tBd4kf+DSHvVgMU/8M=; b=SF/dxn65YYYo0yvQ2KDfjGCUBbNKl8b87uR6ix5EMzG0s/K4DmZxUDVj f6T9lclGvDUxA1v5UR4GQIYWRwXYARcB9hbRh6h23u11fm9ow+tq6SCky zvY7tF2y5LB4ku2D3NnT2Ll28xe5wJdUNMtVlQDCTGUf7+YATJi1m8F28 dc/hBQRdPXjIPf5Qp+onExgceMrEBhonwkIY+9TPG7241Y8unV8XF6i1a LsRLnIdOyUYY5AEfbInsmbTw1cRtMSSNWmcUdDxfyuy9v+KOKbl4Cjw07 MARlHZc+837RyJlXZzoZqxnc21MOOurpIGzxGoZJnqLvkWWYIHG5O3/7V g==; X-CSE-ConnectionGUID: gZpvQPjpQuiBSRY1rdynPw== X-CSE-MsgGUID: qXWOdOOwRa2GQ0wllKvYUQ== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55402064" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55402064" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:18:03 -0700 X-CSE-ConnectionGUID: vtMsu7eRSsCTmkinYXKa3w== X-CSE-MsgGUID: mneJnZk0QlShqxm/wu1ARw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157400" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:17:48 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 13/14] mm: Unpoison pcpu chunks with base address tag Date: Fri, 4 Apr 2025 15:14:17 +0200 Message-ID: <61033ef5b70277039ceeb8f6173e8b3fbc271c08.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 729C81C0005 X-Stat-Signature: xhg9jikhumkdbjubr4xiftbbatbz1in3 X-HE-Tag: 1743772684-482534 X-HE-Meta: U2FsdGVkX18mFxQgt9nJw8D4ew6ZmIuNVW8eoGz8dC82zsaZcio12SCNjNLY3Vrr78bKkWhjANPwJxH8LzsMC6nQZ3GQnKx6JMwReiclrkGrrlw4mwHdB4eoCDlNZKGukqII8rasWYqfKXbJdeE+N/ca6Zg8rdpJDbq+TBsZGpvxKLhn07HmhytH6UjnPqRRnl/hANNogGMfPg4RucVg+9w8UWzJYuwavUTkbYG8p+bUbEGUULxDD/7yeWV1iAI5C2s47xoG9qJnPFPJkm/aTrZKkrAIBr9aAPpFG8WnTvb1pK5e9eEpkG27H+ZPLWAxYvsHFDnu5Gf9EH1vJ2MZsU+dE9O96wos0vPSfC7av3WxALIFBT5SBxgyRkkaPHSHJGIL9wQXEGJrVWTRqZdBFJaZfNRLOGtSgVtESikgm1u8+eNESFI8vrYNsLL4bpqROndnCZEzmseiY+vZZFGDDMRmVYc53r24k4XkxbobJEuUWHVEXA3uGwNMcRvNsyDmDwViN4EJIO4WATFBksxaV4L3Qi+qPoZ3gNp7R4f2WCuSJiqnikBSy7pof6nhksRtX4RKDodiEEeNBe0rciXbbFv577jDtHDK0hgmPRgTzjtgNnxlj7O7pwodS/4mz0VWlpvLCpkVOOdqPzroHcMwhKjlgRrkHHaUeYP39jimezoW/kG3G30X8wbIALcJhcxKF2Aqurd0HWWY9efQDQ++DeTkdpfjTeCxxibxA8Uqm1OPw+70Zai5SWncNnKV+RbIrQOmMvbYvRgsgPfji76qfqdfTCXt19QswPD6HmGqAOrNrx8b9fQ7jcUytfT8e0n4w8ryhem0x2B2bu0YgNnW7402rM1hQ+W4zNmaxvlMY9m9dTozDmkvM+UUj3icCYsnAGYTAflxj+Ty6gG8jXACvNZoR9hDiGCEWSRO+qxFtU1Il2CV/OOW8/bjREkkHLxFVosq779Ebw+qXSc4yxA SxN0cQJB DYr4JmI76IHwC0wzmxb9LM7yVrLNLniJ6yv3ZSY3YnD4KCUUmdxMRnWS+A5YqBr0jEZW3nxXXjGBaGGEpOr7E7JBNbzHw4W3Ux2nzUHP/KOd+lV9OoQlFCnPYGeIe4sieiA1XmzGv6hCQUPB44HM0sAIZZh2crX6N03H/eHlm0Zw6L88qNyYAV4H9UYNV56uqf0tEq0oxGbNqW8kRXjS2pYfAp+qAibCo5W6mZpvhIv788Xgj+Lcy/WPgyYymmKD/aBePBdM9zD8CslV8p6FPPu7RvVG2qdKRZcNr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The problem presented here is related to NUMA systems and tag-based KASAN mode. Getting to it can be explained in the following points: 1. A new chunk is created with pcpu_create_chunk() and vm_structs are allocated. On systems with one NUMA node only one is allocated, but with more NUMA nodes at least a second one will be allocated too. 2. chunk->base_addr is assigned the modified value of vms[0]->addr and thus inherits the tag of this allocated structure. 3. In pcpu_alloc() for each possible cpu pcpu_chunk_addr() is executed which calculates per cpu pointers that correspond to the vms structure addresses. The calculations are based on adding an offset from a table to chunk->base_addr. Here the problem presents itself since for addresses based on vms[1] and up, the tag will be different than the ones based on vms[0] (base_addr). The tag mismatch happens and an error is reported. Unpoison all the vms[]->addr with the same tag to resolve the mismatch. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Remove last version of this patch that just resets the tag on base_addr and add this patch that unpoisons all areas with the same tag instead. include/linux/kasan.h | 10 ++++++++++ mm/kasan/shadow.c | 11 +++++++++++ mm/vmalloc.c | 3 +-- 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 54481f8c30c5..bd033b2ba383 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -613,6 +613,13 @@ static __always_inline void kasan_poison_vmalloc(const void *start, __kasan_poison_vmalloc(start, size); } +void __kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms); +static __always_inline void kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ + if (kasan_enabled()) + __kasan_unpoison_vmap_areas(vms, nr_vms); +} + #else /* CONFIG_KASAN_VMALLOC */ static inline void kasan_populate_early_vm_area_shadow(void *start, @@ -637,6 +644,9 @@ static inline void *kasan_unpoison_vmalloc(const void *start, static inline void kasan_poison_vmalloc(const void *start, unsigned long size) { } +static inline void kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ } + #endif /* CONFIG_KASAN_VMALLOC */ #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 88d1c9dcb507..9496f256bc0f 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -582,6 +582,17 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); } +void __kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ + int area; + + for (area = 0 ; area < nr_vms ; area++) { + kasan_poison(vms[area]->addr, vms[area]->size, + arch_kasan_get_tag(vms[0]->addr), false); + arch_kasan_set_tag(vms[area]->addr, arch_kasan_get_tag(vms[0]->addr)); + } +} + #else /* CONFIG_KASAN_VMALLOC */ int kasan_alloc_module_shadow(void *addr, size_t size, gfp_t gfp_mask) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 61981ee1c9d2..fbd56bf8aeb2 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -4783,8 +4783,7 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ for (area = 0; area < nr_vms; area++) - vms[area]->addr = kasan_unpoison_vmalloc(vms[area]->addr, - vms[area]->size, KASAN_VMALLOC_PROT_NORMAL); + kasan_unpoison_vmap_areas(vms, nr_vms); kfree(vas); return vms; From patchwork Fri Apr 4 13:14:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maciej Wieczor-Retman X-Patchwork-Id: 14038465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF62AC36014 for ; Fri, 4 Apr 2025 13:18:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98266280013; Fri, 4 Apr 2025 09:18:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 90CCE280001; Fri, 4 Apr 2025 09:18:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75F9F280013; Fri, 4 Apr 2025 09:18:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 56314280001 for ; Fri, 4 Apr 2025 09:18:20 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C4F2314018B for ; Fri, 4 Apr 2025 13:18:21 +0000 (UTC) X-FDA: 83296415202.15.C74208B Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf20.hostedemail.com (Postfix) with ESMTP id 792821C0002 for ; Fri, 4 Apr 2025 13:18:19 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=U4e235vT; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743772699; a=rsa-sha256; cv=none; b=SpaFkOby34/m7JRHkRXjxAtgFlcjA6BrOqeUF1NQEt2pgGwE0VKefD6Dbpu8YNZL8Gu+9l w1jez/QaxBj1TqbavTsOcpjXFA6X8A95Y5k0z8hOtrsAcDVSxkasB7UyjMPbAPhQ5/PQnU 7NdbDVTcz0U1yIydq7f7H61GEzc1Gj0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=U4e235vT; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf20.hostedemail.com: domain of maciej.wieczor-retman@intel.com designates 198.175.65.11 as permitted sender) smtp.mailfrom=maciej.wieczor-retman@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743772699; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8gqGofCgAw35fL6rFp8nm8qS6bGOYi033PWynYkryUI=; b=BbI+x1JieKkYO9KFEWnZ1bQn6YFMb5kZZL6dLWfImEqfwyVavBuK5h13fc+9KGZuFR0qGS PO67c6hNSBWWXvNQ6GlGBntxmtKEKSCHEb86us8ySGHT60eqw6BrbC3EqTaOFOelAqUdmH XNPJecHs89SfbYhXnql3JjnS8fEzuzE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1743772700; x=1775308700; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LXNzuXLeRTm8jG3p+pKkuZRDyf0+LkHmCSi04lxWu3Q=; b=U4e235vTdsx9XBHbcovg/D74dMgFJTYKDPyTyWf1NGpuanIi1rYPmtT9 3fHuctG9QgTTR1PAdY0fHUOL59EJgTVBXzxXJcuCa6I5Q7p9CJd1gLORo C88gg5MW8QWLmsB9GPxfGWPrZ0R0BPcA75LEqk4GNC9B82l7/0cD9w1wS 6D5Z63QpybU5p3kF2d/5bluyhhP3qkicAh4fz3LhgpI/BQF3YeJ7rw0Hx Wf5YkJ7FNZflcs9Tq0OgqDdJvmMOu5nkBaDaBG+6NVtsRMvn9PJ/tJ1eb qf4TUX17DzlWvunOvJXm++zkkI0SS8OnLByMw0viZ3gdILsDzDOBHFT0s g==; X-CSE-ConnectionGUID: up1e13o5RmOO4U5CIvS1Sg== X-CSE-MsgGUID: +yW8c4tGQ+Wsv619L7ekzw== X-IronPort-AV: E=McAfee;i="6700,10204,11394"; a="55402087" X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="55402087" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:18:18 -0700 X-CSE-ConnectionGUID: JIG12BvgSLmwXeiVgXuG4w== X-CSE-MsgGUID: 6Y3E6zPDSmCFZvSwlS/caA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,188,1739865600"; d="scan'208";a="128157431" Received: from opintica-mobl1 (HELO wieczorr-mobl1.intel.com) ([10.245.245.50]) by fmviesa009-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 06:18:03 -0700 From: Maciej Wieczor-Retman To: hpa@zytor.com, hch@infradead.org, nick.desaulniers+lkml@gmail.com, kuan-ying.lee@canonical.com, masahiroy@kernel.org, samuel.holland@sifive.com, mingo@redhat.com, corbet@lwn.net, ryabinin.a.a@gmail.com, guoweikang.kernel@gmail.com, jpoimboe@kernel.org, ardb@kernel.org, vincenzo.frascino@arm.com, glider@google.com, kirill.shutemov@linux.intel.com, apopple@nvidia.com, samitolvanen@google.com, maciej.wieczor-retman@intel.com, kaleshsingh@google.com, jgross@suse.com, andreyknvl@gmail.com, scott@os.amperecomputing.com, tony.luck@intel.com, dvyukov@google.com, pasha.tatashin@soleen.com, ziy@nvidia.com, broonie@kernel.org, gatlin.newhouse@gmail.com, jackmanb@google.com, wangkefeng.wang@huawei.com, thiago.bauermann@linaro.org, tglx@linutronix.de, kees@kernel.org, akpm@linux-foundation.org, jason.andryuk@amd.com, snovitoll@gmail.com, xin@zytor.com, jan.kiszka@siemens.com, bp@alien8.de, rppt@kernel.org, peterz@infradead.org, pankaj.gupta@amd.com, thuth@redhat.com, andriy.shevchenko@linux.intel.com, joel.granados@kernel.org, kbingham@kernel.org, nicolas@fjasle.eu, mark.rutland@arm.com, surenb@google.com, catalin.marinas@arm.com, morbo@google.com, justinstitt@google.com, ubizjak@gmail.com, jhubbard@nvidia.com, urezki@gmail.com, dave.hansen@linux.intel.com, bhe@redhat.com, luto@kernel.org, baohua@kernel.org, nathan@kernel.org, will@kernel.org, brgerst@gmail.com Cc: llvm@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, x86@kernel.org Subject: [PATCH v3 14/14] x86: Make software tag-based kasan available Date: Fri, 4 Apr 2025 15:14:18 +0200 Message-ID: <3ed2c4baaf9b182c9d9716db95387ee14d98c99c.1743772053.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 792821C0002 X-Stat-Signature: ddzsq7a17dyru9iqrsd8gnthtztmun84 X-Rspam-User: X-HE-Tag: 1743772699-616028 X-HE-Meta: U2FsdGVkX1/xYAdiTjExYajKN3PwJ3/gn7nA3i+X1XJixhx/cvBbneX5ct+l5zma52ai0+WczJLFa/VGqaVnZJkphm7PeSgdLP+62+cE/iDQU9Bqj56YZ/MEPySAPj8+TK0AcfOAjTQafKOKgrg+6daCPaMF4HIY7OrFtZIoGONzA5S/TpPoV2exg4HqFgD02+bjT+W9VKbmYpLPYJiVQWcv9ILQUbh8DgJRtavgwTZ7NCfK8FPGg0UC+BMOqYLKuldwm5QjtBgwD6ynoqMV4MtjbGYwt1ALCL/MHKQpwXHHlKOWuOou2U1vzOONEIMd77SaBvKVpbKGWZIQzF/6OtCkZuheMppnOdJzLXLPO+DfzwWOS6pUjGiyfgvbHRBdmabTFL43yHyQC67Yu0Gz3UkXZbeTKM5eDVyru5m01u8AlEuNWtHDeY7De4EIDjCf8LVGGXmyrhGbxqZCP0w5O9QME0eN0FFaeahMBSuCHlZdyGHTfyHiyAY56QPjRjL2I7H9hWMO6ixffk208qrDZljWAf+a5WFVmq613HlE99UsUiZqBvHsvnhqgzwM48C+alOHqf+y4t50bpbZqNhpH0Lf38xyvrc4h5syUTFNOJGnwTSVAebF8xBlONPK1RGbqdGmNmvMwFDbrRrD4jGILdzYwhfr63uWFxd0D05zprw38k9Qv8lQ5/DxqG2FSbmI9Zuqodjj7fGuLOHRRr8eiWzUahOQYhIhViztzeDn9ODEAt6wwRezoTpqPy0F/XNP+jZ17MGyezw/5opaGCa6wsqrgoGvyUjA34e+bpRVpdkk1ALBmU5bZgdX0EmF+8C3MHzxuk6XxrKYCG06esgt8WP9AeVoVAize7VufTGv3kPivp+/gKVd0F8dqQrhOMSjX6Hfg4MNssMtyLYkYX+lAs2VmTci5bmgC28/Ms3j1AlGKT836+kV2qkJSUclsAVvozH6s9MlT1qjf6T2IdA bYzAJq9E iFMuFXQVMCFsVjdREcpXBnAQ58NzVBssVflD/oBhnQUIz9S/LZ7vygP2uw+f/jVmcFP7dqzUuM43PAEE5axH9xV6YlY3in5MZ7S35XQV/mLntaJPLePRRf5XlzZ/J5w6f+ZjdB8TEDZM+0Tsk08Q48AiirXjqjcSsLVgIg7P+qqFswbid3UVjxcUN+Kp0LMssTrlL5H3R72/JQd4epW2cqLcKGTx94WBPKqQZmKIn7xhjF5g9cRUapZe/MQis/nrAEl1o X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make CONFIG_KASAN_SW_TAGS available for x86 machines if they have ADDRESS_MASKING enabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform. Set scale macro based on KASAN mode: in software tag-based mode 16 bytes of memory map to one shadow byte and 8 in generic mode. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Remove runtime_const from previous patch and merge the rest here. - Move scale shift definition back to header file. - Add new kasan offset for software tag based mode. - Fix patch message typo 32 -> 16, and 16 -> 8. - Update lib/Kconfig.kasan with x86 now having software tag-based support. Changelog v2: - Remove KASAN dense code. Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 5 +++-- arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/kasan.h | 9 +++++++++ arch/x86/kernel/setup.c | 2 ++ arch/x86/mm/kasan_init_64.c | 2 +- lib/Kconfig.kasan | 3 ++- 7 files changed, 22 insertions(+), 6 deletions(-) diff --git a/Documentation/arch/x86/x86_64/mm.rst b/Documentation/arch/x86/x86_64/mm.rst index f2db178b353f..1e2d6b3ae231 100644 --- a/Documentation/arch/x86/x86_64/mm.rst +++ b/Documentation/arch/x86/x86_64/mm.rst @@ -60,7 +60,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory (generic mode) + fffff40000000000 | -8 TB | fffffc0000000000 | 8 TB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 56-bit one from here on: @@ -130,7 +131,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory (generic mode) + ffdffc0000000000 | -6 PB | ffeffc0000000000 | 4 PB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on: diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 15f346f02af0..cfe1cb15950e 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -197,6 +197,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB @@ -402,8 +403,8 @@ config AUDIT_ARCH config KASAN_SHADOW_OFFSET hex - depends on KASAN - default 0xdffffc0000000000 + default 0xdffffc0000000000 if KASAN_GENERIC + default 0xffeffc0000000000 if KASAN_SW_TAGS config HAVE_INTEL_TXT def_bool y diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index dd8d1a85f671..f6a87e9ad200 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -13,6 +13,7 @@ #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN #undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS #define __NO_FORTIFY diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 212218622963..d2eedaa092d5 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -6,8 +6,16 @@ #include #include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_SCALE_SHIFT 4 +#else #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_END KASAN_SHADOW_OFFSET +#define KASAN_SHADOW_START (KASAN_SHADOW_END - ((UL(1)) << (__VIRTUAL_MASK_SHIFT - KASAN_SHADOW_SCALE_SHIFT))) +#else /* * Compiler uses shadow offset assuming that addresses start * from 0. Kernel addresses don't start from 0, so shadow @@ -24,6 +32,7 @@ #define KASAN_SHADOW_END (KASAN_SHADOW_START + \ (1ULL << (__VIRTUAL_MASK_SHIFT - \ KASAN_SHADOW_SCALE_SHIFT))) +#endif #ifndef __ASSEMBLER__ #include diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index c7164a8de983..a40d66da69f4 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1182,6 +1182,8 @@ void __init setup_arch(char **cmdline_p) kasan_init(); + kasan_init_sw_tags(); + /* * Sync back kernel address range. * diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index e8a451cafc8c..b5cf3dca6954 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -371,7 +371,7 @@ void __init kasan_init(void) * bunch of things like kernel code, modules, EFI mapping, etc. * We need to take extra steps to not overwrite them. */ - if (pgtable_l5_enabled()) { + if (pgtable_l5_enabled() && !IS_ENABLED(CONFIG_KASAN_SW_TAGS)) { void *ptr; ptr = (void *)pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_END)); diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index f82889a830fa..9ddbc6aeb5d5 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -100,7 +100,8 @@ config KASAN_SW_TAGS Requires GCC 11+ or Clang. - Supported only on arm64 CPUs and relies on Top Byte Ignore. + Supported on arm64 CPUs that support Top Byte Ignore and on x86 CPUs + that support Linear Address Masking. Consumes about 1/16th of available memory at kernel start and add an overhead of ~20% for dynamic allocations.