From patchwork Tue Dec 22 20:00:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9196C433DB for ; Tue, 22 Dec 2020 20:00:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 56ADA20795 for ; Tue, 22 Dec 2020 20:00:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 56ADA20795 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9CF1B6B0093; Tue, 22 Dec 2020 15:00:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A40F6B0095; Tue, 22 Dec 2020 15:00:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BB596B0096; Tue, 22 Dec 2020 15:00:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id 75CE26B0093 for ; Tue, 22 Dec 2020 15:00:10 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 35AFE180AD807 for ; Tue, 22 Dec 2020 20:00:10 +0000 (UTC) X-FDA: 77621984580.05.fan83_4712d3127462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 180BA18039009 for ; Tue, 22 Dec 2020 20:00:10 +0000 (UTC) X-HE-Tag: fan83_4712d3127462 X-Filterd-Recvd-Size: 11475 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 939FB20700; Tue, 22 Dec 2020 20:00:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667208; bh=AexJ2vMYIopIJOlUAiEGDR3hvR4BxmRpff6tzMYmp+c=; h=Date:From:To:Subject:In-Reply-To:From; b=JmOCtgw4BJt5sXy/Ye6D1AaB/6XwfgmOzxOY0Qq6lQMeWxtxCucI6fqGJxVkJexVT +u0HKW1owxHPFf2EVPO/yJn7L7X680cGxCKcDQEc7WzVTaK0u+FsgK1juIYHsEW435 sjZYf7uZa4/WmB0vzlMEcLWsQ99450ZZ145MhBQA= Date: Tue, 22 Dec 2020 12:00:07 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 01/60] kasan: drop unnecessary GPL text from comment headers Message-ID: <20201222200007.xqW96_0hC%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: drop unnecessary GPL text from comment headers Patch series "kasan: add hardware tag-based mode for arm64", v11. This patchset adds a new hardware tag-based mode to KASAN [1]. The new mode is similar to the existing software tag-based KASAN, but relies on arm64 Memory Tagging Extension (MTE) [2] to perform memory and pointer tagging (instead of shadow memory and compiler instrumentation). This patchset is co-developed and tested by Vincenzo Frascino . This patchset is available here: https://github.com/xairy/linux/tree/up-kasan-mte-v11 For testing in QEMU hardware tag-based KASAN requires: 1. QEMU built from master [4] (use "-machine virt,mte=on -cpu max" arguments to run). 2. GCC version 10. [1] https://www.kernel.org/doc/html/latest/dev-tools/kasan.html [2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety [3] git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/mte [4] https://github.com/qemu/qemu ====== Overview The underlying ideas of the approach used by hardware tag-based KASAN are: 1. By relying on the Top Byte Ignore (TBI) arm64 CPU feature, pointer tags are stored in the top byte of each kernel pointer. 2. With the Memory Tagging Extension (MTE) arm64 CPU feature, memory tags for kernel memory allocations are stored in a dedicated memory not accessible via normal instuctions. 3. On each memory allocation, a random tag is generated, embedded it into the returned pointer, and the corresponding memory is tagged with the same tag value. 4. With MTE the CPU performs a check on each memory access to make sure that the pointer tag matches the memory tag. 5. On a tag mismatch the CPU generates a tag fault, and a KASAN report is printed. Same as other KASAN modes, hardware tag-based KASAN is intended as a debugging feature at this point. ====== Rationale There are two main reasons for this new hardware tag-based mode: 1. Previously implemented software tag-based KASAN is being successfully used on dogfood testing devices due to its low memory overhead (as initially planned). The new hardware mode keeps the same low memory overhead, and is expected to have significantly lower performance impact, due to the tag checks being performed by the hardware. Therefore the new mode can be used as a better alternative in dogfood testing for hardware that supports MTE. 2. The new mode lays the groundwork for the planned in-kernel MTE-based memory corruption mitigation to be used in production. ====== Technical details Considering the implementation perspective, hardware tag-based KASAN is almost identical to the software mode. The key difference is using MTE for assigning and checking tags. Compared to the software mode, the hardware mode uses 4 bits per tag, as dictated by MTE. Pointer tags are stored in bits [56:60), the top 4 bits have the normal value 0xF. Having less distict tags increases the probablity of false negatives (from ~1/256 to ~1/16) in certain cases. Only synchronous exceptions are set up and used by hardware tag-based KASAN. ====== Benchmarks Note: all measurements have been performed with software emulation of Memory Tagging Extension, performance numbers for hardware tag-based KASAN on the actual hardware are expected to be better. Boot time [1]: * 2.8 sec for clean kernel * 5.7 sec for hardware tag-based KASAN * 11.8 sec for software tag-based KASAN * 11.6 sec for generic KASAN Slab memory usage after boot [2]: * 7.0 kb for clean kernel * 9.7 kb for hardware tag-based KASAN * 9.7 kb for software tag-based KASAN * 41.3 kb for generic KASAN Measurements have been performed with: * defconfig-based configs * Manually built QEMU master * QEMU arguments: -machine virt,mte=on -cpu max * CONFIG_KASAN_STACK_ENABLE disabled * CONFIG_KASAN_INLINE enabled * clang-10 as the compiler and gcc-10 as the assembler [1] Time before the ext4 driver is initialized. [2] Measured as `cat /proc/meminfo | grep Slab`. ====== Notes The cover letter for software tag-based KASAN patchset can be found here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0116523cfffa62aeb5aa3b85ce7419f3dae0c1b8 ===== Tags Tested-by: Vincenzo Frascino This patch (of 41): Don't mention "GNU General Public License version 2" text explicitly, as it's already covered by the SPDX-License-Identifier. Link: https://lkml.kernel.org/r/cover.1606161801.git.andreyknvl@google.com Link: https://lkml.kernel.org/r/6ea9f5f4aa9dbbffa0d0c0a780b37699a4531034.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Catalin Marinas Cc: Will Deacon Cc: Dmitry Vyukov Cc: Andrey Ryabinin Cc: Evgenii Stepanov Cc: Branislav Rankov Cc: Kevin Brodsky Cc: Vasily Gorbik Signed-off-by: Andrew Morton --- mm/kasan/common.c | 5 ----- mm/kasan/generic.c | 5 ----- mm/kasan/generic_report.c | 5 ----- mm/kasan/init.c | 5 ----- mm/kasan/quarantine.c | 10 ---------- mm/kasan/report.c | 5 ----- mm/kasan/tags.c | 5 ----- mm/kasan/tags_report.c | 5 ----- 8 files changed, 45 deletions(-) --- a/mm/kasan/common.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/common.c @@ -7,11 +7,6 @@ * * Some code borrowed from https://github.com/xairy/kasan-prototype by * Andrey Konovalov - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #include --- a/mm/kasan/generic.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/generic.c @@ -7,11 +7,6 @@ * * Some code borrowed from https://github.com/xairy/kasan-prototype by * Andrey Konovalov - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt --- a/mm/kasan/generic_report.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/generic_report.c @@ -7,11 +7,6 @@ * * Some code borrowed from https://github.com/xairy/kasan-prototype by * Andrey Konovalov - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #include --- a/mm/kasan/init.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/init.c @@ -4,11 +4,6 @@ * * Copyright (c) 2015 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #include --- a/mm/kasan/quarantine.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/quarantine.c @@ -6,16 +6,6 @@ * Copyright (C) 2016 Google, Inc. * * Based on code by Dmitry Chernenkov. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License - * version 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License for more details. - * */ #include --- a/mm/kasan/report.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/report.c @@ -7,11 +7,6 @@ * * Some code borrowed from https://github.com/xairy/kasan-prototype by * Andrey Konovalov - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #include --- a/mm/kasan/tags.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/tags.c @@ -4,11 +4,6 @@ * * Copyright (c) 2018 Google, Inc. * Author: Andrey Konovalov - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt --- a/mm/kasan/tags_report.c~kasan-drop-unnecessary-gpl-text-from-comment-headers +++ a/mm/kasan/tags_report.c @@ -7,11 +7,6 @@ * * Some code borrowed from https://github.com/xairy/kasan-prototype by * Andrey Konovalov - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * */ #include From patchwork Tue Dec 22 20:00:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27348C433E0 for ; Tue, 22 Dec 2020 20:00:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B22520867 for ; Tue, 22 Dec 2020 20:00:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B22520867 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0A6996B0096; Tue, 22 Dec 2020 15:00:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 007DC6B0098; Tue, 22 Dec 2020 15:00:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC3DB6B0099; Tue, 22 Dec 2020 15:00:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id BE9BE6B0096 for ; Tue, 22 Dec 2020 15:00:13 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 848AB1F08 for ; Tue, 22 Dec 2020 20:00:13 +0000 (UTC) X-FDA: 77621984706.16.lip20_40028a227462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 56703100E6903 for ; Tue, 22 Dec 2020 20:00:13 +0000 (UTC) X-HE-Tag: lip20_40028a227462 X-Filterd-Recvd-Size: 2941 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3545220705; Tue, 22 Dec 2020 20:00:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667211; bh=iEJgPSTIOZrOrqUbae8/LF4D4oibSp4dMwOwMLPsWWM=; h=Date:From:To:Subject:In-Reply-To:From; b=ye1GRVt20qp6D4DEQRibjAvuLEY/46Eh8rix2K8/qzXjlDthqRiF9A6+JgeQ7GeGH sWb8Vb9gNHDvEZXT8jnh8IS/3NkaDQdX4tJ0itXcKh+zKZIrU0L/D1iGU+CMRtbvim Q5Wzy/yBCRiNxBa6aEKk71ULbVkRrf81p0HtvF5c= Date: Tue, 22 Dec 2020 12:00:10 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 02/60] kasan: KASAN_VMALLOC depends on KASAN_GENERIC Message-ID: <20201222200010.DCyKVfsxL%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: KASAN_VMALLOC depends on KASAN_GENERIC Currently only generic KASAN mode supports vmalloc, reflect that in the config. Link: https://lkml.kernel.org/r/0c493d3a065ad95b04313d00244e884a7e2498ff.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/Kconfig.kasan | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/lib/Kconfig.kasan~kasan-kasan_vmalloc-depends-on-kasan_generic +++ a/lib/Kconfig.kasan @@ -146,7 +146,7 @@ config KASAN_SW_TAGS_IDENTIFY config KASAN_VMALLOC bool "Back mappings in vmalloc space with real shadow memory" - depends on HAVE_ARCH_KASAN_VMALLOC + depends on KASAN_GENERIC && HAVE_ARCH_KASAN_VMALLOC help By default, the shadow region for vmalloc space is the read-only zero page. This means that KASAN cannot detect errors involving From patchwork Tue Dec 22 20:00:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8F99C433E0 for ; Tue, 22 Dec 2020 20:00:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8D1F222ADC for ; Tue, 22 Dec 2020 20:00:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D1F222ADC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 780F26B00A7; Tue, 22 Dec 2020 15:00:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70BDF6B00AC; Tue, 22 Dec 2020 15:00:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50E626B00A8; Tue, 22 Dec 2020 15:00:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id 2AC876B00AC for ; Tue, 22 Dec 2020 15:00:34 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D5B54363D for ; Tue, 22 Dec 2020 20:00:33 +0000 (UTC) X-FDA: 77621985546.12.grape80_520d94c27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id DAB3A1802B9F0 for ; Tue, 22 Dec 2020 20:00:17 +0000 (UTC) X-HE-Tag: grape80_520d94c27462 X-Filterd-Recvd-Size: 7967 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B57F221D1B; Tue, 22 Dec 2020 20:00:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667215; bh=xVl2M5glN7HCbaxCh+UhQW7KpJYvRw/xwxfZzL4AlqA=; h=Date:From:To:Subject:In-Reply-To:From; b=tsqVdllC4p47vINUwyjeBm2dZxyo9lpylLleOpYKptdA9EAMuoOiQFx4zURLbwlJR WSLZstVRNoSogLUedwjBK1Eb8IXiykruY5/4jPCsY9zhbKfFqP6hLLHWKkJhDB4qkL Y7acB1r8H8KxJfEU3r0W3u7ebp+QQ8N3WoeqaXeo= Date: Tue, 22 Dec 2020 12:00:14 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 03/60] kasan: group vmalloc code Message-ID: <20201222200014.SG0vtFsQD%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: group vmalloc code This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Group all vmalloc-related function declarations in include/linux/kasan.h, and their implementations in mm/kasan/common.c. No functional changes. Link: https://lkml.kernel.org/r/80a6fdd29b039962843bd6cf22ce2643a7c8904e.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 41 +++++++++++---------- mm/kasan/common.c | 78 ++++++++++++++++++++-------------------- 2 files changed, 63 insertions(+), 56 deletions(-) --- a/include/linux/kasan.h~kasan-group-vmalloc-code +++ a/include/linux/kasan.h @@ -75,19 +75,6 @@ struct kasan_cache { int free_meta_offset; }; -/* - * These functions provide a special case to support backing module - * allocations with real shadow memory. With KASAN vmalloc, the special - * case is unnecessary, as the work is handled in the generic case. - */ -#ifndef CONFIG_KASAN_VMALLOC -int kasan_module_alloc(void *addr, size_t size); -void kasan_free_shadow(const struct vm_struct *vm); -#else -static inline int kasan_module_alloc(void *addr, size_t size) { return 0; } -static inline void kasan_free_shadow(const struct vm_struct *vm) {} -#endif - int kasan_add_zero_shadow(void *start, unsigned long size); void kasan_remove_zero_shadow(void *start, unsigned long size); @@ -156,9 +143,6 @@ static inline bool kasan_slab_free(struc return false; } -static inline int kasan_module_alloc(void *addr, size_t size) { return 0; } -static inline void kasan_free_shadow(const struct vm_struct *vm) {} - static inline int kasan_add_zero_shadow(void *start, unsigned long size) { return 0; @@ -211,13 +195,16 @@ static inline void *kasan_reset_tag(cons #endif /* CONFIG_KASAN_SW_TAGS */ #ifdef CONFIG_KASAN_VMALLOC + int kasan_populate_vmalloc(unsigned long addr, unsigned long size); void kasan_poison_vmalloc(const void *start, unsigned long size); void kasan_unpoison_vmalloc(const void *start, unsigned long size); void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, unsigned long free_region_end); -#else + +#else /* CONFIG_KASAN_VMALLOC */ + static inline int kasan_populate_vmalloc(unsigned long start, unsigned long size) { @@ -232,7 +219,25 @@ static inline void kasan_release_vmalloc unsigned long end, unsigned long free_region_start, unsigned long free_region_end) {} -#endif + +#endif /* CONFIG_KASAN_VMALLOC */ + +#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC) + +/* + * These functions provide a special case to support backing module + * allocations with real shadow memory. With KASAN vmalloc, the special + * case is unnecessary, as the work is handled in the generic case. + */ +int kasan_module_alloc(void *addr, size_t size); +void kasan_free_shadow(const struct vm_struct *vm); + +#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */ + +static inline int kasan_module_alloc(void *addr, size_t size) { return 0; } +static inline void kasan_free_shadow(const struct vm_struct *vm) {} + +#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */ #ifdef CONFIG_KASAN_INLINE void kasan_non_canonical_hook(unsigned long addr); --- a/mm/kasan/common.c~kasan-group-vmalloc-code +++ a/mm/kasan/common.c @@ -536,44 +536,6 @@ void kasan_kfree_large(void *ptr, unsign /* The object will be poisoned by page_alloc. */ } -#ifndef CONFIG_KASAN_VMALLOC -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - __memset(ret, KASAN_SHADOW_INIT, shadow_size); - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} -#endif - #ifdef CONFIG_MEMORY_HOTPLUG static bool shadow_mapped(unsigned long addr) { @@ -685,6 +647,7 @@ core_initcall(kasan_memhotplug_init); #endif #ifdef CONFIG_KASAN_VMALLOC + static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, void *unused) { @@ -923,4 +886,43 @@ void kasan_release_vmalloc(unsigned long (unsigned long)shadow_end); } } + +#else /* CONFIG_KASAN_VMALLOC */ + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + #endif From patchwork Tue Dec 22 20:00:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACE2BC433DB for ; Tue, 22 Dec 2020 20:00:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 42C5620867 for ; Tue, 22 Dec 2020 20:00:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 42C5620867 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D036E6B0099; Tue, 22 Dec 2020 15:00:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB0FC6B009E; Tue, 22 Dec 2020 15:00:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B79D86B00A2; Tue, 22 Dec 2020 15:00:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 99FEE6B0099 for ; Tue, 22 Dec 2020 15:00:20 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 4518E3638 for ; Tue, 22 Dec 2020 20:00:20 +0000 (UTC) X-FDA: 77621985000.10.push06_221069627462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 23BC316A0C3 for ; Tue, 22 Dec 2020 20:00:20 +0000 (UTC) X-HE-Tag: push06_221069627462 X-Filterd-Recvd-Size: 5757 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 26D2E20771; Tue, 22 Dec 2020 20:00:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667218; bh=Tbjr1ls2W/E9ex/nUEg5tpPEJIGUedvscg041utElv8=; h=Date:From:To:Subject:In-Reply-To:From; b=pTH1XNuQpQOBuYB65Qc2IbCyTxFR+s6Eig+6SvwXAXWTCmcoua4D6PZtVpkgrP6zb Jg61w0lAVeDaA2bCyy/shs1NgHeGDVt23F31jN6lAXw7HBkSeA1MKSwPjP2FkquuEo 35hqGwCRe/K+VGAYvox7OLlDIIkACEyNV3sZxc3s= Date: Tue, 22 Dec 2020 12:00:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, valdis.kletnieks@vt.edu, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 04/60] kasan: shadow declarations only for software modes Message-ID: <20201222200017.5F-9o0NKu%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: shadow declarations only for software modes This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Group shadow-related KASAN function declarations and only define them for the two existing software modes. No functional changes for software modes. Link: https://lkml.kernel.org/r/e88d94eff94db883a65dca52e1736d80d28dd9bc.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Signed-off-by: Valdis Kletnieks Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon [valdis.kletnieks@vt.edu: fix build issue with asmlinkage] Link: https://lkml.kernel.org/r/35126.1606402815@turing-police Link: https://lore.kernel.org/linux-arm-kernel/24105.1606397102@turing-police/ Signed-off-by: Andrew Morton --- include/linux/kasan.h | 48 ++++++++++++++++++++++++++-------------- 1 file changed, 32 insertions(+), 16 deletions(-) --- a/include/linux/kasan.h~kasan-shadow-declarations-only-for-software-modes +++ a/include/linux/kasan.h @@ -11,7 +11,7 @@ struct task_struct; #ifdef CONFIG_KASAN -#include +#include #include /* kasan_data struct is used in KUnit tests for KASAN expected failures */ @@ -20,6 +20,20 @@ struct kunit_kasan_expectation { bool report_found; }; +#endif + +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) + +#include + +/* Software KASAN implementations use shadow memory. */ + +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_INIT 0xFF +#else +#define KASAN_SHADOW_INIT 0 +#endif + extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE]; extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD]; @@ -35,6 +49,23 @@ static inline void *kasan_mem_to_shadow( + KASAN_SHADOW_OFFSET; } +int kasan_add_zero_shadow(void *start, unsigned long size); +void kasan_remove_zero_shadow(void *start, unsigned long size); + +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + +static inline int kasan_add_zero_shadow(void *start, unsigned long size) +{ + return 0; +} +static inline void kasan_remove_zero_shadow(void *start, + unsigned long size) +{} + +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + +#ifdef CONFIG_KASAN + /* Enable reporting bugs after kasan_disable_current() */ extern void kasan_enable_current(void); @@ -75,9 +106,6 @@ struct kasan_cache { int free_meta_offset; }; -int kasan_add_zero_shadow(void *start, unsigned long size); -void kasan_remove_zero_shadow(void *start, unsigned long size); - size_t __ksize(const void *); static inline void kasan_unpoison_slab(const void *ptr) { @@ -143,14 +171,6 @@ static inline bool kasan_slab_free(struc return false; } -static inline int kasan_add_zero_shadow(void *start, unsigned long size) -{ - return 0; -} -static inline void kasan_remove_zero_shadow(void *start, - unsigned long size) -{} - static inline void kasan_unpoison_slab(const void *ptr) { } static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } @@ -158,8 +178,6 @@ static inline size_t kasan_metadata_size #ifdef CONFIG_KASAN_GENERIC -#define KASAN_SHADOW_INIT 0 - void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_record_aux_stack(void *ptr); @@ -174,8 +192,6 @@ static inline void kasan_record_aux_stac #ifdef CONFIG_KASAN_SW_TAGS -#define KASAN_SHADOW_INIT 0xFF - void kasan_init_tags(void); void *kasan_reset_tag(const void *addr); From patchwork Tue Dec 22 20:00:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 288CFC433E0 for ; Tue, 22 Dec 2020 20:00:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B543A20795 for ; Tue, 22 Dec 2020 20:00:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B543A20795 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 560166B00A2; Tue, 22 Dec 2020 15:00:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5102E6B00A3; Tue, 22 Dec 2020 15:00:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D7DE6B00A4; Tue, 22 Dec 2020 15:00:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0219.hostedemail.com [216.40.44.219]) by kanga.kvack.org (Postfix) with ESMTP id 1A43F6B00A2 for ; Tue, 22 Dec 2020 15:00:24 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D08CA1EE6 for ; Tue, 22 Dec 2020 20:00:23 +0000 (UTC) X-FDA: 77621985126.26.tiger75_1b13ca327462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id B490E1804B65C for ; Tue, 22 Dec 2020 20:00:23 +0000 (UTC) X-HE-Tag: tiger75_1b13ca327462 X-Filterd-Recvd-Size: 13334 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A25CA21D81; Tue, 22 Dec 2020 20:00:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667222; bh=w9jd/ejKZA/3A39DIjQfsxAp9CDDf0bS76vUXyt2lvg=; h=Date:From:To:Subject:In-Reply-To:From; b=EpNhyFRpeTz400nmBiNNi6+fsWeEkNwCIkPiXZUXpI5jFKaI694xElgcA6NHuG/bM UPUPT2hCSkmYTRSlbvJO7CPXEAs+Lv+YKU/c0zjQgTcgUYa14laMkQi+caGW6J4vo9 SEO78JHHvbXPUxCmjp+FTbKBbI5fR8C+mNiFgO74= Date: Tue, 22 Dec 2020 12:00:21 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 05/60] kasan: rename (un)poison_shadow to (un)poison_range Message-ID: <20201222200021.iTDQ9xiWD%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename (un)poison_shadow to (un)poison_range This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. The new mode won't be using shadow memory. Rename external annotation kasan_unpoison_shadow() to kasan_unpoison_range(), and introduce internal functions (un)poison_range() (without kasan_ prefix). Co-developed-by: Marco Elver Link: https://lkml.kernel.org/r/fccdcaa13dc6b2211bf363d6c6d499279a54fe3a.1606161801.git.andreyknvl@google.com Signed-off-by: Marco Elver Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 6 ++-- kernel/fork.c | 4 +-- mm/kasan/common.c | 49 ++++++++++++++++++++++------------------ mm/kasan/generic.c | 23 ++++++++---------- mm/kasan/kasan.h | 3 +- mm/kasan/tags.c | 2 - mm/slab_common.c | 2 - 7 files changed, 47 insertions(+), 42 deletions(-) --- a/include/linux/kasan.h~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/include/linux/kasan.h @@ -72,7 +72,7 @@ extern void kasan_enable_current(void); /* Disable reporting bugs for current task */ extern void kasan_disable_current(void); -void kasan_unpoison_shadow(const void *address, size_t size); +void kasan_unpoison_range(const void *address, size_t size); void kasan_unpoison_task_stack(struct task_struct *task); @@ -109,7 +109,7 @@ struct kasan_cache { size_t __ksize(const void *); static inline void kasan_unpoison_slab(const void *ptr) { - kasan_unpoison_shadow(ptr, __ksize(ptr)); + kasan_unpoison_range(ptr, __ksize(ptr)); } size_t kasan_metadata_size(struct kmem_cache *cache); @@ -118,7 +118,7 @@ void kasan_restore_multi_shot(bool enabl #else /* CONFIG_KASAN */ -static inline void kasan_unpoison_shadow(const void *address, size_t size) {} +static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_unpoison_task_stack(struct task_struct *task) {} --- a/kernel/fork.c~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/kernel/fork.c @@ -225,8 +225,8 @@ static unsigned long *alloc_thread_stack if (!s) continue; - /* Clear the KASAN shadow of the stack. */ - kasan_unpoison_shadow(s->addr, THREAD_SIZE); + /* Mark stack accessible for KASAN. */ + kasan_unpoison_range(s->addr, THREAD_SIZE); /* Clear stale pointers from reused stack. */ memset(s->addr, 0, THREAD_SIZE); --- a/mm/kasan/common.c~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/mm/kasan/common.c @@ -108,7 +108,7 @@ void *memcpy(void *dest, const void *src * Poisons the shadow memory for 'size' bytes starting from 'addr'. * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. */ -void kasan_poison_shadow(const void *address, size_t size, u8 value) +void poison_range(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; @@ -125,7 +125,7 @@ void kasan_poison_shadow(const void *add __memset(shadow_start, value, shadow_end - shadow_start); } -void kasan_unpoison_shadow(const void *address, size_t size) +void unpoison_range(const void *address, size_t size) { u8 tag = get_tag(address); @@ -136,7 +136,7 @@ void kasan_unpoison_shadow(const void *a */ address = reset_tag(address); - kasan_poison_shadow(address, size, tag); + poison_range(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); @@ -148,12 +148,17 @@ void kasan_unpoison_shadow(const void *a } } +void kasan_unpoison_range(const void *address, size_t size) +{ + unpoison_range(address, size); +} + static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) { void *base = task_stack_page(task); size_t size = sp - base; - kasan_unpoison_shadow(base, size); + unpoison_range(base, size); } /* Unpoison the entire stack for a task. */ @@ -172,7 +177,7 @@ asmlinkage void kasan_unpoison_task_stac */ void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - kasan_unpoison_shadow(base, watermark - base); + unpoison_range(base, watermark - base); } void kasan_alloc_pages(struct page *page, unsigned int order) @@ -186,13 +191,13 @@ void kasan_alloc_pages(struct page *page tag = random_tag(); for (i = 0; i < (1 << order); i++) page_kasan_tag_set(page + i, tag); - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + unpoison_range(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) { if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), + poison_range(page_address(page), PAGE_SIZE << order, KASAN_FREE_PAGE); } @@ -284,18 +289,18 @@ void kasan_poison_slab(struct page *page for (i = 0; i < compound_nr(page); i++) page_kasan_tag_reset(page + i); - kasan_poison_shadow(page_address(page), page_size(page), - KASAN_KMALLOC_REDZONE); + poison_range(page_address(page), page_size(page), + KASAN_KMALLOC_REDZONE); } void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) { - kasan_unpoison_shadow(object, cache->object_size); + unpoison_range(object, cache->object_size); } void kasan_poison_object_data(struct kmem_cache *cache, void *object) { - kasan_poison_shadow(object, + poison_range(object, round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), KASAN_KMALLOC_REDZONE); } @@ -408,7 +413,7 @@ static bool __kasan_slab_free(struct kme } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE); if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || unlikely(!(cache->flags & SLAB_KASAN))) @@ -448,9 +453,9 @@ static void *__kasan_kmalloc(struct kmem tag = assign_tag(cache, object, false, keep_tag); /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ - kasan_unpoison_shadow(set_tag(object, tag), size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); + unpoison_range(set_tag(object, tag), size); + poison_range((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags); @@ -489,9 +494,9 @@ void * __must_check kasan_kmalloc_large( KASAN_SHADOW_SCALE_SIZE); redzone_end = (unsigned long)ptr + page_size(page); - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); + unpoison_range(ptr, size); + poison_range((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); return (void *)ptr; } @@ -523,7 +528,7 @@ void kasan_poison_kfree(void *ptr, unsig kasan_report_invalid_free(ptr, ip); return; } - kasan_poison_shadow(ptr, page_size(page), KASAN_FREE_PAGE); + poison_range(ptr, page_size(page), KASAN_FREE_PAGE); } else { __kasan_slab_free(page->slab_cache, ptr, ip, false); } @@ -709,7 +714,7 @@ int kasan_populate_vmalloc(unsigned long * // vmalloc() allocates memory * // let a = area->addr * // we reach kasan_populate_vmalloc - * // and call kasan_unpoison_shadow: + * // and call unpoison_range: * STORE shadow(a), unpoison_val * ... * STORE shadow(a+99), unpoison_val x = LOAD p @@ -744,7 +749,7 @@ void kasan_poison_vmalloc(const void *st return; size = round_up(size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(start, size, KASAN_VMALLOC_INVALID); + poison_range(start, size, KASAN_VMALLOC_INVALID); } void kasan_unpoison_vmalloc(const void *start, unsigned long size) @@ -752,7 +757,7 @@ void kasan_unpoison_vmalloc(const void * if (!is_vmalloc_or_module_addr(start)) return; - kasan_unpoison_shadow(start, size); + unpoison_range(start, size); } static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, --- a/mm/kasan/generic.c~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/mm/kasan/generic.c @@ -202,11 +202,11 @@ static void register_global(struct kasan { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(global->beg, global->size); + unpoison_range(global->beg, global->size); - kasan_poison_shadow(global->beg + aligned_size, - global->size_with_redzone - aligned_size, - KASAN_GLOBAL_REDZONE); + poison_range(global->beg + aligned_size, + global->size_with_redzone - aligned_size, + KASAN_GLOBAL_REDZONE); } void __asan_register_globals(struct kasan_global *globals, size_t size) @@ -285,13 +285,12 @@ void __asan_alloca_poison(unsigned long WARN_ON(!IS_ALIGNED(addr, KASAN_ALLOCA_REDZONE_SIZE)); - kasan_unpoison_shadow((const void *)(addr + rounded_down_size), - size - rounded_down_size); - kasan_poison_shadow(left_redzone, KASAN_ALLOCA_REDZONE_SIZE, - KASAN_ALLOCA_LEFT); - kasan_poison_shadow(right_redzone, - padding_size + KASAN_ALLOCA_REDZONE_SIZE, - KASAN_ALLOCA_RIGHT); + unpoison_range((const void *)(addr + rounded_down_size), + size - rounded_down_size); + poison_range(left_redzone, KASAN_ALLOCA_REDZONE_SIZE, + KASAN_ALLOCA_LEFT); + poison_range(right_redzone, padding_size + KASAN_ALLOCA_REDZONE_SIZE, + KASAN_ALLOCA_RIGHT); } EXPORT_SYMBOL(__asan_alloca_poison); @@ -301,7 +300,7 @@ void __asan_allocas_unpoison(const void if (unlikely(!stack_top || stack_top > stack_bottom)) return; - kasan_unpoison_shadow(stack_top, stack_bottom - stack_top); + unpoison_range(stack_top, stack_bottom - stack_top); } EXPORT_SYMBOL(__asan_allocas_unpoison); --- a/mm/kasan/kasan.h~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/mm/kasan/kasan.h @@ -150,7 +150,8 @@ static inline bool addr_has_shadow(const return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); } -void kasan_poison_shadow(const void *address, size_t size, u8 value); +void poison_range(const void *address, size_t size, u8 value); +void unpoison_range(const void *address, size_t size); /** * check_memory_region - Check memory region, and report if invalid access. --- a/mm/kasan/tags.c~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/mm/kasan/tags.c @@ -153,7 +153,7 @@ EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { - kasan_poison_shadow((void *)addr, size, tag); + poison_range((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); --- a/mm/slab_common.c~kasan-rename-unpoison_shadow-to-unpoison_range +++ a/mm/slab_common.c @@ -1176,7 +1176,7 @@ size_t ksize(const void *objp) * We assume that ksize callers could use whole allocated area, * so we need to unpoison this area. */ - kasan_unpoison_shadow(objp, size); + kasan_unpoison_range(objp, size); return size; } EXPORT_SYMBOL(ksize); From patchwork Tue Dec 22 20:00:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5601EC433DB for ; Tue, 22 Dec 2020 20:00:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BE31F221FF for ; Tue, 22 Dec 2020 20:00:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE31F221FF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 510FE6B00AF; Tue, 22 Dec 2020 15:00:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 49C3D6B00AE; Tue, 22 Dec 2020 15:00:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3210B6B00A7; Tue, 22 Dec 2020 15:00:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0005.hostedemail.com [216.40.44.5]) by kanga.kvack.org (Postfix) with ESMTP id 0FD316B00A7 for ; Tue, 22 Dec 2020 15:00:34 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B9B543638 for ; Tue, 22 Dec 2020 20:00:33 +0000 (UTC) X-FDA: 77621985546.01.turn74_5e083ab27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 408351004EF71 for ; Tue, 22 Dec 2020 20:00:27 +0000 (UTC) X-HE-Tag: turn74_5e083ab27462 X-Filterd-Recvd-Size: 16796 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1AB8520867; Tue, 22 Dec 2020 20:00:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667225; bh=NfBmqfB1pNKkD1p9gAgulCyTNRF42GVopbp9die7ir8=; h=Date:From:To:Subject:In-Reply-To:From; b=1P3BIcO5tZ+5sYSEPavHSlVcDbVhqHZJ2Jlb0xNw8eOd35FYfEH3mkFz48xt1yadi Ww5wuVvQfIRqY3N7Zxyrmj+dTGIPtbDTwlPZ7CI/rWnhVUQ1r6nHOrSwIN7dCHEi7K 7zFa0f9QSnDsShFmy4DHoCpkMObkqftXbgu5bYT4= Date: Tue, 22 Dec 2020 12:00:24 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 06/60] kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* Message-ID: <20201222200024.QWUaQaZDF%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. The new mode won't be using shadow memory, but will still use the concept of memory granules. Each memory granule maps to a single metadata entry: 8 bytes per one shadow byte for generic mode, 16 bytes per one shadow byte for software tag-based mode, and 16 bytes per one allocation tag for hardware tag-based mode. Rename KASAN_SHADOW_SCALE_SIZE to KASAN_GRANULE_SIZE, and KASAN_SHADOW_MASK to KASAN_GRANULE_MASK. Also use MASK when used as a mask, otherwise use SIZE. No functional changes. Link: https://lkml.kernel.org/r/939b5754e47f528a6e6a6f28ffc5815d8d128033.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/dev-tools/kasan.rst | 2 - lib/test_kasan.c | 2 - lib/test_kasan_module.c | 2 - mm/kasan/common.c | 39 ++++++++++++++-------------- mm/kasan/generic.c | 14 +++++----- mm/kasan/generic_report.c | 8 ++--- mm/kasan/init.c | 8 ++--- mm/kasan/kasan.h | 4 +- mm/kasan/report.c | 10 +++---- mm/kasan/tags_report.c | 2 - 10 files changed, 46 insertions(+), 45 deletions(-) --- a/Documentation/dev-tools/kasan.rst~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/Documentation/dev-tools/kasan.rst @@ -265,7 +265,7 @@ Most mappings in vmalloc space are small page of shadow space. Allocating a full shadow page per mapping would therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to -``KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE``. +``KASAN_GRANULE_SIZE * PAGE_SIZE``. Instead, we share backing space across multiple mappings. We allocate a backing page when a mapping in vmalloc space uses a particular page --- a/lib/test_kasan.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/lib/test_kasan.c @@ -25,7 +25,7 @@ #include "../mm/kasan/kasan.h" -#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE) +#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE) /* * We assign some test results to these globals to make sure the tests --- a/lib/test_kasan_module.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/lib/test_kasan_module.c @@ -15,7 +15,7 @@ #include "../mm/kasan/kasan.h" -#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_SHADOW_SCALE_SIZE) +#define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE) static noinline void __init copy_user_test(void) { --- a/mm/kasan/common.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/common.c @@ -106,7 +106,7 @@ void *memcpy(void *dest, const void *src /* * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + * Memory addresses should be aligned to KASAN_GRANULE_SIZE. */ void poison_range(const void *address, size_t size, u8 value) { @@ -138,13 +138,13 @@ void unpoison_range(const void *address, poison_range(address, size, tag); - if (size & KASAN_SHADOW_MASK) { + if (size & KASAN_GRANULE_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) *shadow = tag; else - *shadow = size & KASAN_SHADOW_MASK; + *shadow = size & KASAN_GRANULE_MASK; } } @@ -301,7 +301,7 @@ void kasan_unpoison_object_data(struct k void kasan_poison_object_data(struct kmem_cache *cache, void *object) { poison_range(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + round_up(cache->object_size, KASAN_GRANULE_SIZE), KASAN_KMALLOC_REDZONE); } @@ -373,7 +373,7 @@ static inline bool shadow_invalid(u8 tag { if (IS_ENABLED(CONFIG_KASAN_GENERIC)) return shadow_byte < 0 || - shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + shadow_byte >= KASAN_GRANULE_SIZE; /* else CONFIG_KASAN_SW_TAGS: */ if ((u8)shadow_byte == KASAN_TAG_INVALID) @@ -412,7 +412,7 @@ static bool __kasan_slab_free(struct kme return true; } - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE); if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || @@ -445,9 +445,9 @@ static void *__kasan_kmalloc(struct kmem return NULL; redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); + KASAN_GRANULE_SIZE); redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); + KASAN_GRANULE_SIZE); if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) tag = assign_tag(cache, object, false, keep_tag); @@ -491,7 +491,7 @@ void * __must_check kasan_kmalloc_large( page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); + KASAN_GRANULE_SIZE); redzone_end = (unsigned long)ptr + page_size(page); unpoison_range(ptr, size); @@ -589,8 +589,8 @@ static int __meminit kasan_mem_notifier( shadow_size = nr_shadow_pages << PAGE_SHIFT; shadow_end = shadow_start + shadow_size; - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) || + WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT))) return NOTIFY_BAD; switch (action) { @@ -748,7 +748,7 @@ void kasan_poison_vmalloc(const void *st if (!is_vmalloc_or_module_addr(start)) return; - size = round_up(size, KASAN_SHADOW_SCALE_SIZE); + size = round_up(size, KASAN_GRANULE_SIZE); poison_range(start, size, KASAN_VMALLOC_INVALID); } @@ -861,22 +861,22 @@ void kasan_release_vmalloc(unsigned long unsigned long region_start, region_end; unsigned long size; - region_start = ALIGN(start, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); - region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE); + region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE); free_region_start = ALIGN(free_region_start, - PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + PAGE_SIZE * KASAN_GRANULE_SIZE); if (start != region_start && free_region_start < region_start) - region_start -= PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE; + region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE; free_region_end = ALIGN_DOWN(free_region_end, - PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + PAGE_SIZE * KASAN_GRANULE_SIZE); if (end != region_end && free_region_end > region_end) - region_end += PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE; + region_end += PAGE_SIZE * KASAN_GRANULE_SIZE; shadow_start = kasan_mem_to_shadow((void *)region_start); shadow_end = kasan_mem_to_shadow((void *)region_end); @@ -902,7 +902,8 @@ int kasan_module_alloc(void *addr, size_ unsigned long shadow_start; shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + scaled_size = (size + KASAN_GRANULE_SIZE - 1) >> + KASAN_SHADOW_SCALE_SHIFT; shadow_size = round_up(scaled_size, PAGE_SIZE); if (WARN_ON(!PAGE_ALIGNED(shadow_start))) --- a/mm/kasan/generic.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/generic.c @@ -46,7 +46,7 @@ static __always_inline bool memory_is_po s8 shadow_value = *(s8 *)kasan_mem_to_shadow((void *)addr); if (unlikely(shadow_value)) { - s8 last_accessible_byte = addr & KASAN_SHADOW_MASK; + s8 last_accessible_byte = addr & KASAN_GRANULE_MASK; return unlikely(last_accessible_byte >= shadow_value); } @@ -62,7 +62,7 @@ static __always_inline bool memory_is_po * Access crosses 8(shadow size)-byte boundary. Such access maps * into 2 shadow bytes, so we need to check them both. */ - if (unlikely(((addr + size - 1) & KASAN_SHADOW_MASK) < size - 1)) + if (unlikely(((addr + size - 1) & KASAN_GRANULE_MASK) < size - 1)) return *shadow_addr || memory_is_poisoned_1(addr + size - 1); return memory_is_poisoned_1(addr + size - 1); @@ -73,7 +73,7 @@ static __always_inline bool memory_is_po u16 *shadow_addr = (u16 *)kasan_mem_to_shadow((void *)addr); /* Unaligned 16-bytes access maps into 3 shadow bytes. */ - if (unlikely(!IS_ALIGNED(addr, KASAN_SHADOW_SCALE_SIZE))) + if (unlikely(!IS_ALIGNED(addr, KASAN_GRANULE_SIZE))) return *shadow_addr || memory_is_poisoned_1(addr + 15); return *shadow_addr; @@ -134,7 +134,7 @@ static __always_inline bool memory_is_po s8 *last_shadow = (s8 *)kasan_mem_to_shadow((void *)last_byte); if (unlikely(ret != (unsigned long)last_shadow || - ((long)(last_byte & KASAN_SHADOW_MASK) >= *last_shadow))) + ((long)(last_byte & KASAN_GRANULE_MASK) >= *last_shadow))) return true; } return false; @@ -200,7 +200,7 @@ void kasan_cache_shutdown(struct kmem_ca static void register_global(struct kasan_global *global) { - size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); + size_t aligned_size = round_up(global->size, KASAN_GRANULE_SIZE); unpoison_range(global->beg, global->size); @@ -274,10 +274,10 @@ EXPORT_SYMBOL(__asan_handle_no_return); /* Emitted by compiler to poison alloca()ed objects. */ void __asan_alloca_poison(unsigned long addr, size_t size) { - size_t rounded_up_size = round_up(size, KASAN_SHADOW_SCALE_SIZE); + size_t rounded_up_size = round_up(size, KASAN_GRANULE_SIZE); size_t padding_size = round_up(size, KASAN_ALLOCA_REDZONE_SIZE) - rounded_up_size; - size_t rounded_down_size = round_down(size, KASAN_SHADOW_SCALE_SIZE); + size_t rounded_down_size = round_down(size, KASAN_GRANULE_SIZE); const void *left_redzone = (const void *)(addr - KASAN_ALLOCA_REDZONE_SIZE); --- a/mm/kasan/generic_report.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/generic_report.c @@ -34,7 +34,7 @@ void *find_first_bad_addr(void *addr, si void *p = addr; while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p))) - p += KASAN_SHADOW_SCALE_SIZE; + p += KASAN_GRANULE_SIZE; return p; } @@ -46,14 +46,14 @@ static const char *get_shadow_bug_type(s shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * If shadow byte value is in [0, KASAN_GRANULE_SIZE) we can look * at the next shadow byte to determine the type of the bad access. */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + if (*shadow_addr > 0 && *shadow_addr <= KASAN_GRANULE_SIZE - 1) shadow_addr++; switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + case 0 ... KASAN_GRANULE_SIZE - 1: /* * In theory it's still possible to see these shadow values * due to a data race in the kernel code. --- a/mm/kasan/init.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/init.c @@ -442,8 +442,8 @@ void kasan_remove_zero_shadow(void *star end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT); if (WARN_ON((unsigned long)start % - (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) || - WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE))) + (KASAN_GRANULE_SIZE * PAGE_SIZE)) || + WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE))) return; for (; addr < end; addr = next) { @@ -477,8 +477,8 @@ int kasan_add_zero_shadow(void *start, u shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT); if (WARN_ON((unsigned long)start % - (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE)) || - WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE))) + (KASAN_GRANULE_SIZE * PAGE_SIZE)) || + WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE))) return -EINVAL; ret = kasan_populate_early_shadow(shadow_start, shadow_end); --- a/mm/kasan/kasan.h~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/kasan.h @@ -5,8 +5,8 @@ #include #include -#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) -#define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) +#define KASAN_GRANULE_MASK (KASAN_GRANULE_SIZE - 1) #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ --- a/mm/kasan/report.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/report.c @@ -314,24 +314,24 @@ static bool __must_check get_address_sta return false; aligned_addr = round_down((unsigned long)addr, sizeof(long)); - mem_ptr = round_down(aligned_addr, KASAN_SHADOW_SCALE_SIZE); + mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE); shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr); shadow_bottom = kasan_mem_to_shadow(end_of_stack(current)); while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) { shadow_ptr--; - mem_ptr -= KASAN_SHADOW_SCALE_SIZE; + mem_ptr -= KASAN_GRANULE_SIZE; } while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) { shadow_ptr--; - mem_ptr -= KASAN_SHADOW_SCALE_SIZE; + mem_ptr -= KASAN_GRANULE_SIZE; } if (shadow_ptr < shadow_bottom) return false; - frame = (const unsigned long *)(mem_ptr + KASAN_SHADOW_SCALE_SIZE); + frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE); if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) { pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n", frame[0]); @@ -599,6 +599,6 @@ void kasan_non_canonical_hook(unsigned l else bug_type = "maybe wild-memory-access"; pr_alert("KASAN: %s in range [0x%016lx-0x%016lx]\n", bug_type, - orig_addr, orig_addr + KASAN_SHADOW_MASK); + orig_addr, orig_addr + KASAN_GRANULE_SIZE - 1); } #endif --- a/mm/kasan/tags_report.c~kasan-rename-kasan_shadow_-to-kasan_granule_ +++ a/mm/kasan/tags_report.c @@ -76,7 +76,7 @@ void *find_first_bad_addr(void *addr, si void *end = p + size; while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) - p += KASAN_SHADOW_SCALE_SIZE; + p += KASAN_GRANULE_SIZE; return p; } From patchwork Tue Dec 22 20:00:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4166CC433DB for ; Tue, 22 Dec 2020 20:00:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E08C0221FF for ; Tue, 22 Dec 2020 20:00:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E08C0221FF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7D11C6B00A3; Tue, 22 Dec 2020 15:00:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 759CF6B00A4; Tue, 22 Dec 2020 15:00:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 621EA6B00A6; Tue, 22 Dec 2020 15:00:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0214.hostedemail.com [216.40.44.214]) by kanga.kvack.org (Postfix) with ESMTP id 444046B00A3 for ; Tue, 22 Dec 2020 15:00:31 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id ECB19180AD807 for ; Tue, 22 Dec 2020 20:00:30 +0000 (UTC) X-FDA: 77621985420.23.book38_300a72927462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id B479037604 for ; Tue, 22 Dec 2020 20:00:30 +0000 (UTC) X-HE-Tag: book38_300a72927462 X-Filterd-Recvd-Size: 3684 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8482A20795; Tue, 22 Dec 2020 20:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667229; bh=FFOGr2o/XL8jMtH5M70PWuYqpSKtfQGlvgjA8O4gOJs=; h=Date:From:To:Subject:In-Reply-To:From; b=eA9vHzQMLYlhS88rJ4uX9C5SRzqxXhjds454S6WakI+AISSGO2cLR0nICrCM/RffX wM8NUvr12V+z4soIHzDzLDIUiAT4i0awUg+rrR1eiSHKMvzIepLPU4CRoNGGfUhE8D FpwqGDryper3q82LywktiSC8qq1Fpk/ZyalZM+Cc= Date: Tue, 22 Dec 2020 12:00:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 07/60] kasan: only build init.c for software modes Message-ID: <20201222200028.4cehFhEiH%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: only build init.c for software modes This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. The new mode won't be using shadow memory, so only build init.c that contains shadow initialization code for software modes. No functional changes for software modes. Link: https://lkml.kernel.org/r/bae0a6a35b7a9b1a443803c1a55e6e3fecc311c9.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/Makefile | 6 +++--- mm/kasan/init.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) --- a/mm/kasan/init.c~kasan-only-build-initc-for-software-modes +++ a/mm/kasan/init.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * This file contains some kasan initialization code. + * This file contains KASAN shadow initialization code. * * Copyright (c) 2015 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin --- a/mm/kasan/Makefile~kasan-only-build-initc-for-software-modes +++ a/mm/kasan/Makefile @@ -29,6 +29,6 @@ CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNT CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME) -obj-$(CONFIG_KASAN) := common.o init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o +obj-$(CONFIG_KASAN) := common.o report.o +obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o From patchwork Tue Dec 22 20:00:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A40DC433DB for ; Tue, 22 Dec 2020 20:00:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E56D423133 for ; Tue, 22 Dec 2020 20:00:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E56D423133 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 90B1C6B00A8; Tue, 22 Dec 2020 15:00:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8928C6B00AC; Tue, 22 Dec 2020 15:00:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 736606B00AE; Tue, 22 Dec 2020 15:00:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 41F6C6B00A8 for ; Tue, 22 Dec 2020 15:00:35 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F26608249980 for ; Tue, 22 Dec 2020 20:00:34 +0000 (UTC) X-FDA: 77621985588.08.clam13_2103bc727462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id DA28D1819E62A for ; Tue, 22 Dec 2020 20:00:34 +0000 (UTC) X-HE-Tag: clam13_2103bc727462 X-Filterd-Recvd-Size: 36129 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6BF5122583; Tue, 22 Dec 2020 20:00:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667233; bh=4o9oMLHsEqdeBNKpnN9AdjCVifymzhwm4YGlK3E7f/4=; h=Date:From:To:Subject:In-Reply-To:From; b=O0Vtj56mKlCC0xBjyrKmZJq99zJg4gnd8jUPKcoSHlac+TsBOXXzAI3LjxKnw0Gx7 w4Yi2suLqfi9NEnn6FhnIjgYd3XnuyURVN0GOOscB1E2r/XcH2db3T1w6a+NoleLPY /VklUM3CCAyBEAUYyHbh8it5teblpc9aubHnOdTI= Date: Tue, 22 Dec 2020 12:00:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 08/60] kasan: split out shadow.c from common.c Message-ID: <20201222200032.13otl20TG%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: split out shadow.c from common.c This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. The new mode won't be using shadow memory. Move all shadow-related code to shadow.c, which is only enabled for software KASAN modes that use shadow memory. No functional changes for software modes. Link: https://lkml.kernel.org/r/17d95cfa7d5cf9c4fcd9bf415f2a8dea911668df.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/Makefile | 6 mm/kasan/common.c | 486 ------------------------------------------ mm/kasan/shadow.c | 505 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 510 insertions(+), 487 deletions(-) --- a/mm/kasan/common.c~kasan-split-out-shadowc-from-commonc +++ a/mm/kasan/common.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * This file contains common generic and tag-based KASAN code. + * This file contains common KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -13,7 +13,6 @@ #include #include #include -#include #include #include #include @@ -26,12 +25,8 @@ #include #include #include -#include #include -#include -#include - #include "kasan.h" #include "../slab.h" @@ -61,93 +56,6 @@ void kasan_disable_current(void) current->kasan_depth--; } -bool __kasan_check_read(const volatile void *p, unsigned int size) -{ - return check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__kasan_check_read); - -bool __kasan_check_write(const volatile void *p, unsigned int size) -{ - return check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_)) - return NULL; - - return __memset(addr, c, len); -} - -#ifdef __HAVE_ARCH_MEMMOVE -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) || - !check_memory_region((unsigned long)dest, len, true, _RET_IP_)) - return NULL; - - return __memmove(dest, src, len); -} -#endif - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) || - !check_memory_region((unsigned long)dest, len, true, _RET_IP_)) - return NULL; - - return __memcpy(dest, src, len); -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_GRANULE_SIZE. - */ -void poison_range(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - /* - * Perform shadow offset calculation based on untagged address, as - * some of the callers (e.g. kasan_poison_object_data) pass tagged - * addresses to this function. - */ - address = reset_tag(address); - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - __memset(shadow_start, value, shadow_end - shadow_start); -} - -void unpoison_range(const void *address, size_t size) -{ - u8 tag = get_tag(address); - - /* - * Perform shadow offset calculation based on untagged address, as - * some of the callers (e.g. kasan_unpoison_object_data) pass tagged - * addresses to this function. - */ - address = reset_tag(address); - - poison_range(address, size, tag); - - if (size & KASAN_GRANULE_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) - *shadow = tag; - else - *shadow = size & KASAN_GRANULE_MASK; - } -} - void kasan_unpoison_range(const void *address, size_t size) { unpoison_range(address, size); @@ -540,395 +448,3 @@ void kasan_kfree_large(void *ptr, unsign kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) || - WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif - -#ifdef CONFIG_KASAN_VMALLOC - -static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, - void *unused) -{ - unsigned long page; - pte_t pte; - - if (likely(!pte_none(*ptep))) - return 0; - - page = __get_free_page(GFP_KERNEL); - if (!page) - return -ENOMEM; - - memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE); - pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL); - - spin_lock(&init_mm.page_table_lock); - if (likely(pte_none(*ptep))) { - set_pte_at(&init_mm, addr, ptep, pte); - page = 0; - } - spin_unlock(&init_mm.page_table_lock); - if (page) - free_page(page); - return 0; -} - -int kasan_populate_vmalloc(unsigned long addr, unsigned long size) -{ - unsigned long shadow_start, shadow_end; - int ret; - - if (!is_vmalloc_or_module_addr((void *)addr)) - return 0; - - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr); - shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE); - shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size); - shadow_end = ALIGN(shadow_end, PAGE_SIZE); - - ret = apply_to_page_range(&init_mm, shadow_start, - shadow_end - shadow_start, - kasan_populate_vmalloc_pte, NULL); - if (ret) - return ret; - - flush_cache_vmap(shadow_start, shadow_end); - - /* - * We need to be careful about inter-cpu effects here. Consider: - * - * CPU#0 CPU#1 - * WRITE_ONCE(p, vmalloc(100)); while (x = READ_ONCE(p)) ; - * p[99] = 1; - * - * With compiler instrumentation, that ends up looking like this: - * - * CPU#0 CPU#1 - * // vmalloc() allocates memory - * // let a = area->addr - * // we reach kasan_populate_vmalloc - * // and call unpoison_range: - * STORE shadow(a), unpoison_val - * ... - * STORE shadow(a+99), unpoison_val x = LOAD p - * // rest of vmalloc process - * STORE p, a LOAD shadow(x+99) - * - * If there is no barrier between the end of unpoisioning the shadow - * and the store of the result to p, the stores could be committed - * in a different order by CPU#0, and CPU#1 could erroneously observe - * poison in the shadow. - * - * We need some sort of barrier between the stores. - * - * In the vmalloc() case, this is provided by a smp_wmb() in - * clear_vm_uninitialized_flag(). In the per-cpu allocator and in - * get_vm_area() and friends, the caller gets shadow allocated but - * doesn't have any pages mapped into the virtual address space that - * has been reserved. Mapping those pages in will involve taking and - * releasing a page-table lock, which will provide the barrier. - */ - - return 0; -} - -/* - * Poison the shadow for a vmalloc region. Called as part of the - * freeing process at the time the region is freed. - */ -void kasan_poison_vmalloc(const void *start, unsigned long size) -{ - if (!is_vmalloc_or_module_addr(start)) - return; - - size = round_up(size, KASAN_GRANULE_SIZE); - poison_range(start, size, KASAN_VMALLOC_INVALID); -} - -void kasan_unpoison_vmalloc(const void *start, unsigned long size) -{ - if (!is_vmalloc_or_module_addr(start)) - return; - - unpoison_range(start, size); -} - -static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, - void *unused) -{ - unsigned long page; - - page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT); - - spin_lock(&init_mm.page_table_lock); - - if (likely(!pte_none(*ptep))) { - pte_clear(&init_mm, addr, ptep); - free_page(page); - } - spin_unlock(&init_mm.page_table_lock); - - return 0; -} - -/* - * Release the backing for the vmalloc region [start, end), which - * lies within the free region [free_region_start, free_region_end). - * - * This can be run lazily, long after the region was freed. It runs - * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap - * infrastructure. - * - * How does this work? - * ------------------- - * - * We have a region that is page aligned, labelled as A. - * That might not map onto the shadow in a way that is page-aligned: - * - * start end - * v v - * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc - * -------- -------- -------- -------- -------- - * | | | | | - * | | | /-------/ | - * \-------\|/------/ |/---------------/ - * ||| || - * |??AAAAAA|AAAAAAAA|AA??????| < shadow - * (1) (2) (3) - * - * First we align the start upwards and the end downwards, so that the - * shadow of the region aligns with shadow page boundaries. In the - * example, this gives us the shadow page (2). This is the shadow entirely - * covered by this allocation. - * - * Then we have the tricky bits. We want to know if we can free the - * partially covered shadow pages - (1) and (3) in the example. For this, - * we are given the start and end of the free region that contains this - * allocation. Extending our previous example, we could have: - * - * free_region_start free_region_end - * | start end | - * v v v v - * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc - * -------- -------- -------- -------- -------- - * | | | | | - * | | | /-------/ | - * \-------\|/------/ |/---------------/ - * ||| || - * |FFAAAAAA|AAAAAAAA|AAF?????| < shadow - * (1) (2) (3) - * - * Once again, we align the start of the free region up, and the end of - * the free region down so that the shadow is page aligned. So we can free - * page (1) - we know no allocation currently uses anything in that page, - * because all of it is in the vmalloc free region. But we cannot free - * page (3), because we can't be sure that the rest of it is unused. - * - * We only consider pages that contain part of the original region for - * freeing: we don't try to free other pages from the free region or we'd - * end up trying to free huge chunks of virtual address space. - * - * Concurrency - * ----------- - * - * How do we know that we're not freeing a page that is simultaneously - * being used for a fresh allocation in kasan_populate_vmalloc(_pte)? - * - * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running - * at the same time. While we run under free_vmap_area_lock, the population - * code does not. - * - * free_vmap_area_lock instead operates to ensure that the larger range - * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and - * the per-cpu region-finding algorithm both run under free_vmap_area_lock, - * no space identified as free will become used while we are running. This - * means that so long as we are careful with alignment and only free shadow - * pages entirely covered by the free region, we will not run in to any - * trouble - any simultaneous allocations will be for disjoint regions. - */ -void kasan_release_vmalloc(unsigned long start, unsigned long end, - unsigned long free_region_start, - unsigned long free_region_end) -{ - void *shadow_start, *shadow_end; - unsigned long region_start, region_end; - unsigned long size; - - region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE); - region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE); - - free_region_start = ALIGN(free_region_start, - PAGE_SIZE * KASAN_GRANULE_SIZE); - - if (start != region_start && - free_region_start < region_start) - region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE; - - free_region_end = ALIGN_DOWN(free_region_end, - PAGE_SIZE * KASAN_GRANULE_SIZE); - - if (end != region_end && - free_region_end > region_end) - region_end += PAGE_SIZE * KASAN_GRANULE_SIZE; - - shadow_start = kasan_mem_to_shadow((void *)region_start); - shadow_end = kasan_mem_to_shadow((void *)region_end); - - if (shadow_end > shadow_start) { - size = shadow_end - shadow_start; - apply_to_existing_page_range(&init_mm, - (unsigned long)shadow_start, - size, kasan_depopulate_vmalloc_pte, - NULL); - flush_tlb_kernel_range((unsigned long)shadow_start, - (unsigned long)shadow_end); - } -} - -#else /* CONFIG_KASAN_VMALLOC */ - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_GRANULE_SIZE - 1) >> - KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - __memset(ret, KASAN_SHADOW_INIT, shadow_size); - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - -#endif --- a/mm/kasan/Makefile~kasan-split-out-shadowc-from-commonc +++ a/mm/kasan/Makefile @@ -10,6 +10,7 @@ CFLAGS_REMOVE_generic_report.o = $(CC_FL CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE) @@ -26,9 +27,10 @@ CFLAGS_generic_report.o := $(CC_FLAGS_KA CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME) obj-$(CONFIG_KASAN) := common.o report.o -obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += init.o tags.o tags_report.o +obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o --- /dev/null +++ a/mm/kasan/shadow.c @@ -0,0 +1,505 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * This file contains KASAN runtime code that manages shadow memory for + * generic and software tag-based KASAN modes. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "kasan.h" + +bool __kasan_check_read(const volatile void *p, unsigned int size) +{ + return check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__kasan_check_read); + +bool __kasan_check_write(const volatile void *p, unsigned int size) +{ + return check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_)) + return NULL; + + return __memset(addr, c, len); +} + +#ifdef __HAVE_ARCH_MEMMOVE +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) || + !check_memory_region((unsigned long)dest, len, true, _RET_IP_)) + return NULL; + + return __memmove(dest, src, len); +} +#endif + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) || + !check_memory_region((unsigned long)dest, len, true, _RET_IP_)) + return NULL; + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_GRANULE_SIZE. + */ +void poison_range(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_poison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void unpoison_range(const void *address, size_t size) +{ + u8 tag = get_tag(address); + + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_unpoison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + + poison_range(address, size, tag); + + if (size & KASAN_GRANULE_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + *shadow = tag; + else + *shadow = size & KASAN_GRANULE_MASK; + } +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) || + WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif + +#ifdef CONFIG_KASAN_VMALLOC + +static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, + void *unused) +{ + unsigned long page; + pte_t pte; + + if (likely(!pte_none(*ptep))) + return 0; + + page = __get_free_page(GFP_KERNEL); + if (!page) + return -ENOMEM; + + memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE); + pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL); + + spin_lock(&init_mm.page_table_lock); + if (likely(pte_none(*ptep))) { + set_pte_at(&init_mm, addr, ptep, pte); + page = 0; + } + spin_unlock(&init_mm.page_table_lock); + if (page) + free_page(page); + return 0; +} + +int kasan_populate_vmalloc(unsigned long addr, unsigned long size) +{ + unsigned long shadow_start, shadow_end; + int ret; + + if (!is_vmalloc_or_module_addr((void *)addr)) + return 0; + + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr); + shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE); + shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size); + shadow_end = ALIGN(shadow_end, PAGE_SIZE); + + ret = apply_to_page_range(&init_mm, shadow_start, + shadow_end - shadow_start, + kasan_populate_vmalloc_pte, NULL); + if (ret) + return ret; + + flush_cache_vmap(shadow_start, shadow_end); + + /* + * We need to be careful about inter-cpu effects here. Consider: + * + * CPU#0 CPU#1 + * WRITE_ONCE(p, vmalloc(100)); while (x = READ_ONCE(p)) ; + * p[99] = 1; + * + * With compiler instrumentation, that ends up looking like this: + * + * CPU#0 CPU#1 + * // vmalloc() allocates memory + * // let a = area->addr + * // we reach kasan_populate_vmalloc + * // and call unpoison_range: + * STORE shadow(a), unpoison_val + * ... + * STORE shadow(a+99), unpoison_val x = LOAD p + * // rest of vmalloc process + * STORE p, a LOAD shadow(x+99) + * + * If there is no barrier between the end of unpoisioning the shadow + * and the store of the result to p, the stores could be committed + * in a different order by CPU#0, and CPU#1 could erroneously observe + * poison in the shadow. + * + * We need some sort of barrier between the stores. + * + * In the vmalloc() case, this is provided by a smp_wmb() in + * clear_vm_uninitialized_flag(). In the per-cpu allocator and in + * get_vm_area() and friends, the caller gets shadow allocated but + * doesn't have any pages mapped into the virtual address space that + * has been reserved. Mapping those pages in will involve taking and + * releasing a page-table lock, which will provide the barrier. + */ + + return 0; +} + +/* + * Poison the shadow for a vmalloc region. Called as part of the + * freeing process at the time the region is freed. + */ +void kasan_poison_vmalloc(const void *start, unsigned long size) +{ + if (!is_vmalloc_or_module_addr(start)) + return; + + size = round_up(size, KASAN_GRANULE_SIZE); + poison_range(start, size, KASAN_VMALLOC_INVALID); +} + +void kasan_unpoison_vmalloc(const void *start, unsigned long size) +{ + if (!is_vmalloc_or_module_addr(start)) + return; + + unpoison_range(start, size); +} + +static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, + void *unused) +{ + unsigned long page; + + page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT); + + spin_lock(&init_mm.page_table_lock); + + if (likely(!pte_none(*ptep))) { + pte_clear(&init_mm, addr, ptep); + free_page(page); + } + spin_unlock(&init_mm.page_table_lock); + + return 0; +} + +/* + * Release the backing for the vmalloc region [start, end), which + * lies within the free region [free_region_start, free_region_end). + * + * This can be run lazily, long after the region was freed. It runs + * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap + * infrastructure. + * + * How does this work? + * ------------------- + * + * We have a region that is page aligned, labelled as A. + * That might not map onto the shadow in a way that is page-aligned: + * + * start end + * v v + * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc + * -------- -------- -------- -------- -------- + * | | | | | + * | | | /-------/ | + * \-------\|/------/ |/---------------/ + * ||| || + * |??AAAAAA|AAAAAAAA|AA??????| < shadow + * (1) (2) (3) + * + * First we align the start upwards and the end downwards, so that the + * shadow of the region aligns with shadow page boundaries. In the + * example, this gives us the shadow page (2). This is the shadow entirely + * covered by this allocation. + * + * Then we have the tricky bits. We want to know if we can free the + * partially covered shadow pages - (1) and (3) in the example. For this, + * we are given the start and end of the free region that contains this + * allocation. Extending our previous example, we could have: + * + * free_region_start free_region_end + * | start end | + * v v v v + * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc + * -------- -------- -------- -------- -------- + * | | | | | + * | | | /-------/ | + * \-------\|/------/ |/---------------/ + * ||| || + * |FFAAAAAA|AAAAAAAA|AAF?????| < shadow + * (1) (2) (3) + * + * Once again, we align the start of the free region up, and the end of + * the free region down so that the shadow is page aligned. So we can free + * page (1) - we know no allocation currently uses anything in that page, + * because all of it is in the vmalloc free region. But we cannot free + * page (3), because we can't be sure that the rest of it is unused. + * + * We only consider pages that contain part of the original region for + * freeing: we don't try to free other pages from the free region or we'd + * end up trying to free huge chunks of virtual address space. + * + * Concurrency + * ----------- + * + * How do we know that we're not freeing a page that is simultaneously + * being used for a fresh allocation in kasan_populate_vmalloc(_pte)? + * + * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running + * at the same time. While we run under free_vmap_area_lock, the population + * code does not. + * + * free_vmap_area_lock instead operates to ensure that the larger range + * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and + * the per-cpu region-finding algorithm both run under free_vmap_area_lock, + * no space identified as free will become used while we are running. This + * means that so long as we are careful with alignment and only free shadow + * pages entirely covered by the free region, we will not run in to any + * trouble - any simultaneous allocations will be for disjoint regions. + */ +void kasan_release_vmalloc(unsigned long start, unsigned long end, + unsigned long free_region_start, + unsigned long free_region_end) +{ + void *shadow_start, *shadow_end; + unsigned long region_start, region_end; + unsigned long size; + + region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE); + region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE); + + free_region_start = ALIGN(free_region_start, + PAGE_SIZE * KASAN_GRANULE_SIZE); + + if (start != region_start && + free_region_start < region_start) + region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE; + + free_region_end = ALIGN_DOWN(free_region_end, + PAGE_SIZE * KASAN_GRANULE_SIZE); + + if (end != region_end && + free_region_end > region_end) + region_end += PAGE_SIZE * KASAN_GRANULE_SIZE; + + shadow_start = kasan_mem_to_shadow((void *)region_start); + shadow_end = kasan_mem_to_shadow((void *)region_end); + + if (shadow_end > shadow_start) { + size = shadow_end - shadow_start; + apply_to_existing_page_range(&init_mm, + (unsigned long)shadow_start, + size, kasan_depopulate_vmalloc_pte, + NULL); + flush_tlb_kernel_range((unsigned long)shadow_start, + (unsigned long)shadow_end); + } +} + +#else /* CONFIG_KASAN_VMALLOC */ + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_GRANULE_SIZE - 1) >> + KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#endif From patchwork Tue Dec 22 20:00:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADC7BC433E0 for ; Tue, 22 Dec 2020 20:00:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 28CA523137 for ; Tue, 22 Dec 2020 20:00:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28CA523137 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D76026B00AE; Tue, 22 Dec 2020 15:00:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D004D6B00B0; Tue, 22 Dec 2020 15:00:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B53E26B00B1; Tue, 22 Dec 2020 15:00:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id 9BD8E6B00AE for ; Tue, 22 Dec 2020 15:00:38 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 69DAD8249980 for ; Tue, 22 Dec 2020 20:00:38 +0000 (UTC) X-FDA: 77621985756.03.month53_081056427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 2D4B928A4E9 for ; Tue, 22 Dec 2020 20:00:38 +0000 (UTC) X-HE-Tag: month53_081056427462 X-Filterd-Recvd-Size: 5902 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EE866221FF; Tue, 22 Dec 2020 20:00:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667236; bh=qlqKg0D1FBUunJ/7Gr2LXyck+8gOv/WVqpxxiORWXLQ=; h=Date:From:To:Subject:In-Reply-To:From; b=1EozPhv9ekEcBBT31Wbxle0CwbRwHSo/uQf7pQ6wjVM+gVTJbcT420oCR4YpOmfTe BxibUCrIIwCrSEeKxQcvppmHCYwwnR1ZSe/XPOWaKNR/J9dFcC77PohUkFzEWs1p8+ ONbDjpUYvvOEesDMUvFsPYBKlbDOKzz/MWjRFuVQ= Date: Tue, 22 Dec 2020 12:00:35 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 09/60] kasan: define KASAN_MEMORY_PER_SHADOW_PAGE Message-ID: <20201222200035.5ghw6cTXt%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: define KASAN_MEMORY_PER_SHADOW_PAGE Define KASAN_MEMORY_PER_SHADOW_PAGE as (KASAN_GRANULE_SIZE << PAGE_SHIFT), which is the same as (KASAN_GRANULE_SIZE * PAGE_SIZE) for software modes that use shadow memory, and use it across KASAN code to simplify it. Link: https://lkml.kernel.org/r/8329391cfe14b5cffd3decf3b5c535b6ce21eef6.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/init.c | 10 ++++------ mm/kasan/kasan.h | 2 ++ mm/kasan/shadow.c | 16 +++++++--------- 3 files changed, 13 insertions(+), 15 deletions(-) --- a/mm/kasan/init.c~kasan-define-kasan_memory_per_shadow_page +++ a/mm/kasan/init.c @@ -441,9 +441,8 @@ void kasan_remove_zero_shadow(void *star addr = (unsigned long)kasan_mem_to_shadow(start); end = addr + (size >> KASAN_SHADOW_SCALE_SHIFT); - if (WARN_ON((unsigned long)start % - (KASAN_GRANULE_SIZE * PAGE_SIZE)) || - WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE))) + if (WARN_ON((unsigned long)start % KASAN_MEMORY_PER_SHADOW_PAGE) || + WARN_ON(size % KASAN_MEMORY_PER_SHADOW_PAGE)) return; for (; addr < end; addr = next) { @@ -476,9 +475,8 @@ int kasan_add_zero_shadow(void *start, u shadow_start = kasan_mem_to_shadow(start); shadow_end = shadow_start + (size >> KASAN_SHADOW_SCALE_SHIFT); - if (WARN_ON((unsigned long)start % - (KASAN_GRANULE_SIZE * PAGE_SIZE)) || - WARN_ON(size % (KASAN_GRANULE_SIZE * PAGE_SIZE))) + if (WARN_ON((unsigned long)start % KASAN_MEMORY_PER_SHADOW_PAGE) || + WARN_ON(size % KASAN_MEMORY_PER_SHADOW_PAGE)) return -EINVAL; ret = kasan_populate_early_shadow(shadow_start, shadow_end); --- a/mm/kasan/kasan.h~kasan-define-kasan_memory_per_shadow_page +++ a/mm/kasan/kasan.h @@ -8,6 +8,8 @@ #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_GRANULE_MASK (KASAN_GRANULE_SIZE - 1) +#define KASAN_MEMORY_PER_SHADOW_PAGE (KASAN_GRANULE_SIZE << PAGE_SHIFT) + #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ --- a/mm/kasan/shadow.c~kasan-define-kasan_memory_per_shadow_page +++ a/mm/kasan/shadow.c @@ -161,7 +161,7 @@ static int __meminit kasan_mem_notifier( shadow_end = shadow_start + shadow_size; if (WARN_ON(mem_data->nr_pages % KASAN_GRANULE_SIZE) || - WARN_ON(start_kaddr % (KASAN_GRANULE_SIZE << PAGE_SHIFT))) + WARN_ON(start_kaddr % KASAN_MEMORY_PER_SHADOW_PAGE)) return NOTIFY_BAD; switch (action) { @@ -432,22 +432,20 @@ void kasan_release_vmalloc(unsigned long unsigned long region_start, region_end; unsigned long size; - region_start = ALIGN(start, PAGE_SIZE * KASAN_GRANULE_SIZE); - region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_GRANULE_SIZE); + region_start = ALIGN(start, KASAN_MEMORY_PER_SHADOW_PAGE); + region_end = ALIGN_DOWN(end, KASAN_MEMORY_PER_SHADOW_PAGE); - free_region_start = ALIGN(free_region_start, - PAGE_SIZE * KASAN_GRANULE_SIZE); + free_region_start = ALIGN(free_region_start, KASAN_MEMORY_PER_SHADOW_PAGE); if (start != region_start && free_region_start < region_start) - region_start -= PAGE_SIZE * KASAN_GRANULE_SIZE; + region_start -= KASAN_MEMORY_PER_SHADOW_PAGE; - free_region_end = ALIGN_DOWN(free_region_end, - PAGE_SIZE * KASAN_GRANULE_SIZE); + free_region_end = ALIGN_DOWN(free_region_end, KASAN_MEMORY_PER_SHADOW_PAGE); if (end != region_end && free_region_end > region_end) - region_end += PAGE_SIZE * KASAN_GRANULE_SIZE; + region_end += KASAN_MEMORY_PER_SHADOW_PAGE; shadow_start = kasan_mem_to_shadow((void *)region_start); shadow_end = kasan_mem_to_shadow((void *)region_end); From patchwork Tue Dec 22 20:00:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D23EDC433DB for ; Tue, 22 Dec 2020 20:00:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7752D23139 for ; Tue, 22 Dec 2020 20:00:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7752D23139 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C17A6B00B1; Tue, 22 Dec 2020 15:00:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 649566B00B2; Tue, 22 Dec 2020 15:00:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EC1E6B00B3; Tue, 22 Dec 2020 15:00:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0067.hostedemail.com [216.40.44.67]) by kanga.kvack.org (Postfix) with ESMTP id 28B626B00B1 for ; Tue, 22 Dec 2020 15:00:42 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E4481181AEF3F for ; Tue, 22 Dec 2020 20:00:41 +0000 (UTC) X-FDA: 77621985882.10.rake89_5d12ca027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id C1B6116A0C3 for ; Tue, 22 Dec 2020 20:00:41 +0000 (UTC) X-HE-Tag: rake89_5d12ca027462 X-Filterd-Recvd-Size: 5837 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 96E8622ADC; Tue, 22 Dec 2020 20:00:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667240; bh=WrCp/mYoc7bkusflRSZG45RSf8Huec1dFaE455zpBKc=; h=Date:From:To:Subject:In-Reply-To:From; b=hVw1kjlnxLO3NehC0LO5/zeYipJ/8NhwD0FW1sw9Xg9OmlM9z2Ia/VWaYOtt/Z4Em HeETAyIBszp+In02kRGI5hkotWYQTVTE1pVrqmKzxTiM3SqzXVAHjX1r4zd9T1SQg8 yduu/zQTZGUWY+5qCaw0IclSYK/KkPJ86MkgDyXs= Date: Tue, 22 Dec 2020 12:00:39 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 10/60] kasan: rename report and tags files Message-ID: <20201222200039.X1Pa4j3L2%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename report and tags files Rename generic_report.c to report_generic.c and tags_report.c to report_sw_tags.c, as their content is more relevant to report.c file. Also rename tags.c to sw_tags.c to better reflect that this file contains code for software tag-based mode. No functional changes. Link: https://lkml.kernel.org/r/a6105d416da97d389580015afed66c4c3cfd4c08.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/Makefile | 16 +- mm/kasan/generic_report.c | 160 ----------------------------- mm/kasan/report.c | 2 mm/kasan/report_generic.c | 160 +++++++++++++++++++++++++++++ mm/kasan/report_sw_tags.c | 88 ++++++++++++++++ mm/kasan/sw_tags.c | 195 ++++++++++++++++++++++++++++++++++++ mm/kasan/tags.c | 195 ------------------------------------ mm/kasan/tags_report.c | 88 ---------------- 8 files changed, 452 insertions(+), 452 deletions(-) --- a/mm/kasan/Makefile~kasan-rename-report-and-tags-files +++ a/mm/kasan/Makefile @@ -6,13 +6,13 @@ KCOV_INSTRUMENT := n # Disable ftrace to avoid recursion. CFLAGS_REMOVE_common.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_generic.o = $(CC_FLAGS_FTRACE) -CFLAGS_REMOVE_generic_report.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE) -CFLAGS_REMOVE_tags.o = $(CC_FLAGS_FTRACE) -CFLAGS_REMOVE_tags_report.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE) # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 @@ -23,14 +23,14 @@ CC_FLAGS_KASAN_RUNTIME += -DDISABLE_BRAN CFLAGS_common.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_generic.o := $(CC_FLAGS_KASAN_RUNTIME) -CFLAGS_generic_report.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME) -CFLAGS_tags.o := $(CC_FLAGS_KASAN_RUNTIME) -CFLAGS_tags_report.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME) obj-$(CONFIG_KASAN) := common.o report.o -obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o generic_report.o shadow.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += init.o shadow.o tags.o tags_report.o +obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o --- a/mm/kasan/report.c~kasan-rename-report-and-tags-files +++ a/mm/kasan/report.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * This file contains common generic and tag-based KASAN error reporting code. + * This file contains common KASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin diff --git a/mm/kasan/generic_report.c b/mm/kasan/report_generic.c similarity index 100% rename from mm/kasan/generic_report.c rename to mm/kasan/report_generic.c diff --git a/mm/kasan/tags_report.c b/mm/kasan/report_sw_tags.c similarity index 100% rename from mm/kasan/tags_report.c rename to mm/kasan/report_sw_tags.c diff --git a/mm/kasan/tags.c b/mm/kasan/sw_tags.c similarity index 100% rename from mm/kasan/tags.c rename to mm/kasan/sw_tags.c From patchwork Tue Dec 22 20:00:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 436E6C433DB for ; Tue, 22 Dec 2020 20:00:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C916820700 for ; Tue, 22 Dec 2020 20:00:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C916820700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0A64F6B00B5; Tue, 22 Dec 2020 15:00:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 00B636B00B4; Tue, 22 Dec 2020 15:00:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D29F36B00B5; Tue, 22 Dec 2020 15:00:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id B525B6B00B3 for ; Tue, 22 Dec 2020 15:00:45 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7702A181AF5C2 for ; Tue, 22 Dec 2020 20:00:45 +0000 (UTC) X-FDA: 77621986050.22.week19_1105c5f27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 4885B18038E60 for ; Tue, 22 Dec 2020 20:00:45 +0000 (UTC) X-HE-Tag: week19_1105c5f27462 X-Filterd-Recvd-Size: 3590 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 32F9623133; Tue, 22 Dec 2020 20:00:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667243; bh=BnKABBFXehNVsAf/DxE7GIkSrsUgxtLfAbUVA2DgJtY=; h=Date:From:To:Subject:In-Reply-To:From; b=jtrIYAPlKB4UuiTVvsW1INcaGBHvGdr3rT5UlZcW82r8DfXPCKv/BIkGek6yCYe8w YVwSB5LZ+kELtHil6BC6tLqMENmwi9/olzqi/TJKmHTX6RSL50I+HJqw9gsw1cvo8x CMxCyF75I14l6sRKVbKlqiPrZ0GXHIR2ipl5UH00= Date: Tue, 22 Dec 2020 12:00:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 11/60] kasan: don't duplicate config dependencies Message-ID: <20201222200042.iCRSiCYmC%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: don't duplicate config dependencies Both KASAN_GENERIC and KASAN_SW_TAGS have common dependencies, move those to KASAN. Link: https://lkml.kernel.org/r/c1cc0d562608a318c607afe22db5ec2a7af72e47.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/Kconfig.kasan | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) --- a/lib/Kconfig.kasan~kasan-dont-duplicate-config-dependencies +++ a/lib/Kconfig.kasan @@ -24,6 +24,8 @@ menuconfig KASAN (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS + select CONSTRUCTORS + select STACKDEPOT help Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, designed to find out-of-bounds accesses and use-after-free bugs. @@ -46,10 +48,7 @@ choice config KASAN_GENERIC bool "Generic mode" depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC - depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB - select CONSTRUCTORS - select STACKDEPOT help Enables generic KASAN mode. @@ -70,10 +69,7 @@ config KASAN_GENERIC config KASAN_SW_TAGS bool "Software tag-based mode" depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS - depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB - select CONSTRUCTORS - select STACKDEPOT help Enables software tag-based KASAN mode. From patchwork Tue Dec 22 20:00:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EF90C433E0 for ; Tue, 22 Dec 2020 20:00:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C52A20705 for ; Tue, 22 Dec 2020 20:00:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C52A20705 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E79E6B00B4; Tue, 22 Dec 2020 15:00:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 66FD86B00B6; Tue, 22 Dec 2020 15:00:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5378B6B00B7; Tue, 22 Dec 2020 15:00:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id 37E6C6B00B4 for ; Tue, 22 Dec 2020 15:00:50 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id F14F48249980 for ; Tue, 22 Dec 2020 20:00:49 +0000 (UTC) X-FDA: 77621986218.25.boot79_0307f0227462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 02EA41804E3AF for ; Tue, 22 Dec 2020 20:00:48 +0000 (UTC) X-HE-Tag: boot79_0307f0227462 X-Filterd-Recvd-Size: 5543 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BA93723137; Tue, 22 Dec 2020 20:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667247; bh=l3lg1QHfWNiKhxkb3gVgUjhwfevdtIV2NAFdxq0ggH0=; h=Date:From:To:Subject:In-Reply-To:From; b=Ncc0R+aq8vmgNfHAZFO1XEy5auFNqVHgPYh8uoDovmfWKriBDu9CUWniCd1UhVYxT pUuqm6+Xpy/NFJuU/SIiuNWAcw3O4qrSc7smLSlAH03ge9AS+unM2QVWOfURw/iaPO dlm9Sm0CklkptV7lkgHOGRMKAvACitzMjsAmDuRI= Date: Tue, 22 Dec 2020 12:00:46 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 12/60] kasan: hide invalid free check implementation Message-ID: <20201222200046.skGG0SPl-%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: hide invalid free check implementation This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. For software KASAN modes the check is based on the value in the shadow memory. Hardware tag-based KASAN won't be using shadow, so hide the implementation of the check in check_invalid_free(). Also simplify the code for software tag-based mode. No functional changes for software modes. Link: https://lkml.kernel.org/r/d01534a4b977f97d87515dc590e6348e1406de81.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 19 +------------------ mm/kasan/generic.c | 7 +++++++ mm/kasan/kasan.h | 2 ++ mm/kasan/sw_tags.c | 9 +++++++++ 4 files changed, 19 insertions(+), 18 deletions(-) --- a/mm/kasan/common.c~kasan-hide-invalid-free-check-implementation +++ a/mm/kasan/common.c @@ -277,25 +277,9 @@ void * __must_check kasan_init_slab_obj( return (void *)object; } -static inline bool shadow_invalid(u8 tag, s8 shadow_byte) -{ - if (IS_ENABLED(CONFIG_KASAN_GENERIC)) - return shadow_byte < 0 || - shadow_byte >= KASAN_GRANULE_SIZE; - - /* else CONFIG_KASAN_SW_TAGS: */ - if ((u8)shadow_byte == KASAN_TAG_INVALID) - return true; - if ((tag != KASAN_TAG_KERNEL) && (tag != (u8)shadow_byte)) - return true; - - return false; -} - static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { - s8 shadow_byte; u8 tag; void *tagged_object; unsigned long rounded_up_size; @@ -314,8 +298,7 @@ static bool __kasan_slab_free(struct kme if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) return false; - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_invalid(tag, shadow_byte)) { + if (check_invalid_free(tagged_object)) { kasan_report_invalid_free(tagged_object, ip); return true; } --- a/mm/kasan/generic.c~kasan-hide-invalid-free-check-implementation +++ a/mm/kasan/generic.c @@ -187,6 +187,13 @@ bool check_memory_region(unsigned long a return check_memory_region_inline(addr, size, write, ret_ip); } +bool check_invalid_free(void *addr) +{ + s8 shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr)); + + return shadow_byte < 0 || shadow_byte >= KASAN_GRANULE_SIZE; +} + void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); --- a/mm/kasan/kasan.h~kasan-hide-invalid-free-check-implementation +++ a/mm/kasan/kasan.h @@ -166,6 +166,8 @@ void unpoison_range(const void *address, bool check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +bool check_invalid_free(void *addr); + void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); --- a/mm/kasan/sw_tags.c~kasan-hide-invalid-free-check-implementation +++ a/mm/kasan/sw_tags.c @@ -121,6 +121,15 @@ bool check_memory_region(unsigned long a return true; } +bool check_invalid_free(void *addr) +{ + u8 tag = get_tag(addr); + u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr))); + + return (shadow_byte == KASAN_TAG_INVALID) || + (tag != KASAN_TAG_KERNEL && tag != shadow_byte); +} + #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ From patchwork Tue Dec 22 20:00:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 188A9C433E0 for ; Tue, 22 Dec 2020 20:00:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B521621D1B for ; Tue, 22 Dec 2020 20:00:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B521621D1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 47F8F6B00B7; Tue, 22 Dec 2020 15:00:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 33D096B00B8; Tue, 22 Dec 2020 15:00:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DBF16B00B9; Tue, 22 Dec 2020 15:00:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0105.hostedemail.com [216.40.44.105]) by kanga.kvack.org (Postfix) with ESMTP id EF5776B00B7 for ; Tue, 22 Dec 2020 15:00:52 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B943B1EF1 for ; Tue, 22 Dec 2020 20:00:52 +0000 (UTC) X-FDA: 77621986344.05.maid66_1006f8727462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 98C8318027FBA for ; Tue, 22 Dec 2020 20:00:52 +0000 (UTC) X-HE-Tag: maid66_1006f8727462 X-Filterd-Recvd-Size: 13237 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5519923139; Tue, 22 Dec 2020 20:00:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667251; bh=756GrEw9riVx6R5mXVZ+zUzRgjk0NVBevAxEpXN7LqI=; h=Date:From:To:Subject:In-Reply-To:From; b=KXpKMuh6Egev5mUViKglX1v2H1ffDw91eWTiBk0elzYHpgByDXUZCexw7/EX6x934 rV5OdPMDrO8EUO+HtSgrVsacbOVb2eerUiiDpJyMSteB6ZPSo5gBIb6e5RFINDudlo IUKzPAbACzOkM+QBUgOUQwJidv7HScBDbD7CieAc= Date: Tue, 22 Dec 2020 12:00:49 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 13/60] kasan: decode stack frame only with KASAN_STACK_ENABLE Message-ID: <20201222200049.DgifxXrKv%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: decode stack frame only with KASAN_STACK_ENABLE Decoding routines aren't needed when CONFIG_KASAN_STACK_ENABLE is not enabled. Currently only generic KASAN mode implements stack error reporting. No functional changes for software modes. Link: https://lkml.kernel.org/r/05a24db36f5ec876af876a299bbea98c29468ebd.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/kasan.h | 6 + mm/kasan/report.c | 162 ------------------------------------ mm/kasan/report_generic.c | 162 ++++++++++++++++++++++++++++++++++++ 3 files changed, 168 insertions(+), 162 deletions(-) --- a/mm/kasan/kasan.h~kasan-decode-stack-frame-only-with-kasan_stack_enable +++ a/mm/kasan/kasan.h @@ -171,6 +171,12 @@ bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); +#if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK +void print_address_stack_frame(const void *addr); +#else +static inline void print_address_stack_frame(const void *addr) { } +#endif + bool kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); --- a/mm/kasan/report.c~kasan-decode-stack-frame-only-with-kasan_stack_enable +++ a/mm/kasan/report.c @@ -211,168 +211,6 @@ static inline bool init_task_stack_addr( sizeof(init_thread_union.stack)); } -static bool __must_check tokenize_frame_descr(const char **frame_descr, - char *token, size_t max_tok_len, - unsigned long *value) -{ - const char *sep = strchr(*frame_descr, ' '); - - if (sep == NULL) - sep = *frame_descr + strlen(*frame_descr); - - if (token != NULL) { - const size_t tok_len = sep - *frame_descr; - - if (tok_len + 1 > max_tok_len) { - pr_err("KASAN internal error: frame description too long: %s\n", - *frame_descr); - return false; - } - - /* Copy token (+ 1 byte for '\0'). */ - strlcpy(token, *frame_descr, tok_len + 1); - } - - /* Advance frame_descr past separator. */ - *frame_descr = sep + 1; - - if (value != NULL && kstrtoul(token, 10, value)) { - pr_err("KASAN internal error: not a valid number: %s\n", token); - return false; - } - - return true; -} - -static void print_decoded_frame_descr(const char *frame_descr) -{ - /* - * We need to parse the following string: - * "n alloc_1 alloc_2 ... alloc_n" - * where alloc_i looks like - * "offset size len name" - * or "offset size len name:line". - */ - - char token[64]; - unsigned long num_objects; - - if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), - &num_objects)) - return; - - pr_err("\n"); - pr_err("this frame has %lu %s:\n", num_objects, - num_objects == 1 ? "object" : "objects"); - - while (num_objects--) { - unsigned long offset; - unsigned long size; - - /* access offset */ - if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), - &offset)) - return; - /* access size */ - if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), - &size)) - return; - /* name length (unused) */ - if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL)) - return; - /* object name */ - if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), - NULL)) - return; - - /* Strip line number; without filename it's not very helpful. */ - strreplace(token, ':', '\0'); - - /* Finally, print object information. */ - pr_err(" [%lu, %lu) '%s'", offset, offset + size, token); - } -} - -static bool __must_check get_address_stack_frame_info(const void *addr, - unsigned long *offset, - const char **frame_descr, - const void **frame_pc) -{ - unsigned long aligned_addr; - unsigned long mem_ptr; - const u8 *shadow_bottom; - const u8 *shadow_ptr; - const unsigned long *frame; - - BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP)); - - /* - * NOTE: We currently only support printing frame information for - * accesses to the task's own stack. - */ - if (!object_is_on_stack(addr)) - return false; - - aligned_addr = round_down((unsigned long)addr, sizeof(long)); - mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE); - shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr); - shadow_bottom = kasan_mem_to_shadow(end_of_stack(current)); - - while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) { - shadow_ptr--; - mem_ptr -= KASAN_GRANULE_SIZE; - } - - while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) { - shadow_ptr--; - mem_ptr -= KASAN_GRANULE_SIZE; - } - - if (shadow_ptr < shadow_bottom) - return false; - - frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE); - if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) { - pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n", - frame[0]); - return false; - } - - *offset = (unsigned long)addr - (unsigned long)frame; - *frame_descr = (const char *)frame[1]; - *frame_pc = (void *)frame[2]; - - return true; -} - -static void print_address_stack_frame(const void *addr) -{ - unsigned long offset; - const char *frame_descr; - const void *frame_pc; - - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) - return; - - if (!get_address_stack_frame_info(addr, &offset, &frame_descr, - &frame_pc)) - return; - - /* - * get_address_stack_frame_info only returns true if the given addr is - * on the current task's stack. - */ - pr_err("\n"); - pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n", - addr, current->comm, task_pid_nr(current), offset); - pr_err(" %pS\n", frame_pc); - - if (!frame_descr) - return; - - print_decoded_frame_descr(frame_descr); -} - static void print_address_description(void *addr, u8 tag) { struct page *page = kasan_addr_to_page(addr); --- a/mm/kasan/report_generic.c~kasan-decode-stack-frame-only-with-kasan_stack_enable +++ a/mm/kasan/report_generic.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -122,6 +123,167 @@ const char *get_bug_type(struct kasan_ac return get_wild_bug_type(info); } +#if CONFIG_KASAN_STACK +static bool __must_check tokenize_frame_descr(const char **frame_descr, + char *token, size_t max_tok_len, + unsigned long *value) +{ + const char *sep = strchr(*frame_descr, ' '); + + if (sep == NULL) + sep = *frame_descr + strlen(*frame_descr); + + if (token != NULL) { + const size_t tok_len = sep - *frame_descr; + + if (tok_len + 1 > max_tok_len) { + pr_err("KASAN internal error: frame description too long: %s\n", + *frame_descr); + return false; + } + + /* Copy token (+ 1 byte for '\0'). */ + strlcpy(token, *frame_descr, tok_len + 1); + } + + /* Advance frame_descr past separator. */ + *frame_descr = sep + 1; + + if (value != NULL && kstrtoul(token, 10, value)) { + pr_err("KASAN internal error: not a valid number: %s\n", token); + return false; + } + + return true; +} + +static void print_decoded_frame_descr(const char *frame_descr) +{ + /* + * We need to parse the following string: + * "n alloc_1 alloc_2 ... alloc_n" + * where alloc_i looks like + * "offset size len name" + * or "offset size len name:line". + */ + + char token[64]; + unsigned long num_objects; + + if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), + &num_objects)) + return; + + pr_err("\n"); + pr_err("this frame has %lu %s:\n", num_objects, + num_objects == 1 ? "object" : "objects"); + + while (num_objects--) { + unsigned long offset; + unsigned long size; + + /* access offset */ + if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), + &offset)) + return; + /* access size */ + if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), + &size)) + return; + /* name length (unused) */ + if (!tokenize_frame_descr(&frame_descr, NULL, 0, NULL)) + return; + /* object name */ + if (!tokenize_frame_descr(&frame_descr, token, sizeof(token), + NULL)) + return; + + /* Strip line number; without filename it's not very helpful. */ + strreplace(token, ':', '\0'); + + /* Finally, print object information. */ + pr_err(" [%lu, %lu) '%s'", offset, offset + size, token); + } +} + +static bool __must_check get_address_stack_frame_info(const void *addr, + unsigned long *offset, + const char **frame_descr, + const void **frame_pc) +{ + unsigned long aligned_addr; + unsigned long mem_ptr; + const u8 *shadow_bottom; + const u8 *shadow_ptr; + const unsigned long *frame; + + BUILD_BUG_ON(IS_ENABLED(CONFIG_STACK_GROWSUP)); + + /* + * NOTE: We currently only support printing frame information for + * accesses to the task's own stack. + */ + if (!object_is_on_stack(addr)) + return false; + + aligned_addr = round_down((unsigned long)addr, sizeof(long)); + mem_ptr = round_down(aligned_addr, KASAN_GRANULE_SIZE); + shadow_ptr = kasan_mem_to_shadow((void *)aligned_addr); + shadow_bottom = kasan_mem_to_shadow(end_of_stack(current)); + + while (shadow_ptr >= shadow_bottom && *shadow_ptr != KASAN_STACK_LEFT) { + shadow_ptr--; + mem_ptr -= KASAN_GRANULE_SIZE; + } + + while (shadow_ptr >= shadow_bottom && *shadow_ptr == KASAN_STACK_LEFT) { + shadow_ptr--; + mem_ptr -= KASAN_GRANULE_SIZE; + } + + if (shadow_ptr < shadow_bottom) + return false; + + frame = (const unsigned long *)(mem_ptr + KASAN_GRANULE_SIZE); + if (frame[0] != KASAN_CURRENT_STACK_FRAME_MAGIC) { + pr_err("KASAN internal error: frame info validation failed; invalid marker: %lu\n", + frame[0]); + return false; + } + + *offset = (unsigned long)addr - (unsigned long)frame; + *frame_descr = (const char *)frame[1]; + *frame_pc = (void *)frame[2]; + + return true; +} + +void print_address_stack_frame(const void *addr) +{ + unsigned long offset; + const char *frame_descr; + const void *frame_pc; + + if (!get_address_stack_frame_info(addr, &offset, &frame_descr, + &frame_pc)) + return; + + /* + * get_address_stack_frame_info only returns true if the given addr is + * on the current task's stack. + */ + pr_err("\n"); + pr_err("addr %px is located in stack of task %s/%d at offset %lu in frame:\n", + addr, current->comm, task_pid_nr(current), offset); + pr_err(" %pS\n", frame_pc); + + if (!frame_descr) + return; + + print_decoded_frame_descr(frame_descr); +} +#endif /* CONFIG_KASAN_STACK */ + #define DEFINE_ASAN_REPORT_LOAD(size) \ void __asan_report_load##size##_noabort(unsigned long addr) \ { \ From patchwork Tue Dec 22 20:00:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2652CC433E0 for ; Tue, 22 Dec 2020 20:01:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BAEEB21D1B for ; Tue, 22 Dec 2020 20:01:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BAEEB21D1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A1ECE6B00B9; Tue, 22 Dec 2020 15:00:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 97D546B00BA; Tue, 22 Dec 2020 15:00:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FAAA6B00BB; Tue, 22 Dec 2020 15:00:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0103.hostedemail.com [216.40.44.103]) by kanga.kvack.org (Postfix) with ESMTP id 5772F6B00B9 for ; Tue, 22 Dec 2020 15:00:56 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1AE06180ACF75 for ; Tue, 22 Dec 2020 20:00:56 +0000 (UTC) X-FDA: 77621986512.27.hope84_3302a8727462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id D79003D670 for ; Tue, 22 Dec 2020 20:00:55 +0000 (UTC) X-HE-Tag: hope84_3302a8727462 X-Filterd-Recvd-Size: 4772 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D98FE20700; Tue, 22 Dec 2020 20:00:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667254; bh=wdKlI9yL+xh6v3JC0ySPdCRxNpCE+sflhhU2MWYOChE=; h=Date:From:To:Subject:In-Reply-To:From; b=zp6VCnJ3w5817iZ+c0BnrMJ9fBEXM3MmRA7Qd2v/7hAYNO3Sfxzak+kPXo0/M8Jb2 4fAEZCwF0V8wxDxtnFDnp0qkem0kFwh/oYUx+TkFxiiIoQkJpDKiq8AmbrKeccyciI MNlThdtDqyG82xAwIT+a939ZKIJXg7+Mc9Tc2cUE= Date: Tue, 22 Dec 2020 12:00:53 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 14/60] kasan, arm64: only init shadow for software modes Message-ID: <20201222200053.8m3hZ9VEE%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: only init shadow for software modes This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Hardware tag-based KASAN won't be using shadow memory. Only initialize it when one of the software KASAN modes are enabled. No functional changes for software modes. Link: https://lkml.kernel.org/r/d1742eea2cd728d150d49b144e49b6433405c7ba.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/kasan.h | 8 ++++++-- arch/arm64/mm/kasan_init.c | 15 ++++++++++++++- 2 files changed, 20 insertions(+), 3 deletions(-) --- a/arch/arm64/include/asm/kasan.h~kasan-arm64-only-init-shadow-for-software-modes +++ a/arch/arm64/include/asm/kasan.h @@ -13,6 +13,12 @@ #define arch_kasan_get_tag(addr) __tag_get(addr) #ifdef CONFIG_KASAN +void kasan_init(void); +#else +static inline void kasan_init(void) { } +#endif + +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) /* * KASAN_SHADOW_START: beginning of the kernel virtual addresses. @@ -33,12 +39,10 @@ #define _KASAN_SHADOW_START(va) (KASAN_SHADOW_END - (1UL << ((va) - KASAN_SHADOW_SCALE_SHIFT))) #define KASAN_SHADOW_START _KASAN_SHADOW_START(vabits_actual) -void kasan_init(void); void kasan_copy_shadow(pgd_t *pgdir); asmlinkage void kasan_early_init(void); #else -static inline void kasan_init(void) { } static inline void kasan_copy_shadow(pgd_t *pgdir) { } #endif --- a/arch/arm64/mm/kasan_init.c~kasan-arm64-only-init-shadow-for-software-modes +++ a/arch/arm64/mm/kasan_init.c @@ -21,6 +21,8 @@ #include #include +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) + static pgd_t tmp_pg_dir[PTRS_PER_PGD] __initdata __aligned(PGD_SIZE); /* @@ -208,7 +210,7 @@ static void __init clear_pgds(unsigned l set_pgd(pgd_offset_k(start), __pgd(0)); } -void __init kasan_init(void) +static void __init kasan_init_shadow(void) { u64 kimg_shadow_start, kimg_shadow_end; u64 mod_shadow_start, mod_shadow_end; @@ -269,6 +271,17 @@ void __init kasan_init(void) memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); +} + +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */ + +static inline void __init kasan_init_shadow(void) { } + +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + +void __init kasan_init(void) +{ + kasan_init_shadow(); /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; From patchwork Tue Dec 22 20:00:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9A9BC43381 for ; Tue, 22 Dec 2020 20:01:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 868D421D1B for ; Tue, 22 Dec 2020 20:01:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 868D421D1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D299D6B00BB; Tue, 22 Dec 2020 15:00:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB2006B00BC; Tue, 22 Dec 2020 15:00:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B054C6B00BD; Tue, 22 Dec 2020 15:00:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0107.hostedemail.com [216.40.44.107]) by kanga.kvack.org (Postfix) with ESMTP id 9543A6B00BB for ; Tue, 22 Dec 2020 15:00:59 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 5E6DA1EE6 for ; Tue, 22 Dec 2020 20:00:59 +0000 (UTC) X-FDA: 77621986638.18.back83_1c04b8c27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 30E06100ED3B0 for ; Tue, 22 Dec 2020 20:00:59 +0000 (UTC) X-HE-Tag: back83_1c04b8c27462 X-Filterd-Recvd-Size: 7117 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:00:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3FA2620705; Tue, 22 Dec 2020 20:00:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667257; bh=rO//3/1UfidKt13OxnpGFo0kBSvNUoU6eix40zc5yKE=; h=Date:From:To:Subject:In-Reply-To:From; b=1+e5G7hLYPu8SnZrD3c4Wg0WCEmcsoFzE7Tj7Uns/6cqNGULa587C8fFSSshCYsif PjjPH53AkmMxroIxSGwWySoEmCNgBnv2GUG0bCgZ0seavEPRTmp60eb1oVN6qTxBHs CG7n6bak397k96U91WJuTsA57h/eGkTVZGCw/KTo= Date: Tue, 22 Dec 2020 12:00:56 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 15/60] kasan, arm64: only use kasan_depth for software modes Message-ID: <20201222200056.QaXj4OC5P%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: only use kasan_depth for software modes This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Hardware tag-based KASAN won't use kasan_depth. Only define and use it when one of the software KASAN modes are enabled. No functional changes for software modes. Link: https://lkml.kernel.org/r/e16f15aeda90bc7fb4dfc2e243a14b74cc5c8219.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/mm/kasan_init.c | 11 ++++++++--- include/linux/kasan.h | 18 +++++++++--------- include/linux/sched.h | 2 +- init/init_task.c | 2 +- mm/kasan/common.c | 2 ++ mm/kasan/report.c | 2 ++ 6 files changed, 23 insertions(+), 14 deletions(-) --- a/arch/arm64/mm/kasan_init.c~kasan-arm64-only-use-kasan_depth-for-software-modes +++ a/arch/arm64/mm/kasan_init.c @@ -273,17 +273,22 @@ static void __init kasan_init_shadow(voi cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); } +static void __init kasan_init_depth(void) +{ + init_task.kasan_depth = 0; +} + #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */ static inline void __init kasan_init_shadow(void) { } +static inline void __init kasan_init_depth(void) { } + #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ void __init kasan_init(void) { kasan_init_shadow(); - - /* At this point kasan is fully initialized. Enable error messages */ - init_task.kasan_depth = 0; + kasan_init_depth(); pr_info("KernelAddressSanitizer initialized\n"); } --- a/include/linux/kasan.h~kasan-arm64-only-use-kasan_depth-for-software-modes +++ a/include/linux/kasan.h @@ -52,6 +52,12 @@ static inline void *kasan_mem_to_shadow( int kasan_add_zero_shadow(void *start, unsigned long size); void kasan_remove_zero_shadow(void *start, unsigned long size); +/* Enable reporting bugs after kasan_disable_current() */ +extern void kasan_enable_current(void); + +/* Disable reporting bugs for current task */ +extern void kasan_disable_current(void); + #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ static inline int kasan_add_zero_shadow(void *start, unsigned long size) @@ -62,16 +68,13 @@ static inline void kasan_remove_zero_sha unsigned long size) {} +static inline void kasan_enable_current(void) {} +static inline void kasan_disable_current(void) {} + #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ #ifdef CONFIG_KASAN -/* Enable reporting bugs after kasan_disable_current() */ -extern void kasan_enable_current(void); - -/* Disable reporting bugs for current task */ -extern void kasan_disable_current(void); - void kasan_unpoison_range(const void *address, size_t size); void kasan_unpoison_task_stack(struct task_struct *task); @@ -122,9 +125,6 @@ static inline void kasan_unpoison_range( static inline void kasan_unpoison_task_stack(struct task_struct *task) {} -static inline void kasan_enable_current(void) {} -static inline void kasan_disable_current(void) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} --- a/include/linux/sched.h~kasan-arm64-only-use-kasan_depth-for-software-modes +++ a/include/linux/sched.h @@ -1234,7 +1234,7 @@ struct task_struct { u64 timer_slack_ns; u64 default_timer_slack_ns; -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) unsigned int kasan_depth; #endif --- a/init/init_task.c~kasan-arm64-only-use-kasan_depth-for-software-modes +++ a/init/init_task.c @@ -176,7 +176,7 @@ struct task_struct init_task .numa_group = NULL, .numa_faults = NULL, #endif -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) .kasan_depth = 1, #endif #ifdef CONFIG_KCSAN --- a/mm/kasan/common.c~kasan-arm64-only-use-kasan_depth-for-software-modes +++ a/mm/kasan/common.c @@ -46,6 +46,7 @@ void kasan_set_track(struct kasan_track track->stack = kasan_save_stack(flags); } +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) void kasan_enable_current(void) { current->kasan_depth++; @@ -55,6 +56,7 @@ void kasan_disable_current(void) { current->kasan_depth--; } +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ void kasan_unpoison_range(const void *address, size_t size) { --- a/mm/kasan/report.c~kasan-arm64-only-use-kasan_depth-for-software-modes +++ a/mm/kasan/report.c @@ -292,8 +292,10 @@ static void print_shadow_for_address(con static bool report_enabled(void) { +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) if (current->kasan_depth) return false; +#endif if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) return true; return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); From patchwork Tue Dec 22 20:01:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05378C433E0 for ; Tue, 22 Dec 2020 20:01:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 895FD2313F for ; Tue, 22 Dec 2020 20:01:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 895FD2313F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7F86D6B00BD; Tue, 22 Dec 2020 15:01:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A9DC6B00BE; Tue, 22 Dec 2020 15:01:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FAA96B00BF; Tue, 22 Dec 2020 15:01:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 41C436B00BD for ; Tue, 22 Dec 2020 15:01:03 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E5B963624 for ; Tue, 22 Dec 2020 20:01:02 +0000 (UTC) X-FDA: 77621986764.26.love01_3d16cd027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id AE64D1804B667 for ; Tue, 22 Dec 2020 20:01:02 +0000 (UTC) X-HE-Tag: love01_3d16cd027462 X-Filterd-Recvd-Size: 5289 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8FB0720771; Tue, 22 Dec 2020 20:01:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667261; bh=0o31pG6AdEpFva+9QxweYjoptW0XURaqpwcvMaqTi7c=; h=Date:From:To:Subject:In-Reply-To:From; b=p8OWnqgwnlk4z+rCYNbPALpAbqflSxLl2VVgpxRQqndUZMA1ZnxNj+nDbBVWdqq8Q pXE57mqlEy0rbiiR+SMDls6PTxDCE8jiRXi9ybjHB2O/ak62n7VxArs+Gfgp1geQUj NA8UCxDH7FqnRGdkTcVtm5w02ie/31QHcHn4Jxjc= Date: Tue, 22 Dec 2020 12:01:00 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 16/60] kasan, arm64: move initialization message Message-ID: <20201222200100.nMe3EBu9c%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: move initialization message Software tag-based KASAN mode is fully initialized with kasan_init_tags(), while the generic mode only requires kasan_init(). Move the initialization message for tag-based mode into kasan_init_tags(). Also fix pr_fmt() usage for KASAN code: generic.c doesn't need it as it doesn't use any printing functions; tag-based mode should use "kasan:" instead of KBUILD_MODNAME (which stands for file name). Link: https://lkml.kernel.org/r/29a30ea4e1750450dd1f693d25b7b6cb05913ecf.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/kasan.h | 9 +++------ arch/arm64/mm/kasan_init.c | 13 +++++-------- mm/kasan/generic.c | 2 -- mm/kasan/sw_tags.c | 4 +++- 4 files changed, 11 insertions(+), 17 deletions(-) --- a/arch/arm64/include/asm/kasan.h~kasan-arm64-move-initialization-message +++ a/arch/arm64/include/asm/kasan.h @@ -12,14 +12,10 @@ #define arch_kasan_reset_tag(addr) __tag_reset(addr) #define arch_kasan_get_tag(addr) __tag_get(addr) -#ifdef CONFIG_KASAN -void kasan_init(void); -#else -static inline void kasan_init(void) { } -#endif - #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) +void kasan_init(void); + /* * KASAN_SHADOW_START: beginning of the kernel virtual addresses. * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses, @@ -43,6 +39,7 @@ void kasan_copy_shadow(pgd_t *pgdir); asmlinkage void kasan_early_init(void); #else +static inline void kasan_init(void) { } static inline void kasan_copy_shadow(pgd_t *pgdir) { } #endif --- a/arch/arm64/mm/kasan_init.c~kasan-arm64-move-initialization-message +++ a/arch/arm64/mm/kasan_init.c @@ -278,17 +278,14 @@ static void __init kasan_init_depth(void init_task.kasan_depth = 0; } -#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) */ - -static inline void __init kasan_init_shadow(void) { } - -static inline void __init kasan_init_depth(void) { } - -#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ - void __init kasan_init(void) { kasan_init_shadow(); kasan_init_depth(); +#if defined(CONFIG_KASAN_GENERIC) + /* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */ pr_info("KernelAddressSanitizer initialized\n"); +#endif } + +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ --- a/mm/kasan/generic.c~kasan-arm64-move-initialization-message +++ a/mm/kasan/generic.c @@ -9,8 +9,6 @@ * Andrey Konovalov */ -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt - #include #include #include --- a/mm/kasan/sw_tags.c~kasan-arm64-move-initialization-message +++ a/mm/kasan/sw_tags.c @@ -6,7 +6,7 @@ * Author: Andrey Konovalov */ -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define pr_fmt(fmt) "kasan: " fmt #include #include @@ -41,6 +41,8 @@ void kasan_init_tags(void) for_each_possible_cpu(cpu) per_cpu(prng_state, cpu) = (u32)get_cycles(); + + pr_info("KernelAddressSanitizer initialized\n"); } /* From patchwork Tue Dec 22 20:01:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E655C433DB for ; Tue, 22 Dec 2020 20:01:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A612023159 for ; Tue, 22 Dec 2020 20:01:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A612023159 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 251186B00BF; Tue, 22 Dec 2020 15:01:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B4C16B00C0; Tue, 22 Dec 2020 15:01:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A3456B00C1; Tue, 22 Dec 2020 15:01:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id DD53F6B00BF for ; Tue, 22 Dec 2020 15:01:06 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9E1DA8249980 for ; Tue, 22 Dec 2020 20:01:06 +0000 (UTC) X-FDA: 77621986932.14.game70_0e060c027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 5303E18010A71 for ; Tue, 22 Dec 2020 20:01:06 +0000 (UTC) X-HE-Tag: game70_0e060c027462 X-Filterd-Recvd-Size: 4347 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 094402313C; Tue, 22 Dec 2020 20:01:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667264; bh=IV8/Ek5M+HJU0AOVakQNsPSidNEzQjIvcDfJcOLMfh4=; h=Date:From:To:Subject:In-Reply-To:From; b=Q2MPz3j5h+ca2IN0oLQihpuu3h/Ki6M9jLUc4QGq4ccBhJNEYLDni/fEXF0Jb3wUn 69nubKoaqh7a4YpWe6ImuYXYUHFGnAHEf26E3LkahC/r7QcHgid+0I1DGbq1D82Suf H0xI+GbANRUD2RbOupvyWCnZkgTJDCRIZjSCN5lQ= Date: Tue, 22 Dec 2020 12:01:03 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 17/60] kasan, arm64: rename kasan_init_tags and mark as __init Message-ID: <20201222200103._POlSBJhz%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: rename kasan_init_tags and mark as __init Rename kasan_init_tags() to kasan_init_sw_tags() as the upcoming hardware tag-based KASAN mode will have its own initialization routine. Also similarly to kasan_init() mark kasan_init_tags() as __init. Link: https://lkml.kernel.org/r/71e52af72a09f4b50c8042f16101c60e50649fbb.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/kernel/setup.c | 2 +- arch/arm64/mm/kasan_init.c | 2 +- include/linux/kasan.h | 4 ++-- mm/kasan/sw_tags.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) --- a/arch/arm64/kernel/setup.c~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init +++ a/arch/arm64/kernel/setup.c @@ -358,7 +358,7 @@ void __init __no_sanitize_address setup_ smp_build_mpidr_hash(); /* Init percpu seeds for random tags after cpus are set up. */ - kasan_init_tags(); + kasan_init_sw_tags(); #ifdef CONFIG_ARM64_SW_TTBR0_PAN /* --- a/arch/arm64/mm/kasan_init.c~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init +++ a/arch/arm64/mm/kasan_init.c @@ -283,7 +283,7 @@ void __init kasan_init(void) kasan_init_shadow(); kasan_init_depth(); #if defined(CONFIG_KASAN_GENERIC) - /* CONFIG_KASAN_SW_TAGS also requires kasan_init_tags(). */ + /* CONFIG_KASAN_SW_TAGS also requires kasan_init_sw_tags(). */ pr_info("KernelAddressSanitizer initialized\n"); #endif } --- a/include/linux/kasan.h~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init +++ a/include/linux/kasan.h @@ -192,7 +192,7 @@ static inline void kasan_record_aux_stac #ifdef CONFIG_KASAN_SW_TAGS -void kasan_init_tags(void); +void __init kasan_init_sw_tags(void); void *kasan_reset_tag(const void *addr); @@ -201,7 +201,7 @@ bool kasan_report(unsigned long addr, si #else /* CONFIG_KASAN_SW_TAGS */ -static inline void kasan_init_tags(void) { } +static inline void kasan_init_sw_tags(void) { } static inline void *kasan_reset_tag(const void *addr) { --- a/mm/kasan/sw_tags.c~kasan-arm64-rename-kasan_init_tags-and-mark-as-__init +++ a/mm/kasan/sw_tags.c @@ -35,7 +35,7 @@ static DEFINE_PER_CPU(u32, prng_state); -void kasan_init_tags(void) +void __init kasan_init_sw_tags(void) { int cpu; From patchwork Tue Dec 22 20:01:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 630B0C433DB for ; Tue, 22 Dec 2020 20:01:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EF642332A for ; Tue, 22 Dec 2020 20:01:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EF642332A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 363C56B00C0; Tue, 22 Dec 2020 15:01:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EE376B00C1; Tue, 22 Dec 2020 15:01:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DC006B00C2; Tue, 22 Dec 2020 15:01:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 006C96B00C0 for ; Tue, 22 Dec 2020 15:01:09 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C58A91EE6 for ; Tue, 22 Dec 2020 20:01:09 +0000 (UTC) X-FDA: 77621987058.08.pail85_0d0ed2027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id A544D1819E793 for ; Tue, 22 Dec 2020 20:01:09 +0000 (UTC) X-HE-Tag: pail85_0d0ed2027462 X-Filterd-Recvd-Size: 4537 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6627E21D1B; Tue, 22 Dec 2020 20:01:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667268; bh=CntEdFeEhpEy8nANGl/CKXpvCpvnknYL6NlgBt4vUtk=; h=Date:From:To:Subject:In-Reply-To:From; b=AXw+dT5Kk5nTnDmzxLgWHprV59YVX6/0wkUurtNU1bRzlPQ2BQXIoytYsqXjI4FNi 3LvqXqKasrGAtd+vbMSHsngOS/AQoFLBnh+BuPESnI5Zm4+mIGv3fh+13Hb5T2iU1R pSS1G+bwe+mLH9VpgIDx7a+6jX6xDg5hUhjzHijo= Date: Tue, 22 Dec 2020 12:01:07 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 18/60] kasan: rename addr_has_shadow to addr_has_metadata Message-ID: <20201222200107.iVjy6s2QW%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000309, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename addr_has_shadow to addr_has_metadata This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Hardware tag-based KASAN won't be using shadow memory, but will reuse this function. Rename "shadow" to implementation-neutral "metadata". No functional changes. Link: https://lkml.kernel.org/r/370466fba590a4596b55ffd38adfd990f8886db4.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/kasan.h | 2 +- mm/kasan/report.c | 6 +++--- mm/kasan/report_generic.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) --- a/mm/kasan/kasan.h~kasan-rename-addr_has_shadow-to-addr_has_metadata +++ a/mm/kasan/kasan.h @@ -147,7 +147,7 @@ static inline const void *kasan_shadow_t << KASAN_SHADOW_SCALE_SHIFT); } -static inline bool addr_has_shadow(const void *addr) +static inline bool addr_has_metadata(const void *addr) { return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); } --- a/mm/kasan/report.c~kasan-rename-addr_has_shadow-to-addr_has_metadata +++ a/mm/kasan/report.c @@ -361,7 +361,7 @@ static void __kasan_report(unsigned long untagged_addr = reset_tag(tagged_addr); info.access_addr = tagged_addr; - if (addr_has_shadow(untagged_addr)) + if (addr_has_metadata(untagged_addr)) info.first_bad_addr = find_first_bad_addr(tagged_addr, size); else info.first_bad_addr = untagged_addr; @@ -372,11 +372,11 @@ static void __kasan_report(unsigned long start_report(&flags); print_error_description(&info); - if (addr_has_shadow(untagged_addr)) + if (addr_has_metadata(untagged_addr)) print_tags(get_tag(tagged_addr), info.first_bad_addr); pr_err("\n"); - if (addr_has_shadow(untagged_addr)) { + if (addr_has_metadata(untagged_addr)) { print_address_description(untagged_addr, get_tag(tagged_addr)); pr_err("\n"); print_shadow_for_address(info.first_bad_addr); --- a/mm/kasan/report_generic.c~kasan-rename-addr_has_shadow-to-addr_has_metadata +++ a/mm/kasan/report_generic.c @@ -118,7 +118,7 @@ const char *get_bug_type(struct kasan_ac if (info->access_addr + info->access_size < info->access_addr) return "out-of-bounds"; - if (addr_has_shadow(info->access_addr)) + if (addr_has_metadata(info->access_addr)) return get_shadow_bug_type(info); return get_wild_bug_type(info); } From patchwork Tue Dec 22 20:01:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF39CC433E0 for ; Tue, 22 Dec 2020 20:01:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5988923331 for ; Tue, 22 Dec 2020 20:01:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5988923331 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 908E76B00C1; Tue, 22 Dec 2020 15:01:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 86A3F6B00C3; Tue, 22 Dec 2020 15:01:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70BD26B00C4; Tue, 22 Dec 2020 15:01:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 5342C6B00C1 for ; Tue, 22 Dec 2020 15:01:13 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 18E0D181AEF32 for ; Tue, 22 Dec 2020 20:01:13 +0000 (UTC) X-FDA: 77621987226.19.tramp30_1e1693427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id DCA6C1ACEAD for ; Tue, 22 Dec 2020 20:01:12 +0000 (UTC) X-HE-Tag: tramp30_1e1693427462 X-Filterd-Recvd-Size: 3675 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EA34921D81; Tue, 22 Dec 2020 20:01:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667271; bh=2B/NNjgKynQJZv4Y6anRy8CVgpGnS6XKSXDRJna02cY=; h=Date:From:To:Subject:In-Reply-To:From; b=frmpgrx6Dr9aWTySZuUwloIMZnmG3NWspw52TjkqGthVAqLSWuwlYsLUzSS4FfPSS TcTLyoD1ORQsCsHRzj8PoEI0fTUHWwbovttI5/eMZboBNd6dAHgSjUva18hUGNaAit G5pA8hpNyPd7Ka0LmnAOvuCwevyiqvks/EzkERNQ= Date: Tue, 22 Dec 2020 12:01:10 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 19/60] kasan: rename print_shadow_for_address to print_memory_metadata Message-ID: <20201222200110.QNReXF7AZ%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename print_shadow_for_address to print_memory_metadata This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Hardware tag-based KASAN won't be using shadow memory, but will reuse this function. Rename "shadow" to implementation-neutral "metadata". No functional changes. Link: https://lkml.kernel.org/r/dd955c5aadaee16aef451a6189d19172166a23f5.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/report.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/mm/kasan/report.c~kasan-rename-print_shadow_for_address-to-print_memory_metadata +++ a/mm/kasan/report.c @@ -252,7 +252,7 @@ static int shadow_pointer_offset(const v (shadow - row) / SHADOW_BYTES_PER_BLOCK + 1; } -static void print_shadow_for_address(const void *addr) +static void print_memory_metadata(const void *addr) { int i; const void *shadow = kasan_mem_to_shadow(addr); @@ -338,7 +338,7 @@ void kasan_report_invalid_free(void *obj pr_err("\n"); print_address_description(object, tag); pr_err("\n"); - print_shadow_for_address(object); + print_memory_metadata(object); end_report(&flags); } @@ -379,7 +379,7 @@ static void __kasan_report(unsigned long if (addr_has_metadata(untagged_addr)) { print_address_description(untagged_addr, get_tag(tagged_addr)); pr_err("\n"); - print_shadow_for_address(info.first_bad_addr); + print_memory_metadata(info.first_bad_addr); } else { dump_stack(); } From patchwork Tue Dec 22 20:01:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1468C433DB for ; Tue, 22 Dec 2020 20:01:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4352D23332 for ; Tue, 22 Dec 2020 20:01:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4352D23332 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6219B6B00C4; Tue, 22 Dec 2020 15:01:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5AB206B00C5; Tue, 22 Dec 2020 15:01:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44D8D6B00C6; Tue, 22 Dec 2020 15:01:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 233B86B00C4 for ; Tue, 22 Dec 2020 15:01:17 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D307E3624 for ; Tue, 22 Dec 2020 20:01:16 +0000 (UTC) X-FDA: 77621987352.04.soup10_520b8c127462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 891C7800CE0E for ; Tue, 22 Dec 2020 20:01:16 +0000 (UTC) X-HE-Tag: soup10_520b8c127462 X-Filterd-Recvd-Size: 5499 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 84F1C2313F; Tue, 22 Dec 2020 20:01:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667275; bh=WkJP5ueegwzdiDDMvDzqZf2U3rSH1/aHsOmLVGy5lhQ=; h=Date:From:To:Subject:In-Reply-To:From; b=FkFYZ+YIHPLA8OrDGuOC9XRX6LR/LQWCz8kC+gpM27b1RIc6tDM3Wz1qiiOATH2Ty owRai9uRIbVWbhpzLxJ59hQl5wKBpwEybHPAlDJLnsRnLvpOIFNfvf9WddESCAzQCF lqohOeme5kqh/Y0IF1Yjntv7yoku8b6s9Af2Yyu4= Date: Tue, 22 Dec 2020 12:01:14 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 20/60] kasan: rename SHADOW layout macros to META Message-ID: <20201222200114.mt0mOLilb%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename SHADOW layout macros to META This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Hardware tag-based KASAN won't be using shadow memory, but will reuse these macros. Rename "SHADOW" to implementation-neutral "META". No functional changes. Link: https://lkml.kernel.org/r/f96244ec59dc17db35173ec352c5592b14aefaf8.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/report.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) --- a/mm/kasan/report.c~kasan-rename-shadow-layout-macros-to-meta +++ a/mm/kasan/report.c @@ -33,11 +33,11 @@ #include "kasan.h" #include "../slab.h" -/* Shadow layout customization. */ -#define SHADOW_BYTES_PER_BLOCK 1 -#define SHADOW_BLOCKS_PER_ROW 16 -#define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) -#define SHADOW_ROWS_AROUND_ADDR 2 +/* Metadata layout customization. */ +#define META_BYTES_PER_BLOCK 1 +#define META_BLOCKS_PER_ROW 16 +#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK) +#define META_ROWS_AROUND_ADDR 2 static unsigned long kasan_flags; @@ -240,7 +240,7 @@ static void print_address_description(vo static bool row_is_guilty(const void *row, const void *guilty) { - return (row <= guilty) && (guilty < row + SHADOW_BYTES_PER_ROW); + return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW); } static int shadow_pointer_offset(const void *row, const void *shadow) @@ -249,7 +249,7 @@ static int shadow_pointer_offset(const v * 3 + (BITS_PER_LONG/8)*2 chars. */ return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 + - (shadow - row) / SHADOW_BYTES_PER_BLOCK + 1; + (shadow - row) / META_BYTES_PER_BLOCK + 1; } static void print_memory_metadata(const void *addr) @@ -259,15 +259,15 @@ static void print_memory_metadata(const const void *shadow_row; shadow_row = (void *)round_down((unsigned long)shadow, - SHADOW_BYTES_PER_ROW) - - SHADOW_ROWS_AROUND_ADDR * SHADOW_BYTES_PER_ROW; + META_BYTES_PER_ROW) + - META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW; pr_err("Memory state around the buggy address:\n"); - for (i = -SHADOW_ROWS_AROUND_ADDR; i <= SHADOW_ROWS_AROUND_ADDR; i++) { + for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) { const void *kaddr = kasan_shadow_to_mem(shadow_row); char buffer[4 + (BITS_PER_LONG/8)*2]; - char shadow_buf[SHADOW_BYTES_PER_ROW]; + char shadow_buf[META_BYTES_PER_ROW]; snprintf(buffer, sizeof(buffer), (i == 0) ? ">%px: " : " %px: ", kaddr); @@ -276,17 +276,17 @@ static void print_memory_metadata(const * function, because generic functions may try to * access kasan mapping for the passed address. */ - memcpy(shadow_buf, shadow_row, SHADOW_BYTES_PER_ROW); + memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW); print_hex_dump(KERN_ERR, buffer, - DUMP_PREFIX_NONE, SHADOW_BYTES_PER_ROW, 1, - shadow_buf, SHADOW_BYTES_PER_ROW, 0); + DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1, + shadow_buf, META_BYTES_PER_ROW, 0); if (row_is_guilty(shadow_row, shadow)) pr_err("%*c\n", shadow_pointer_offset(shadow_row, shadow), '^'); - shadow_row += SHADOW_BYTES_PER_ROW; + shadow_row += META_BYTES_PER_ROW; } } From patchwork Tue Dec 22 20:01:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80438C433E9 for ; Tue, 22 Dec 2020 20:01:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 270C92333B for ; Tue, 22 Dec 2020 20:01:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 270C92333B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B79866B00C6; Tue, 22 Dec 2020 15:01:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB3846B00C7; Tue, 22 Dec 2020 15:01:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 956866B00C8; Tue, 22 Dec 2020 15:01:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 79FA16B00C6 for ; Tue, 22 Dec 2020 15:01:20 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3AB9C8249980 for ; Tue, 22 Dec 2020 20:01:20 +0000 (UTC) X-FDA: 77621987520.22.wheel03_0e02f8027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 16A5D18038E68 for ; Tue, 22 Dec 2020 20:01:20 +0000 (UTC) X-HE-Tag: wheel03_0e02f8027462 X-Filterd-Recvd-Size: 8029 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0A3D023159; Tue, 22 Dec 2020 20:01:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667278; bh=SZTQn9fjqbA0ifLQitsdafMEg0vq3YSpL7vQ+CatsAs=; h=Date:From:To:Subject:In-Reply-To:From; b=gCrS98moG/qMWsHVeV+Vsh79MPJ2oDoMLFb1vBvfOdjdC5EM5UXZ4BXE2eNwslGSa UCNH9aAzmhr61mq+Q0370sDiCxX1VxVHGk2LTxxHholrNdxKQziSYS2PthAU1fnSwH tkxAkk/cyvg2eFK1Iw9GN3oX+wBFwVtPHY7TBXfw= Date: Tue, 22 Dec 2020 12:01:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 21/60] kasan: separate metadata_fetch_row for each mode Message-ID: <20201222200117.RQORRRZUR%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: separate metadata_fetch_row for each mode This is a preparatory commit for the upcoming addition of a new hardware tag-based (MTE-based) KASAN mode. Rework print_memory_metadata() to make it agnostic with regard to the way metadata is stored. Allow providing a separate metadata_fetch_row() implementation for each KASAN mode. Hardware tag-based KASAN will provide its own implementation that doesn't use shadow memory. No functional changes for software modes. Link: https://lkml.kernel.org/r/5fb1ec0152bb1f521505017800387ec3e36ffe18.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/kasan.h | 8 +++++ mm/kasan/report.c | 56 +++++++++++++++++------------------- mm/kasan/report_generic.c | 5 +++ mm/kasan/report_sw_tags.c | 5 +++ 4 files changed, 45 insertions(+), 29 deletions(-) --- a/mm/kasan/kasan.h~kasan-separate-metadata_fetch_row-for-each-mode +++ a/mm/kasan/kasan.h @@ -58,6 +58,13 @@ #define KASAN_ABI_VERSION 1 #endif +/* Metadata layout customization. */ +#define META_BYTES_PER_BLOCK 1 +#define META_BLOCKS_PER_ROW 16 +#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK) +#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_GRANULE_SIZE) +#define META_ROWS_AROUND_ADDR 2 + struct kasan_access_info { const void *access_addr; const void *first_bad_addr; @@ -170,6 +177,7 @@ bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); +void metadata_fetch_row(char *buffer, void *row); #if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK void print_address_stack_frame(const void *addr); --- a/mm/kasan/report.c~kasan-separate-metadata_fetch_row-for-each-mode +++ a/mm/kasan/report.c @@ -33,12 +33,6 @@ #include "kasan.h" #include "../slab.h" -/* Metadata layout customization. */ -#define META_BYTES_PER_BLOCK 1 -#define META_BLOCKS_PER_ROW 16 -#define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK) -#define META_ROWS_AROUND_ADDR 2 - static unsigned long kasan_flags; #define KASAN_BIT_REPORTED 0 @@ -238,55 +232,59 @@ static void print_address_description(vo print_address_stack_frame(addr); } -static bool row_is_guilty(const void *row, const void *guilty) +static bool meta_row_is_guilty(const void *row, const void *addr) { - return (row <= guilty) && (guilty < row + META_BYTES_PER_ROW); + return (row <= addr) && (addr < row + META_MEM_BYTES_PER_ROW); } -static int shadow_pointer_offset(const void *row, const void *shadow) +static int meta_pointer_offset(const void *row, const void *addr) { - /* The length of ">ff00ff00ff00ff00: " is - * 3 + (BITS_PER_LONG/8)*2 chars. + /* + * Memory state around the buggy address: + * ff00ff00ff00ff00: 00 00 00 05 fe fe fe fe fe fe fe fe fe fe fe fe + * ... + * + * The length of ">ff00ff00ff00ff00: " is + * 3 + (BITS_PER_LONG / 8) * 2 chars. + * The length of each granule metadata is 2 bytes + * plus 1 byte for space. */ - return 3 + (BITS_PER_LONG/8)*2 + (shadow - row)*2 + - (shadow - row) / META_BYTES_PER_BLOCK + 1; + return 3 + (BITS_PER_LONG / 8) * 2 + + (addr - row) / KASAN_GRANULE_SIZE * 3 + 1; } static void print_memory_metadata(const void *addr) { int i; - const void *shadow = kasan_mem_to_shadow(addr); - const void *shadow_row; + void *row; - shadow_row = (void *)round_down((unsigned long)shadow, - META_BYTES_PER_ROW) - - META_ROWS_AROUND_ADDR * META_BYTES_PER_ROW; + row = (void *)round_down((unsigned long)addr, META_MEM_BYTES_PER_ROW) + - META_ROWS_AROUND_ADDR * META_MEM_BYTES_PER_ROW; pr_err("Memory state around the buggy address:\n"); for (i = -META_ROWS_AROUND_ADDR; i <= META_ROWS_AROUND_ADDR; i++) { - const void *kaddr = kasan_shadow_to_mem(shadow_row); - char buffer[4 + (BITS_PER_LONG/8)*2]; - char shadow_buf[META_BYTES_PER_ROW]; + char buffer[4 + (BITS_PER_LONG / 8) * 2]; + char metadata[META_BYTES_PER_ROW]; snprintf(buffer, sizeof(buffer), - (i == 0) ? ">%px: " : " %px: ", kaddr); + (i == 0) ? ">%px: " : " %px: ", row); + /* * We should not pass a shadow pointer to generic * function, because generic functions may try to * access kasan mapping for the passed address. */ - memcpy(shadow_buf, shadow_row, META_BYTES_PER_ROW); + metadata_fetch_row(&metadata[0], row); + print_hex_dump(KERN_ERR, buffer, DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1, - shadow_buf, META_BYTES_PER_ROW, 0); + metadata, META_BYTES_PER_ROW, 0); - if (row_is_guilty(shadow_row, shadow)) - pr_err("%*c\n", - shadow_pointer_offset(shadow_row, shadow), - '^'); + if (meta_row_is_guilty(row, addr)) + pr_err("%*c\n", meta_pointer_offset(row, addr), '^'); - shadow_row += META_BYTES_PER_ROW; + row += META_MEM_BYTES_PER_ROW; } } --- a/mm/kasan/report_generic.c~kasan-separate-metadata_fetch_row-for-each-mode +++ a/mm/kasan/report_generic.c @@ -123,6 +123,11 @@ const char *get_bug_type(struct kasan_ac return get_wild_bug_type(info); } +void metadata_fetch_row(char *buffer, void *row) +{ + memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW); +} + #if CONFIG_KASAN_STACK static bool __must_check tokenize_frame_descr(const char **frame_descr, char *token, size_t max_tok_len, --- a/mm/kasan/report_sw_tags.c~kasan-separate-metadata_fetch_row-for-each-mode +++ a/mm/kasan/report_sw_tags.c @@ -80,6 +80,11 @@ void *find_first_bad_addr(void *addr, si return p; } +void metadata_fetch_row(char *buffer, void *row) +{ + memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW); +} + void print_tags(u8 addr_tag, const void *addr) { u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); From patchwork Tue Dec 22 20:01:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0718C433E0 for ; Tue, 22 Dec 2020 20:01:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8ADB120867 for ; Tue, 22 Dec 2020 20:01:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8ADB120867 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 64DCA6B00C8; Tue, 22 Dec 2020 15:01:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 55C7B6B00C9; Tue, 22 Dec 2020 15:01:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3AFD76B00CA; Tue, 22 Dec 2020 15:01:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0032.hostedemail.com [216.40.44.32]) by kanga.kvack.org (Postfix) with ESMTP id 1C1796B00C8 for ; Tue, 22 Dec 2020 15:01:24 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CDC9D8249980 for ; Tue, 22 Dec 2020 20:01:23 +0000 (UTC) X-FDA: 77621987646.21.talk30_5307a8027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 7B1F9180442CB for ; Tue, 22 Dec 2020 20:01:23 +0000 (UTC) X-HE-Tag: talk30_5307a8027462 X-Filterd-Recvd-Size: 7625 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5817A2332A; Tue, 22 Dec 2020 20:01:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667282; bh=aZ20VFyvvs6LomSn07Oc4uZtBKQwry8f4qJ1pOVDoBk=; h=Date:From:To:Subject:In-Reply-To:From; b=SEr5TZATQbH29/gA87nSbl2F7O5wlJWXX0VqlCl/cmCyWVQOi2CHvTIbOUhJsVzXf eMK+RPzbaYEbTEl0kSd8Eknqu48vuqjfFTlv0+YgCOPQeexEcQORcKvh+oh20JsH8U eYoXg0PUTqi+QMffJOCV0nfKX3gheQg3TP4eYXEQ= Date: Tue, 22 Dec 2020 12:01:20 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 22/60] kasan: introduce CONFIG_KASAN_HW_TAGS Message-ID: <20201222200120.Kv4I4h0r3%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: introduce CONFIG_KASAN_HW_TAGS This patch adds a configuration option for a new KASAN mode called hardware tag-based KASAN. This mode uses the memory tagging approach like the software tag-based mode, but relies on arm64 Memory Tagging Extension feature for tag management and access checking. Link: https://lkml.kernel.org/r/44906a209d3a44f9c6f5a21841e90988e365601e.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/Kconfig.kasan | 61 +++++++++++++++++++++++++++++++------------- 1 file changed, 44 insertions(+), 17 deletions(-) --- a/lib/Kconfig.kasan~kasan-introduce-config_kasan_hw_tags +++ a/lib/Kconfig.kasan @@ -6,7 +6,10 @@ config HAVE_ARCH_KASAN config HAVE_ARCH_KASAN_SW_TAGS bool -config HAVE_ARCH_KASAN_VMALLOC +config HAVE_ARCH_KASAN_HW_TAGS + bool + +config HAVE_ARCH_KASAN_VMALLOC bool config CC_HAS_KASAN_GENERIC @@ -15,16 +18,19 @@ config CC_HAS_KASAN_GENERIC config CC_HAS_KASAN_SW_TAGS def_bool $(cc-option, -fsanitize=kernel-hwaddress) +# This option is only required for software KASAN modes. +# Old GCC versions don't have proper support for no_sanitize_address. +# See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89124 for details. config CC_HAS_WORKING_NOSANITIZE_ADDRESS def_bool !CC_IS_GCC || GCC_VERSION >= 80300 menuconfig KASAN bool "KASAN: runtime memory debugger" - depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ - (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) + depends on (((HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ + (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS)) && \ + CC_HAS_WORKING_NOSANITIZE_ADDRESS) || \ + HAVE_ARCH_KASAN_HW_TAGS depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) - depends on CC_HAS_WORKING_NOSANITIZE_ADDRESS - select CONSTRUCTORS select STACKDEPOT help Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, @@ -37,18 +43,24 @@ choice prompt "KASAN mode" default KASAN_GENERIC help - KASAN has two modes: generic KASAN (similar to userspace ASan, - x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and - software tag-based KASAN (a version based on software memory - tagging, arm64 only, similar to userspace HWASan, enabled with - CONFIG_KASAN_SW_TAGS). + KASAN has three modes: + 1. generic KASAN (similar to userspace ASan, + x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC), + 2. software tag-based KASAN (arm64 only, based on software + memory tagging (similar to userspace HWASan), enabled with + CONFIG_KASAN_SW_TAGS), and + 3. hardware tag-based KASAN (arm64 only, based on hardware + memory tagging, enabled with CONFIG_KASAN_HW_TAGS). + + All KASAN modes are strictly debugging features. - Both generic and tag-based KASAN are strictly debugging features. + For better error reports enable CONFIG_STACKTRACE. config KASAN_GENERIC bool "Generic mode" depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC select SLUB_DEBUG if SLUB + select CONSTRUCTORS help Enables generic KASAN mode. @@ -61,8 +73,6 @@ config KASAN_GENERIC and introduces an overhead of ~x1.5 for the rest of the allocations. The performance slowdown is ~x3. - For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). @@ -70,11 +80,15 @@ config KASAN_SW_TAGS bool "Software tag-based mode" depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS select SLUB_DEBUG if SLUB + select CONSTRUCTORS help Enables software tag-based KASAN mode. - This mode requires Top Byte Ignore support by the CPU and therefore - is only supported for arm64. This mode requires Clang. + This mode require software memory tagging support in the form of + HWASan-like compiler instrumentation. + + Currently this mode is only implemented for arm64 CPUs and relies on + Top Byte Ignore. This mode requires Clang. This mode consumes about 1/16th of available memory at kernel start and introduces an overhead of ~20% for the rest of the allocations. @@ -82,15 +96,27 @@ config KASAN_SW_TAGS casting and comparison, as it embeds tags into the top byte of each pointer. - For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +config KASAN_HW_TAGS + bool "Hardware tag-based mode" + depends on HAVE_ARCH_KASAN_HW_TAGS + depends on SLUB + help + Enables hardware tag-based KASAN mode. + + This mode requires hardware memory tagging support, and can be used + by any architecture that provides it. + + Currently this mode is only implemented for arm64 CPUs starting from + ARMv8.5 and relies on Memory Tagging Extension and Top Byte Ignore. + endchoice choice prompt "Instrumentation type" + depends on KASAN_GENERIC || KASAN_SW_TAGS default KASAN_OUTLINE config KASAN_OUTLINE @@ -114,6 +140,7 @@ endchoice config KASAN_STACK_ENABLE bool "Enable stack instrumentation (unsafe)" if CC_IS_CLANG && !COMPILE_TEST + depends on KASAN_GENERIC || KASAN_SW_TAGS help The LLVM stack address sanitizer has a know problem that causes excessive stack usage in a lot of functions, see From patchwork Tue Dec 22 20:01:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 929B3C433E9 for ; Tue, 22 Dec 2020 20:01:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2F50120700 for ; Tue, 22 Dec 2020 20:01:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F50120700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BED408D000F; Tue, 22 Dec 2020 15:01:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B75488D0003; Tue, 22 Dec 2020 15:01:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8DB48D000F; Tue, 22 Dec 2020 15:01:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id 8B4208D0003 for ; Tue, 22 Dec 2020 15:01:32 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 4D53D3638 for ; Tue, 22 Dec 2020 20:01:32 +0000 (UTC) X-FDA: 77621988024.06.cow32_1e0aabc27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id E04DD10046C86 for ; Tue, 22 Dec 2020 20:01:26 +0000 (UTC) X-HE-Tag: cow32_1e0aabc27462 X-Filterd-Recvd-Size: 3677 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DFC8B23331; Tue, 22 Dec 2020 20:01:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667285; bh=o4LWwQlSOpED0V2os9lzQIgkMFv6BxZ2jMv9cz3gbxo=; h=Date:From:To:Subject:In-Reply-To:From; b=kgjqxayu+tLuqIE2MD1+9Xt6dFKLNWUe70Vb07j0ztC/cFTkHaoShefYqtQVJbMUB oHVoiFHrpnJaNxm4MmsIxRNFmCmDMLd5Y5W7xgFmNXJPh2DDhKmvJDtZV2kAIcUW6g lLjQNHBjde21VWEjvfa6mcJFsiSgVjP6OEswWiAM= Date: Tue, 22 Dec 2020 12:01:24 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 23/60] arm64: enable armv8.5-a asm-arch option Message-ID: <20201222200124.8TIWn8vqx%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: enable armv8.5-a asm-arch option Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) which is an armv8.5-a architecture extension. Enable the correct asm option when the compiler supports it in order to allow the usage of ALTERNATIVE()s with MTE instructions. Link: https://lkml.kernel.org/r/d03d1157124ea3532eaeb77507988733f5734986.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 4 ++++ arch/arm64/Makefile | 5 +++++ 2 files changed, 9 insertions(+) --- a/arch/arm64/Kconfig~arm64-enable-armv85-a-asm-arch-option +++ a/arch/arm64/Kconfig @@ -1571,6 +1571,9 @@ endmenu menu "ARMv8.5 architectural features" +config AS_HAS_ARMV8_5 + def_bool $(cc-option,-Wa$(comma)-march=armv8.5-a) + config ARM64_BTI bool "Branch Target Identification support" default y @@ -1645,6 +1648,7 @@ config ARM64_MTE bool "Memory Tagging Extension support" default y depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI + depends on AS_HAS_ARMV8_5 select ARCH_USES_HIGH_VMA_FLAGS help Memory Tagging (part of the ARMv8.5 Extensions) provides --- a/arch/arm64/Makefile~arm64-enable-armv85-a-asm-arch-option +++ a/arch/arm64/Makefile @@ -96,6 +96,11 @@ ifeq ($(CONFIG_AS_HAS_ARMV8_4), y) asm-arch := armv8.4-a endif +ifeq ($(CONFIG_AS_HAS_ARMV8_5), y) +# make sure to pass the newest target architecture to -march. +asm-arch := armv8.5-a +endif + ifdef asm-arch KBUILD_CFLAGS += -Wa,-march=$(asm-arch) \ -DARM64_ASM_ARCH='"$(asm-arch)"' From patchwork Tue Dec 22 20:01:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B078AC433E0 for ; Tue, 22 Dec 2020 20:01:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4624E20867 for ; Tue, 22 Dec 2020 20:01:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4624E20867 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 278596B00CA; Tue, 22 Dec 2020 15:01:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D9336B00CB; Tue, 22 Dec 2020 15:01:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C8CD6B00CC; Tue, 22 Dec 2020 15:01:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id E5F846B00CA for ; Tue, 22 Dec 2020 15:01:30 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A8F2F180AD83B for ; Tue, 22 Dec 2020 20:01:30 +0000 (UTC) X-FDA: 77621987940.24.store81_3c0a63427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 8A2531A4A0 for ; Tue, 22 Dec 2020 20:01:30 +0000 (UTC) X-HE-Tag: store81_3c0a63427462 X-Filterd-Recvd-Size: 8782 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6D2BD23333; Tue, 22 Dec 2020 20:01:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667289; bh=76oWDf5f4w1BlWSph6gBIkwhthhPKhw/sXWIAqJa58Y=; h=Date:From:To:Subject:In-Reply-To:From; b=CBeOPS8V8UbYMScp9wk0/OTyzx+VcMCbs3xL08QImyRxenBESNUZJl2RuNgb0O+cX 8UCxG+GlQKppdZrvTKk/VtTVJGUsCbWyjKjEn6L8Pn18DMTOnqhsjma0BGjLFbWxX3 SwvP42DlugeFwU+X7rmYmMTAUX6t54CRR8E7SDyc= Date: Tue, 22 Dec 2020 12:01:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 24/60] arm64: mte: add in-kernel MTE helpers Message-ID: <20201222200128.a-NXc9wRV%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: mte: add in-kernel MTE helpers Provide helper functions to manipulate allocation and pointer tags for kernel addresses. Low-level helper functions (mte_assign_*, written in assembly) operate tag values from the [0x0, 0xF] range. High-level helper functions (mte_get/set_*) use the [0xF0, 0xFF] range to preserve compatibility with normal kernel pointers that have 0xFF in their top byte. MTE_GRANULE_SIZE and related definitions are moved to mte-def.h header that doesn't have any dependencies and is safe to include into any low-level header. Link: https://lkml.kernel.org/r/c31bf759b4411b2d98cdd801eb928e241584fd1f.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/esr.h | 1 arch/arm64/include/asm/mte-def.h | 15 +++++++ arch/arm64/include/asm/mte-kasan.h | 56 +++++++++++++++++++++++++++ arch/arm64/include/asm/mte.h | 20 ++++++--- arch/arm64/kernel/mte.c | 48 +++++++++++++++++++++++ arch/arm64/lib/mte.S | 16 +++++++ 6 files changed, 150 insertions(+), 6 deletions(-) --- a/arch/arm64/include/asm/esr.h~arm64-mte-add-in-kernel-mte-helpers +++ a/arch/arm64/include/asm/esr.h @@ -106,6 +106,7 @@ #define ESR_ELx_FSC_TYPE (0x3C) #define ESR_ELx_FSC_LEVEL (0x03) #define ESR_ELx_FSC_EXTABT (0x10) +#define ESR_ELx_FSC_MTE (0x11) #define ESR_ELx_FSC_SERROR (0x11) #define ESR_ELx_FSC_ACCESS (0x08) #define ESR_ELx_FSC_FAULT (0x04) --- /dev/null +++ a/arch/arm64/include/asm/mte-def.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020 ARM Ltd. + */ +#ifndef __ASM_MTE_DEF_H +#define __ASM_MTE_DEF_H + +#define MTE_GRANULE_SIZE UL(16) +#define MTE_GRANULE_MASK (~(MTE_GRANULE_SIZE - 1)) +#define MTE_TAG_SHIFT 56 +#define MTE_TAG_SIZE 4 +#define MTE_TAG_MASK GENMASK((MTE_TAG_SHIFT + (MTE_TAG_SIZE - 1)), MTE_TAG_SHIFT) +#define MTE_TAG_MAX (MTE_TAG_MASK >> MTE_TAG_SHIFT) + +#endif /* __ASM_MTE_DEF_H */ --- a/arch/arm64/include/asm/mte.h~arm64-mte-add-in-kernel-mte-helpers +++ a/arch/arm64/include/asm/mte.h @@ -5,14 +5,16 @@ #ifndef __ASM_MTE_H #define __ASM_MTE_H -#define MTE_GRANULE_SIZE UL(16) -#define MTE_GRANULE_MASK (~(MTE_GRANULE_SIZE - 1)) -#define MTE_TAG_SHIFT 56 -#define MTE_TAG_SIZE 4 +#include +#include + +#define __MTE_PREAMBLE ARM64_ASM_PREAMBLE ".arch_extension memtag\n" #ifndef __ASSEMBLY__ +#include #include +#include #include @@ -45,7 +47,9 @@ long get_mte_ctrl(struct task_struct *ta int mte_ptrace_copy_tags(struct task_struct *child, long request, unsigned long addr, unsigned long data); -#else +void mte_assign_mem_tag_range(void *addr, size_t size); + +#else /* CONFIG_ARM64_MTE */ /* unused if !CONFIG_ARM64_MTE, silence the compiler */ #define PG_mte_tagged 0 @@ -80,7 +84,11 @@ static inline int mte_ptrace_copy_tags(s return -EIO; } -#endif +static inline void mte_assign_mem_tag_range(void *addr, size_t size) +{ +} + +#endif /* CONFIG_ARM64_MTE */ #endif /* __ASSEMBLY__ */ #endif /* __ASM_MTE_H */ --- /dev/null +++ a/arch/arm64/include/asm/mte-kasan.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020 ARM Ltd. + */ +#ifndef __ASM_MTE_KASAN_H +#define __ASM_MTE_KASAN_H + +#include + +#ifndef __ASSEMBLY__ + +#include + +/* + * The functions below are meant to be used only for the + * KASAN_HW_TAGS interface defined in asm/memory.h. + */ +#ifdef CONFIG_ARM64_MTE + +static inline u8 mte_get_ptr_tag(void *ptr) +{ + /* Note: The format of KASAN tags is 0xF */ + u8 tag = 0xF0 | (u8)(((u64)(ptr)) >> MTE_TAG_SHIFT); + + return tag; +} + +u8 mte_get_mem_tag(void *addr); +u8 mte_get_random_tag(void); +void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag); + +#else /* CONFIG_ARM64_MTE */ + +static inline u8 mte_get_ptr_tag(void *ptr) +{ + return 0xFF; +} + +static inline u8 mte_get_mem_tag(void *addr) +{ + return 0xFF; +} +static inline u8 mte_get_random_tag(void) +{ + return 0xFF; +} +static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) +{ + return addr; +} + +#endif /* CONFIG_ARM64_MTE */ + +#endif /* __ASSEMBLY__ */ + +#endif /* __ASM_MTE_KASAN_H */ --- a/arch/arm64/kernel/mte.c~arm64-mte-add-in-kernel-mte-helpers +++ a/arch/arm64/kernel/mte.c @@ -13,10 +13,13 @@ #include #include #include +#include #include +#include #include #include +#include #include #include @@ -72,6 +75,51 @@ int memcmp_pages(struct page *page1, str return ret; } +u8 mte_get_mem_tag(void *addr) +{ + if (!system_supports_mte()) + return 0xFF; + + asm(__MTE_PREAMBLE "ldg %0, [%0]" + : "+r" (addr)); + + return mte_get_ptr_tag(addr); +} + +u8 mte_get_random_tag(void) +{ + void *addr; + + if (!system_supports_mte()) + return 0xFF; + + asm(__MTE_PREAMBLE "irg %0, %0" + : "+r" (addr)); + + return mte_get_ptr_tag(addr); +} + +void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) +{ + void *ptr = addr; + + if ((!system_supports_mte()) || (size == 0)) + return addr; + + /* Make sure that size is MTE granule aligned. */ + WARN_ON(size & (MTE_GRANULE_SIZE - 1)); + + /* Make sure that the address is MTE granule aligned. */ + WARN_ON((u64)addr & (MTE_GRANULE_SIZE - 1)); + + tag = 0xF0 | tag; + ptr = (void *)__tag_set(ptr, tag); + + mte_assign_mem_tag_range(ptr, size); + + return ptr; +} + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ --- a/arch/arm64/lib/mte.S~arm64-mte-add-in-kernel-mte-helpers +++ a/arch/arm64/lib/mte.S @@ -149,3 +149,19 @@ SYM_FUNC_START(mte_restore_page_tags) ret SYM_FUNC_END(mte_restore_page_tags) + +/* + * Assign allocation tags for a region of memory based on the pointer tag + * x0 - source pointer + * x1 - size + * + * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and + * size must be non-zero and MTE_GRANULE_SIZE aligned. + */ +SYM_FUNC_START(mte_assign_mem_tag_range) +1: stg x0, [x0] + add x0, x0, #MTE_GRANULE_SIZE + subs x1, x1, #MTE_GRANULE_SIZE + b.gt 1b + ret +SYM_FUNC_END(mte_assign_mem_tag_range) From patchwork Tue Dec 22 20:01:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8BA3C433E6 for ; Tue, 22 Dec 2020 20:01:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4CFFE20705 for ; Tue, 22 Dec 2020 20:01:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4CFFE20705 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 906888D0010; Tue, 22 Dec 2020 15:01:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8945A8D0003; Tue, 22 Dec 2020 15:01:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BB788D0010; Tue, 22 Dec 2020 15:01:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 537388D0003 for ; Tue, 22 Dec 2020 15:01:34 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1616A181AEF3F for ; Tue, 22 Dec 2020 20:01:34 +0000 (UTC) X-FDA: 77621988108.24.hen29_4f01b7027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id E42F01A4A7 for ; Tue, 22 Dec 2020 20:01:33 +0000 (UTC) X-HE-Tag: hen29_4f01b7027462 X-Filterd-Recvd-Size: 5784 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C559A20795; Tue, 22 Dec 2020 20:01:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667292; bh=ah5TPYDRLqIHq32JpImdGs7kOi1DMjEaUHb3vestQn4=; h=Date:From:To:Subject:In-Reply-To:From; b=kUZpMS9q7Y2GIchJspsYF1RHE1r2GdsB5bOfL1hLiO/Pv6Bra6/R2qHskwcM+LF3z OCPv65XlkHQVQByZjPlJaXB2kVEeXVplxdWDm5BliTRq/d4UD0oVLbd+EVM98WB014 YQCWM3ZNsdmxW+hInnbaTyiZJMWf+9YaCuPn+G4k= Date: Tue, 22 Dec 2020 12:01:31 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 25/60] arm64: mte: reset the page tag in page->flags Message-ID: <20201222200131.HtyUt5vvG%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: mte: reset the page tag in page->flags The hardware tag-based KASAN for compatibility with the other modes stores the tag associated to a page in page->flags. Due to this the kernel faults on access when it allocates a page with an initial tag and the user changes the tags. Reset the tag associated by the kernel to a page in all the meaningful places to prevent kernel faults on access. Note: An alternative to this approach could be to modify page_to_virt(). This though could end up being racy, in fact if a CPU checks the PG_mte_tagged bit and decides that the page is not tagged but another CPU maps the same with PROT_MTE and becomes tagged the subsequent kernel access would fail. Link: https://lkml.kernel.org/r/9073d4e973747a6f78d5bdd7ebe17f290d087096.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/kernel/hibernate.c | 5 +++++ arch/arm64/kernel/mte.c | 9 +++++++++ arch/arm64/mm/copypage.c | 9 +++++++++ arch/arm64/mm/mteswap.c | 9 +++++++++ 4 files changed, 32 insertions(+) --- a/arch/arm64/kernel/hibernate.c~arm64-mte-reset-the-page-tag-in-page-flags +++ a/arch/arm64/kernel/hibernate.c @@ -371,6 +371,11 @@ static void swsusp_mte_restore_tags(void unsigned long pfn = xa_state.xa_index; struct page *page = pfn_to_online_page(pfn); + /* + * It is not required to invoke page_kasan_tag_reset(page) + * at this point since the tags stored in page->flags are + * already restored. + */ mte_restore_page_tags(page_address(page), tags); mte_free_tag_storage(tags); --- a/arch/arm64/kernel/mte.c~arm64-mte-reset-the-page-tag-in-page-flags +++ a/arch/arm64/kernel/mte.c @@ -34,6 +34,15 @@ static void mte_sync_page_tags(struct pa return; } + page_kasan_tag_reset(page); + /* + * We need smp_wmb() in between setting the flags and clearing the + * tags because if another thread reads page->flags and builds a + * tagged address out of it, there is an actual dependency to the + * memory access, but on the current thread we do not guarantee that + * the new page->flags are visible before the tags were updated. + */ + smp_wmb(); mte_clear_page_tags(page_address(page)); } --- a/arch/arm64/mm/copypage.c~arm64-mte-reset-the-page-tag-in-page-flags +++ a/arch/arm64/mm/copypage.c @@ -23,6 +23,15 @@ void copy_highpage(struct page *to, stru if (system_supports_mte() && test_bit(PG_mte_tagged, &from->flags)) { set_bit(PG_mte_tagged, &to->flags); + page_kasan_tag_reset(to); + /* + * We need smp_wmb() in between setting the flags and clearing the + * tags because if another thread reads page->flags and builds a + * tagged address out of it, there is an actual dependency to the + * memory access, but on the current thread we do not guarantee that + * the new page->flags are visible before the tags were updated. + */ + smp_wmb(); mte_copy_page_tags(kto, kfrom); } } --- a/arch/arm64/mm/mteswap.c~arm64-mte-reset-the-page-tag-in-page-flags +++ a/arch/arm64/mm/mteswap.c @@ -53,6 +53,15 @@ bool mte_restore_tags(swp_entry_t entry, if (!tags) return false; + page_kasan_tag_reset(page); + /* + * We need smp_wmb() in between setting the flags and clearing the + * tags because if another thread reads page->flags and builds a + * tagged address out of it, there is an actual dependency to the + * memory access, but on the current thread we do not guarantee that + * the new page->flags are visible before the tags were updated. + */ + smp_wmb(); mte_restore_page_tags(page_address(page), tags); return true; From patchwork Tue Dec 22 20:01:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB464C433E9 for ; Tue, 22 Dec 2020 20:01:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7AAFB20771 for ; Tue, 22 Dec 2020 20:01:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AAFB20771 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9F1AB8D0013; Tue, 22 Dec 2020 15:01:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 97FD48D0012; Tue, 22 Dec 2020 15:01:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 846698D0013; Tue, 22 Dec 2020 15:01:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0148.hostedemail.com [216.40.44.148]) by kanga.kvack.org (Postfix) with ESMTP id 68E828D0012 for ; Tue, 22 Dec 2020 15:01:41 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2CC1A8249980 for ; Tue, 22 Dec 2020 20:01:41 +0000 (UTC) X-FDA: 77621988402.25.plate04_3e06af027462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id BAEBE1804E3A9 for ; Tue, 22 Dec 2020 20:01:37 +0000 (UTC) X-HE-Tag: plate04_3e06af027462 X-Filterd-Recvd-Size: 6711 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 612A1221FF; Tue, 22 Dec 2020 20:01:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667296; bh=qpMIML1/gII246ISYK6mXLCd8OVwFtW20z8VkUZtJjY=; h=Date:From:To:Subject:In-Reply-To:From; b=puLsTPjeZ0HNJ30FXoUw6b77hWljrgx7iVHvFMUjMNEp9XLKasPWgFNZxklZgXjT+ PzmeJvQuIqRoqo+2E6ZTtIBQFVFgOr9Fsp8YdotB8QlPNGi5C7aJKLOzUXL1UT6oWL SfBPKdFsxB66J72DOp0FpiNDCFQnZvP23J5YQYAU= Date: Tue, 22 Dec 2020 12:01:35 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 26/60] arm64: mte: add in-kernel tag fault handler Message-ID: <20201222200135.bp8Ma_xt8%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: mte: add in-kernel tag fault handler Add the implementation of the in-kernel fault handler. When a tag fault happens on a kernel address: * MTE is disabled on the current CPU, * the execution continues. When a tag fault happens on a user address: * the kernel executes do_bad_area() and panics. The tag fault handler for kernel addresses is currently empty and will be filled in by a future commit. Link: https://lkml.kernel.org/r/ad31529b073e22840b7a2246172c2b67747ed7c4.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov Signed-off-by: Catalin Marinas Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon [catalin.marinas@arm.com: ensure CONFIG_ARM64_PAN is enabled with MTE] Link: https://lkml.kernel.org/r/20201203102628.GB2224@gaia Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 2 + arch/arm64/include/asm/uaccess.h | 23 ++++++++++++++ arch/arm64/mm/fault.c | 45 +++++++++++++++++++++++++++++ 3 files changed, 70 insertions(+) --- a/arch/arm64/include/asm/uaccess.h~arm64-mte-add-in-kernel-tag-fault-handler +++ a/arch/arm64/include/asm/uaccess.h @@ -159,8 +159,28 @@ static inline void __uaccess_enable_hw_p CONFIG_ARM64_PAN)); } +/* + * The Tag Check Flag (TCF) mode for MTE is per EL, hence TCF0 + * affects EL0 and TCF affects EL1 irrespective of which TTBR is + * used. + * The kernel accesses TTBR0 usually with LDTR/STTR instructions + * when UAO is available, so these would act as EL0 accesses using + * TCF0. + * However futex.h code uses exclusives which would be executed as + * EL1, this can potentially cause a tag check fault even if the + * user disables TCF0. + * + * To address the problem we set the PSTATE.TCO bit in uaccess_enable() + * and reset it in uaccess_disable(). + * + * The Tag check override (TCO) bit disables temporarily the tag checking + * preventing the issue. + */ static inline void uaccess_disable_privileged(void) { + asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(0), + ARM64_MTE, CONFIG_KASAN_HW_TAGS)); + if (uaccess_ttbr0_disable()) return; @@ -169,6 +189,9 @@ static inline void uaccess_disable_privi static inline void uaccess_enable_privileged(void) { + asm volatile(ALTERNATIVE("nop", SET_PSTATE_TCO(1), + ARM64_MTE, CONFIG_KASAN_HW_TAGS)); + if (uaccess_ttbr0_enable()) return; --- a/arch/arm64/Kconfig~arm64-mte-add-in-kernel-tag-fault-handler +++ a/arch/arm64/Kconfig @@ -1649,6 +1649,8 @@ config ARM64_MTE default y depends on ARM64_AS_HAS_MTE && ARM64_TAGGED_ADDR_ABI depends on AS_HAS_ARMV8_5 + # Required for tag checking in the uaccess routines + depends on ARM64_PAN select ARCH_USES_HIGH_VMA_FLAGS help Memory Tagging (part of the ARMv8.5 Extensions) provides --- a/arch/arm64/mm/fault.c~arm64-mte-add-in-kernel-tag-fault-handler +++ a/arch/arm64/mm/fault.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include #include @@ -296,6 +297,44 @@ static void die_kernel_fault(const char do_exit(SIGKILL); } +static void report_tag_fault(unsigned long addr, unsigned int esr, + struct pt_regs *regs) +{ +} + +static void do_tag_recovery(unsigned long addr, unsigned int esr, + struct pt_regs *regs) +{ + static bool reported; + + if (!READ_ONCE(reported)) { + report_tag_fault(addr, esr, regs); + WRITE_ONCE(reported, true); + } + + /* + * Disable MTE Tag Checking on the local CPU for the current EL. + * It will be done lazily on the other CPUs when they will hit a + * tag fault. + */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_NONE); + isb(); +} + +static bool is_el1_mte_sync_tag_check_fault(unsigned int esr) +{ + unsigned int ec = ESR_ELx_EC(esr); + unsigned int fsc = esr & ESR_ELx_FSC; + + if (ec != ESR_ELx_EC_DABT_CUR) + return false; + + if (fsc == ESR_ELx_FSC_MTE) + return true; + + return false; +} + static void __do_kernel_fault(unsigned long addr, unsigned int esr, struct pt_regs *regs) { @@ -312,6 +351,12 @@ static void __do_kernel_fault(unsigned l "Ignoring spurious kernel translation fault at virtual address %016lx\n", addr)) return; + if (is_el1_mte_sync_tag_check_fault(esr)) { + do_tag_recovery(addr, esr, regs); + + return; + } + if (is_el1_permission_fault(addr, esr, regs)) { if (esr & ESR_ELx_WNR) msg = "write to read-only memory"; From patchwork Tue Dec 22 20:01:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59B47C433DB for ; Tue, 22 Dec 2020 20:01:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E507C20795 for ; Tue, 22 Dec 2020 20:01:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E507C20795 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC9978D0015; Tue, 22 Dec 2020 15:01:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B28508D0012; Tue, 22 Dec 2020 15:01:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F11F8D0015; Tue, 22 Dec 2020 15:01:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0108.hostedemail.com [216.40.44.108]) by kanga.kvack.org (Postfix) with ESMTP id 82E838D0012 for ; Tue, 22 Dec 2020 15:01:42 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 519198249980 for ; Tue, 22 Dec 2020 20:01:42 +0000 (UTC) X-FDA: 77621988444.29.beast99_0a0b97527462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 6E650180868E0 for ; Tue, 22 Dec 2020 20:01:41 +0000 (UTC) X-HE-Tag: beast99_0a0b97527462 X-Filterd-Recvd-Size: 5962 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0B13222ADC; Tue, 22 Dec 2020 20:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667299; bh=gh4y1Xk5AtcuXlMWZAg1eDaRrWbiCuy7OvwHp+REC6g=; h=Date:From:To:Subject:In-Reply-To:From; b=LlaZPQlMFAyuMjT7xYNTebh2VCknVxcPYy2WhqC76+VsNydGVqEnZ19XSY9z4icXA j8a8FJICKCOccNG0JT4dkYj/Q64xoNTCJVrxpXwjyeB1h2wOFb09GTccjtPXx9deXc b3WtakCyOsA+51v3ThvB10KlhE6UmhfwDTYEMWnw= Date: Tue, 22 Dec 2020 12:01:38 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 27/60] arm64: kasan: allow enabling in-kernel MTE Message-ID: <20201222200138.aIQST2piJ%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: kasan: allow enabling in-kernel MTE Hardware tag-based KASAN relies on Memory Tagging Extension (MTE) feature and requires it to be enabled. MTE supports This patch adds a new mte_enable_kernel() helper, that enables MTE in Synchronous mode in EL1 and is intended to be called from KASAN runtime during initialization. The Tag Checking operation causes a synchronous data abort as a consequence of a tag check fault when MTE is configured in synchronous mode. As part of this change enable match-all tag for EL1 to allow the kernel to access user pages without faulting. This is required because the kernel does not have knowledge of the tags set by the user in a page. Note: For MTE, the TCF bit field in SCTLR_EL1 affects only EL1 in a similar way as TCF0 affects EL0. MTE that is built on top of the Top Byte Ignore (TBI) feature hence we enable it as part of this patch as well. Link: https://lkml.kernel.org/r/7352b0a0899af65c2785416c8ca6bf3845b66fa1.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/mte-kasan.h | 6 ++++++ arch/arm64/kernel/mte.c | 7 +++++++ arch/arm64/mm/proc.S | 23 ++++++++++++++++++++--- 3 files changed, 33 insertions(+), 3 deletions(-) --- a/arch/arm64/include/asm/mte-kasan.h~arm64-kasan-allow-enabling-in-kernel-mte +++ a/arch/arm64/include/asm/mte-kasan.h @@ -29,6 +29,8 @@ u8 mte_get_mem_tag(void *addr); u8 mte_get_random_tag(void); void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag); +void mte_enable_kernel(void); + #else /* CONFIG_ARM64_MTE */ static inline u8 mte_get_ptr_tag(void *ptr) @@ -49,6 +51,10 @@ static inline void *mte_set_mem_tag_rang return addr; } +static inline void mte_enable_kernel(void) +{ +} + #endif /* CONFIG_ARM64_MTE */ #endif /* __ASSEMBLY__ */ --- a/arch/arm64/kernel/mte.c~arm64-kasan-allow-enabling-in-kernel-mte +++ a/arch/arm64/kernel/mte.c @@ -129,6 +129,13 @@ void *mte_set_mem_tag_range(void *addr, return ptr; } +void mte_enable_kernel(void) +{ + /* Enable MTE Sync Mode for EL1. */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + isb(); +} + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ --- a/arch/arm64/mm/proc.S~arm64-kasan-allow-enabling-in-kernel-mte +++ a/arch/arm64/mm/proc.S @@ -40,9 +40,15 @@ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA #ifdef CONFIG_KASAN_SW_TAGS -#define TCR_KASAN_FLAGS TCR_TBI1 | TCR_TBID1 +#define TCR_KASAN_SW_FLAGS TCR_TBI1 | TCR_TBID1 #else -#define TCR_KASAN_FLAGS 0 +#define TCR_KASAN_SW_FLAGS 0 +#endif + +#ifdef CONFIG_KASAN_HW_TAGS +#define TCR_KASAN_HW_FLAGS SYS_TCR_EL1_TCMA1 | TCR_TBI1 +#else +#define TCR_KASAN_HW_FLAGS 0 #endif /* @@ -427,6 +433,10 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x5, MAIR_EL1_SET #ifdef CONFIG_ARM64_MTE + mte_tcr .req x20 + + mov mte_tcr, #0 + /* * Update MAIR_EL1, GCR_EL1 and TFSR*_EL1 if MTE is supported * (ID_AA64PFR1_EL1[11:8] > 1). @@ -447,6 +457,9 @@ SYM_FUNC_START(__cpu_setup) /* clear any pending tag check faults in TFSR*_EL1 */ msr_s SYS_TFSR_EL1, xzr msr_s SYS_TFSRE0_EL1, xzr + + /* set the TCR_EL1 bits */ + mov_q mte_tcr, TCR_KASAN_HW_FLAGS 1: #endif msr mair_el1, x5 @@ -456,7 +469,11 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS + TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS +#ifdef CONFIG_ARM64_MTE + orr x10, x10, mte_tcr + .unreq mte_tcr +#endif tcr_clear_errata_bits x10, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52 From patchwork Tue Dec 22 20:01:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 005F9C433DB for ; Tue, 22 Dec 2020 20:02:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9772E21D1B for ; Tue, 22 Dec 2020 20:02:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9772E21D1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B28AD8D0016; Tue, 22 Dec 2020 15:01:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB7268D0017; Tue, 22 Dec 2020 15:01:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 953D38D0016; Tue, 22 Dec 2020 15:01:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id 7CF808D0012 for ; Tue, 22 Dec 2020 15:01:45 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3B5181F06 for ; Tue, 22 Dec 2020 20:01:45 +0000 (UTC) X-FDA: 77621988570.05.kite71_340a5aa27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 107FE1801ED61 for ; Tue, 22 Dec 2020 20:01:45 +0000 (UTC) X-HE-Tag: kite71_340a5aa27462 X-Filterd-Recvd-Size: 6259 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D206022583; Tue, 22 Dec 2020 20:01:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667303; bh=DDDH2RVDluwp8itwATWBD8/EvB5Hgdjo8/Xdk/NWnkg=; h=Date:From:To:Subject:In-Reply-To:From; b=qEXg0vgyKFYriHUgfT5UgYkNSdwl39se4/S6jmPzp6jkcICATClKVG/SFQFiVC812 ayijQGBNP/0j7DKP6kzeykNBi6XCZmlrfBtvqASwPZPLv+zwSlO7pSYyhz7bMwcE/a PYdH5stzcvDhpY2ClpNWDqbb3p500oqiWV8t9p+M= Date: Tue, 22 Dec 2020 12:01:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 28/60] arm64: mte: convert gcr_user into an exclude mask Message-ID: <20201222200142.y7aeQtVE1%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: mte: convert gcr_user into an exclude mask The gcr_user mask is a per thread mask that represents the tags that are excluded from random generation when the Memory Tagging Extension is present and an 'irg' instruction is invoked. gcr_user affects the behavior on EL0 only. Currently that mask is an include mask and it is controlled by the user via prctl() while GCR_EL1 accepts an exclude mask. Convert the include mask into an exclude one to make it easier the register setting. Note: This change will affect gcr_kernel (for EL1) introduced with a future patch. Link: https://lkml.kernel.org/r/946dd31be833b660334c4f93410acf6d6c4cf3c4.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/processor.h | 2 - arch/arm64/kernel/mte.c | 29 +++++++++++++-------------- 2 files changed, 16 insertions(+), 15 deletions(-) --- a/arch/arm64/include/asm/processor.h~arm64-mte-convert-gcr_user-into-an-exclude-mask +++ a/arch/arm64/include/asm/processor.h @@ -152,7 +152,7 @@ struct thread_struct { #endif #ifdef CONFIG_ARM64_MTE u64 sctlr_tcf0; - u64 gcr_user_incl; + u64 gcr_user_excl; #endif }; --- a/arch/arm64/kernel/mte.c~arm64-mte-convert-gcr_user-into-an-exclude-mask +++ a/arch/arm64/kernel/mte.c @@ -156,23 +156,22 @@ static void set_sctlr_el1_tcf0(u64 tcf0) preempt_enable(); } -static void update_gcr_el1_excl(u64 incl) +static void update_gcr_el1_excl(u64 excl) { - u64 excl = ~incl & SYS_GCR_EL1_EXCL_MASK; /* - * Note that 'incl' is an include mask (controlled by the user via - * prctl()) while GCR_EL1 accepts an exclude mask. + * Note that the mask controlled by the user via prctl() is an + * include while GCR_EL1 accepts an exclude mask. * No need for ISB since this only affects EL0 currently, implicit * with ERET. */ sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl); } -static void set_gcr_el1_excl(u64 incl) +static void set_gcr_el1_excl(u64 excl) { - current->thread.gcr_user_incl = incl; - update_gcr_el1_excl(incl); + current->thread.gcr_user_excl = excl; + update_gcr_el1_excl(excl); } void flush_mte_state(void) @@ -187,7 +186,7 @@ void flush_mte_state(void) /* disable tag checking */ set_sctlr_el1_tcf0(SCTLR_EL1_TCF0_NONE); /* reset tag generation mask */ - set_gcr_el1_excl(0); + set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK); } void mte_thread_switch(struct task_struct *next) @@ -198,7 +197,7 @@ void mte_thread_switch(struct task_struc /* avoid expensive SCTLR_EL1 accesses if no change */ if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); - update_gcr_el1_excl(next->thread.gcr_user_incl); + update_gcr_el1_excl(next->thread.gcr_user_excl); } void mte_suspend_exit(void) @@ -206,13 +205,14 @@ void mte_suspend_exit(void) if (!system_supports_mte()) return; - update_gcr_el1_excl(current->thread.gcr_user_incl); + update_gcr_el1_excl(current->thread.gcr_user_excl); } long set_mte_ctrl(struct task_struct *task, unsigned long arg) { u64 tcf0; - u64 gcr_incl = (arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT; + u64 gcr_excl = ~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & + SYS_GCR_EL1_EXCL_MASK; if (!system_supports_mte()) return 0; @@ -233,10 +233,10 @@ long set_mte_ctrl(struct task_struct *ta if (task != current) { task->thread.sctlr_tcf0 = tcf0; - task->thread.gcr_user_incl = gcr_incl; + task->thread.gcr_user_excl = gcr_excl; } else { set_sctlr_el1_tcf0(tcf0); - set_gcr_el1_excl(gcr_incl); + set_gcr_el1_excl(gcr_excl); } return 0; @@ -245,11 +245,12 @@ long set_mte_ctrl(struct task_struct *ta long get_mte_ctrl(struct task_struct *task) { unsigned long ret; + u64 incl = ~task->thread.gcr_user_excl & SYS_GCR_EL1_EXCL_MASK; if (!system_supports_mte()) return 0; - ret = task->thread.gcr_user_incl << PR_MTE_TAG_SHIFT; + ret = incl << PR_MTE_TAG_SHIFT; switch (task->thread.sctlr_tcf0) { case SCTLR_EL1_TCF0_NONE: From patchwork Tue Dec 22 20:01:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E764C433E0 for ; Tue, 22 Dec 2020 20:02:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 12E9121D1B for ; Tue, 22 Dec 2020 20:02:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12E9121D1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1DBB78D0018; Tue, 22 Dec 2020 15:01:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 163468D0017; Tue, 22 Dec 2020 15:01:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F218F8D0018; Tue, 22 Dec 2020 15:01:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id D2C318D0017 for ; Tue, 22 Dec 2020 15:01:48 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8ED748249980 for ; Tue, 22 Dec 2020 20:01:48 +0000 (UTC) X-FDA: 77621988696.14.chess44_010b33427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 6D08E180070C9 for ; Tue, 22 Dec 2020 20:01:48 +0000 (UTC) X-HE-Tag: chess44_010b33427462 X-Filterd-Recvd-Size: 8584 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:47 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5653523137; Tue, 22 Dec 2020 20:01:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667307; bh=K55WA5i+MZqHFYpmOdSCc9fuL4mhAA32VNirWDBvmWY=; h=Date:From:To:Subject:In-Reply-To:From; b=inbFPmBrXywI470sMYhux6YtyoIFLTbwY0C48OfawI9JDPrmOgs3EqKpcNTinAUlw UndWp2hNEsx2TIdhZfCpHblgG/9Ue4L7XTzmkfQ9K9h3ROzOo0i2SJqB/hWtgwdQLy +UnUoosxBikW32ljx7rSpOP0BjaoeZ+a4i8Qtg1I= Date: Tue, 22 Dec 2020 12:01:45 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 29/60] arm64: mte: switch GCR_EL1 in kernel entry and exit Message-ID: <20201222200145.zvzM87pBj%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: arm64: mte: switch GCR_EL1 in kernel entry and exit When MTE is present, the GCR_EL1 register contains the tags mask that allows to exclude tags from the random generation via the IRG instruction. With the introduction of the new Tag-Based KASAN API that provides a mechanism to reserve tags for special reasons, the MTE implementation has to make sure that the GCR_EL1 setting for the kernel does not affect the userspace processes and viceversa. Save and restore the kernel/user mask in GCR_EL1 in kernel entry and exit. Link: https://lkml.kernel.org/r/578b03294708cc7258fad0dc9c2a2e809e5a8214.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Co-developed-by: Andrey Konovalov Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/mte-def.h | 1 arch/arm64/include/asm/mte-kasan.h | 5 +++ arch/arm64/include/asm/mte.h | 2 + arch/arm64/kernel/asm-offsets.c | 3 + arch/arm64/kernel/entry.S | 41 +++++++++++++++++++++++++++ arch/arm64/kernel/mte.c | 31 ++++++++++++++++++-- 6 files changed, 79 insertions(+), 4 deletions(-) --- a/arch/arm64/include/asm/mte-def.h~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit +++ a/arch/arm64/include/asm/mte-def.h @@ -10,6 +10,5 @@ #define MTE_TAG_SHIFT 56 #define MTE_TAG_SIZE 4 #define MTE_TAG_MASK GENMASK((MTE_TAG_SHIFT + (MTE_TAG_SIZE - 1)), MTE_TAG_SHIFT) -#define MTE_TAG_MAX (MTE_TAG_MASK >> MTE_TAG_SHIFT) #endif /* __ASM_MTE_DEF_H */ --- a/arch/arm64/include/asm/mte.h~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit +++ a/arch/arm64/include/asm/mte.h @@ -18,6 +18,8 @@ #include +extern u64 gcr_kernel_excl; + void mte_clear_page_tags(void *addr); unsigned long mte_copy_tags_from_user(void *to, const void __user *from, unsigned long n); --- a/arch/arm64/include/asm/mte-kasan.h~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit +++ a/arch/arm64/include/asm/mte-kasan.h @@ -30,6 +30,7 @@ u8 mte_get_random_tag(void); void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag); void mte_enable_kernel(void); +void mte_init_tags(u64 max_tag); #else /* CONFIG_ARM64_MTE */ @@ -55,6 +56,10 @@ static inline void mte_enable_kernel(voi { } +static inline void mte_init_tags(u64 max_tag) +{ +} + #endif /* CONFIG_ARM64_MTE */ #endif /* __ASSEMBLY__ */ --- a/arch/arm64/kernel/asm-offsets.c~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit +++ a/arch/arm64/kernel/asm-offsets.c @@ -47,6 +47,9 @@ int main(void) DEFINE(THREAD_KEYS_USER, offsetof(struct task_struct, thread.keys_user)); DEFINE(THREAD_KEYS_KERNEL, offsetof(struct task_struct, thread.keys_kernel)); #endif +#ifdef CONFIG_ARM64_MTE + DEFINE(THREAD_GCR_EL1_USER, offsetof(struct task_struct, thread.gcr_user_excl)); +#endif BLANK(); DEFINE(S_X0, offsetof(struct pt_regs, regs[0])); DEFINE(S_X2, offsetof(struct pt_regs, regs[2])); --- a/arch/arm64/kernel/entry.S~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit +++ a/arch/arm64/kernel/entry.S @@ -173,6 +173,43 @@ alternative_else_nop_endif #endif .endm + .macro mte_set_gcr, tmp, tmp2 +#ifdef CONFIG_ARM64_MTE + /* + * Calculate and set the exclude mask preserving + * the RRND (bit[16]) setting. + */ + mrs_s \tmp2, SYS_GCR_EL1 + bfi \tmp2, \tmp, #0, #16 + msr_s SYS_GCR_EL1, \tmp2 + isb +#endif + .endm + + .macro mte_set_kernel_gcr, tmp, tmp2 +#ifdef CONFIG_KASAN_HW_TAGS +alternative_if_not ARM64_MTE + b 1f +alternative_else_nop_endif + ldr_l \tmp, gcr_kernel_excl + + mte_set_gcr \tmp, \tmp2 +1: +#endif + .endm + + .macro mte_set_user_gcr, tsk, tmp, tmp2 +#ifdef CONFIG_ARM64_MTE +alternative_if_not ARM64_MTE + b 1f +alternative_else_nop_endif + ldr \tmp, [\tsk, #THREAD_GCR_EL1_USER] + + mte_set_gcr \tmp, \tmp2 +1: +#endif + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -212,6 +249,8 @@ alternative_else_nop_endif ptrauth_keys_install_kernel tsk, x20, x22, x23 + mte_set_kernel_gcr x22, x23 + scs_load tsk, x20 .else add x21, sp, #S_FRAME_SIZE @@ -315,6 +354,8 @@ alternative_else_nop_endif /* No kernel C function calls after this as user keys are set. */ ptrauth_keys_install_user tsk, x0, x1, x2 + mte_set_user_gcr tsk, x0, x1 + apply_ssbd 0, x0, x1 .endif --- a/arch/arm64/kernel/mte.c~arm64-mte-switch-gcr_el1-in-kernel-entry-and-exit +++ a/arch/arm64/kernel/mte.c @@ -23,6 +23,8 @@ #include #include +u64 gcr_kernel_excl __ro_after_init; + static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_swap) { pte_t old_pte = READ_ONCE(*ptep); @@ -129,6 +131,26 @@ void *mte_set_mem_tag_range(void *addr, return ptr; } +void mte_init_tags(u64 max_tag) +{ + static bool gcr_kernel_excl_initialized; + + if (!gcr_kernel_excl_initialized) { + /* + * The format of the tags in KASAN is 0xFF and in MTE is 0xF. + * This conversion extracts an MTE tag from a KASAN tag. + */ + u64 incl = GENMASK(FIELD_GET(MTE_TAG_MASK >> MTE_TAG_SHIFT, + max_tag), 0); + + gcr_kernel_excl = ~incl & SYS_GCR_EL1_EXCL_MASK; + gcr_kernel_excl_initialized = true; + } + + /* Enable the kernel exclude mask for random tags generation. */ + write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1); +} + void mte_enable_kernel(void) { /* Enable MTE Sync Mode for EL1. */ @@ -171,7 +193,11 @@ static void update_gcr_el1_excl(u64 excl static void set_gcr_el1_excl(u64 excl) { current->thread.gcr_user_excl = excl; - update_gcr_el1_excl(excl); + + /* + * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value + * by mte_set_user_gcr() in kernel_exit, + */ } void flush_mte_state(void) @@ -197,7 +223,6 @@ void mte_thread_switch(struct task_struc /* avoid expensive SCTLR_EL1 accesses if no change */ if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); - update_gcr_el1_excl(next->thread.gcr_user_excl); } void mte_suspend_exit(void) @@ -205,7 +230,7 @@ void mte_suspend_exit(void) if (!system_supports_mte()) return; - update_gcr_el1_excl(current->thread.gcr_user_excl); + update_gcr_el1_excl(gcr_kernel_excl); } long set_mte_ctrl(struct task_struct *task, unsigned long arg) From patchwork Tue Dec 22 20:01:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69D83C433DB for ; Tue, 22 Dec 2020 20:02:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EAF7D21D81 for ; Tue, 22 Dec 2020 20:02:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EAF7D21D81 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 64ADF8D001A; Tue, 22 Dec 2020 15:01:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D15A8D0019; Tue, 22 Dec 2020 15:01:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4707A8D001A; Tue, 22 Dec 2020 15:01:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id 2E55C8D0019 for ; Tue, 22 Dec 2020 15:01:52 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E8DE6824999B for ; Tue, 22 Dec 2020 20:01:51 +0000 (UTC) X-FDA: 77621988822.15.rate71_6314a8527462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id BC3851814B0C7 for ; Tue, 22 Dec 2020 20:01:51 +0000 (UTC) X-HE-Tag: rate71_6314a8527462 X-Filterd-Recvd-Size: 3203 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DFB9120700; Tue, 22 Dec 2020 20:01:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667310; bh=rXPig3Y2BiXNEZi+1w0PI3jdh8Ii07tQ40TtSpXIZlQ=; h=Date:From:To:Subject:In-Reply-To:From; b=n4PJIavFFOdKcM4PaeR1OhWVk1SCIJ6K0nz4FK8seo+0ebIso9PIlRgXUZgOIYuf5 W32lVn18Ow9LNYiSORE5M0kGa9F/IXHcP7H8bFFCB9LOVKuE7YSXezWqzu0QQudGhe mQMpipd5L0UdZeBK3rlHAfq45CpAPuFfQozwDpM8= Date: Tue, 22 Dec 2020 12:01:49 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 30/60] kasan, mm: untag page address in free_reserved_area Message-ID: <20201222200149.zhH1IPBra%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: kasan, mm: untag page address in free_reserved_area free_reserved_area() memsets the pages belonging to a given memory area. As that memory hasn't been allocated via page_alloc, the KASAN tags that those pages have are 0x00. As the result the memset might result in a tag mismatch. Untag the address to avoid spurious faults. Link: https://lkml.kernel.org/r/ebef6425f4468d063e2f09c1b62ccbb2236b71d3.1606161801.git.andreyknvl@google.com Cc: Andrew Morton Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/page_alloc.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/mm/page_alloc.c~kasan-mm-untag-page-address-in-free_reserved_area +++ a/mm/page_alloc.c @@ -7671,6 +7671,11 @@ unsigned long free_reserved_area(void *s * alias for the memset(). */ direct_map_addr = page_address(page); + /* + * Perform a kasan-unchecked memset() since this memory + * has not been initialized. + */ + direct_map_addr = kasan_reset_tag(direct_map_addr); if ((unsigned int)poison <= 0xFF) memset(direct_map_addr, poison, PAGE_SIZE); From patchwork Tue Dec 22 20:01:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EFF7C433DB for ; Tue, 22 Dec 2020 20:02:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC66B21D81 for ; Tue, 22 Dec 2020 20:02:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC66B21D81 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B2A8C8D001E; Tue, 22 Dec 2020 15:01:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB1988D001D; Tue, 22 Dec 2020 15:01:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97AD28D001E; Tue, 22 Dec 2020 15:01:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id 7EDB08D001D for ; Tue, 22 Dec 2020 15:01:55 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 419CD181AEF32 for ; Tue, 22 Dec 2020 20:01:55 +0000 (UTC) X-FDA: 77621988990.22.cord58_2e1678927462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 169B618038E60 for ; Tue, 22 Dec 2020 20:01:55 +0000 (UTC) X-HE-Tag: cord58_2e1678927462 X-Filterd-Recvd-Size: 3128 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4DDF820705; Tue, 22 Dec 2020 20:01:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667313; bh=2lgIsQoKW+rr7/1buXqUkMofgxltHlPWM9zrtbujye4=; h=Date:From:To:Subject:In-Reply-To:From; b=1hIgiVOCEgyW8expTmaRV2hhCjJAJR9+fruk1GhhsKJPguhFC75o7WGeed1UZ6epg LERK6I/0vYexnnbB/tzaGOCJHyWyZoJ457cY0Owo2S6P+6hZVTXlWhXFtmnvUijKEM 0TF0Odvmi1xwZBGnocSe91I3WUe7qagvPH0akZkY= Date: Tue, 22 Dec 2020 12:01:52 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 31/60] arm64: kasan: align allocations for HW_TAGS Message-ID: <20201222200152.6iQkzqXqp%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: arm64: kasan: align allocations for HW_TAGS Hardware tag-based KASAN uses the memory tagging approach, which requires all allocations to be aligned to the memory granule size. Align the allocations to MTE_GRANULE_SIZE via ARCH_SLAB_MINALIGN when CONFIG_KASAN_HW_TAGS is enabled. Link: https://lkml.kernel.org/r/fe64131606b1c2aabfd34ae99554c0d9df18eb19.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/cache.h | 3 +++ 1 file changed, 3 insertions(+) --- a/arch/arm64/include/asm/cache.h~arm64-kasan-align-allocations-for-hw_tags +++ a/arch/arm64/include/asm/cache.h @@ -6,6 +6,7 @@ #define __ASM_CACHE_H #include +#include #define CTR_L1IP_SHIFT 14 #define CTR_L1IP_MASK 3 @@ -51,6 +52,8 @@ #ifdef CONFIG_KASAN_SW_TAGS #define ARCH_SLAB_MINALIGN (1ULL << KASAN_SHADOW_SCALE_SHIFT) +#elif defined(CONFIG_KASAN_HW_TAGS) +#define ARCH_SLAB_MINALIGN MTE_GRANULE_SIZE #endif #ifndef __ASSEMBLY__ From patchwork Tue Dec 22 20:01:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E823FC433DB for ; Tue, 22 Dec 2020 20:02:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 91AB921D81 for ; Tue, 22 Dec 2020 20:02:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91AB921D81 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 53A118D0020; Tue, 22 Dec 2020 15:01:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AFDF8D001F; Tue, 22 Dec 2020 15:01:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B2FC8D0020; Tue, 22 Dec 2020 15:01:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0024.hostedemail.com [216.40.44.24]) by kanga.kvack.org (Postfix) with ESMTP id 025FC8D001F for ; Tue, 22 Dec 2020 15:01:58 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BAE901EE6 for ; Tue, 22 Dec 2020 20:01:58 +0000 (UTC) X-FDA: 77621989116.23.ray10_2d058f227462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 97A4837604 for ; Tue, 22 Dec 2020 20:01:58 +0000 (UTC) X-HE-Tag: ray10_2d058f227462 X-Filterd-Recvd-Size: 4968 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:01:57 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AED0C20771; Tue, 22 Dec 2020 20:01:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667317; bh=8O226vQPibDszPVjnip5QVA2WJlOWY1esr9ASN8tjZQ=; h=Date:From:To:Subject:In-Reply-To:From; b=uzivVGqI6Q5nrAXYJhQJ8dfNihUIKSFg3uQ2QtHcZ9gKSkn70bGbsfs/Kgi7nlSDV pExo0dR9ZC1B5cY8wPSJNegOElvcJ9d8zMq2FJlqAOK5dmZe64CNDSRFi4NiyqELQA 7rcbktUlUKgQhVW8O9gdhkmRzzMpVUmV7D90GP8s= Date: Tue, 22 Dec 2020 12:01:56 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 32/60] arm64: kasan: add arch layer for memory tagging helpers Message-ID: <20201222200156.bCJvLZ_dr%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: arm64: kasan: add arch layer for memory tagging helpers This patch add a set of arch_*() memory tagging helpers currently only defined for arm64 when hardware tag-based KASAN is enabled. These helpers will be used by KASAN runtime to implement the hardware tag-based mode. The arch-level indirection level is introduced to simplify adding hardware tag-based KASAN support for other architectures in the future by defining the appropriate arch_*() macros. Link: https://lkml.kernel.org/r/fc9e5bb71201c03131a2fc00a74125723568dda9.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/memory.h | 9 +++++++++ mm/kasan/kasan.h | 26 ++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) --- a/arch/arm64/include/asm/memory.h~arm64-kasan-add-arch-layer-for-memory-tagging-helpers +++ a/arch/arm64/include/asm/memory.h @@ -230,6 +230,15 @@ static inline const void *__tag_set(cons return (const void *)(__addr | __tag_shifted(tag)); } +#ifdef CONFIG_KASAN_HW_TAGS +#define arch_enable_tagging() mte_enable_kernel() +#define arch_init_tags(max_tag) mte_init_tags(max_tag) +#define arch_get_random_tag() mte_get_random_tag() +#define arch_get_mem_tag(addr) mte_get_mem_tag(addr) +#define arch_set_mem_tag_range(addr, size, tag) \ + mte_set_mem_tag_range((addr), (size), (tag)) +#endif /* CONFIG_KASAN_HW_TAGS */ + /* * Physical vs virtual RAM address space conversion. These are * private definitions which should NOT be used outside memory.h --- a/mm/kasan/kasan.h~arm64-kasan-add-arch-layer-for-memory-tagging-helpers +++ a/mm/kasan/kasan.h @@ -243,6 +243,32 @@ static inline const void *arch_kasan_set #define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) #define get_tag(addr) arch_kasan_get_tag(addr) +#ifdef CONFIG_KASAN_HW_TAGS + +#ifndef arch_enable_tagging +#define arch_enable_tagging() +#endif +#ifndef arch_init_tags +#define arch_init_tags(max_tag) +#endif +#ifndef arch_get_random_tag +#define arch_get_random_tag() (0xFF) +#endif +#ifndef arch_get_mem_tag +#define arch_get_mem_tag(addr) (0xFF) +#endif +#ifndef arch_set_mem_tag_range +#define arch_set_mem_tag_range(addr, size, tag) ((void *)(addr)) +#endif + +#define hw_enable_tagging() arch_enable_tagging() +#define hw_init_tags(max_tag) arch_init_tags(max_tag) +#define hw_get_random_tag() arch_get_random_tag() +#define hw_get_mem_tag(addr) arch_get_mem_tag(addr) +#define hw_set_mem_tag_range(addr, size, tag) arch_set_mem_tag_range((addr), (size), (tag)) + +#endif /* CONFIG_KASAN_HW_TAGS */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Tue Dec 22 20:01:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C38B5C433DB for ; Tue, 22 Dec 2020 20:02:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5C10B21D1B for ; Tue, 22 Dec 2020 20:02:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C10B21D1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B04A98D0021; Tue, 22 Dec 2020 15:02:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A8A908D001F; Tue, 22 Dec 2020 15:02:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97B3E8D0021; Tue, 22 Dec 2020 15:02:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 7CD2F8D001F for ; Tue, 22 Dec 2020 15:02:02 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 39AFB180AD83B for ; Tue, 22 Dec 2020 20:02:02 +0000 (UTC) X-FDA: 77621989284.23.hat13_190a5fb27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 0B10B37609 for ; Tue, 22 Dec 2020 20:02:02 +0000 (UTC) X-HE-Tag: hat13_190a5fb27462 X-Filterd-Recvd-Size: 3033 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1A75620795; Tue, 22 Dec 2020 20:02:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667320; bh=QMYVZBUjFli/H6X7hOz80AwmmUTQTOwCo6dvVlQUpWY=; h=Date:From:To:Subject:In-Reply-To:From; b=K0tvxtieWWAju6MzigDHSH+IYA40ZTbSBrDPGkIVqjNFa/H9DKCnWi3HnMOe4nM0i DD64OtLKAbTB0xzcmg8AzPxS5EDos+fthxl/MQSlSORmzjNatabyS/n49LkTWwOAIq nSoCzN+A8ka5f3WksaTFVxo5PYtxBeP/CNp9VgvU= Date: Tue, 22 Dec 2020 12:01:59 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 33/60] kasan: define KASAN_GRANULE_SIZE for HW_TAGS Message-ID: <20201222200159.AjHp86Rpc%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: define KASAN_GRANULE_SIZE for HW_TAGS Hardware tag-based KASAN has granules of MTE_GRANULE_SIZE. Define KASAN_GRANULE_SIZE to MTE_GRANULE_SIZE for CONFIG_KASAN_HW_TAGS. Link: https://lkml.kernel.org/r/3d15794b3d1b27447fd7fdf862c073192ba657bd.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/kasan.h | 6 ++++++ 1 file changed, 6 insertions(+) --- a/mm/kasan/kasan.h~kasan-define-kasan_granule_size-for-hw_tags +++ a/mm/kasan/kasan.h @@ -5,7 +5,13 @@ #include #include +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) +#else +#include +#define KASAN_GRANULE_SIZE MTE_GRANULE_SIZE +#endif + #define KASAN_GRANULE_MASK (KASAN_GRANULE_SIZE - 1) #define KASAN_MEMORY_PER_SHADOW_PAGE (KASAN_GRANULE_SIZE << PAGE_SHIFT) From patchwork Tue Dec 22 20:02:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 821E3C433E6 for ; Tue, 22 Dec 2020 20:02:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D86123133 for ; Tue, 22 Dec 2020 20:02:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D86123133 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E37458D0023; Tue, 22 Dec 2020 15:02:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D4C5C8D0022; Tue, 22 Dec 2020 15:02:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C38988D0023; Tue, 22 Dec 2020 15:02:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0108.hostedemail.com [216.40.44.108]) by kanga.kvack.org (Postfix) with ESMTP id A99348D0022 for ; Tue, 22 Dec 2020 15:02:06 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 75197363D for ; Tue, 22 Dec 2020 20:02:06 +0000 (UTC) X-FDA: 77621989452.06.night77_4003dd227462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 3494310044B19 for ; Tue, 22 Dec 2020 20:02:06 +0000 (UTC) X-HE-Tag: night77_4003dd227462 X-Filterd-Recvd-Size: 3498 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 90D9C20867; Tue, 22 Dec 2020 20:02:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667324; bh=bPiKkZ6Z/+1py3lpgzBMnzQUWTR7th/BFIrQJlJkvTc=; h=Date:From:To:Subject:In-Reply-To:From; b=nSDtfTGjwJJOKnF0g+p8b7Hg6DIHjgJ6ip7Yb1zfC8CmiTxGkSS970aOmDqPwwe+h MiD2gBTauGOeMMLXBNs1cBZHIn3HorP3LSRDlxYdqyNkwXEEhOJJiyHxKtcPMxye+b XUCWi2LzHXfb6rbGg3wNYobEwJ1nFttw4ZiC3xOA= Date: Tue, 22 Dec 2020 12:02:03 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 34/60] kasan, x86, s390: update undef CONFIG_KASAN Message-ID: <20201222200203.jZX-wM-Vv%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, x86, s390: update undef CONFIG_KASAN With the intoduction of hardware tag-based KASAN some kernel checks of this kind: ifdef CONFIG_KASAN will be updated to: if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) x86 and s390 use a trick to #undef CONFIG_KASAN for some of the code that isn't linked with KASAN runtime and shouldn't have any KASAN annotations. Also #undef CONFIG_KASAN_GENERIC with CONFIG_KASAN. Link: https://lkml.kernel.org/r/9d84bfaaf8fabe0fc89f913c9e420a30bd31a260.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Acked-by: Vasily Gorbik Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/s390/boot/string.c | 1 + arch/x86/boot/compressed/misc.h | 1 + 2 files changed, 2 insertions(+) --- a/arch/s390/boot/string.c~kasan-x86-s390-update-undef-config_kasan +++ a/arch/s390/boot/string.c @@ -3,6 +3,7 @@ #include #include #undef CONFIG_KASAN +#undef CONFIG_KASAN_GENERIC #include "../lib/string.c" int strncmp(const char *cs, const char *ct, size_t count) --- a/arch/x86/boot/compressed/misc.h~kasan-x86-s390-update-undef-config_kasan +++ a/arch/x86/boot/compressed/misc.h @@ -12,6 +12,7 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN +#undef CONFIG_KASAN_GENERIC /* cpu_feature_enabled() cannot be used this early */ #define USE_EARLY_PGTABLE_L5 From patchwork Tue Dec 22 20:02:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A1F7C433E0 for ; Tue, 22 Dec 2020 20:02:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2CAC420771 for ; Tue, 22 Dec 2020 20:02:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2CAC420771 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD4ED8D0025; Tue, 22 Dec 2020 15:02:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C5FB28D0024; Tue, 22 Dec 2020 15:02:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B73A48D0025; Tue, 22 Dec 2020 15:02:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 97BA58D0024 for ; Tue, 22 Dec 2020 15:02:09 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5C1EF180AD807 for ; Tue, 22 Dec 2020 20:02:09 +0000 (UTC) X-FDA: 77621989578.29.fall96_5a0056a27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 33C9D180868DC for ; Tue, 22 Dec 2020 20:02:09 +0000 (UTC) X-HE-Tag: fall96_5a0056a27462 X-Filterd-Recvd-Size: 14550 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E856320700; Tue, 22 Dec 2020 20:02:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667327; bh=mphpU/kKZiL4hPerdJpms8BqxL07kb338OrSwnPQhrQ=; h=Date:From:To:Subject:In-Reply-To:From; b=Cquy94+JT5R2YooukclKkDdhvBC/Qumn3lR1ol7r9i1CBUM0oVfKD0XKeQeR+VsxH /rqbt5tOXE6X5qLRASB/Rq8XZVzQw7Gdi9vqzriUfoguIbQYNvenSNSb04qII9b1Tu gqaNXrkNypm3e7Gnj4HiLH2TCj68Zo1udnE3aPEc= Date: Tue, 22 Dec 2020 12:02:06 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 35/60] kasan, arm64: expand CONFIG_KASAN checks Message-ID: <20201222200206.DFpU_FeHE%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: expand CONFIG_KASAN checks Some #ifdef CONFIG_KASAN checks are only relevant for software KASAN modes (either related to shadow memory or compiler instrumentation). Expand those into CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS. Link: https://lkml.kernel.org/r/e6971e432dbd72bb897ff14134ebb7e169bdcf0c.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 2 +- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/assembler.h | 2 +- arch/arm64/include/asm/memory.h | 2 +- arch/arm64/include/asm/string.h | 5 +++-- arch/arm64/kernel/head.S | 2 +- arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kernel/kaslr.c | 3 ++- arch/arm64/kernel/module.c | 6 ++++-- arch/arm64/mm/ptdump.c | 6 +++--- include/linux/kasan-checks.h | 2 +- include/linux/kasan.h | 7 ++++--- include/linux/moduleloader.h | 3 ++- include/linux/string.h | 2 +- mm/ptdump.c | 13 ++++++++----- scripts/Makefile.lib | 2 ++ 16 files changed, 36 insertions(+), 25 deletions(-) --- a/arch/arm64/include/asm/assembler.h~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/include/asm/assembler.h @@ -473,7 +473,7 @@ USER(\label, ic ivau, \tmp2) // invali #define NOKPROBE(x) #endif -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define EXPORT_SYMBOL_NOKASAN(name) #else #define EXPORT_SYMBOL_NOKASAN(name) EXPORT_SYMBOL(name) --- a/arch/arm64/include/asm/memory.h~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/include/asm/memory.h @@ -72,7 +72,7 @@ * address space for the shadow region respectively. They can bloat the stack * significantly, so double the (minimum) stack size when they are in use. */ -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) \ + KASAN_SHADOW_OFFSET) --- a/arch/arm64/include/asm/string.h~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/include/asm/string.h @@ -5,7 +5,7 @@ #ifndef __ASM_STRING_H #define __ASM_STRING_H -#ifndef CONFIG_KASAN +#if !(defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) #define __HAVE_ARCH_STRRCHR extern char *strrchr(const char *, int c); @@ -48,7 +48,8 @@ extern void *__memset(void *, int, __ker void memcpy_flushcache(void *dst, const void *src, size_t cnt); #endif -#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__) +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ + !defined(__SANITIZE_ADDRESS__) /* * For files that are not instrumented (e.g. mm/slub.c) we --- a/arch/arm64/Kconfig~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/Kconfig @@ -334,7 +334,7 @@ config BROKEN_GAS_INST config KASAN_SHADOW_OFFSET hex - depends on KASAN + depends on KASAN_GENERIC || KASAN_SW_TAGS default 0xdfff800000000000 if (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) && !KASAN_SW_TAGS default 0xdfffc00000000000 if ARM64_VA_BITS_47 && !KASAN_SW_TAGS default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS --- a/arch/arm64/kernel/head.S~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/kernel/head.S @@ -433,7 +433,7 @@ SYM_FUNC_START_LOCAL(__primary_switched) bl __pi_memset dsb ishst // Make zero page visible to PTW -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) bl kasan_early_init #endif #ifdef CONFIG_RANDOMIZE_BASE --- a/arch/arm64/kernel/image-vars.h~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/kernel/image-vars.h @@ -37,7 +37,7 @@ __efistub_strncmp = __pi_strncmp; __efistub_strrchr = __pi_strrchr; __efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc; -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) __efistub___memcpy = __pi_memcpy; __efistub___memmove = __pi_memmove; __efistub___memset = __pi_memset; --- a/arch/arm64/kernel/kaslr.c~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/kernel/kaslr.c @@ -161,7 +161,8 @@ u64 __init kaslr_early_init(u64 dt_phys) /* use the top 16 bits to randomize the linear region */ memstart_offset_seed = seed >> 48; - if (IS_ENABLED(CONFIG_KASAN)) + if (IS_ENABLED(CONFIG_KASAN_GENERIC) || + IS_ENABLED(CONFIG_KASAN_SW_TAGS)) /* * KASAN does not expect the module region to intersect the * vmalloc region, since shadow memory is allocated for each --- a/arch/arm64/kernel/module.c~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/kernel/module.c @@ -30,7 +30,8 @@ void *module_alloc(unsigned long size) if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS)) gfp_mask |= __GFP_NOWARN; - if (IS_ENABLED(CONFIG_KASAN)) + if (IS_ENABLED(CONFIG_KASAN_GENERIC) || + IS_ENABLED(CONFIG_KASAN_SW_TAGS)) /* don't exceed the static module region - see below */ module_alloc_end = MODULES_END; @@ -39,7 +40,8 @@ void *module_alloc(unsigned long size) NUMA_NO_NODE, __builtin_return_address(0)); if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) && - !IS_ENABLED(CONFIG_KASAN)) + !IS_ENABLED(CONFIG_KASAN_GENERIC) && + !IS_ENABLED(CONFIG_KASAN_SW_TAGS)) /* * KASAN can only deal with module allocations being served * from the reserved module region, since the remainder of --- a/arch/arm64/Makefile~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/Makefile @@ -137,7 +137,7 @@ head-y := arch/arm64/kernel/head.o ifeq ($(CONFIG_KASAN_SW_TAGS), y) KASAN_SHADOW_SCALE_SHIFT := 4 -else +else ifeq ($(CONFIG_KASAN_GENERIC), y) KASAN_SHADOW_SCALE_SHIFT := 3 endif --- a/arch/arm64/mm/ptdump.c~kasan-arm64-expand-config_kasan-checks +++ a/arch/arm64/mm/ptdump.c @@ -29,7 +29,7 @@ enum address_markers_idx { PAGE_OFFSET_NR = 0, PAGE_END_NR, -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) KASAN_START_NR, #endif }; @@ -37,7 +37,7 @@ enum address_markers_idx { static struct addr_marker address_markers[] = { { PAGE_OFFSET, "Linear Mapping start" }, { 0 /* PAGE_END */, "Linear Mapping end" }, -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) { 0 /* KASAN_SHADOW_START */, "Kasan shadow start" }, { KASAN_SHADOW_END, "Kasan shadow end" }, #endif @@ -383,7 +383,7 @@ void ptdump_check_wx(void) static int ptdump_init(void) { address_markers[PAGE_END_NR].start_address = PAGE_END; -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) address_markers[KASAN_START_NR].start_address = KASAN_SHADOW_START; #endif ptdump_initialize(); --- a/include/linux/kasan-checks.h~kasan-arm64-expand-config_kasan-checks +++ a/include/linux/kasan-checks.h @@ -9,7 +9,7 @@ * even in compilation units that selectively disable KASAN, but must use KASAN * to validate access to an address. Never use these in header files! */ -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) bool __kasan_check_read(const volatile void *p, unsigned int size); bool __kasan_check_write(const volatile void *p, unsigned int size); #else --- a/include/linux/kasan.h~kasan-arm64-expand-config_kasan-checks +++ a/include/linux/kasan.h @@ -238,7 +238,8 @@ static inline void kasan_release_vmalloc #endif /* CONFIG_KASAN_VMALLOC */ -#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC) +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ + !defined(CONFIG_KASAN_VMALLOC) /* * These functions provide a special case to support backing module @@ -248,12 +249,12 @@ static inline void kasan_release_vmalloc int kasan_module_alloc(void *addr, size_t size); void kasan_free_shadow(const struct vm_struct *vm); -#else /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */ +#else /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */ static inline int kasan_module_alloc(void *addr, size_t size) { return 0; } static inline void kasan_free_shadow(const struct vm_struct *vm) {} -#endif /* CONFIG_KASAN && !CONFIG_KASAN_VMALLOC */ +#endif /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */ #ifdef CONFIG_KASAN_INLINE void kasan_non_canonical_hook(unsigned long addr); --- a/include/linux/moduleloader.h~kasan-arm64-expand-config_kasan-checks +++ a/include/linux/moduleloader.h @@ -96,7 +96,8 @@ void module_arch_cleanup(struct module * /* Any cleanup before freeing mod->module_init */ void module_arch_freeing_init(struct module *mod); -#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC) +#if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ + !defined(CONFIG_KASAN_VMALLOC) #include #define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT) #else --- a/include/linux/string.h~kasan-arm64-expand-config_kasan-checks +++ a/include/linux/string.h @@ -267,7 +267,7 @@ void __write_overflow(void) __compiletim #if !defined(__NO_FORTIFY) && defined(__OPTIMIZE__) && defined(CONFIG_FORTIFY_SOURCE) -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) extern void *__underlying_memchr(const void *p, int c, __kernel_size_t size) __RENAME(memchr); extern int __underlying_memcmp(const void *p, const void *q, __kernel_size_t size) __RENAME(memcmp); extern void *__underlying_memcpy(void *p, const void *q, __kernel_size_t size) __RENAME(memcpy); --- a/mm/ptdump.c~kasan-arm64-expand-config_kasan-checks +++ a/mm/ptdump.c @@ -4,7 +4,7 @@ #include #include -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) /* * This is an optimization for KASAN=y case. Since all kasan page tables * eventually point to the kasan_early_shadow_page we could call note_page() @@ -31,7 +31,8 @@ static int ptdump_pgd_entry(pgd_t *pgd, struct ptdump_state *st = walk->private; pgd_t val = READ_ONCE(*pgd); -#if CONFIG_PGTABLE_LEVELS > 4 && defined(CONFIG_KASAN) +#if CONFIG_PGTABLE_LEVELS > 4 && \ + (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) if (pgd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_p4d))) return note_kasan_page_table(walk, addr); #endif @@ -51,7 +52,8 @@ static int ptdump_p4d_entry(p4d_t *p4d, struct ptdump_state *st = walk->private; p4d_t val = READ_ONCE(*p4d); -#if CONFIG_PGTABLE_LEVELS > 3 && defined(CONFIG_KASAN) +#if CONFIG_PGTABLE_LEVELS > 3 && \ + (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) if (p4d_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pud))) return note_kasan_page_table(walk, addr); #endif @@ -71,7 +73,8 @@ static int ptdump_pud_entry(pud_t *pud, struct ptdump_state *st = walk->private; pud_t val = READ_ONCE(*pud); -#if CONFIG_PGTABLE_LEVELS > 2 && defined(CONFIG_KASAN) +#if CONFIG_PGTABLE_LEVELS > 2 && \ + (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) if (pud_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pmd))) return note_kasan_page_table(walk, addr); #endif @@ -91,7 +94,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, struct ptdump_state *st = walk->private; pmd_t val = READ_ONCE(*pmd); -#if defined(CONFIG_KASAN) +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) if (pmd_page(val) == virt_to_page(lm_alias(kasan_early_shadow_pte))) return note_kasan_page_table(walk, addr); #endif --- a/scripts/Makefile.lib~kasan-arm64-expand-config_kasan-checks +++ a/scripts/Makefile.lib @@ -148,10 +148,12 @@ endif # we don't want to check (depends on variables KASAN_SANITIZE_obj.o, KASAN_SANITIZE) # ifeq ($(CONFIG_KASAN),y) +ifneq ($(CONFIG_KASAN_HW_TAGS),y) _c_flags += $(if $(patsubst n%,, \ $(KASAN_SANITIZE_$(basetarget).o)$(KASAN_SANITIZE)y), \ $(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE)) endif +endif ifeq ($(CONFIG_UBSAN),y) _c_flags += $(if $(patsubst n%,, \ From patchwork Tue Dec 22 20:02:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97EE9C433DB for ; Tue, 22 Dec 2020 20:02:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 196CA20700 for ; Tue, 22 Dec 2020 20:02:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 196CA20700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6BD278D0026; Tue, 22 Dec 2020 15:02:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6453F8D0024; Tue, 22 Dec 2020 15:02:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5329E8D0026; Tue, 22 Dec 2020 15:02:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id 3AF058D0024 for ; Tue, 22 Dec 2020 15:02:13 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E4B553638 for ; Tue, 22 Dec 2020 20:02:12 +0000 (UTC) X-FDA: 77621989704.06.bell71_080132227462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id AA9D0100349A8 for ; Tue, 22 Dec 2020 20:02:12 +0000 (UTC) X-HE-Tag: bell71_080132227462 X-Filterd-Recvd-Size: 16323 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7C28220705; Tue, 22 Dec 2020 20:02:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667331; bh=14hTgMhvxcDg5e5CrHlxvJbgpL3KAuzaehlcujLSvg4=; h=Date:From:To:Subject:In-Reply-To:From; b=cWjpG4a8ZpqiiFoR9oWgREbhY0l/oILN/HQCyrrW5VB8pWXSN4K5hkKOnGR8k1WiC XaiTROnhjLi73soS0QC2OYXLJmQO6/i6xhnIfrGEzkZgFStHZThfW+d2cB//hlvFUY DyIJ4F35tyZw6vhEtTIDlc4KrwS9Sa7SKfGclUNI= Date: Tue, 22 Dec 2020 12:02:10 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 36/60] kasan, arm64: implement HW_TAGS runtime Message-ID: <20201222200210.CkKPSORQz%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: implement HW_TAGS runtime Provide implementation of KASAN functions required for the hardware tag-based mode. Those include core functions for memory and pointer tagging (tags_hw.c) and bug reporting (report_tags_hw.c). Also adapt common KASAN code to support the new mode. Link: https://lkml.kernel.org/r/cfd0fbede579a6b66755c98c88c108e54f9c56bf.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Acked-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/memory.h | 4 - arch/arm64/kernel/cpufeature.c | 3 + arch/arm64/kernel/smp.c | 2 include/linux/kasan.h | 24 +++++--- include/linux/mm.h | 2 include/linux/page-flags-layout.h | 2 mm/kasan/Makefile | 5 + mm/kasan/common.c | 15 ++--- mm/kasan/hw_tags.c | 80 ++++++++++++++++++++++++++++ mm/kasan/kasan.h | 19 +++++- mm/kasan/report_hw_tags.c | 42 ++++++++++++++ mm/kasan/report_sw_tags.c | 2 mm/kasan/shadow.c | 2 mm/kasan/sw_tags.c | 2 14 files changed, 178 insertions(+), 26 deletions(-) --- a/arch/arm64/include/asm/memory.h~kasan-arm64-implement-hw_tags-runtime +++ a/arch/arm64/include/asm/memory.h @@ -214,7 +214,7 @@ static inline unsigned long kaslr_offset (__force __typeof__(addr))__addr; \ }) -#ifdef CONFIG_KASAN_SW_TAGS +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) #define __tag_shifted(tag) ((u64)(tag) << 56) #define __tag_reset(addr) __untagged_addr(addr) #define __tag_get(addr) (__u8)((u64)(addr) >> 56) @@ -222,7 +222,7 @@ static inline unsigned long kaslr_offset #define __tag_shifted(tag) 0UL #define __tag_reset(addr) (addr) #define __tag_get(addr) 0 -#endif /* CONFIG_KASAN_SW_TAGS */ +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ static inline const void *__tag_set(const void *addr, u8 tag) { --- a/arch/arm64/kernel/cpufeature.c~kasan-arm64-implement-hw_tags-runtime +++ a/arch/arm64/kernel/cpufeature.c @@ -70,6 +70,7 @@ #include #include #include +#include #include #include #include @@ -1710,6 +1711,8 @@ static void cpu_enable_mte(struct arm64_ cleared_zero_page = true; mte_clear_page_tags(lm_alias(empty_zero_page)); } + + kasan_init_hw_tags_cpu(); } #endif /* CONFIG_ARM64_MTE */ --- a/arch/arm64/kernel/smp.c~kasan-arm64-implement-hw_tags-runtime +++ a/arch/arm64/kernel/smp.c @@ -462,6 +462,8 @@ void __init smp_prepare_boot_cpu(void) /* Conditionally switch to GIC PMR for interrupt masking */ if (system_uses_irq_prio_masking()) init_gic_priority_masking(); + + kasan_init_hw_tags(); } static u64 __init of_get_cpu_mpidr(struct device_node *dn) --- a/include/linux/kasan.h~kasan-arm64-implement-hw_tags-runtime +++ a/include/linux/kasan.h @@ -190,25 +190,35 @@ static inline void kasan_record_aux_stac #endif /* CONFIG_KASAN_GENERIC */ -#ifdef CONFIG_KASAN_SW_TAGS - -void __init kasan_init_sw_tags(void); +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) void *kasan_reset_tag(const void *addr); bool kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); -#else /* CONFIG_KASAN_SW_TAGS */ - -static inline void kasan_init_sw_tags(void) { } +#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ static inline void *kasan_reset_tag(const void *addr) { return (void *)addr; } -#endif /* CONFIG_KASAN_SW_TAGS */ +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS*/ + +#ifdef CONFIG_KASAN_SW_TAGS +void __init kasan_init_sw_tags(void); +#else +static inline void kasan_init_sw_tags(void) { } +#endif + +#ifdef CONFIG_KASAN_HW_TAGS +void kasan_init_hw_tags_cpu(void); +void __init kasan_init_hw_tags(void); +#else +static inline void kasan_init_hw_tags_cpu(void) { } +static inline void kasan_init_hw_tags(void) { } +#endif #ifdef CONFIG_KASAN_VMALLOC --- a/include/linux/mm.h~kasan-arm64-implement-hw_tags-runtime +++ a/include/linux/mm.h @@ -1421,7 +1421,7 @@ static inline bool cpupid_match_pid(stru } #endif /* CONFIG_NUMA_BALANCING */ -#ifdef CONFIG_KASAN_SW_TAGS +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) static inline u8 page_kasan_tag(const struct page *page) { return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; --- a/include/linux/page-flags-layout.h~kasan-arm64-implement-hw_tags-runtime +++ a/include/linux/page-flags-layout.h @@ -77,7 +77,7 @@ #define LAST_CPUPID_SHIFT 0 #endif -#ifdef CONFIG_KASAN_SW_TAGS +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) #define KASAN_TAG_WIDTH 8 #else #define KASAN_TAG_WIDTH 0 --- a/mm/kasan/common.c~kasan-arm64-implement-hw_tags-runtime +++ a/mm/kasan/common.c @@ -118,7 +118,7 @@ void kasan_free_pages(struct page *page, */ static inline unsigned int optimal_redzone(unsigned int object_size) { - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) return 0; return @@ -183,14 +183,14 @@ size_t kasan_metadata_size(struct kmem_c struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, const void *object) { - return (void *)object + cache->kasan_info.alloc_meta_offset; + return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; } struct kasan_free_meta *get_free_info(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; + return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; } void kasan_poison_slab(struct page *page) @@ -272,9 +272,8 @@ void * __must_check kasan_init_slab_obj( alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) - object = set_tag(object, - assign_tag(cache, object, true, false)); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) + object = set_tag(object, assign_tag(cache, object, true, false)); return (void *)object; } @@ -342,10 +341,10 @@ static void *__kasan_kmalloc(struct kmem redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_GRANULE_SIZE); - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) tag = assign_tag(cache, object, false, keep_tag); - /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ + /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */ unpoison_range(set_tag(object, tag), size); poison_range((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); --- /dev/null +++ a/mm/kasan/hw_tags.c @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * This file contains core hardware tag-based KASAN code. + * + * Copyright (c) 2020 Google, Inc. + * Author: Andrey Konovalov + */ + +#define pr_fmt(fmt) "kasan: " fmt + +#include +#include +#include +#include +#include +#include + +#include "kasan.h" + +/* kasan_init_hw_tags_cpu() is called for each CPU. */ +void kasan_init_hw_tags_cpu(void) +{ + hw_init_tags(KASAN_TAG_MAX); + hw_enable_tagging(); +} + +/* kasan_init_hw_tags() is called once on boot CPU. */ +void __init kasan_init_hw_tags(void) +{ + pr_info("KernelAddressSanitizer initialized\n"); +} + +void *kasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + +void poison_range(const void *address, size_t size, u8 value) +{ + hw_set_mem_tag_range(reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), value); +} + +void unpoison_range(const void *address, size_t size) +{ + hw_set_mem_tag_range(reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); +} + +u8 random_tag(void) +{ + return hw_get_random_tag(); +} + +bool check_invalid_free(void *addr) +{ + u8 ptr_tag = get_tag(addr); + u8 mem_tag = hw_get_mem_tag(addr); + + return (mem_tag == KASAN_TAG_INVALID) || + (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); +} + +void kasan_set_free_info(struct kmem_cache *cache, + void *object, u8 tag) +{ + struct kasan_alloc_meta *alloc_meta; + + alloc_meta = get_alloc_info(cache, object); + kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); +} + +struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, + void *object, u8 tag) +{ + struct kasan_alloc_meta *alloc_meta; + + alloc_meta = get_alloc_info(cache, object); + return &alloc_meta->free_track[0]; +} --- a/mm/kasan/kasan.h~kasan-arm64-implement-hw_tags-runtime +++ a/mm/kasan/kasan.h @@ -154,6 +154,11 @@ struct kasan_alloc_meta *get_alloc_info( struct kasan_free_meta *get_free_info(struct kmem_cache *cache, const void *object); +void poison_range(const void *address, size_t size, u8 value); +void unpoison_range(const void *address, size_t size); + +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) + static inline const void *kasan_shadow_to_mem(const void *shadow_addr) { return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET) @@ -165,9 +170,6 @@ static inline bool addr_has_metadata(con return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); } -void poison_range(const void *address, size_t size, u8 value); -void unpoison_range(const void *address, size_t size); - /** * check_memory_region - Check memory region, and report if invalid access. * @addr: the accessed address @@ -179,6 +181,15 @@ void unpoison_range(const void *address, bool check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + +static inline bool addr_has_metadata(const void *addr) +{ + return true; +} + +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); @@ -215,7 +226,7 @@ static inline void quarantine_reduce(voi static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif -#ifdef CONFIG_KASAN_SW_TAGS +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) void print_tags(u8 addr_tag, const void *addr); --- a/mm/kasan/Makefile~kasan-arm64-implement-hw_tags-runtime +++ a/mm/kasan/Makefile @@ -10,8 +10,10 @@ CFLAGS_REMOVE_init.o = $(CC_FLAGS_FTRACE CFLAGS_REMOVE_quarantine.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_report.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_report_generic.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_report_hw_tags.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_report_sw_tags.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_shadow.o = $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_hw_tags.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_sw_tags.o = $(CC_FLAGS_FTRACE) # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 @@ -27,10 +29,13 @@ CFLAGS_init.o := $(CC_FLAGS_KASAN_RUNTIM CFLAGS_quarantine.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_report.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_report_generic.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_report_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_report_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_shadow.o := $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_hw_tags.o := $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_sw_tags.o := $(CC_FLAGS_KASAN_RUNTIME) obj-$(CONFIG_KASAN) := common.o report.o obj-$(CONFIG_KASAN_GENERIC) += init.o generic.o report_generic.o shadow.o quarantine.o +obj-$(CONFIG_KASAN_HW_TAGS) += hw_tags.o report_hw_tags.o obj-$(CONFIG_KASAN_SW_TAGS) += init.o report_sw_tags.o shadow.o sw_tags.o --- /dev/null +++ a/mm/kasan/report_hw_tags.c @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * This file contains hardware tag-based KASAN specific error reporting code. + * + * Copyright (c) 2020 Google, Inc. + * Author: Andrey Konovalov + */ + +#include +#include +#include +#include +#include +#include + +#include "kasan.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} + +void *find_first_bad_addr(void *addr, size_t size) +{ + return reset_tag(addr); +} + +void metadata_fetch_row(char *buffer, void *row) +{ + int i; + + for (i = 0; i < META_BYTES_PER_ROW; i++) + buffer[i] = hw_get_mem_tag(row + i * KASAN_GRANULE_SIZE); +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 memory_tag = hw_get_mem_tag((void *)addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", + addr_tag, memory_tag); +} --- a/mm/kasan/report_sw_tags.c~kasan-arm64-implement-hw_tags-runtime +++ a/mm/kasan/report_sw_tags.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * This file contains tag-based KASAN specific error reporting code. + * This file contains software tag-based KASAN specific error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin --- a/mm/kasan/shadow.c~kasan-arm64-implement-hw_tags-runtime +++ a/mm/kasan/shadow.c @@ -107,7 +107,7 @@ void unpoison_range(const void *address, if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) *shadow = tag; - else + else /* CONFIG_KASAN_GENERIC */ *shadow = size & KASAN_GRANULE_MASK; } } --- a/mm/kasan/sw_tags.c~kasan-arm64-implement-hw_tags-runtime +++ a/mm/kasan/sw_tags.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 /* - * This file contains core tag-based KASAN code. + * This file contains core software tag-based KASAN code. * * Copyright (c) 2018 Google, Inc. * Author: Andrey Konovalov From patchwork Tue Dec 22 20:02:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68242C433DB for ; Tue, 22 Dec 2020 20:02:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1135320795 for ; Tue, 22 Dec 2020 20:02:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1135320795 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DC1328D0028; Tue, 22 Dec 2020 15:02:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D49A18D0027; Tue, 22 Dec 2020 15:02:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C11108D0028; Tue, 22 Dec 2020 15:02:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id A896E8D0027 for ; Tue, 22 Dec 2020 15:02:16 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6937D1EF1 for ; Tue, 22 Dec 2020 20:02:16 +0000 (UTC) X-FDA: 77621989872.20.coal95_240331127462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 4BF36180C07AB for ; Tue, 22 Dec 2020 20:02:16 +0000 (UTC) X-HE-Tag: coal95_240331127462 X-Filterd-Recvd-Size: 4748 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 215DA21D1B; Tue, 22 Dec 2020 20:02:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667334; bh=PrQJM9rlJRzM+cN/hE0wVm8FU6/mdbHuXXko4hMlZVg=; h=Date:From:To:Subject:In-Reply-To:From; b=vttNYF3jjn3odV0CidymostduPv6Lat0JL8TYxLHCadkYRSLk58I6hKIfG3cZVgVl Y6hu2HEDvNE1b17YL51JQQrk0pqch7QMty/qheN8h+lnv8K+ekjFmPFVH5MIHnTzeS 0GFqqUKBYEndy/OEYxF7wiR3fchT2HsrKRzNymE0= Date: Tue, 22 Dec 2020 12:02:13 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 37/60] kasan, arm64: print report from tag fault handler Message-ID: <20201222200213.YGSu-WgCL%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: print report from tag fault handler Add error reporting for hardware tag-based KASAN. When CONFIG_KASAN_HW_TAGS is enabled, print KASAN report from the arm64 tag fault handler. SAS bits aren't set in ESR for all faults reported in EL1, so it's impossible to find out the size of the access the caused the fault. Adapt KASAN reporting code to handle this case. Link: https://lkml.kernel.org/r/b559c82b6a969afedf53b4694b475f0234067a1a.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/mm/fault.c | 14 ++++++++++++++ mm/kasan/report.c | 11 ++++++++--- 2 files changed, 22 insertions(+), 3 deletions(-) --- a/arch/arm64/mm/fault.c~kasan-arm64-print-report-from-tag-fault-handler +++ a/arch/arm64/mm/fault.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -297,10 +298,23 @@ static void die_kernel_fault(const char do_exit(SIGKILL); } +#ifdef CONFIG_KASAN_HW_TAGS static void report_tag_fault(unsigned long addr, unsigned int esr, struct pt_regs *regs) { + bool is_write = ((esr & ESR_ELx_WNR) >> ESR_ELx_WNR_SHIFT) != 0; + + /* + * SAS bits aren't set for all faults reported in EL1, so we can't + * find out access size. + */ + kasan_report(addr, 0, is_write, regs->pc); } +#else +/* Tag faults aren't enabled without CONFIG_KASAN_HW_TAGS. */ +static inline void report_tag_fault(unsigned long addr, unsigned int esr, + struct pt_regs *regs) { } +#endif static void do_tag_recovery(unsigned long addr, unsigned int esr, struct pt_regs *regs) --- a/mm/kasan/report.c~kasan-arm64-print-report-from-tag-fault-handler +++ a/mm/kasan/report.c @@ -62,9 +62,14 @@ static void print_error_description(stru { pr_err("BUG: KASAN: %s in %pS\n", get_bug_type(info), (void *)info->ip); - pr_err("%s of size %zu at addr %px by task %s/%d\n", - info->is_write ? "Write" : "Read", info->access_size, - info->access_addr, current->comm, task_pid_nr(current)); + if (info->access_size) + pr_err("%s of size %zu at addr %px by task %s/%d\n", + info->is_write ? "Write" : "Read", info->access_size, + info->access_addr, current->comm, task_pid_nr(current)); + else + pr_err("%s at addr %px by task %s/%d\n", + info->is_write ? "Write" : "Read", + info->access_addr, current->comm, task_pid_nr(current)); } static DEFINE_SPINLOCK(report_lock); From patchwork Tue Dec 22 20:02:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7CC6C433DB for ; Tue, 22 Dec 2020 20:02:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5A2D620855 for ; Tue, 22 Dec 2020 20:02:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A2D620855 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 360078D0029; Tue, 22 Dec 2020 15:02:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E8638D002A; Tue, 22 Dec 2020 15:02:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FE448D0029; Tue, 22 Dec 2020 15:02:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 07B4D8D0027 for ; Tue, 22 Dec 2020 15:02:20 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C5364180AD815 for ; Tue, 22 Dec 2020 20:02:19 +0000 (UTC) X-FDA: 77621989998.07.view00_3a17c6d27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id A521E1803F9A8 for ; Tue, 22 Dec 2020 20:02:19 +0000 (UTC) X-HE-Tag: view00_3a17c6d27462 X-Filterd-Recvd-Size: 8012 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 92FEE23133; Tue, 22 Dec 2020 20:02:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667338; bh=SVHQFaEU4AA89s82OpI6oHUPgqGsTJGJuIRydzHoxG8=; h=Date:From:To:Subject:In-Reply-To:From; b=vlmaBK06bH1Zv2EOdt+L6hWCErRcHemZU73hZ0a8jYX+gUlSKG0NX2K2JxbSOOdzz +XCbO06YK8eCLQRQmr1RBYTryrITqhurMj19u41rn5IAU7ZNGyqHHbDWZCAKE25i6R ZRd53NQVZLXkzfHrgeJNz8F3fOEUse/NOvsiAyDo= Date: Tue, 22 Dec 2020 12:02:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 38/60] kasan, mm: reset tags when accessing metadata Message-ID: <20201222200217.9hOE1mcZ2%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, mm: reset tags when accessing metadata Kernel allocator code accesses metadata for slab objects, that may lie out-of-bounds of the object itself, or be accessed when an object is freed. Such accesses trigger tag faults and lead to false-positive reports with hardware tag-based KASAN. Software KASAN modes disable instrumentation for allocator code via KASAN_SANITIZE Makefile macro, and rely on kasan_enable/disable_current() annotations which are used to ignore KASAN reports. With hardware tag-based KASAN neither of those options are available, as it doesn't use compiler instrumetation, no tag faults are ignored, and MTE is disabled after the first one. Instead, reset tags when accessing metadata (currently only for SLUB). Link: https://lkml.kernel.org/r/a0f3cefbc49f34c843b664110842de4db28179d0.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Acked-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/page_alloc.c | 4 +++- mm/page_poison.c | 2 +- mm/slub.c | 29 ++++++++++++++++------------- 3 files changed, 20 insertions(+), 15 deletions(-) --- a/mm/page_alloc.c~kasan-mm-reset-tags-when-accessing-metadata +++ a/mm/page_alloc.c @@ -1204,8 +1204,10 @@ static void kernel_init_free_pages(struc /* s390's use of memset() could override KASAN redzones. */ kasan_disable_current(); - for (i = 0; i < numpages; i++) + for (i = 0; i < numpages; i++) { + page_kasan_tag_reset(page + i); clear_highpage(page + i); + } kasan_enable_current(); } --- a/mm/page_poison.c~kasan-mm-reset-tags-when-accessing-metadata +++ a/mm/page_poison.c @@ -25,7 +25,7 @@ static void poison_page(struct page *pag /* KASAN still think the page is in-use, so skip it. */ kasan_disable_current(); - memset(addr, PAGE_POISON, PAGE_SIZE); + memset(kasan_reset_tag(addr), PAGE_POISON, PAGE_SIZE); kasan_enable_current(); kunmap_atomic(addr); } --- a/mm/slub.c~kasan-mm-reset-tags-when-accessing-metadata +++ a/mm/slub.c @@ -249,7 +249,7 @@ static inline void *freelist_ptr(const s { #ifdef CONFIG_SLAB_FREELIST_HARDENED /* - * When CONFIG_KASAN_SW_TAGS is enabled, ptr_addr might be tagged. + * When CONFIG_KASAN_SW/HW_TAGS is enabled, ptr_addr might be tagged. * Normally, this doesn't cause any issues, as both set_freepointer() * and get_freepointer() are called with a pointer with the same tag. * However, there are some issues with CONFIG_SLUB_DEBUG code. For @@ -275,6 +275,7 @@ static inline void *freelist_dereference static inline void *get_freepointer(struct kmem_cache *s, void *object) { + object = kasan_reset_tag(object); return freelist_dereference(s, object + s->offset); } @@ -304,6 +305,7 @@ static inline void set_freepointer(struc BUG_ON(object == fp); /* naive detection of double free or corruption */ #endif + freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr); *(void **)freeptr_addr = freelist_ptr(s, fp, freeptr_addr); } @@ -538,8 +540,8 @@ static void print_section(char *level, c unsigned int length) { metadata_access_enable(); - print_hex_dump(level, text, DUMP_PREFIX_ADDRESS, 16, 1, addr, - length, 1); + print_hex_dump(level, kasan_reset_tag(text), DUMP_PREFIX_ADDRESS, + 16, 1, addr, length, 1); metadata_access_disable(); } @@ -570,7 +572,7 @@ static struct track *get_track(struct km p = object + get_info_end(s); - return p + alloc; + return kasan_reset_tag(p + alloc); } static void set_track(struct kmem_cache *s, void *object, @@ -583,7 +585,8 @@ static void set_track(struct kmem_cache unsigned int nr_entries; metadata_access_enable(); - nr_entries = stack_trace_save(p->addrs, TRACK_ADDRS_COUNT, 3); + nr_entries = stack_trace_save(kasan_reset_tag(p->addrs), + TRACK_ADDRS_COUNT, 3); metadata_access_disable(); if (nr_entries < TRACK_ADDRS_COUNT) @@ -747,7 +750,7 @@ static __printf(3, 4) void slab_err(stru static void init_object(struct kmem_cache *s, void *object, u8 val) { - u8 *p = object; + u8 *p = kasan_reset_tag(object); if (s->flags & SLAB_RED_ZONE) memset(p - s->red_left_pad, val, s->red_left_pad); @@ -777,7 +780,7 @@ static int check_bytes_and_report(struct u8 *addr = page_address(page); metadata_access_enable(); - fault = memchr_inv(start, value, bytes); + fault = memchr_inv(kasan_reset_tag(start), value, bytes); metadata_access_disable(); if (!fault) return 1; @@ -873,7 +876,7 @@ static int slab_pad_check(struct kmem_ca pad = end - remainder; metadata_access_enable(); - fault = memchr_inv(pad, POISON_INUSE, remainder); + fault = memchr_inv(kasan_reset_tag(pad), POISON_INUSE, remainder); metadata_access_disable(); if (!fault) return 1; @@ -1118,7 +1121,7 @@ void setup_page_debug(struct kmem_cache return; metadata_access_enable(); - memset(addr, POISON_INUSE, page_size(page)); + memset(kasan_reset_tag(addr), POISON_INUSE, page_size(page)); metadata_access_disable(); } @@ -1566,10 +1569,10 @@ static inline bool slab_free_freelist_ho * Clear the object and the metadata, but don't touch * the redzone. */ - memset(object, 0, s->object_size); + memset(kasan_reset_tag(object), 0, s->object_size); rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad : 0; - memset((char *)object + s->inuse, 0, + memset((char *)kasan_reset_tag(object) + s->inuse, 0, s->size - s->inuse - rsize); } @@ -2881,10 +2884,10 @@ redo: stat(s, ALLOC_FASTPATH); } - maybe_wipe_obj_freeptr(s, object); + maybe_wipe_obj_freeptr(s, kasan_reset_tag(object)); if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object) - memset(object, 0, s->object_size); + memset(kasan_reset_tag(object), 0, s->object_size); slab_post_alloc_hook(s, objcg, gfpflags, 1, &object); From patchwork Tue Dec 22 20:02:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AFD3C433DB for ; Tue, 22 Dec 2020 20:03:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E2876229C5 for ; Tue, 22 Dec 2020 20:03:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2876229C5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 09F078D002B; Tue, 22 Dec 2020 15:02:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 050CE8D002A; Tue, 22 Dec 2020 15:02:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECDE48D002B; Tue, 22 Dec 2020 15:02:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id D2D1E8D002A for ; Tue, 22 Dec 2020 15:02:23 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 81FAA3638 for ; Tue, 22 Dec 2020 20:02:23 +0000 (UTC) X-FDA: 77621990166.30.force93_1f1809727462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 3581A180B3C8E for ; Tue, 22 Dec 2020 20:02:23 +0000 (UTC) X-HE-Tag: force93_1f1809727462 X-Filterd-Recvd-Size: 2892 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1E7F023137; Tue, 22 Dec 2020 20:02:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667341; bh=24kg3PNyKlXnB6Xe61RW6pY/C7fqdxm5pJlEpurIZYw=; h=Date:From:To:Subject:In-Reply-To:From; b=OGpn/ioGrNub06J0iS4FlJNbQT2o0xADJff5SMLvDGocPf0rRsufJRR7ZUYIIuqIi 1zu/sLHTSj1gXyGZbJCeCnu8iZR7svdPqV8/AKSvrL696r9f9jTkri5/52aF2tSK+y Q7/n7EBLchOuFv3VY4BPBLMh4SQIrONxDhYobBb0= Date: Tue, 22 Dec 2020 12:02:20 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 39/60] kasan, arm64: enable CONFIG_KASAN_HW_TAGS Message-ID: <20201222200220.wzQYtRYE1%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: enable CONFIG_KASAN_HW_TAGS Hardware tag-based KASAN is now ready, enable the configuration option. Link: https://lkml.kernel.org/r/a6fa50d3bb6b318e05c6389a44095be96442b8b0.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Acked-by: Catalin Marinas Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) --- a/arch/arm64/Kconfig~kasan-arm64-enable-config_kasan_hw_tags +++ a/arch/arm64/Kconfig @@ -137,6 +137,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN + select HAVE_ARCH_KASAN_HW_TAGS if (HAVE_ARCH_KASAN && ARM64_MTE) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT From patchwork Tue Dec 22 20:02:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EA63C433E0 for ; Tue, 22 Dec 2020 20:03:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 27FB1229C5 for ; Tue, 22 Dec 2020 20:03:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27FB1229C5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4700C8D002D; Tue, 22 Dec 2020 15:02:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 422098D002C; Tue, 22 Dec 2020 15:02:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30F8B8D002D; Tue, 22 Dec 2020 15:02:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 136378D002C for ; Tue, 22 Dec 2020 15:02:27 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CB2798249980 for ; Tue, 22 Dec 2020 20:02:26 +0000 (UTC) X-FDA: 77621990292.03.shake42_021243327462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 9CE8D28A4E9 for ; Tue, 22 Dec 2020 20:02:26 +0000 (UTC) X-HE-Tag: shake42_021243327462 X-Filterd-Recvd-Size: 9092 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7EC5120700; Tue, 22 Dec 2020 20:02:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667345; bh=tAxHNT7XQoQvZrsc0GxwdWXao08bTrbBlGwiVEcT74w=; h=Date:From:To:Subject:In-Reply-To:From; b=KjpAQHwT/bZns4HdKznUu3dtNKTV03QQP+WrD6l5jm/mhV9nsI/LYClQnSywKGKtl l0ri9DgYD7uRHLmC1k5Ur2L2We/38iCL+bgZLN6I5Up1VCAoOBpNX4Upk6D57dabqb XXRdHt8KH5NchZPJWOLtXrDROyKCFUXDQVl25mFs= Date: Tue, 22 Dec 2020 12:02:24 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 40/60] kasan: add documentation for hardware tag-based mode Message-ID: <20201222200224.JRY9mJCgq%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add documentation for hardware tag-based mode Add documentation for hardware tag-based KASAN mode and also add some clarifications for software tag-based mode. Link: https://lkml.kernel.org/r/20ed1d387685e89fc31be068f890f070ef9fd5d5.1606161801.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Vincenzo Frascino Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Tested-by: Vincenzo Frascino Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/dev-tools/kasan.rst | 84 ++++++++++++++++++++-------- 1 file changed, 61 insertions(+), 23 deletions(-) --- a/Documentation/dev-tools/kasan.rst~kasan-add-documentation-for-hardware-tag-based-mode +++ a/Documentation/dev-tools/kasan.rst @@ -5,12 +5,14 @@ Overview -------- KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to -find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN -(similar to userspace ASan) and software tag-based KASAN (similar to userspace -HWASan). - -KASAN uses compile-time instrumentation to insert validity checks before every -memory access, and therefore requires a compiler version that supports that. +find out-of-bound and use-after-free bugs. KASAN has three modes: +1. generic KASAN (similar to userspace ASan), +2. software tag-based KASAN (similar to userspace HWASan), +3. hardware tag-based KASAN (based on hardware memory tagging). + +Software KASAN modes (1 and 2) use compile-time instrumentation to insert +validity checks before every memory access, and therefore require a compiler +version that supports that. Generic KASAN is supported in both GCC and Clang. With GCC it requires version 8.3.0 or later. Any supported Clang version is compatible, but detection of @@ -19,7 +21,7 @@ out-of-bounds accesses for global variab Tag-based KASAN is only supported in Clang. Currently generic KASAN is supported for the x86_64, arm64, xtensa, s390 and -riscv architectures, and tag-based KASAN is supported only for arm64. +and riscv architectures, and tag-based KASAN modes are supported only for arm64. Usage ----- @@ -28,14 +30,16 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and -CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN). - -You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. -Outline and inline are compiler instrumentation types. The former produces -smaller binary while the latter is 1.1 - 2 times faster. +and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN), +CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN), and +CONFIG_KASAN_HW_TAGS (to enable hardware tag-based KASAN). + +For software modes, you also need to choose between CONFIG_KASAN_OUTLINE and +CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. +The former produces smaller binary while the latter is 1.1 - 2 times faster. -Both KASAN modes work with both SLUB and SLAB memory allocators. +Both software KASAN modes work with both SLUB and SLAB memory allocators, +hardware tag-based KASAN currently only support SLUB. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To augment reports with last allocation and freeing stack of the physical page, @@ -197,17 +201,24 @@ call_rcu() and workqueue queuing. Software tag-based KASAN ~~~~~~~~~~~~~~~~~~~~~~~~ -Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to -store a pointer tag in the top byte of kernel pointers. Like generic KASAN it -uses shadow memory to store memory tags associated with each 16-byte memory +Software tag-based KASAN requires software memory tagging support in the form +of HWASan-like compiler instrumentation (see HWASan documentation for details). + +Software tag-based KASAN is currently only implemented for arm64 architecture. + +Software tag-based KASAN uses the Top Byte Ignore (TBI) feature of arm64 CPUs +to store a pointer tag in the top byte of kernel pointers. Like generic KASAN +it uses shadow memory to store memory tags associated with each 16-byte memory cell (therefore it dedicates 1/16th of the kernel memory for shadow memory). -On each memory allocation tag-based KASAN generates a random tag, tags the -allocated memory with this tag, and embeds this tag into the returned pointer. +On each memory allocation software tag-based KASAN generates a random tag, tags +the allocated memory with this tag, and embeds this tag into the returned +pointer. + Software tag-based KASAN uses compile-time instrumentation to insert checks before each memory access. These checks make sure that tag of the memory that is being accessed is equal to tag of the pointer that is used to access this -memory. In case of a tag mismatch tag-based KASAN prints a bug report. +memory. In case of a tag mismatch software tag-based KASAN prints a bug report. Software tag-based KASAN also has two instrumentation modes (outline, that emits callbacks to check memory accesses; and inline, that performs the shadow @@ -216,9 +227,36 @@ simply printed from the function that pe instrumentation a brk instruction is emitted by the compiler, and a dedicated brk handler is used to print bug reports. -A potential expansion of this mode is a hardware tag-based mode, which would -use hardware memory tagging support instead of compiler instrumentation and -manual shadow memory manipulation. +Software tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through +pointers with 0xFF pointer tag aren't checked). The value 0xFE is currently +reserved to tag freed memory regions. + +Software tag-based KASAN currently only supports tagging of +kmem_cache_alloc/kmalloc and page_alloc memory. + +Hardware tag-based KASAN +~~~~~~~~~~~~~~~~~~~~~~~~ + +Hardware tag-based KASAN is similar to the software mode in concept, but uses +hardware memory tagging support instead of compiler instrumentation and +shadow memory. + +Hardware tag-based KASAN is currently only implemented for arm64 architecture +and based on both arm64 Memory Tagging Extension (MTE) introduced in ARMv8.5 +Instruction Set Architecture, and Top Byte Ignore (TBI). + +Special arm64 instructions are used to assign memory tags for each allocation. +Same tags are assigned to pointers to those allocations. On every memory +access, hardware makes sure that tag of the memory that is being accessed is +equal to tag of the pointer that is used to access this memory. In case of a +tag mismatch a fault is generated and a report is printed. + +Hardware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through +pointers with 0xFF pointer tag aren't checked). The value 0xFE is currently +reserved to tag freed memory regions. + +Hardware tag-based KASAN currently only supports tagging of +kmem_cache_alloc/kmalloc and page_alloc memory. What memory accesses are sanitised by KASAN? -------------------------------------------- From patchwork Tue Dec 22 20:02:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC88DC433E0 for ; Tue, 22 Dec 2020 20:03:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 79AFC21D81 for ; Tue, 22 Dec 2020 20:03:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79AFC21D81 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92BD08D002E; Tue, 22 Dec 2020 15:02:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9010D8D002C; Tue, 22 Dec 2020 15:02:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A2388D002E; Tue, 22 Dec 2020 15:02:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id 6324E8D002C for ; Tue, 22 Dec 2020 15:02:30 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 32B9F180AD815 for ; Tue, 22 Dec 2020 20:02:30 +0000 (UTC) X-FDA: 77621990460.25.moon26_1305deb27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 0EECF1804E3A8 for ; Tue, 22 Dec 2020 20:02:30 +0000 (UTC) X-HE-Tag: moon26_1305deb27462 X-Filterd-Recvd-Size: 6892 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 151C421D81; Tue, 22 Dec 2020 20:02:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667348; bh=+xLkThKDUWSr3xAxWlQfGBf1+rjemeW56IA8qmmEWa0=; h=Date:From:To:Subject:In-Reply-To:From; b=J5yF6wRzZ9ivDavR+kIqKMYilNSIeKPaFo5rLOTQgPLt3lAVsJAF6LchlxJkF6Gy8 nVsI1vmF8+OF2fw7VaQK8T3w6KT1tyGSY9pi8VvKSuf1x6QR3y/vppmwpof2nfzC71 4Iq2YHeyxhOsPNe+zDQksq/2yJqBv5eMRW7+AbCI= Date: Tue, 22 Dec 2020 12:02:27 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 41/60] kselftest/arm64: check GCR_EL1 after context switch Message-ID: <20201222200227.EUa3TTzcm%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000017, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincenzo Frascino Subject: kselftest/arm64: check GCR_EL1 after context switch This test is specific to MTE and verifies that the GCR_EL1 register is context switched correctly. It spawns 1024 processes and each process spawns 5 threads. Each thread writes a random setting of GCR_EL1 through the prctl() system call and reads it back verifying that it is the same. If the values are not the same it reports a failure. Note: The test has been extended to verify that even SYNC and ASYNC mode setting is preserved correctly over context switching. Link: https://lkml.kernel.org/r/b51a165426e906e7ec8a68d806ef3f8cd92581a6.1606161801.git.andreyknvl@google.com Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Acked-by: Catalin Marinas Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- tools/testing/selftests/arm64/mte/Makefile | 2 tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c | 155 ++++++++++ 2 files changed, 156 insertions(+), 1 deletion(-) --- /dev/null +++ a/tools/testing/selftests/arm64/mte/check_gcr_el1_cswitch.c @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (C) 2020 ARM Limited + +#define _GNU_SOURCE + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kselftest.h" +#include "mte_common_util.h" + +#define PR_SET_TAGGED_ADDR_CTRL 55 +#define PR_GET_TAGGED_ADDR_CTRL 56 +# define PR_TAGGED_ADDR_ENABLE (1UL << 0) +# define PR_MTE_TCF_SHIFT 1 +# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TAG_SHIFT 3 +# define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT) + +#include "mte_def.h" + +#define NUM_ITERATIONS 1024 +#define MAX_THREADS 5 +#define THREAD_ITERATIONS 1000 + +void *execute_thread(void *x) +{ + pid_t pid = *((pid_t *)x); + pid_t tid = gettid(); + uint64_t prctl_tag_mask; + uint64_t prctl_set; + uint64_t prctl_get; + uint64_t prctl_tcf; + + srand(time(NULL) ^ (pid << 16) ^ (tid << 16)); + + prctl_tag_mask = rand() & 0xffff; + + if (prctl_tag_mask % 2) + prctl_tcf = PR_MTE_TCF_SYNC; + else + prctl_tcf = PR_MTE_TCF_ASYNC; + + prctl_set = PR_TAGGED_ADDR_ENABLE | prctl_tcf | (prctl_tag_mask << PR_MTE_TAG_SHIFT); + + for (int j = 0; j < THREAD_ITERATIONS; j++) { + if (prctl(PR_SET_TAGGED_ADDR_CTRL, prctl_set, 0, 0, 0)) { + perror("prctl() failed"); + goto fail; + } + + prctl_get = prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0); + + if (prctl_set != prctl_get) { + ksft_print_msg("Error: prctl_set: 0x%lx != prctl_get: 0x%lx\n", + prctl_set, prctl_get); + goto fail; + } + } + + return (void *)KSFT_PASS; + +fail: + return (void *)KSFT_FAIL; +} + +int execute_test(pid_t pid) +{ + pthread_t thread_id[MAX_THREADS]; + int thread_data[MAX_THREADS]; + + for (int i = 0; i < MAX_THREADS; i++) + pthread_create(&thread_id[i], NULL, + execute_thread, (void *)&pid); + + for (int i = 0; i < MAX_THREADS; i++) + pthread_join(thread_id[i], (void *)&thread_data[i]); + + for (int i = 0; i < MAX_THREADS; i++) + if (thread_data[i] == KSFT_FAIL) + return KSFT_FAIL; + + return KSFT_PASS; +} + +int mte_gcr_fork_test(void) +{ + pid_t pid; + int results[NUM_ITERATIONS]; + pid_t cpid; + int res; + + for (int i = 0; i < NUM_ITERATIONS; i++) { + pid = fork(); + + if (pid < 0) + return KSFT_FAIL; + + if (pid == 0) { + cpid = getpid(); + + res = execute_test(cpid); + + exit(res); + } + } + + for (int i = 0; i < NUM_ITERATIONS; i++) { + wait(&res); + + if (WIFEXITED(res)) + results[i] = WEXITSTATUS(res); + else + --i; + } + + for (int i = 0; i < NUM_ITERATIONS; i++) + if (results[i] == KSFT_FAIL) + return KSFT_FAIL; + + return KSFT_PASS; +} + +int main(int argc, char *argv[]) +{ + int err; + + err = mte_default_setup(); + if (err) + return err; + + ksft_set_plan(1); + + evaluate_test(mte_gcr_fork_test(), + "Verify that GCR_EL1 is set correctly on context switch\n"); + + mte_restore_setup(); + ksft_print_cnts(); + + return ksft_get_fail_cnt() == 0 ? KSFT_PASS : KSFT_FAIL; +} + --- a/tools/testing/selftests/arm64/mte/Makefile~kselftest-arm64-check-gcr_el1-after-context-switch +++ a/tools/testing/selftests/arm64/mte/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (C) 2020 ARM Limited -CFLAGS += -std=gnu99 -I. +CFLAGS += -std=gnu99 -I. -lpthread SRCS := $(filter-out mte_common_util.c,$(wildcard *.c)) PROGS := $(patsubst %.c,%,$(SRCS)) From patchwork Tue Dec 22 20:02:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72D77C433DB for ; Tue, 22 Dec 2020 20:03:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE61A2313F for ; Tue, 22 Dec 2020 20:03:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE61A2313F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3C568D0030; Tue, 22 Dec 2020 15:02:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A148C8D002F; Tue, 22 Dec 2020 15:02:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83FA68D0030; Tue, 22 Dec 2020 15:02:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0115.hostedemail.com [216.40.44.115]) by kanga.kvack.org (Postfix) with ESMTP id 6B8B78D002F for ; Tue, 22 Dec 2020 15:02:34 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2D1C1363D for ; Tue, 22 Dec 2020 20:02:34 +0000 (UTC) X-FDA: 77621990628.22.food04_5f17dbe27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id DE58718038E68 for ; Tue, 22 Dec 2020 20:02:33 +0000 (UTC) X-HE-Tag: food04_5f17dbe27462 X-Filterd-Recvd-Size: 8435 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AF16422583; Tue, 22 Dec 2020 20:02:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667352; bh=Qa/lJeHYl8b4KzgVDEcPnrNSL3TndyoXcPsKu0X9a9U=; h=Date:From:To:Subject:In-Reply-To:From; b=mUWhNkUyiKWY5tfl1pcxafrTyckzyFBQayNDhF//ValEyZASIoZvF8w4QOZmxKR7k BX6CIZWw/1tZjDGzdfl7pWoeOCHY/LAdizck28a+3oGbtpAyXpXTvoySIL3BEYIf3H F7ceG/DLgYkz5e4KX9B62O44mSpiR3arw1mtfBqE= Date: Tue, 22 Dec 2020 12:02:31 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 42/60] kasan: simplify quarantine_put call site Message-ID: <20201222200231.4cOQTwTiP%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: simplify quarantine_put call site Patch series "kasan: boot parameters for hardware tag-based mode", v4. === Overview Hardware tag-based KASAN mode [1] is intended to eventually be used in production as a security mitigation. Therefore there's a need for finer control over KASAN features and for an existence of a kill switch. This patchset adds a few boot parameters for hardware tag-based KASAN that allow to disable or otherwise control particular KASAN features, as well as provides some initial optimizations for running KASAN in production. There's another planned patchset what will further optimize hardware tag-based KASAN, provide proper benchmarking and tests, and will fully enable tag-based KASAN for production use. Hardware tag-based KASAN relies on arm64 Memory Tagging Extension (MTE) [2] to perform memory and pointer tagging. Please see [3] and [4] for detailed analysis of how MTE helps to fight memory safety problems. The features that can be controlled are: 1. Whether KASAN is enabled at all. 2. Whether KASAN collects and saves alloc/free stacks. 3. Whether KASAN panics on a detected bug or not. The patch titled "kasan: add and integrate kasan boot parameters" of this series adds a few new boot parameters. kasan.mode allows to choose one of three main modes: - kasan.mode=off - KASAN is disabled, no tag checks are performed - kasan.mode=prod - only essential production features are enabled - kasan.mode=full - all KASAN features are enabled The chosen mode provides default control values for the features mentioned above. However it's also possible to override the default values by providing: - kasan.stacktrace=off/on - enable stacks collection (default: on for mode=full, otherwise off) - kasan.fault=report/panic - only report tag fault or also panic (default: report) If kasan.mode parameter is not provided, it defaults to full when CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise. It is essential that switching between these modes doesn't require rebuilding the kernel with different configs, as this is required by the Android GKI (Generic Kernel Image) initiative. === Benchmarks For now I've only performed a few simple benchmarks such as measuring kernel boot time and slab memory usage after boot. There's an upcoming patchset which will optimize KASAN further and include more detailed benchmarking results. The benchmarks were performed in QEMU and the results below exclude the slowdown caused by QEMU memory tagging emulation (as it's different from the slowdown that will be introduced by hardware and is therefore irrelevant). KASAN_HW_TAGS=y + kasan.mode=off introduces no performance or memory impact compared to KASAN_HW_TAGS=n. kasan.mode=prod (manually excluding tagging) introduces 3% of performance and no memory impact (except memory used by hardware to store tags) compared to kasan.mode=off. kasan.mode=full has about 40% performance and 30% memory impact over kasan.mode=prod. Both come from alloc/free stack collection. === Notes This patchset is available here: https://github.com/xairy/linux/tree/up-boot-mte-v4 This patchset is based on v11 of "kasan: add hardware tag-based mode for arm64" patchset [1]. For testing in QEMU hardware tag-based KASAN requires: 1. QEMU built from master [6] (use "-machine virt,mte=on -cpu max" arguments to run). 2. GCC version 10. [1] https://lore.kernel.org/linux-arm-kernel/cover.1606161801.git.andreyknvl@google.com/T/#t [2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety [3] https://arxiv.org/pdf/1802.09517.pdf [4] https://github.com/microsoft/MSRC-Security-Research/blob/master/papers/2020/Security%20analysis%20of%20memory%20tagging.pdf [5] https://source.android.com/devices/architecture/kernel/generic-kernel-image [6] https://github.com/qemu/qemu === Tags Tested-by: Vincenzo Frascino This patch (of 19): Move get_free_info() call into quarantine_put() to simplify the call site. No functional changes. Link: https://lkml.kernel.org/r/cover.1606162397.git.andreyknvl@google.com Link: https://lkml.kernel.org/r/312d0a3ef92cc6dc4fa5452cbc1714f9393ca239.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Iab0f04e7ebf8d83247024b7190c67c3c34c7940f Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Catalin Marinas Cc: Will Deacon Cc: Andrey Ryabinin Cc: Alexander Potapenko Cc: Evgenii Stepanov Cc: Branislav Rankov Cc: Kevin Brodsky Cc: Vasily Gorbik Signed-off-by: Andrew Morton --- mm/kasan/common.c | 2 +- mm/kasan/kasan.h | 5 ++--- mm/kasan/quarantine.c | 3 ++- 3 files changed, 5 insertions(+), 5 deletions(-) --- a/mm/kasan/common.c~kasan-simplify-quarantine_put-call-site +++ a/mm/kasan/common.c @@ -313,7 +313,7 @@ static bool __kasan_slab_free(struct kme kasan_set_free_info(cache, object, tag); - quarantine_put(get_free_info(cache, object), cache); + quarantine_put(cache, object); return IS_ENABLED(CONFIG_KASAN_GENERIC); } --- a/mm/kasan/kasan.h~kasan-simplify-quarantine_put-call-site +++ a/mm/kasan/kasan.h @@ -216,12 +216,11 @@ struct kasan_track *kasan_get_free_track #if defined(CONFIG_KASAN_GENERIC) && \ (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); +void quarantine_put(struct kmem_cache *cache, void *object); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); #else -static inline void quarantine_put(struct kasan_free_meta *info, - struct kmem_cache *cache) { } +static inline void quarantine_put(struct kmem_cache *cache, void *object) { } static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif --- a/mm/kasan/quarantine.c~kasan-simplify-quarantine_put-call-site +++ a/mm/kasan/quarantine.c @@ -163,11 +163,12 @@ static void qlist_free_all(struct qlist_ qlist_init(q); } -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache) +void quarantine_put(struct kmem_cache *cache, void *object) { unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; + struct kasan_free_meta *info = get_free_info(cache, object); /* * Note: irq must be disabled until after we move the batch to the From patchwork Tue Dec 22 20:02:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B583C433E0 for ; Tue, 22 Dec 2020 20:03:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 925E0229C5 for ; Tue, 22 Dec 2020 20:03:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 925E0229C5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EF7EC8D0032; Tue, 22 Dec 2020 15:02:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ECF098D0031; Tue, 22 Dec 2020 15:02:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D47DA8D0032; Tue, 22 Dec 2020 15:02:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id BEE8D8D0031 for ; Tue, 22 Dec 2020 15:02:37 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 870031F06 for ; Tue, 22 Dec 2020 20:02:37 +0000 (UTC) X-FDA: 77621990754.16.roll23_340db7627462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 60389100E6903 for ; Tue, 22 Dec 2020 20:02:37 +0000 (UTC) X-HE-Tag: roll23_340db7627462 X-Filterd-Recvd-Size: 10648 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 47F6721D1B; Tue, 22 Dec 2020 20:02:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667355; bh=rguwOdBpFIiStR5CoWFIuhHyeFE/fvoDsI3SLYmES1Q=; h=Date:From:To:Subject:In-Reply-To:From; b=TcA+9sRkbrS7BdPp1fzTDNXQZ0FayCoxZXg9mv09kaGlDE3mj7sVEQo6peC980rUR 5HNCZUeXio9dh/A/v+VYApCuyJVU1Y56pKOlHieOq/WEVMAyuWadMKvF0vjQ8j+DAq 9h9zOgBtJKrZgJ5YeO7Gws7uEAQL19tENcypbYw8= Date: Tue, 22 Dec 2020 12:02:34 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 43/60] kasan: rename get_alloc/free_info Message-ID: <20201222200234.Nu5AUtsvM%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename get_alloc/free_info Rename get_alloc_info() and get_free_info() to kasan_get_alloc_meta() and kasan_get_free_meta() to better reflect what those do and avoid confusion with kasan_set_free_info(). No functional changes. Link: https://lkml.kernel.org/r/27b7c036b754af15a2839e945f6d8bfce32b4c2f.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ib6e4ba61c8b12112b403d3479a9799ac8fff8de1 Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 16 ++++++++-------- mm/kasan/generic.c | 12 ++++++------ mm/kasan/hw_tags.c | 4 ++-- mm/kasan/kasan.h | 8 ++++---- mm/kasan/quarantine.c | 4 ++-- mm/kasan/report.c | 12 ++++++------ mm/kasan/report_sw_tags.c | 2 +- mm/kasan/sw_tags.c | 4 ++-- 8 files changed, 31 insertions(+), 31 deletions(-) --- a/mm/kasan/common.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/common.c @@ -180,14 +180,14 @@ size_t kasan_metadata_size(struct kmem_c sizeof(struct kasan_free_meta) : 0); } -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object) { return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; } -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; @@ -264,13 +264,13 @@ static u8 assign_tag(struct kmem_cache * void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; if (!(cache->flags & SLAB_KASAN)) return (void *)object; - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -350,7 +350,7 @@ static void *__kasan_kmalloc(struct kmem KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags); + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); return set_tag(object, tag); } --- a/mm/kasan/generic.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/generic.c @@ -328,7 +328,7 @@ void kasan_record_aux_stack(void *addr) { struct page *page = kasan_addr_to_page(addr); struct kmem_cache *cache; - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; void *object; if (!(page && PageSlab(page))) @@ -336,10 +336,10 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); - alloc_info = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); - alloc_info->aux_stack[1] = alloc_info->aux_stack[0]; - alloc_info->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); + alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; + alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } void kasan_set_free_info(struct kmem_cache *cache, @@ -347,7 +347,7 @@ void kasan_set_free_info(struct kmem_cac { struct kasan_free_meta *free_meta; - free_meta = get_free_info(cache, object); + free_meta = kasan_get_free_meta(cache, object); kasan_set_track(&free_meta->free_track, GFP_NOWAIT); /* @@ -361,5 +361,5 @@ struct kasan_track *kasan_get_free_track { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; - return &get_free_info(cache, object)->free_track; + return &kasan_get_free_meta(cache, object)->free_track; } --- a/mm/kasan/hw_tags.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/hw_tags.c @@ -66,7 +66,7 @@ void kasan_set_free_info(struct kmem_cac { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } @@ -75,6 +75,6 @@ struct kasan_track *kasan_get_free_track { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); return &alloc_meta->free_track[0]; } --- a/mm/kasan/kasan.h~kasan-rename-get_alloc-free_info +++ a/mm/kasan/kasan.h @@ -149,10 +149,10 @@ struct kasan_free_meta { #endif }; -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object); -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object); +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object); +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object); void poison_range(const void *address, size_t size, u8 value); void unpoison_range(const void *address, size_t size); --- a/mm/kasan/quarantine.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/quarantine.c @@ -168,7 +168,7 @@ void quarantine_put(struct kmem_cache *c unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; - struct kasan_free_meta *info = get_free_info(cache, object); + struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); /* * Note: irq must be disabled until after we move the batch to the @@ -185,7 +185,7 @@ void quarantine_put(struct kmem_cache *c local_irq_restore(flags); return; } - qlist_put(q, &info->quarantine_link, cache->size); + qlist_put(q, &meta->quarantine_link, cache->size); if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) { qlist_move_all(q, &temp); --- a/mm/kasan/report.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/report.c @@ -164,12 +164,12 @@ static void describe_object_addr(struct static void describe_object(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object); + struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); if (cache->flags & SLAB_KASAN) { struct kasan_track *free_track; - print_track(&alloc_info->alloc_track, "Allocated"); + print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); free_track = kasan_get_free_track(cache, object, tag); if (free_track) { @@ -178,14 +178,14 @@ static void describe_object(struct kmem_ } #ifdef CONFIG_KASAN_GENERIC - if (alloc_info->aux_stack[0]) { + if (alloc_meta->aux_stack[0]) { pr_err("Last potentially related work creation:\n"); - print_stack(alloc_info->aux_stack[0]); + print_stack(alloc_meta->aux_stack[0]); pr_err("\n"); } - if (alloc_info->aux_stack[1]) { + if (alloc_meta->aux_stack[1]) { pr_err("Second to last potentially related work creation:\n"); - print_stack(alloc_info->aux_stack[1]); + print_stack(alloc_meta->aux_stack[1]); pr_err("\n"); } #endif --- a/mm/kasan/report_sw_tags.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/report_sw_tags.c @@ -46,7 +46,7 @@ const char *get_bug_type(struct kasan_ac if (page && PageSlab(page)) { cache = page->slab_cache; object = nearest_obj(cache, page, (void *)addr); - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); for (i = 0; i < KASAN_NR_FREE_STACKS; i++) if (alloc_meta->free_pointer_tag[i] == tag) --- a/mm/kasan/sw_tags.c~kasan-rename-get_alloc-free_info +++ a/mm/kasan/sw_tags.c @@ -174,7 +174,7 @@ void kasan_set_free_info(struct kmem_cac struct kasan_alloc_meta *alloc_meta; u8 idx = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -191,7 +191,7 @@ struct kasan_track *kasan_get_free_track struct kasan_alloc_meta *alloc_meta; int i = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Tue Dec 22 20:02:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F7DFC433DB for ; Tue, 22 Dec 2020 20:03:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 208B623139 for ; Tue, 22 Dec 2020 20:03:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 208B623139 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 66B528D0034; Tue, 22 Dec 2020 15:02:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 642888D0033; Tue, 22 Dec 2020 15:02:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 536788D0034; Tue, 22 Dec 2020 15:02:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 36B778D0033 for ; Tue, 22 Dec 2020 15:02:41 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id EDF4B181AEF32 for ; Tue, 22 Dec 2020 20:02:40 +0000 (UTC) X-FDA: 77621990880.04.wire18_2f0fa5727462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id D458D800E098 for ; Tue, 22 Dec 2020 20:02:40 +0000 (UTC) X-HE-Tag: wire18_2f0fa5727462 X-Filterd-Recvd-Size: 3343 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D087B2313C; Tue, 22 Dec 2020 20:02:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667359; bh=t0cd2hZ+0IGmC6YZOXAl7iIEiMiy2IvhFrjitqSK3sE=; h=Date:From:To:Subject:In-Reply-To:From; b=ScF+l/yplvCw8J6GYIozS3M10rS9uhPqH+MkBnZOJXbdpUrAo41PW58wqmYAYal9k NVPh5N+TuMNNneV70sSI8aReCnZNytoQrBGzwPdBHOntuzLCV4RwSrbBv81gFaAnJC BK1cQxDGV0YCK/EGVnWQJe77ng6rB/1gJ1dfnDYs= Date: Tue, 22 Dec 2020 12:02:38 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 44/60] kasan: introduce set_alloc_info Message-ID: <20201222200238.37xoLOd-R%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: introduce set_alloc_info Add set_alloc_info() helper and move kasan_set_track() into it. This will simplify the code for one of the upcoming changes. No functional changes. Link: https://lkml.kernel.org/r/b2393e8f1e311a70fc3aaa2196461b6acdee7d21.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I0316193cbb4ecc9b87b7c2eee0dd79f8ec908c1a Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) --- a/mm/kasan/common.c~kasan-introduce-set_alloc_info +++ a/mm/kasan/common.c @@ -323,6 +323,11 @@ bool kasan_slab_free(struct kmem_cache * return __kasan_slab_free(cache, object, ip, true); } +static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) +{ + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); +} + static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { @@ -350,7 +355,7 @@ static void *__kasan_kmalloc(struct kmem KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); } From patchwork Tue Dec 22 20:02:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22009C433E6 for ; Tue, 22 Dec 2020 20:03:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0D6920771 for ; Tue, 22 Dec 2020 20:03:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0D6920771 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 273B58D0036; Tue, 22 Dec 2020 15:02:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2010B8D0035; Tue, 22 Dec 2020 15:02:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 075588D0036; Tue, 22 Dec 2020 15:02:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id E1A398D0035 for ; Tue, 22 Dec 2020 15:02:44 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id AD2F93638 for ; Tue, 22 Dec 2020 20:02:44 +0000 (UTC) X-FDA: 77621991048.02.bag92_5a0051527462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 93DC310097AA0 for ; Tue, 22 Dec 2020 20:02:44 +0000 (UTC) X-HE-Tag: bag92_5a0051527462 X-Filterd-Recvd-Size: 5658 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:43 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7A4CB23133; Tue, 22 Dec 2020 20:02:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667363; bh=WNKVRpkZq69N5UEbck7KrA2+8iIlHIEGJXiayP/E030=; h=Date:From:To:Subject:In-Reply-To:From; b=ZDVJlZjsw1Gdr45ZH6YWl3IMS79zWl55jWI6EmoOZvb53VzdC7UysN0FlKq2x2ZMR /eQJexKdxsPpG2H5DAd/B96v8D2FnO+Fkj+55hej3WoWtKzGMCdqVAflJ923cv+vl8 U3HqIS22KrGEgbXNdlsgR0ejWfHgFqxCUEv1CgcM= Date: Tue, 22 Dec 2020 12:02:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 45/60] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK Message-ID: <20201222200242.WjED4h0iJ%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK There's a config option CONFIG_KASAN_STACK that has to be enabled for KASAN to use stack instrumentation and perform validity checks for stack variables. There's no need to unpoison stack when CONFIG_KASAN_STACK is not enabled. Only call kasan_unpoison_task_stack[_below]() when CONFIG_KASAN_STACK is enabled. Note, that CONFIG_KASAN_STACK is an option that is currently always defined when CONFIG_KASAN is enabled, and therefore has to be tested with #if instead of #ifdef. Link: https://lkml.kernel.org/r/d09dd3f8abb388da397fd11598c5edeaa83fe559.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/If8a891e9fe01ea543e00b576852685afec0887e3 Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Acked-by: Catalin Marinas Reviewed-by: Dmitry Vyukov Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/kernel/sleep.S | 2 +- arch/x86/kernel/acpi/wakeup_64.S | 2 +- include/linux/kasan.h | 10 ++++++---- mm/kasan/common.c | 2 ++ 4 files changed, 10 insertions(+), 6 deletions(-) --- a/arch/arm64/kernel/sleep.S~kasan-arm64-unpoison-stack-only-with-config_kasan_stack +++ a/arch/arm64/kernel/sleep.S @@ -133,7 +133,7 @@ SYM_FUNC_START(_cpu_resume) */ bl cpu_do_resume -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK mov x0, sp bl kasan_unpoison_task_stack_below #endif --- a/arch/x86/kernel/acpi/wakeup_64.S~kasan-arm64-unpoison-stack-only-with-config_kasan_stack +++ a/arch/x86/kernel/acpi/wakeup_64.S @@ -112,7 +112,7 @@ SYM_FUNC_START(do_suspend_lowlevel) movq pt_regs_r14(%rax), %r14 movq pt_regs_r15(%rax), %r15 -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK /* * The suspend path may have poisoned some areas deeper in the stack, * which we now need to unpoison. --- a/include/linux/kasan.h~kasan-arm64-unpoison-stack-only-with-config_kasan_stack +++ a/include/linux/kasan.h @@ -77,8 +77,6 @@ static inline void kasan_disable_current void kasan_unpoison_range(const void *address, size_t size); -void kasan_unpoison_task_stack(struct task_struct *task); - void kasan_alloc_pages(struct page *page, unsigned int order); void kasan_free_pages(struct page *page, unsigned int order); @@ -123,8 +121,6 @@ void kasan_restore_multi_shot(bool enabl static inline void kasan_unpoison_range(const void *address, size_t size) {} -static inline void kasan_unpoison_task_stack(struct task_struct *task) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} @@ -176,6 +172,12 @@ static inline size_t kasan_metadata_size #endif /* CONFIG_KASAN */ +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +void kasan_unpoison_task_stack(struct task_struct *task); +#else +static inline void kasan_unpoison_task_stack(struct task_struct *task) {} +#endif + #ifdef CONFIG_KASAN_GENERIC void kasan_cache_shrink(struct kmem_cache *cache); --- a/mm/kasan/common.c~kasan-arm64-unpoison-stack-only-with-config_kasan_stack +++ a/mm/kasan/common.c @@ -63,6 +63,7 @@ void kasan_unpoison_range(const void *ad unpoison_range(address, size); } +#if CONFIG_KASAN_STACK static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) { void *base = task_stack_page(task); @@ -89,6 +90,7 @@ asmlinkage void kasan_unpoison_task_stac unpoison_range(base, watermark - base); } +#endif /* CONFIG_KASAN_STACK */ void kasan_alloc_pages(struct page *page, unsigned int order) { From patchwork Tue Dec 22 20:02:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3A5BC433E6 for ; Tue, 22 Dec 2020 20:03:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6CA3320867 for ; Tue, 22 Dec 2020 20:03:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6CA3320867 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D21AC8D0037; Tue, 22 Dec 2020 15:02:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD1DF8D0035; Tue, 22 Dec 2020 15:02:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B733D8D0037; Tue, 22 Dec 2020 15:02:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0043.hostedemail.com [216.40.44.43]) by kanga.kvack.org (Postfix) with ESMTP id 9E2928D0035 for ; Tue, 22 Dec 2020 15:02:48 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 637BD3638 for ; Tue, 22 Dec 2020 20:02:48 +0000 (UTC) X-FDA: 77621991216.25.run18_2f02df427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 360771804E3A8 for ; Tue, 22 Dec 2020 20:02:48 +0000 (UTC) X-HE-Tag: run18_2f02df427462 X-Filterd-Recvd-Size: 3508 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:47 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2224220700; Tue, 22 Dec 2020 20:02:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667366; bh=rn5Neti983/U06+GwPVruLgh6wBz38/ROfpXbbmR6Qk=; h=Date:From:To:Subject:In-Reply-To:From; b=y4bLIbM8OatYgZuYkvWE2Xn3m3hJspUTWrfDCm+sIWtzCQsBg137yfQipDsst/EwX 0Ct+/p49oe34TZTCxUosVdjhsV8dsedK0LtiT+/fJcFlTsRtr59hPYO+4jaKkOIkVU SmS+5/gFuO7Y0LC7AUCATSjuX3+1qIc+9q4ku98g= Date: Tue, 22 Dec 2020 12:02:45 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 46/60] kasan: allow VMAP_STACK for HW_TAGS mode Message-ID: <20201222200245.lfDSG5aGm%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: allow VMAP_STACK for HW_TAGS mode Even though hardware tag-based mode currently doesn't support checking vmalloc allocations, it doesn't use shadow memory and works with VMAP_STACK as is. Change VMAP_STACK definition accordingly. Link: https://lkml.kernel.org/r/ecdb2a1658ebd88eb276dee2493518ac0e82de41.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I3552cbc12321dec82cd7372676e9372a2eb452ac Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Acked-by: Catalin Marinas Reviewed-by: Dmitry Vyukov Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/Kconfig | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/arch/Kconfig~kasan-allow-vmap_stack-for-hw_tags-mode +++ a/arch/Kconfig @@ -976,16 +976,16 @@ config VMAP_STACK default y bool "Use a virtually-mapped stack" depends on HAVE_ARCH_VMAP_STACK - depends on !KASAN || KASAN_VMALLOC + depends on !KASAN || KASAN_HW_TAGS || KASAN_VMALLOC help Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately rather than causing difficult-to-diagnose corruption. - To use this with KASAN, the architecture must support backing - virtual mappings with real shadow memory, and KASAN_VMALLOC must - be enabled. + To use this with software KASAN modes, the architecture must support + backing virtual mappings with real shadow memory, and KASAN_VMALLOC + must be enabled. config ARCH_OPTIONAL_KERNEL_RWX def_bool n From patchwork Tue Dec 22 20:02:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF4BDC433E6 for ; Tue, 22 Dec 2020 20:03:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 544C520700 for ; Tue, 22 Dec 2020 20:03:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 544C520700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3FE5F8D003B; Tue, 22 Dec 2020 15:02:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 35B328D0038; Tue, 22 Dec 2020 15:02:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 271FB8D003B; Tue, 22 Dec 2020 15:02:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id 091A08D0038 for ; Tue, 22 Dec 2020 15:02:52 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C5903180AD807 for ; Tue, 22 Dec 2020 20:02:51 +0000 (UTC) X-FDA: 77621991342.11.stick49_1102e7427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 92586180F8B87 for ; Tue, 22 Dec 2020 20:02:51 +0000 (UTC) X-HE-Tag: stick49_1102e7427462 X-Filterd-Recvd-Size: 3291 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A63DF20771; Tue, 22 Dec 2020 20:02:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667370; bh=qTjQu+aAiKh3WeHDrcy3Ad4bAKWypLWyNRllSXK4W20=; h=Date:From:To:Subject:In-Reply-To:From; b=vV789QWAgVBT5A0euD4P8/nHuuqivsdYaCOmWjIBCj565zpMHb8JCcRG2m9aCHQTm KFrZLWCKwYs3GDy5OY23Gg2RXn/O/lvn8M8kOsidgjyBF0xGoXjQaASaKUnirZuxd+ jiTlLF7PaB0/xnk1QnRnprE1eDjppk3eK+VH977Y= Date: Tue, 22 Dec 2020 12:02:49 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 47/60] kasan: remove __kasan_unpoison_stack Message-ID: <20201222200249.pSPi936nP%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: remove __kasan_unpoison_stack There's no need for __kasan_unpoison_stack() helper, as it's only currently used in a single place. Removing it also removes unneeded arithmetic. No functional changes. Link: https://lkml.kernel.org/r/93e78948704a42ea92f6248ff8a725613d721161.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ie5ba549d445292fe629b4a96735e4034957bcc50 Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) --- a/mm/kasan/common.c~kasan-remove-__kasan_unpoison_stack +++ a/mm/kasan/common.c @@ -64,18 +64,12 @@ void kasan_unpoison_range(const void *ad } #if CONFIG_KASAN_STACK -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - unpoison_range(base, size); -} - /* Unpoison the entire stack for a task. */ void kasan_unpoison_task_stack(struct task_struct *task) { - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); + void *base = task_stack_page(task); + + unpoison_range(base, THREAD_SIZE); } /* Unpoison the stack for the current task beyond a watermark sp value. */ From patchwork Tue Dec 22 20:02:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80E5EC433DB for ; Tue, 22 Dec 2020 20:03:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 227FB20700 for ; Tue, 22 Dec 2020 20:03:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 227FB20700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DE6B38D0041; Tue, 22 Dec 2020 15:02:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DA4188D0040; Tue, 22 Dec 2020 15:02:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF3FC8D0041; Tue, 22 Dec 2020 15:02:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id A603F8D0040 for ; Tue, 22 Dec 2020 15:02:55 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 62A5A1EE6 for ; Tue, 22 Dec 2020 20:02:55 +0000 (UTC) X-FDA: 77621991510.15.geese35_0c1147427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 1E8411814B0CC for ; Tue, 22 Dec 2020 20:02:55 +0000 (UTC) X-HE-Tag: geese35_0c1147427462 X-Filterd-Recvd-Size: 9542 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1FEB820795; Tue, 22 Dec 2020 20:02:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667373; bh=1JdPjsnJ0/YV2Ga7nrDr6NvzRTzyVicg3a5t4tWvz6g=; h=Date:From:To:Subject:In-Reply-To:From; b=ZYjatn4hWZU1epW4anEtCTT5HXyTFB0Uyk2W80K5QQreD8NzgmWwXBjYGque+C4q8 R8A6deHmHKn1V+0sAFBxlRbZTutF++giunhUDBpv5glu7XjgUR90qfY6urBGOTHs4t CLgcFf7esDQpTPNwJOEZlj7xwpVHSvbY46ee9Rw8= Date: Tue, 22 Dec 2020 12:02:52 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 48/60] kasan: inline kasan_reset_tag for tag-based modes Message-ID: <20201222200252.cNQz1sGuW%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: inline kasan_reset_tag for tag-based modes Using kasan_reset_tag() currently results in a function call. As it's called quite often from the allocator code, this leads to a noticeable slowdown. Move it to include/linux/kasan.h and turn it into a static inline function. Also remove the now unneeded reset_tag() internal KASAN macro and use kasan_reset_tag() instead. Link: https://lkml.kernel.org/r/6940383a3a9dfb416134d338d8fac97a9ebb8686.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I4d2061acfe91d480a75df00b07c22d8494ef14b5 Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 5 ++++- mm/kasan/common.c | 6 +++--- mm/kasan/hw_tags.c | 9 ++------- mm/kasan/kasan.h | 4 ---- mm/kasan/report.c | 4 ++-- mm/kasan/report_hw_tags.c | 2 +- mm/kasan/report_sw_tags.c | 4 ++-- mm/kasan/shadow.c | 4 ++-- mm/kasan/sw_tags.c | 9 ++------- 9 files changed, 18 insertions(+), 29 deletions(-) --- a/include/linux/kasan.h~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/include/linux/kasan.h @@ -194,7 +194,10 @@ static inline void kasan_record_aux_stac #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -void *kasan_reset_tag(const void *addr); +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)arch_kasan_reset_tag(addr); +} bool kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); --- a/mm/kasan/common.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/common.c @@ -179,14 +179,14 @@ size_t kasan_metadata_size(struct kmem_c struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { - return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } void kasan_poison_slab(struct page *page) @@ -283,7 +283,7 @@ static bool __kasan_slab_free(struct kme tag = get_tag(object); tagged_object = object; - object = reset_tag(object); + object = kasan_reset_tag(object); if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { --- a/mm/kasan/hw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/hw_tags.c @@ -30,20 +30,15 @@ void __init kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - void poison_range(const void *address, size_t size, u8 value) { - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), value); } void unpoison_range(const void *address, size_t size) { - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } --- a/mm/kasan/kasan.h~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/kasan.h @@ -248,15 +248,11 @@ static inline const void *arch_kasan_set return addr; } #endif -#ifndef arch_kasan_reset_tag -#define arch_kasan_reset_tag(addr) ((void *)(addr)) -#endif #ifndef arch_kasan_get_tag #define arch_kasan_get_tag(addr) 0 #endif #define set_tag(addr, tag) ((void *)arch_kasan_set_tag((addr), (tag))) -#define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) #define get_tag(addr) arch_kasan_get_tag(addr) #ifdef CONFIG_KASAN_HW_TAGS --- a/mm/kasan/report.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/report.c @@ -328,7 +328,7 @@ void kasan_report_invalid_free(void *obj unsigned long flags; u8 tag = get_tag(object); - object = reset_tag(object); + object = kasan_reset_tag(object); #if IS_ENABLED(CONFIG_KUNIT) if (current->kunit_test) @@ -361,7 +361,7 @@ static void __kasan_report(unsigned long disable_trace_on_warning(); tagged_addr = (void *)addr; - untagged_addr = reset_tag(tagged_addr); + untagged_addr = kasan_reset_tag(tagged_addr); info.access_addr = tagged_addr; if (addr_has_metadata(untagged_addr)) --- a/mm/kasan/report_hw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/report_hw_tags.c @@ -22,7 +22,7 @@ const char *get_bug_type(struct kasan_ac void *find_first_bad_addr(void *addr, size_t size) { - return reset_tag(addr); + return kasan_reset_tag(addr); } void metadata_fetch_row(char *buffer, void *row) --- a/mm/kasan/report_sw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/report_sw_tags.c @@ -41,7 +41,7 @@ const char *get_bug_type(struct kasan_ac int i; tag = get_tag(info->access_addr); - addr = reset_tag(info->access_addr); + addr = kasan_reset_tag(info->access_addr); page = kasan_addr_to_page(addr); if (page && PageSlab(page)) { cache = page->slab_cache; @@ -72,7 +72,7 @@ const char *get_bug_type(struct kasan_ac void *find_first_bad_addr(void *addr, size_t size) { u8 tag = get_tag(addr); - void *p = reset_tag(addr); + void *p = kasan_reset_tag(addr); void *end = p + size; while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) --- a/mm/kasan/shadow.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/shadow.c @@ -81,7 +81,7 @@ void poison_range(const void *address, s * some of the callers (e.g. kasan_poison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -98,7 +98,7 @@ void unpoison_range(const void *address, * some of the callers (e.g. kasan_unpoison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); poison_range(address, size, tag); --- a/mm/kasan/sw_tags.c~kasan-inline-kasan_reset_tag-for-tag-based-modes +++ a/mm/kasan/sw_tags.c @@ -67,11 +67,6 @@ u8 random_tag(void) return (u8)(state % (KASAN_TAG_MAX + 1)); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - bool check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { @@ -107,7 +102,7 @@ bool check_memory_region(unsigned long a if (tag == KASAN_TAG_KERNEL) return true; - untagged_addr = reset_tag((const void *)addr); + untagged_addr = kasan_reset_tag((const void *)addr); if (unlikely(untagged_addr < kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { return !kasan_report(addr, size, write, ret_ip); @@ -126,7 +121,7 @@ bool check_memory_region(unsigned long a bool check_invalid_free(void *addr) { u8 tag = get_tag(addr); - u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr))); + u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr))); return (shadow_byte == KASAN_TAG_INVALID) || (tag != KASAN_TAG_KERNEL && tag != shadow_byte); From patchwork Tue Dec 22 20:02:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F0CBC433E0 for ; Tue, 22 Dec 2020 20:03:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CF64B20700 for ; Tue, 22 Dec 2020 20:03:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF64B20700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6903E8D0047; Tue, 22 Dec 2020 15:02:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 642CE8D0040; Tue, 22 Dec 2020 15:02:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50A868D0047; Tue, 22 Dec 2020 15:02:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 344BD8D0040 for ; Tue, 22 Dec 2020 15:02:59 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id F40BA181AEF32 for ; Tue, 22 Dec 2020 20:02:58 +0000 (UTC) X-FDA: 77621991678.30.camp52_1e11f7927462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id D1F8A180B3AA7 for ; Tue, 22 Dec 2020 20:02:58 +0000 (UTC) X-HE-Tag: camp52_1e11f7927462 X-Filterd-Recvd-Size: 4581 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:02:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AC1EB20855; Tue, 22 Dec 2020 20:02:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667377; bh=KtQ8zqniFHEwjOHUzzt9kqDNd7huicdLPzTDnHqEmzo=; h=Date:From:To:Subject:In-Reply-To:From; b=wnXTFkBd1Gqn7XG7XVAnYtu3u8Xij+u75Uols6Hc71Y8ww64ZGFc5L1IZ03eQEsF6 xyPp6ev7bYLeJaq9L+k2/rcL66I50HNcc5mTWYOT44O95OUReqn8rB7QUwVidGaWOt xTUiLTUCBWyC6y/Oh6vJSLpLUCKuBbGqdSdSHjC0= Date: Tue, 22 Dec 2020 12:02:56 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 49/60] kasan: inline random_tag for HW_TAGS Message-ID: <20201222200256.8RSW2lfSj%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: inline random_tag for HW_TAGS Using random_tag() currently results in a function call. Move its definition to mm/kasan/kasan.h and turn it into a static inline function for hardware tag-based mode to avoid uneeded function calls. Link: https://lkml.kernel.org/r/be438471690e351e1d792e6bb432e8c03ccb15d3.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Iac5b2faf9a912900e16cca6834d621f5d4abf427 Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/hw_tags.c | 5 ----- mm/kasan/kasan.h | 31 ++++++++++++++----------------- 2 files changed, 14 insertions(+), 22 deletions(-) --- a/mm/kasan/hw_tags.c~kasan-inline-random_tag-for-hw_tags +++ a/mm/kasan/hw_tags.c @@ -42,11 +42,6 @@ void unpoison_range(const void *address, round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -u8 random_tag(void) -{ - return hw_get_random_tag(); -} - bool check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); --- a/mm/kasan/kasan.h~kasan-inline-random_tag-for-hw_tags +++ a/mm/kasan/kasan.h @@ -190,6 +190,12 @@ static inline bool addr_has_metadata(con #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { } +#endif + bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); @@ -225,23 +231,6 @@ static inline void quarantine_reduce(voi static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) - -void print_tags(u8 addr_tag, const void *addr); - -u8 random_tag(void); - -#else - -static inline void print_tags(u8 addr_tag, const void *addr) { } - -static inline u8 random_tag(void) -{ - return 0; -} - -#endif - #ifndef arch_kasan_set_tag static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) { @@ -281,6 +270,14 @@ static inline const void *arch_kasan_set #endif /* CONFIG_KASAN_HW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +u8 random_tag(void); +#elif defined(CONFIG_KASAN_HW_TAGS) +static inline u8 random_tag(void) { return hw_get_random_tag(); } +#else +static inline u8 random_tag(void) { return 0; } +#endif + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Tue Dec 22 20:02:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 618EDC433E6 for ; Tue, 22 Dec 2020 20:04:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0448020700 for ; Tue, 22 Dec 2020 20:04:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0448020700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 272908D0049; Tue, 22 Dec 2020 15:03:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D6B98D0048; Tue, 22 Dec 2020 15:03:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09CC58D0049; Tue, 22 Dec 2020 15:03:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id E52B78D0048 for ; Tue, 22 Dec 2020 15:03:02 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B11861EE6 for ; Tue, 22 Dec 2020 20:03:02 +0000 (UTC) X-FDA: 77621991804.20.wish63_3a04f4c27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 88330180C07AB for ; Tue, 22 Dec 2020 20:03:02 +0000 (UTC) X-HE-Tag: wish63_3a04f4c27462 X-Filterd-Recvd-Size: 3856 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6389920867; Tue, 22 Dec 2020 20:03:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667381; bh=uwEfFpsDitRQNrxaDV+NeBYhlLTFGRyRjUl+0c90tqg=; h=Date:From:To:Subject:In-Reply-To:From; b=ZoAKU+5MVS0baC2DO3XSD6C5VsJn3Es6C5IKZRTu/ZlSjTLFqMzx/uarQCS5yi7ht bRhDtqW8vtjUvXS2dhzC5lQOqvkET5sc+zRQNEjRzJzulGoOjGjmBAeGHhMG5HsXLh RSq8g59KGt26hO7mNQTrsdCtJAD54JGr0rJwv/4U= Date: Tue, 22 Dec 2020 12:02:59 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 50/60] kasan: open-code kasan_unpoison_slab Message-ID: <20201222200259.Y9RejNqMY%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: open-code kasan_unpoison_slab There's the external annotation kasan_unpoison_slab() that is currently defined as static inline and uses kasan_unpoison_range(). Open-code this function in mempool.c. Otherwise with an upcoming change this function will result in an unnecessary function call. Link: https://lkml.kernel.org/r/131a6694a978a9a8b150187e539eecc8bcbf759b.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ia7c8b659f79209935cbaab3913bf7f082cc43a0e Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 6 ------ mm/mempool.c | 2 +- 2 files changed, 1 insertion(+), 7 deletions(-) --- a/include/linux/kasan.h~kasan-open-code-kasan_unpoison_slab +++ a/include/linux/kasan.h @@ -107,11 +107,6 @@ struct kasan_cache { int free_meta_offset; }; -size_t __ksize(const void *); -static inline void kasan_unpoison_slab(const void *ptr) -{ - kasan_unpoison_range(ptr, __ksize(ptr)); -} size_t kasan_metadata_size(struct kmem_cache *cache); bool kasan_save_enable_multi_shot(void); @@ -167,7 +162,6 @@ static inline bool kasan_slab_free(struc return false; } -static inline void kasan_unpoison_slab(const void *ptr) { } static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ --- a/mm/mempool.c~kasan-open-code-kasan_unpoison_slab +++ a/mm/mempool.c @@ -112,7 +112,7 @@ static __always_inline void kasan_poison static void kasan_unpoison_element(mempool_t *pool, void *element) { if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) - kasan_unpoison_slab(element); + kasan_unpoison_range(element, __ksize(element)); else if (pool->alloc == mempool_alloc_pages) kasan_alloc_pages(element, (unsigned long)pool->pool_data); } From patchwork Tue Dec 22 20:03:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3454C433E0 for ; Tue, 22 Dec 2020 20:04:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7382420700 for ; Tue, 22 Dec 2020 20:04:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7382420700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 864198D004A; Tue, 22 Dec 2020 15:03:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8153B8D0048; Tue, 22 Dec 2020 15:03:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 730F68D004A; Tue, 22 Dec 2020 15:03:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id 58F3F8D0048 for ; Tue, 22 Dec 2020 15:03:06 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 23137181AEF32 for ; Tue, 22 Dec 2020 20:03:06 +0000 (UTC) X-FDA: 77621991972.16.step28_071286427462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id F0363100E6903 for ; Tue, 22 Dec 2020 20:03:05 +0000 (UTC) X-HE-Tag: step28_071286427462 X-Filterd-Recvd-Size: 5676 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id F18DA20700; Tue, 22 Dec 2020 20:03:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667384; bh=wPvi+UsUpTxgCzcocHc0vONZ7AR6L8dOcpE8h6uun9o=; h=Date:From:To:Subject:In-Reply-To:From; b=b8FF/zUnen67BY4xiFx/uRWePB4rUwcb8BAH/CjhDdoFB1kxNsHb4hiLN6NbyPdqc +QqByp74CulQt+zBcQt8ZKH8dP4sa75HuT6XS8J/6MbsevkiCn+tKvFsX1QhfZilnT EtNeKyC2QbDpvvMED7dlI6ICiKp1owe0ILZVwVUU= Date: Tue, 22 Dec 2020 12:03:03 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 51/60] kasan: inline (un)poison_range and check_invalid_free Message-ID: <20201222200303.rJO6f-L6E%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: inline (un)poison_range and check_invalid_free Using (un)poison_range() or check_invalid_free() currently results in function calls. Move their definitions to mm/kasan/kasan.h and turn them into static inline functions for hardware tag-based mode to avoid unneeded function calls. Link: https://lkml.kernel.org/r/7007955b69eb31b5376a7dc1e0f4ac49138504f2.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ia9d8191024a12d1374675b3d27197f10193f50bb Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/hw_tags.c | 21 --------------------- mm/kasan/kasan.h | 36 +++++++++++++++++++++++++++++++----- 2 files changed, 31 insertions(+), 26 deletions(-) --- a/mm/kasan/hw_tags.c~kasan-inline-unpoison_range-and-check_invalid_free +++ a/mm/kasan/hw_tags.c @@ -30,27 +30,6 @@ void __init kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void poison_range(const void *address, size_t size, u8 value) -{ - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), value); -} - -void unpoison_range(const void *address, size_t size) -{ - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); -} - -bool check_invalid_free(void *addr) -{ - u8 ptr_tag = get_tag(addr); - u8 mem_tag = hw_get_mem_tag(addr); - - return (mem_tag == KASAN_TAG_INVALID) || - (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); -} - void kasan_set_free_info(struct kmem_cache *cache, void *object, u8 tag) { --- a/mm/kasan/kasan.h~kasan-inline-unpoison_range-and-check_invalid_free +++ a/mm/kasan/kasan.h @@ -154,9 +154,6 @@ struct kasan_alloc_meta *kasan_get_alloc struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); -void poison_range(const void *address, size_t size, u8 value); -void unpoison_range(const void *address, size_t size); - #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) static inline const void *kasan_shadow_to_mem(const void *shadow_addr) @@ -196,8 +193,6 @@ void print_tags(u8 addr_tag, const void static inline void print_tags(u8 addr_tag, const void *addr) { } #endif -bool check_invalid_free(void *addr); - void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void metadata_fetch_row(char *buffer, void *row); @@ -278,6 +273,37 @@ static inline u8 random_tag(void) { retu static inline u8 random_tag(void) { return 0; } #endif +#ifdef CONFIG_KASAN_HW_TAGS + +static inline void poison_range(const void *address, size_t size, u8 value) +{ + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), value); +} + +static inline void unpoison_range(const void *address, size_t size) +{ + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); +} + +static inline bool check_invalid_free(void *addr) +{ + u8 ptr_tag = get_tag(addr); + u8 mem_tag = hw_get_mem_tag(addr); + + return (mem_tag == KASAN_TAG_INVALID) || + (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); +} + +#else /* CONFIG_KASAN_HW_TAGS */ + +void poison_range(const void *address, size_t size, u8 value); +void unpoison_range(const void *address, size_t size); +bool check_invalid_free(void *addr); + +#endif /* CONFIG_KASAN_HW_TAGS */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Tue Dec 22 20:03:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A025C433DB for ; Tue, 22 Dec 2020 20:04:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D217420700 for ; Tue, 22 Dec 2020 20:04:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D217420700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 027C28D004C; Tue, 22 Dec 2020 15:03:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F1C608D0048; Tue, 22 Dec 2020 15:03:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0D118D004C; Tue, 22 Dec 2020 15:03:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id C729E8D0048 for ; Tue, 22 Dec 2020 15:03:09 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 88C54180AD83B for ; Tue, 22 Dec 2020 20:03:09 +0000 (UTC) X-FDA: 77621992098.28.sun47_18075c227462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 5AF226C2C for ; Tue, 22 Dec 2020 20:03:09 +0000 (UTC) X-HE-Tag: sun47_18075c227462 X-Filterd-Recvd-Size: 12911 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3CA1123137; Tue, 22 Dec 2020 20:03:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667387; bh=hyYfzSohot5OY4xtsFYuRbnkcFMBQrgeMqhDRbh4nro=; h=Date:From:To:Subject:In-Reply-To:From; b=AEPR80h5NNkOWu2+61+hX1Z8EYAQ4d30+bXgrrJRRve1irfxbWq8YOhtFOc2FdVg8 vyNGRV2qxg8uc/rBzEwKWmaC82U0tOfpkhe2482gsV7DEmdguwACa9lxeGYspZI8Xu iUiuaZ14TjPJdrc//HibNBYyDiCO2ce3yvk6SfX8= Date: Tue, 22 Dec 2020 12:03:06 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 52/60] kasan: add and integrate kasan boot parameters Message-ID: <20201222200306.eGuADKVc8%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add and integrate kasan boot parameters Hardware tag-based KASAN mode is intended to eventually be used in production as a security mitigation. Therefore there's a need for finer control over KASAN features and for an existence of a kill switch. This change adds a few boot parameters for hardware tag-based KASAN that allow to disable or otherwise control particular KASAN features. The features that can be controlled are: 1. Whether KASAN is enabled at all. 2. Whether KASAN collects and saves alloc/free stacks. 3. Whether KASAN panics on a detected bug or not. With this change a new boot parameter kasan.mode allows to choose one of three main modes: - kasan.mode=off - KASAN is disabled, no tag checks are performed - kasan.mode=prod - only essential production features are enabled - kasan.mode=full - all KASAN features are enabled The chosen mode provides default control values for the features mentioned above. However it's also possible to override the default values by providing: - kasan.stacktrace=off/on - enable alloc/free stack collection (default: on for mode=full, otherwise off) - kasan.fault=report/panic - only report tag fault or also panic (default: report) If kasan.mode parameter is not provided, it defaults to full when CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise. It is essential that switching between these modes doesn't require rebuilding the kernel with different configs, as this is required by the Android GKI (Generic Kernel Image) initiative [1]. [1] https://source.android.com/devices/architecture/kernel/generic-kernel-image [andreyknvl@google.com: don't use read-only static keys] Link: https://lkml.kernel.org/r/f2ded589eba1597f7360a972226083de9afd86e2.1607537948.git.andreyknvl@google.com Link: https://lkml.kernel.org/r/cb093613879d8d8841173f090133eddeb4c35f1f.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4 Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 22 ++++-- mm/kasan/hw_tags.c | 151 +++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 16 ++++ mm/kasan/report.c | 14 +++ 4 files changed, 196 insertions(+), 7 deletions(-) --- a/mm/kasan/common.c~kasan-add-and-integrate-kasan-boot-parameters +++ a/mm/kasan/common.c @@ -134,6 +134,11 @@ void kasan_cache_create(struct kmem_cach unsigned int redzone_size; int redzone_adjust; + if (!kasan_stack_collection_enabled()) { + *flags |= SLAB_KASAN; + return; + } + /* Add alloc meta. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); @@ -170,6 +175,8 @@ void kasan_cache_create(struct kmem_cach size_t kasan_metadata_size(struct kmem_cache *cache) { + if (!kasan_stack_collection_enabled()) + return 0; return (cache->kasan_info.alloc_meta_offset ? sizeof(struct kasan_alloc_meta) : 0) + (cache->kasan_info.free_meta_offset ? @@ -262,11 +269,13 @@ void * __must_check kasan_init_slab_obj( { struct kasan_alloc_meta *alloc_meta; - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; + if (kasan_stack_collection_enabled()) { + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -303,6 +312,9 @@ static bool __kasan_slab_free(struct kme rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE); + if (!kasan_stack_collection_enabled()) + return false; + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || unlikely(!(cache->flags & SLAB_KASAN))) return false; @@ -350,7 +362,7 @@ static void *__kasan_kmalloc(struct kmem poison_range((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) + if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); --- a/mm/kasan/hw_tags.c~kasan-add-and-integrate-kasan-boot-parameters +++ a/mm/kasan/hw_tags.c @@ -8,18 +8,115 @@ #define pr_fmt(fmt) "kasan: " fmt +#include #include #include #include #include +#include #include #include #include "kasan.h" +enum kasan_arg_mode { + KASAN_ARG_MODE_DEFAULT, + KASAN_ARG_MODE_OFF, + KASAN_ARG_MODE_PROD, + KASAN_ARG_MODE_FULL, +}; + +enum kasan_arg_stacktrace { + KASAN_ARG_STACKTRACE_DEFAULT, + KASAN_ARG_STACKTRACE_OFF, + KASAN_ARG_STACKTRACE_ON, +}; + +enum kasan_arg_fault { + KASAN_ARG_FAULT_DEFAULT, + KASAN_ARG_FAULT_REPORT, + KASAN_ARG_FAULT_PANIC, +}; + +static enum kasan_arg_mode kasan_arg_mode __ro_after_init; +static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init; +static enum kasan_arg_fault kasan_arg_fault __ro_after_init; + +/* Whether KASAN is enabled at all. */ +DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled); +EXPORT_SYMBOL(kasan_flag_enabled); + +/* Whether to collect alloc/free stack traces. */ +DEFINE_STATIC_KEY_FALSE(kasan_flag_stacktrace); + +/* Whether panic or disable tag checking on fault. */ +bool kasan_flag_panic __ro_after_init; + +/* kasan.mode=off/prod/full */ +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_mode = KASAN_ARG_MODE_OFF; + else if (!strcmp(arg, "prod")) + kasan_arg_mode = KASAN_ARG_MODE_PROD; + else if (!strcmp(arg, "full")) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + return -EINVAL; + + return 0; +} +early_param("kasan.mode", early_kasan_mode); + +/* kasan.stack=off/on */ +static int __init early_kasan_flag_stacktrace(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_OFF; + else if (!strcmp(arg, "on")) + kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_ON; + else + return -EINVAL; + + return 0; +} +early_param("kasan.stacktrace", early_kasan_flag_stacktrace); + +/* kasan.fault=report/panic */ +static int __init early_kasan_fault(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "report")) + kasan_arg_fault = KASAN_ARG_FAULT_REPORT; + else if (!strcmp(arg, "panic")) + kasan_arg_fault = KASAN_ARG_FAULT_PANIC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.fault", early_kasan_fault); + /* kasan_init_hw_tags_cpu() is called for each CPU. */ void kasan_init_hw_tags_cpu(void) { + /* + * There's no need to check that the hardware is MTE-capable here, + * as this function is only called for MTE-capable hardware. + */ + + /* If KASAN is disabled, do nothing. */ + if (kasan_arg_mode == KASAN_ARG_MODE_OFF) + return; + hw_init_tags(KASAN_TAG_MAX); hw_enable_tagging(); } @@ -27,6 +124,60 @@ void kasan_init_hw_tags_cpu(void) /* kasan_init_hw_tags() is called once on boot CPU. */ void __init kasan_init_hw_tags(void) { + /* If hardware doesn't support MTE, do nothing. */ + if (!system_supports_mte()) + return; + + /* Choose KASAN mode if kasan boot parameter is not provided. */ + if (kasan_arg_mode == KASAN_ARG_MODE_DEFAULT) { + if (IS_ENABLED(CONFIG_DEBUG_KERNEL)) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + kasan_arg_mode = KASAN_ARG_MODE_PROD; + } + + /* Preset parameter values based on the mode. */ + switch (kasan_arg_mode) { + case KASAN_ARG_MODE_DEFAULT: + /* Shouldn't happen as per the check above. */ + WARN_ON(1); + return; + case KASAN_ARG_MODE_OFF: + /* If KASAN is disabled, do nothing. */ + return; + case KASAN_ARG_MODE_PROD: + static_branch_enable(&kasan_flag_enabled); + break; + case KASAN_ARG_MODE_FULL: + static_branch_enable(&kasan_flag_enabled); + static_branch_enable(&kasan_flag_stacktrace); + break; + } + + /* Now, optionally override the presets. */ + + switch (kasan_arg_stacktrace) { + case KASAN_ARG_STACKTRACE_DEFAULT: + break; + case KASAN_ARG_STACKTRACE_OFF: + static_branch_disable(&kasan_flag_stacktrace); + break; + case KASAN_ARG_STACKTRACE_ON: + static_branch_enable(&kasan_flag_stacktrace); + break; + } + + switch (kasan_arg_fault) { + case KASAN_ARG_FAULT_DEFAULT: + break; + case KASAN_ARG_FAULT_REPORT: + kasan_flag_panic = false; + break; + case KASAN_ARG_FAULT_PANIC: + kasan_flag_panic = true; + break; + } + pr_info("KernelAddressSanitizer initialized\n"); } --- a/mm/kasan/kasan.h~kasan-add-and-integrate-kasan-boot-parameters +++ a/mm/kasan/kasan.h @@ -5,6 +5,22 @@ #include #include +#ifdef CONFIG_KASAN_HW_TAGS +#include +DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace); +static inline bool kasan_stack_collection_enabled(void) +{ + return static_branch_unlikely(&kasan_flag_stacktrace); +} +#else +static inline bool kasan_stack_collection_enabled(void) +{ + return true; +} +#endif + +extern bool kasan_flag_panic __ro_after_init; + #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #else --- a/mm/kasan/report.c~kasan-add-and-integrate-kasan-boot-parameters +++ a/mm/kasan/report.c @@ -99,6 +99,10 @@ static void end_report(unsigned long *fl panic_on_warn = 0; panic("panic_on_warn set ...\n"); } +#ifdef CONFIG_KASAN_HW_TAGS + if (kasan_flag_panic) + panic("kasan.fault=panic set ...\n"); +#endif kasan_enable_current(); } @@ -161,8 +165,8 @@ static void describe_object_addr(struct (void *)(object_addr + cache->object_size)); } -static void describe_object(struct kmem_cache *cache, void *object, - const void *addr, u8 tag) +static void describe_object_stacks(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) { struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); @@ -190,7 +194,13 @@ static void describe_object(struct kmem_ } #endif } +} +static void describe_object(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) +{ + if (kasan_stack_collection_enabled()) + describe_object_stacks(cache, object, addr, tag); describe_object_addr(cache, object, addr); } From patchwork Tue Dec 22 20:03:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 637E6C433E6 for ; Tue, 22 Dec 2020 20:04:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E122C20700 for ; Tue, 22 Dec 2020 20:04:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E122C20700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8440F8D004E; Tue, 22 Dec 2020 15:03:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CA448D004D; Tue, 22 Dec 2020 15:03:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68FF88D004E; Tue, 22 Dec 2020 15:03:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5053B8D004D for ; Tue, 22 Dec 2020 15:03:13 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 14CBB3638 for ; Tue, 22 Dec 2020 20:03:13 +0000 (UTC) X-FDA: 77621992266.18.print02_6005ef727462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id E0728100ED3B0 for ; Tue, 22 Dec 2020 20:03:12 +0000 (UTC) X-HE-Tag: print02_6005ef727462 X-Filterd-Recvd-Size: 19410 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A60D420795; Tue, 22 Dec 2020 20:03:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667391; bh=HeMNj/mDEtimD9mVrilCzSQB2upiy3W+vAHdTEgIQRQ=; h=Date:From:To:Subject:In-Reply-To:From; b=HMhnRChzwaTpfGh5ePFBnTASKfzTtju1c9d1Up3vPpepErTCFOsFG1d7U9jEFsY/0 5BzXckiALnyHcvN+k5tEFL1CBkWcZP7mr3yu8xH44TM6w84UqztMrQmhxlD47ducNq W1dSAPDs+70xsQZHTeR/HNAd2zSG+2cLUuuNBsf0= Date: Tue, 22 Dec 2020 12:03:10 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, Vincenzo.Frascino@arm.com, will.deacon@arm.com Subject: [patch 53/60] kasan, mm: check kasan_enabled in annotations Message-ID: <20201222200310.VzS1ry-6-%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, mm: check kasan_enabled in annotations Declare the kasan_enabled static key in include/linux/kasan.h and in include/linux/mm.h and check it in all kasan annotations. This allows to avoid any slowdown caused by function calls when kasan_enabled is disabled. Link: https://lkml.kernel.org/r/9f90e3c0aa840dbb4833367c2335193299f69023.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I2589451d3c96c97abbcbf714baabe6161c6f153e Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 217 ++++++++++++++++++++++++++++++---------- include/linux/mm.h | 22 ++-- mm/kasan/common.c | 56 +++++----- 3 files changed, 212 insertions(+), 83 deletions(-) --- a/include/linux/kasan.h~kasan-mm-check-kasan_enabled-in-annotations +++ a/include/linux/kasan.h @@ -2,6 +2,7 @@ #ifndef _LINUX_KASAN_H #define _LINUX_KASAN_H +#include #include struct kmem_cache; @@ -75,54 +76,176 @@ static inline void kasan_disable_current #ifdef CONFIG_KASAN -void kasan_unpoison_range(const void *address, size_t size); - -void kasan_alloc_pages(struct page *page, unsigned int order); -void kasan_free_pages(struct page *page, unsigned int order); - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags); - -void kasan_poison_slab(struct page *page); -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); -void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, - const void *object); - -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, - gfp_t flags); -void kasan_kfree_large(void *ptr, unsigned long ip); -void kasan_poison_kfree(void *ptr, unsigned long ip); -void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags); -void * __must_check kasan_krealloc(const void *object, size_t new_size, - gfp_t flags); - -void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags); -bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); - struct kasan_cache { int alloc_meta_offset; int free_meta_offset; }; -size_t kasan_metadata_size(struct kmem_cache *cache); +#ifdef CONFIG_KASAN_HW_TAGS +DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); +static __always_inline bool kasan_enabled(void) +{ + return static_branch_likely(&kasan_flag_enabled); +} +#else +static inline bool kasan_enabled(void) +{ + return true; +} +#endif + +void __kasan_unpoison_range(const void *addr, size_t size); +static __always_inline void kasan_unpoison_range(const void *addr, size_t size) +{ + if (kasan_enabled()) + __kasan_unpoison_range(addr, size); +} + +void __kasan_alloc_pages(struct page *page, unsigned int order); +static __always_inline void kasan_alloc_pages(struct page *page, + unsigned int order) +{ + if (kasan_enabled()) + __kasan_alloc_pages(page, order); +} + +void __kasan_free_pages(struct page *page, unsigned int order); +static __always_inline void kasan_free_pages(struct page *page, + unsigned int order) +{ + if (kasan_enabled()) + __kasan_free_pages(page, order); +} + +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags); +static __always_inline void kasan_cache_create(struct kmem_cache *cache, + unsigned int *size, slab_flags_t *flags) +{ + if (kasan_enabled()) + __kasan_cache_create(cache, size, flags); +} + +size_t __kasan_metadata_size(struct kmem_cache *cache); +static __always_inline size_t kasan_metadata_size(struct kmem_cache *cache) +{ + if (kasan_enabled()) + return __kasan_metadata_size(cache); + return 0; +} + +void __kasan_poison_slab(struct page *page); +static __always_inline void kasan_poison_slab(struct page *page) +{ + if (kasan_enabled()) + __kasan_poison_slab(page); +} + +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object); +static __always_inline void kasan_unpoison_object_data(struct kmem_cache *cache, + void *object) +{ + if (kasan_enabled()) + __kasan_unpoison_object_data(cache, object); +} + +void __kasan_poison_object_data(struct kmem_cache *cache, void *object); +static __always_inline void kasan_poison_object_data(struct kmem_cache *cache, + void *object) +{ + if (kasan_enabled()) + __kasan_poison_object_data(cache, object); +} + +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); +static __always_inline void * __must_check kasan_init_slab_obj( + struct kmem_cache *cache, const void *object) +{ + if (kasan_enabled()) + return __kasan_init_slab_obj(cache, object); + return (void *)object; +} + +bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object, + unsigned long ip) +{ + if (kasan_enabled()) + return __kasan_slab_free(s, object, ip); + return false; +} + +void * __must_check __kasan_slab_alloc(struct kmem_cache *s, + void *object, gfp_t flags); +static __always_inline void * __must_check kasan_slab_alloc( + struct kmem_cache *s, void *object, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_slab_alloc(s, object, flags); + return object; +} + +void * __must_check __kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +static __always_inline void * __must_check kasan_kmalloc(struct kmem_cache *s, + const void *object, size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc(s, object, size, flags); + return (void *)object; +} + +void * __must_check __kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags); +static __always_inline void * __must_check kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc_large(ptr, size, flags); + return (void *)ptr; +} + +void * __must_check __kasan_krealloc(const void *object, + size_t new_size, gfp_t flags); +static __always_inline void * __must_check kasan_krealloc(const void *object, + size_t new_size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_krealloc(object, new_size, flags); + return (void *)object; +} + +void __kasan_poison_kfree(void *ptr, unsigned long ip); +static __always_inline void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_poison_kfree(ptr, ip); +} + +void __kasan_kfree_large(void *ptr, unsigned long ip); +static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_kfree_large(ptr, ip); +} bool kasan_save_enable_multi_shot(void); void kasan_restore_multi_shot(bool enabled); #else /* CONFIG_KASAN */ +static inline bool kasan_enabled(void) +{ + return false; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} - static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} - +static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} @@ -133,36 +256,32 @@ static inline void *kasan_init_slab_obj( { return (void *)object; } - -static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, + unsigned long ip) { - return ptr; + return false; +} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; } -static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags) { return (void *)object; } +static inline void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + return (void *)ptr; +} static inline void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags) { return (void *)object; } - -static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) -{ - return object; -} -static inline bool kasan_slab_free(struct kmem_cache *s, void *object, - unsigned long ip) -{ - return false; -} - -static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } +static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} +static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ --- a/include/linux/mm.h~kasan-mm-check-kasan_enabled-in-annotations +++ a/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -1422,22 +1423,30 @@ static inline bool cpupid_match_pid(stru #endif /* CONFIG_NUMA_BALANCING */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) + static inline u8 page_kasan_tag(const struct page *page) { - return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + if (kasan_enabled()) + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + return 0xff; } static inline void page_kasan_tag_set(struct page *page, u8 tag) { - page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); - page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + if (kasan_enabled()) { + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + } } static inline void page_kasan_tag_reset(struct page *page) { - page_kasan_tag_set(page, 0xff); + if (kasan_enabled()) + page_kasan_tag_set(page, 0xff); } -#else + +#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ + static inline u8 page_kasan_tag(const struct page *page) { return 0xff; @@ -1445,7 +1454,8 @@ static inline u8 page_kasan_tag(const st static inline void page_kasan_tag_set(struct page *page, u8 tag) { } static inline void page_kasan_tag_reset(struct page *page) { } -#endif + +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ static inline struct zone *page_zone(const struct page *page) { --- a/mm/kasan/common.c~kasan-mm-check-kasan_enabled-in-annotations +++ a/mm/kasan/common.c @@ -58,7 +58,7 @@ void kasan_disable_current(void) } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ -void kasan_unpoison_range(const void *address, size_t size) +void __kasan_unpoison_range(const void *address, size_t size) { unpoison_range(address, size); } @@ -86,7 +86,7 @@ asmlinkage void kasan_unpoison_task_stac } #endif /* CONFIG_KASAN_STACK */ -void kasan_alloc_pages(struct page *page, unsigned int order) +void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; unsigned long i; @@ -100,7 +100,7 @@ void kasan_alloc_pages(struct page *page unpoison_range(page_address(page), PAGE_SIZE << order); } -void kasan_free_pages(struct page *page, unsigned int order) +void __kasan_free_pages(struct page *page, unsigned int order) { if (likely(!PageHighMem(page))) poison_range(page_address(page), @@ -127,8 +127,8 @@ static inline unsigned int optimal_redzo object_size <= (1 << 16) - 1024 ? 1024 : 2048; } -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) { unsigned int orig_size = *size; unsigned int redzone_size; @@ -173,7 +173,7 @@ void kasan_cache_create(struct kmem_cach *flags |= SLAB_KASAN; } -size_t kasan_metadata_size(struct kmem_cache *cache) +size_t __kasan_metadata_size(struct kmem_cache *cache) { if (!kasan_stack_collection_enabled()) return 0; @@ -196,7 +196,7 @@ struct kasan_free_meta *kasan_get_free_m return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } -void kasan_poison_slab(struct page *page) +void __kasan_poison_slab(struct page *page) { unsigned long i; @@ -206,12 +206,12 @@ void kasan_poison_slab(struct page *page KASAN_KMALLOC_REDZONE); } -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) { unpoison_range(object, cache->object_size); } -void kasan_poison_object_data(struct kmem_cache *cache, void *object) +void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { poison_range(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), @@ -264,7 +264,7 @@ static u8 assign_tag(struct kmem_cache * #endif } -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_meta; @@ -283,7 +283,7 @@ void * __must_check kasan_init_slab_obj( return (void *)object; } -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, +static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { u8 tag; @@ -326,9 +326,9 @@ static bool __kasan_slab_free(struct kme return IS_ENABLED(CONFIG_KASAN_GENERIC); } -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) { - return __kasan_slab_free(cache, object, ip, true); + return ____kasan_slab_free(cache, object, ip, true); } static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) @@ -336,7 +336,7 @@ static void set_alloc_info(struct kmem_c kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); } -static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, +static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { unsigned long redzone_start; @@ -368,20 +368,20 @@ static void *__kasan_kmalloc(struct kmem return set_tag(object, tag); } -void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, - gfp_t flags) +void * __must_check __kasan_slab_alloc(struct kmem_cache *cache, + void *object, gfp_t flags) { - return __kasan_kmalloc(cache, object, cache->object_size, flags, false); + return ____kasan_kmalloc(cache, object, cache->object_size, flags, false); } -void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, - size_t size, gfp_t flags) +void * __must_check __kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { - return __kasan_kmalloc(cache, object, size, flags, true); + return ____kasan_kmalloc(cache, object, size, flags, true); } -EXPORT_SYMBOL(kasan_kmalloc); +EXPORT_SYMBOL(__kasan_kmalloc); -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, +void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; @@ -406,7 +406,7 @@ void * __must_check kasan_kmalloc_large( return (void *)ptr; } -void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; @@ -416,13 +416,13 @@ void * __must_check kasan_krealloc(const page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); + return __kasan_kmalloc_large(object, size, flags); else - return __kasan_kmalloc(page->slab_cache, object, size, + return ____kasan_kmalloc(page->slab_cache, object, size, flags, true); } -void kasan_poison_kfree(void *ptr, unsigned long ip) +void __kasan_poison_kfree(void *ptr, unsigned long ip) { struct page *page; @@ -435,11 +435,11 @@ void kasan_poison_kfree(void *ptr, unsig } poison_range(ptr, page_size(page), KASAN_FREE_PAGE); } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); + ____kasan_slab_free(page->slab_cache, ptr, ip, false); } } -void kasan_kfree_large(void *ptr, unsigned long ip) +void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); From patchwork Tue Dec 22 20:03:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7103C433DB for ; Tue, 22 Dec 2020 20:04:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 91DE220700 for ; Tue, 22 Dec 2020 20:04:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91DE220700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C101D8D004F; Tue, 22 Dec 2020 15:03:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE70D8D004D; Tue, 22 Dec 2020 15:03:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A869C8D004F; Tue, 22 Dec 2020 15:03:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id 933078D004D for ; Tue, 22 Dec 2020 15:03:16 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 624E98249980 for ; Tue, 22 Dec 2020 20:03:16 +0000 (UTC) X-FDA: 77621992392.17.spade85_5412c0b27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 4277F180D0180 for ; Tue, 22 Dec 2020 20:03:16 +0000 (UTC) X-HE-Tag: spade85_5412c0b27462 X-Filterd-Recvd-Size: 6414 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1F1C821D81; Tue, 22 Dec 2020 20:03:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667394; bh=9TsHQ6uDKwIEBcEbiKo3oU2d3k0N/gw6C3j6ssode7M=; h=Date:From:To:Subject:In-Reply-To:From; b=g4ZxiF5BD9oUF0yMZvieVnSLMmYApQTke2TZDfr4nhvMIvzmII+yIfkiUBkESKOXw IH3V8ZDa3hDxzyABoA8UtXUeyjq/Xa9ccHSm001GSn7z5HjWeDNP2SreCs5SGacTpq QP1PAPOgyGYETIAeJBv4weLDnB55ARjqeo35nS6Q= Date: Tue, 22 Dec 2020 12:03:13 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 54/60] kasan, mm: rename kasan_poison_kfree Message-ID: <20201222200313.0kdHaewuD%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, mm: rename kasan_poison_kfree Rename kasan_poison_kfree() to kasan_slab_free_mempool() as it better reflects what this annotation does. Also add a comment that explains the PageSlab() check. No functional changes. Link: https://lkml.kernel.org/r/141675fb493555e984c5dca555e9d9f768c7bbaa.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I5026f87364e556b506ef1baee725144bb04b8810 Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 16 ++++++++-------- mm/kasan/common.c | 40 +++++++++++++++++++++++----------------- mm/mempool.c | 2 +- 3 files changed, 32 insertions(+), 26 deletions(-) --- a/include/linux/kasan.h~kasan-mm-rename-kasan_poison_kfree +++ a/include/linux/kasan.h @@ -176,6 +176,13 @@ static __always_inline bool kasan_slab_f return false; } +void __kasan_slab_free_mempool(void *ptr, unsigned long ip); +static __always_inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_slab_free_mempool(ptr, ip); +} + void * __must_check __kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); static __always_inline void * __must_check kasan_slab_alloc( @@ -216,13 +223,6 @@ static __always_inline void * __must_che return (void *)object; } -void __kasan_poison_kfree(void *ptr, unsigned long ip); -static __always_inline void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - if (kasan_enabled()) - __kasan_poison_kfree(ptr, ip); -} - void __kasan_kfree_large(void *ptr, unsigned long ip); static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip) { @@ -261,6 +261,7 @@ static inline bool kasan_slab_free(struc { return false; } +static inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) {} static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) { @@ -280,7 +281,6 @@ static inline void *kasan_krealloc(const { return (void *)object; } -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ --- a/mm/kasan/common.c~kasan-mm-rename-kasan_poison_kfree +++ a/mm/kasan/common.c @@ -331,6 +331,29 @@ bool __kasan_slab_free(struct kmem_cache return ____kasan_slab_free(cache, object, ip, true); } +void __kasan_slab_free_mempool(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + /* + * Even though this function is only called for kmem_cache_alloc and + * kmalloc backed mempool allocations, those allocations can still be + * !PageSlab() when the size provided to kmalloc is larger than + * KMALLOC_MAX_SIZE, and kmalloc falls back onto page_alloc. + */ + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + poison_range(ptr, page_size(page), KASAN_FREE_PAGE); + } else { + ____kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); @@ -422,23 +445,6 @@ void * __must_check __kasan_krealloc(con flags, true); } -void __kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - poison_range(ptr, page_size(page), KASAN_FREE_PAGE); - } else { - ____kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) --- a/mm/mempool.c~kasan-mm-rename-kasan_poison_kfree +++ a/mm/mempool.c @@ -104,7 +104,7 @@ static inline void poison_element(mempoo static __always_inline void kasan_poison_element(mempool_t *pool, void *element) { if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) - kasan_poison_kfree(element, _RET_IP_); + kasan_slab_free_mempool(element, _RET_IP_); else if (pool->alloc == mempool_alloc_pages) kasan_free_pages(element, (unsigned long)pool->pool_data); } From patchwork Tue Dec 22 20:03:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D23ADC433DB for ; Tue, 22 Dec 2020 20:04:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7D42A20700 for ; Tue, 22 Dec 2020 20:04:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D42A20700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 558B48D0050; Tue, 22 Dec 2020 15:03:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E3A38D004D; Tue, 22 Dec 2020 15:03:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3ADFF8D0050; Tue, 22 Dec 2020 15:03:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id 2133E8D004D for ; Tue, 22 Dec 2020 15:03:20 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DEDC61EE6 for ; Tue, 22 Dec 2020 20:03:19 +0000 (UTC) X-FDA: 77621992518.30.week92_4e0930927462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id B68D9180B3C94 for ; Tue, 22 Dec 2020 20:03:19 +0000 (UTC) X-HE-Tag: week92_4e0930927462 X-Filterd-Recvd-Size: 3890 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B031E21D1B; Tue, 22 Dec 2020 20:03:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667398; bh=yp1OULphLpM+nd+iKILWI/XZeC4mNqXX649a/tA6Mm4=; h=Date:From:To:Subject:In-Reply-To:From; b=Lquvbzc4+EdHIhl/btBLVR1892D+s+BFC5QQJuR+vs8mFfEy4CwBZGQaupIWfwgpp 1la0MEN07txM4K+7ZkNAEGFeb07yXNfgjsN1GvNocM26Xiu3rZU9Qa9bG9GGaWn9ri 2fwVAIXud6xf+nDWhLV0tydS93j1eH7z/fihSzIE= Date: Tue, 22 Dec 2020 12:03:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 55/60] kasan: don't round_up too much Message-ID: <20201222200317.GjVde9W0u%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: don't round_up too much For hardware tag-based mode kasan_poison_memory() already rounds up the size. Do the same for software modes and remove round_up() from the common code. Link: https://lkml.kernel.org/r/47b232474f1f89dc072aeda0fa58daa6efade377.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ib397128fac6eba874008662b4964d65352db4aa4 Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 8 ++------ mm/kasan/shadow.c | 1 + 2 files changed, 3 insertions(+), 6 deletions(-) --- a/mm/kasan/common.c~kasan-dont-round_up-too-much +++ a/mm/kasan/common.c @@ -213,9 +213,7 @@ void __kasan_unpoison_object_data(struct void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { - poison_range(object, - round_up(cache->object_size, KASAN_GRANULE_SIZE), - KASAN_KMALLOC_REDZONE); + poison_range(object, cache->object_size, KASAN_KMALLOC_REDZONE); } /* @@ -288,7 +286,6 @@ static bool ____kasan_slab_free(struct k { u8 tag; void *tagged_object; - unsigned long rounded_up_size; tag = get_tag(object); tagged_object = object; @@ -309,8 +306,7 @@ static bool ____kasan_slab_free(struct k return true; } - rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); - poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE); + poison_range(object, cache->object_size, KASAN_KMALLOC_FREE); if (!kasan_stack_collection_enabled()) return false; --- a/mm/kasan/shadow.c~kasan-dont-round_up-too-much +++ a/mm/kasan/shadow.c @@ -82,6 +82,7 @@ void poison_range(const void *address, s * addresses to this function. */ address = kasan_reset_tag(address); + size = round_up(size, KASAN_GRANULE_SIZE); shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); From patchwork Tue Dec 22 20:03:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5F7CC433E0 for ; Tue, 22 Dec 2020 20:04:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5AA3520700 for ; Tue, 22 Dec 2020 20:04:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AA3520700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DBB818D0053; Tue, 22 Dec 2020 15:03:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D6B048D0051; Tue, 22 Dec 2020 15:03:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5C0A8D0053; Tue, 22 Dec 2020 15:03:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0035.hostedemail.com [216.40.44.35]) by kanga.kvack.org (Postfix) with ESMTP id AAB8E8D0051 for ; Tue, 22 Dec 2020 15:03:23 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6F820181AEF21 for ; Tue, 22 Dec 2020 20:03:23 +0000 (UTC) X-FDA: 77621992686.19.chess24_40072ee27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 5CA521AD1BA for ; Tue, 22 Dec 2020 20:03:23 +0000 (UTC) X-HE-Tag: chess24_40072ee27462 X-Filterd-Recvd-Size: 4204 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6CB0722583; Tue, 22 Dec 2020 20:03:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667402; bh=ly+Js3FOuGOqMb/jC+AuE0wJvoWr61MuGVX2dNpum2g=; h=Date:From:To:Subject:In-Reply-To:From; b=wHt+KvX3kgQDkqzfQFI6B7uIYPqSSKQFivQtuw9b+L/KVW7agCSLMFbKlgHwd9PEc dZJUC1vnfqogBuuoYvzgTzGsfONPvzH7L8yAJXU1wamqfKVEZKKMdcH4/aftLovd6F Y72XcCBJLW+LlUPSDHzfs+Ei4qA1T2RQQbwNDQ7M= Date: Tue, 22 Dec 2020 12:03:20 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 56/60] kasan: simplify assign_tag and set_tag calls Message-ID: <20201222200320._nL5WSSxl%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: simplify assign_tag and set_tag calls set_tag() already ignores the tag for the generic mode, so just call it as is. Add a check for the generic mode to assign_tag(), and simplify its call in ____kasan_kmalloc(). Link: https://lkml.kernel.org/r/121eeab245f98555862b289d2ba9269c868fbbcf.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I18905ca78fb4a3d60e1a34a4ca00247272480438 Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) --- a/mm/kasan/common.c~kasan-simplify-assign_tag-and-set_tag-calls +++ a/mm/kasan/common.c @@ -233,6 +233,9 @@ void __kasan_poison_object_data(struct k static u8 assign_tag(struct kmem_cache *cache, const void *object, bool init, bool keep_tag) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return 0xff; + /* * 1. When an object is kmalloc()'ed, two hooks are called: * kasan_slab_alloc() and kasan_kmalloc(). We assign the @@ -275,8 +278,8 @@ void * __must_check __kasan_init_slab_ob __memset(alloc_meta, 0, sizeof(*alloc_meta)); } - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - object = set_tag(object, assign_tag(cache, object, true, false)); + /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ + object = set_tag(object, assign_tag(cache, object, true, false)); return (void *)object; } @@ -360,7 +363,7 @@ static void *____kasan_kmalloc(struct km { unsigned long redzone_start; unsigned long redzone_end; - u8 tag = 0xff; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -372,9 +375,7 @@ static void *____kasan_kmalloc(struct km KASAN_GRANULE_SIZE); redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_GRANULE_SIZE); - - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - tag = assign_tag(cache, object, false, keep_tag); + tag = assign_tag(cache, object, false, keep_tag); /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */ unpoison_range(set_tag(object, tag), size); From patchwork Tue Dec 22 20:03:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8854DC433DB for ; Tue, 22 Dec 2020 20:04:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 22B0E20795 for ; Tue, 22 Dec 2020 20:04:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 22B0E20795 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A12E48D0054; Tue, 22 Dec 2020 15:03:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C3718D0051; Tue, 22 Dec 2020 15:03:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88A9F8D0054; Tue, 22 Dec 2020 15:03:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0229.hostedemail.com [216.40.44.229]) by kanga.kvack.org (Postfix) with ESMTP id 73B028D0051 for ; Tue, 22 Dec 2020 15:03:27 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 380A9180AD83B for ; Tue, 22 Dec 2020 20:03:27 +0000 (UTC) X-FDA: 77621992854.29.curve26_33157d327462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 07967180868CA for ; Tue, 22 Dec 2020 20:03:27 +0000 (UTC) X-HE-Tag: curve26_33157d327462 X-Filterd-Recvd-Size: 2924 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0850B23133; Tue, 22 Dec 2020 20:03:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667405; bh=JM8QWrDg7sBMI5LsEMe0za0IAKlgysbCmniUFjiFTvI=; h=Date:From:To:Subject:In-Reply-To:From; b=nclu423SdrM8qiuSa+xzq5QFk/c3R0I3QHYtD77yKQ1vuKW7+zC5vE0I6w9m3IovR cYgZENXCZ4maG9OZlsPnIPWddDUKDH1jtjeU1VPZ88zcJnRghTd3e0UQ09CG8BFVCV MP8zc2ICFc4FuBY4/9kpWGKsAt/0wMUU1aOk+BAg= Date: Tue, 22 Dec 2020 12:03:24 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 57/60] kasan: clarify comment in __kasan_kfree_large Message-ID: <20201222200324.fNkB_D1hi%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: clarify comment in __kasan_kfree_large Currently it says that the memory gets poisoned by page_alloc code. Clarify this by mentioning the specific callback that poisons the memory. Link: https://lkml.kernel.org/r/1c8380fe0332a3bcc720fe29f1e0bef2e2974416.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I1334dffb69b87d7986fab88a1a039cc3ea764725 Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/kasan/common.c~kasan-clarify-comment-in-__kasan_kfree_large +++ a/mm/kasan/common.c @@ -446,5 +446,5 @@ void __kasan_kfree_large(void *ptr, unsi { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ + /* The object will be poisoned by kasan_free_pages(). */ } From patchwork Tue Dec 22 20:03:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F404C433E0 for ; Tue, 22 Dec 2020 20:04:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0F60820700 for ; Tue, 22 Dec 2020 20:04:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F60820700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9514B8D0056; Tue, 22 Dec 2020 15:03:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 925828D0055; Tue, 22 Dec 2020 15:03:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C6F98D0056; Tue, 22 Dec 2020 15:03:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 624FA8D0055 for ; Tue, 22 Dec 2020 15:03:31 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 29E70181AEF23 for ; Tue, 22 Dec 2020 20:03:31 +0000 (UTC) X-FDA: 77621993022.19.cry56_000447527462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 02A701AD1B7 for ; Tue, 22 Dec 2020 20:03:30 +0000 (UTC) X-HE-Tag: cry56_000447527462 X-Filterd-Recvd-Size: 18779 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A434E229C5; Tue, 22 Dec 2020 20:03:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667409; bh=Of/jzA8+Tr9XIYgBpVpaCKX4IG0ekMKLFRIaldPs10E=; h=Date:From:To:Subject:In-Reply-To:From; b=B+8WWd/9LUVBHPr9jQJGDWartcjU4dy2D8r7JI3JeBdtqmljg66/Nj9xMTVveqUDY OXTTptoNKauuwkWTWENA4h9NeC1/v+i1TdeoSZBYF+T5lCPytP7laaJvuzkj4U+qvb gbOa3/yrxQgngtot43d0r4Gxzh6cWiPqR7qH95yg= Date: Tue, 22 Dec 2020 12:03:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, Vincenzo.Frascino@arm.com, will.deacon@arm.com Subject: [patch 58/60] kasan: sanitize objects when metadata doesn't fit Message-ID: <20201222200328.D_oFzGWqb%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: sanitize objects when metadata doesn't fit KASAN marks caches that are sanitized with the SLAB_KASAN cache flag. Currently if the metadata that is appended after the object (stores e.g. stack trace ids) doesn't fit into KMALLOC_MAX_SIZE (can only happen with SLAB, see the comment in the patch), KASAN turns off sanitization completely. With this change sanitization of the object data is always enabled. However the metadata is only stored when it fits. Instead of checking for SLAB_KASAN flag accross the code to find out whether the metadata is there, use cache->kasan_info.alloc/free_meta_offset. As 0 can be a valid value for free_meta_offset, introduce KASAN_NO_FREE_META as an indicator that the free metadata is missing. Without this change all sanitized KASAN objects would be put into quarantine with generic KASAN. With this change, only the objects that have metadata (i.e. when it fits) are put into quarantine, the rest is freed right away. Along the way rework __kasan_cache_create() and add claryfying comments. Link: https://lkml.kernel.org/r/aee34b87a5e4afe586c2ac6a0b32db8dc4dcc2dc.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Icd947e2bea054cb5cfbdc6cf6652227d97032dcb Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/kasan/common.c | 116 ++++++++++++++++++++++-------------- mm/kasan/generic.c | 9 +- mm/kasan/hw_tags.c | 6 + mm/kasan/kasan.h | 17 ++++- mm/kasan/quarantine.c | 18 ++++- mm/kasan/report.c | 43 +++++++------ mm/kasan/report_sw_tags.c | 9 +- mm/kasan/sw_tags.c | 4 + 8 files changed, 147 insertions(+), 75 deletions(-) --- a/mm/kasan/common.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/common.c @@ -114,9 +114,6 @@ void __kasan_free_pages(struct page *pag */ static inline unsigned int optimal_redzone(unsigned int object_size) { - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) - return 0; - return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -130,47 +127,77 @@ static inline unsigned int optimal_redzo void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { - unsigned int orig_size = *size; - unsigned int redzone_size; - int redzone_adjust; + unsigned int ok_size; + unsigned int optimal_size; + + /* + * SLAB_KASAN is used to mark caches as ones that are sanitized by + * KASAN. Currently this flag is used in two places: + * 1. In slab_ksize() when calculating the size of the accessible + * memory within the object. + * 2. In slab_common.c to prevent merging of sanitized caches. + */ + *flags |= SLAB_KASAN; - if (!kasan_stack_collection_enabled()) { - *flags |= SLAB_KASAN; + if (!kasan_stack_collection_enabled()) return; - } - /* Add alloc meta. */ + ok_size = *size; + + /* Add alloc meta into redzone. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); - /* Add free meta. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC) && - (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta))) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); + /* + * If alloc meta doesn't fit, don't add it. + * This can only happen with SLAB, as it has KMALLOC_MAX_SIZE equal + * to KMALLOC_MAX_CACHE_SIZE and doesn't fall back to page_alloc for + * larger sizes. + */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.alloc_meta_offset = 0; + *size = ok_size; + /* Continue, since free meta might still fit. */ } - redzone_size = optimal_redzone(cache->object_size); - redzone_adjust = redzone_size - (*size - cache->object_size); - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + redzone_size)); + /* Only the generic mode uses free meta or flexible redzones. */ + if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + return; + } /* - * If the metadata doesn't fit, don't enable KASAN at all. + * Add free meta into redzone when it's not possible to store + * it in the object. This is the case when: + * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that it can + * be touched after it was freed, or + * 2. Object has a constructor, which means it's expected to + * retain its content until the next allocation, or + * 3. Object is too small. + * Otherwise cache->kasan_info.free_meta_offset = 0 is implied. */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; + if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + ok_size = *size; + + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + + /* If free meta doesn't fit, don't add it. */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + *size = ok_size; + } } - *flags |= SLAB_KASAN; + /* Calculate size with optimal redzone. */ + optimal_size = cache->object_size + optimal_redzone(cache->object_size); + /* Limit it with KMALLOC_MAX_SIZE (relevant for SLAB only). */ + if (optimal_size > KMALLOC_MAX_SIZE) + optimal_size = KMALLOC_MAX_SIZE; + /* Use optimal size if the size with added metas is not large enough. */ + if (*size < optimal_size) + *size = optimal_size; } size_t __kasan_metadata_size(struct kmem_cache *cache) @@ -186,15 +213,21 @@ size_t __kasan_metadata_size(struct kmem struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { + if (!cache->kasan_info.alloc_meta_offset) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + if (cache->kasan_info.free_meta_offset == KASAN_NO_FREE_META) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } +#endif void __kasan_poison_slab(struct page *page) { @@ -271,11 +304,9 @@ void * __must_check __kasan_init_slab_ob struct kasan_alloc_meta *alloc_meta; if (kasan_stack_collection_enabled()) { - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + if (alloc_meta) + __memset(alloc_meta, 0, sizeof(*alloc_meta)); } /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ @@ -314,15 +345,12 @@ static bool ____kasan_slab_free(struct k if (!kasan_stack_collection_enabled()) return false; - if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || - unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine)) return false; kasan_set_free_info(cache, object, tag); - quarantine_put(cache, object); - - return IS_ENABLED(CONFIG_KASAN_GENERIC); + return quarantine_put(cache, object); } bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -355,7 +383,11 @@ void __kasan_slab_free_mempool(void *ptr static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + struct kasan_alloc_meta *alloc_meta; + + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) + kasan_set_track(&alloc_meta->alloc_track, flags); } static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, @@ -382,7 +414,7 @@ static void *____kasan_kmalloc(struct km poison_range((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) + if (kasan_stack_collection_enabled()) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); --- a/mm/kasan/generic.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/generic.c @@ -348,11 +348,11 @@ void kasan_set_free_info(struct kmem_cac struct kasan_free_meta *free_meta; free_meta = kasan_get_free_meta(cache, object); - kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + if (!free_meta) + return; - /* - * the object was freed and has free track set - */ + kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + /* The object was freed and has free track set. */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREETRACK; } @@ -361,5 +361,6 @@ struct kasan_track *kasan_get_free_track { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; + /* Free meta must be present with KASAN_KMALLOC_FREETRACK. */ return &kasan_get_free_meta(cache, object)->free_track; } --- a/mm/kasan/hw_tags.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/hw_tags.c @@ -187,7 +187,8 @@ void kasan_set_free_info(struct kmem_cac struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); - kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); + if (alloc_meta) + kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, @@ -196,5 +197,8 @@ struct kasan_track *kasan_get_free_track struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; + return &alloc_meta->free_track[0]; } --- a/mm/kasan/kasan.h~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/kasan.h @@ -155,20 +155,31 @@ struct kasan_alloc_meta { struct qlist_node { struct qlist_node *next; }; + +/* + * Generic mode either stores free meta in the object itself or in the redzone + * after the object. In the former case free meta offset is 0, in the latter + * case it has some sane value smaller than INT_MAX. Use INT_MAX as free meta + * offset when free meta isn't present. + */ +#define KASAN_NO_FREE_META INT_MAX + struct kasan_free_meta { +#ifdef CONFIG_KASAN_GENERIC /* This field is used while the object is in the quarantine. * Otherwise it might be used for the allocator freelist. */ struct qlist_node quarantine_link; -#ifdef CONFIG_KASAN_GENERIC struct kasan_track free_track; #endif }; struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object); +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); +#endif #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) @@ -233,11 +244,11 @@ struct kasan_track *kasan_get_free_track #if defined(CONFIG_KASAN_GENERIC) && \ (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) -void quarantine_put(struct kmem_cache *cache, void *object); +bool quarantine_put(struct kmem_cache *cache, void *object); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); #else -static inline void quarantine_put(struct kmem_cache *cache, void *object) { } +static inline bool quarantine_put(struct kmem_cache *cache, void *object) { return false; } static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif --- a/mm/kasan/quarantine.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/quarantine.c @@ -137,7 +137,12 @@ static void qlink_free(struct qlist_node if (IS_ENABLED(CONFIG_SLAB)) local_irq_save(flags); + /* + * As the object now gets freed from the quaratine, assume that its + * free track is no longer valid. + */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREE; + ___cache_free(cache, object, _THIS_IP_); if (IS_ENABLED(CONFIG_SLAB)) @@ -163,7 +168,7 @@ static void qlist_free_all(struct qlist_ qlist_init(q); } -void quarantine_put(struct kmem_cache *cache, void *object) +bool quarantine_put(struct kmem_cache *cache, void *object) { unsigned long flags; struct qlist_head *q; @@ -171,6 +176,13 @@ void quarantine_put(struct kmem_cache *c struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); /* + * If there's no metadata for this object, don't put it into + * quarantine. + */ + if (!meta) + return false; + + /* * Note: irq must be disabled until after we move the batch to the * global quarantine. Otherwise quarantine_remove_cache() can miss * some objects belonging to the cache if they are in our local temp @@ -183,7 +195,7 @@ void quarantine_put(struct kmem_cache *c q = this_cpu_ptr(&cpu_quarantine); if (q->offline) { local_irq_restore(flags); - return; + return false; } qlist_put(q, &meta->quarantine_link, cache->size); if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) { @@ -206,6 +218,8 @@ void quarantine_put(struct kmem_cache *c } local_irq_restore(flags); + + return true; } void quarantine_reduce(void) --- a/mm/kasan/report.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/report.c @@ -168,32 +168,35 @@ static void describe_object_addr(struct static void describe_object_stacks(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); - - if (cache->flags & SLAB_KASAN) { - struct kasan_track *free_track; + struct kasan_alloc_meta *alloc_meta; + struct kasan_track *free_track; + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) { print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); - free_track = kasan_get_free_track(cache, object, tag); - if (free_track) { - print_track(free_track, "Freed"); - pr_err("\n"); - } + } + + free_track = kasan_get_free_track(cache, object, tag); + if (free_track) { + print_track(free_track, "Freed"); + pr_err("\n"); + } #ifdef CONFIG_KASAN_GENERIC - if (alloc_meta->aux_stack[0]) { - pr_err("Last potentially related work creation:\n"); - print_stack(alloc_meta->aux_stack[0]); - pr_err("\n"); - } - if (alloc_meta->aux_stack[1]) { - pr_err("Second to last potentially related work creation:\n"); - print_stack(alloc_meta->aux_stack[1]); - pr_err("\n"); - } -#endif + if (!alloc_meta) + return; + if (alloc_meta->aux_stack[0]) { + pr_err("Last potentially related work creation:\n"); + print_stack(alloc_meta->aux_stack[0]); + pr_err("\n"); } + if (alloc_meta->aux_stack[1]) { + pr_err("Second to last potentially related work creation:\n"); + print_stack(alloc_meta->aux_stack[1]); + pr_err("\n"); + } +#endif } static void describe_object(struct kmem_cache *cache, void *object, --- a/mm/kasan/report_sw_tags.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/report_sw_tags.c @@ -48,9 +48,12 @@ const char *get_bug_type(struct kasan_ac object = nearest_obj(cache, page, (void *)addr); alloc_meta = kasan_get_alloc_meta(cache, object); - for (i = 0; i < KASAN_NR_FREE_STACKS; i++) - if (alloc_meta->free_pointer_tag[i] == tag) - return "use-after-free"; + if (alloc_meta) { + for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { + if (alloc_meta->free_pointer_tag[i] == tag) + return "use-after-free"; + } + } return "out-of-bounds"; } --- a/mm/kasan/sw_tags.c~kasan-sanitize-objects-when-metadata-doesnt-fit +++ a/mm/kasan/sw_tags.c @@ -170,6 +170,8 @@ void kasan_set_free_info(struct kmem_cac u8 idx = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -187,6 +189,8 @@ struct kasan_track *kasan_get_free_track int i = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Tue Dec 22 20:03:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E687C433E0 for ; Tue, 22 Dec 2020 20:04:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B315220700 for ; Tue, 22 Dec 2020 20:04:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B315220700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D1EF58D0057; Tue, 22 Dec 2020 15:03:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD10D8D0055; Tue, 22 Dec 2020 15:03:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0B788D0057; Tue, 22 Dec 2020 15:03:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id AD1908D0055 for ; Tue, 22 Dec 2020 15:03:34 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7B4998249980 for ; Tue, 22 Dec 2020 20:03:34 +0000 (UTC) X-FDA: 77621993148.03.toes58_0f0e65327462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 512E228A4E8 for ; Tue, 22 Dec 2020 20:03:34 +0000 (UTC) X-HE-Tag: toes58_0f0e65327462 X-Filterd-Recvd-Size: 5638 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 48F9B21D81; Tue, 22 Dec 2020 20:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667412; bh=uJ4An1wnldMhr2Wy6GlXvJSV0e7Q73eZYXMEC7AsUTo=; h=Date:From:To:Subject:In-Reply-To:From; b=1sqhDCabdwBndTgXl9gB0zYgBUGv6HEwCfc4htKOC0Mc/bujXB+NigFe3kL67q4dG rDYlegh+SMODa8IsDSB7aTcIPNkn8/IVUVQS4YkKMk5aJddD+QYx//CWbtulJBRiTu NWnY8bEm+HRGPRcgGhKOdXPzFU+3mfBtrVlIoRgQ= Date: Tue, 22 Dec 2020 12:03:31 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, Vincenzo.Frascino@arm.com, will.deacon@arm.com Subject: [patch 59/60] kasan, mm: allow cache merging with no metadata Message-ID: <20201222200331.wcT_Yhdtg%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, mm: allow cache merging with no metadata The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Link: https://lkml.kernel.org/r/37497e940bfd4b32c0a93a702a9ae4cf061d5392.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) --- a/include/linux/kasan.h~kasan-mm-allow-cache-merging-with-no-metadata +++ a/include/linux/kasan.h @@ -82,17 +82,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -239,6 +252,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} --- a/mm/kasan/common.c~kasan-mm-allow-cache-merging-with-no-metadata +++ a/mm/kasan/common.c @@ -86,6 +86,17 @@ asmlinkage void kasan_unpoison_task_stac } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; --- a/mm/slab_common.c~kasan-mm-allow-cache-merging-with-no-metadata +++ a/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -53,7 +54,7 @@ static DECLARE_WORK(slab_caches_to_rcu_d */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) From patchwork Tue Dec 22 20:03:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11987315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8220C433DB for ; Tue, 22 Dec 2020 20:04:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8C23C20700 for ; Tue, 22 Dec 2020 20:04:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C23C20700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C0E348D0059; Tue, 22 Dec 2020 15:03:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BBF0C8D0055; Tue, 22 Dec 2020 15:03:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD84B8D0059; Tue, 22 Dec 2020 15:03:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0148.hostedemail.com [216.40.44.148]) by kanga.kvack.org (Postfix) with ESMTP id 94AAC8D0055 for ; Tue, 22 Dec 2020 15:03:39 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 5E8621F06 for ; Tue, 22 Dec 2020 20:03:39 +0000 (UTC) X-FDA: 77621993358.12.fan62_18176bb27462 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id C83781800E52A for ; Tue, 22 Dec 2020 20:03:37 +0000 (UTC) X-HE-Tag: fan62_18176bb27462 X-Filterd-Recvd-Size: 15268 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Dec 2020 20:03:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BB69E21D1B; Tue, 22 Dec 2020 20:03:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1608667416; bh=8DcJNR4WmHnL0Hv0fKmZFOYV6gb4BwyMrwvUViDgsMg=; h=Date:From:To:Subject:In-Reply-To:From; b=ELJIoXbC559xqqQbG6DHPTgGdgj6GJP9bHPGR9SADPTGlV759JZFC100JIe1Kpean y9ao1NdYOlBfLIg36vSJBu+Y4xIlRdrKrpniQxl3OlNj6FMlj4HXCcqBjquJoU9fSZ raaCqOmwCayrAPDZ+IfpX5auBq6HkTiOyjv4gOGQ= Date: Tue, 22 Dec 2020 12:03:35 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, gor@linux.ibm.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 60/60] kasan: update documentation Message-ID: <20201222200335.RHdnHDjIK%akpm@linux-foundation.org> In-Reply-To: <20201222115844.d30aaef7df6f5b120d3e0c3d@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: update documentation This change updates KASAN documentation to reflect the addition of boot parameters and also reworks and clarifies some of the existing sections, in particular: defines what a memory granule is, mentions quarantine, makes Kunit section more readable. Link: https://lkml.kernel.org/r/748daf013e17d925b0fe00c1c3b5dce726dd2430.1606162397.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Ib1f83e91be273264b25f42b04448ac96b858849f Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Tested-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/dev-tools/kasan.rst | 188 +++++++++++++++++----------- 1 file changed, 117 insertions(+), 71 deletions(-) --- a/Documentation/dev-tools/kasan.rst~kasan-update-documentation +++ a/Documentation/dev-tools/kasan.rst @@ -4,8 +4,9 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to -find out-of-bound and use-after-free bugs. KASAN has three modes: +KernelAddressSANitizer (KASAN) is a dynamic memory safety error detector +designed to find out-of-bound and use-after-free bugs. KASAN has three modes: + 1. generic KASAN (similar to userspace ASan), 2. software tag-based KASAN (similar to userspace HWASan), 3. hardware tag-based KASAN (based on hardware memory tagging). @@ -39,23 +40,13 @@ CONFIG_KASAN_INLINE. Outline and inline The former produces smaller binary while the latter is 1.1 - 2 times faster. Both software KASAN modes work with both SLUB and SLAB memory allocators, -hardware tag-based KASAN currently only support SLUB. -For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. +while the hardware tag-based KASAN currently only support SLUB. + +For better error reports that include stack traces, enable CONFIG_STACKTRACE. To augment reports with last allocation and freeing stack of the physical page, it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. -To disable instrumentation for specific files or directories, add a line -similar to the following to the respective kernel Makefile: - -- For a single file (e.g. main.o):: - - KASAN_SANITIZE_main.o := n - -- For all files in one directory:: - - KASAN_SANITIZE := n - Error reports ~~~~~~~~~~~~~ @@ -140,22 +131,75 @@ freed (in case of a use-after-free bug r the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. -Reading this part requires some understanding of how KASAN works. - -The state of each 8 aligned bytes of memory is encoded in one shadow byte. -Those 8 bytes can be accessible, partially accessible, freed or be a redzone. -We use the following encoding for each shadow byte: 0 means that all 8 bytes -of the corresponding memory region are accessible; number N (1 <= N <= 7) means -that the first N bytes are accessible, and other (8 - N) bytes are not; -any negative value indicates that the entire 8-byte word is inaccessible. -We use different negative values to distinguish between different kinds of -inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). +Internally KASAN tracks memory state separately for each memory granule, which +is either 8 or 16 aligned bytes depending on KASAN mode. Each number in the +memory state section of the report shows the state of one of the memory +granules that surround the accessed address. + +For generic KASAN the size of each memory granule is 8. The state of each +granule is encoded in one shadow byte. Those 8 bytes can be accessible, +partially accessible, freed or be a part of a redzone. KASAN uses the following +encoding for each shadow byte: 0 means that all 8 bytes of the corresponding +memory region are accessible; number N (1 <= N <= 7) means that the first N +bytes are accessible, and other (8 - N) bytes are not; any negative value +indicates that the entire 8-byte word is inaccessible. KASAN uses different +negative values to distinguish between different kinds of inaccessible memory +like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. For tag-based KASAN this last report section shows the memory tags around the -accessed address (see Implementation details section). +accessed address (see `Implementation details`_ section). + +Boot parameters +~~~~~~~~~~~~~~~ + +Hardware tag-based KASAN mode (see the section about different mode below) is +intended for use in production as a security mitigation. Therefore it supports +boot parameters that allow to disable KASAN competely or otherwise control +particular KASAN features. + +The things that can be controlled are: + +1. Whether KASAN is enabled at all. +2. Whether KASAN collects and saves alloc/free stacks. +3. Whether KASAN panics on a detected bug or not. + +The ``kasan.mode`` boot parameter allows to choose one of three main modes: + +- ``kasan.mode=off`` - KASAN is disabled, no tag checks are performed +- ``kasan.mode=prod`` - only essential production features are enabled +- ``kasan.mode=full`` - all KASAN features are enabled + +The chosen mode provides default control values for the features mentioned +above. However it's also possible to override the default values by providing: + +- ``kasan.stacktrace=off`` or ``=on`` - enable alloc/free stack collection + (default: ``on`` for ``mode=full``, + otherwise ``off``) +- ``kasan.fault=report`` or ``=panic`` - only print KASAN report or also panic + (default: ``report``) + +If ``kasan.mode`` parameter is not provided, it defaults to ``full`` when +``CONFIG_DEBUG_KERNEL`` is enabled, and to ``prod`` otherwise. + +For developers +~~~~~~~~~~~~~~ + +Software KASAN modes use compiler instrumentation to insert validity checks. +Such instrumentation might be incompatible with some part of the kernel, and +therefore needs to be disabled. To disable instrumentation for specific files +or directories, add a line similar to the following to the respective kernel +Makefile: + +- For a single file (e.g. main.o):: + + KASAN_SANITIZE_main.o := n + +- For all files in one directory:: + + KASAN_SANITIZE := n Implementation details @@ -164,10 +208,10 @@ Implementation details Generic KASAN ~~~~~~~~~~~~~ -From a high level, our approach to memory error detection is similar to that -of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to insert checks of shadow -memory on each memory access. +From a high level perspective, KASAN's approach to memory error detection is +similar to that of kmemcheck: use shadow memory to record whether each byte of +memory is safe to access, and use compile-time instrumentation to insert checks +of shadow memory on each memory access. Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and offset to @@ -198,6 +242,9 @@ Generic KASAN also reports the last 2 ca potentially has access to an object. Call stacks for the following are shown: call_rcu() and workqueue queuing. +Generic KASAN is the only mode that delays the reuse of freed object via +quarantine (see mm/kasan/quarantine.c for implementation). + Software tag-based KASAN ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -305,15 +352,15 @@ therefore be wasteful. Furthermore, to e use different shadow pages, mappings would have to be aligned to ``KASAN_GRANULE_SIZE * PAGE_SIZE``. -Instead, we share backing space across multiple mappings. We allocate +Instead, KASAN shares backing space across multiple mappings. It allocates a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. -We hook in to the vmap infrastructure to lazily clean up unused shadow +KASAN hooks into the vmap infrastructure to lazily clean up unused shadow memory. -To avoid the difficulties around swapping mappings around, we expect +To avoid the difficulties around swapping mappings around, KASAN expects that the part of the shadow region that covers the vmalloc space will not be covered by the early shadow page, but will be left unmapped. This will require changes in arch-specific code. @@ -324,24 +371,31 @@ architectures that do not have a fixed m CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE -------------------------------------------------- -``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing. -This means each test focuses on a small unit of functionality and -there are a few ways these tests can be run. +KASAN tests consist on two parts: + +1. Tests that are integrated with the KUnit Test Framework. Enabled with +``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified +automatically in a few different ways, see the instructions below. + +2. Tests that are currently incompatible with KUnit. Enabled with +``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can +only be verified manually, by loading the kernel module and inspecting the +kernel log for KASAN reports. -Each test will print the KASAN report if an error is detected and then -print the number of the test and the status of the test: +Each KUnit-compatible KASAN test prints a KASAN report if an error is detected. +Then the test prints its number and status. -pass:: +When a test passes:: ok 28 - kmalloc_double_kzfree -or, if kmalloc failed:: +When a test fails due to a failed ``kmalloc``:: # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 Expected ptr is not null, but is not ok 4 - kmalloc_large_oob_right -or, if a KASAN report was expected, but not found:: +When a test fails due to a missing KASAN report:: # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 Expected kasan_data->report_expected == kasan_data->report_found, but @@ -349,46 +403,38 @@ or, if a KASAN report was expected, but kasan_data->report_found == 0 not ok 28 - kmalloc_double_kzfree -All test statuses are tracked as they run and an overall status will -be printed at the end:: +At the end the cumulative status of all KASAN tests is printed. On success:: ok 1 - kasan -or:: +Or, if one of the tests failed:: not ok 1 - kasan -(1) Loadable Module -~~~~~~~~~~~~~~~~~~~~ + +There are a few ways to run KUnit-compatible KASAN tests. + +1. Loadable module +~~~~~~~~~~~~~~~~~~ With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as -a loadable module and run on any architecture that supports KASAN -using something like insmod or modprobe. The module is called ``test_kasan``. +a loadable module and run on any architecture that supports KASAN by loading +the module with insmod or modprobe. The module is called ``test_kasan``. -(2) Built-In -~~~~~~~~~~~~~ +2. Built-In +~~~~~~~~~~~ With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in -on any architecture that supports KASAN. These and any other KUnit -tests enabled will run and print the results at boot as a late-init -call. - -(3) Using kunit_tool -~~~~~~~~~~~~~~~~~~~~~ - -With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also -use kunit_tool to see the results of these along with other KUnit -tests in a more readable way. This will not print the KASAN reports -of tests that passed. Use `KUnit documentation `_ for more up-to-date -information on kunit_tool. +on any architecure that supports KASAN. These and any other KUnit tests enabled +will run and print the results at boot as a late-init call. -.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html +3. Using kunit_tool +~~~~~~~~~~~~~~~~~~~ -``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be -converted to KUnit. These tests can be run only as a module with -``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and -``CONFIG_KASAN`` built-in. The type of error expected and the -function being run is printed before the expression expected to give -an error. Then the error is printed, if found, and that test -should be interpreted to pass only if the error was the one expected -by the test. +With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also +possible use ``kunit_tool`` to see the results of these and other KUnit tests +in a more readable way. This will not print the KASAN reports of the tests that +passed. Use `KUnit documentation `_ +for more up-to-date information on ``kunit_tool``. + +.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html