From patchwork Fri Aug 14 17:26:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11715027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C39A618 for ; Fri, 14 Aug 2020 17:29:37 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 15E4C20708 for ; Fri, 14 Aug 2020 17:29:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="kOeiJUTm"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Xg76fkW8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15E4C20708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:Mime-Version:Message-Id:Date: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=dS6W2zeMP/zVhG+Z645P//ZOKsot6hUZHaNzNgq2Fto=; b=kOeiJUTmsLZIS2thrXw47rghGG wbFkMYAnIb0Uvvu/0RBqmiLg2RFvJwGk9zcQCEvQKPmN/0eDvHm6TYajay5tY/91WkgcBabEFyROG rT5mltPtAAy6T4ZK2xv825u8zN018Cp3CNSWQrCfL0sqoGoBe6Mh39wqDsUunVQJTdxvVxOr9qrKZ OydeQ9o3miGHqh6FTXQOBPCIhB51Q9LciPZFS+f6GdABPCIG10uVv4dnx2x9YQCSCFIbDPfMNpIlk HQMvz5f5vjI8kvcXXyxm4t/GcnaI+zHQpELwfiqx2yrrCKffSmlblik7bln51WgZ1R1BJuLz61J4f bPijolEQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k6dUK-0003Bf-Kc; Fri, 14 Aug 2020 17:27:32 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k6dUF-00037C-Fv for linux-arm-kernel@lists.infradead.org; Fri, 14 Aug 2020 17:27:29 +0000 Received: by mail-qt1-x849.google.com with SMTP id m13so7430530qth.16 for ; Fri, 14 Aug 2020 10:27:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=0cNgcsyuYAwNS3ucnNThPdPKsO+dwuJKPiI6RmFliVA=; b=Xg76fkW8M2c7cn589Bvqn4szeZLaNA2SA6S32k9aRrLPAsDykQkJ0MRoFqGRyre7my rEpWakxnYZIxWRwUsUc/30YkVzIWQ+T69BnOx2nvTXmeuA7reFa5uT6tNd/O0Ifow/oa 5o70At9eIqXqDvNI38meVhkkQx+DAOiwwQcbgZeJpZsm38Q+Krwiya1dctIAHJllgV33 xlrhFFMmkci+YzLbgz8eUQkpHFu2XiVBIiSamjonwPvLvga604RDymxQm3+QOdvexAqM QPPpozO0r+/IrskYJiqnGN2AoUZ4NdZu7PskxH/DHVviMMe5RI6zQNK8DQbikhFb1Czo PoWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=0cNgcsyuYAwNS3ucnNThPdPKsO+dwuJKPiI6RmFliVA=; b=MXUvNDnX8QswZxUQ/3Ka/HMNHE0wNq8uiojTo/daK5qLyriuisV/f0MJdfOARBITLO V8PIDR8fDc7qYctEM6LCUfwXckt/O48nuYC7PAS1upUA8IrQN+PtxwpwTBb1MS1Lz8Ow mQv5Z3as6BkY4Qz3M9fgMmuw8IexjqK6wfvxQaIr58uM02q97B44FFn6b0fhwyC3pdiv Brg/XEPjGMj1pLM082FynY6FidHpOA3D+kvYmgfmmFz0CxRFdBN2NbEaulgK4A2d1H7+ nL3AY9TWAseM0Gz3bbs0k1lexKOG1rTwdRtz0hgC9XALF9jmdIqbvIPfO45ydE5wTJ5K u8xg== X-Gm-Message-State: AOAM532qgRHnyN7u7HM5JUfFa7ci0CpkhSOU71QD6HoZ1GaJ6qyEMtjz y3Zco04iwbooyTrf6w3SVasRZyXvjQEhvmFy X-Google-Smtp-Source: ABdhPJyJACKOly1+LVDe6Gou4x4jDxqWHURgjAjdmnX4i5puSGRaag6KUTy9s/R0iOXjyYBs7alYuVsgM/G/zkze X-Received: by 2002:ad4:4c0a:: with SMTP id bz10mr3506370qvb.78.1597426043462; Fri, 14 Aug 2020 10:27:23 -0700 (PDT) Date: Fri, 14 Aug 2020 19:26:42 +0200 Message-Id: Mime-Version: 1.0 X-Mailer: git-send-email 2.28.0.220.ged08abb693-goog Subject: [PATCH 00/35] kasan: add hardware tag-based mode for arm64 From: Andrey Konovalov To: Dmitry Vyukov , Vincenzo Frascino , Catalin Marinas , kasan-dev@googlegroups.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200814_132727_633599_384D67B4 X-CRM114-Status: GOOD ( 26.00 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:849 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marco Elver , Elena Petrova , Andrey Konovalov , Kevin Brodsky , Will Deacon , Branislav Rankov , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Potapenko , linux-arm-kernel@lists.infradead.org, Andrey Ryabinin , Andrew Morton , Evgenii Stepanov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This patchset adds a new hardware tag-based mode to KASAN [1]. The new mode is similar to the existing software tag-based KASAN, but relies on arm64 Memory Tagging Extension (MTE) [2] to perform memory and pointer tagging (instead of shadow memory and compiler instrumentation). This patchset is available here: https://github.com/xairy/linux/tree/up-kasan-mte-v1 and has also been uploaded to the Linux kernel Gerrit instance: https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/2700 This patchset is based on the v7 of the user MTE patches [3], along with some KASAN patches cherry-picked from the current mainline. This patchset consists of three parts: 1. Rework KASAN code to allow easier integration of the hardware tag-based mode. 2. Introduce core in-kernel MTE routines. 3. Combine the two parts together and introduce the new mode. For testing in QEMU hardware tag-based KASAN requires: 1. QEMU built from master [4] with these patches [5] on top. 2. GCC version 10. [1] https://www.kernel.org/doc/html/latest/dev-tools/kasan.html [2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety [3] git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/mte [4] https://github.com/qemu/qemu [5] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg02677.html ====== Overview The underlying ideas of the approach used by hardware tag-based KASAN are: 1. By relying on the Top Byte Ignore (TBI) arm64 CPU feature, pointer tags are stored in the top byte of each kernel pointer. 2. With the Memory Tagging Extension (MTE) arm64 CPU feature, memory tags for kernel memory allocations are stored in a dedicated memory not accessible via normal instuctions. 3. On each memory allocation, a random tag is generated, embedded it into the returned pointer, and the corresponding memory is tagged with the same tag value. 4. With MTE the CPU performs a check on each memory access to make sure that the pointer tag matches the memory tag. 5. On a tag mismatch the CPU generates a tag fault, and a KASAN report is printed. Same as other KASAN modes, hardware tag-based KASAN is intended as a debugging feature at this point. ====== Rationale There are two main reasons for this new hardware tag-based mode: 1. Previously implemented software tag-based KASAN is being successfully used on dogfood testing devices due to its low memory overhead (as initially planned). The new hardware mode keeps the same low memory overhead, and is expected to have significantly lower performance impact, due to the tag checks being performed by the hardware. Therefore the new mode can be used as a better alternative in dogfood testing for hardware that supports MTE. 2. The new mode lays the groundwork for the planned in-kernel MTE-based memory corruption mitigation to be used in production. ====== Technical details From the implementation perspective, hardware tag-based KASAN is almost identical to the software mode. The key difference is using MTE for assigning and checking tags. Compared to the software mode, the hardware mode uses 4 bits per tag, as dictated by MTE. Pointer tags are stored in bits [56:60), the top 4 bits have the normal value 0xF. Having less distict tags increases the probablity of false negatives (from ~1/256 to ~1/16) in certain cases. Only synchronous exceptions are set up and used by hardware tag-based KASAN. ====== Benchmarks Note: all measurements have been performed with software emulation of Memory Tagging Extension, performance numbers for hardware tag-based KASAN on the actual hardware are expected to be better. Boot time [1]: * 2.8 sec for clean kernel * 5.7 sec for hardware tag-based KASAN * 11.8 sec for software tag-based KASAN * 11.6 sec for generic KASAN Slab memory usage after boot [2]: * 7.0 kb for clean kernel * 9.7 kb for hardware tag-based KASAN * 9.7 kb for software tag-based KASAN * 41.3 kb for generic KASAN Measurements have been performed with: * defconfig-based configs * Manually built QEMU master with these patches on top [3] * QEMU arguments: -machine virt,mte=on -cpu max * CONFIG_KASAN_STACK_ENABLE disabled * CONFIG_KASAN_INLINE enabled * clang-10 as the compiler and gcc-10 as the assembler [1] Time before the ext4 driver is initialized. [2] Measured as `cat /proc/meminfo | grep Slab`. [3] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg02677.html ====== Notes The cover letter for software tag-based KASAN patchset can be found here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0116523cfffa62aeb5aa3b85ce7419f3dae0c1b8 Andrey Konovalov (29): kasan: KASAN_VMALLOC depends on KASAN_GENERIC kasan: group vmalloc code kasan: shadow declarations only for software modes kasan: rename (un)poison_shadow to (un)poison_memory kasan: rename KASAN_SHADOW_* to KASAN_GRANULE_* kasan: only build init.c for software modes kasan: split out shadow.c from common.c kasan: rename generic/tags_report.c files kasan: don't duplicate config dependencies kasan: hide invalid free check implementation kasan: decode stack frame only with KASAN_STACK_ENABLE kasan, arm64: only init shadow for software modes kasan, arm64: only use kasan_depth for software modes kasan: rename addr_has_shadow to addr_has_metadata kasan: rename print_shadow_for_address to print_memory_metadata kasan: kasan_non_canonical_hook only for software modes kasan: rename SHADOW layout macros to META kasan: separate metadata_fetch_row for each mode kasan: don't allow SW_TAGS with ARM64_MTE kasan: introduce CONFIG_KASAN_HW_TAGS kasan, arm64: align allocations for HW_TAGS kasan: define KASAN_GRANULE_SIZE for HW_TAGS kasan, x86, s390: update undef CONFIG_KASAN kasan, arm64: expand CONFIG_KASAN checks kasan, arm64: implement HW_TAGS runtime kasan, arm64: print report from tag fault handler kasan, slub: reset tags when accessing metadata kasan, arm64: enable CONFIG_KASAN_HW_TAGS kasan: add documentation for hardware tag-based mode Vincenzo Frascino (6): arm64: mte: Add in-kernel MTE helpers arm64: mte: Add in-kernel tag fault handler arm64: mte: Enable in-kernel MTE arm64: mte: Convert gcr_user into an exclude mask arm64: mte: Switch GCR_EL1 in kernel entry and exit kasan, arm64: Enable TBI EL1 Documentation/dev-tools/kasan.rst | 75 ++- arch/arm64/Kconfig | 3 +- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/assembler.h | 2 +- arch/arm64/include/asm/cache.h | 3 + arch/arm64/include/asm/esr.h | 1 + arch/arm64/include/asm/kasan.h | 8 +- arch/arm64/include/asm/memory.h | 6 +- arch/arm64/include/asm/mte.h | 54 ++- arch/arm64/include/asm/mte_asm.h | 10 + arch/arm64/include/asm/processor.h | 2 +- arch/arm64/include/asm/string.h | 5 +- arch/arm64/kernel/asm-offsets.c | 3 + arch/arm64/kernel/cpufeature.c | 11 +- arch/arm64/kernel/entry.S | 28 ++ arch/arm64/kernel/head.S | 2 +- arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kernel/mte.c | 87 +++- arch/arm64/kernel/setup.c | 1 - arch/arm64/lib/mte.S | 41 ++ arch/arm64/mm/dump.c | 6 +- arch/arm64/mm/fault.c | 59 ++- arch/arm64/mm/kasan_init.c | 22 +- arch/arm64/mm/proc.S | 2 +- arch/s390/boot/string.c | 1 + arch/x86/boot/compressed/misc.h | 1 + include/linux/kasan-checks.h | 2 +- include/linux/kasan.h | 110 +++-- include/linux/mm.h | 2 +- include/linux/moduleloader.h | 3 +- include/linux/page-flags-layout.h | 2 +- include/linux/sched.h | 2 +- include/linux/string.h | 2 +- init/init_task.c | 2 +- kernel/fork.c | 4 +- lib/Kconfig.kasan | 60 ++- lib/test_kasan.c | 2 +- mm/kasan/Makefile | 21 +- mm/kasan/common.c | 554 ++-------------------- mm/kasan/generic.c | 33 +- mm/kasan/generic_report.c | 165 ------- mm/kasan/init.c | 10 +- mm/kasan/kasan.h | 45 +- mm/kasan/mte.c | 76 +++ mm/kasan/report.c | 254 ++-------- mm/kasan/report_generic.c | 331 +++++++++++++ mm/kasan/report_mte.c | 47 ++ mm/kasan/{tags_report.c => report_tags.c} | 7 +- mm/kasan/shadow.c | 509 ++++++++++++++++++++ mm/kasan/tags.c | 14 +- mm/page_poison.c | 2 +- mm/ptdump.c | 13 +- mm/slab_common.c | 2 +- mm/slub.c | 25 +- scripts/Makefile.lib | 2 + 55 files changed, 1653 insertions(+), 1085 deletions(-) create mode 100644 arch/arm64/include/asm/mte_asm.h delete mode 100644 mm/kasan/generic_report.c create mode 100644 mm/kasan/mte.c create mode 100644 mm/kasan/report_generic.c create mode 100644 mm/kasan/report_mte.c rename mm/kasan/{tags_report.c => report_tags.c} (94%) create mode 100644 mm/kasan/shadow.c