From patchwork Fri Oct 18 09:42:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11197997 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF04913B1 for ; Fri, 18 Oct 2019 09:44:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE99420820 for ; Fri, 18 Oct 2019 09:44:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dHJwyo4n" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE99420820 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 794DB8E002B; Fri, 18 Oct 2019 05:44:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 71DB38E0003; Fri, 18 Oct 2019 05:44:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60A398E002B; Fri, 18 Oct 2019 05:44:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3596D8E0003 for ; Fri, 18 Oct 2019 05:44:02 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id D80C1A2D0 for ; Fri, 18 Oct 2019 09:44:01 +0000 (UTC) X-FDA: 76056419082.20.hand70_7a6ed3fafca0c X-Spam-Summary: 10,1,0,175d4173fd21541a,d41d8cd98f00b204,3yimpxqykca0tyvqr4t11tyr.p1zyv07a-zzx8npx.14t@flex--glider.bounces.google.com,:glider@google.com:axboe@kernel.dk:luto@kernel.org:vegard.nossum@oracle.com:dvyukov@google.com:,RULES_HIT:1:41:152:334:355:379:541:617:800:960:973:982:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1690:1730:1747:1777:1792:1801:2197:2198:2199:2200:2393:2553:2559:2562:2637:2897:2904:3138:3139:3140:3141:3142:3152:3653:3865:3866:3867:3868:3870:3871:4250:4321:4605:5007:6119:6120:6261:6653:7875:7901:7903:7974:8603:8660:8784:9969:10004:11026:11232:11473:11657:11658:11914:12043:12296:12297:12438:12555:12895:12986:13146:13148:13230:13846:14394:14659:14877:21080:21220:21221:21365:21433:21444:21451:21611:21627:21740:21795:21966:30003:30051:30054:30064:30069:30070:30080:30090,0,RBL:209.85.222.202:@flex--glider.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Domain Cache:0, X-HE-Tag: hand70_7a6ed3fafca0c X-Filterd-Recvd-Size: 13791 Received: from mail-qk1-f202.google.com (mail-qk1-f202.google.com [209.85.222.202]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Fri, 18 Oct 2019 09:44:01 +0000 (UTC) Received: by mail-qk1-f202.google.com with SMTP id c4so390210qkg.22 for ; Fri, 18 Oct 2019 02:44:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wzjhPFLgc4dnmBIO1rbgzyvtsjkcRh993a9RjYmhJo0=; b=dHJwyo4nrVc46fqMRsanExMWZtTOE9bZ6QQn990dE2mogazVXL84AUfIDE4z71ueLt vKDO2tMjXPhZF6Ehn1kQzfgBap/9+fqmK7QvHsrVVA37Ud4zxCwOh8RmJ6ER7FpUJJCy 30ZFUMXaf5Qaat6aX2uiEW/zw/ciJJI4k1EOZ3vRgLaU/9jeUzLsEp6N6g0/0hAw8hrY VjORpF//e4faHlQvdm8xkzWH8M2IUwmCBdodugjhfv4PnIj3RNxKzgoeGPORKYatQ5yz 19E8PBQH8eohAH0BXH9iJAAxr35hWO+klbNQ0s6vGHve/IdH6aj07tD8Ez2eZkqPHMoe Yszg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wzjhPFLgc4dnmBIO1rbgzyvtsjkcRh993a9RjYmhJo0=; b=iNEUawvkN6cTKwKGmoX/uiLqdKC7RRyfQjZfSlDTe/j6z1NKEhn6p5vAcFt9UKbaju wT/acfoV19FDPCc9gngm5EeAdr6WAf9vIx2XRmFX3I434BmRsRseaVxaqBNdn79N4Vq1 Fm0R/dsBPgdno51DPwgSxK6Y5fN4YkIYq/+STOs7ra+ZdXMDVM1MPi9m+aT4SHELtdAE Fm646nl1F4pwzCtRjMJARK/nPPp9U2UuxK4vi+sdZ79T1+ytsPUczfgF7aO6ocZqtbLW nJD4MT41UZQ/WJduoMyMK8HRXZRuyMUi6Jsw/YyxEBDPxTpF+SVtXUocRj0rSrWlQez6 QuJg== X-Gm-Message-State: APjAAAXZJK3CbBmypv3uwrhrVK/B0MH7D9E3s7m7dpIL+IMOi1LTWB4c VQKjL4UDvyBhYL6CtLwuC8qY0hAaPMA= X-Google-Smtp-Source: APXvYqwxZZ1jrItTC477pbeNJSNjQfHcElRa4bN9IKVhHx0k6ZwSMEioykWGZc8/zd05MEQnu/hAX2dbn5Y= X-Received: by 2002:a0c:f792:: with SMTP id s18mr8669465qvn.20.1571391840593; Fri, 18 Oct 2019 02:44:00 -0700 (PDT) Date: Fri, 18 Oct 2019 11:42:54 +0200 In-Reply-To: <20191018094304.37056-1-glider@google.com> Message-Id: <20191018094304.37056-17-glider@google.com> Mime-Version: 1.0 References: <20191018094304.37056-1-glider@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH RFC v1 16/26] kmsan: Changing existing files to enable KMSAN builds From: glider@google.com To: Alexander Potapenko Cc: Jens Axboe , Andy Lutomirski , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do the following to make KMSAN usable: - add KMSAN declarations to struct page and struct task_struct; - add Makefile.kmsan to top-level Makefile; - call KMSAN initialization from init/main.c - add asm hooks to arch/x86/entry/entry_64.S; - increase task stack size under KMSAN; - disable page merging in block/blk.h; - disable CONFIG_KMSAN in arch/x86/boot/compressed/misc.h to use default string functions instead of KMSAN ones. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Jens Axboe Cc: Andy Lutomirski Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I37e0b7f2d2f2b0aeac5753ff9d6b411485fc374e --- Makefile | 3 ++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/entry/entry_64.S | 18 ++++++++++++++++++ arch/x86/include/asm/page_64_types.h | 12 +++++++----- block/blk.h | 7 +++++++ include/linux/mm_types.h | 9 +++++++++ include/linux/sched.h | 5 +++++ init/main.c | 3 +++ lib/stackdepot.c | 7 ++++--- mm/Makefile | 1 + scripts/Makefile.lib | 6 ++++++ 11 files changed, 63 insertions(+), 9 deletions(-) diff --git a/Makefile b/Makefile index ffd7a912fc46..5041e908f051 100644 --- a/Makefile +++ b/Makefile @@ -478,7 +478,7 @@ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE -export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN +export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KMSAN export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL @@ -898,6 +898,7 @@ KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none) endif include scripts/Makefile.kasan +include scripts/Makefile.kmsan include scripts/Makefile.extrawarn include scripts/Makefile.ubsan diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index c8181392f70d..dd4bd8c5d97a 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -12,6 +12,7 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN +#undef CONFIG_KMSAN /* cpu_feature_enabled() cannot be used this early */ #define USE_EARLY_PGTABLE_L5 diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index b7c3ea4cb19d..1218672bad85 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -168,12 +169,14 @@ GLOBAL(entry_SYSCALL_64_after_hwframe) PUSH_AND_CLEAR_REGS rax=$-ENOSYS TRACE_IRQS_OFF + KMSAN_SYSCALL_ENTER /* IRQs are off. */ movq %rax, %rdi movq %rsp, %rsi call do_syscall_64 /* returns with IRQs disabled */ + KMSAN_SYSCALL_EXIT TRACE_IRQS_IRETQ /* we're about to change IF */ /* @@ -575,6 +578,7 @@ ENTRY(interrupt_entry) 1: ENTER_IRQ_STACK old_rsp=%rdi save_ret=1 + KMSAN_INTERRUPT_ENTER /* We entered an interrupt context - irqs are off: */ TRACE_IRQS_OFF @@ -604,12 +608,14 @@ common_interrupt: addq $-0x80, (%rsp) /* Adjust vector to [-256, -1] range */ call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call do_IRQ /* rdi points to pt_regs */ /* 0(%rsp): old RSP */ ret_from_intr: DISABLE_INTERRUPTS(CLBR_ANY) TRACE_IRQS_OFF + KMSAN_INTERRUPT_EXIT LEAVE_IRQ_STACK testb $3, CS(%rsp) @@ -802,6 +808,7 @@ ENTRY(\sym) .Lcommon_\sym: call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call \do_sym /* rdi points to pt_regs */ jmp ret_from_intr END(\sym) @@ -909,15 +916,18 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt .if \shift_ist != -1 subq $\ist_offset, CPU_TSS_IST(\shift_ist) + KMSAN_IST_ENTER(\shift_ist) .endif .if \read_cr2 movq %r12, %rdx /* Move CR2 into 3rd argument */ .endif + KMSAN_UNPOISON_PT_REGS call \do_sym .if \shift_ist != -1 + KMSAN_IST_EXIT(\shift_ist) addq $\ist_offset, CPU_TSS_IST(\shift_ist) .endif @@ -1079,7 +1089,9 @@ ENTRY(do_softirq_own_stack) pushq %rbp mov %rsp, %rbp ENTER_IRQ_STACK regs=0 old_rsp=%r11 + KMSAN_SOFTIRQ_ENTER call __do_softirq + KMSAN_SOFTIRQ_EXIT LEAVE_IRQ_STACK regs=0 leaveq ret @@ -1466,9 +1478,12 @@ ENTRY(nmi) * done with the NMI stack. */ + KMSAN_NMI_ENTER movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* * Return back to user mode. We must *not* do the normal exit @@ -1678,10 +1693,13 @@ end_repeat_nmi: call paranoid_entry UNWIND_HINT_REGS + KMSAN_NMI_ENTER /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */ movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* Always restore stashed CR3 value (see paranoid_entry) */ RESTORE_CR3 scratch_reg=%r15 save_reg=%r14 diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h index 288b065955b7..ea9fbf09f43b 100644 --- a/arch/x86/include/asm/page_64_types.h +++ b/arch/x86/include/asm/page_64_types.h @@ -7,18 +7,20 @@ #endif #ifdef CONFIG_KASAN -#define KASAN_STACK_ORDER 1 +#define EXTRA_STACK_ORDER 1 +#elif defined(CONFIG_KMSAN) +#define EXTRA_STACK_ORDER 2 #else -#define KASAN_STACK_ORDER 0 +#define EXTRA_STACK_ORDER 0 #endif -#define THREAD_SIZE_ORDER (2 + KASAN_STACK_ORDER) +#define THREAD_SIZE_ORDER (2 + EXTRA_STACK_ORDER) #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) -#define EXCEPTION_STACK_ORDER (0 + KASAN_STACK_ORDER) +#define EXCEPTION_STACK_ORDER (0 + EXTRA_STACK_ORDER) #define EXCEPTION_STKSZ (PAGE_SIZE << EXCEPTION_STACK_ORDER) -#define IRQ_STACK_ORDER (2 + KASAN_STACK_ORDER) +#define IRQ_STACK_ORDER (2 + EXTRA_STACK_ORDER) #define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER) /* diff --git a/block/blk.h b/block/blk.h index 47fba9362e60..9ee271a22423 100644 --- a/block/blk.h +++ b/block/blk.h @@ -78,6 +78,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q, phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset; phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset; +#ifdef CONFIG_KMSAN + /* + * Merging consequent physical pages may not work correctly under KMSAN + * if their metadata pages aren't consequent. Just disable merging. + */ + return false; +#endif if (addr1 + vec1->bv_len != addr2) return false; if (xen_domain() && !xen_biovec_phys_mergeable(vec1, vec2->bv_page)) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 2222fa795284..c87c5416a802 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -216,6 +216,15 @@ struct page { not kmapped, ie. highmem) */ #endif /* WANT_PAGE_VIRTUAL */ +#ifdef CONFIG_KMSAN + /* + * Bits in struct page are scarce, so the LSB in *shadow is used to + * indicate whether the page should be ignored by KMSAN or not. + */ + struct page *shadow; + struct page *origin; +#endif + #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS int _last_cpupid; #endif diff --git a/include/linux/sched.h b/include/linux/sched.h index 7417e0207b48..e0f6e95c22ed 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -1172,6 +1173,10 @@ struct task_struct { unsigned int kasan_depth; #endif +#ifdef CONFIG_KMSAN + struct kmsan_task_state kmsan; +#endif + #ifdef CONFIG_FUNCTION_GRAPH_TRACER /* Index of current stored address in ret_stack: */ int curr_ret_stack; diff --git a/init/main.c b/init/main.c index 91f6ebb30ef0..afcca2a38348 100644 --- a/init/main.c +++ b/init/main.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include #include @@ -554,6 +555,7 @@ static void __init mm_init(void) */ page_ext_init_flatmem(); report_meminit(); + kmsan_initialize_shadow(); mem_init(); kmem_cache_init(); kmemleak_init(); @@ -625,6 +627,7 @@ asmlinkage __visible void __init start_kernel(void) sort_main_extable(); trap_init(); mm_init(); + kmsan_initialize(); ftrace_init(); diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 460efd3ef742..529a86eb1dd1 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -233,8 +233,8 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (parts.slabindex > depot_index) { WARN(1, "slab index %d out of bounds (%d) for stack id %08x\n", parts.slabindex, depot_index, handle); - __memset(trace, 0, sizeof(*trace)); - return; + *entries = NULL; + return 0; } slab = stack_slabs[parts.slabindex]; stack = slab + offset; @@ -299,7 +299,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, * contexts and I/O. */ alloc_flags &= ~GFP_ZONEMASK; - alloc_flags &= (GFP_ATOMIC | GFP_KERNEL); + alloc_flags &= (GFP_ATOMIC | GFP_KERNEL | + __GFP_NO_KMSAN_SHADOW); alloc_flags |= __GFP_NOWARN; page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER); if (page) diff --git a/mm/Makefile b/mm/Makefile index d996846697ef..419e6e02dfaf 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -71,6 +71,7 @@ obj-$(CONFIG_PAGE_POISONING) += page_poison.o obj-$(CONFIG_SLAB) += slab.o obj-$(CONFIG_SLUB) += slub.o obj-$(CONFIG_KASAN) += kasan/ +obj-$(CONFIG_KMSAN) += kmsan/ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_MEMTEST) += memtest.o diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 179d55af5852..f9f38f7c1cd4 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -140,6 +140,12 @@ _c_flags += $(if $(patsubst n%,, \ $(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE)) endif +ifeq ($(CONFIG_KMSAN),y) +_c_flags += $(if $(patsubst n%,, \ + $(KMSAN_SANITIZE_$(basetarget).o)$(KMSAN_SANITIZE)y), \ + $(CFLAGS_KMSAN)) +endif + ifeq ($(CONFIG_UBSAN),y) _c_flags += $(if $(patsubst n%,, \ $(UBSAN_SANITIZE_$(basetarget).o)$(UBSAN_SANITIZE)$(CONFIG_UBSAN_SANITIZE_ALL)), \