From patchwork Fri Dec 20 18:49:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11306153 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D805139A for ; Fri, 20 Dec 2019 18:50:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F3C60206DA for ; Fri, 20 Dec 2019 18:50:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="R93bydj9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3C60206DA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A1E5B8E01AD; Fri, 20 Dec 2019 13:50:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F49F8E019D; Fri, 20 Dec 2019 13:50:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 90ABD8E01AD; Fri, 20 Dec 2019 13:50:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0059.hostedemail.com [216.40.44.59]) by kanga.kvack.org (Postfix) with ESMTP id 79B8C8E019D for ; Fri, 20 Dec 2019 13:50:34 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 38C1B52D9 for ; Fri, 20 Dec 2019 18:50:34 +0000 (UTC) X-FDA: 76286410788.24.crush16_7951607c9b42c X-Spam-Summary: 2,0,0,c53849cd67485154,d41d8cd98f00b204,3-bf9xqykcbmz41wxaz77z4x.v75416dg-553etv3.7az@flex--glider.bounces.google.com,:axboe@kernel.dk:luto@kernel.org:wsa@the-dreams.de:hch@lst.de:vegard.nossum@oracle.com:dvyukov@google.com:elver@google.com:andreyknvl@google.com::glider@google.com:viro@zeniv.linux.org.uk:adilger.kernel@dilger.ca:akpm@linux-foundation.org:aryabinin@virtuozzo.com:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@infradead.org:darrick.wong@oracle.com:davem@davemloft.net:dmitry.torokhov@gmail.com:ebiggers@google.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:iii@linux.ibm.com:mingo@elte.hu:jasowang@redhat.com:m.szyprowski@samsung.com:mark.rutland@arm.com:martin.petersen@oracle.com:schwidefsky@de.ibm.com:willy@infradead.org:mst@redhat.com:mhocko@suse.com:monstr@monstr.eu:pmladek@suse.com:cai@lca.pw:rdunlap@infradead.org:robin.murphy@arm.com:sergey.senozhatsky@gmail.com:rostedt@goodmi s.org:ti X-HE-Tag: crush16_7951607c9b42c X-Filterd-Recvd-Size: 11430 Received: from mail-vk1-f201.google.com (mail-vk1-f201.google.com [209.85.221.201]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Fri, 20 Dec 2019 18:50:33 +0000 (UTC) Received: by mail-vk1-f201.google.com with SMTP id 198so4314033vkz.9 for ; Fri, 20 Dec 2019 10:50:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=U2J1dD/ZXcniGKT5OYyEH3nV07OVYnSE+3Bb6+bjCsE=; b=R93bydj9/p0wkWkJp2na66k2CLfBvBuXRwTTRju+TChtiTS+lEHW/BlbuJbQ9/VWDG Os6sUJJib4VJmkpuXX08aqoXLjIrQ15D/cAxbtuu0k8nHPHweEIITQzzeDE/0PaNkvlE Xe+/Bx9jiPKErDaF8yrtUVSsb379g7Silw25cfWIIC9QTYubtjWeoW3GllCB+dxyL6Qs oEn/BjlWisej8NAst0RO4jjDKMF2VGwLj9Qv7YLTw6Gi5N9iXx25QnE8yjv2gcEm3e30 JiXcdLf63xDXtgghLrIHpbhZP20Rbn+NN/DzDk7g50wBKTWbycYIznes1bPpwquofX7Z tO8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=U2J1dD/ZXcniGKT5OYyEH3nV07OVYnSE+3Bb6+bjCsE=; b=hBQy01IiNqzDViXe3IDeua7nNjkoley3DPT4wybmezo1nUgAiTvvngxps2InK5e6r+ HClFXHdTz2aAKw9e0y0iMExupwmZjvCJbuIKRf/EAGxBoXCKaFVWsifO07q3H1pRc4wa B023r7cS5NP4PbCK7bZPsvt48ygAohpZ2BqStGBZ5BpAjTNfU1/LX1bg8VLKLDw3rJp0 gqqdk33xF1fKYfvgI+swgWJiAvPdn84ft6DdQlLWYIgzPGkDzB9HAx76ltAug50SKXke uvenBX8ry8uWxT2VBWgUDmkOqvOFVkaP09pqVHrBmhlEIenVGKmayQhcDRNjO0yx8i4U sogQ== X-Gm-Message-State: APjAAAVTeDkCKk5uFgeR/nc7NLo8YxNUQO3oPfmSsfKTOlmydBLCPQjh lqe06PQ+ECGBDUKeks42yVo6mc5n8Xg= X-Google-Smtp-Source: APXvYqy4TANZRc7b/nIjki5uDWFduvol008/KiHCSvLWXK4FkQPiOCW7rmuYlakM7y3N/ws8+Xplw1xy54M= X-Received: by 2002:ab0:738c:: with SMTP id l12mr10259321uap.135.1576867832869; Fri, 20 Dec 2019 10:50:32 -0800 (PST) Date: Fri, 20 Dec 2019 19:49:23 +0100 In-Reply-To: <20191220184955.223741-1-glider@google.com> Message-Id: <20191220184955.223741-11-glider@google.com> Mime-Version: 1.0 References: <20191220184955.223741-1-glider@google.com> X-Mailer: git-send-email 2.24.1.735.g03f4e72817-goog Subject: [PATCH RFC v4 10/42] kmsan: KMSAN compiler API implementation From: glider@google.com To: Jens Axboe , Andy Lutomirski , Wolfram Sang , Christoph Hellwig , Vegard Nossum , Dmitry Vyukov , Marco Elver , Andrey Konovalov , linux-mm@kvack.org Cc: glider@google.com, viro@zeniv.linux.org.uk, adilger.kernel@dilger.ca, akpm@linux-foundation.org, aryabinin@virtuozzo.com, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@infradead.org, darrick.wong@oracle.com, davem@davemloft.net, dmitry.torokhov@gmail.com, ebiggers@google.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, iii@linux.ibm.com, mingo@elte.hu, jasowang@redhat.com, m.szyprowski@samsung.com, mark.rutland@arm.com, martin.petersen@oracle.com, schwidefsky@de.ibm.com, willy@infradead.org, mst@redhat.com, mhocko@suse.com, monstr@monstr.eu, pmladek@suse.com, cai@lca.pw, rdunlap@infradead.org, robin.murphy@arm.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, gor@linux.ibm.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: kmsan_instr.c contains the functions called by KMSAN instrumentation. These include functions that: - return shadow/origin pointers for memory accesses; - poison and unpoison local variables; - provide KMSAN context state to pass metadata for function arguments; - perform string operations (mem*) on metadata; - tell KMSAN to report an error. This patch has been split away from the rest of KMSAN runtime to simplify the review process. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Jens Axboe Cc: Andy Lutomirski Cc: Wolfram Sang Cc: Christoph Hellwig Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Marco Elver Cc: Andrey Konovalov Cc: linux-mm@kvack.org --- v4: - split this patch away as requested by Andrey Konovalov - removed redundant address checks when copying shadow - fix __msan_memmove prototype Change-Id: I826272ed2ebe8ab8ef61a9d4cccdcf07c7b6b499 --- mm/kmsan/kmsan_instr.c | 229 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 229 insertions(+) create mode 100644 mm/kmsan/kmsan_instr.c diff --git a/mm/kmsan/kmsan_instr.c b/mm/kmsan/kmsan_instr.c new file mode 100644 index 000000000000..0de8aafac510 --- /dev/null +++ b/mm/kmsan/kmsan_instr.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN compiler API. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include "kmsan.h" +#include +#include + +static bool is_bad_asm_addr(void *addr, u64 size, bool is_store) +{ + if ((u64)addr < TASK_SIZE) + return true; + if (!kmsan_get_metadata(addr, size, META_SHADOW)) + return true; + return false; +} + +struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, u64 size) +{ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n); + +struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, u64 size) +{ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n); + +#define DECLARE_METADATA_PTR_GETTER(size) \ +struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size(void *addr) \ +{ \ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); \ +} \ +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \ + \ +struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size(void *addr) \ +{ \ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); \ +} \ +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size) + +DECLARE_METADATA_PTR_GETTER(1); +DECLARE_METADATA_PTR_GETTER(2); +DECLARE_METADATA_PTR_GETTER(4); +DECLARE_METADATA_PTR_GETTER(8); + +void __msan_instrument_asm_store(void *addr, u64 size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + /* + * Most of the accesses are below 32 bytes. The two exceptions so far + * are clwb() (64 bytes) and FPU state (512 bytes). + * It's unlikely that the assembly will touch more than 512 bytes. + */ + if (size > 512) { + WARN_ONCE(1, "assembly store size too big: %d\n", size); + size = 8; + } + if (is_bad_asm_addr(addr, size, /*is_store*/true)) + return; + irq_flags = kmsan_enter_runtime(); + /* Unpoisoning the memory on best effort. */ + kmsan_internal_unpoison_shadow(addr, size, /*checked*/false); + kmsan_leave_runtime(irq_flags); +} +EXPORT_SYMBOL(__msan_instrument_asm_store); + +void *__msan_memmove(void *dst, const void *src, size_t n) +{ + void *result; + + result = __memmove(dst, src, n); + if (!n) + /* Some people call memmove() with zero length. */ + return result; + if (!kmsan_ready || kmsan_in_runtime()) + return result; + + kmsan_memmove_metadata(dst, (void *)src, n); + + return result; +} +EXPORT_SYMBOL(__msan_memmove); + +void *__msan_memmove_nosanitize(void *dst, void *src, u64 n) +{ + return __memmove(dst, src, n); +} +EXPORT_SYMBOL(__msan_memmove_nosanitize); + +void *__msan_memcpy(void *dst, const void *src, u64 n) +{ + void *result; + + result = __memcpy(dst, src, n); + if (!n) + /* Some people call memcpy() with zero length. */ + return result; + + if (!kmsan_ready || kmsan_in_runtime()) + return result; + + kmsan_memcpy_metadata(dst, (void *)src, n); + + return result; +} +EXPORT_SYMBOL(__msan_memcpy); + +void *__msan_memcpy_nosanitize(void *dst, void *src, u64 n) +{ + return __memcpy(dst, src, n); +} +EXPORT_SYMBOL(__msan_memcpy_nosanitize); + +void *__msan_memset(void *dst, int c, size_t n) +{ + void *result; + unsigned long irq_flags; + + result = __memset(dst, c, n); + if (!kmsan_ready || kmsan_in_runtime()) + return result; + + irq_flags = kmsan_enter_runtime(); + /* + * Clang doesn't pass parameter metadata here, so it is impossible to + * use shadow of @c to set up the shadow for @dst. + */ + kmsan_internal_unpoison_shadow(dst, n, /*checked*/false); + kmsan_leave_runtime(irq_flags); + + return result; +} +EXPORT_SYMBOL(__msan_memset); + +void *__msan_memset_nosanitize(void *dst, int c, size_t n) +{ + return __memset(dst, c, n); +} +EXPORT_SYMBOL(__msan_memset_nosanitize); + +depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin) +{ + depot_stack_handle_t ret = 0; + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return ret; + + /* Creating new origins may allocate memory. */ + irq_flags = kmsan_enter_runtime(); + ret = kmsan_internal_chain_origin(origin); + kmsan_leave_runtime(irq_flags); + return ret; +} +EXPORT_SYMBOL(__msan_chain_origin); + +void __msan_poison_alloca(void *address, u64 size, char *descr) +{ + depot_stack_handle_t handle; + unsigned long entries[4]; + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + + kmsan_internal_memset_shadow(address, -1, size, /*checked*/true); + + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN; + entries[1] = (u64)descr; + entries[2] = (u64)__builtin_return_address(0); + entries[3] = (u64)kmsan_internal_return_address(1); + + /* stack_depot_save() may allocate memory. */ + irq_flags = kmsan_enter_runtime(); + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC); + kmsan_leave_runtime(irq_flags); + kmsan_internal_set_origin(address, size, handle); +} +EXPORT_SYMBOL(__msan_poison_alloca); + +void __msan_unpoison_alloca(void *address, u64 size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + + irq_flags = kmsan_enter_runtime(); + kmsan_internal_unpoison_shadow(address, size, /*checked*/true); + kmsan_leave_runtime(irq_flags); +} +EXPORT_SYMBOL(__msan_unpoison_alloca); + +void __msan_warning(u32 origin) +{ + unsigned long irq_flags; + + if (!kmsan_ready || kmsan_in_runtime()) + return; + irq_flags = kmsan_enter_runtime(); + kmsan_report(origin, /*address*/0, /*size*/0, + /*off_first*/0, /*off_last*/0, /*user_addr*/0, REASON_ANY); + kmsan_leave_runtime(irq_flags); +} +EXPORT_SYMBOL(__msan_warning); + +struct kmsan_context_state *__msan_get_context_state(void) +{ + struct kmsan_context_state *ret; + + ret = kmsan_task_context_state(); + BUG_ON(!ret); + return ret; +} +EXPORT_SYMBOL(__msan_get_context_state);