From patchwork Fri Jul 1 14:22:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8002C43334 for ; Fri, 1 Jul 2022 14:23:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A7EC6B0073; Fri, 1 Jul 2022 10:23:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 756AF6B0074; Fri, 1 Jul 2022 10:23:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F9656B0075; Fri, 1 Jul 2022 10:23:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 511E96B0073 for ; Fri, 1 Jul 2022 10:23:21 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1F0682165C for ; Fri, 1 Jul 2022 14:23:21 +0000 (UTC) X-FDA: 79638748602.10.EB79AE8 Received: from mail-lf1-f73.google.com (mail-lf1-f73.google.com [209.85.167.73]) by imf02.hostedemail.com (Postfix) with ESMTP id BD57080051 for ; Fri, 1 Jul 2022 14:23:20 +0000 (UTC) Received: by mail-lf1-f73.google.com with SMTP id bq4-20020a056512150400b0047f7f36efc6so1167728lfb.9 for ; Fri, 01 Jul 2022 07:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=/yIHROt1QYP0boO+vuzmQ126QU/rx8ZuEvPrfW89F9Y=; b=RMeY6Ur3ouU5biN/Uz6Pi1vECHINkdUQ1XcdDhpxam1K7Tmx2sZMlF9fAiiBGiDeb0 U7Aa8QqJ+VzQP8OpEO/pKz35Tdfyk3WWhmM4pgL6mDH0PoavMplCXSLjU0240DaBpzjO QbDGael0DJV5cJgd2vGBzYz8gomx8CJ50pgg+tLhgpdbZUtX5ayyxuW6RmdLsX6TBH+C o0DtNWdueVL830o3hq+mgXMzlCK2+/F+OKqDr/zxODRC5vpVa5UwODat1/WfKYe86OhU lS0UhtJlWn+ziWveaLeyRQpu/EtFNlZ1zDnOV0otO8OsP7edPw4pO0p+qmnwXROxdlBV mvtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=/yIHROt1QYP0boO+vuzmQ126QU/rx8ZuEvPrfW89F9Y=; b=wpBCEJbIM9wsh3y/GcAv+u5NQQW3jG3wEabY1rp3FiPcEan0LxDG+cS5F3ZHx9SuXa Zba6Uy3O/SxFloafS9W89Y5ixXeLX6Vs8UC8f2ZqDXomwlhLLEARSplyQruohuYU8k0Z enw/fgBORpQW544NjXfbnk8RW+loP1t0MObIycq87c+RTdoVTM+F/Y01RD8W2peKs+wR athzFPJLc3vYMJ81vTzjjUSAZJmc18/hB3++V3pROVQh7euWzn0thg2yyFQUGcmY1wjt Xe/ByePMHGYCpSGFrwZmn3tWEhuWs4h0lliqIzlRYydhhQ0BwYzW0coVWW80r/fdgcFG CQ4A== X-Gm-Message-State: AJIora9yCXdAnPJl9w3lLJBNQuF1aQWMxRS/Iq7r7rkVdIErmF4qUX0O uEhWvOzqA9OBGfYeoqsAa/B9yX+lZW4= X-Google-Smtp-Source: AGRyM1uD+3Ia18lHCKIGMfEeSdvCpSjDt0Xq3SAkrsJ1OFDc0LZZ0ntvnmsmvOrkKwNT5g2Svgk1nfZ8tIw= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6512:1588:b0:481:1a00:4f10 with SMTP id bp8-20020a056512158800b004811a004f10mr9629583lfb.435.1656685398820; Fri, 01 Jul 2022 07:23:18 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:26 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-2-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 01/45] x86: add missing include to sparsemem.h From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=RMeY6Ur3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3VgO_YgYKCHUZebWXkZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--glider.bounces.google.com designates 209.85.167.73 as permitted sender) smtp.mailfrom=3VgO_YgYKCHUZebWXkZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685400; a=rsa-sha256; cv=none; b=Nbq4TvODE/t8E84dES955LIis7kTbulsoBAQX1XZ9ZmMqagc5VvlvOfUFXRSvYO/eSVxpH ghr2Z+Pe39HyivekkgCAyRF0ZDeoTzQzY4Vq5yTX6xTbObtUndjAwLID3CvibemLUQoDN+ Tgr88shIXtT/rnP4H/FVjPMX2Ihb6OM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685400; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/yIHROt1QYP0boO+vuzmQ126QU/rx8ZuEvPrfW89F9Y=; b=QQK8UxIyg0F/pOhRZP3DPNsu7YeMX7Bqqa2uIFll4NukpcvVdNAdCkxmrIDvfkYRYNCwxO q8VOwZXOsaPEQF5gV56DZLMpVH/nn+dEJN81ab+gtKc3WwD8vzTheAOY192jGJzZsCk4vP LQ3A1ZVbiajwJGPanxpSrSNcY8fbL1Q= X-Stat-Signature: u9fy84tig45ko8ocyp51msdfa1trikjs X-Rspamd-Queue-Id: BD57080051 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=RMeY6Ur3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3VgO_YgYKCHUZebWXkZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--glider.bounces.google.com designates 209.85.167.73 as permitted sender) smtp.mailfrom=3VgO_YgYKCHUZebWXkZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--glider.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1656685400-526176 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dmitry Vyukov Including sparsemem.h from other files (e.g. transitively via asm/pgtable_64_types.h) results in compilation errors due to unknown types: sparsemem.h:34:32: error: unknown type name 'phys_addr_t' extern int phys_to_target_node(phys_addr_t start); ^ sparsemem.h:36:39: error: unknown type name 'u64' extern int memory_add_physaddr_to_nid(u64 start); ^ Fix these errors by including linux/types.h from sparsemem.h This is required for the upcoming KMSAN patches. Signed-off-by: Dmitry Vyukov Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Ifae221ce85d870d8f8d17173bd44d5cf9be2950f --- arch/x86/include/asm/sparsemem.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/include/asm/sparsemem.h b/arch/x86/include/asm/sparsemem.h index 6a9ccc1b2be5d..64df897c0ee30 100644 --- a/arch/x86/include/asm/sparsemem.h +++ b/arch/x86/include/asm/sparsemem.h @@ -2,6 +2,8 @@ #ifndef _ASM_X86_SPARSEMEM_H #define _ASM_X86_SPARSEMEM_H +#include + #ifdef CONFIG_SPARSEMEM /* * generic non-linear memory support: From patchwork Fri Jul 1 14:22:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 316FBC43334 for ; Fri, 1 Jul 2022 14:23:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9F4D6B0074; Fri, 1 Jul 2022 10:23:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4E226B0075; Fri, 1 Jul 2022 10:23:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A2476B0078; Fri, 1 Jul 2022 10:23:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8788C6B0074 for ; Fri, 1 Jul 2022 10:23:23 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 606186014D for ; Fri, 1 Jul 2022 14:23:23 +0000 (UTC) X-FDA: 79638748686.10.13C46F2 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf23.hostedemail.com (Postfix) with ESMTP id D772A140051 for ; Fri, 1 Jul 2022 14:23:22 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id w22-20020a05640234d600b00435ba41dbaaso1881609edc.12 for ; Fri, 01 Jul 2022 07:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=CwX7/lGI7AR80Huh7duB2nRTvZ0igKEJIu5G+Gv7rzk=; b=EefzW4eCsab6lg9j9txLhgrpvzV91MfK4k882eIXelhXIGW49nRx+Vkl0S6AjloVus CEVFsEtDWMgPio6Ew24N3e/2qVos+6BEMGIEbu7lvFn6Muwxy/zISWLFwuqR6fGdLMBN 2tku9H44HTLCZA5NeuXHw+1MoY8QVtR8Mg1LSRR2WkUQoapUihhwHOAMsUQQN4Yn7flr j0Kas8iETv4z1IssW3/67+BZDDunWiULUUADJc0QZmcsLlZe3ldJKCogl6owDfE4FtqK qqpMzoLNGv97PTOeYSb48qYJ+g9+CyJrcpPucpaLyV4BMMrPC001mr00qlH3yuK5BnkE H9sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=CwX7/lGI7AR80Huh7duB2nRTvZ0igKEJIu5G+Gv7rzk=; b=bA0MXfj5tProHnXpsQJgkamO6DcjEQB0aHnNN7773Tes9hUxxSaLDtwM1p9IFmODjq YmnWqr9dKUCL1r2zVoura/kFas5Donuncf8zMzQlOidA4L4WU/33mqYNcjABsTjspBrk Nj0jVsTenG+mv5ESH2ck+08qV5aRhsvCVo9SqnR4mAu3Wh+3S4MlfN4ECv0E0k9iSqEK Cd35UQlnwZ9a5ok8v/SrNKwtQ5g1MivzehIOMbT5JP/0BqkOZgGYfRDSpTNEekMY+jsg F7i2by4XnljukLlPljOLOHkZ0HUEBRha0KfWQqU50j/0QkDr+zew+5TBZu6B9zFqKsEJ GXZw== X-Gm-Message-State: AJIora/QY5B7Iiejou+8yZNDuPWquN06+kVGxHpu+6o7Ptoni7ZTlfM5 oLAgXUnVb87TPckognjzwSDKj0IM/JU= X-Google-Smtp-Source: AGRyM1sR36JM+xYMiOJXstdRXO4cKvRGr5Giyf+e9Q7fYGOcPa0FKG83UM6ydtXOqJdeEXHdTiqSWF+nftc= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:7da1:b0:726:9562:b09f with SMTP id oz33-20020a1709077da100b007269562b09fmr15604327ejc.11.1656685401482; Fri, 01 Jul 2022 07:23:21 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:27 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-3-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 02/45] stackdepot: reserve 5 extra bits in depot_stack_handle_t From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=EefzW4eC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf23.hostedemail.com: domain of 3WQO_YgYKCHgcheZanckkcha.Ykihejqt-iigrWYg.knc@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3WQO_YgYKCHgcheZanckkcha.Ykihejqt-iigrWYg.knc@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685402; a=rsa-sha256; cv=none; b=nDsXjKd2f1JaEAkIvJxo3UPQ4tDpiOccM0Fn6szk5zn3AhnpxWtRUW12o9U1goJz7/qd3j rfWQZ/+Kz/k6Tr2Kdv0vMFRSxo4XHDBOHmff+h5Cb4JL5lcr1YbUV0L8R1mQ0GkMNky3yX /buccaVUIOk81vSjpqz7o7kpN6I8zRU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CwX7/lGI7AR80Huh7duB2nRTvZ0igKEJIu5G+Gv7rzk=; b=kRbOEn+LuxWfmhIOgPm2deF0jmfzs+SWgTGV38OVELZXzw5lpQYCn2/Pz3uhsE4yzoKGtc 7HGxssyZi7R+OFVBfFFwS499AJvzQUVqpyZ/ju/9//9ebS2TLyXo7s1kLy7SH+G1n0la5t wR3Aw4XUztaH73DSRLKDHwvyj5g2Hew= X-Stat-Signature: 3bmfg9n4fuj9d6rdmp3sjp5bbohuu1f8 X-Rspamd-Queue-Id: D772A140051 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=EefzW4eC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf23.hostedemail.com: domain of 3WQO_YgYKCHgcheZanckkcha.Ykihejqt-iigrWYg.knc@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3WQO_YgYKCHgcheZanckkcha.Ykihejqt-iigrWYg.knc@flex--glider.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1656685402-321767 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some users (currently only KMSAN) may want to use spare bits in depot_stack_handle_t. Let them do so by adding @extra_bits to __stack_depot_save() to store arbitrary flags, and providing stack_depot_get_extra_bits() to retrieve those flags. Also adapt KASAN to the new prototype by passing extra_bits=0, as KASAN does not intend to store additional information in the stack handle. Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v4: -- per Marco Elver's request, fold "kasan: common: adapt to the new prototype of __stack_depot_save()" into this patch to prevent bisection breakages. Link: https://linux-review.googlesource.com/id/I0587f6c777667864768daf07821d594bce6d8ff9 --- include/linux/stackdepot.h | 8 ++++++++ lib/stackdepot.c | 29 ++++++++++++++++++++++++----- mm/kasan/common.c | 2 +- 3 files changed, 33 insertions(+), 6 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index bc2797955de90..9ca7798d7a318 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -14,9 +14,15 @@ #include typedef u32 depot_stack_handle_t; +/* + * Number of bits in the handle that stack depot doesn't use. Users may store + * information in them. + */ +#define STACK_DEPOT_EXTRA_BITS 5 depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, + unsigned int extra_bits, gfp_t gfp_flags, bool can_alloc); /* @@ -59,6 +65,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries); +unsigned int stack_depot_get_extra_bits(depot_stack_handle_t handle); + int stack_depot_snprint(depot_stack_handle_t handle, char *buf, size_t size, int spaces); diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 5ca0d086ef4a3..3d1dbdd5a87f6 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -42,7 +42,8 @@ #define STACK_ALLOC_OFFSET_BITS (STACK_ALLOC_ORDER + PAGE_SHIFT - \ STACK_ALLOC_ALIGN) #define STACK_ALLOC_INDEX_BITS (DEPOT_STACK_BITS - \ - STACK_ALLOC_NULL_PROTECTION_BITS - STACK_ALLOC_OFFSET_BITS) + STACK_ALLOC_NULL_PROTECTION_BITS - \ + STACK_ALLOC_OFFSET_BITS - STACK_DEPOT_EXTRA_BITS) #define STACK_ALLOC_SLABS_CAP 8192 #define STACK_ALLOC_MAX_SLABS \ (((1LL << (STACK_ALLOC_INDEX_BITS)) < STACK_ALLOC_SLABS_CAP) ? \ @@ -55,6 +56,7 @@ union handle_parts { u32 slabindex : STACK_ALLOC_INDEX_BITS; u32 offset : STACK_ALLOC_OFFSET_BITS; u32 valid : STACK_ALLOC_NULL_PROTECTION_BITS; + u32 extra : STACK_DEPOT_EXTRA_BITS; }; }; @@ -76,6 +78,14 @@ static int next_slab_inited; static size_t depot_offset; static DEFINE_RAW_SPINLOCK(depot_lock); +unsigned int stack_depot_get_extra_bits(depot_stack_handle_t handle) +{ + union handle_parts parts = { .handle = handle }; + + return parts.extra; +} +EXPORT_SYMBOL(stack_depot_get_extra_bits); + static bool init_stack_slab(void **prealloc) { if (!*prealloc) @@ -139,6 +149,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->handle.slabindex = depot_index; stack->handle.offset = depot_offset >> STACK_ALLOC_ALIGN; stack->handle.valid = 1; + stack->handle.extra = 0; memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); depot_offset += required_size; @@ -343,6 +354,7 @@ EXPORT_SYMBOL_GPL(stack_depot_fetch); * * @entries: Pointer to storage array * @nr_entries: Size of the storage array + * @extra_bits: Flags to store in unused bits of depot_stack_handle_t * @alloc_flags: Allocation gfp flags * @can_alloc: Allocate stack slabs (increased chance of failure if false) * @@ -354,6 +366,10 @@ EXPORT_SYMBOL_GPL(stack_depot_fetch); * If the stack trace in @entries is from an interrupt, only the portion up to * interrupt entry is saved. * + * Additional opaque flags can be passed in @extra_bits, stored in the unused + * bits of the stack handle, and retrieved using stack_depot_get_extra_bits() + * without calling stack_depot_fetch(). + * * Context: Any context, but setting @can_alloc to %false is required if * alloc_pages() cannot be used from the current context. Currently * this is the case from contexts where neither %GFP_ATOMIC nor @@ -363,10 +379,11 @@ EXPORT_SYMBOL_GPL(stack_depot_fetch); */ depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, + unsigned int extra_bits, gfp_t alloc_flags, bool can_alloc) { struct stack_record *found = NULL, **bucket; - depot_stack_handle_t retval = 0; + union handle_parts retval = { .handle = 0 }; struct page *page = NULL; void *prealloc = NULL; unsigned long flags; @@ -450,9 +467,11 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, free_pages((unsigned long)prealloc, STACK_ALLOC_ORDER); } if (found) - retval = found->handle.handle; + retval.handle = found->handle.handle; fast_exit: - return retval; + retval.extra = extra_bits; + + return retval.handle; } EXPORT_SYMBOL_GPL(__stack_depot_save); @@ -472,6 +491,6 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t alloc_flags) { - return __stack_depot_save(entries, nr_entries, alloc_flags, true); + return __stack_depot_save(entries, nr_entries, 0, alloc_flags, true); } EXPORT_SYMBOL_GPL(stack_depot_save); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index c40c0e7b3b5f1..ba4fceeec173c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -36,7 +36,7 @@ depot_stack_handle_t kasan_save_stack(gfp_t flags, bool can_alloc) unsigned int nr_entries; nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 0); - return __stack_depot_save(entries, nr_entries, flags, can_alloc); + return __stack_depot_save(entries, nr_entries, 0, flags, can_alloc); } void kasan_set_track(struct kasan_track *track, gfp_t flags) From patchwork Fri Jul 1 14:22:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DBDAC433EF for ; Fri, 1 Jul 2022 14:23:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 312AC6B0075; Fri, 1 Jul 2022 10:23:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C2F36B0078; Fri, 1 Jul 2022 10:23:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 164AB6B007B; Fri, 1 Jul 2022 10:23:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 06F6A6B0075 for ; Fri, 1 Jul 2022 10:23:26 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D3743359E7 for ; Fri, 1 Jul 2022 14:23:25 +0000 (UTC) X-FDA: 79638748770.04.15BD125 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf16.hostedemail.com (Postfix) with ESMTP id 7C54018002A for ; Fri, 1 Jul 2022 14:23:25 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id v16-20020a056402349000b00435a1c942a9so1890321edc.15 for ; Fri, 01 Jul 2022 07:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=DM2lItfZBVLN/C7PGBgjPtkXFa/DP0+N1ALBTGezmzM=; b=BrwjnieB1ihiMJjhT7I5u//E7HjI1WLLjlXBmbnDhgMb7SpTYsEuazuBiSutpdswxC 5pvB8SRUOYAiKuMPD3S9k8yYpGxtgwA4cZZf6JHUocr1/F0k13itXtMMpFzSqF04Alsp chy+0E2s4SsndfHwLVSyYJzYMcaxAhoa6oqKJOe11TNjTNr10CeQHAcSf+2V5GU9ne2/ XLFDmkZaS+nan9HDxgfTnVYnCoqy44FIEOMq2nA0ViTNF2vno2mvx43ZLEeDlEnkJ1qy mGeEtfCFTEFHfRnQWwnwzmz9FeJeVkjle7Ho0tF4WBjphb+reY+Lt6qdvhMZ0CVbxkQi c7Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DM2lItfZBVLN/C7PGBgjPtkXFa/DP0+N1ALBTGezmzM=; b=zoaFfROfPWpMdz6KfYbdTrh5zixCkjiJsDbtdMeMPTX4fRtLefESPIQ3h6gHbNqJIJ arWCtKg24Yk03RzBe9grS6/ohje7lHLOY8DYNH9ql13lD4lQnojjCnV/CTS5UXa13vND i4SaS4KcXgzQ4xXMaagRuFRWZQzmLBGud0HDFWuLcJ74r+ZIxf1DoOyC744GX1fgWaYi AUBZBxWfSp0UJ5o/fWGa1ND4hFo1ycc2ex6HHIymYb9njLUFYHZQFIVvIqyOe6uBv6AW sR/I9d/ak/aF3vfUKXOTG4UDuF27Tu3yMvEVztRQDr7CByU/5HnnePOtErLJpcizknFT 1T8Q== X-Gm-Message-State: AJIora9tXKA0YXUyJWR7EoZkB6kUEuY8F3xQ5YsKwR+5xqYVu2c+3NZ9 D7s0Hwx3hdstgkaenXdFLi+ueWY1g0Y= X-Google-Smtp-Source: AGRyM1uQju4QFNdvGz9cfWoWELsmxEhkDZif288LkxUpLP859+uBjIeNd+i+MrlITpHeO3F2JnA8CnG2JDk= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:aa7:c2cf:0:b0:435:6576:b7c0 with SMTP id m15-20020aa7c2cf000000b004356576b7c0mr19716710edp.18.1656685404238; Fri, 01 Jul 2022 07:23:24 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:28 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-4-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 03/45] instrumented.h: allow instrumenting both sides of copy_from_user() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685405; a=rsa-sha256; cv=none; b=chYZHDslKaiS0FYCF3lk8pOP5jfahx5+7EPp34vgOBMvI7ikgH9SIVtdhk8WW+SmZ9y+GU eqGXGxFBf8Ox58caNsWh8r3lX1ny+bWafphH9hNl5oh+Y0+LjHPA4RLaQOKnIZcVbfaa+5 KA7GNfcl2WWpI+Q5NVI11WOet65f+g0= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=BrwjnieB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3XAO_YgYKCHsfkhcdqfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3XAO_YgYKCHsfkhcdqfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DM2lItfZBVLN/C7PGBgjPtkXFa/DP0+N1ALBTGezmzM=; b=xYb6YMz7Txl5TWHuobNHe6rilOtacS8/GuhAZX/BqGU3IHHlm7whlrPLVvVeJoPVXDNWtT iY3UyOQXR0wrl831pdVCYb3njjmsvLsqEjtZrgoJqfVyf6hUXlgNpe8i7VM2NVx1N6gjIU vzyh64ogdBEXHmjuZcSZYIDxSujb0+Q= X-Rspamd-Queue-Id: 7C54018002A X-Rspam-User: Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=BrwjnieB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3XAO_YgYKCHsfkhcdqfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3XAO_YgYKCHsfkhcdqfnnfkd.bnlkhmtw-lljuZbj.nqf@flex--glider.bounces.google.com X-Rspamd-Server: rspam02 X-Stat-Signature: ynag8ujidzw98cq5ixijcwmki8cjq9yc X-HE-Tag: 1656685405-756245 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce instrument_copy_from_user_before() and instrument_copy_from_user_after() hooks to be invoked before and after the call to copy_from_user(). KASAN and KCSAN will be only using instrument_copy_from_user_before(), but for KMSAN we'll need to insert code after copy_from_user(). Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v4: -- fix _copy_from_user_key() in arch/s390/lib/uaccess.c (Reported-by: kernel test robot ) Link: https://linux-review.googlesource.com/id/I855034578f0b0f126734cbd734fb4ae1d3a6af99 --- arch/s390/lib/uaccess.c | 3 ++- include/linux/instrumented.h | 21 +++++++++++++++++++-- include/linux/uaccess.h | 19 ++++++++++++++----- lib/iov_iter.c | 9 ++++++--- lib/usercopy.c | 3 ++- 5 files changed, 43 insertions(+), 12 deletions(-) diff --git a/arch/s390/lib/uaccess.c b/arch/s390/lib/uaccess.c index d7b3b193d1088..58033dfcb6d45 100644 --- a/arch/s390/lib/uaccess.c +++ b/arch/s390/lib/uaccess.c @@ -81,8 +81,9 @@ unsigned long _copy_from_user_key(void *to, const void __user *from, might_fault(); if (!should_fail_usercopy()) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user_key(to, from, n, key); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); diff --git a/include/linux/instrumented.h b/include/linux/instrumented.h index 42faebbaa202a..ee8f7d17d34f5 100644 --- a/include/linux/instrumented.h +++ b/include/linux/instrumented.h @@ -120,7 +120,7 @@ instrument_copy_to_user(void __user *to, const void *from, unsigned long n) } /** - * instrument_copy_from_user - instrument writes of copy_from_user + * instrument_copy_from_user_before - add instrumentation before copy_from_user * * Instrument writes to kernel memory, that are due to copy_from_user (and * variants). The instrumentation should be inserted before the accesses. @@ -130,10 +130,27 @@ instrument_copy_to_user(void __user *to, const void *from, unsigned long n) * @n number of bytes to copy */ static __always_inline void -instrument_copy_from_user(const void *to, const void __user *from, unsigned long n) +instrument_copy_from_user_before(const void *to, const void __user *from, unsigned long n) { kasan_check_write(to, n); kcsan_check_write(to, n); } +/** + * instrument_copy_from_user_after - add instrumentation after copy_from_user + * + * Instrument writes to kernel memory, that are due to copy_from_user (and + * variants). The instrumentation should be inserted after the accesses. + * + * @to destination address + * @from source address + * @n number of bytes to copy + * @left number of bytes not copied (as returned by copy_from_user) + */ +static __always_inline void +instrument_copy_from_user_after(const void *to, const void __user *from, + unsigned long n, unsigned long left) +{ +} + #endif /* _LINUX_INSTRUMENTED_H */ diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index 5a328cf02b75e..da16e96680cf1 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -58,20 +58,28 @@ static __always_inline __must_check unsigned long __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n) { - instrument_copy_from_user(to, from, n); + unsigned long res; + + instrument_copy_from_user_before(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); + return res; } static __always_inline __must_check unsigned long __copy_from_user(void *to, const void __user *from, unsigned long n) { + unsigned long res; + might_fault(); + instrument_copy_from_user_before(to, from, n); if (should_fail_usercopy()) return n; - instrument_copy_from_user(to, from, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); + return res; } /** @@ -115,8 +123,9 @@ _copy_from_user(void *to, const void __user *from, unsigned long n) unsigned long res = n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 0b64695ab632f..fe5d169314dbf 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -159,13 +159,16 @@ static int copyout(void __user *to, const void *from, size_t n) static int copyin(void *to, const void __user *from, size_t n) { + size_t res = n; + if (should_fail_usercopy()) return n; if (access_ok(from, n)) { - instrument_copy_from_user(to, from, n); - n = raw_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); + res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } - return n; + return res; } static size_t copy_page_to_iter_iovec(struct page *page, size_t offset, size_t bytes, diff --git a/lib/usercopy.c b/lib/usercopy.c index 7413dd300516e..1505a52f23a01 100644 --- a/lib/usercopy.c +++ b/lib/usercopy.c @@ -12,8 +12,9 @@ unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n unsigned long res = n; might_fault(); if (!should_fail_usercopy() && likely(access_ok(from, n))) { - instrument_copy_from_user(to, from, n); + instrument_copy_from_user_before(to, from, n); res = raw_copy_from_user(to, from, n); + instrument_copy_from_user_after(to, from, n, res); } if (unlikely(res)) memset(to + (n - res), 0, res); From patchwork Fri Jul 1 14:22:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85CF8C433EF for ; Fri, 1 Jul 2022 14:23:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 201CA6B0078; Fri, 1 Jul 2022 10:23:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B1A96B007B; Fri, 1 Jul 2022 10:23:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0525B6B007D; Fri, 1 Jul 2022 10:23:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EB6806B0078 for ; Fri, 1 Jul 2022 10:23:28 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C00D4359BF for ; Fri, 1 Jul 2022 14:23:28 +0000 (UTC) X-FDA: 79638748896.10.875AA78 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf05.hostedemail.com (Postfix) with ESMTP id 516E2100059 for ; Fri, 1 Jul 2022 14:23:28 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id y18-20020a056402441200b0043564cdf765so1890350eda.11 for ; Fri, 01 Jul 2022 07:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=g6pC2DAwiUBeK1TSxgdl5U3IOE2AcDc9FK6KiXQeXic=; b=AHKrqjSKbIhFOsa2VQSybdRJw40/76dIOouQ3LxiX50cNdW9EKH77HHDfNAcImNKZf GeMQtHcRemc/Cl5NUHQ8aiLcNwc8TpgRsU6Ln8BGBoNJ4WhY8e4fJesF8eINu4Q1m6rE 894rX+GOm0y1pWTY+02nHyNLuCMkwHH0SiMztDb+tBivwkIU2GyInzjk7TGEmVGfgXT7 ikUy1pi4Zeduvw3a5o1EtaeInq31rAlsXTucAeSAezlmwAdOugIN+Mt9tp5v0YNDJcZ8 rg9kw8pUI9ailA5iyfhYRap74iZrdGMioHk8KCDVS57lmwlU8Cc/kwuLKwtlSc7RZkg8 qAyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=g6pC2DAwiUBeK1TSxgdl5U3IOE2AcDc9FK6KiXQeXic=; b=Tp6sxgJSsVTQY0i6vSbGM923WU7EXNIVY0yYU4zl9pVKxZrx+bq/LjVcMRO4NEe01l 2e4eNXHqHWWI+ZLEbR1e7Nks1MoKC6nGdkfs7uip4tmHnWBB1etSN0aogG6o+ohD7W3L DzEhuzISNfR4awq6Zv8eTOlGv+H1dwMauAFWcQMLvp2z8itgd8IcxXO78lx6RRX/uvhB GP9X/mV4kotC+7/u5wgKujd/s1wcprdRJlKfHFCzOwNaGGllfwiVM8i6vr5FWX+L27vB 7VNlawNuZ49YUJMOXitBZ0NYRjJyI8kmxpHKHFV36pTpu27JfZlzQSK5Dq/hu8x/Im1J Mghg== X-Gm-Message-State: AJIora+6rbdpqaDkobvJ/onS/+ZijorO3n2XCX+/d+3D3BMY/W+Sqlyo cB27b+Y/eHODL0uMNbvahWQ4+frdMbM= X-Google-Smtp-Source: AGRyM1uxLYmWR//LYr+qyE7iiW3wADsN021yfuxu7jUqUOU6cZfOOtwpDxxr6LrgXLqxLicromQ2HETc9Qk= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:97d1:b0:722:e6fc:a04 with SMTP id js17-20020a17090797d100b00722e6fc0a04mr14570630ejc.217.1656685406971; Fri, 01 Jul 2022 07:23:26 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:29 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-5-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 04/45] x86: asm: instrument usercopy in get_user() and __put_user_size() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685408; a=rsa-sha256; cv=none; b=bbJxYO/C9hLdWSdPPX0i50Nc2JyJhybjRLVg7s6pRJxEBaNhdRgX6656QZ8TGORbeDJsuC 3KP1b18M7HmJmGSnmik4YeMmeQljJm8PNSUOHXIgV8q1yIq3I9wAsH3Gf81VS/QUNp0Ed1 AXqkSxCsG3XBV9FKpLrhcUVHXLFVH08= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=AHKrqjSK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3XgO_YgYKCH0hmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3XgO_YgYKCH0hmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g6pC2DAwiUBeK1TSxgdl5U3IOE2AcDc9FK6KiXQeXic=; b=RmyuRFnX/y+7ivQ8wMKuWxXE1NX/yqPB43FT4jQOTatCjly2xjWFWuIT49dVWR0XJtPWUA WLdk+VdVqo5eWSW7cyxuNDkBS41mOqlwtsCRX42oxgXPBtc7REAwiNpLOBkTv2slo0sjQK oSPkelmuZELT5xx6OoFDpvAn/Pw+7nA= X-Stat-Signature: egms1fr8b8g5g1nne1ocz3amaujjok9z X-Rspamd-Queue-Id: 516E2100059 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=AHKrqjSK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3XgO_YgYKCH0hmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3XgO_YgYKCH0hmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1656685408-997650 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use hooks from instrumented.h to notify bug detection tools about usercopy events in get_user() and put_user_size(). It's still unclear how to instrument put_user(), which assumes that instrumentation code doesn't clobber RAX. Signed-off-by: Alexander Potapenko Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- Link: https://linux-review.googlesource.com/id/Ia9f12bfe5832623250e20f1859fdf5cc485a2fce --- arch/x86/include/asm/uaccess.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h index 913e593a3b45f..1a8b5a234474f 100644 --- a/arch/x86/include/asm/uaccess.h +++ b/arch/x86/include/asm/uaccess.h @@ -5,6 +5,7 @@ * User space memory access functions */ #include +#include #include #include #include @@ -99,11 +100,13 @@ extern int __get_user_bad(void); int __ret_gu; \ register __inttype(*(ptr)) __val_gu asm("%"_ASM_DX); \ __chk_user_ptr(ptr); \ + instrument_copy_from_user_before((void *)&(x), ptr, sizeof(*(ptr))); \ asm volatile("call __" #fn "_%P4" \ : "=a" (__ret_gu), "=r" (__val_gu), \ ASM_CALL_CONSTRAINT \ : "0" (ptr), "i" (sizeof(*(ptr)))); \ (x) = (__force __typeof__(*(ptr))) __val_gu; \ + instrument_copy_from_user_after((void *)&(x), ptr, sizeof(*(ptr)), 0); \ __builtin_expect(__ret_gu, 0); \ }) @@ -248,7 +251,9 @@ extern void __put_user_nocheck_8(void); #define __put_user_size(x, ptr, size, label) \ do { \ + __typeof__(*(ptr)) __pus_val = x; \ __chk_user_ptr(ptr); \ + instrument_copy_to_user(ptr, &(__pus_val), size); \ switch (size) { \ case 1: \ __put_user_goto(x, ptr, "b", "iq", label); \ @@ -286,6 +291,7 @@ do { \ #define __get_user_size(x, ptr, size, label) \ do { \ __chk_user_ptr(ptr); \ + instrument_copy_from_user_before((void *)&(x), ptr, size); \ switch (size) { \ case 1: { \ unsigned char x_u8__; \ @@ -305,6 +311,7 @@ do { \ default: \ (x) = __get_user_bad(); \ } \ + instrument_copy_from_user_after((void *)&(x), ptr, size, 0); \ } while (0) #define __get_user_asm(x, addr, itype, ltype, label) \ From patchwork Fri Jul 1 14:22:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A11CC43334 for ; Fri, 1 Jul 2022 14:23:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFF346B007E; Fri, 1 Jul 2022 10:23:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C87476B007D; Fri, 1 Jul 2022 10:23:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B27B06B007E; Fri, 1 Jul 2022 10:23:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A25756B007B for ; Fri, 1 Jul 2022 10:23:31 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7FC5235AA8 for ; Fri, 1 Jul 2022 14:23:31 +0000 (UTC) X-FDA: 79638749022.02.285B27E Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf01.hostedemail.com (Postfix) with ESMTP id 16D9D40047 for ; Fri, 1 Jul 2022 14:23:30 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id hp8-20020a1709073e0800b0072629757566so851533ejc.0 for ; Fri, 01 Jul 2022 07:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=pc6R18JS+sE1u3dlHmLsZDbE6VaTO3tMmut2L/JxCPM=; b=S+rhPYqifq/yKHIbjTCu4N0+Z7NmA9VGY0y8mmzTIhWAs4YaTE8GZZ/d43VgjMb7UK az5tdkPj9m+23R3TaCa0oezo79iWHEHsoOzkGkp2TjQQUNIdG/x1dKhPZaJZhlGLwZdu OjzMamB1js4+neGeWSDKWbw+2TwxoP4epnmauaWHGLcm5vCoQvR64fFA9xoH25lw6mWd RLVZUTkg987xWln4gHc8u6k+gTKox0fy6iKvm0z+TJUaI4tHLawej5SvXm+skA79uSIb qPS9h+WO1w5qn9e14232gpg/imXEIDcl9Inw5qmcqk5oV1wYuWBucbpYRkDhcT0pyMzj XIag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=pc6R18JS+sE1u3dlHmLsZDbE6VaTO3tMmut2L/JxCPM=; b=sNk4k5DciAOkO5h7/BC6MFN9iBnnGwQh539wFyt7ZSqBLmarspsGxHeUbx7pHbSRAH XOHgHUK4hrRjS3cjXCUkATfl18K58z2wSYHmt0fISfDmn0Ks536xG2Y2UxTB8appLPGs rY9bBSsWQFCEslQXGAC/S626Ect/fqdlH9H/jwom+9h02pdm/rtGuPu3nBz5Swbv1AEl 9/K9+gjRZ4qDy37ApmvgOv73y5GE061UjxQ1Zlkw/eMzHgLI+plnxIxJkOw5+IDKoytv cHzspao6zU4c4M5NfkLVQnpkTZxZ5SnydLUoHoWTB2UV2j4ddV4lYl0aNOKxEKZwmOP1 l2Wg== X-Gm-Message-State: AJIora8uiF5WCkVJzfI1ZHblfPrz4tfIhqVGffsYDV843PFpq3ptpa+j 8cByV33z6kEJU+r4DblY9GVL7AbbreI= X-Google-Smtp-Source: AGRyM1tGUA53rIsnnKld405DIJFvq8TS/2FfFMNmonYGXrqujK7Cf69LRpSQi0Bl4dPyssWWDST2mmIbkrQ= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:3514:b0:435:f24a:fbad with SMTP id b20-20020a056402351400b00435f24afbadmr18590781edd.311.1656685409756; Fri, 01 Jul 2022 07:23:29 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:30 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-6-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 05/45] asm-generic: instrument usercopy in cacheflush.h From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685411; a=rsa-sha256; cv=none; b=rv4vAEN836T5ZmPaCbLQWZg388JwsHrsJRPOOpJTbkS+KvID8Vb0q5BMeo5hDJBVskIQRh VTzRpdihWsvsw++0irM12Wsu3NjKKbRjxEMuw/1m1S2fqMnjaoIpIIwpHSniL+BZ5LWb7b CCEiSYviU4/sX6g+lH/Vu13K9c2mRL0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=S+rhPYqi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3YQO_YgYKCIAkpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3YQO_YgYKCIAkpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685411; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pc6R18JS+sE1u3dlHmLsZDbE6VaTO3tMmut2L/JxCPM=; b=D6c4FiC5UcK9P2kWe6mDlbCqRvv8F7ZrDB09c1SIc70AMJdlPd0cx7/Ozk1QKO9emBC9YY qL+JeOlLiFhMreRmhdTMBNLnEEq2GJ8VS4IpqgMXKuLqqgkI0bs7G2diGvA6IQtKHuvlz/ HeImFzVYOpnBUtWQ+SwjJIkuUes6lbE= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 16D9D40047 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=S+rhPYqi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3YQO_YgYKCIAkpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3YQO_YgYKCIAkpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: uq99utsurcd8pe3i7n7zaqwrxs8xw7nx X-HE-Tag: 1656685410-768883 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Notify memory tools about usercopy events in copy_to_user_page() and copy_from_user_page(). Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- Link: https://linux-review.googlesource.com/id/Ic1ee8da1886325f46ad67f52176f48c2c836c48f --- include/asm-generic/cacheflush.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h index 4f07afacbc239..0f63eb325025f 100644 --- a/include/asm-generic/cacheflush.h +++ b/include/asm-generic/cacheflush.h @@ -2,6 +2,8 @@ #ifndef _ASM_GENERIC_CACHEFLUSH_H #define _ASM_GENERIC_CACHEFLUSH_H +#include + struct mm_struct; struct vm_area_struct; struct page; @@ -105,6 +107,7 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) #ifndef copy_to_user_page #define copy_to_user_page(vma, page, vaddr, dst, src, len) \ do { \ + instrument_copy_to_user(dst, src, len); \ memcpy(dst, src, len); \ flush_icache_user_page(vma, page, vaddr, len); \ } while (0) @@ -112,7 +115,11 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) #ifndef copy_from_user_page #define copy_from_user_page(vma, page, vaddr, dst, src, len) \ - memcpy(dst, src, len) + do { \ + instrument_copy_from_user_before(dst, src, len); \ + memcpy(dst, src, len); \ + instrument_copy_from_user_after(dst, src, len, 0); \ + } while (0) #endif #endif /* _ASM_GENERIC_CACHEFLUSH_H */ From patchwork Fri Jul 1 14:22:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9218C433EF for ; Fri, 1 Jul 2022 14:23:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4EAAD6B007B; Fri, 1 Jul 2022 10:23:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 498AE6B007D; Fri, 1 Jul 2022 10:23:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3382C6B0080; Fri, 1 Jul 2022 10:23:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1D1156B007B for ; Fri, 1 Jul 2022 10:23:34 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DA0ED21593 for ; Fri, 1 Jul 2022 14:23:33 +0000 (UTC) X-FDA: 79638749106.30.B4BE8D0 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf21.hostedemail.com (Postfix) with ESMTP id 84BFB1C0036 for ; Fri, 1 Jul 2022 14:23:33 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id qk8-20020a1709077f8800b00722fcbfdcf7so844123ejc.2 for ; Fri, 01 Jul 2022 07:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Pcx/5TatZO+PLwZ5TV1XrBLvINh7By4hSIuvqXTbyqY=; b=KQ8tvNcabYh5b0OfzPUfNuqNjrPFbvcOHLtOZmj6c2zdlusah8UdaLRsbGMoEfn5+1 Ic2UfeaAJEIVF+dE9aAsnU5QMyKBMW25Jy/tJli1xr8YjwAnhVyVO94NN9QlGBwUrugD FnP+SYMfLs3Y7Holkyu5VoelECZ60e9TFnH+ELwn0HWy4a+EW+ZIv/Phg6y7cuCEyFEH lnB9SqRSK71xVx4j7ssP+HbVjl9mVJkWupHPlAXW7avNtDaRd5v5tv2yEdPJ7sSuMJcH 3z4NGToFAhlXmKf9cfT7mDP1R6z3I7Ujtr943BSZ+03icFE3s+LhQqd+ur5Ktc9E8uv2 Vh8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Pcx/5TatZO+PLwZ5TV1XrBLvINh7By4hSIuvqXTbyqY=; b=eWOa53MMjjQkjXUduc5YIXPxkVLx7agUw7fWmNPg1O+rOPDOIREZb214hAD+83xtf9 SIzP36KprvANow6WxgWCVtFwktpccCjbnqUFBqPWNy7pUIL2Pzs88fw5hQz3jqOnYq0j udSnCc+IwCMvazOeprIEnjvkX8CfUQwVCQF9Gdk353dehh8W8r5P2pRJGFzESWwJenym 1H77Jxfg/aiqFB0S7PS+0tL2qsqtW0w75NfZh96672tI89Wp3IBXWu3OUGBRKimCWmvF CVl1I7Ms5gk6OtuvbHPhHclAnC53L9OIlB5c2Mf5y+IOV6yGtB/NshhMtbPr2fe1tIaU XJtQ== X-Gm-Message-State: AJIora/heoDN0+vViP4LBRoylNz44IxPrcLGMdQMS5i1Po2JVLQZ4hD4 l0X8drrzhqyJdf5v6Vq4eqc358Wpbcg= X-Google-Smtp-Source: AGRyM1tytUCyY6mMEcrEEYjY29xRPpYXLxayeGtyR0ecC/t1p0mypoOr1Kv9/2Phu4zxmePJc+29RTIxoGA= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:42d5:b0:433:1727:b31c with SMTP id i21-20020a05640242d500b004331727b31cmr11243413edc.9.1656685412341; Fri, 01 Jul 2022 07:23:32 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:31 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-7-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 06/45] kmsan: add ReST documentation From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685413; a=rsa-sha256; cv=none; b=zC3PN5ZdPf/34sa42Zd/4nzIQLDWAUKxV7k7HdnC2i7Iek+gdmRqYQDK30lTxZx/WUqGwL trqqrqSXkibPy+zEtnQpwuYfI/rGv6R+d2JqgydOcg4xaI1zB92gj9wTxDvrTho2EgjvRR y4PLZrzr/BJK9++YkxaRjnj0fWJzmCo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=KQ8tvNca; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of 3ZAO_YgYKCIMnspklynvvnsl.jvtspu14-ttr2hjr.vyn@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3ZAO_YgYKCIMnspklynvvnsl.jvtspu14-ttr2hjr.vyn@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685413; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Pcx/5TatZO+PLwZ5TV1XrBLvINh7By4hSIuvqXTbyqY=; b=HF0aP4JanIVADaVS90hS8/gUO+VNm/bAnd6KWHIYHIUC/pbXwui9JvlinYDy/0JNSLj+fs uZjtJWjI+64hKnQUCore9RxwF6aHFwMSfsadBle6o4Az8027goodcSbchJy1hNK99wELGR 1djlhB4QxEZk00Fl5h9nyfh1ex5WaU8= Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=KQ8tvNca; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of 3ZAO_YgYKCIMnspklynvvnsl.jvtspu14-ttr2hjr.vyn@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3ZAO_YgYKCIMnspklynvvnsl.jvtspu14-ttr2hjr.vyn@flex--glider.bounces.google.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 84BFB1C0036 X-Stat-Signature: brjfju9kmy9ks5chq4prqe5t8xfwryzc X-Rspam-User: X-HE-Tag: 1656685413-585749 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add Documentation/dev-tools/kmsan.rst and reference it in the dev-tools index. Signed-off-by: Alexander Potapenko --- v2: -- added a note that KMSAN is not intended for production use v4: -- describe CONFIG_KMSAN_CHECK_PARAM_RETVAL -- drop mentions of cpu_entry_area -- add SPDX license Link: https://linux-review.googlesource.com/id/I751586f79418b95550a83c6035c650b5b01567cc --- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kmsan.rst | 422 ++++++++++++++++++++++++++++++ 2 files changed, 423 insertions(+) create mode 100644 Documentation/dev-tools/kmsan.rst diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index 4621eac290f46..6b0663075dc04 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -24,6 +24,7 @@ Documentation/dev-tools/testing-overview.rst kcov gcov kasan + kmsan ubsan kmemleak kcsan diff --git a/Documentation/dev-tools/kmsan.rst b/Documentation/dev-tools/kmsan.rst new file mode 100644 index 0000000000000..3fa5d7fb222c9 --- /dev/null +++ b/Documentation/dev-tools/kmsan.rst @@ -0,0 +1,422 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. Copyright (C) 2022, Google LLC. + +============================= +KernelMemorySanitizer (KMSAN) +============================= + +KMSAN is a dynamic error detector aimed at finding uses of uninitialized +values. It is based on compiler instrumentation, and is quite similar to the +userspace `MemorySanitizer tool`_. + +An important note is that KMSAN is not intended for production use, because it +drastically increases kernel memory footprint and slows the whole system down. + +Example report +============== + +Here is an example of a KMSAN report:: + + ===================================================== + BUG: KMSAN: uninit-value in test_uninit_kmsan_check_memory+0x1be/0x380 [kmsan_test] + test_uninit_kmsan_check_memory+0x1be/0x380 mm/kmsan/kmsan_test.c:273 + kunit_run_case_internal lib/kunit/test.c:333 + kunit_try_run_case+0x206/0x420 lib/kunit/test.c:374 + kunit_generic_run_threadfn_adapter+0x6d/0xc0 lib/kunit/try-catch.c:28 + kthread+0x721/0x850 kernel/kthread.c:327 + ret_from_fork+0x1f/0x30 ??:? + + Uninit was stored to memory at: + do_uninit_local_array+0xfa/0x110 mm/kmsan/kmsan_test.c:260 + test_uninit_kmsan_check_memory+0x1a2/0x380 mm/kmsan/kmsan_test.c:271 + kunit_run_case_internal lib/kunit/test.c:333 + kunit_try_run_case+0x206/0x420 lib/kunit/test.c:374 + kunit_generic_run_threadfn_adapter+0x6d/0xc0 lib/kunit/try-catch.c:28 + kthread+0x721/0x850 kernel/kthread.c:327 + ret_from_fork+0x1f/0x30 ??:? + + Local variable uninit created at: + do_uninit_local_array+0x4a/0x110 mm/kmsan/kmsan_test.c:256 + test_uninit_kmsan_check_memory+0x1a2/0x380 mm/kmsan/kmsan_test.c:271 + + Bytes 4-7 of 8 are uninitialized + Memory access of size 8 starts at ffff888083fe3da0 + + CPU: 0 PID: 6731 Comm: kunit_try_catch Tainted: G B E 5.16.0-rc3+ #104 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014 + ===================================================== + + +The report says that the local variable ``uninit`` was created uninitialized in +``do_uninit_local_array()``. The lower stack trace corresponds to the place +where this variable was created. + +The upper stack shows where the uninit value was used - in +``test_uninit_kmsan_check_memory()``. The tool shows the bytes which were left +uninitialized in the local variable, as well as the stack where the value was +copied to another memory location before use. + +A use of uninitialized value ``v`` is reported by KMSAN in the following cases: + - in a condition, e.g. ``if (v) { ... }``; + - in an indexing or pointer dereferencing, e.g. ``array[v]`` or ``*v``; + - when it is copied to userspace or hardware, e.g. ``copy_to_user(..., &v, ...)``; + - when it is passed as an argument to a function, and + ``CONFIG_KMSAN_CHECK_PARAM_RETVAL`` is enabled (see below). + +The mentioned cases (apart from copying data to userspace or hardware, which is +a security issue) are considered undefined behavior from the C11 Standard point +of view. + +KMSAN and Clang +=============== + +In order for KMSAN to work the kernel must be built with Clang, which so far is +the only compiler that has KMSAN support. The kernel instrumentation pass is +based on the userspace `MemorySanitizer tool`_. + +How to build +============ + +In order to build a kernel with KMSAN you will need a fresh Clang (14.0.0+). +Please refer to `LLVM documentation`_ for the instructions on how to build Clang. + +Now configure and build the kernel with CONFIG_KMSAN enabled. + +How KMSAN works +=============== + +KMSAN shadow memory +------------------- + +KMSAN associates a metadata byte (also called shadow byte) with every byte of +kernel memory. A bit in the shadow byte is set iff the corresponding bit of the +kernel memory byte is uninitialized. Marking the memory uninitialized (i.e. +setting its shadow bytes to ``0xff``) is called poisoning, marking it +initialized (setting the shadow bytes to ``0x00``) is called unpoisoning. + +When a new variable is allocated on the stack, it is poisoned by default by +instrumentation code inserted by the compiler (unless it is a stack variable +that is immediately initialized). Any new heap allocation done without +``__GFP_ZERO`` is also poisoned. + +Compiler instrumentation also tracks the shadow values with the help from the +runtime library in ``mm/kmsan/``. + +The shadow value of a basic or compound type is an array of bytes of the same +length. When a constant value is written into memory, that memory is unpoisoned. +When a value is read from memory, its shadow memory is also obtained and +propagated into all the operations which use that value. For every instruction +that takes one or more values the compiler generates code that calculates the +shadow of the result depending on those values and their shadows. + +Example:: + + int a = 0xff; // i.e. 0x000000ff + int b; + int c = a | b; + +In this case the shadow of ``a`` is ``0``, shadow of ``b`` is ``0xffffffff``, +shadow of ``c`` is ``0xffffff00``. This means that the upper three bytes of +``c`` are uninitialized, while the lower byte is initialized. + + +Origin tracking +--------------- + +Every four bytes of kernel memory also have a so-called origin assigned to +them. This origin describes the point in program execution at which the +uninitialized value was created. Every origin is associated with either the +full allocation stack (for heap-allocated memory), or the function containing +the uninitialized variable (for locals). + +When an uninitialized variable is allocated on stack or heap, a new origin +value is created, and that variable's origin is filled with that value. +When a value is read from memory, its origin is also read and kept together +with the shadow. For every instruction that takes one or more values the origin +of the result is one of the origins corresponding to any of the uninitialized +inputs. If a poisoned value is written into memory, its origin is written to the +corresponding storage as well. + +Example 1:: + + int a = 42; + int b; + int c = a + b; + +In this case the origin of ``b`` is generated upon function entry, and is +stored to the origin of ``c`` right before the addition result is written into +memory. + +Several variables may share the same origin address, if they are stored in the +same four-byte chunk. In this case every write to either variable updates the +origin for all of them. We have to sacrifice precision in this case, because +storing origins for individual bits (and even bytes) would be too costly. + +Example 2:: + + int combine(short a, short b) { + union ret_t { + int i; + short s[2]; + } ret; + ret.s[0] = a; + ret.s[1] = b; + return ret.i; + } + +If ``a`` is initialized and ``b`` is not, the shadow of the result would be +0xffff0000, and the origin of the result would be the origin of ``b``. +``ret.s[0]`` would have the same origin, but it will be never used, because +that variable is initialized. + +If both function arguments are uninitialized, only the origin of the second +argument is preserved. + +Origin chaining +~~~~~~~~~~~~~~~ + +To ease debugging, KMSAN creates a new origin for every store of an +uninitialized value to memory. The new origin references both its creation stack +and the previous origin the value had. This may cause increased memory +consumption, so we limit the length of origin chains in the runtime. + +Clang instrumentation API +------------------------- + +Clang instrumentation pass inserts calls to functions defined in +``mm/kmsan/instrumentation.c`` into the kernel code. + +Shadow manipulation +~~~~~~~~~~~~~~~~~~~ + +For every memory access the compiler emits a call to a function that returns a +pair of pointers to the shadow and origin addresses of the given memory:: + + typedef struct { + void *shadow, *origin; + } shadow_origin_ptr_t + + shadow_origin_ptr_t __msan_metadata_ptr_for_load_{1,2,4,8}(void *addr) + shadow_origin_ptr_t __msan_metadata_ptr_for_store_{1,2,4,8}(void *addr) + shadow_origin_ptr_t __msan_metadata_ptr_for_load_n(void *addr, uintptr_t size) + shadow_origin_ptr_t __msan_metadata_ptr_for_store_n(void *addr, uintptr_t size) + +The function name depends on the memory access size. + +The compiler makes sure that for every loaded value its shadow and origin +values are read from memory. When a value is stored to memory, its shadow and +origin are also stored using the metadata pointers. + +Handling locals +~~~~~~~~~~~~~~~ + +A special function is used to create a new origin value for a local variable and +set the origin of that variable to that value:: + + void __msan_poison_alloca(void *addr, uintptr_t size, char *descr) + +Access to per-task data +~~~~~~~~~~~~~~~~~~~~~~~~~ + +At the beginning of every instrumented function KMSAN inserts a call to +``__msan_get_context_state()``:: + + kmsan_context_state *__msan_get_context_state(void) + +``kmsan_context_state`` is declared in ``include/linux/kmsan.h``:: + + struct kmsan_context_state { + char param_tls[KMSAN_PARAM_SIZE]; + char retval_tls[KMSAN_RETVAL_SIZE]; + char va_arg_tls[KMSAN_PARAM_SIZE]; + char va_arg_origin_tls[KMSAN_PARAM_SIZE]; + u64 va_arg_overflow_size_tls; + char param_origin_tls[KMSAN_PARAM_SIZE]; + depot_stack_handle_t retval_origin_tls; + }; + +This structure is used by KMSAN to pass parameter shadows and origins between +instrumented functions (unless the parameters are checked immediately by +``CONFIG_KMSAN_CHECK_PARAM_RETVAL``). + +Passing uninitialized values to functions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +KMSAN instrumentation pass has an option, ``-fsanitize-memory-param-retval``, +which makes the compiler check function parameters passed by value, as well as +function return values. + +The option is controlled by ``CONFIG_KMSAN_CHECK_PARAM_RETVAL``, which is +enabled by default to let KMSAN report uninitialized values earlier. +Please refer to the `LKML discussion`_ for more details. + +Because of the way the checks are implemented in LLVM (they are only applied to +parameters marked as ``noundef``), not all parameters are guaranteed to be +checked, so we cannot give up the metadata storage in ``kmsan_context_state``. + +String functions +~~~~~~~~~~~~~~~~ + +The compiler replaces calls to ``memcpy()``/``memmove()``/``memset()`` with the +following functions. These functions are also called when data structures are +initialized or copied, making sure shadow and origin values are copied alongside +with the data:: + + void *__msan_memcpy(void *dst, void *src, uintptr_t n) + void *__msan_memmove(void *dst, void *src, uintptr_t n) + void *__msan_memset(void *dst, int c, uintptr_t n) + +Error reporting +~~~~~~~~~~~~~~~ + +For each use of a value the compiler emits a shadow check that calls +``__msan_warning()`` in the case that value is poisoned:: + + void __msan_warning(u32 origin) + +``__msan_warning()`` causes KMSAN runtime to print an error report. + +Inline assembly instrumentation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +KMSAN instruments every inline assembly output with a call to:: + + void __msan_instrument_asm_store(void *addr, uintptr_t size) + +, which unpoisons the memory region. + +This approach may mask certain errors, but it also helps to avoid a lot of +false positives in bitwise operations, atomics etc. + +Sometimes the pointers passed into inline assembly do not point to valid memory. +In such cases they are ignored at runtime. + +Disabling the instrumentation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A function can be marked with ``__no_kmsan_checks``. Doing so makes KMSAN +ignore uninitialized values in that function and mark its output as initialized. +As a result, the user will not get KMSAN reports related to that function. + +Another function attribute supported by KMSAN is ``__no_sanitize_memory``. +Applying this attribute to a function will result in KMSAN not instrumenting it, +which can be helpful if we do not want the compiler to mess up some low-level +code (e.g. that marked with ``noinstr``). + +This however comes at a cost: stack allocations from such functions will have +incorrect shadow/origin values, likely leading to false positives. Functions +called from non-instrumented code may also receive incorrect metadata for their +parameters. + +As a rule of thumb, avoid using ``__no_sanitize_memory`` explicitly. + +It is also possible to disable KMSAN for a single file (e.g. main.o):: + + KMSAN_SANITIZE_main.o := n + +or for the whole directory:: + + KMSAN_SANITIZE := n + +in the Makefile. Think of this as applying ``__no_sanitize_memory`` to every +function in the file or directory. Most users won't need KMSAN_SANITIZE, unless +their code gets broken by KMSAN (e.g. runs at early boot time). + +Runtime library +--------------- + +The code is located in ``mm/kmsan/``. + +Per-task KMSAN state +~~~~~~~~~~~~~~~~~~~~ + +Every task_struct has an associated KMSAN task state that holds the KMSAN +context (see above) and a per-task flag disallowing KMSAN reports:: + + struct kmsan_context { + ... + bool allow_reporting; + struct kmsan_context_state cstate; + ... + } + + struct task_struct { + ... + struct kmsan_context kmsan; + ... + } + + +KMSAN contexts +~~~~~~~~~~~~~~ + +When running in a kernel task context, KMSAN uses ``current->kmsan.cstate`` to +hold the metadata for function parameters and return values. + +But in the case the kernel is running in the interrupt, softirq or NMI context, +where ``current`` is unavailable, KMSAN switches to per-cpu interrupt state:: + + DEFINE_PER_CPU(struct kmsan_ctx, kmsan_percpu_ctx); + +Metadata allocation +~~~~~~~~~~~~~~~~~~~ + +There are several places in the kernel for which the metadata is stored. + +1. Each ``struct page`` instance contains two pointers to its shadow and +origin pages:: + + struct page { + ... + struct page *shadow, *origin; + ... + }; + +At boot-time, the kernel allocates shadow and origin pages for every available +kernel page. This is done quite late, when the kernel address space is already +fragmented, so normal data pages may arbitrarily interleave with the metadata +pages. + +This means that in general for two contiguous memory pages their shadow/origin +pages may not be contiguous. So, if a memory access crosses the boundary +of a memory block, accesses to shadow/origin memory may potentially corrupt +other pages or read incorrect values from them. + +In practice, contiguous memory pages returned by the same ``alloc_pages()`` +call will have contiguous metadata, whereas if these pages belong to two +different allocations their metadata pages can be fragmented. + +For the kernel data (``.data``, ``.bss`` etc.) and percpu memory regions +there also are no guarantees on metadata contiguity. + +In the case ``__msan_metadata_ptr_for_XXX_YYY()`` hits the border between two +pages with non-contiguous metadata, it returns pointers to fake shadow/origin regions:: + + char dummy_load_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); + char dummy_store_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); + +``dummy_load_page`` is zero-initialized, so reads from it always yield zeroes. +All stores to ``dummy_store_page`` are ignored. + +2. For vmalloc memory and modules, there is a direct mapping between the memory +range, its shadow and origin. KMSAN reduces the vmalloc area by 3/4, making only +the first quarter available to ``vmalloc()``. The second quarter of the vmalloc +area contains shadow memory for the first quarter, the third one holds the +origins. A small part of the fourth quarter contains shadow and origins for the +kernel modules. Please refer to ``arch/x86/include/asm/pgtable_64_types.h`` for +more details. + +When an array of pages is mapped into a contiguous virtual memory space, their +shadow and origin pages are similarly mapped into contiguous regions. + +References +========== + +E. Stepanov, K. Serebryany. `MemorySanitizer: fast detector of uninitialized +memory use in C++ +`_. +In Proceedings of CGO 2015. + +.. _MemorySanitizer tool: https://clang.llvm.org/docs/MemorySanitizer.html +.. _LLVM documentation: https://llvm.org/docs/GettingStarted.html +.. _LKML discussion: https://lore.kernel.org/all/20220614144853.3693273-1-glider@google.com/ From patchwork Fri Jul 1 14:22:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6868CC43334 for ; Fri, 1 Jul 2022 14:23:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFA056B007D; Fri, 1 Jul 2022 10:23:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E82896B0080; Fri, 1 Jul 2022 10:23:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFD3A6B0081; Fri, 1 Jul 2022 10:23:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BFAB36B007D for ; Fri, 1 Jul 2022 10:23:36 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 79BB5615E2 for ; Fri, 1 Jul 2022 14:23:36 +0000 (UTC) X-FDA: 79638749232.09.8F53680 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf23.hostedemail.com (Postfix) with ESMTP id 28021140044 for ; Fri, 1 Jul 2022 14:23:35 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id m8-20020a056402430800b00435cfa7c6d1so1886234edc.9 for ; Fri, 01 Jul 2022 07:23:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=gkw9YA7vb5ti0YajzjGN1OXbVHAqDO+SEh0OPXXltUo=; b=DJbRcfFJX1wyQKX8l1FZMyPmyPhiG8QaxtSYEHL+jg9LVV4C2ZYVwI87iVYJ5A4E5H sNiz7RovSUX82iKvetSTT3ivRgXe5SpIKUpLrVUzsQuWrPS2jZdEVUvd0d8c9xpIi90R qYQHmigg/c73A/xfmHKA2YHRRIAqFEe1j/TS5WTo13oUcktHSiV8OM6WEW0kvknYU24w qmZDUmfFklZTJQaCpjJHDgMF8cHmn9jm2Clttqa12yyLWz5x4d/+8IezIzi4zJyCX3cm qbQ7Z3421wCmEB8O7JvaGEmI1uQEvIdKykYZFz7qXILVLVkF14Shh8wwyANYIgOrQMXI WcWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gkw9YA7vb5ti0YajzjGN1OXbVHAqDO+SEh0OPXXltUo=; b=LOp6s9W3nMpjK4piXgQdOmQ7Ab+ukUzfrafgaG5E4sxbGacJODjIiW6qd5DVIdBoph QxswAKG0zx8QJwbFZFqWSRqCR2uKudnKv8xBvTVuXx7tAXuLWos36lL1a3GkJ7nlHJ2v dd7PFOoctpu71jeXgN7sEZQ1oALL3TMAkwccLZnlSezTPAKpdHDd0xpSCwQ4zlrtHVEa eANas4u0uQOJNf1lThODo2s4o0VISPBg952P39VHDpRDfgddqxiMm89GsjXER1FCHO2D CvJ2kHeCUS5ou2Y1bTOpupLesYugP0L273Nb+iZRBAy+5oizsNHURthw3yid4dqzBlJ1 JBcw== X-Gm-Message-State: AJIora+PWTOHMLcktDKvU14pGSKGMDKynRtbDVJZwa2ZyDr9mLJCtV65 ekZ6zQ5TP2wUNM1Oh/GEF7qIpOUNIRc= X-Google-Smtp-Source: AGRyM1uCNIkqQBwTVhpOJ3Y5EwonQhFZw2dysbIFTeg8qz1++vtqH5YHyoXPS6hxY4s/hJxG0C172twlxOY= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:518b:b0:435:c1ed:3121 with SMTP id q11-20020a056402518b00b00435c1ed3121mr19002047edd.405.1656685414953; Fri, 01 Jul 2022 07:23:34 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:32 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-8-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 07/45] kmsan: introduce __no_sanitize_memory and __no_kmsan_checks From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=DJbRcfFJ; spf=pass (imf23.hostedemail.com: domain of 3ZgO_YgYKCIUpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3ZgO_YgYKCIUpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685416; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gkw9YA7vb5ti0YajzjGN1OXbVHAqDO+SEh0OPXXltUo=; b=eKk38tqLz6mgno0W0nCLchP9d/wyQ4cCBPng1VBBFoQWlK4jdmeVwXfA+kD+sFgG6XCcHQ 2nm6agbXoNpZex4XcBJUnW5VJoydwVVQRVaqelGfUAD5NJwVvMIBzmnqYIotrMqb1rkYp9 EblXISbeHt2OSTEdsMeBYFRVBpbtcy0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685416; a=rsa-sha256; cv=none; b=IWYeBBw0hLNkLRAqsxbn/UYbz15muYI44WNozwXxLFzc8PkrpQxkTKoRBImVef0uhdnKk7 V7yOCqNOXW1/YdNpq+lwAiJikVHbffwCnA7ktrWrdtfZ8GZwapFoILwQakyYLXboWD6ULO Wgl5GUCTLe7WTerAklspv4WHutReVWc= X-Stat-Signature: ep89b8yiuso37suk4aanef8gje5o8xzi X-Rspamd-Queue-Id: 28021140044 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=DJbRcfFJ; spf=pass (imf23.hostedemail.com: domain of 3ZgO_YgYKCIUpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3ZgO_YgYKCIUpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-HE-Tag: 1656685415-687138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __no_sanitize_memory is a function attribute that instructs KMSAN to skip a function during instrumentation. This is needed to e.g. implement the noinstr functions. __no_kmsan_checks is a function attribute that makes KMSAN ignore the uninitialized values coming from the function's inputs, and initialize the function's outputs. Functions marked with this attribute can't be inlined into functions not marked with it, and vice versa. This behavior is overridden by __always_inline. __SANITIZE_MEMORY__ is a macro that's defined iff the file is instrumented with KMSAN. This is not the same as CONFIG_KMSAN, which is defined for every file. Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- Link: https://linux-review.googlesource.com/id/I004ff0360c918d3cd8b18767ddd1381c6d3281be --- include/linux/compiler-clang.h | 23 +++++++++++++++++++++++ include/linux/compiler-gcc.h | 6 ++++++ 2 files changed, 29 insertions(+) diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index c84fec767445d..4fa0cc4cbd2c8 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -51,6 +51,29 @@ #define __no_sanitize_undefined #endif +#if __has_feature(memory_sanitizer) +#define __SANITIZE_MEMORY__ +/* + * Unlike other sanitizers, KMSAN still inserts code into functions marked with + * no_sanitize("kernel-memory"). Using disable_sanitizer_instrumentation + * provides the behavior consistent with other __no_sanitize_ attributes, + * guaranteeing that __no_sanitize_memory functions remain uninstrumented. + */ +#define __no_sanitize_memory __disable_sanitizer_instrumentation + +/* + * The __no_kmsan_checks attribute ensures that a function does not produce + * false positive reports by: + * - initializing all local variables and memory stores in this function; + * - skipping all shadow checks; + * - passing initialized arguments to this function's callees. + */ +#define __no_kmsan_checks __attribute__((no_sanitize("kernel-memory"))) +#else +#define __no_sanitize_memory +#define __no_kmsan_checks +#endif + /* * Support for __has_feature(coverage_sanitizer) was added in Clang 13 together * with no_sanitize("coverage"). Prior versions of Clang support coverage diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index a0c55eeaeaf16..63eb90eddad77 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -125,6 +125,12 @@ #define __SANITIZE_ADDRESS__ #endif +/* + * GCC does not support KMSAN. + */ +#define __no_sanitize_memory +#define __no_kmsan_checks + /* * Turn individual warnings and errors on and off locally, depending * on version. From patchwork Fri Jul 1 14:22:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 688F0C433EF for ; Fri, 1 Jul 2022 14:23:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 006946B0080; Fri, 1 Jul 2022 10:23:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED0AD6B0081; Fri, 1 Jul 2022 10:23:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4AE26B0082; Fri, 1 Jul 2022 10:23:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C4DE26B0080 for ; Fri, 1 Jul 2022 10:23:39 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 9F46A8031D for ; Fri, 1 Jul 2022 14:23:39 +0000 (UTC) X-FDA: 79638749358.28.1A7EA04 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf17.hostedemail.com (Postfix) with ESMTP id 3AA684003F for ; Fri, 1 Jul 2022 14:23:39 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id c20-20020a05640227d400b004369cf00c6bso1887527ede.22 for ; Fri, 01 Jul 2022 07:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=os0Zx5VZv6p+Rm8I/hkYFMfJuZRHr+O61ed8WspeRIk=; b=OV7vbBjecQ+2aw/zjF5UI7FBbnLXSdlTtfDvhPtucRYMTl+svaPlqkq36KZQsvGhP/ ne8e4l8I3/0GRvxegtfKUO6cwS2qPksp9NrQQ5lnpQwHWvK+HWlKJBwNEAVhhJIwF7u5 Ns847PNFYy8k60pdHRyM2PsMAizAqLlGYbkafF+wOEnPqhwIwE4aYtsL1hd1Jq6fCqFz ce+lc///QcEti81QlLr7Y54ODBIMH5AChXdc7cPCvHBnFhpkGjgRkrjcUhJb8u74OZGE PCvurUMEpDHYof925nelBMEr785dzxSUlTT5q2hRH/q3uOyGgTsSk9NCo4066mule3qb +5Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=os0Zx5VZv6p+Rm8I/hkYFMfJuZRHr+O61ed8WspeRIk=; b=6wV41jXW9wV5KX/G6eIXxgQgrW4aWqTqOBrgkP1p0dcI6uEhsWwHQE3zunstvn3ANt a93Mzhs6WM86Ta0yfdHsV49WsdHROl451fBhqt9YCJSNlAdq2CNThDDXrZiaH8oPu81s i3lCgnO1g+gCOm66jLhKoqSBy7U9T/uuKxNGZrGs0I72dTxFsQu/R+2AurBGH0Df+yP7 TA9si9BmKxM1MJDpfhonlmTDnWgiQD3fk6oaXd5/jLWbjBZyoc3P2Qnt1laEQ1XPBFYK drG2DDPVoVAo0Ek7MvjAm6/pEl2iOkexV3fVMw7Fuag7/NwcgmnXZcjJeboDXrUXe1iF V8Ng== X-Gm-Message-State: AJIora+IEOuqn7Ywfci1dIo8uJKlgzm1cjum7q0YT7OQ/ywGWDslrFs7 8mxF/VFNIlWiAfB3O+85Kgdu/EoZ4qk= X-Google-Smtp-Source: AGRyM1sWpivMRCzyWBt2NMDYpIZXoVXItcZn0l1qEGR/DeU09TxwkO65LsmcVUOum8/EZGKYMgfViYv/FwA= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:4244:b0:437:726c:e1a with SMTP id g4-20020a056402424400b00437726c0e1amr19866573edb.107.1656685417915; Fri, 01 Jul 2022 07:23:37 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:33 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-9-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 08/45] kmsan: mark noinstr as __no_sanitize_memory From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685419; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=os0Zx5VZv6p+Rm8I/hkYFMfJuZRHr+O61ed8WspeRIk=; b=e/B/rIukoKklUzD5lF7OrGjHluX9EDRKbGS19FLps5mdUO+DhZb7bVJofbtXJgH2Wu+wRE gfsdXhq0FaXrHG4jtgBfk7U1g/WhECNL2GxdX1qVfydhxZSmh9/F0P2+wusNHxp2roy8tg eMb3E7/9PwKUxMgdH+D4kbetNcMjNks= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685419; a=rsa-sha256; cv=none; b=LcuK0wTP+iT4ul61Y/KcOvelOCYy9Gx/5lbCiAXYxd4hbhFOIh8FOGriqCDLz9jCwwPdiu 5nskUy8rwb9zEqplmXdshpyvEmNSjwlczaIzbFJui3LKi3wYBT2enN5tbVWGuEgRUkAx2t i7a+4DQ8FSKxX+CObLoocyO7RqALOB0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OV7vbBje; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3aQO_YgYKCIgsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3aQO_YgYKCIgsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OV7vbBje; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3aQO_YgYKCIgsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3aQO_YgYKCIgsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com X-Rspamd-Server: rspam03 X-Rspam-User: X-Stat-Signature: t5k7ejcem5u7pfdpiedkrnkkw6hkuq9o X-Rspamd-Queue-Id: 3AA684003F X-HE-Tag: 1656685419-391381 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: noinstr functions should never be instrumented, so make KMSAN skip them by applying the __no_sanitize_memory attribute. Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v2: -- moved this patch earlier in the series per Mark Rutland's request Link: https://linux-review.googlesource.com/id/I3c9abe860b97b49bc0c8026918b17a50448dec0d --- include/linux/compiler_types.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index d08dfcb0ac687..fb5777e5228e7 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -227,7 +227,8 @@ struct ftrace_likely_data { /* Section for code which can't be instrumented at all */ #define noinstr \ noinline notrace __attribute((__section__(".noinstr.text"))) \ - __no_kcsan __no_sanitize_address __no_profile __no_sanitize_coverage + __no_kcsan __no_sanitize_address __no_profile __no_sanitize_coverage \ + __no_sanitize_memory #endif /* __KERNEL__ */ From patchwork Fri Jul 1 14:22:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903366 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B702C43334 for ; Fri, 1 Jul 2022 14:23:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C9F66B0071; Fri, 1 Jul 2022 10:23:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 979B86B0081; Fri, 1 Jul 2022 10:23:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F4096B0082; Fri, 1 Jul 2022 10:23:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 70AE26B0071 for ; Fri, 1 Jul 2022 10:23:43 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A519334B48 for ; Fri, 1 Jul 2022 14:23:42 +0000 (UTC) X-FDA: 79638749484.10.907600F Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf06.hostedemail.com (Postfix) with ESMTP id 3F294180033 for ; Fri, 1 Jul 2022 14:23:42 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id z19-20020a05640240d300b00437633081abso1901914edb.0 for ; Fri, 01 Jul 2022 07:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9CmKhXrgq4R//9cDj1zsNax3tT0d2FywVlyLyC3NqDY=; b=U+fVjLRG5oNjqME2B9ccBc/zjLmfwrEmyw/ej3ru79fJemqPnxYlMMrth6VkT3UGNR HYhunHxxINt63ndT6iLXlgZzqmTWlNiZWKNbO5DwAV9ASu3H9cFPMY4yM0S7lI8xD5Rd EdfdxYo8yK9PcvxDjTY8AkAfIw+DM3ORaR0VjSQ6WImxJGGIPu0tQkrosIMl1vZ0/2Ji NWuO1Z/jcEr4SrlBYEg338PXfVrC2Q6IRrJfBy14cKv5aKi1rkr6MnEskoTrso8+Vuwy cR78mRBYVCFAKZOPuO/rTahaUWaznyDz6n12VcGqjAy/jcfSNqAIl2kMglHKUYWNqXly zlCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9CmKhXrgq4R//9cDj1zsNax3tT0d2FywVlyLyC3NqDY=; b=MmGfD+B0b/C5buR6puUM0z4l0GIuKF7AAB8wWVaJiHoHloyhyAxc55Ii005y3Ktwb8 lCNWXW/6m9CXI8ofu+Pd3sReS0VxdHgvRIxPB8qCOeh+ZSVPXGcmncgffpr6n8Zc25DX diPUHdYIYTNQzzDAaw1WeKFfurjlZOg1ccUqXAvxo6v2bWnU76BBSH3EOWoecV7ScN9m 0kyxH7QXxj0d4AKedFo71g0sZzKrSrq8Y1pEiWf3fApacpnaLCsMcRGJ3nHSIUysEMsW KT7/958R3PRAzkWWSKYH9xEMz+JD6HfuMbGbdjktrW9RQPReEUzvb/JRmeqZ5hjsFkKz 7zOA== X-Gm-Message-State: AJIora+hGKUc28KHjdsaaBZIVgbnY9loQtPZ9qSUVXQ+FkL7OS1vl10B Uloguu3MskSRVnvIsBLFVSLF/Xrdrq4= X-Google-Smtp-Source: AGRyM1tC0/9ONSX2Oj+NF6gPMNaP4csfTn6MQttvUu3tGGrRxI+8L442tFZ55fo0CQX675W66GnAMJtI6Fs= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:1914:b0:437:8f32:96e5 with SMTP id e20-20020a056402191400b004378f3296e5mr19396026edz.218.1656685421053; Fri, 01 Jul 2022 07:23:41 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:34 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-10-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 09/45] x86: kmsan: pgtable: reduce vmalloc space From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685422; a=rsa-sha256; cv=none; b=FSRYsdiT27cSxU3hY4hX7QPczxNk76+tC4bZxZ7iAJIOcw+i/bvMw+y3W1JMsYRRKVvV1/ NpBxO1Yhga2F38N6x5SP8rUr+raSalHLsGMD835CJMdmY/Xk4zGqrH64vqpheaoVjKMIk0 nuXxie2VdIFt48i5CyDbVUjSz6kRxDc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=U+fVjLRG; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 3bQO_YgYKCIww1ytu7w44w1u.s421y3AD-220Bqs0.47w@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3bQO_YgYKCIww1ytu7w44w1u.s421y3AD-220Bqs0.47w@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9CmKhXrgq4R//9cDj1zsNax3tT0d2FywVlyLyC3NqDY=; b=UQkPu0byr/3Y2fG0hQNInGQqWiDiWm3ZNn7GlCuPXWJIQb5O7nFJtqnJ7ZRWBxHBlOgIhj cf016XgIo0GoXMRLpY03IqyJXY8Y2qRMYxVmn3fOHRSWCBKSB0NdxzuXo4/5+N/Ba/Qi3M KPwm02/+OY8EbHmT3Tp+Ft8YgufMj+8= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3F294180033 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=U+fVjLRG; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 3bQO_YgYKCIww1ytu7w44w1u.s421y3AD-220Bqs0.47w@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3bQO_YgYKCIww1ytu7w44w1u.s421y3AD-220Bqs0.47w@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: ja9er1zcz8asg6gfmmy3n1ar9ffrq9p5 X-HE-Tag: 1656685422-709936 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN is going to use 3/4 of existing vmalloc space to hold the metadata, therefore we lower VMALLOC_END to make sure vmalloc() doesn't allocate past the first 1/4. Signed-off-by: Alexander Potapenko --- v2: -- added x86: to the title Link: https://linux-review.googlesource.com/id/I9d8b7f0a88a639f1263bc693cbd5c136626f7efd --- arch/x86/include/asm/pgtable_64_types.h | 41 ++++++++++++++++++++++++- arch/x86/mm/init_64.c | 2 +- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 70e360a2e5fb7..ad6ded5b1dedf 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -139,7 +139,46 @@ extern unsigned int ptrs_per_p4d; # define VMEMMAP_START __VMEMMAP_BASE_L4 #endif /* CONFIG_DYNAMIC_MEMORY_LAYOUT */ -#define VMALLOC_END (VMALLOC_START + (VMALLOC_SIZE_TB << 40) - 1) +#define VMEMORY_END (VMALLOC_START + (VMALLOC_SIZE_TB << 40) - 1) + +#ifndef CONFIG_KMSAN +#define VMALLOC_END VMEMORY_END +#else +/* + * In KMSAN builds vmalloc area is four times smaller, and the remaining 3/4 + * are used to keep the metadata for virtual pages. The memory formerly + * belonging to vmalloc area is now laid out as follows: + * + * 1st quarter: VMALLOC_START to VMALLOC_END - new vmalloc area + * 2nd quarter: KMSAN_VMALLOC_SHADOW_START to + * VMALLOC_END+KMSAN_VMALLOC_SHADOW_OFFSET - vmalloc area shadow + * 3rd quarter: KMSAN_VMALLOC_ORIGIN_START to + * VMALLOC_END+KMSAN_VMALLOC_ORIGIN_OFFSET - vmalloc area origins + * 4th quarter: KMSAN_MODULES_SHADOW_START to KMSAN_MODULES_ORIGIN_START + * - shadow for modules, + * KMSAN_MODULES_ORIGIN_START to + * KMSAN_MODULES_ORIGIN_START + MODULES_LEN - origins for modules. + */ +#define VMALLOC_QUARTER_SIZE ((VMALLOC_SIZE_TB << 40) >> 2) +#define VMALLOC_END (VMALLOC_START + VMALLOC_QUARTER_SIZE - 1) + +/* + * vmalloc metadata addresses are calculated by adding shadow/origin offsets + * to vmalloc address. + */ +#define KMSAN_VMALLOC_SHADOW_OFFSET VMALLOC_QUARTER_SIZE +#define KMSAN_VMALLOC_ORIGIN_OFFSET (VMALLOC_QUARTER_SIZE << 1) + +#define KMSAN_VMALLOC_SHADOW_START (VMALLOC_START + KMSAN_VMALLOC_SHADOW_OFFSET) +#define KMSAN_VMALLOC_ORIGIN_START (VMALLOC_START + KMSAN_VMALLOC_ORIGIN_OFFSET) + +/* + * The shadow/origin for modules are placed one by one in the last 1/4 of + * vmalloc space. + */ +#define KMSAN_MODULES_SHADOW_START (VMALLOC_END + KMSAN_VMALLOC_ORIGIN_OFFSET + 1) +#define KMSAN_MODULES_ORIGIN_START (KMSAN_MODULES_SHADOW_START + MODULES_LEN) +#endif /* CONFIG_KMSAN */ #define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE) /* The module sections ends with the start of the fixmap */ diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 39c5246964a91..5806331172361 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1287,7 +1287,7 @@ static void __init preallocate_vmalloc_pages(void) unsigned long addr; const char *lvl; - for (addr = VMALLOC_START; addr <= VMALLOC_END; addr = ALIGN(addr + 1, PGDIR_SIZE)) { + for (addr = VMALLOC_START; addr <= VMEMORY_END; addr = ALIGN(addr + 1, PGDIR_SIZE)) { pgd_t *pgd = pgd_offset_k(addr); p4d_t *p4d; pud_t *pud; From patchwork Fri Jul 1 14:22:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A54DC43334 for ; Fri, 1 Jul 2022 14:23:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A2EB6B0081; Fri, 1 Jul 2022 10:23:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92D396B0082; Fri, 1 Jul 2022 10:23:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A6486B0083; Fri, 1 Jul 2022 10:23:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6A97B6B0081 for ; Fri, 1 Jul 2022 10:23:45 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 47AF2EC7 for ; Fri, 1 Jul 2022 14:23:45 +0000 (UTC) X-FDA: 79638749610.14.3989A61 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf04.hostedemail.com (Postfix) with ESMTP id CB0B440045 for ; Fri, 1 Jul 2022 14:23:44 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id k7-20020a1709062a4700b006fe92440164so844990eje.23 for ; Fri, 01 Jul 2022 07:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=AkHvx6ZxtZ0IjsJvZfH9AmZNAyLy9uG4lwf6iPpek9Y=; b=rHudIZCMEvVcL0J/tpORMCU1TSzScrZ6kJnwi8au7+a6JF30h8PGv5ybkos7NSpKVF l8ItJ4ldc7rg9EUkrGxk8hBljzCbzYjQgUwA1nGDUYpjSYBpSv1cZheZLHTOEc3UGcpz GN3MhPn5PyMEGYWYnmCKoNlKEkO4c/obwSOro9mv0hP92gHSxiMzcqtoWoMzuEAzed2q wII50gyDZqPcOzaGsoh2O4Imju9tw8/pg9ZgJAzo8hKdBhlLYoNWrG5KLaa/L/jFhzEF QS0SOmwTV1hAlRt1ITxae7Jk30DIUPW7GuCpXSA6Yvjsk9qUvcwrEZ8Tfb30DarFex/+ W40w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=AkHvx6ZxtZ0IjsJvZfH9AmZNAyLy9uG4lwf6iPpek9Y=; b=K+4T91QNlSeFSbdJubvW3gdapSt4/Gy5G4/nlvKN3kOgRh5iOhxF8FuaNJrQqui2xI WbEdlOYBMMXgu+246FFLN164TPKzKgUKJUdYxrtu3sNWRWBxg1Df1CmowiD/oGpFRAtF GAMK3Aw64k9wn/KJTf3jO8babHu+N4YTtpCl2fU894c74ALo7ceNVaHXPQ2IVzoOx6Cf sjoguus8XtXvmmBObd/O+qoWZgvQn3CKtsnuYy97IAdJF94wIXZsoDVQkF0p4+ulwdPw +wI/SC85LMvyuz9oZb++mQGqajnm3qVA/LpiNBF6j49Ufr8rLzQuPWr3LiRTaM9FIspI Ctdw== X-Gm-Message-State: AJIora/pHGsbJFGWZ9bQNQdoRvzkmWciXL4rXlN1fk5mjveQi2lyA0WR gSNVQ01hBcniUKz6fG60HHNCx3QLnag= X-Google-Smtp-Source: AGRyM1s+kuVh9fols3fC269jhlEDP/Rp/CRoJ+oGxYSY9r03d5HAcg3xOh5GvCrJCoCJuu3kbfRaEWJ10XY= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a50:fe83:0:b0:437:9c60:12f3 with SMTP id d3-20020a50fe83000000b004379c6012f3mr19071968edt.120.1656685423650; Fri, 01 Jul 2022 07:23:43 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:35 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-11-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 10/45] libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rHudIZCM; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 3bwO_YgYKCI4y30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3bwO_YgYKCI4y30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685424; a=rsa-sha256; cv=none; b=TzwRg1MoKqqzJ6Jtyx8Zi3y1LK2GEeOKz2wtF5vshkncRbNBlOKpRFZwlThY5/EnIVxBi0 E3t6Bl2TMsyCWIXBhMd7Mp30tyv3yENT+towwsrBO+L6gJIijDkKrT1c+JFtHhP/PaEug1 U18a71Yphky9Ug0060bZ56ejyzn/ywM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AkHvx6ZxtZ0IjsJvZfH9AmZNAyLy9uG4lwf6iPpek9Y=; b=Ik/gklbSUi9AgD2ERzrjUeWxRtHSogIDCqEUVtwdA8XSpcD4XuDExrDBeZWECNVTRayI+7 pN5Qr9ls469NXzWf6ZpoD/5CtqTyNhg55HMNJSztaqmdN+fWIvPScA250166I6XQrovcqU RRZS7KAyiL4JyyK11PTdHD6hpf/L01A= X-Stat-Signature: z8etfz3bg7thuiwy1c7iqr9ngttu9n49 X-Rspamd-Queue-Id: CB0B440045 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=rHudIZCM; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 3bwO_YgYKCI4y30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3bwO_YgYKCI4y30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1656685424-283182 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN adds extra metadata fields to struct page, so it does not fit into 64 bytes anymore. Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver Reported-by: Jeff Moyer --- Link: https://linux-review.googlesource.com/id/I353796acc6a850bfd7bb342aa1b63e616fc614f1 --- drivers/nvdimm/nd.h | 2 +- drivers/nvdimm/pfn_devs.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index ec5219680092d..85ca5b4da3cf3 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -652,7 +652,7 @@ void devm_namespace_disable(struct device *dev, struct nd_namespace_common *ndns); #if IS_ENABLED(CONFIG_ND_CLAIM) /* max struct page size independent of kernel config */ -#define MAX_STRUCT_PAGE_SIZE 64 +#define MAX_STRUCT_PAGE_SIZE 128 int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, struct dev_pagemap *pgmap); #else static inline int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c index 0e92ab4b32833..61af072ac98f9 100644 --- a/drivers/nvdimm/pfn_devs.c +++ b/drivers/nvdimm/pfn_devs.c @@ -787,7 +787,7 @@ static int nd_pfn_init(struct nd_pfn *nd_pfn) * when populating the vmemmap. This *should* be equal to * PMD_SIZE for most architectures. * - * Also make sure size of struct page is less than 64. We + * Also make sure size of struct page is less than 128. We * want to make sure we use large enough size here so that * we don't have a dynamic reserve space depending on * struct page size. But we also want to make sure we notice From patchwork Fri Jul 1 14:22:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903368 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E70ECC43334 for ; Fri, 1 Jul 2022 14:23:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F5746B0082; Fri, 1 Jul 2022 10:23:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A6FC6B0083; Fri, 1 Jul 2022 10:23:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55D276B0085; Fri, 1 Jul 2022 10:23:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 40BAF6B0082 for ; Fri, 1 Jul 2022 10:23:48 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 17F0B20EDB for ; Fri, 1 Jul 2022 14:23:48 +0000 (UTC) X-FDA: 79638749736.28.3330428 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf03.hostedemail.com (Postfix) with ESMTP id A06E32003B for ; Fri, 1 Jul 2022 14:23:47 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id g8-20020a056402090800b00433940d207eso1896434edz.1 for ; Fri, 01 Jul 2022 07:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=voao7VAcevzSxeDPcEE+UOm7zOWlvOnF9Q9KaHmc0WI=; b=n6per+2fS6fxcmNk/pKus1qoPabP5/ExXg1lWFjoKQ5r3/RyB8/KmMaWlunsm5YwS0 D6FQUYWxDrhPz/QV9D+ECwYEfDHobSBuRxdX8EVCnb7yk9MqmKwLEY3/CK7AZJcVKCO5 mq+Gcof6UV74aHVElWKlVHtE6QY+hJSmo1SQb4rVVeXrr8d9NpTq7tUCQvYsmBa9Xmf7 vgvUxUYp7rR5Ruu9LphLqy4SmZg17HH6BeTekKgX7KqM6ssoKtg4zH5dXfG5TnKBw9Ym IKbFZbp7+Kckt8xuTjS39eOOv3+1tfK/6LWiD2A1/PKu4JoT/WfjVC9DgpEKd4wqxRf/ PMHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=voao7VAcevzSxeDPcEE+UOm7zOWlvOnF9Q9KaHmc0WI=; b=vwEvScBxCsMaX/VmmolEwcyxdnJ6vNTGm07f0LVCt1nUxuqBICtYY9t/MLa6XQpyad YPSbnJH6eYwRHQcPb24AJ/MmngSAPmSlAH7kdUG1n2uIARMnEcLWsgLrU1DRvlY+1GV5 3fxC0Gs8B0wn0UC0EsjJrf4ByWPyD7LCBJvmMCXppTAF8/M2Fsz7sYt6sZrShSguXcHB 1CBkLD9vKXX4i6+D9vzt9WxZKQ/encSb14UoiS2oGSJQ1g5A/4WinfN4CVwVhLzqz3B9 KwLvIrPDBdNxRg5xJk2fLLO9vk64p+0ifr9UreU+m6r95bzxTZDiksAcWH0jTt9pBrjF cWFg== X-Gm-Message-State: AJIora/mPSZfnwitCWojz/7QFOksjl8cc71YCQ61jJ3XTRWRgyKv9KyL K3o2Q2fm7tRhnkwpJQD8SdZmwOjSZT0= X-Google-Smtp-Source: AGRyM1sce+V0aHnfNNbnjPPVU88/OdFpY3Ep32JqlcG2OMmBResjZl7ormQWXVc9C8Rk4QIFOSUkfCMJIDo= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:35d6:b0:435:9f33:3363 with SMTP id z22-20020a05640235d600b004359f333363mr19445562edc.349.1656685426463; Fri, 01 Jul 2022 07:23:46 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:36 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-12-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 11/45] kmsan: add KMSAN runtime core From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685427; a=rsa-sha256; cv=none; b=gnxx4x5gwK28iKsYtRTCTNLfyXoGGAgMK+c0qjie3kgH8UsBzSPhKtvBOPPlAegk/hEFYy IzPRX8yKyh54N5xMW00Huz4M2uhzzMV4lmnIZZgeH2F7AzdxvMwghVo7UB5gIAqCWI8Krk HjnPiw0lCtDoB5RBPHf7j//N6nOlXVA= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=n6per+2f; spf=pass (imf03.hostedemail.com: domain of 3cgO_YgYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3cgO_YgYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685427; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=voao7VAcevzSxeDPcEE+UOm7zOWlvOnF9Q9KaHmc0WI=; b=3uL9WtZzbLnUeaWLADg0FZxoV2cRCrYvG2Df2Y2bYJ8M/ZezISXkD2g+C/T7QsyPq6Wzf1 21nCOwavP3WEguWWGIlyCFN/WLVQ+mA0ZHeO/7Cmk9nAbTNiaS8c6L8K+HWiVdvnry0v/U bdZDs8kbKHbez+5hWOtTECx2KUvUcVM= X-Rspam-User: X-Rspamd-Server: rspam04 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=n6per+2f; spf=pass (imf03.hostedemail.com: domain of 3cgO_YgYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3cgO_YgYKCJE163yzC19916z.x97638FI-775Gvx5.9C1@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: ycx6zm5kysqqndscnw1wzdjkpt3ro4po X-Rspamd-Queue-Id: A06E32003B X-HE-Tag: 1656685427-596478 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For each memory location KernelMemorySanitizer maintains two types of metadata: 1. The so-called shadow of that location - а byte:byte mapping describing whether or not individual bits of memory are initialized (shadow is 0) or not (shadow is 1). 2. The origins of that location - а 4-byte:4-byte mapping containing 4-byte IDs of the stack traces where uninitialized values were created. Each struct page now contains pointers to two struct pages holding KMSAN metadata (shadow and origins) for the original struct page. Utility routines in mm/kmsan/core.c and mm/kmsan/shadow.c handle the metadata creation, addressing, copying and checking. mm/kmsan/report.c performs error reporting in the cases an uninitialized value is used in a way that leads to undefined behavior. KMSAN compiler instrumentation is responsible for tracking the metadata along with the kernel memory. mm/kmsan/instrumentation.c provides the implementation for instrumentation hooks that are called from files compiled with -fsanitize=kernel-memory. To aid parameter passing (also done at instrumentation level), each task_struct now contains a struct kmsan_task_state used to track the metadata of function parameters and return values for that task. Finally, this patch provides CONFIG_KMSAN that enables KMSAN, and declares CFLAGS_KMSAN, which are applied to files compiled with KMSAN. The KMSAN_SANITIZE:=n Makefile directive can be used to completely disable KMSAN instrumentation for certain files. Similarly, KMSAN_ENABLE_CHECKS:=n disables KMSAN checks and makes newly created stack memory initialized. Users can also use functions from include/linux/kmsan-checks.h to mark certain memory regions as uninitialized or initialized (this is called "poisoning" and "unpoisoning") or check that a particular region is initialized. Signed-off-by: Alexander Potapenko Acked-by: Marco Elver --- v2: -- as requested by Greg K-H, moved hooks for different subsystems to respective patches, rewrote the patch description; -- addressed comments by Dmitry Vyukov; -- added a note about KMSAN being not intended for production use. -- fix case of unaligned dst in kmsan_internal_memmove_metadata() v3: -- print build IDs in reports where applicable -- drop redundant filter_irq_stacks(), unpoison the local passed to __stack_depot_save() -- remove a stray BUG() v4: (mostly fixes suggested by Marco Elver) -- add missing SPDX headers -- move CC_IS_CLANG && CLANG_VERSION under HAVE_KMSAN_COMPILER -- replace occurrences of |var| with @var -- reflow KMSAN_WARN_ON(), fix code comments -- remove x86-specific code from shadow.c to improve portability -- convert kmsan_report_lock to raw spinlock -- add enter_runtime/exit_runtime around kmsan_internal_memmove_metadata() -- remove unnecessary include from kmsan.h (reported by ) -- introduce CONFIG_KMSAN_CHECK_PARAM_RETVAL (on by default), which maps to -fsanitize-memory-param-retval and makes KMSAN eagerly check values passed as function parameters and returned from functions. -- use real shadow in instrumented functions called from runtime Link: https://linux-review.googlesource.com/id/I9b71bfe3425466c97159f9de0062e5e8e4fec866 --- Makefile | 1 + include/linux/kmsan-checks.h | 64 +++++ include/linux/kmsan.h | 46 ++++ include/linux/mm_types.h | 12 + include/linux/sched.h | 5 + lib/Kconfig.debug | 1 + lib/Kconfig.kmsan | 50 ++++ mm/Makefile | 1 + mm/kmsan/Makefile | 23 ++ mm/kmsan/core.c | 458 +++++++++++++++++++++++++++++++++++ mm/kmsan/hooks.c | 66 +++++ mm/kmsan/instrumentation.c | 271 +++++++++++++++++++++ mm/kmsan/kmsan.h | 190 +++++++++++++++ mm/kmsan/report.c | 211 ++++++++++++++++ mm/kmsan/shadow.c | 147 +++++++++++ scripts/Makefile.kmsan | 8 + scripts/Makefile.lib | 9 + 17 files changed, 1563 insertions(+) create mode 100644 include/linux/kmsan-checks.h create mode 100644 include/linux/kmsan.h create mode 100644 lib/Kconfig.kmsan create mode 100644 mm/kmsan/Makefile create mode 100644 mm/kmsan/core.c create mode 100644 mm/kmsan/hooks.c create mode 100644 mm/kmsan/instrumentation.c create mode 100644 mm/kmsan/kmsan.h create mode 100644 mm/kmsan/report.c create mode 100644 mm/kmsan/shadow.c create mode 100644 scripts/Makefile.kmsan diff --git a/Makefile b/Makefile index 8973b285ce6c7..7c93482f6df3d 100644 --- a/Makefile +++ b/Makefile @@ -1014,6 +1014,7 @@ include-y := scripts/Makefile.extrawarn include-$(CONFIG_DEBUG_INFO) += scripts/Makefile.debug include-$(CONFIG_KASAN) += scripts/Makefile.kasan include-$(CONFIG_KCSAN) += scripts/Makefile.kcsan +include-$(CONFIG_KMSAN) += scripts/Makefile.kmsan include-$(CONFIG_UBSAN) += scripts/Makefile.ubsan include-$(CONFIG_KCOV) += scripts/Makefile.kcov include-$(CONFIG_RANDSTRUCT) += scripts/Makefile.randstruct diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h new file mode 100644 index 0000000000000..a6522a0c28df9 --- /dev/null +++ b/include/linux/kmsan-checks.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KMSAN checks to be used for one-off annotations in subsystems. + * + * Copyright (C) 2017-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#ifndef _LINUX_KMSAN_CHECKS_H +#define _LINUX_KMSAN_CHECKS_H + +#include + +#ifdef CONFIG_KMSAN + +/** + * kmsan_poison_memory() - Mark the memory range as uninitialized. + * @address: address to start with. + * @size: size of buffer to poison. + * @flags: GFP flags for allocations done by this function. + * + * Until other data is written to this range, KMSAN will treat it as + * uninitialized. Error reports for this memory will reference the call site of + * kmsan_poison_memory() as origin. + */ +void kmsan_poison_memory(const void *address, size_t size, gfp_t flags); + +/** + * kmsan_unpoison_memory() - Mark the memory range as initialized. + * @address: address to start with. + * @size: size of buffer to unpoison. + * + * Until other data is written to this range, KMSAN will treat it as + * initialized. + */ +void kmsan_unpoison_memory(const void *address, size_t size); + +/** + * kmsan_check_memory() - Check the memory range for being initialized. + * @address: address to start with. + * @size: size of buffer to check. + * + * If any piece of the given range is marked as uninitialized, KMSAN will report + * an error. + */ +void kmsan_check_memory(const void *address, size_t size); + +#else + +static inline void kmsan_poison_memory(const void *address, size_t size, + gfp_t flags) +{ +} +static inline void kmsan_unpoison_memory(const void *address, size_t size) +{ +} +static inline void kmsan_check_memory(const void *address, size_t size) +{ +} + +#endif + +#endif /* _LINUX_KMSAN_CHECKS_H */ diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h new file mode 100644 index 0000000000000..99e48c6b049d9 --- /dev/null +++ b/include/linux/kmsan.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KMSAN API for subsystems. + * + * Copyright (C) 2017-2022 Google LLC + * Author: Alexander Potapenko + * + */ +#ifndef _LINUX_KMSAN_H +#define _LINUX_KMSAN_H + +#include +#include +#include +#include + +struct page; + +#ifdef CONFIG_KMSAN + +/* These constants are defined in the MSan LLVM instrumentation pass. */ +#define KMSAN_RETVAL_SIZE 800 +#define KMSAN_PARAM_SIZE 800 + +struct kmsan_context_state { + char param_tls[KMSAN_PARAM_SIZE]; + char retval_tls[KMSAN_RETVAL_SIZE]; + char va_arg_tls[KMSAN_PARAM_SIZE]; + char va_arg_origin_tls[KMSAN_PARAM_SIZE]; + u64 va_arg_overflow_size_tls; + char param_origin_tls[KMSAN_PARAM_SIZE]; + depot_stack_handle_t retval_origin_tls; +}; + +#undef KMSAN_PARAM_SIZE +#undef KMSAN_RETVAL_SIZE + +struct kmsan_ctx { + struct kmsan_context_state cstate; + int kmsan_in_runtime; + bool allow_reporting; +}; + +#endif + +#endif /* _LINUX_KMSAN_H */ diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index c29ab4c0cd5c6..3cc0ebdd9625f 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -218,6 +218,18 @@ struct page { not kmapped, ie. highmem) */ #endif /* WANT_PAGE_VIRTUAL */ +#ifdef CONFIG_KMSAN + /* + * KMSAN metadata for this page: + * - shadow page: every bit indicates whether the corresponding + * bit of the original page is initialized (0) or not (1); + * - origin page: every 4 bytes contain an id of the stack trace + * where the uninitialized value was created. + */ + struct page *kmsan_shadow; + struct page *kmsan_origin; +#endif + #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS int _last_cpupid; #endif diff --git a/include/linux/sched.h b/include/linux/sched.h index c46f3a63b758f..f9bb2c954e794 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -1353,6 +1354,10 @@ struct task_struct { #endif #endif +#ifdef CONFIG_KMSAN + struct kmsan_ctx kmsan_ctx; +#endif + #if IS_ENABLED(CONFIG_KUNIT) struct kunit *kunit_test; #endif diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 2e24db4bff192..59819e6fa5865 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -963,6 +963,7 @@ config DEBUG_STACKOVERFLOW source "lib/Kconfig.kasan" source "lib/Kconfig.kfence" +source "lib/Kconfig.kmsan" endmenu # "Memory Debugging" diff --git a/lib/Kconfig.kmsan b/lib/Kconfig.kmsan new file mode 100644 index 0000000000000..8f768d4034e3c --- /dev/null +++ b/lib/Kconfig.kmsan @@ -0,0 +1,50 @@ +# SPDX-License-Identifier: GPL-2.0-only +config HAVE_ARCH_KMSAN + bool + +config HAVE_KMSAN_COMPILER + # Clang versions <14.0.0 also support -fsanitize=kernel-memory, but not + # all the features necessary to build the kernel with KMSAN. + depends on CC_IS_CLANG && CLANG_VERSION >= 140000 + def_bool $(cc-option,-fsanitize=kernel-memory -mllvm -msan-disable-checks=1) + +config HAVE_KMSAN_PARAM_RETVAL + # Separate check for -fsanitize-memory-param-retval support. + depends on CC_IS_CLANG && CLANG_VERSION >= 140000 + def_bool $(cc-option,-fsanitize=kernel-memory -fsanitize-memory-param-retval) + + +config KMSAN + bool "KMSAN: detector of uninitialized values use" + depends on HAVE_ARCH_KMSAN && HAVE_KMSAN_COMPILER + depends on SLUB && DEBUG_KERNEL && !KASAN && !KCSAN + select STACKDEPOT + select STACKDEPOT_ALWAYS_INIT + help + KernelMemorySanitizer (KMSAN) is a dynamic detector of uses of + uninitialized values in the kernel. It is based on compiler + instrumentation provided by Clang and thus requires Clang to build. + + An important note is that KMSAN is not intended for production use, + because it drastically increases kernel memory footprint and slows + the whole system down. + + See for more details. + +if KMSAN + +config KMSAN_CHECK_PARAM_RETVAL + bool "Check for uninitialized values passed to and returned from functions" + default HAVE_KMSAN_PARAM_RETVAL + help + If the compiler supports -fsanitize-memory-param-retval, KMSAN will + eagerly check every function parameter passed by value and every + function return value. + + Disabling KMSAN_CHECK_PARAM_RETVAL will result in tracking shadow for + function parameters and return values across function borders. This + is a more relaxed mode, but it generates more instrumentation code and + may potentially report errors in corner cases when non-instrumented + functions call instrumented ones. + +endif diff --git a/mm/Makefile b/mm/Makefile index 6f9ffa968a1a1..ff96830153221 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -89,6 +89,7 @@ obj-$(CONFIG_SLAB) += slab.o obj-$(CONFIG_SLUB) += slub.o obj-$(CONFIG_KASAN) += kasan/ obj-$(CONFIG_KFENCE) += kfence/ +obj-$(CONFIG_KMSAN) += kmsan/ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMTEST) += memtest.o obj-$(CONFIG_MIGRATION) += migrate.o diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile new file mode 100644 index 0000000000000..550ad8625e4f9 --- /dev/null +++ b/mm/kmsan/Makefile @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for KernelMemorySanitizer (KMSAN). +# +# +obj-y := core.o instrumentation.o hooks.o report.o shadow.o + +KMSAN_SANITIZE := n +KCOV_INSTRUMENT := n +UBSAN_SANITIZE := n + +# Disable instrumentation of KMSAN runtime with other tools. +CC_FLAGS_KMSAN_RUNTIME := -fno-stack-protector +CC_FLAGS_KMSAN_RUNTIME += $(call cc-option,-fno-conserve-stack) +CC_FLAGS_KMSAN_RUNTIME += -DDISABLE_BRANCH_PROFILING + +CFLAGS_REMOVE.o = $(CC_FLAGS_FTRACE) + +CFLAGS_core.o := $(CC_FLAGS_KMSAN_RUNTIME) +CFLAGS_hooks.o := $(CC_FLAGS_KMSAN_RUNTIME) +CFLAGS_instrumentation.o := $(CC_FLAGS_KMSAN_RUNTIME) +CFLAGS_report.o := $(CC_FLAGS_KMSAN_RUNTIME) +CFLAGS_shadow.o := $(CC_FLAGS_KMSAN_RUNTIME) diff --git a/mm/kmsan/core.c b/mm/kmsan/core.c new file mode 100644 index 0000000000000..16fb8880a9c6d --- /dev/null +++ b/mm/kmsan/core.c @@ -0,0 +1,458 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN runtime library. + * + * Copyright (C) 2017-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../slab.h" +#include "kmsan.h" + +/* + * Avoid creating too long origin chains, these are unlikely to participate in + * real reports. + */ +#define MAX_CHAIN_DEPTH 7 +#define NUM_SKIPPED_TO_WARN 10000 + +bool kmsan_enabled __read_mostly; + +/* + * Per-CPU KMSAN context to be used in interrupts, where current->kmsan is + * unavaliable. + */ +DEFINE_PER_CPU(struct kmsan_ctx, kmsan_percpu_ctx); + +void kmsan_internal_poison_memory(void *address, size_t size, gfp_t flags, + unsigned int poison_flags) +{ + u32 extra_bits = + kmsan_extra_bits(/*depth*/ 0, poison_flags & KMSAN_POISON_FREE); + bool checked = poison_flags & KMSAN_POISON_CHECK; + depot_stack_handle_t handle; + + handle = kmsan_save_stack_with_flags(flags, extra_bits); + kmsan_internal_set_shadow_origin(address, size, -1, handle, checked); +} + +void kmsan_internal_unpoison_memory(void *address, size_t size, bool checked) +{ + kmsan_internal_set_shadow_origin(address, size, 0, 0, checked); +} + +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, + unsigned int extra) +{ + unsigned long entries[KMSAN_STACK_DEPTH]; + unsigned int nr_entries; + + nr_entries = stack_trace_save(entries, KMSAN_STACK_DEPTH, 0); + + /* Don't sleep (see might_sleep_if() in __alloc_pages_nodemask()). */ + flags &= ~__GFP_DIRECT_RECLAIM; + + return __stack_depot_save(entries, nr_entries, extra, flags, true); +} + +/* Copy the metadata following the memmove() behavior. */ +void kmsan_internal_memmove_metadata(void *dst, void *src, size_t n) +{ + depot_stack_handle_t old_origin = 0, new_origin = 0; + int src_slots, dst_slots, i, iter, step, skip_bits; + depot_stack_handle_t *origin_src, *origin_dst; + void *shadow_src, *shadow_dst; + u32 *align_shadow_src, shadow; + bool backwards; + + shadow_dst = kmsan_get_metadata(dst, KMSAN_META_SHADOW); + if (!shadow_dst) + return; + KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(dst, n)); + + shadow_src = kmsan_get_metadata(src, KMSAN_META_SHADOW); + if (!shadow_src) { + /* + * @src is untracked: zero out destination shadow, ignore the + * origins, we're done. + */ + __memset(shadow_dst, 0, n); + return; + } + KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(src, n)); + + __memmove(shadow_dst, shadow_src, n); + + origin_dst = kmsan_get_metadata(dst, KMSAN_META_ORIGIN); + origin_src = kmsan_get_metadata(src, KMSAN_META_ORIGIN); + KMSAN_WARN_ON(!origin_dst || !origin_src); + src_slots = (ALIGN((u64)src + n, KMSAN_ORIGIN_SIZE) - + ALIGN_DOWN((u64)src, KMSAN_ORIGIN_SIZE)) / + KMSAN_ORIGIN_SIZE; + dst_slots = (ALIGN((u64)dst + n, KMSAN_ORIGIN_SIZE) - + ALIGN_DOWN((u64)dst, KMSAN_ORIGIN_SIZE)) / + KMSAN_ORIGIN_SIZE; + KMSAN_WARN_ON((src_slots < 1) || (dst_slots < 1)); + KMSAN_WARN_ON((src_slots - dst_slots > 1) || + (dst_slots - src_slots < -1)); + + backwards = dst > src; + i = backwards ? min(src_slots, dst_slots) - 1 : 0; + iter = backwards ? -1 : 1; + + align_shadow_src = + (u32 *)ALIGN_DOWN((u64)shadow_src, KMSAN_ORIGIN_SIZE); + for (step = 0; step < min(src_slots, dst_slots); step++, i += iter) { + KMSAN_WARN_ON(i < 0); + shadow = align_shadow_src[i]; + if (i == 0) { + /* + * If @src isn't aligned on KMSAN_ORIGIN_SIZE, don't + * look at the first @src % KMSAN_ORIGIN_SIZE bytes + * of the first shadow slot. + */ + skip_bits = ((u64)src % KMSAN_ORIGIN_SIZE) * 8; + shadow = (shadow >> skip_bits) << skip_bits; + } + if (i == src_slots - 1) { + /* + * If @src + n isn't aligned on + * KMSAN_ORIGIN_SIZE, don't look at the last + * (@src + n) % KMSAN_ORIGIN_SIZE bytes of the + * last shadow slot. + */ + skip_bits = (((u64)src + n) % KMSAN_ORIGIN_SIZE) * 8; + shadow = (shadow << skip_bits) >> skip_bits; + } + /* + * Overwrite the origin only if the corresponding + * shadow is nonempty. + */ + if (origin_src[i] && (origin_src[i] != old_origin) && shadow) { + old_origin = origin_src[i]; + new_origin = kmsan_internal_chain_origin(old_origin); + /* + * kmsan_internal_chain_origin() may return + * NULL, but we don't want to lose the previous + * origin value. + */ + if (!new_origin) + new_origin = old_origin; + } + if (shadow) + origin_dst[i] = new_origin; + else + origin_dst[i] = 0; + } + /* + * If dst_slots is greater than src_slots (i.e. + * dst_slots == src_slots + 1), there is an extra origin slot at the + * beginning or end of the destination buffer, for which we take the + * origin from the previous slot. + * This is only done if the part of the source shadow corresponding to + * slot is non-zero. + * + * E.g. if we copy 8 aligned bytes that are marked as uninitialized + * and have origins o111 and o222, to an unaligned buffer with offset 1, + * these two origins are copied to three origin slots, so one of then + * needs to be duplicated, depending on the copy direction (@backwards) + * + * src shadow: |uuuu|uuuu|....| + * src origin: |o111|o222|....| + * + * backwards = 0: + * dst shadow: |.uuu|uuuu|u...| + * dst origin: |....|o111|o222| - fill the empty slot with o111 + * backwards = 1: + * dst shadow: |.uuu|uuuu|u...| + * dst origin: |o111|o222|....| - fill the empty slot with o222 + */ + if (src_slots < dst_slots) { + if (backwards) { + shadow = align_shadow_src[src_slots - 1]; + skip_bits = (((u64)dst + n) % KMSAN_ORIGIN_SIZE) * 8; + shadow = (shadow << skip_bits) >> skip_bits; + if (shadow) + /* src_slots > 0, therefore dst_slots is at least 2 */ + origin_dst[dst_slots - 1] = origin_dst[dst_slots - 2]; + } else { + shadow = align_shadow_src[0]; + skip_bits = ((u64)dst % KMSAN_ORIGIN_SIZE) * 8; + shadow = (shadow >> skip_bits) << skip_bits; + if (shadow) + origin_dst[0] = origin_dst[1]; + } + } +} + +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id) +{ + unsigned long entries[3]; + u32 extra_bits; + int depth; + bool uaf; + + if (!id) + return id; + /* + * Make sure we have enough spare bits in @id to hold the UAF bit and + * the chain depth. + */ + BUILD_BUG_ON((1 << STACK_DEPOT_EXTRA_BITS) <= (MAX_CHAIN_DEPTH << 1)); + + extra_bits = stack_depot_get_extra_bits(id); + depth = kmsan_depth_from_eb(extra_bits); + uaf = kmsan_uaf_from_eb(extra_bits); + + if (depth >= MAX_CHAIN_DEPTH) { + static atomic_long_t kmsan_skipped_origins; + long skipped = atomic_long_inc_return(&kmsan_skipped_origins); + + if (skipped % NUM_SKIPPED_TO_WARN == 0) { + pr_warn("not chained %ld origins\n", skipped); + dump_stack(); + kmsan_print_origin(id); + } + return id; + } + depth++; + extra_bits = kmsan_extra_bits(depth, uaf); + + entries[0] = KMSAN_CHAIN_MAGIC_ORIGIN; + entries[1] = kmsan_save_stack_with_flags(GFP_ATOMIC, 0); + entries[2] = id; + /* + * @entries is a local var in non-instrumented code, so KMSAN does not + * know it is initialized. Explicitly unpoison it to avoid false + * positives when __stack_depot_save() passes it to instrumented code. + */ + kmsan_internal_unpoison_memory(entries, sizeof(entries), false); + return __stack_depot_save(entries, ARRAY_SIZE(entries), extra_bits, + GFP_ATOMIC, true); +} + +void kmsan_internal_set_shadow_origin(void *addr, size_t size, int b, + u32 origin, bool checked) +{ + u64 address = (u64)addr; + void *shadow_start; + u32 *origin_start; + size_t pad = 0; + int i; + + KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(addr, size)); + shadow_start = kmsan_get_metadata(addr, KMSAN_META_SHADOW); + if (!shadow_start) { + /* + * kmsan_metadata_is_contiguous() is true, so either all shadow + * and origin pages are NULL, or all are non-NULL. + */ + if (checked) { + pr_err("%s: not memsetting %ld bytes starting at %px, because the shadow is NULL\n", + __func__, size, addr); + KMSAN_WARN_ON(true); + } + return; + } + __memset(shadow_start, b, size); + + if (!IS_ALIGNED(address, KMSAN_ORIGIN_SIZE)) { + pad = address % KMSAN_ORIGIN_SIZE; + address -= pad; + size += pad; + } + size = ALIGN(size, KMSAN_ORIGIN_SIZE); + origin_start = + (u32 *)kmsan_get_metadata((void *)address, KMSAN_META_ORIGIN); + + for (i = 0; i < size / KMSAN_ORIGIN_SIZE; i++) + origin_start[i] = origin; +} + +struct page *kmsan_vmalloc_to_page_or_null(void *vaddr) +{ + struct page *page; + + if (!kmsan_internal_is_vmalloc_addr(vaddr) && + !kmsan_internal_is_module_addr(vaddr)) + return NULL; + page = vmalloc_to_page(vaddr); + if (pfn_valid(page_to_pfn(page))) + return page; + else + return NULL; +} + +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr, + int reason) +{ + depot_stack_handle_t cur_origin = 0, new_origin = 0; + unsigned long addr64 = (unsigned long)addr; + depot_stack_handle_t *origin = NULL; + unsigned char *shadow = NULL; + int cur_off_start = -1; + int i, chunk_size; + size_t pos = 0; + + if (!size) + return; + KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(addr, size)); + while (pos < size) { + chunk_size = min(size - pos, + PAGE_SIZE - ((addr64 + pos) % PAGE_SIZE)); + shadow = kmsan_get_metadata((void *)(addr64 + pos), + KMSAN_META_SHADOW); + if (!shadow) { + /* + * This page is untracked. If there were uninitialized + * bytes before, report them. + */ + if (cur_origin) { + kmsan_enter_runtime(); + kmsan_report(cur_origin, addr, size, + cur_off_start, pos - 1, user_addr, + reason); + kmsan_leave_runtime(); + } + cur_origin = 0; + cur_off_start = -1; + pos += chunk_size; + continue; + } + for (i = 0; i < chunk_size; i++) { + if (!shadow[i]) { + /* + * This byte is unpoisoned. If there were + * poisoned bytes before, report them. + */ + if (cur_origin) { + kmsan_enter_runtime(); + kmsan_report(cur_origin, addr, size, + cur_off_start, pos + i - 1, + user_addr, reason); + kmsan_leave_runtime(); + } + cur_origin = 0; + cur_off_start = -1; + continue; + } + origin = kmsan_get_metadata((void *)(addr64 + pos + i), + KMSAN_META_ORIGIN); + KMSAN_WARN_ON(!origin); + new_origin = *origin; + /* + * Encountered new origin - report the previous + * uninitialized range. + */ + if (cur_origin != new_origin) { + if (cur_origin) { + kmsan_enter_runtime(); + kmsan_report(cur_origin, addr, size, + cur_off_start, pos + i - 1, + user_addr, reason); + kmsan_leave_runtime(); + } + cur_origin = new_origin; + cur_off_start = pos + i; + } + } + pos += chunk_size; + } + KMSAN_WARN_ON(pos != size); + if (cur_origin) { + kmsan_enter_runtime(); + kmsan_report(cur_origin, addr, size, cur_off_start, pos - 1, + user_addr, reason); + kmsan_leave_runtime(); + } +} + +bool kmsan_metadata_is_contiguous(void *addr, size_t size) +{ + char *cur_shadow = NULL, *next_shadow = NULL, *cur_origin = NULL, + *next_origin = NULL; + u64 cur_addr = (u64)addr, next_addr = cur_addr + PAGE_SIZE; + depot_stack_handle_t *origin_p; + bool all_untracked = false; + + if (!size) + return true; + + /* The whole range belongs to the same page. */ + if (ALIGN_DOWN(cur_addr + size - 1, PAGE_SIZE) == + ALIGN_DOWN(cur_addr, PAGE_SIZE)) + return true; + + cur_shadow = kmsan_get_metadata((void *)cur_addr, /*is_origin*/ false); + if (!cur_shadow) + all_untracked = true; + cur_origin = kmsan_get_metadata((void *)cur_addr, /*is_origin*/ true); + if (all_untracked && cur_origin) + goto report; + + for (; next_addr < (u64)addr + size; + cur_addr = next_addr, cur_shadow = next_shadow, + cur_origin = next_origin, next_addr += PAGE_SIZE) { + next_shadow = kmsan_get_metadata((void *)next_addr, false); + next_origin = kmsan_get_metadata((void *)next_addr, true); + if (all_untracked) { + if (next_shadow || next_origin) + goto report; + if (!next_shadow && !next_origin) + continue; + } + if (((u64)cur_shadow == ((u64)next_shadow - PAGE_SIZE)) && + ((u64)cur_origin == ((u64)next_origin - PAGE_SIZE))) + continue; + goto report; + } + return true; + +report: + pr_err("%s: attempting to access two shadow page ranges.\n", __func__); + pr_err("Access of size %ld at %px.\n", size, addr); + pr_err("Addresses belonging to different ranges: %px and %px\n", + (void *)cur_addr, (void *)next_addr); + pr_err("page[0].shadow: %px, page[1].shadow: %px\n", cur_shadow, + next_shadow); + pr_err("page[0].origin: %px, page[1].origin: %px\n", cur_origin, + next_origin); + origin_p = kmsan_get_metadata(addr, KMSAN_META_ORIGIN); + if (origin_p) { + pr_err("Origin: %08x\n", *origin_p); + kmsan_print_origin(*origin_p); + } else { + pr_err("Origin: unavailable\n"); + } + return false; +} + +bool kmsan_internal_is_module_addr(void *vaddr) +{ + return ((u64)vaddr >= MODULES_VADDR) && ((u64)vaddr < MODULES_END); +} + +bool kmsan_internal_is_vmalloc_addr(void *addr) +{ + return ((u64)addr >= VMALLOC_START) && ((u64)addr < VMALLOC_END); +} diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c new file mode 100644 index 0000000000000..4ac62fa67a02a --- /dev/null +++ b/mm/kmsan/hooks.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN hooks for kernel subsystems. + * + * These functions handle creation of KMSAN metadata for memory allocations. + * + * Copyright (C) 2018-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#include +#include +#include +#include +#include +#include + +#include "../internal.h" +#include "../slab.h" +#include "kmsan.h" + +/* + * Instrumented functions shouldn't be called under + * kmsan_enter_runtime()/kmsan_leave_runtime(), because this will lead to + * skipping effects of functions like memset() inside instrumented code. + */ + +/* Functions from kmsan-checks.h follow. */ +void kmsan_poison_memory(const void *address, size_t size, gfp_t flags) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + kmsan_enter_runtime(); + /* The users may want to poison/unpoison random memory. */ + kmsan_internal_poison_memory((void *)address, size, flags, + KMSAN_POISON_NOCHECK); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_poison_memory); + +void kmsan_unpoison_memory(const void *address, size_t size) +{ + unsigned long ua_flags; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + ua_flags = user_access_save(); + kmsan_enter_runtime(); + /* The users may want to poison/unpoison random memory. */ + kmsan_internal_unpoison_memory((void *)address, size, + KMSAN_POISON_NOCHECK); + kmsan_leave_runtime(); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(kmsan_unpoison_memory); + +void kmsan_check_memory(const void *addr, size_t size) +{ + if (!kmsan_enabled) + return; + return kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0, + REASON_ANY); +} +EXPORT_SYMBOL(kmsan_check_memory); diff --git a/mm/kmsan/instrumentation.c b/mm/kmsan/instrumentation.c new file mode 100644 index 0000000000000..1b705162be8c2 --- /dev/null +++ b/mm/kmsan/instrumentation.c @@ -0,0 +1,271 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN compiler API. + * + * This file implements __msan_XXX hooks that Clang inserts into the code + * compiled with -fsanitize=kernel-memory. + * See Documentation/dev-tools/kmsan.rst for more information on how KMSAN + * instrumentation works. + * + * Copyright (C) 2017-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#include "kmsan.h" +#include +#include +#include + +static inline bool is_bad_asm_addr(void *addr, uintptr_t size, bool is_store) +{ + if ((u64)addr < TASK_SIZE) + return true; + if (!kmsan_get_metadata(addr, KMSAN_META_SHADOW)) + return true; + return false; +} + +static inline struct shadow_origin_ptr +get_shadow_origin_ptr(void *addr, u64 size, bool store) +{ + unsigned long ua_flags = user_access_save(); + struct shadow_origin_ptr ret; + + ret = kmsan_get_shadow_origin_ptr(addr, size, store); + user_access_restore(ua_flags); + return ret; +} + +/* Get shadow and origin pointers for a memory load with non-standard size. */ +struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, + uintptr_t size) +{ + return get_shadow_origin_ptr(addr, size, /*store*/ false); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n); + +/* Get shadow and origin pointers for a memory store with non-standard size. */ +struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, + uintptr_t size) +{ + return get_shadow_origin_ptr(addr, size, /*store*/ true); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n); + +/* + * Declare functions that obtain shadow/origin pointers for loads and stores + * with fixed size. + */ +#define DECLARE_METADATA_PTR_GETTER(size) \ + struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size( \ + void *addr) \ + { \ + return get_shadow_origin_ptr(addr, size, /*store*/ false); \ + } \ + EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \ + struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size( \ + void *addr) \ + { \ + return get_shadow_origin_ptr(addr, size, /*store*/ true); \ + } \ + EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size) + +DECLARE_METADATA_PTR_GETTER(1); +DECLARE_METADATA_PTR_GETTER(2); +DECLARE_METADATA_PTR_GETTER(4); +DECLARE_METADATA_PTR_GETTER(8); + +/* + * Handle a memory store performed by inline assembly. KMSAN conservatively + * attempts to unpoison the outputs of asm() directives to prevent false + * positives caused by missed stores. + */ +void __msan_instrument_asm_store(void *addr, uintptr_t size) +{ + unsigned long ua_flags; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + ua_flags = user_access_save(); + /* + * Most of the accesses are below 32 bytes. The two exceptions so far + * are clwb() (64 bytes) and FPU state (512 bytes). + * It's unlikely that the assembly will touch more than 512 bytes. + */ + if (size > 512) { + WARN_ONCE(1, "assembly store size too big: %ld\n", size); + size = 8; + } + if (is_bad_asm_addr(addr, size, /*is_store*/ true)) { + user_access_restore(ua_flags); + return; + } + kmsan_enter_runtime(); + /* Unpoisoning the memory on best effort. */ + kmsan_internal_unpoison_memory(addr, size, /*checked*/ false); + kmsan_leave_runtime(); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(__msan_instrument_asm_store); + +/* Handle llvm.memmove intrinsic. */ +void *__msan_memmove(void *dst, const void *src, uintptr_t n) +{ + void *result; + + result = __memmove(dst, src, n); + if (!n) + /* Some people call memmove() with zero length. */ + return result; + if (!kmsan_enabled || kmsan_in_runtime()) + return result; + + kmsan_enter_runtime(); + kmsan_internal_memmove_metadata(dst, (void *)src, n); + kmsan_leave_runtime(); + + return result; +} +EXPORT_SYMBOL(__msan_memmove); + +/* Handle llvm.memcpy intrinsic. */ +void *__msan_memcpy(void *dst, const void *src, uintptr_t n) +{ + void *result; + + result = __memcpy(dst, src, n); + if (!n) + /* Some people call memcpy() with zero length. */ + return result; + + if (!kmsan_enabled || kmsan_in_runtime()) + return result; + + kmsan_enter_runtime(); + /* Using memmove instead of memcpy doesn't affect correctness. */ + kmsan_internal_memmove_metadata(dst, (void *)src, n); + kmsan_leave_runtime(); + + return result; +} +EXPORT_SYMBOL(__msan_memcpy); + +/* Handle llvm.memset intrinsic. */ +void *__msan_memset(void *dst, int c, uintptr_t n) +{ + void *result; + + result = __memset(dst, c, n); + if (!kmsan_enabled || kmsan_in_runtime()) + return result; + + kmsan_enter_runtime(); + /* + * Clang doesn't pass parameter metadata here, so it is impossible to + * use shadow of @c to set up the shadow for @dst. + */ + kmsan_internal_unpoison_memory(dst, n, /*checked*/ false); + kmsan_leave_runtime(); + + return result; +} +EXPORT_SYMBOL(__msan_memset); + +/* + * Create a new origin from an old one. This is done when storing an + * uninitialized value to memory. When reporting an error, KMSAN unrolls and + * prints the whole chain of stores that preceded the use of this value. + */ +depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin) +{ + depot_stack_handle_t ret = 0; + unsigned long ua_flags; + + if (!kmsan_enabled || kmsan_in_runtime()) + return ret; + + ua_flags = user_access_save(); + + /* Creating new origins may allocate memory. */ + kmsan_enter_runtime(); + ret = kmsan_internal_chain_origin(origin); + kmsan_leave_runtime(); + user_access_restore(ua_flags); + return ret; +} +EXPORT_SYMBOL(__msan_chain_origin); + +/* Poison a local variable when entering a function. */ +void __msan_poison_alloca(void *address, uintptr_t size, char *descr) +{ + depot_stack_handle_t handle; + unsigned long entries[4]; + unsigned long ua_flags; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + ua_flags = user_access_save(); + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN; + entries[1] = (u64)descr; + entries[2] = (u64)__builtin_return_address(0); + /* + * With frame pointers enabled, it is possible to quickly fetch the + * second frame of the caller stack without calling the unwinder. + * Without them, simply do not bother. + */ + if (IS_ENABLED(CONFIG_UNWINDER_FRAME_POINTER)) + entries[3] = (u64)__builtin_return_address(1); + else + entries[3] = 0; + + /* stack_depot_save() may allocate memory. */ + kmsan_enter_runtime(); + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC); + kmsan_leave_runtime(); + + kmsan_internal_set_shadow_origin(address, size, -1, handle, + /*checked*/ true); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(__msan_poison_alloca); + +/* Unpoison a local variable. */ +void __msan_unpoison_alloca(void *address, uintptr_t size) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + kmsan_enter_runtime(); + kmsan_internal_unpoison_memory(address, size, /*checked*/ true); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(__msan_unpoison_alloca); + +/* + * Report that an uninitialized value with the given origin was used in a way + * that constituted undefined behavior. + */ +void __msan_warning(u32 origin) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + kmsan_enter_runtime(); + kmsan_report(origin, /*address*/ 0, /*size*/ 0, + /*off_first*/ 0, /*off_last*/ 0, /*user_addr*/ 0, + REASON_ANY); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(__msan_warning); + +/* + * At the beginning of an instrumented function, obtain the pointer to + * `struct kmsan_context_state` holding the metadata for function parameters. + */ +struct kmsan_context_state *__msan_get_context_state(void) +{ + return &kmsan_get_context()->cstate; +} +EXPORT_SYMBOL(__msan_get_context_state); diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h new file mode 100644 index 0000000000000..d3c400ca097ba --- /dev/null +++ b/mm/kmsan/kmsan.h @@ -0,0 +1,190 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Functions used by the KMSAN runtime. + * + * Copyright (C) 2017-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#ifndef __MM_KMSAN_KMSAN_H +#define __MM_KMSAN_KMSAN_H + +#include +#include +#include +#include +#include +#include +#include +#include + +#define KMSAN_ALLOCA_MAGIC_ORIGIN 0xabcd0100 +#define KMSAN_CHAIN_MAGIC_ORIGIN 0xabcd0200 + +#define KMSAN_POISON_NOCHECK 0x0 +#define KMSAN_POISON_CHECK 0x1 +#define KMSAN_POISON_FREE 0x2 + +#define KMSAN_ORIGIN_SIZE 4 + +#define KMSAN_STACK_DEPTH 64 + +#define KMSAN_META_SHADOW (false) +#define KMSAN_META_ORIGIN (true) + +extern bool kmsan_enabled; +extern int panic_on_kmsan; + +/* + * KMSAN performs a lot of consistency checks that are currently enabled by + * default. BUG_ON is normally discouraged in the kernel, unless used for + * debugging, but KMSAN itself is a debugging tool, so it makes little sense to + * recover if something goes wrong. + */ +#define KMSAN_WARN_ON(cond) \ + ({ \ + const bool __cond = WARN_ON(cond); \ + if (unlikely(__cond)) { \ + WRITE_ONCE(kmsan_enabled, false); \ + if (panic_on_kmsan) { \ + /* Can't call panic() here because */ \ + /* of uaccess checks. */ \ + BUG(); \ + } \ + } \ + __cond; \ + }) + +/* + * A pair of metadata pointers to be returned by the instrumentation functions. + */ +struct shadow_origin_ptr { + void *shadow, *origin; +}; + +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size, + bool store); +void *kmsan_get_metadata(void *addr, bool is_origin); + +enum kmsan_bug_reason { + REASON_ANY, + REASON_COPY_TO_USER, + REASON_SUBMIT_URB, +}; + +void kmsan_print_origin(depot_stack_handle_t origin); + +/** + * kmsan_report() - Report a use of uninitialized value. + * @origin: Stack ID of the uninitialized value. + * @address: Address at which the memory access happens. + * @size: Memory access size. + * @off_first: Offset (from @address) of the first byte to be reported. + * @off_last: Offset (from @address) of the last byte to be reported. + * @user_addr: When non-NULL, denotes the userspace address to which the kernel + * is leaking data. + * @reason: Error type from enum kmsan_bug_reason. + * + * kmsan_report() prints an error message for a consequent group of bytes + * sharing the same origin. If an uninitialized value is used in a comparison, + * this function is called once without specifying the addresses. When checking + * a memory range, KMSAN may call kmsan_report() multiple times with the same + * @address, @size, @user_addr and @reason, but different @off_first and + * @off_last corresponding to different @origin values. + */ +void kmsan_report(depot_stack_handle_t origin, void *address, int size, + int off_first, int off_last, const void *user_addr, + enum kmsan_bug_reason reason); + +DECLARE_PER_CPU(struct kmsan_ctx, kmsan_percpu_ctx); + +static __always_inline struct kmsan_ctx *kmsan_get_context(void) +{ + return in_task() ? ¤t->kmsan_ctx : raw_cpu_ptr(&kmsan_percpu_ctx); +} + +/* + * When a compiler hook or KMSAN runtime function is invoked, it may make a + * call to instrumented code and eventually call itself recursively. To avoid + * that, we guard the runtime entry regions with + * kmsan_enter_runtime()/kmsan_leave_runtime() and exit the hook if + * kmsan_in_runtime() is true. + * + * Non-runtime code may occasionally get executed in nested IRQs from the + * runtime code (e.g. when called via smp_call_function_single()). Because some + * KMSAN routines may take locks (e.g. for memory allocation), we conservatively + * bail out instead of calling them. To minimize the effect of this (potentially + * missing initialization events) kmsan_in_runtime() is not checked in + * non-blocking runtime functions. + */ +static __always_inline bool kmsan_in_runtime(void) +{ + if ((hardirq_count() >> HARDIRQ_SHIFT) > 1) + return true; + return kmsan_get_context()->kmsan_in_runtime; +} + +static __always_inline void kmsan_enter_runtime(void) +{ + struct kmsan_ctx *ctx; + + ctx = kmsan_get_context(); + KMSAN_WARN_ON(ctx->kmsan_in_runtime++); +} + +static __always_inline void kmsan_leave_runtime(void) +{ + struct kmsan_ctx *ctx = kmsan_get_context(); + + KMSAN_WARN_ON(--ctx->kmsan_in_runtime); +} + +depot_stack_handle_t kmsan_save_stack(void); +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, + unsigned int extra_bits); + +/* + * Pack and unpack the origin chain depth and UAF flag to/from the extra bits + * provided by the stack depot. + * The UAF flag is stored in the lowest bit, followed by the depth in the upper + * bits. + * set_dsh_extra_bits() is responsible for clamping the value. + */ +static __always_inline unsigned int kmsan_extra_bits(unsigned int depth, + bool uaf) +{ + return (depth << 1) | uaf; +} + +static __always_inline bool kmsan_uaf_from_eb(unsigned int extra_bits) +{ + return extra_bits & 1; +} + +static __always_inline unsigned int kmsan_depth_from_eb(unsigned int extra_bits) +{ + return extra_bits >> 1; +} + +/* + * kmsan_internal_ functions are supposed to be very simple and not require the + * kmsan_in_runtime() checks. + */ +void kmsan_internal_memmove_metadata(void *dst, void *src, size_t n); +void kmsan_internal_poison_memory(void *address, size_t size, gfp_t flags, + unsigned int poison_flags); +void kmsan_internal_unpoison_memory(void *address, size_t size, bool checked); +void kmsan_internal_set_shadow_origin(void *address, size_t size, int b, + u32 origin, bool checked); +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id); + +bool kmsan_metadata_is_contiguous(void *addr, size_t size); +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr, + int reason); +bool kmsan_internal_is_module_addr(void *vaddr); +bool kmsan_internal_is_vmalloc_addr(void *addr); + +struct page *kmsan_vmalloc_to_page_or_null(void *vaddr); + +#endif /* __MM_KMSAN_KMSAN_H */ diff --git a/mm/kmsan/report.c b/mm/kmsan/report.c new file mode 100644 index 0000000000000..c298edcf49ee5 --- /dev/null +++ b/mm/kmsan/report.c @@ -0,0 +1,211 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN error reporting routines. + * + * Copyright (C) 2019-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#include +#include +#include +#include +#include + +#include "kmsan.h" + +static DEFINE_RAW_SPINLOCK(kmsan_report_lock); +#define DESCR_SIZE 128 +/* Protected by kmsan_report_lock */ +static char report_local_descr[DESCR_SIZE]; +int panic_on_kmsan __read_mostly; + +#ifdef MODULE_PARAM_PREFIX +#undef MODULE_PARAM_PREFIX +#endif +#define MODULE_PARAM_PREFIX "kmsan." +module_param_named(panic, panic_on_kmsan, int, 0); + +/* + * Skip internal KMSAN frames. + */ +static int get_stack_skipnr(const unsigned long stack_entries[], + int num_entries) +{ + int len, skip; + char buf[64]; + + for (skip = 0; skip < num_entries; ++skip) { + len = scnprintf(buf, sizeof(buf), "%ps", + (void *)stack_entries[skip]); + + /* Never show __msan_* or kmsan_* functions. */ + if ((strnstr(buf, "__msan_", len) == buf) || + (strnstr(buf, "kmsan_", len) == buf)) + continue; + + /* + * No match for runtime functions -- @skip entries to skip to + * get to first frame of interest. + */ + break; + } + + return skip; +} + +/* + * Currently the descriptions of locals generated by Clang look as follows: + * ----local_name@function_name + * We want to print only the name of the local, as other information in that + * description can be confusing. + * The meaningful part of the description is copied to a global buffer to avoid + * allocating memory. + */ +static char *pretty_descr(char *descr) +{ + int i, pos = 0, len = strlen(descr); + + for (i = 0; i < len; i++) { + if (descr[i] == '@') + break; + if (descr[i] == '-') + continue; + report_local_descr[pos] = descr[i]; + if (pos + 1 == DESCR_SIZE) + break; + pos++; + } + report_local_descr[pos] = 0; + return report_local_descr; +} + +void kmsan_print_origin(depot_stack_handle_t origin) +{ + unsigned long *entries = NULL, *chained_entries = NULL; + unsigned int nr_entries, chained_nr_entries, skipnr; + void *pc1 = NULL, *pc2 = NULL; + depot_stack_handle_t head; + unsigned long magic; + char *descr = NULL; + + if (!origin) + return; + + while (true) { + nr_entries = stack_depot_fetch(origin, &entries); + magic = nr_entries ? entries[0] : 0; + if ((nr_entries == 4) && (magic == KMSAN_ALLOCA_MAGIC_ORIGIN)) { + descr = (char *)entries[1]; + pc1 = (void *)entries[2]; + pc2 = (void *)entries[3]; + pr_err("Local variable %s created at:\n", + pretty_descr(descr)); + if (pc1) + pr_err(" %pSb\n", pc1); + if (pc2) + pr_err(" %pSb\n", pc2); + break; + } + if ((nr_entries == 3) && (magic == KMSAN_CHAIN_MAGIC_ORIGIN)) { + head = entries[1]; + origin = entries[2]; + pr_err("Uninit was stored to memory at:\n"); + chained_nr_entries = + stack_depot_fetch(head, &chained_entries); + kmsan_internal_unpoison_memory( + chained_entries, + chained_nr_entries * sizeof(*chained_entries), + /*checked*/ false); + skipnr = get_stack_skipnr(chained_entries, + chained_nr_entries); + stack_trace_print(chained_entries + skipnr, + chained_nr_entries - skipnr, 0); + pr_err("\n"); + continue; + } + pr_err("Uninit was created at:\n"); + if (nr_entries) { + skipnr = get_stack_skipnr(entries, nr_entries); + stack_trace_print(entries + skipnr, nr_entries - skipnr, + 0); + } else { + pr_err("(stack is not available)\n"); + } + break; + } +} + +void kmsan_report(depot_stack_handle_t origin, void *address, int size, + int off_first, int off_last, const void *user_addr, + enum kmsan_bug_reason reason) +{ + unsigned long stack_entries[KMSAN_STACK_DEPTH]; + int num_stack_entries, skipnr; + char *bug_type = NULL; + unsigned long ua_flags; + bool is_uaf; + + if (!kmsan_enabled) + return; + if (!current->kmsan_ctx.allow_reporting) + return; + if (!origin) + return; + + current->kmsan_ctx.allow_reporting = false; + ua_flags = user_access_save(); + raw_spin_lock(&kmsan_report_lock); + pr_err("=====================================================\n"); + is_uaf = kmsan_uaf_from_eb(stack_depot_get_extra_bits(origin)); + switch (reason) { + case REASON_ANY: + bug_type = is_uaf ? "use-after-free" : "uninit-value"; + break; + case REASON_COPY_TO_USER: + bug_type = is_uaf ? "kernel-infoleak-after-free" : + "kernel-infoleak"; + break; + case REASON_SUBMIT_URB: + bug_type = is_uaf ? "kernel-usb-infoleak-after-free" : + "kernel-usb-infoleak"; + break; + } + + num_stack_entries = + stack_trace_save(stack_entries, KMSAN_STACK_DEPTH, 1); + skipnr = get_stack_skipnr(stack_entries, num_stack_entries); + + pr_err("BUG: KMSAN: %s in %pSb\n", + bug_type, (void *)stack_entries[skipnr]); + stack_trace_print(stack_entries + skipnr, num_stack_entries - skipnr, + 0); + pr_err("\n"); + + kmsan_print_origin(origin); + + if (size) { + pr_err("\n"); + if (off_first == off_last) + pr_err("Byte %d of %d is uninitialized\n", off_first, + size); + else + pr_err("Bytes %d-%d of %d are uninitialized\n", + off_first, off_last, size); + } + if (address) + pr_err("Memory access of size %d starts at %px\n", size, + address); + if (user_addr && reason == REASON_COPY_TO_USER) + pr_err("Data copied to user address %px\n", user_addr); + pr_err("\n"); + dump_stack_print_info(KERN_ERR); + pr_err("=====================================================\n"); + add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); + raw_spin_unlock(&kmsan_report_lock); + if (panic_on_kmsan) + panic("kmsan.panic set ...\n"); + user_access_restore(ua_flags); + current->kmsan_ctx.allow_reporting = true; +} diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c new file mode 100644 index 0000000000000..e5ad2972d7362 --- /dev/null +++ b/mm/kmsan/shadow.c @@ -0,0 +1,147 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN shadow implementation. + * + * Copyright (C) 2017-2022 Google LLC + * Author: Alexander Potapenko + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../internal.h" +#include "kmsan.h" + +#define shadow_page_for(page) ((page)->kmsan_shadow) + +#define origin_page_for(page) ((page)->kmsan_origin) + +static void *shadow_ptr_for(struct page *page) +{ + return page_address(shadow_page_for(page)); +} + +static void *origin_ptr_for(struct page *page) +{ + return page_address(origin_page_for(page)); +} + +static bool page_has_metadata(struct page *page) +{ + return shadow_page_for(page) && origin_page_for(page); +} + +static void set_no_shadow_origin_page(struct page *page) +{ + shadow_page_for(page) = NULL; + origin_page_for(page) = NULL; +} + +/* + * Dummy load and store pages to be used when the real metadata is unavailable. + * There are separate pages for loads and stores, so that every load returns a + * zero, and every store doesn't affect other loads. + */ +static char dummy_load_page[PAGE_SIZE] __aligned(PAGE_SIZE); +static char dummy_store_page[PAGE_SIZE] __aligned(PAGE_SIZE); + +static unsigned long vmalloc_meta(void *addr, bool is_origin) +{ + unsigned long addr64 = (unsigned long)addr, off; + + KMSAN_WARN_ON(is_origin && !IS_ALIGNED(addr64, KMSAN_ORIGIN_SIZE)); + if (kmsan_internal_is_vmalloc_addr(addr)) { + off = addr64 - VMALLOC_START; + return off + (is_origin ? KMSAN_VMALLOC_ORIGIN_START : + KMSAN_VMALLOC_SHADOW_START); + } + if (kmsan_internal_is_module_addr(addr)) { + off = addr64 - MODULES_VADDR; + return off + (is_origin ? KMSAN_MODULES_ORIGIN_START : + KMSAN_MODULES_SHADOW_START); + } + return 0; +} + +static struct page *virt_to_page_or_null(void *vaddr) +{ + if (kmsan_virt_addr_valid(vaddr)) + return virt_to_page(vaddr); + else + return NULL; +} + +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *address, u64 size, + bool store) +{ + struct shadow_origin_ptr ret; + void *shadow; + + /* + * Even if we redirect this memory access to the dummy page, it will + * go out of bounds. + */ + KMSAN_WARN_ON(size > PAGE_SIZE); + + if (!kmsan_enabled) + goto return_dummy; + + KMSAN_WARN_ON(!kmsan_metadata_is_contiguous(address, size)); + shadow = kmsan_get_metadata(address, KMSAN_META_SHADOW); + if (!shadow) + goto return_dummy; + + ret.shadow = shadow; + ret.origin = kmsan_get_metadata(address, KMSAN_META_ORIGIN); + return ret; + +return_dummy: + if (store) { + /* Ignore this store. */ + ret.shadow = dummy_store_page; + ret.origin = dummy_store_page; + } else { + /* This load will return zero. */ + ret.shadow = dummy_load_page; + ret.origin = dummy_load_page; + } + return ret; +} + +/* + * Obtain the shadow or origin pointer for the given address, or NULL if there's + * none. The caller must check the return value for being non-NULL if needed. + * The return value of this function should not depend on whether we're in the + * runtime or not. + */ +void *kmsan_get_metadata(void *address, bool is_origin) +{ + u64 addr = (u64)address, pad, off; + struct page *page; + + if (is_origin && !IS_ALIGNED(addr, KMSAN_ORIGIN_SIZE)) { + pad = addr % KMSAN_ORIGIN_SIZE; + addr -= pad; + } + address = (void *)addr; + if (kmsan_internal_is_vmalloc_addr(address) || + kmsan_internal_is_module_addr(address)) + return (void *)vmalloc_meta(address, is_origin); + + page = virt_to_page_or_null(address); + if (!page) + return NULL; + if (!page_has_metadata(page)) + return NULL; + off = addr % PAGE_SIZE; + + return (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off; +} diff --git a/scripts/Makefile.kmsan b/scripts/Makefile.kmsan new file mode 100644 index 0000000000000..b5b0aa61322ec --- /dev/null +++ b/scripts/Makefile.kmsan @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +kmsan-cflags := -fsanitize=kernel-memory + +ifdef CONFIG_KMSAN_CHECK_PARAM_RETVAL +kmsan-cflags += -fsanitize-memory-param-retval +endif + +export CFLAGS_KMSAN := $(kmsan-cflags) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index d1425778664b9..46ebf7cb081f6 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -157,6 +157,15 @@ _c_flags += $(if $(patsubst n%,, \ endif endif +ifeq ($(CONFIG_KMSAN),y) +_c_flags += $(if $(patsubst n%,, \ + $(KMSAN_SANITIZE_$(basetarget).o)$(KMSAN_SANITIZE)y), \ + $(CFLAGS_KMSAN)) +_c_flags += $(if $(patsubst n%,, \ + $(KMSAN_ENABLE_CHECKS_$(basetarget).o)$(KMSAN_ENABLE_CHECKS)y), \ + , -mllvm -msan-disable-checks=1) +endif + ifeq ($(CONFIG_UBSAN),y) _c_flags += $(if $(patsubst n%,, \ $(UBSAN_SANITIZE_$(basetarget).o)$(UBSAN_SANITIZE)$(CONFIG_UBSAN_SANITIZE_ALL)), \ From patchwork Fri Jul 1 14:22:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3BD8C433EF for ; Fri, 1 Jul 2022 14:23:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 695E86B0083; Fri, 1 Jul 2022 10:23:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6458F6B0085; Fri, 1 Jul 2022 10:23:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 537346B0087; Fri, 1 Jul 2022 10:23:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3BCF96B0083 for ; Fri, 1 Jul 2022 10:23:51 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 18B538085F for ; Fri, 1 Jul 2022 14:23:51 +0000 (UTC) X-FDA: 79638749862.02.AA30AD1 Received: from mail-lj1-f202.google.com (mail-lj1-f202.google.com [209.85.208.202]) by imf03.hostedemail.com (Postfix) with ESMTP id A3AC320039 for ; Fri, 1 Jul 2022 14:23:50 +0000 (UTC) Received: by mail-lj1-f202.google.com with SMTP id d24-20020a2eb058000000b0025a7f5ccae6so501666ljl.14 for ; Fri, 01 Jul 2022 07:23:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=prMZkaWSelYFGUo1Yp0oSlk2FGuv/jH48kix0SjfSqo=; b=hUrPjZ5xQAK1rq1Z9VjevL+FrrINSG0tRRhgiAbMolgJVMWGIx4r5RfIMIwzpxc6ms gUg7hKctDb4wQE3VOaCHWIV2LGy5Ucp2ZAB7kF9GJdFo1NEyX0t952wecwLOOKno1+2E 1rgmYSk4NtJQfzw9K/W+3BAffRewOQGfttd7giwJGo9MV8LuZwy/QsokSAd/dtQcFE3M z+L8uHLCgsAlvhMmnJc8Wq3E2JUFEG6y6kF7zDOvDO2SeAdTgKBtF3nkCOG4j2QwCqre z530iem10UaCweSv2liUUURMK8s4nPk+0u5brQ3OdzHhBaeQjctMcqzwscixL/Fwjs+v C1qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=prMZkaWSelYFGUo1Yp0oSlk2FGuv/jH48kix0SjfSqo=; b=xPOYAst7ciLQZ2ws6ojxSeesYY1AU4HzUOPHa9GXK4QHkYQYdxNj5yalb/q7aJALZw Wccz4xnWpHZTk+W5EzScB6ouO+F2/BZCKfOMM6hn7VByHY4zmbH6nuVOPOCagdE15JYT /NEowTk3f1bNbFQTcXjyybsJ1oDGLJOsMXL+704FPrVq6zB5jYfYUDgJ2UIHwVRP3Txi gcEdld0uY3rAc2shsUVMqx7ybueRXxWFbtN2HMeIVvZcFbzt0kT8CArfnUCJYMIFbuhd tQxUFUUlbgcp2ATHsuHyvJzncBiIAazxetqMs1HlhyocTa221VClQucVJBDuV46VYW1+ 64tQ== X-Gm-Message-State: AJIora9QBO/5k0cFlUd6fBXD9pDaBrr5LSO/1QVYNFzrCXm1fNvYBRPw RWwlfqmNiUnhzc/THo6Kvb3rdLskaSw= X-Google-Smtp-Source: AGRyM1tkPv2qxZS7ch8ZP9uL8wXLQqYDrfQpsgzEv2crKp3uD9M2y3fPjLDqyCgehGEb/p3HkajEMnoQMWA= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6512:2622:b0:481:5b17:58e7 with SMTP id bt34-20020a056512262200b004815b1758e7mr2552760lfb.600.1656685428893; Fri, 01 Jul 2022 07:23:48 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:37 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-13-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 12/45] kmsan: disable instrumentation of unsupported common kernel code From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=hUrPjZ5x; spf=pass (imf03.hostedemail.com: domain of 3dAO_YgYKCJM38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com designates 209.85.208.202 as permitted sender) smtp.mailfrom=3dAO_YgYKCJM38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=prMZkaWSelYFGUo1Yp0oSlk2FGuv/jH48kix0SjfSqo=; b=HT0h78QzUPsWUUmegNdnnpBRgkwnoh3fQurrf5SNOmbBO+/ZhAmAPPBSIVGLGGauiScj8/ aWKMBPCfIiR7Z3EJ5/wYznVxdW9XF9ovJJyDImHc7LofaS5N1hh7PGDmtV+vTePAkJBtOZ ibgLDmALffAekZ2qHu2BZUqQ8s3oz5I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685430; a=rsa-sha256; cv=none; b=xx0afgJJgIfIT20dpepcYhl4+TQqZQVuIz25RvQ2h4RONCp1LhoQxNBOUpMRNscdEKY9L7 BXgtADM3bwknQnlgJJf597/DyM2y7EqJ7B8EOaUh8umrspBhIE/bwEEkYEy+7QL5rGak/O Qg7bYH9WPQwPpPLnEACAYro2CvWziOU= X-Stat-Signature: 49t4xg8hzgfhbdfaart6fu7nt45u8eba X-Rspamd-Queue-Id: A3AC320039 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=hUrPjZ5x; spf=pass (imf03.hostedemail.com: domain of 3dAO_YgYKCJM38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com designates 209.85.208.202 as permitted sender) smtp.mailfrom=3dAO_YgYKCJM38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-HE-Tag: 1656685430-993189 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: EFI stub cannot be linked with KMSAN runtime, so we disable instrumentation for it. Instrumenting kcov, stackdepot or lockdep leads to infinite recursion caused by instrumentation hooks calling instrumented code again. This patch was previously part of "kmsan: disable KMSAN instrumentation for certain kernel parts", but was split away per Mark Rutland's request. Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- Link: https://linux-review.googlesource.com/id/I41ae706bd3474f074f6a870bfc3f0f90e9c720f7 --- drivers/firmware/efi/libstub/Makefile | 1 + kernel/Makefile | 1 + kernel/locking/Makefile | 3 ++- lib/Makefile | 1 + 4 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index d0537573501e9..81432d0c904b1 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -46,6 +46,7 @@ GCOV_PROFILE := n # Sanitizer runtimes are unavailable and cannot be linked here. KASAN_SANITIZE := n KCSAN_SANITIZE := n +KMSAN_SANITIZE := n UBSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y diff --git a/kernel/Makefile b/kernel/Makefile index a7e1f49ab2b3b..e47f0526c987f 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -38,6 +38,7 @@ KCOV_INSTRUMENT_kcov.o := n KASAN_SANITIZE_kcov.o := n KCSAN_SANITIZE_kcov.o := n UBSAN_SANITIZE_kcov.o := n +KMSAN_SANITIZE_kcov.o := n CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) -fno-stack-protector # Don't instrument error handlers diff --git a/kernel/locking/Makefile b/kernel/locking/Makefile index d51cabf28f382..ea925731fa40f 100644 --- a/kernel/locking/Makefile +++ b/kernel/locking/Makefile @@ -5,8 +5,9 @@ KCOV_INSTRUMENT := n obj-y += mutex.o semaphore.o rwsem.o percpu-rwsem.o -# Avoid recursion lockdep -> KCSAN -> ... -> lockdep. +# Avoid recursion lockdep -> sanitizer -> ... -> lockdep. KCSAN_SANITIZE_lockdep.o := n +KMSAN_SANITIZE_lockdep.o := n ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_lockdep.o = $(CC_FLAGS_FTRACE) diff --git a/lib/Makefile b/lib/Makefile index f99bf61f8bbc6..5056769d00bb6 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -272,6 +272,7 @@ obj-$(CONFIG_POLYNOMIAL) += polynomial.o CFLAGS_stackdepot.o += -fno-builtin obj-$(CONFIG_STACKDEPOT) += stackdepot.o KASAN_SANITIZE_stackdepot.o := n +KMSAN_SANITIZE_stackdepot.o := n KCOV_INSTRUMENT_stackdepot.o := n obj-$(CONFIG_REF_TRACKER) += ref_tracker.o From patchwork Fri Jul 1 14:22:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37AEDC433EF for ; Fri, 1 Jul 2022 14:23:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BED0A6B0085; Fri, 1 Jul 2022 10:23:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B509D6B0087; Fri, 1 Jul 2022 10:23:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A172B6B0088; Fri, 1 Jul 2022 10:23:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8DE4B6B0085 for ; Fri, 1 Jul 2022 10:23:53 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 673FF120C23 for ; Fri, 1 Jul 2022 14:23:53 +0000 (UTC) X-FDA: 79638749946.03.C139277 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf26.hostedemail.com (Postfix) with ESMTP id CE0CA140024 for ; Fri, 1 Jul 2022 14:23:52 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id x2-20020a1709065ac200b006d9b316257fso839780ejs.12 for ; Fri, 01 Jul 2022 07:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PCXlNDV7asGuiYne5zMLVF95C5LbNPo0tZu8h92qf24=; b=At0kpR+7GbMYGbD3FSMWgm2JP0BV8RH8Rto+ZODMMn/z0B3vgzhxxWFmIqaMmM9L3C aNCmXfo6YUoUIiTJElUOiEnHhSBIqPy8a7onAGUAl9n4baUa/KcBUobt9fplCCRo7R+2 FJo7QcbMRDLlB8vftJOnq+M2J1oN7gTUZDWR3uyxo+NUMhcERhdykf6ykg37DhNfFVtM x3I5LV5DF8QOxuVcmdOlWAmT5YLzMlab3EWZE6++Fq0YccDaGwEWg4rtgrg7cGiHpldO RsbX66G8NwOBHTBYFwYfODhKpIkfxrnU8VZ4zPBZ+qzKi6r+qU7sLkEnyV6MCpb4vpNM 2+Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PCXlNDV7asGuiYne5zMLVF95C5LbNPo0tZu8h92qf24=; b=j21z1H7TCfh9PLu+FUFpXR6PKQGxyUrdsIaYd2ehzzIRCTKcBqpys2pzv6ZHXHKZx5 KA0Ds2YT6g5BAfm33+T9VtGbaTJltM/QLbDz8COAfaAlsSOTE7p7mHjyOSws0HD/xyvk DoekQeshUauPS09B0kbplq9lL3ORAL/pc9syXqrgrTNE0JO4sgB9jkG47l85zAiDmeR5 htmk1mJOwh/CYHlSPft7yr6Kp/G+Q95sHhjsT0FUv3AO+9O2YfpuqGup0i4MAWJNZ7Vb Y2J6Np8YBngh/79gufpJQnK6kBDfOjZF0+VDJ+lkNa0bbkNaDa7A51E2Uuv3g25b+LQi A3jA== X-Gm-Message-State: AJIora9nGBgJqPAa3s20jtFJ1tRaOLTitUat82ATcWh/ZxGrRihTmeuZ OIqPHM0cXGZyjkNke4i20tQXZ/+SQLk= X-Google-Smtp-Source: AGRyM1u9I17I9UxxKSpcqjnVIo/mN4dyAvFgl5Bm+pBE1XybOt2h9kEiW1jwTQ35IIGsBNaKQVlCw0/gS1E= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:906:c781:b0:726:c967:8d1b with SMTP id cw1-20020a170906c78100b00726c9678d1bmr14665371ejb.54.1656685431730; Fri, 01 Jul 2022 07:23:51 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:38 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-14-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 13/45] MAINTAINERS: add entry for KMSAN From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685433; a=rsa-sha256; cv=none; b=0H6WuXmJpRnZaJRJiK37wWNzb3PI9ECdx7kNuohfU5ksY1mSq/JP0Iw0MLLl8/hc5L4xbF 0HVjicadTO/jHZfpCr5e601LZqeNPOsg0x6P0UPdG6QcjeV2H5ynCmcM/VCQKo4vVycDkW BBZFzwLKLCMc6iVJsQ4QITMi8icKkww= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=At0kpR+7; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 3dwO_YgYKCJY6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3dwO_YgYKCJY6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PCXlNDV7asGuiYne5zMLVF95C5LbNPo0tZu8h92qf24=; b=nVbIB6MOIz9/fB+EPgFd4knARqcNSVyi4Uu8+eU8QoS6PT+Vhy3+XUt09XaaomCzuMmVDO jD1lFgZsDXs01B1A5UQl2riouJe570BioPYsHX+YRVq5OfmctThtIHYvGL1/NyLuPJ5JPT FHx7iZuTAwbC2w9Rv5gcm35UI0H/aCo= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CE0CA140024 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=At0kpR+7; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 3dwO_YgYKCJY6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3dwO_YgYKCJY6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: czcdfszm3pe4j6dsmbwqk1kzn4fbq6ez X-HE-Tag: 1656685432-141583 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add entry for KMSAN maintainers/reviewers. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Ic5836c2bceb6b63f71a60d3327d18af3aa3dab77 --- MAINTAINERS | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index fe5daf1415013..f56281df30284 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -11106,6 +11106,18 @@ F: kernel/kmod.c F: lib/test_kmod.c F: tools/testing/selftests/kmod/ +KMSAN +M: Alexander Potapenko +R: Marco Elver +R: Dmitry Vyukov +L: kasan-dev@googlegroups.com +S: Maintained +F: Documentation/dev-tools/kmsan.rst +F: include/linux/kmsan*.h +F: lib/Kconfig.kmsan +F: mm/kmsan/ +F: scripts/Makefile.kmsan + KPROBES M: Naveen N. Rao M: Anil S Keshavamurthy From patchwork Fri Jul 1 14:22:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86BDEC433EF for ; Fri, 1 Jul 2022 14:23:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B3C26B0087; Fri, 1 Jul 2022 10:23:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 164D26B0088; Fri, 1 Jul 2022 10:23:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 004176B0089; Fri, 1 Jul 2022 10:23:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DE1266B0087 for ; Fri, 1 Jul 2022 10:23:57 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9BD4D34B88 for ; Fri, 1 Jul 2022 14:23:57 +0000 (UTC) X-FDA: 79638750114.17.8679025 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf18.hostedemail.com (Postfix) with ESMTP id 1745E1C002B for ; Fri, 1 Jul 2022 14:23:55 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id y5-20020a056402358500b0043592ac3961so1866180edc.6 for ; Fri, 01 Jul 2022 07:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zch90Kb1PH444z16gANLEx2ctA8AD9SJjapQIjqFsi4=; b=pnk7oQtFARPFwXoB7UQH8a+lShrkN/dZfogG5GgCvhFXsN88RluWz+ZjGnD7oYUdIw cSAA2elsGgUBj0N1M87IT0lO+gzJw5NsZk0pUoabL84U06CrW3epHcEoxKZDoTfe1LxE IC2iQFQHw18iRFcs9AhivljPkYhG9+EsemtgcZvDv+4iPrTkYcJTPWF5Qp1gEXnao22F WuPHtwwaA30+tR4AmC6g+PR+HpqAoB04SL8qGnjhJHcUOE6O9aYUG1Gt0rxanbg58DoW CIOkkK5G42M6r1U3xVSfIsqfE42vyc9QnRfxKZmDfDHpDwCFuzIcbOQxQHcD5gp/n6gE jbzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zch90Kb1PH444z16gANLEx2ctA8AD9SJjapQIjqFsi4=; b=C9tGxonPYG9Np2k9GyixSiK7ah7WerZ89URxYUCpr/VnGPykLlnLBuikwSXuxY3nA2 j6N0jBzZPJOQAW79eFdYOSSEoG1zBT/5k3Q4gKjv0lvG2GM+03rR1O5PdpFYvvmvSb8r 8ywVCUjua59qd5z+5Q6Lxrhk2SfG8IpYW/OSt3G+qfk5lcwbthuaGR/B2mZKq3IrSNbh BkvkJTCFcH+/jySXgiA1eIYIfRqBqx4LCFefxp08dKatnDuscM0eCeb6QaisYCfprYwp iJzgXA4DHxr4K1wwypav0DYCLEzmGBC8SLdXhJJYck/5/vGKvkOH7RaQ0kyv4cs6P6tm prWw== X-Gm-Message-State: AJIora+K42zisYtJ0ZDppog7lgMgwJ1ZQJWJGCQEkrSBp6Cf4SHvHWVF +LTzKHarKx/wf2bo/pxJGvsBo9bztVQ= X-Google-Smtp-Source: AGRyM1sz81cWisEJq9r2TpHBDU5N+Ew2N2b9W2QTXS0gGFF23C1Ujb2ac0Cgjhi/K2VMxzuq9KL7J9yxvPI= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:3f1d:b0:726:c927:769b with SMTP id hq29-20020a1709073f1d00b00726c927769bmr14568479ejc.754.1656685434644; Fri, 01 Jul 2022 07:23:54 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:39 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-15-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 14/45] mm: kmsan: maintain KMSAN metadata for page operations From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685436; a=rsa-sha256; cv=none; b=NFUrmKoCv2Dl7z/rDZ6z85cWVV+o/05UpGM3NR3L8R/+UMMl+mrej0hbtTyEXDXP1PXtS9 1/LcdLHH1fZ0aVcj1FZP4LtWtPcr0t/Qc64ZY1mqCVioMOejoaTAOlHawzhEGrGVQosUK7 O6uBMOLTjsFjW+Fc2U3PDeUOGzIk0F4= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=pnk7oQtF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of 3egO_YgYKCJk9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3egO_YgYKCJk9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685436; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zch90Kb1PH444z16gANLEx2ctA8AD9SJjapQIjqFsi4=; b=T348Xurwt3/31x1T3VVVi2gUwcKwx0wYl+Bg6lteOctS1KOd6QlfK5yqDXyTwQZugFyknL deBGrmToXW7xnMnRq5TN3KKgXPVzj7MAT0mEdJTjusTwwlfzOjpr9OONd+TSOarsiBC7rf Sehy6Z3HzdiyoFwvx+bx5mYMNcnTVmE= Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=pnk7oQtF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of 3egO_YgYKCJk9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3egO_YgYKCJk9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1745E1C002B X-Stat-Signature: e919k8oar4fwgjynx1we44z4wjeg378i X-Rspam-User: X-HE-Tag: 1656685435-107956 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Insert KMSAN hooks that make the necessary bookkeeping changes: - poison page shadow and origins in alloc_pages()/free_page(); - clear page shadow and origins in clear_page(), copy_user_highpage(); - copy page metadata in copy_highpage(), wp_page_copy(); - handle vmap()/vunmap()/iounmap(); Signed-off-by: Alexander Potapenko --- v2: -- move page metadata hooks implementation here -- remove call to kmsan_memblock_free_pages() v3: -- use PAGE_SHIFT in kmsan_ioremap_page_range() v4: -- change sizeof(type) to sizeof(*ptr) -- replace occurrences of |var| with @var -- swap mm: and kmsan: in the subject -- drop __no_sanitize_memory from clear_page() Link: https://linux-review.googlesource.com/id/I6d4f53a0e7eab46fa29f0348f3095d9f2e326850 --- arch/x86/include/asm/page_64.h | 12 ++++ arch/x86/mm/ioremap.c | 3 + include/linux/highmem.h | 3 + include/linux/kmsan.h | 123 +++++++++++++++++++++++++++++++++ mm/internal.h | 6 ++ mm/kmsan/hooks.c | 87 +++++++++++++++++++++++ mm/kmsan/shadow.c | 114 ++++++++++++++++++++++++++++++ mm/memory.c | 2 + mm/page_alloc.c | 11 +++ mm/vmalloc.c | 20 +++++- 10 files changed, 379 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index baa70451b8df5..227dd33eb4efb 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -45,14 +45,26 @@ void clear_page_orig(void *page); void clear_page_rep(void *page); void clear_page_erms(void *page); +/* This is an assembly header, avoid including too much of kmsan.h */ +#ifdef CONFIG_KMSAN +void kmsan_unpoison_memory(const void *addr, size_t size); +#endif static inline void clear_page(void *page) { +#ifdef CONFIG_KMSAN + /* alternative_call_2() changes @page. */ + void *page_copy = page; +#endif alternative_call_2(clear_page_orig, clear_page_rep, X86_FEATURE_REP_GOOD, clear_page_erms, X86_FEATURE_ERMS, "=D" (page), "0" (page) : "cc", "memory", "rax", "rcx"); +#ifdef CONFIG_KMSAN + /* Clear KMSAN shadow for the pages that have it. */ + kmsan_unpoison_memory(page_copy, PAGE_SIZE); +#endif } void copy_page(void *to, void *from); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index 1ad0228f8ceb9..78c5bc654cff5 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include @@ -479,6 +480,8 @@ void iounmap(volatile void __iomem *addr) return; } + kmsan_iounmap_page_range((unsigned long)addr, + (unsigned long)addr + get_vm_area_size(p)); memtype_free(p->phys_addr, p->phys_addr + get_vm_area_size(p)); /* Finally remove it */ diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 3af34de54330c..ae82c5aefb018 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -302,6 +303,7 @@ static inline void copy_user_highpage(struct page *to, struct page *from, vfrom = kmap_local_page(from); vto = kmap_local_page(to); copy_user_page(vto, vfrom, vaddr, to); + kmsan_unpoison_memory(page_address(to), PAGE_SIZE); kunmap_local(vto); kunmap_local(vfrom); } @@ -317,6 +319,7 @@ static inline void copy_highpage(struct page *to, struct page *from) vfrom = kmap_local_page(from); vto = kmap_local_page(to); copy_page(vto, vfrom); + kmsan_copy_page_meta(to, from); kunmap_local(vto); kunmap_local(vfrom); } diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index 99e48c6b049d9..699fe4f5b3bee 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -41,6 +41,129 @@ struct kmsan_ctx { bool allow_reporting; }; +/** + * kmsan_alloc_page() - Notify KMSAN about an alloc_pages() call. + * @page: struct page pointer returned by alloc_pages(). + * @order: order of allocated struct page. + * @flags: GFP flags used by alloc_pages() + * + * KMSAN marks 1<<@order pages starting at @page as uninitialized, unless + * @flags contain __GFP_ZERO. + */ +void kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags); + +/** + * kmsan_free_page() - Notify KMSAN about a free_pages() call. + * @page: struct page pointer passed to free_pages(). + * @order: order of deallocated struct page. + * + * KMSAN marks freed memory as uninitialized. + */ +void kmsan_free_page(struct page *page, unsigned int order); + +/** + * kmsan_copy_page_meta() - Copy KMSAN metadata between two pages. + * @dst: destination page. + * @src: source page. + * + * KMSAN copies the contents of metadata pages for @src into the metadata pages + * for @dst. If @dst has no associated metadata pages, nothing happens. + * If @src has no associated metadata pages, @dst metadata pages are unpoisoned. + */ +void kmsan_copy_page_meta(struct page *dst, struct page *src); + +/** + * kmsan_map_kernel_range_noflush() - Notify KMSAN about a vmap. + * @start: start of vmapped range. + * @end: end of vmapped range. + * @prot: page protection flags used for vmap. + * @pages: array of pages. + * @page_shift: page_shift passed to vmap_range_noflush(). + * + * KMSAN maps shadow and origin pages of @pages into contiguous ranges in + * vmalloc metadata address range. + */ +void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end, + pgprot_t prot, struct page **pages, + unsigned int page_shift); + +/** + * kmsan_vunmap_kernel_range_noflush() - Notify KMSAN about a vunmap. + * @start: start of vunmapped range. + * @end: end of vunmapped range. + * + * KMSAN unmaps the contiguous metadata ranges created by + * kmsan_map_kernel_range_noflush(). + */ +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end); + +/** + * kmsan_ioremap_page_range() - Notify KMSAN about a ioremap_page_range() call. + * @addr: range start. + * @end: range end. + * @phys_addr: physical range start. + * @prot: page protection flags used for ioremap_page_range(). + * @page_shift: page_shift argument passed to vmap_range_noflush(). + * + * KMSAN creates new metadata pages for the physical pages mapped into the + * virtual memory. + */ +void kmsan_ioremap_page_range(unsigned long addr, unsigned long end, + phys_addr_t phys_addr, pgprot_t prot, + unsigned int page_shift); + +/** + * kmsan_iounmap_page_range() - Notify KMSAN about a iounmap_page_range() call. + * @start: range start. + * @end: range end. + * + * KMSAN unmaps the metadata pages for the given range and, unlike for + * vunmap_page_range(), also deallocates them. + */ +void kmsan_iounmap_page_range(unsigned long start, unsigned long end); + +#else + +static inline int kmsan_alloc_page(struct page *page, unsigned int order, + gfp_t flags) +{ + return 0; +} + +static inline void kmsan_free_page(struct page *page, unsigned int order) +{ +} + +static inline void kmsan_copy_page_meta(struct page *dst, struct page *src) +{ +} + +static inline void kmsan_vmap_pages_range_noflush(unsigned long start, + unsigned long end, + pgprot_t prot, + struct page **pages, + unsigned int page_shift) +{ +} + +static inline void kmsan_vunmap_range_noflush(unsigned long start, + unsigned long end) +{ +} + +static inline void kmsan_ioremap_page_range(unsigned long start, + unsigned long end, + phys_addr_t phys_addr, + pgprot_t prot, + unsigned int page_shift) +{ +} + +static inline void kmsan_iounmap_page_range(unsigned long start, + unsigned long end) +{ +} + #endif #endif /* _LINUX_KMSAN_H */ diff --git a/mm/internal.h b/mm/internal.h index c0f8fbe0445b5..dccdba2ac4ecf 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -847,8 +847,14 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end, } #endif +int __vmap_pages_range_noflush(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, + unsigned int page_shift); + void vunmap_range_noflush(unsigned long start, unsigned long end); +void __vunmap_range_noflush(unsigned long start, unsigned long end); + int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, unsigned long addr, int page_nid, int *flags); diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 4ac62fa67a02a..070756be70e3a 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -26,6 +26,93 @@ * skipping effects of functions like memset() inside instrumented code. */ +static unsigned long vmalloc_shadow(unsigned long addr) +{ + return (unsigned long)kmsan_get_metadata((void *)addr, + KMSAN_META_SHADOW); +} + +static unsigned long vmalloc_origin(unsigned long addr) +{ + return (unsigned long)kmsan_get_metadata((void *)addr, + KMSAN_META_ORIGIN); +} + +void kmsan_vunmap_range_noflush(unsigned long start, unsigned long end) +{ + __vunmap_range_noflush(vmalloc_shadow(start), vmalloc_shadow(end)); + __vunmap_range_noflush(vmalloc_origin(start), vmalloc_origin(end)); + flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end)); + flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end)); +} +EXPORT_SYMBOL(kmsan_vunmap_range_noflush); + +/* + * This function creates new shadow/origin pages for the physical pages mapped + * into the virtual memory. If those physical pages already had shadow/origin, + * those are ignored. + */ +void kmsan_ioremap_page_range(unsigned long start, unsigned long end, + phys_addr_t phys_addr, pgprot_t prot, + unsigned int page_shift) +{ + gfp_t gfp_mask = GFP_KERNEL | __GFP_ZERO; + struct page *shadow, *origin; + unsigned long off = 0; + int i, nr; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + nr = (end - start) / PAGE_SIZE; + kmsan_enter_runtime(); + for (i = 0; i < nr; i++, off += PAGE_SIZE) { + shadow = alloc_pages(gfp_mask, 1); + origin = alloc_pages(gfp_mask, 1); + __vmap_pages_range_noflush( + vmalloc_shadow(start + off), + vmalloc_shadow(start + off + PAGE_SIZE), prot, &shadow, + PAGE_SHIFT); + __vmap_pages_range_noflush( + vmalloc_origin(start + off), + vmalloc_origin(start + off + PAGE_SIZE), prot, &origin, + PAGE_SHIFT); + } + flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end)); + flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end)); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_ioremap_page_range); + +void kmsan_iounmap_page_range(unsigned long start, unsigned long end) +{ + unsigned long v_shadow, v_origin; + struct page *shadow, *origin; + int i, nr; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + nr = (end - start) / PAGE_SIZE; + kmsan_enter_runtime(); + v_shadow = (unsigned long)vmalloc_shadow(start); + v_origin = (unsigned long)vmalloc_origin(start); + for (i = 0; i < nr; i++, v_shadow += PAGE_SIZE, v_origin += PAGE_SIZE) { + shadow = kmsan_vmalloc_to_page_or_null((void *)v_shadow); + origin = kmsan_vmalloc_to_page_or_null((void *)v_origin); + __vunmap_range_noflush(v_shadow, vmalloc_shadow(end)); + __vunmap_range_noflush(v_origin, vmalloc_origin(end)); + if (shadow) + __free_pages(shadow, 1); + if (origin) + __free_pages(origin, 1); + } + flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end)); + flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end)); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_iounmap_page_range); + /* Functions from kmsan-checks.h follow. */ void kmsan_poison_memory(const void *address, size_t size, gfp_t flags) { diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c index e5ad2972d7362..416cb85487a1a 100644 --- a/mm/kmsan/shadow.c +++ b/mm/kmsan/shadow.c @@ -145,3 +145,117 @@ void *kmsan_get_metadata(void *address, bool is_origin) return (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off; } + +void kmsan_copy_page_meta(struct page *dst, struct page *src) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + if (!dst || !page_has_metadata(dst)) + return; + if (!src || !page_has_metadata(src)) { + kmsan_internal_unpoison_memory(page_address(dst), PAGE_SIZE, + /*checked*/ false); + return; + } + + kmsan_enter_runtime(); + __memcpy(shadow_ptr_for(dst), shadow_ptr_for(src), PAGE_SIZE); + __memcpy(origin_ptr_for(dst), origin_ptr_for(src), PAGE_SIZE); + kmsan_leave_runtime(); +} + +void kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags) +{ + bool initialized = (flags & __GFP_ZERO) || !kmsan_enabled; + struct page *shadow, *origin; + depot_stack_handle_t handle; + int pages = 1 << order; + int i; + + if (!page) + return; + + shadow = shadow_page_for(page); + origin = origin_page_for(page); + + if (initialized) { + __memset(page_address(shadow), 0, PAGE_SIZE * pages); + __memset(page_address(origin), 0, PAGE_SIZE * pages); + return; + } + + /* Zero pages allocated by the runtime should also be initialized. */ + if (kmsan_in_runtime()) + return; + + __memset(page_address(shadow), -1, PAGE_SIZE * pages); + kmsan_enter_runtime(); + handle = kmsan_save_stack_with_flags(flags, /*extra_bits*/ 0); + kmsan_leave_runtime(); + /* + * Addresses are page-aligned, pages are contiguous, so it's ok + * to just fill the origin pages with @handle. + */ + for (i = 0; i < PAGE_SIZE * pages / sizeof(handle); i++) + ((depot_stack_handle_t *)page_address(origin))[i] = handle; +} + +void kmsan_free_page(struct page *page, unsigned int order) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + kmsan_enter_runtime(); + kmsan_internal_poison_memory(page_address(page), + PAGE_SIZE << compound_order(page), + GFP_KERNEL, + KMSAN_POISON_CHECK | KMSAN_POISON_FREE); + kmsan_leave_runtime(); +} + +void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end, + pgprot_t prot, struct page **pages, + unsigned int page_shift) +{ + unsigned long shadow_start, origin_start, shadow_end, origin_end; + struct page **s_pages, **o_pages; + int nr, i, mapped; + + if (!kmsan_enabled) + return; + + shadow_start = vmalloc_meta((void *)start, KMSAN_META_SHADOW); + shadow_end = vmalloc_meta((void *)end, KMSAN_META_SHADOW); + if (!shadow_start) + return; + + nr = (end - start) / PAGE_SIZE; + s_pages = kcalloc(nr, sizeof(*s_pages), GFP_KERNEL); + o_pages = kcalloc(nr, sizeof(*o_pages), GFP_KERNEL); + if (!s_pages || !o_pages) + goto ret; + for (i = 0; i < nr; i++) { + s_pages[i] = shadow_page_for(pages[i]); + o_pages[i] = origin_page_for(pages[i]); + } + prot = __pgprot(pgprot_val(prot) | _PAGE_NX); + prot = PAGE_KERNEL; + + origin_start = vmalloc_meta((void *)start, KMSAN_META_ORIGIN); + origin_end = vmalloc_meta((void *)end, KMSAN_META_ORIGIN); + kmsan_enter_runtime(); + mapped = __vmap_pages_range_noflush(shadow_start, shadow_end, prot, + s_pages, page_shift); + KMSAN_WARN_ON(mapped); + mapped = __vmap_pages_range_noflush(origin_start, origin_end, prot, + o_pages, page_shift); + KMSAN_WARN_ON(mapped); + kmsan_leave_runtime(); + flush_tlb_kernel_range(shadow_start, shadow_end); + flush_tlb_kernel_range(origin_start, origin_end); + flush_cache_vmap(shadow_start, shadow_end); + flush_cache_vmap(origin_start, origin_end); + +ret: + kfree(s_pages); + kfree(o_pages); +} diff --git a/mm/memory.c b/mm/memory.c index 7a089145cad4b..947349399e05c 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -3120,6 +3121,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) delayacct_wpcopy_end(); return 0; } + kmsan_copy_page_meta(new_page, old_page); } if (mem_cgroup_charge(page_folio(new_page), mm, GFP_KERNEL)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e008a3df0485c..785459251145e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1320,6 +1321,7 @@ static __always_inline bool free_pages_prepare(struct page *page, VM_BUG_ON_PAGE(PageTail(page), page); trace_mm_page_free(page, order); + kmsan_free_page(page, order); if (unlikely(PageHWPoison(page)) && !order) { /* @@ -3711,6 +3713,14 @@ static struct page *rmqueue_pcplist(struct zone *preferred_zone, /* * Allocate a page from the given zone. Use pcplists for order-0 allocations. */ + +/* + * Do not instrument rmqueue() with KMSAN. This function may call + * __msan_poison_alloca() through a call to set_pfnblock_flags_mask(). + * If __msan_poison_alloca() attempts to allocate pages for the stack depot, it + * may call rmqueue() again, which will result in a deadlock. + */ +__no_sanitize_memory static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, @@ -5446,6 +5456,7 @@ struct page *__alloc_pages(gfp_t gfp, unsigned int order, int preferred_nid, } trace_mm_page_alloc(page, order, alloc_gfp, ac.migratetype); + kmsan_alloc_page(page, order, alloc_gfp); return page; } diff --git a/mm/vmalloc.c b/mm/vmalloc.c index effd1ff6a4b41..6973d7f1ef934 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -320,6 +320,9 @@ int ioremap_page_range(unsigned long addr, unsigned long end, err = vmap_range_noflush(addr, end, phys_addr, pgprot_nx(prot), ioremap_max_page_shift); flush_cache_vmap(addr, end); + if (!err) + kmsan_ioremap_page_range(addr, end, phys_addr, prot, + ioremap_max_page_shift); return err; } @@ -416,7 +419,7 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, * * This is an internal function only. Do not use outside mm/. */ -void vunmap_range_noflush(unsigned long start, unsigned long end) +void __vunmap_range_noflush(unsigned long start, unsigned long end) { unsigned long next; pgd_t *pgd; @@ -438,6 +441,12 @@ void vunmap_range_noflush(unsigned long start, unsigned long end) arch_sync_kernel_mappings(start, end); } +void vunmap_range_noflush(unsigned long start, unsigned long end) +{ + kmsan_vunmap_range_noflush(start, end); + __vunmap_range_noflush(start, end); +} + /** * vunmap_range - unmap kernel virtual addresses * @addr: start of the VM area to unmap @@ -575,7 +584,7 @@ static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end, * * This is an internal function only. Do not use outside mm/. */ -int vmap_pages_range_noflush(unsigned long addr, unsigned long end, +int __vmap_pages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift) { unsigned int i, nr = (end - addr) >> PAGE_SHIFT; @@ -601,6 +610,13 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end, return 0; } +int vmap_pages_range_noflush(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, unsigned int page_shift) +{ + kmsan_vmap_pages_range_noflush(addr, end, prot, pages, page_shift); + return __vmap_pages_range_noflush(addr, end, prot, pages, page_shift); +} + /** * vmap_pages_range - map pages to a kernel virtual address * @addr: start of the VM area to map From patchwork Fri Jul 1 14:22:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 313E9C43334 for ; Fri, 1 Jul 2022 14:24:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E4E46B0088; Fri, 1 Jul 2022 10:23:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66D4B6B0089; Fri, 1 Jul 2022 10:23:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 499146B008A; Fri, 1 Jul 2022 10:23:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 38C7D6B0088 for ; Fri, 1 Jul 2022 10:23:59 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0C88620DFE for ; Fri, 1 Jul 2022 14:23:59 +0000 (UTC) X-FDA: 79638750198.03.832E65F Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf19.hostedemail.com (Postfix) with ESMTP id A97E71A0039 for ; Fri, 1 Jul 2022 14:23:58 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id e20-20020a170906315400b007262bd0111eso841882eje.9 for ; Fri, 01 Jul 2022 07:23:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=yQ768gKjsg7o2qhom6fXRmA7hEqZpzUb/NLIF5UbCVk=; b=ilZQasgNcrZJU5SMLJZGlURLt0pZ82ORL0+V8GocPsRlZs6tRnEGLVOfls8cUZJtgR 0R9M5BhBsomSj6OV1JKUmUwDFagJq1aNYwCqXEMTv4xgB5j2Y8jx/7HCM8Zcnoy1skVZ dgZc5GlmMfXmNgGLxm8ewEotHX/gNzlolDo6mNcw2gS0YiH3R1PNCs+f2m/a2DIemvAz 0eSoyaqeLDV//5dVrVYvDUiFjY6VFUjBaQJQh/0o6dwhxb3psYS4+4I1UgkFHt0hyqY/ 9vnPZRdQC2MezrZyNlxfyeuVLepfrXiJmU5ZUHS2hiJDRKCT9K/pC9q4//yJD4fIKNY/ z5qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=yQ768gKjsg7o2qhom6fXRmA7hEqZpzUb/NLIF5UbCVk=; b=cP/6VR2o4OMMbAa0LKWUIJRwBmS4gw2KqeSbj2oxJaPhh1FcE6lWDfrJSnh684AyVB 5RnKZ60wpAFZNdf7Uh3rIatOlMm3Py52keEvGb0ygXa7JsiPyFbvco4bu+7i/WVdjQY0 2/9cpCdwbCjv7rmnRA34X0Z2OElQPh00DvW5Bme5A2LHscBJHsyoMVnn1+0hTbbVKsUU rHRJm2D05crX0E4slYeVx8NwC2up2Mbo4FDRSu0CSvIwm3VjbCi/tHsUaHcbGQQCG441 WxU7h7PDmWFux9XWNSYACv1OUVexbm1XKrf+3hfSVOAVBKtKCYWCm6n+a9ZOfnz61TT9 Wgmw== X-Gm-Message-State: AJIora80rDoc5il2eZGYNduRuwEYxkoSOZDYMMbrXtcGPWn9BQfCOsUE iqdyox0GMDNa/GrJujdJ7W8i2+N2kEE= X-Google-Smtp-Source: AGRyM1sm+6FJrp4kZ/TS4caUm7SvU0ZnpXFWFSa4DDFgO1nCu3BLqphLUK9DTM/B/KuGGGUeghZwZi10Euw= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:4410:b0:434:f35f:132e with SMTP id y16-20020a056402441000b00434f35f132emr19195845eda.215.1656685437446; Fri, 01 Jul 2022 07:23:57 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:40 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-16-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 15/45] mm: kmsan: call KMSAN hooks from SLUB code From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685438; a=rsa-sha256; cv=none; b=GxEgAcGAJTIJnS8LGT+rCKoi6DZdYskAbnSYfWrMH9hNCcsYopCcoFydpCwYrOuuITM0VP N+5GtXFskwzvikXtDsCTEIAuN+hWZzFgDPcqPGuBQwwAwW3gfm5DExyotKn2Bk9UqhfsZU 5eYIMn2tFVXP3QtX9vuwg9xP2eBTFQ4= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ilZQasgN; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf19.hostedemail.com: domain of 3fQO_YgYKCJwCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3fQO_YgYKCJwCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685438; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yQ768gKjsg7o2qhom6fXRmA7hEqZpzUb/NLIF5UbCVk=; b=SYvCoBf32phth8quMHBl33Ux/CTY5jhXFaQ1bxPWJQabhka7GtpIFoeLO9tnIt/yJQH6r4 YYfB/aB9okRUrCMBAECOS6kmK1N+qNsHROd4JYRekGN76SN2kPsBrtXPS8XOBXSDCAKJ/t qtbK1qP9IFjfqfIhX1FX+IptslfULtc= X-Stat-Signature: ti61x34u1gtfzpf3jzrbn45nxw59rgsg X-Rspamd-Queue-Id: A97E71A0039 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ilZQasgN; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf19.hostedemail.com: domain of 3fQO_YgYKCJwCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3fQO_YgYKCJwCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1656685438-127425 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In order to report uninitialized memory coming from heap allocations KMSAN has to poison them unless they're created with __GFP_ZERO. It's handy that we need KMSAN hooks in the places where init_on_alloc/init_on_free initialization is performed. In addition, we apply __no_kmsan_checks to get_freepointer_safe() to suppress reports when accessing freelist pointers that reside in freed objects. Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v2: -- move the implementation of SLUB hooks here v4: -- change sizeof(type) to sizeof(*ptr) -- swap mm: and kmsan: in the subject -- get rid of kmsan_init(), replace it with __no_kmsan_checks Link: https://linux-review.googlesource.com/id/I6954b386c5c5d7f99f48bb6cbcc74b75136ce86e --- include/linux/kmsan.h | 57 ++++++++++++++++++++++++++++++ mm/kmsan/hooks.c | 80 +++++++++++++++++++++++++++++++++++++++++++ mm/slab.h | 1 + mm/slub.c | 18 ++++++++++ 4 files changed, 156 insertions(+) diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index 699fe4f5b3bee..fd76cea338878 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -15,6 +15,7 @@ #include struct page; +struct kmem_cache; #ifdef CONFIG_KMSAN @@ -72,6 +73,44 @@ void kmsan_free_page(struct page *page, unsigned int order); */ void kmsan_copy_page_meta(struct page *dst, struct page *src); +/** + * kmsan_slab_alloc() - Notify KMSAN about a slab allocation. + * @s: slab cache the object belongs to. + * @object: object pointer. + * @flags: GFP flags passed to the allocator. + * + * Depending on cache flags and GFP flags, KMSAN sets up the metadata of the + * newly created object, marking it as initialized or uninitialized. + */ +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); + +/** + * kmsan_slab_free() - Notify KMSAN about a slab deallocation. + * @s: slab cache the object belongs to. + * @object: object pointer. + * + * KMSAN marks the freed object as uninitialized. + */ +void kmsan_slab_free(struct kmem_cache *s, void *object); + +/** + * kmsan_kmalloc_large() - Notify KMSAN about a large slab allocation. + * @ptr: object pointer. + * @size: object size. + * @flags: GFP flags passed to the allocator. + * + * Similar to kmsan_slab_alloc(), but for large allocations. + */ +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); + +/** + * kmsan_kfree_large() - Notify KMSAN about a large slab deallocation. + * @ptr: object pointer. + * + * Similar to kmsan_slab_free(), but for large allocations. + */ +void kmsan_kfree_large(const void *ptr); + /** * kmsan_map_kernel_range_noflush() - Notify KMSAN about a vmap. * @start: start of vmapped range. @@ -138,6 +177,24 @@ static inline void kmsan_copy_page_meta(struct page *dst, struct page *src) { } +static inline void kmsan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ +} + +static inline void kmsan_slab_free(struct kmem_cache *s, void *object) +{ +} + +static inline void kmsan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags) +{ +} + +static inline void kmsan_kfree_large(const void *ptr) +{ +} + static inline void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end, pgprot_t prot, diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 070756be70e3a..052e17b7a717d 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -26,6 +26,86 @@ * skipping effects of functions like memset() inside instrumented code. */ +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) +{ + if (unlikely(object == NULL)) + return; + if (!kmsan_enabled || kmsan_in_runtime()) + return; + /* + * There's a ctor or this is an RCU cache - do nothing. The memory + * status hasn't changed since last use. + */ + if (s->ctor || (s->flags & SLAB_TYPESAFE_BY_RCU)) + return; + + kmsan_enter_runtime(); + if (flags & __GFP_ZERO) + kmsan_internal_unpoison_memory(object, s->object_size, + KMSAN_POISON_CHECK); + else + kmsan_internal_poison_memory(object, s->object_size, flags, + KMSAN_POISON_CHECK); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_slab_alloc); + +void kmsan_slab_free(struct kmem_cache *s, void *object) +{ + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON))) + return; + /* + * If there's a constructor, freed memory must remain in the same state + * until the next allocation. We cannot save its state to detect + * use-after-free bugs, instead we just keep it unpoisoned. + */ + if (s->ctor) + return; + kmsan_enter_runtime(); + kmsan_internal_poison_memory(object, s->object_size, GFP_KERNEL, + KMSAN_POISON_CHECK | KMSAN_POISON_FREE); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_slab_free); + +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + if (unlikely(ptr == NULL)) + return; + if (!kmsan_enabled || kmsan_in_runtime()) + return; + kmsan_enter_runtime(); + if (flags & __GFP_ZERO) + kmsan_internal_unpoison_memory((void *)ptr, size, + /*checked*/ true); + else + kmsan_internal_poison_memory((void *)ptr, size, flags, + KMSAN_POISON_CHECK); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_kmalloc_large); + +void kmsan_kfree_large(const void *ptr) +{ + struct page *page; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + kmsan_enter_runtime(); + page = virt_to_head_page((void *)ptr); + KMSAN_WARN_ON(ptr != page_address(page)); + kmsan_internal_poison_memory((void *)ptr, + PAGE_SIZE << compound_order(page), + GFP_KERNEL, + KMSAN_POISON_CHECK | KMSAN_POISON_FREE); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_kfree_large); + static unsigned long vmalloc_shadow(unsigned long addr) { return (unsigned long)kmsan_get_metadata((void *)addr, diff --git a/mm/slab.h b/mm/slab.h index db9fb5c8dae73..d0de8195873d8 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -752,6 +752,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, memset(p[i], 0, s->object_size); kmemleak_alloc_recursive(p[i], s->object_size, 1, s->flags, flags); + kmsan_slab_alloc(s, p[i], flags); } memcg_slab_post_alloc_hook(s, objcg, flags, size, p); diff --git a/mm/slub.c b/mm/slub.c index b1281b8654bd3..b8b601f165087 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -359,6 +360,17 @@ static void prefetch_freepointer(const struct kmem_cache *s, void *object) prefetchw(object + s->offset); } +/* + * When running under KMSAN, get_freepointer_safe() may return an uninitialized + * pointer value in the case the current thread loses the race for the next + * memory chunk in the freelist. In that case this_cpu_cmpxchg_double() in + * slab_alloc_node() will fail, so the uninitialized value won't be used, but + * KMSAN will still check all arguments of cmpxchg because of imperfect + * handling of inline assembly. + * To work around this problem, we apply __no_kmsan_checks to ensure that + * get_freepointer_safe() returns initialized memory. + */ +__no_kmsan_checks static inline void *get_freepointer_safe(struct kmem_cache *s, void *object) { unsigned long freepointer_addr; @@ -1709,6 +1721,7 @@ static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) ptr = kasan_kmalloc_large(ptr, size, flags); /* As ptr might get tagged, call kmemleak hook after KASAN. */ kmemleak_alloc(ptr, size, 1, flags); + kmsan_kmalloc_large(ptr, size, flags); return ptr; } @@ -1716,12 +1729,14 @@ static __always_inline void kfree_hook(void *x) { kmemleak_free(x); kasan_kfree_large(x); + kmsan_kfree_large(x); } static __always_inline bool slab_free_hook(struct kmem_cache *s, void *x, bool init) { kmemleak_free_recursive(x, s->flags); + kmsan_slab_free(s, x); debug_check_no_locks_freed(x, s->object_size); @@ -3756,6 +3771,7 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size, */ slab_post_alloc_hook(s, objcg, flags, size, p, slab_want_init_on_alloc(flags, s)); + return i; error: slub_put_cpu_ptr(s->cpu_slab); @@ -5939,6 +5955,7 @@ static char *create_unique_id(struct kmem_cache *s) p += sprintf(p, "%07u", s->size); BUG_ON(p > name + ID_STR_LENGTH - 1); + kmsan_unpoison_memory(name, p - name); return name; } @@ -6040,6 +6057,7 @@ static int sysfs_slab_alias(struct kmem_cache *s, const char *name) al->name = name; al->next = alias_list; alias_list = al; + kmsan_unpoison_memory(al, sizeof(*al)); return 0; } From patchwork Fri Jul 1 14:22:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16236C43334 for ; Fri, 1 Jul 2022 14:24:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8D546B0089; Fri, 1 Jul 2022 10:24:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A3D676B008A; Fri, 1 Jul 2022 10:24:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DDD96B008C; Fri, 1 Jul 2022 10:24:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7B4536B0089 for ; Fri, 1 Jul 2022 10:24:02 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5725335AEE for ; Fri, 1 Jul 2022 14:24:02 +0000 (UTC) X-FDA: 79638750324.21.F5B07B5 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf13.hostedemail.com (Postfix) with ESMTP id ACDDA20050 for ; Fri, 1 Jul 2022 14:24:01 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id y18-20020a056402441200b0043564cdf765so1891212eda.11 for ; Fri, 01 Jul 2022 07:24:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MuviuQokEzhvfg2Z0+25mg/6MRU0laLu6ns1H0kafhg=; b=RS/M11ViQBiSN+sd81NPLXtCXBjB0FGsgDbLZu3p8F4GXPk0D+7t81g6tlYzjC+YwJ 2W2SuOsDI6B6k+EbyeqLWQ5i6XoSdBB/+r1g2aJs+YxxhMsErmeKZJZA7jqnG5QhyArd kCQL6luWnGxlNT+6WLZVxu+1AO53PlCajPc408xi4cgug6IuYEuGFoJhtwpZh92pjpFi LUqCR9BfPmYNdTfDQrXf+Z9/vBRTPln7cJAJKvCI/ZLobPZOQV/2MT0+aDpNak9Etgjo y4ufCn34FXG2xJ2ki47OVbFCxLHBCiQiG9yCQJMgCp4FSLz2CJ4PyO19DAZvw0qWc0JD PPNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MuviuQokEzhvfg2Z0+25mg/6MRU0laLu6ns1H0kafhg=; b=NpxVugu9GyQ7lxli0o45Y8c39kHBgSebmaVD7hgdU7KquSc1AvHy4KR7CsBdADs7/p XY6Yid5Lugs8I609pkSqlaUAyUp5DaFp3E+VwhCm78DVABmA72zHKogRZ2rBCOnXC+v3 viKX4lFpK4/JClGeOX/a0GJjjaBpimDLsGmOynY6dJvK2j8ILN3t/6ImkOivQdsj9mIR GQTB9jt/xUjgf2baj6OEEtUt42eKmUBUNTIzR1ewwJEGRkY9IFHJtWVrkJ9xNEECf+hI 4/Ln9zTz7/jytw0tq38R+mvt5UE35LpI2awUeThhBDNieQaCnI0dkjH8SRyYh1REqy3L 2oyw== X-Gm-Message-State: AJIora87PHf3OW5DcaJPXMOw2IMRxhNL+aknMYFf0chi5xJXhHr7oxQf upO9nxc95ddigNStvzRlEla2+teGVQ4= X-Google-Smtp-Source: AGRyM1svvXk0uhgtL2lyHdhsMHh+dNlSgp0MiLj8ryXpF1AJlade1jJus8xhDzqNuinjfAclnUPFVASuizY= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:430a:b0:435:8ec9:31ec with SMTP id m10-20020a056402430a00b004358ec931ecmr19482900edc.248.1656685440344; Fri, 01 Jul 2022 07:24:00 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:41 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-17-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 16/45] kmsan: handle task creation and exiting From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685441; a=rsa-sha256; cv=none; b=R4vqrBdXgwISQbJ35nEJfpGxKgFQFXKgUEkKETn6coThDnYO3nqj5i6vsvA64xsT9LqGAa 8K2a/qriTx3VMyB5hqp5UBgQCOzlbinFcChBAqz+VYV6OCGkRleSp/xsiMig5XNP/X7gaw tktkWCmAeGcnw5QDzrabnwxSVZxUrwE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="RS/M11Vi"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3gAO_YgYKCJ8FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3gAO_YgYKCJ8FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MuviuQokEzhvfg2Z0+25mg/6MRU0laLu6ns1H0kafhg=; b=I8ilOvnxqc0bk1UCwpPH5FhFth5zV5seQFmlDbNPce0RIAju09HO+rE3gSTIT9wGUJfI0y CcmuXnhILwwJNKMxBpfudA0vzwQEjsEHOWoM5213fJ4vqkAffqwR7w2wUKZgb/cHAOWe+9 xNvuJP4JrKIAp4pupO2c3X2BNqpmNGc= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: ACDDA20050 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="RS/M11Vi"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3gAO_YgYKCJ8FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3gAO_YgYKCJ8FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: zeqm87iyjah77th3eqe577ipxhsizmun X-HE-Tag: 1656685441-172866 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Tell KMSAN that a new task is created, so the tool creates a backing metadata structure for that task. Signed-off-by: Alexander Potapenko --- v2: -- move implementation of kmsan_task_create() and kmsan_task_exit() here v4: -- change sizeof(type) to sizeof(*ptr) Link: https://linux-review.googlesource.com/id/I0f41c3a1c7d66f7e14aabcfdfc7c69addb945805 --- include/linux/kmsan.h | 17 +++++++++++++++++ kernel/exit.c | 2 ++ kernel/fork.c | 2 ++ mm/kmsan/core.c | 10 ++++++++++ mm/kmsan/hooks.c | 19 +++++++++++++++++++ mm/kmsan/kmsan.h | 2 ++ 6 files changed, 52 insertions(+) diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index fd76cea338878..b71e2032222e9 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -16,6 +16,7 @@ struct page; struct kmem_cache; +struct task_struct; #ifdef CONFIG_KMSAN @@ -42,6 +43,14 @@ struct kmsan_ctx { bool allow_reporting; }; +void kmsan_task_create(struct task_struct *task); + +/** + * kmsan_task_exit() - Notify KMSAN that a task has exited. + * @task: task about to finish. + */ +void kmsan_task_exit(struct task_struct *task); + /** * kmsan_alloc_page() - Notify KMSAN about an alloc_pages() call. * @page: struct page pointer returned by alloc_pages(). @@ -163,6 +172,14 @@ void kmsan_iounmap_page_range(unsigned long start, unsigned long end); #else +static inline void kmsan_task_create(struct task_struct *task) +{ +} + +static inline void kmsan_task_exit(struct task_struct *task) +{ +} + static inline int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags) { diff --git a/kernel/exit.c b/kernel/exit.c index f072959fcab7f..1784b7a741ddd 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -60,6 +60,7 @@ #include #include #include +#include #include #include #include @@ -741,6 +742,7 @@ void __noreturn do_exit(long code) WARN_ON(tsk->plug); kcov_task_exit(tsk); + kmsan_task_exit(tsk); coredump_task_exit(tsk); ptrace_event(PTRACE_EVENT_EXIT, code); diff --git a/kernel/fork.c b/kernel/fork.c index 9d44f2d46c696..6dfca6f00ec82 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -1026,6 +1027,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) tsk->worker_private = NULL; kcov_task_init(tsk); + kmsan_task_create(tsk); kmap_local_fork(tsk); #ifdef CONFIG_FAULT_INJECTION diff --git a/mm/kmsan/core.c b/mm/kmsan/core.c index 16fb8880a9c6d..7eabed03ed10b 100644 --- a/mm/kmsan/core.c +++ b/mm/kmsan/core.c @@ -44,6 +44,16 @@ bool kmsan_enabled __read_mostly; */ DEFINE_PER_CPU(struct kmsan_ctx, kmsan_percpu_ctx); +void kmsan_internal_task_create(struct task_struct *task) +{ + struct kmsan_ctx *ctx = &task->kmsan_ctx; + struct thread_info *info = current_thread_info(); + + __memset(ctx, 0, sizeof(*ctx)); + ctx->allow_reporting = true; + kmsan_internal_unpoison_memory(info, sizeof(*info), false); +} + void kmsan_internal_poison_memory(void *address, size_t size, gfp_t flags, unsigned int poison_flags) { diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 052e17b7a717d..43a529569053d 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -26,6 +26,25 @@ * skipping effects of functions like memset() inside instrumented code. */ +void kmsan_task_create(struct task_struct *task) +{ + kmsan_enter_runtime(); + kmsan_internal_task_create(task); + kmsan_leave_runtime(); +} +EXPORT_SYMBOL(kmsan_task_create); + +void kmsan_task_exit(struct task_struct *task) +{ + struct kmsan_ctx *ctx = &task->kmsan_ctx; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + + ctx->allow_reporting = false; +} +EXPORT_SYMBOL(kmsan_task_exit); + void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) { if (unlikely(object == NULL)) diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h index d3c400ca097ba..c7fb8666607e2 100644 --- a/mm/kmsan/kmsan.h +++ b/mm/kmsan/kmsan.h @@ -179,6 +179,8 @@ void kmsan_internal_set_shadow_origin(void *address, size_t size, int b, u32 origin, bool checked); depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id); +void kmsan_internal_task_create(struct task_struct *task); + bool kmsan_metadata_is_contiguous(void *addr, size_t size); void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr, int reason); From patchwork Fri Jul 1 14:22:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AD23C43334 for ; Fri, 1 Jul 2022 14:24:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 361786B008A; Fri, 1 Jul 2022 10:24:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 311D96B008C; Fri, 1 Jul 2022 10:24:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13D596B0092; Fri, 1 Jul 2022 10:24:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 035426B008A for ; Fri, 1 Jul 2022 10:24:05 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D4694358FC for ; Fri, 1 Jul 2022 14:24:04 +0000 (UTC) X-FDA: 79638750408.01.4F2CD5C Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf22.hostedemail.com (Postfix) with ESMTP id 7129DC002E for ; Fri, 1 Jul 2022 14:24:04 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id i9-20020a05640242c900b004373cd1c4d5so1893591edc.2 for ; Fri, 01 Jul 2022 07:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=e6DErphzLhmkwnld+r1h2oFxsMsjnlMc9+N+LMIifaY=; b=Y2pA3EYx4J6UDiI5Jf+Zn8An9KgBKw8qiHP8bdf7BjhtRPQ9n8x3gUZNgu0i4g1Dar SN9kaTFADAxQG2VIloKtReLucjOsfQ+Rl1YC/qfwChAYrP+M6AHtvs8mAV5uRBz4yp+a wQi6lSZ8Ddg0vxEBT6eIxCIJyCby5lDeK9P5fOLDf6AXx45a8Rdnz45/fIveySQy1/ro ghF5ItXe0bGqapgjkhDJOBvaBzwdfHZGYYFU9rqtiV0GOndZzV0qBvt9z+TneSdWyDZj 3HC0Q3vPjDMzpKwLigp8xH1qqlZyxSoALNDKkvJPy9j9i33R4BbEencDJZA7rio7sRid eHJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=e6DErphzLhmkwnld+r1h2oFxsMsjnlMc9+N+LMIifaY=; b=Ck2CYG0jR5WZMxqZaSOg4eqmbhWcogmClDVd5I5dpCnfvTIoJ5h+mVD+NXqFnL5C2g G/3Ol8ZFgX7kmj69ddmSwdKq1eaoYSVvzWXmrBUjbKf25sPXOY0VsUEuUTpSm7afUfDD dXiA2rc6umSPMNAIz+GUQOR2JwnWZ1wPU/WA5SyTyLoO8jvxPy21PO16573+FaLbcK0t RmIwGMxcLeoQvrNMh7RUrST7sfeeivvjoRvsgbx3rLdTayv+fBNUdxpGGvR/2RbXS6gV YtkL3Hr4e/VXA8fjjtUXgL+oro7CRW0T+suW/cSuUs8o49zQlDDG4wTKTsQKh/5RYLbG SvVA== X-Gm-Message-State: AJIora9ApBCoLGJF8vCMqH8yeRRl8hUtkbV1jHh/QiHtaQBSXIdn+u0B f0IoApkZKNtG1aCSQdfkeJhJOF39PQ0= X-Google-Smtp-Source: AGRyM1uKLwv+5FEkA6ahqf1TgDGKMvu1umRTrYztncLlzRJbf0vZxVoRYVguM7zaz+DxU6CpnBXbPnKmeho= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:7207:b0:726:cc89:40ca with SMTP id dr7-20020a170907720700b00726cc8940camr14991650ejc.141.1656685443124; Fri, 01 Jul 2022 07:24:03 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:42 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-18-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 17/45] init: kmsan: call KMSAN initialization routines From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685444; a=rsa-sha256; cv=none; b=hHEYQmaEyeQZWGHIaVbyhamSTmYSCBMoCHTLgkHe1mIMQ5djESGGQHP6QDwyRPv2IoJ1oB 0Gk+hJlyBjB+08HSw6FCKC3s+9FayPEWIbh4jHKR6Qv5U94SXSNCLLc13kZzBlN61k9zWK a8xqOcBBqde5rpZGgDtph1DdqVNVWPY= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Y2pA3EYx; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3gwO_YgYKCKIINKFGTIQQING.EQONKPWZ-OOMXCEM.QTI@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3gwO_YgYKCKIINKFGTIQQING.EQONKPWZ-OOMXCEM.QTI@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685444; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e6DErphzLhmkwnld+r1h2oFxsMsjnlMc9+N+LMIifaY=; b=g70Om4bXahyEE1ERdTdxld2/IAir3AHmnCMsr+kGndDGB3atTGSwBjM6/jZnhN8VJnXOS0 +GAtbKGx4RiuCQwIpN6wd8mhhOYFXghQ5J3fjTZtBpUyYfUD9jJc6w0q1jYTclCYU9yTeh Ez5KtkgL35CdMp0+V8q5WueoJbxuEks= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 7129DC002E Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Y2pA3EYx; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3gwO_YgYKCKIINKFGTIQQING.EQONKPWZ-OOMXCEM.QTI@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3gwO_YgYKCKIINKFGTIQQING.EQONKPWZ-OOMXCEM.QTI@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: xnkgi7soj3zra8gkd5spe13ixx3wtsqx X-HE-Tag: 1656685444-3215 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: kmsan_init_shadow() scans the mappings created at boot time and creates metadata pages for those mappings. When the memblock allocator returns pages to pagealloc, we reserve 2/3 of those pages and use them as metadata for the remaining 1/3. Once KMSAN starts, every page allocated by pagealloc has its associated shadow and origin pages. kmsan_initialize() initializes the bookkeeping for init_task and enables KMSAN. Signed-off-by: Alexander Potapenko --- v2: -- move mm/kmsan/init.c and kmsan_memblock_free_pages() to this patch -- print a warning that KMSAN is a debugging tool (per Greg K-H's request) v4: -- change sizeof(type) to sizeof(*ptr) -- replace occurrences of |var| with @var -- swap init: and kmsan: in the subject -- do not export __init functions Link: https://linux-review.googlesource.com/id/I7bc53706141275914326df2345881ffe0cdd16bd --- include/linux/kmsan.h | 48 +++++++++ init/main.c | 3 + mm/kmsan/Makefile | 3 +- mm/kmsan/init.c | 238 ++++++++++++++++++++++++++++++++++++++++++ mm/kmsan/kmsan.h | 3 + mm/kmsan/shadow.c | 36 +++++++ mm/page_alloc.c | 3 + 7 files changed, 333 insertions(+), 1 deletion(-) create mode 100644 mm/kmsan/init.c diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index b71e2032222e9..82fd564cc72e7 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -51,6 +51,40 @@ void kmsan_task_create(struct task_struct *task); */ void kmsan_task_exit(struct task_struct *task); +/** + * kmsan_init_shadow() - Initialize KMSAN shadow at boot time. + * + * Allocate and initialize KMSAN metadata for early allocations. + */ +void __init kmsan_init_shadow(void); + +/** + * kmsan_init_runtime() - Initialize KMSAN state and enable KMSAN. + */ +void __init kmsan_init_runtime(void); + +/** + * kmsan_memblock_free_pages() - handle freeing of memblock pages. + * @page: struct page to free. + * @order: order of @page. + * + * Freed pages are either returned to buddy allocator or held back to be used + * as metadata pages. + */ +bool __init kmsan_memblock_free_pages(struct page *page, unsigned int order); + +/** + * kmsan_task_create() - Initialize KMSAN state for the task. + * @task: task to initialize. + */ +void kmsan_task_create(struct task_struct *task); + +/** + * kmsan_task_exit() - Notify KMSAN that a task has exited. + * @task: task about to finish. + */ +void kmsan_task_exit(struct task_struct *task); + /** * kmsan_alloc_page() - Notify KMSAN about an alloc_pages() call. * @page: struct page pointer returned by alloc_pages(). @@ -172,6 +206,20 @@ void kmsan_iounmap_page_range(unsigned long start, unsigned long end); #else +static inline void kmsan_init_shadow(void) +{ +} + +static inline void kmsan_init_runtime(void) +{ +} + +static inline bool kmsan_memblock_free_pages(struct page *page, + unsigned int order) +{ + return true; +} + static inline void kmsan_task_create(struct task_struct *task) { } diff --git a/init/main.c b/init/main.c index 0ee39cdcfcac9..7ba48a9ff1d53 100644 --- a/init/main.c +++ b/init/main.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -835,6 +836,7 @@ static void __init mm_init(void) init_mem_debugging_and_hardening(); kfence_alloc_pool(); report_meminit(); + kmsan_init_shadow(); stack_depot_early_init(); mem_init(); mem_init_print_info(); @@ -852,6 +854,7 @@ static void __init mm_init(void) init_espfix_bsp(); /* Should be run after espfix64 is set up. */ pti_init(); + kmsan_init_runtime(); } #ifdef CONFIG_RANDOMIZE_KSTACK_OFFSET diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile index 550ad8625e4f9..401acb1a491ce 100644 --- a/mm/kmsan/Makefile +++ b/mm/kmsan/Makefile @@ -3,7 +3,7 @@ # Makefile for KernelMemorySanitizer (KMSAN). # # -obj-y := core.o instrumentation.o hooks.o report.o shadow.o +obj-y := core.o instrumentation.o init.o hooks.o report.o shadow.o KMSAN_SANITIZE := n KCOV_INSTRUMENT := n @@ -18,6 +18,7 @@ CFLAGS_REMOVE.o = $(CC_FLAGS_FTRACE) CFLAGS_core.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_hooks.o := $(CC_FLAGS_KMSAN_RUNTIME) +CFLAGS_init.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_instrumentation.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_report.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_shadow.o := $(CC_FLAGS_KMSAN_RUNTIME) diff --git a/mm/kmsan/init.c b/mm/kmsan/init.c new file mode 100644 index 0000000000000..abbf595a1e359 --- /dev/null +++ b/mm/kmsan/init.c @@ -0,0 +1,238 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN initialization routines. + * + * Copyright (C) 2017-2021 Google LLC + * Author: Alexander Potapenko + * + */ + +#include "kmsan.h" + +#include +#include +#include + +#include "../internal.h" + +#define NUM_FUTURE_RANGES 128 +struct start_end_pair { + u64 start, end; +}; + +static struct start_end_pair start_end_pairs[NUM_FUTURE_RANGES] __initdata; +static int future_index __initdata; + +/* + * Record a range of memory for which the metadata pages will be created once + * the page allocator becomes available. + */ +static void __init kmsan_record_future_shadow_range(void *start, void *end) +{ + u64 nstart = (u64)start, nend = (u64)end, cstart, cend; + bool merged = false; + int i; + + KMSAN_WARN_ON(future_index == NUM_FUTURE_RANGES); + KMSAN_WARN_ON((nstart >= nend) || !nstart || !nend); + nstart = ALIGN_DOWN(nstart, PAGE_SIZE); + nend = ALIGN(nend, PAGE_SIZE); + + /* + * Scan the existing ranges to see if any of them overlaps with + * [start, end). In that case, merge the two ranges instead of + * creating a new one. + * The number of ranges is less than 20, so there is no need to organize + * them into a more intelligent data structure. + */ + for (i = 0; i < future_index; i++) { + cstart = start_end_pairs[i].start; + cend = start_end_pairs[i].end; + if ((cstart < nstart && cend < nstart) || + (cstart > nend && cend > nend)) + /* ranges are disjoint - do not merge */ + continue; + start_end_pairs[i].start = min(nstart, cstart); + start_end_pairs[i].end = max(nend, cend); + merged = true; + break; + } + if (merged) + return; + start_end_pairs[future_index].start = nstart; + start_end_pairs[future_index].end = nend; + future_index++; +} + +/* + * Initialize the shadow for existing mappings during kernel initialization. + * These include kernel text/data sections, NODE_DATA and future ranges + * registered while creating other data (e.g. percpu). + * + * Allocations via memblock can be only done before slab is initialized. + */ +void __init kmsan_init_shadow(void) +{ + const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE); + phys_addr_t p_start, p_end; + int nid; + u64 i; + + for_each_reserved_mem_range(i, &p_start, &p_end) + kmsan_record_future_shadow_range(phys_to_virt(p_start), + phys_to_virt(p_end)); + /* Allocate shadow for .data */ + kmsan_record_future_shadow_range(_sdata, _edata); + + for_each_online_node(nid) + kmsan_record_future_shadow_range( + NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size); + + for (i = 0; i < future_index; i++) + kmsan_init_alloc_meta_for_range( + (void *)start_end_pairs[i].start, + (void *)start_end_pairs[i].end); +} + +struct page_pair { + struct page *shadow, *origin; +}; +static struct page_pair held_back[MAX_ORDER] __initdata; + +/* + * Eager metadata allocation. When the memblock allocator is freeing pages to + * pagealloc, we use 2/3 of them as metadata for the remaining 1/3. + * We store the pointers to the returned blocks of pages in held_back[] grouped + * by their order: when kmsan_memblock_free_pages() is called for the first + * time with a certain order, it is reserved as a shadow block, for the second + * time - as an origin block. On the third time the incoming block receives its + * shadow and origin ranges from the previously saved shadow and origin blocks, + * after which held_back[order] can be used again. + * + * At the very end there may be leftover blocks in held_back[]. They are + * collected later by kmsan_memblock_discard(). + */ +bool kmsan_memblock_free_pages(struct page *page, unsigned int order) +{ + struct page *shadow, *origin; + + if (!held_back[order].shadow) { + held_back[order].shadow = page; + return false; + } + if (!held_back[order].origin) { + held_back[order].origin = page; + return false; + } + shadow = held_back[order].shadow; + origin = held_back[order].origin; + kmsan_setup_meta(page, shadow, origin, order); + + held_back[order].shadow = NULL; + held_back[order].origin = NULL; + return true; +} + +#define MAX_BLOCKS 8 +struct smallstack { + struct page *items[MAX_BLOCKS]; + int index; + int order; +}; + +static struct smallstack collect = { + .index = 0, + .order = MAX_ORDER, +}; + +static void smallstack_push(struct smallstack *stack, struct page *pages) +{ + KMSAN_WARN_ON(stack->index == MAX_BLOCKS); + stack->items[stack->index] = pages; + stack->index++; +} +#undef MAX_BLOCKS + +static struct page *smallstack_pop(struct smallstack *stack) +{ + struct page *ret; + + KMSAN_WARN_ON(stack->index == 0); + stack->index--; + ret = stack->items[stack->index]; + stack->items[stack->index] = NULL; + return ret; +} + +static void do_collection(void) +{ + struct page *page, *shadow, *origin; + + while (collect.index >= 3) { + page = smallstack_pop(&collect); + shadow = smallstack_pop(&collect); + origin = smallstack_pop(&collect); + kmsan_setup_meta(page, shadow, origin, collect.order); + __free_pages_core(page, collect.order); + } +} + +static void collect_split(void) +{ + struct smallstack tmp = { + .order = collect.order - 1, + .index = 0, + }; + struct page *page; + + if (!collect.order) + return; + while (collect.index) { + page = smallstack_pop(&collect); + smallstack_push(&tmp, &page[0]); + smallstack_push(&tmp, &page[1 << tmp.order]); + } + __memcpy(&collect, &tmp, sizeof(tmp)); +} + +/* + * Memblock is about to go away. Split the page blocks left over in held_back[] + * and return 1/3 of that memory to the system. + */ +static void kmsan_memblock_discard(void) +{ + int i; + + /* + * For each order=N: + * - push held_back[N].shadow and .origin to @collect; + * - while there are >= 3 elements in @collect, do garbage collection: + * - pop 3 ranges from @collect; + * - use two of them as shadow and origin for the third one; + * - repeat; + * - split each remaining element from @collect into 2 ranges of + * order=N-1, + * - repeat. + */ + collect.order = MAX_ORDER - 1; + for (i = MAX_ORDER - 1; i >= 0; i--) { + if (held_back[i].shadow) + smallstack_push(&collect, held_back[i].shadow); + if (held_back[i].origin) + smallstack_push(&collect, held_back[i].origin); + held_back[i].shadow = NULL; + held_back[i].origin = NULL; + do_collection(); + collect_split(); + } +} + +void __init kmsan_init_runtime(void) +{ + /* Assuming current is init_task */ + kmsan_internal_task_create(current); + kmsan_memblock_discard(); + pr_info("Starting KernelMemorySanitizer\n"); + pr_info("ATTENTION: KMSAN is a debugging tool! Do not use it on production machines!\n"); + kmsan_enabled = true; +} diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h index c7fb8666607e2..2f17912ef863f 100644 --- a/mm/kmsan/kmsan.h +++ b/mm/kmsan/kmsan.h @@ -66,6 +66,7 @@ struct shadow_origin_ptr { struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size, bool store); void *kmsan_get_metadata(void *addr, bool is_origin); +void __init kmsan_init_alloc_meta_for_range(void *start, void *end); enum kmsan_bug_reason { REASON_ANY, @@ -188,5 +189,7 @@ bool kmsan_internal_is_module_addr(void *vaddr); bool kmsan_internal_is_vmalloc_addr(void *addr); struct page *kmsan_vmalloc_to_page_or_null(void *vaddr); +void kmsan_setup_meta(struct page *page, struct page *shadow, + struct page *origin, int order); #endif /* __MM_KMSAN_KMSAN_H */ diff --git a/mm/kmsan/shadow.c b/mm/kmsan/shadow.c index 416cb85487a1a..7b254c30d42cc 100644 --- a/mm/kmsan/shadow.c +++ b/mm/kmsan/shadow.c @@ -259,3 +259,39 @@ void kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end, kfree(s_pages); kfree(o_pages); } + +/* Allocate metadata for pages allocated at boot time. */ +void __init kmsan_init_alloc_meta_for_range(void *start, void *end) +{ + struct page *shadow_p, *origin_p; + void *shadow, *origin; + struct page *page; + u64 addr, size; + + start = (void *)ALIGN_DOWN((u64)start, PAGE_SIZE); + size = ALIGN((u64)end - (u64)start, PAGE_SIZE); + shadow = memblock_alloc(size, PAGE_SIZE); + origin = memblock_alloc(size, PAGE_SIZE); + for (addr = 0; addr < size; addr += PAGE_SIZE) { + page = virt_to_page_or_null((char *)start + addr); + shadow_p = virt_to_page_or_null((char *)shadow + addr); + set_no_shadow_origin_page(shadow_p); + shadow_page_for(page) = shadow_p; + origin_p = virt_to_page_or_null((char *)origin + addr); + set_no_shadow_origin_page(origin_p); + origin_page_for(page) = origin_p; + } +} + +void kmsan_setup_meta(struct page *page, struct page *shadow, + struct page *origin, int order) +{ + int i; + + for (i = 0; i < (1 << order); i++) { + set_no_shadow_origin_page(&shadow[i]); + set_no_shadow_origin_page(&origin[i]); + shadow_page_for(&page[i]) = &shadow[i]; + origin_page_for(&page[i]) = &origin[i]; + } +} diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 785459251145e..e8d5a0b2a3264 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1731,6 +1731,9 @@ void __init memblock_free_pages(struct page *page, unsigned long pfn, { if (early_page_uninitialised(pfn)) return; + if (!kmsan_memblock_free_pages(page, order)) + /* KMSAN will take care of these pages. */ + return; __free_pages_core(page, order); } From patchwork Fri Jul 1 14:22:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD839C43334 for ; Fri, 1 Jul 2022 14:24:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66F176B0074; Fri, 1 Jul 2022 10:24:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61EA16B0075; Fri, 1 Jul 2022 10:24:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 499616B007E; Fri, 1 Jul 2022 10:24:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3AF8F6B0074 for ; Fri, 1 Jul 2022 10:24:08 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 051B421668 for ; Fri, 1 Jul 2022 14:24:07 +0000 (UTC) X-FDA: 79638750576.03.70909DD Received: from mail-lf1-f74.google.com (mail-lf1-f74.google.com [209.85.167.74]) by imf11.hostedemail.com (Postfix) with ESMTP id 8946C40054 for ; Fri, 1 Jul 2022 14:24:07 +0000 (UTC) Received: by mail-lf1-f74.google.com with SMTP id o7-20020a056512230700b004810a865709so1183339lfu.3 for ; Fri, 01 Jul 2022 07:24:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=qRpA/KHLRYdo76YWwE6d04+ye7UN8yXoHeTH9XEwQK0=; b=nuIy5sPY97WE4Na2+tDp/B2mAWuutSYXusRYqrf6YxSijHhgLbmnsmS0o2fiYh5/Do /09lw8m4SVNKOYmoEK6HMxPLTu1E3kw4LhBcRBfvwY3nS2wLt+OCVSphGAatbhRfsti+ VVp0F1sWJPyWXiGjZXUPSCQZOElXfKweDeG5Yu1Njxbddsc0VwS3a8L8cZHVuO5Klbj8 qdZN0ySrNX0MJDuYE65IztkpOKOSP3D6akR5YHdge9pHQKisJPIuL47h4k0Q33mfj/9L XeJvoIaLRnLrA4bnmskTdmIuMKwd86AQ7L805VR2oh/iRtOJ1JrzzU9ncKoZ9gIHDb8s YLCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=qRpA/KHLRYdo76YWwE6d04+ye7UN8yXoHeTH9XEwQK0=; b=XyL/rQHQKkMmN3v9FhSCNbTAnKggL+vLkXVnNXPNZo0G6BjBwND94CHcrX6RMTyKUh wFmF7beUDrTKvsRIYXUo56t3yvr6F+SjWZe5HDbQh1TePBSNEdWxb0TM+XG+Qgq7eBYV LLPYm8dQ0tPSRfiNdnkjFvo16zWCHGFqirQMObN6AdtJG5sQw6DDPNg+wDwceva3iRKh r42OE8o9bDTxkMqq/otIoU+oahfHGaBAw8fhrYf9Bu4PmgPCG0qFpv6S4fcD2PjhVjpx tDJX1w1FhMLWbIU3GVjFmXtJPf56l8IUjIXKlNzWa82twpaD4S453g5hsHUbts6sApsO ChLw== X-Gm-Message-State: AJIora+lw/5t/UtYKUReUWEHH/bCAEfxnz7aWx3D5tfHGY0DANFa4R+3 dFyic/oJhH7Kj8+/ybXe4i1tpZixfTU= X-Google-Smtp-Source: AGRyM1sty4IKmaztZhJQX9qrKqisLJMfHz0k4hy5dUHdfFpoWKsT1NmFCA9JTdaWQUOqJX7WaPNwfk8uyOs= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6512:b9b:b0:47f:6aae:ecc5 with SMTP id b27-20020a0565120b9b00b0047f6aaeecc5mr9169900lfv.412.1656685445833; Fri, 01 Jul 2022 07:24:05 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:43 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-19-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 18/45] instrumented.h: add KMSAN support From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685447; a=rsa-sha256; cv=none; b=el7PZntIq6dQ3B3/88sCn2EmHWGjf6ZEwmnWj95VoRh2fIG8mtUdMnm6JriB8Ep/VM+EDu naGMJo8OUH4bSKQL01dur5O4wF7qQRwOjF5N/RPZPAExUiKwJjwBkth1smKOwXsVii37/T dkQSXNQW6eP1nLvqro5INSrRTJHi1x0= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=nuIy5sPY; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 3hQO_YgYKCKQKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com designates 209.85.167.74 as permitted sender) smtp.mailfrom=3hQO_YgYKCKQKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685447; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qRpA/KHLRYdo76YWwE6d04+ye7UN8yXoHeTH9XEwQK0=; b=6hJLMwWSdZZg5n6Da0B1gDMEqOvkBdKaLopDVXZwIB7GWTtVUyJNk7Yu0frPIDYvC14Ml8 JJzl8A0Gc0eGQkTf7TMrF2i8PsTlON8oHhD81BSLIznta1WvUH3pWXgQU/r3niRNKhTbFS cNqGYv7DqeuMqYQxs4w7T8k/yKiiGTU= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8946C40054 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=nuIy5sPY; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 3hQO_YgYKCKQKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com designates 209.85.167.74 as permitted sender) smtp.mailfrom=3hQO_YgYKCKQKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: j1rkbhan5ny6xqdnzydo3bcrrsmhxbrg X-HE-Tag: 1656685447-198581 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To avoid false positives, KMSAN needs to unpoison the data copied from the userspace. To detect infoleaks - check the memory buffer passed to copy_to_user(). Signed-off-by: Alexander Potapenko Reviewed-by: Marco Elver --- v2: -- move implementation of kmsan_copy_to_user() here Link: https://linux-review.googlesource.com/id/I43e93b9c02709e6be8d222342f1b044ac8bdbaaf --- include/linux/instrumented.h | 5 ++++- include/linux/kmsan-checks.h | 19 ++++++++++++++++++ mm/kmsan/hooks.c | 38 ++++++++++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+), 1 deletion(-) diff --git a/include/linux/instrumented.h b/include/linux/instrumented.h index ee8f7d17d34f5..c73c1b19e9227 100644 --- a/include/linux/instrumented.h +++ b/include/linux/instrumented.h @@ -2,7 +2,7 @@ /* * This header provides generic wrappers for memory access instrumentation that - * the compiler cannot emit for: KASAN, KCSAN. + * the compiler cannot emit for: KASAN, KCSAN, KMSAN. */ #ifndef _LINUX_INSTRUMENTED_H #define _LINUX_INSTRUMENTED_H @@ -10,6 +10,7 @@ #include #include #include +#include #include /** @@ -117,6 +118,7 @@ instrument_copy_to_user(void __user *to, const void *from, unsigned long n) { kasan_check_read(from, n); kcsan_check_read(from, n); + kmsan_copy_to_user(to, from, n, 0); } /** @@ -151,6 +153,7 @@ static __always_inline void instrument_copy_from_user_after(const void *to, const void __user *from, unsigned long n, unsigned long left) { + kmsan_unpoison_memory(to, n - left); } #endif /* _LINUX_INSTRUMENTED_H */ diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h index a6522a0c28df9..c4cae333deec5 100644 --- a/include/linux/kmsan-checks.h +++ b/include/linux/kmsan-checks.h @@ -46,6 +46,21 @@ void kmsan_unpoison_memory(const void *address, size_t size); */ void kmsan_check_memory(const void *address, size_t size); +/** + * kmsan_copy_to_user() - Notify KMSAN about a data transfer to userspace. + * @to: destination address in the userspace. + * @from: source address in the kernel. + * @to_copy: number of bytes to copy. + * @left: number of bytes not copied. + * + * If this is a real userspace data transfer, KMSAN checks the bytes that were + * actually copied to ensure there was no information leak. If @to belongs to + * the kernel space (which is possible for compat syscalls), KMSAN just copies + * the metadata. + */ +void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, + size_t left); + #else static inline void kmsan_poison_memory(const void *address, size_t size, @@ -58,6 +73,10 @@ static inline void kmsan_unpoison_memory(const void *address, size_t size) static inline void kmsan_check_memory(const void *address, size_t size) { } +static inline void kmsan_copy_to_user(void __user *to, const void *from, + size_t to_copy, size_t left) +{ +} #endif diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 43a529569053d..1cdb4420977f1 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -212,6 +212,44 @@ void kmsan_iounmap_page_range(unsigned long start, unsigned long end) } EXPORT_SYMBOL(kmsan_iounmap_page_range); +void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, + size_t left) +{ + unsigned long ua_flags; + + if (!kmsan_enabled || kmsan_in_runtime()) + return; + /* + * At this point we've copied the memory already. It's hard to check it + * before copying, as the size of actually copied buffer is unknown. + */ + + /* copy_to_user() may copy zero bytes. No need to check. */ + if (!to_copy) + return; + /* Or maybe copy_to_user() failed to copy anything. */ + if (to_copy <= left) + return; + + ua_flags = user_access_save(); + if ((u64)to < TASK_SIZE) { + /* This is a user memory access, check it. */ + kmsan_internal_check_memory((void *)from, to_copy - left, to, + REASON_COPY_TO_USER); + user_access_restore(ua_flags); + return; + } + /* Otherwise this is a kernel memory access. This happens when a compat + * syscall passes an argument allocated on the kernel stack to a real + * syscall. + * Don't check anything, just copy the shadow of the copied bytes. + */ + kmsan_internal_memmove_metadata((void *)to, (void *)from, + to_copy - left); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(kmsan_copy_to_user); + /* Functions from kmsan-checks.h follow. */ void kmsan_poison_memory(const void *address, size_t size, gfp_t flags) { From patchwork Fri Jul 1 14:22:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4384DC433EF for ; Fri, 1 Jul 2022 14:24:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D25F86B0073; Fri, 1 Jul 2022 10:24:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CBA4D6B0075; Fri, 1 Jul 2022 10:24:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADA396B0078; Fri, 1 Jul 2022 10:24:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9D14F6B0073 for ; Fri, 1 Jul 2022 10:24:10 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 6A64860B9B for ; Fri, 1 Jul 2022 14:24:10 +0000 (UTC) X-FDA: 79638750660.29.37F633E Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf16.hostedemail.com (Postfix) with ESMTP id 061EF18006A for ; Fri, 1 Jul 2022 14:24:09 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id t14-20020a056402524e00b0043595a18b91so1870160edd.13 for ; Fri, 01 Jul 2022 07:24:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GNX0et0VeFDbRNtievhe/rlGdldiDAHSzzvJvy6PdeY=; b=sYf9Flp9INntHEwHp1o9tRTz2HbnWSqOM0+13jbX4U/+5ngXGJ1iU0hkzEzI7Yl6Jk jlH0nE7LDojywEMxRZBi0U0XGDKUUrXHG6C4Qt21C0ilgW5fLbkrrpGtDWltdc6Tw5Ya YHQJqpyVfLRTzczxIItYEgnay8aizN5mERMhqohjf+v62gF9XLNvQGeEcDq68s6syawh PIsw0Dg3A/+St8+47YO/SOm1rxeGm8eQwJ6SDQfP6Egws2HnP2QcRUDhdRO6+G85DaEs 34UNhJTVc5cvYOHCVw3XrE8e3CVUh9QKirtDvWgSK8qukjSrnnfPDSXOiJU5cGqgyDHU tXqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GNX0et0VeFDbRNtievhe/rlGdldiDAHSzzvJvy6PdeY=; b=jzw/idn2fODIsJs5lzrswlT4PVLa/GpR8uc0qcKnVihMtwktM/WFYk82qVVgjtqxz4 Y6rqfm6oMZWMzf+T+9jGUiVb/9Oos70xPKxA7eK5rSViDvZeVi2URrs1fFCIQMK+fSXD AxjiNREkk5lYATP58x8Pnh7W1ZZOGd7rFt2oO7iJokCbeniOzF+wY2ujbyJvpwrekssk zer6QWE/DzZQIKzyUjGjJHjVpfuJxGNJGzcBcW2kxbHbzHf4Mu8vyqf6Lc6JOO56PIai GXS8wTYHoJ4wfcxx5uqcgcIDfVPbcyy588H9x9OdfE+qn4i+laiHtnxONXYj3aBBvfBk LugA== X-Gm-Message-State: AJIora9qr7HmLkBm9HMJBCRXHYpQn5K5PSMDd6NWdMfCIucMOz0YtEGe +2gNZbcFLKRLPzb332O+xVGXk45PbNw= X-Google-Smtp-Source: AGRyM1uzH9PjvdEPicdkmuZyhlg/emV06zmVNYqY3j0pzDHXQSlgbpj3K/6MPdVlVtvQnC8U1qYagZgOlpg= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:2985:b0:439:651b:c1f4 with SMTP id eq5-20020a056402298500b00439651bc1f4mr8429220edb.276.1656685448889; Fri, 01 Jul 2022 07:24:08 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:44 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-20-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 19/45] kmsan: unpoison @tlb in arch_tlb_gather_mmu() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GNX0et0VeFDbRNtievhe/rlGdldiDAHSzzvJvy6PdeY=; b=A6mrYZw7rPCZmzaQzcK0RusTFq+i8tifjPKNNz0FkjiP5iGwXbEhhV+qBx3NtiqSw3Y/c5 nqgB+GBOWX2z2F4ri+Jl4GgOkiFxjsVAVkJPP2fWH+hp6JqGVHtXKskLGRXrq0L2MagyB4 oQvqeZ2jTHtd/muYNuD07r7xVKo3zQ4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685450; a=rsa-sha256; cv=none; b=SWbACiL5Ne4vhqiMWYDp4EzB2LQmSyh95KlU+DWi5pH4tgDeuTp8i5QNh7BCVSei5lFEHZ W5BJ/0yEm+ZqH1ag0k479TpIQma0XyCGnXZzpK9+/X77C25lNeSX4VytoH1jVGQF5acBqP 7mp+D1q5bqbpNrxtJ2dIQaImfFyIp2c= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sYf9Flp9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3iAO_YgYKCKcNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3iAO_YgYKCKcNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com X-Stat-Signature: ruohh5qo3cn48tq14m6xchet6kseox7y X-Rspam-User: Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sYf9Flp9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3iAO_YgYKCKcNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3iAO_YgYKCKcNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 061EF18006A X-HE-Tag: 1656685449-112346 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a hack to reduce stackdepot pressure. struct mmu_gather contains 7 1-bit fields packed into a 32-bit unsigned int value. The remaining 25 bits remain uninitialized and are never used, but KMSAN updates the origin for them in zap_pXX_range() in mm/memory.c, thus creating very long origin chains. This is technically correct, but consumes too much memory. Unpoisoning the whole structure will prevent creating such chains. Signed-off-by: Alexander Potapenko Acked-by: Marco Elver --- Link: https://linux-review.googlesource.com/id/I76abee411b8323acfdbc29bc3a60dca8cff2de77 --- mm/mmu_gather.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c index a71924bd38c0d..add4244e5790d 100644 --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -1,6 +1,7 @@ #include #include #include +#include #include #include #include @@ -265,6 +266,15 @@ void tlb_flush_mmu(struct mmu_gather *tlb) static void __tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, bool fullmm) { + /* + * struct mmu_gather contains 7 1-bit fields packed into a 32-bit + * unsigned int value. The remaining 25 bits remain uninitialized + * and are never used, but KMSAN updates the origin for them in + * zap_pXX_range() in mm/memory.c, thus creating very long origin + * chains. This is technically correct, but consumes too much memory. + * Unpoisoning the whole structure will prevent creating such chains. + */ + kmsan_unpoison_memory(tlb, sizeof(*tlb)); tlb->mm = mm; tlb->fullmm = fullmm; From patchwork Fri Jul 1 14:22:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E86A1CCA479 for ; Fri, 1 Jul 2022 14:24:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86AE86B0075; Fri, 1 Jul 2022 10:24:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 81C116B0078; Fri, 1 Jul 2022 10:24:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BE3D6B007E; Fri, 1 Jul 2022 10:24:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5CC566B0075 for ; Fri, 1 Jul 2022 10:24:13 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 2CA01808D5 for ; Fri, 1 Jul 2022 14:24:13 +0000 (UTC) X-FDA: 79638750786.02.582064D Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf02.hostedemail.com (Postfix) with ESMTP id B8E1280053 for ; Fri, 1 Jul 2022 14:24:12 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id x2-20020a1709065ac200b006d9b316257fso840111ejs.12 for ; Fri, 01 Jul 2022 07:24:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=d5jq2vBXLL0PAqQqIdp+R/eYgdfcHLruJviOlOzbAyw=; b=foA7zxm3d/yxHBn+7dN7Qf5yVA+m4cALhyxiFKss7bKF1M3ODx5vEwdDu8Z7CnbCBu OSIkaWKjVsPLHctCSmjj3EWUvd9aiSVqRJZDm9K2CWqUG+ieeba45Mq3u6VJQJwcNq0r lOhN5cunTXvYPcXTrITquDdeJ+XqRcLXJE5ps4uaJ8K1ae2F4Veo0wjtVjDSDkzSsl+w BbkhLScv8E9JYEftt93Q5opL13Lup7qtptxPZoQ4Dr14N6VU9IQAlCZiwEZVKyBiSk5N b6V/btlcruxgs+blFm7eLtLE3qAI3wOZyf4T1eZP8bk02wLrZgXR4/OGq1K4+0HF+aE0 hslw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=d5jq2vBXLL0PAqQqIdp+R/eYgdfcHLruJviOlOzbAyw=; b=pprGFEKj7cX8QyUuxAkG/KsTxWcQEMWVPzNcC/vcGho92o6xvFXNWooP6brEZ6taw7 xdnvfEPBXD5TDp4A9bHttEKSwPYhzYslv7phBtADUOTikoBD4gQIywPnm63g5+XtEcps UlXJR52rpG2FHLjvoe0Q4Ko5LUfAQkojtfz/eC2H4QRStHS7PjKmfemhkkKqkmi3IocH aPu/kciqR9k8I/ahAV8usZCaZFDAt5yQqDfpMjMLoDRUJrZ7DvhRTCgMe30qsuWxOFzZ xgO5ZYegd5h/fiC7n+qqkHEEcfAbHw18ci/QtluVnP+HCxT1jTNtAVqToS7Bhbju2YU/ ekhg== X-Gm-Message-State: AJIora+WpuvhwQBDKKO+7BhvPL9SDeyw6+8XNSOgV1kT+xYu86/zrCA4 e46Ixb+VroDwW0pLiGL98tq0XnH+cO8= X-Google-Smtp-Source: AGRyM1tOQ9hEzrWjXXF1mGZvlD2bn9kK96D3aHnwRR/ak9ADNhFKggqQeO/Ez5UYkEsxu56uu1Qj7OsLgis= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:aa7:d5c9:0:b0:435:8099:30e6 with SMTP id d9-20020aa7d5c9000000b00435809930e6mr19303788eds.384.1656685451552; Fri, 01 Jul 2022 07:24:11 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:45 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-21-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 20/45] kmsan: add iomap support From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685452; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d5jq2vBXLL0PAqQqIdp+R/eYgdfcHLruJviOlOzbAyw=; b=oP3gceEpHYCuOjdGDsyMiGDzOe+6IsMPTi4oBIIEOnzHoIX6N7SDH+bIaJtrIGR5p7nV/l g3LKaefww22vr1KFppR1s7lrjAjgi4AZkk9/cOtl7Mq3KhIlx2Z/uoOPeHPh5znRAXfOx3 AGS/w4TChNAj7MVIGzGiwutUzLom2KY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685452; a=rsa-sha256; cv=none; b=dE/q2mk1oLHR88je0bHKM+1e21InFn0hvg5A6Epkrp/OqWHv0epzaZVY6bZAvLdCHrK1Wf TG6yd8x2m4tgCR2Z4tDTtTbesTGIUdPKNfsiKpJ4LFBVRqwyab1Nd8Z38AcCjbAzo6ytRm KAjm7XpEAX0ALqGmWZi1raLjLyvlNhU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=foA7zxm3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3iwO_YgYKCKoQVSNObQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3iwO_YgYKCKoQVSNObQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--glider.bounces.google.com X-Stat-Signature: rexhettui4khsxfegcgrpshrc15jm57a X-Rspam-User: Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=foA7zxm3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3iwO_YgYKCKoQVSNObQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3iwO_YgYKCKoQVSNObQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--glider.bounces.google.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B8E1280053 X-HE-Tag: 1656685452-149048 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Functions from lib/iomap.c interact with hardware, so KMSAN must ensure that: - every read function returns an initialized value - every write function checks values before sending them to hardware. Signed-off-by: Alexander Potapenko --- v4: -- switch from __no_sanitize_memory (which now means "no KMSAN instrumentation") to __no_kmsan_checks (i.e. "unpoison everything") Link: https://linux-review.googlesource.com/id/I45527599f09090aca046dfe1a26df453adab100d --- lib/iomap.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/lib/iomap.c b/lib/iomap.c index fbaa3e8f19d6c..4f8b31baa5752 100644 --- a/lib/iomap.c +++ b/lib/iomap.c @@ -6,6 +6,7 @@ */ #include #include +#include #include @@ -70,26 +71,35 @@ static void bad_io_access(unsigned long port, const char *access) #define mmio_read64be(addr) swab64(readq(addr)) #endif +/* + * Here and below, we apply __no_kmsan_checks to functions reading data from + * hardware, to ensure that KMSAN marks their return values as initialized. + */ +__no_kmsan_checks unsigned int ioread8(const void __iomem *addr) { IO_COND(addr, return inb(port), return readb(addr)); return 0xff; } +__no_kmsan_checks unsigned int ioread16(const void __iomem *addr) { IO_COND(addr, return inw(port), return readw(addr)); return 0xffff; } +__no_kmsan_checks unsigned int ioread16be(const void __iomem *addr) { IO_COND(addr, return pio_read16be(port), return mmio_read16be(addr)); return 0xffff; } +__no_kmsan_checks unsigned int ioread32(const void __iomem *addr) { IO_COND(addr, return inl(port), return readl(addr)); return 0xffffffff; } +__no_kmsan_checks unsigned int ioread32be(const void __iomem *addr) { IO_COND(addr, return pio_read32be(port), return mmio_read32be(addr)); @@ -142,18 +152,21 @@ static u64 pio_read64be_hi_lo(unsigned long port) return lo | (hi << 32); } +__no_kmsan_checks u64 ioread64_lo_hi(const void __iomem *addr) { IO_COND(addr, return pio_read64_lo_hi(port), return readq(addr)); return 0xffffffffffffffffULL; } +__no_kmsan_checks u64 ioread64_hi_lo(const void __iomem *addr) { IO_COND(addr, return pio_read64_hi_lo(port), return readq(addr)); return 0xffffffffffffffffULL; } +__no_kmsan_checks u64 ioread64be_lo_hi(const void __iomem *addr) { IO_COND(addr, return pio_read64be_lo_hi(port), @@ -161,6 +174,7 @@ u64 ioread64be_lo_hi(const void __iomem *addr) return 0xffffffffffffffffULL; } +__no_kmsan_checks u64 ioread64be_hi_lo(const void __iomem *addr) { IO_COND(addr, return pio_read64be_hi_lo(port), @@ -188,22 +202,32 @@ EXPORT_SYMBOL(ioread64be_hi_lo); void iowrite8(u8 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, outb(val,port), writeb(val, addr)); } void iowrite16(u16 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, outw(val,port), writew(val, addr)); } void iowrite16be(u16 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, pio_write16be(val,port), mmio_write16be(val, addr)); } void iowrite32(u32 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, outl(val,port), writel(val, addr)); } void iowrite32be(u32 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, pio_write32be(val,port), mmio_write32be(val, addr)); } EXPORT_SYMBOL(iowrite8); @@ -239,24 +263,32 @@ static void pio_write64be_hi_lo(u64 val, unsigned long port) void iowrite64_lo_hi(u64 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, pio_write64_lo_hi(val, port), writeq(val, addr)); } void iowrite64_hi_lo(u64 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, pio_write64_hi_lo(val, port), writeq(val, addr)); } void iowrite64be_lo_hi(u64 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, pio_write64be_lo_hi(val, port), mmio_write64be(val, addr)); } void iowrite64be_hi_lo(u64 val, void __iomem *addr) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(&val, sizeof(val)); IO_COND(addr, pio_write64be_hi_lo(val, port), mmio_write64be(val, addr)); } @@ -328,14 +360,20 @@ static inline void mmio_outsl(void __iomem *addr, const u32 *src, int count) void ioread8_rep(const void __iomem *addr, void *dst, unsigned long count) { IO_COND(addr, insb(port,dst,count), mmio_insb(addr, dst, count)); + /* KMSAN must treat values read from devices as initialized. */ + kmsan_unpoison_memory(dst, count); } void ioread16_rep(const void __iomem *addr, void *dst, unsigned long count) { IO_COND(addr, insw(port,dst,count), mmio_insw(addr, dst, count)); + /* KMSAN must treat values read from devices as initialized. */ + kmsan_unpoison_memory(dst, count * 2); } void ioread32_rep(const void __iomem *addr, void *dst, unsigned long count) { IO_COND(addr, insl(port,dst,count), mmio_insl(addr, dst, count)); + /* KMSAN must treat values read from devices as initialized. */ + kmsan_unpoison_memory(dst, count * 4); } EXPORT_SYMBOL(ioread8_rep); EXPORT_SYMBOL(ioread16_rep); @@ -343,14 +381,20 @@ EXPORT_SYMBOL(ioread32_rep); void iowrite8_rep(void __iomem *addr, const void *src, unsigned long count) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(src, count); IO_COND(addr, outsb(port, src, count), mmio_outsb(addr, src, count)); } void iowrite16_rep(void __iomem *addr, const void *src, unsigned long count) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(src, count * 2); IO_COND(addr, outsw(port, src, count), mmio_outsw(addr, src, count)); } void iowrite32_rep(void __iomem *addr, const void *src, unsigned long count) { + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(src, count * 4); IO_COND(addr, outsl(port, src,count), mmio_outsl(addr, src, count)); } EXPORT_SYMBOL(iowrite8_rep); From patchwork Fri Jul 1 14:22:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6E2ECCA479 for ; Fri, 1 Jul 2022 14:24:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FEE36B0078; Fri, 1 Jul 2022 10:24:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AD976B007E; Fri, 1 Jul 2022 10:24:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2753A6B008C; Fri, 1 Jul 2022 10:24:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 18E0B6B0078 for ; Fri, 1 Jul 2022 10:24:16 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0028C612B6 for ; Fri, 1 Jul 2022 14:24:15 +0000 (UTC) X-FDA: 79638750912.14.BD22467 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf30.hostedemail.com (Postfix) with ESMTP id 927148003D for ; Fri, 1 Jul 2022 14:24:15 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id h16-20020a05640250d000b00435bab1a7b4so1893130edb.10 for ; Fri, 01 Jul 2022 07:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7LZjp1LTGSiY93TRZ3g7FZorW8E9Cp1phv1xqeqiFUY=; b=HUp0QjEtL3POjFsPVmW7Lsr6wHh3uP0c3AIGbejYW5WBVEEA51jhGDXufEvS9+1OoF 4uVyuJc0WeTwbfhCMIs+rmrcOAjTWABoSd8Oaw1teEbFzU0MKdY/z4Slq6iDFA136+6A Dq0ghLvrTYYGXMowWhh/7yEVnmXH6kNFujWdairYknULTqhesx4Yk07JTgMa2g03KfDv ZrTJqfs2WDDcnPD58XjBPdAHHwnHRzjgo5idM+mqSJrORCqZ/jLttEcLx+95oGm5NNfU HQHKB38oaAiXzo52FsKtum2XD5ZN4s5oCz9X6TuacgVJMlaPcY5XhphiQ91hDBCbdhJB tKOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7LZjp1LTGSiY93TRZ3g7FZorW8E9Cp1phv1xqeqiFUY=; b=USvQ9fUxwSWgc4RIC23VUi6DDz7QYKUg1llxBUzTp+6RHlSj9nAZYFoKDaXMS+KfCl NCQWFBIgFLIust6p7+RrYHgNllVyQ0oKN0d1YrKSPqCpyqUQLXesKOpZd3/AOsE6O9PZ lNxtgnaADOnngMlHogXREy1J4sMF9OdyZT3TrY7KEo+dGHzEqp85Bk3oVWm8LnofqMZp P8pLrVvg7ywSoNF0tMAtQqPHGRUuDH0LmXa6RDZwuJ0PqvIOtdFU9aRPNVjeeLGwvBAR Zg4F37ZUlfM21McYVDZ0+7sgTib9ajrnEcKrVSudvyTg8Dgh99Q0WU4kOhAecNKYXTYP W+oA== X-Gm-Message-State: AJIora8yzsLTtlqa+Q81MwNaOKcSTKuAvpAUftirrhLD33S2hrwmt2ZD SadwQGktjLsUdLwaFZ0k0qREl3YGC8g= X-Google-Smtp-Source: AGRyM1sKIpsY5lY7K78S+NlVsi52z6DpTXM615RMvKRvttRlQPej/Daek4sIgXeLlq3X+zfaa9FMqF7k7ig= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:4493:b0:435:8dd5:c951 with SMTP id er19-20020a056402449300b004358dd5c951mr18955210edb.289.1656685454405; Fri, 01 Jul 2022 07:24:14 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:46 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-22-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 21/45] Input: libps2: mark data received in __ps2_command() as initialized From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685455; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7LZjp1LTGSiY93TRZ3g7FZorW8E9Cp1phv1xqeqiFUY=; b=5zzP4wY6DurpdlyKD4lI0JawaeHQyP1xj6h+B+LX4cIyo+jwZfzBmMOKo6P8Lv/WTunBJ6 rZ85ZRtczF3CYCBZV4Rpgpwtbv1Z1II27L8NnQg+Oeek2PKfqWxVwiscHr5pzuAycXLjbJ crwYxMo5OH1oyO7gmflvS8xOLonBFaE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685455; a=rsa-sha256; cv=none; b=tc+jG+5/vA5H2WIu30B1/0bgUCmkieCCoBReCcC1xQdq+deFTpPt5T18IPv7jSB71pjAzL uGpYpIMLov6ivpGFUMS4w09hOrCEze1iBO0zDyKcpZc5U4Nz4hzMQZpbR8Qp6f3eX+ULFR 6T+8Bdj6u4ogkq/oPtW4BvPM51jORQ4= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HUp0QjEt; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3jgO_YgYKCK0TYVQReTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3jgO_YgYKCK0TYVQReTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--glider.bounces.google.com X-Stat-Signature: ybkm1st4mkeufws4dt3am3sfgtztdt1f X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=HUp0QjEt; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3jgO_YgYKCK0TYVQReTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3jgO_YgYKCK0TYVQReTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--glider.bounces.google.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 927148003D X-HE-Tag: 1656685455-508449 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN does not know that the device initializes certain bytes in ps2dev->cmdbuf. Call kmsan_unpoison_memory() to explicitly mark them as initialized. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I2d26f6baa45271d37320d3f4a528c39cb7e545f0 --- drivers/input/serio/libps2.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/input/serio/libps2.c b/drivers/input/serio/libps2.c index 250e213cc80c6..3e19344eda93c 100644 --- a/drivers/input/serio/libps2.c +++ b/drivers/input/serio/libps2.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -294,9 +295,11 @@ int __ps2_command(struct ps2dev *ps2dev, u8 *param, unsigned int command) serio_pause_rx(ps2dev->serio); - if (param) + if (param) { for (i = 0; i < receive; i++) param[i] = ps2dev->cmdbuf[(receive - 1) - i]; + kmsan_unpoison_memory(param, receive); + } if (ps2dev->cmdcnt && (command != PS2_CMD_RESET_BAT || ps2dev->cmdcnt != 1)) { From patchwork Fri Jul 1 14:22:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E0B3C43334 for ; Fri, 1 Jul 2022 14:24:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 166826B0071; Fri, 1 Jul 2022 10:24:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 116A56B007E; Fri, 1 Jul 2022 10:24:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1FAA6B008C; Fri, 1 Jul 2022 10:24:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E37EA6B0071 for ; Fri, 1 Jul 2022 10:24:18 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id C0BB280856 for ; Fri, 1 Jul 2022 14:24:18 +0000 (UTC) X-FDA: 79638750996.30.B8B2C75 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf17.hostedemail.com (Postfix) with ESMTP id 4ACBB40044 for ; Fri, 1 Jul 2022 14:24:18 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id hp8-20020a1709073e0800b0072629757566so852346ejc.0 for ; Fri, 01 Jul 2022 07:24:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=q/n/OecEQokjv5vk7nJWmgO+y+lMcJIuX+yK+Jk36kg=; b=U5LnbAq/BfGg73b3hAPk1LGNT654UNP9cra9ckGKhN/P0fyT5dmbNa1GwGMC/VQSau kRRACO1kwseZjFl8pDjCJnqdw+CVsYmJalihZ0BfiiWO66sGbSWS/QGcsD8uZTf0fZVy 6eRMZPVngQrj635M05w5qDdJQQzaIckJ8YJ7a9RY76OdD8EzT31PanbT0T3p4KWXGYQF +qmU1PTyJ4IBn8XSLLL2PEetWgBrtk9fkaI3OUzV7+LCeZNWoyFZA/kpqxczFaWlH/25 jGYWS+j/JNFVmhC3ACDNxjLd8Gs1XS5BL6t/D7d5YG8xFOVszZWJNMj5ExZnAIg3/Kxj YuYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=q/n/OecEQokjv5vk7nJWmgO+y+lMcJIuX+yK+Jk36kg=; b=WgIdUBshY52ypZ9GLgeKbSSiBeLa7KAiD4WppX06R8JAsoiW4NEB3ih+Fxdr5xefLd 2EJsT630ppxcYapptbZn2cXV7ymR6V8WVjVRodpqF/hL+EYbH2iryiug8rV9cGrv+v+9 uCHiOPfxgLKy4YWmFB7tUPn7CgJMKY1khUmZ1BXlhQznZrXEBgPDXyhDXzcITB8isPhH qH6hGvYDW/GzYKmGyk+dezMS1r4+Yh8vhsVp0z1pFUasMVPBjOmTJY8hDh9pWcWvYfJr HsHKhJGNcRtxBIRpxEtJNHb++XB1aTyX5g/eSbbEqpnRKIpP1YVxqwWBIs2ebvzpdFO/ veBg== X-Gm-Message-State: AJIora84JA9014VJ3U3kM40jORwkvXZlfaZRrfqv1JF3BqBe7PHQ7+EZ ym8UTXHuNM6Qmp00szdkNu5uQyX88pg= X-Google-Smtp-Source: AGRyM1vh1xuSpd10aw8ZdO/pImp9DeAVweQvga8OQDtjXKfbnui60mltRkNXb/YJK5XPxxAY6bzLEimCmls= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:f1b:b0:436:d3c4:aef2 with SMTP id i27-20020a0564020f1b00b00436d3c4aef2mr19579345eda.27.1656685457025; Fri, 01 Jul 2022 07:24:17 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:47 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-23-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 22/45] dma: kmsan: unpoison DMA mappings From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="U5LnbAq/"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3kQO_YgYKCLAWbYTUhWeeWbU.SecbYdkn-ccalQSa.ehW@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3kQO_YgYKCLAWbYTUhWeeWbU.SecbYdkn-ccalQSa.ehW@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685458; a=rsa-sha256; cv=none; b=6w/8/770L2GDAZsUSDUsnnlEfuHl/SBEEwbXO43Tr2rB2ZNpBKrS0vZ2TJiwE3yZvGWL/r Tx9BQF3bcUP+bcgRMZezrQZP6doGH5a0/B7RKrIQvKbKShyok+AThP21PhTN/HLy5gjAjp y0XzYFfIc4cqlaojL61jP/AuicX6Lew= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685458; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q/n/OecEQokjv5vk7nJWmgO+y+lMcJIuX+yK+Jk36kg=; b=wSvPZU5q9SLrb609UeVFJRrZbFtvcLkm+ifraGrwzpUdL0ob89ZFdKrtL2ga+m0S1S15b+ Kz5EH0ztF9pTKZNsQRPyvWgkY44dgx7HxL6IgwH86icni8gLpQnHCRGrKSkOmvcgmdWLb+ KkomUBj8Fs6yc0W+jPS7XPOi3qlciJc= X-Stat-Signature: sebx1nxk7z6ofk45trz4mr8zn793b5m8 X-Rspamd-Queue-Id: 4ACBB40044 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="U5LnbAq/"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3kQO_YgYKCLAWbYTUhWeeWbU.SecbYdkn-ccalQSa.ehW@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3kQO_YgYKCLAWbYTUhWeeWbU.SecbYdkn-ccalQSa.ehW@flex--glider.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1656685458-335563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN doesn't know about DMA memory writes performed by devices. We unpoison such memory when it's mapped to avoid false positive reports. Signed-off-by: Alexander Potapenko --- v2: -- move implementation of kmsan_handle_dma() and kmsan_handle_dma_sg() here v4: -- swap dma: and kmsan: int the subject Link: https://linux-review.googlesource.com/id/Ia162dc4c5a92e74d4686c1be32a4dfeffc5c32cd --- include/linux/kmsan.h | 41 +++++++++++++++++++++++++++++ kernel/dma/mapping.c | 9 ++++--- mm/kmsan/hooks.c | 61 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 108 insertions(+), 3 deletions(-) diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index 82fd564cc72e7..55fe673ee1e84 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -9,6 +9,7 @@ #ifndef _LINUX_KMSAN_H #define _LINUX_KMSAN_H +#include #include #include #include @@ -17,6 +18,7 @@ struct page; struct kmem_cache; struct task_struct; +struct scatterlist; #ifdef CONFIG_KMSAN @@ -204,6 +206,35 @@ void kmsan_ioremap_page_range(unsigned long addr, unsigned long end, */ void kmsan_iounmap_page_range(unsigned long start, unsigned long end); +/** + * kmsan_handle_dma() - Handle a DMA data transfer. + * @page: first page of the buffer. + * @offset: offset of the buffer within the first page. + * @size: buffer size. + * @dir: one of possible dma_data_direction values. + * + * Depending on @direction, KMSAN: + * * checks the buffer, if it is copied to device; + * * initializes the buffer, if it is copied from device; + * * does both, if this is a DMA_BIDIRECTIONAL transfer. + */ +void kmsan_handle_dma(struct page *page, size_t offset, size_t size, + enum dma_data_direction dir); + +/** + * kmsan_handle_dma_sg() - Handle a DMA transfer using scatterlist. + * @sg: scatterlist holding DMA buffers. + * @nents: number of scatterlist entries. + * @dir: one of possible dma_data_direction values. + * + * Depending on @direction, KMSAN: + * * checks the buffers in the scatterlist, if they are copied to device; + * * initializes the buffers, if they are copied from device; + * * does both, if this is a DMA_BIDIRECTIONAL transfer. + */ +void kmsan_handle_dma_sg(struct scatterlist *sg, int nents, + enum dma_data_direction dir); + #else static inline void kmsan_init_shadow(void) @@ -286,6 +317,16 @@ static inline void kmsan_iounmap_page_range(unsigned long start, { } +static inline void kmsan_handle_dma(struct page *page, size_t offset, + size_t size, enum dma_data_direction dir) +{ +} + +static inline void kmsan_handle_dma_sg(struct scatterlist *sg, int nents, + enum dma_data_direction dir) +{ +} + #endif #endif /* _LINUX_KMSAN_H */ diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index db7244291b745..5d17d5d62166b 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -156,6 +156,7 @@ dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, addr = dma_direct_map_page(dev, page, offset, size, dir, attrs); else addr = ops->map_page(dev, page, offset, size, dir, attrs); + kmsan_handle_dma(page, offset, size, dir); debug_dma_map_page(dev, page, offset, size, dir, addr, attrs); return addr; @@ -194,11 +195,13 @@ static int __dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, else ents = ops->map_sg(dev, sg, nents, dir, attrs); - if (ents > 0) + if (ents > 0) { + kmsan_handle_dma_sg(sg, nents, dir); debug_dma_map_sg(dev, sg, nents, ents, dir, attrs); - else if (WARN_ON_ONCE(ents != -EINVAL && ents != -ENOMEM && - ents != -EIO)) + } else if (WARN_ON_ONCE(ents != -EINVAL && ents != -ENOMEM && + ents != -EIO)) { return -EIO; + } return ents; } diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 1cdb4420977f1..8a6947a2a2f22 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -10,9 +10,11 @@ */ #include +#include #include #include #include +#include #include #include @@ -250,6 +252,65 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, } EXPORT_SYMBOL(kmsan_copy_to_user); +static void kmsan_handle_dma_page(const void *addr, size_t size, + enum dma_data_direction dir) +{ + switch (dir) { + case DMA_BIDIRECTIONAL: + kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0, + REASON_ANY); + kmsan_internal_unpoison_memory((void *)addr, size, + /*checked*/ false); + break; + case DMA_TO_DEVICE: + kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0, + REASON_ANY); + break; + case DMA_FROM_DEVICE: + kmsan_internal_unpoison_memory((void *)addr, size, + /*checked*/ false); + break; + case DMA_NONE: + break; + } +} + +/* Helper function to handle DMA data transfers. */ +void kmsan_handle_dma(struct page *page, size_t offset, size_t size, + enum dma_data_direction dir) +{ + u64 page_offset, to_go, addr; + + if (PageHighMem(page)) + return; + addr = (u64)page_address(page) + offset; + /* + * The kernel may occasionally give us adjacent DMA pages not belonging + * to the same allocation. Process them separately to avoid triggering + * internal KMSAN checks. + */ + while (size > 0) { + page_offset = addr % PAGE_SIZE; + to_go = min(PAGE_SIZE - page_offset, (u64)size); + kmsan_handle_dma_page((void *)addr, to_go, dir); + addr += to_go; + size -= to_go; + } +} +EXPORT_SYMBOL(kmsan_handle_dma); + +void kmsan_handle_dma_sg(struct scatterlist *sg, int nents, + enum dma_data_direction dir) +{ + struct scatterlist *item; + int i; + + for_each_sg(sg, item, nents, i) + kmsan_handle_dma(sg_page(item), item->offset, item->length, + dir); +} +EXPORT_SYMBOL(kmsan_handle_dma_sg); + /* Functions from kmsan-checks.h follow. */ void kmsan_poison_memory(const void *address, size_t size, gfp_t flags) { From patchwork Fri Jul 1 14:22:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 379C5C43334 for ; Fri, 1 Jul 2022 14:24:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB0AC6B007E; Fri, 1 Jul 2022 10:24:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C60356B0080; Fri, 1 Jul 2022 10:24:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B00F16B0083; Fri, 1 Jul 2022 10:24:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A06786B007E for ; Fri, 1 Jul 2022 10:24:21 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 772572FFC0 for ; Fri, 1 Jul 2022 14:24:21 +0000 (UTC) X-FDA: 79638751122.26.0872B65 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf09.hostedemail.com (Postfix) with ESMTP id F1679140049 for ; Fri, 1 Jul 2022 14:24:20 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id c20-20020a05640227d400b004369cf00c6bso1888601ede.22 for ; Fri, 01 Jul 2022 07:24:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+K29lQHYx/QboL6+Fy2jFmZJqrUMV6q0RSetNFINlHU=; b=Lp2eEEvz0+TgiBntUx4ua/9fGqKccdfR6XH7uHLacnvYo4bPCaLpszQq+8mA0Y5gts YCzUjj3dnLK0eoahSun1hhzXa9zo2wYpNUMZwR4kY7OA35DYyKZ9LQhFeSUbQH6W8LBU cEi1G7HBTkR7zPLvT/TESyx9jjytrH3anjLjFrwMdooDJEmIyePYHuF/VlAGlAiBGHgQ /2HrW5IdeOp02Byommjy5nWcpNJpE2ztvAY4y3tZZGaGC81RVSmcis/j1833duSMcHp5 Z+3w/2IiHJlg4Mg0b3tNbahk09J0jUi+gYRmg2qSYy97kQYMJVderfkH45qNQi5yNzLp 8KnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+K29lQHYx/QboL6+Fy2jFmZJqrUMV6q0RSetNFINlHU=; b=IeWEgbn6GoWbmRfdJKJlSkhiXmMLxc1Zfu+7QKqI8pgn9MQoaiA09zSxfUP+pOX/Hn Ec4FwSvYZ0bZ0Kp2zQakO5LcwP5cIvBVKdHOaAvFZL5oEz1EGNehcEOXjNHcD3UxSY9s 8/NcJB3Uv4EMe3LKNcQlT6fIflvlYaIefzOnTleyCi2yPN5bo2f/FljyaGfHAWdKdz4R qC/4oDjSxM80wDEFBsGQxrA70y6Pk6ID1GF+2IxHnsajycjrO8fLOlP2zfhD7mY+r3ZH cN+igoy9lUOO7nn48hYEPQxWpjhw20oo//TtusI6XBuTljJNgK9yPywMRzW34TD+tIis hdhQ== X-Gm-Message-State: AJIora/YZd9x3374pAxF2EhsXHC6WmVXtbeQj4CMPXRgwf9jGz8n+h/9 LpOr5J/Zuvn5wRQXO2drRK3KkgQWm8o= X-Google-Smtp-Source: AGRyM1v+DIWwLp7Wzl6B8tncc37UxB9rTXYufdTzcwNSGLPhyYz8kU9A/vUtw7VqYleTmtYO9LI0fVniQsI= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:15a:b0:431:71b9:86f3 with SMTP id s26-20020a056402015a00b0043171b986f3mr18869662edu.249.1656685459864; Fri, 01 Jul 2022 07:24:19 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:48 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-24-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 23/45] virtio: kmsan: check/unpoison scatterlist in vring_map_one_sg() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685461; a=rsa-sha256; cv=none; b=lQH9hXiW3nLM+EmEhkj9Mgd4pUm+KmW+KKGNxKtCxB6Y4QlOtImtesEa832Q+DdHp8YE+A jxYux74uZjaKoZqtzFHHhgvG5/WxfhEOviYbPqrDfpz+87XgtSckNiJXD+tuyRHm+ok6Z+ wLUHHC5HMjEvnTI3r7+Hc4+4QK/kXfc= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Lp2eEEvz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3kwO_YgYKCLIYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3kwO_YgYKCLIYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685461; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+K29lQHYx/QboL6+Fy2jFmZJqrUMV6q0RSetNFINlHU=; b=q/4xHmaCzTNkHDa7dXIBOKi39UknC78g4oqpw13FUBO0a0jZyQtf8zBZ2J3B6Nyr+1igSD rJ6dz8aGxupxFSXletjL9QTg65U+WSUHHHq6VgIuBdtWkFOjoxlEEg3MsOEzqUxtljOYKo J7ALr5dXlYqHVhoNi6lLbgnwkpTu2OE= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: F1679140049 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Lp2eEEvz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3kwO_YgYKCLIYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3kwO_YgYKCLIYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: rrbzkezi11geybudn3a4q6e8dpnhg9f1 X-HE-Tag: 1656685460-586032 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If vring doesn't use the DMA API, KMSAN is unable to tell whether the memory is initialized by hardware. Explicitly call kmsan_handle_dma() from vring_map_one_sg() in this case to prevent false positives. Signed-off-by: Alexander Potapenko Acked-by: Michael S. Tsirkin --- v4: -- swap virtio: and kmsan: in the subject Link: https://linux-review.googlesource.com/id/I211533ecb86a66624e151551f83ddd749536b3af --- drivers/virtio/virtio_ring.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 13a7348cedfff..2d42a4b38e628 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -329,8 +330,15 @@ static dma_addr_t vring_map_one_sg(const struct vring_virtqueue *vq, struct scatterlist *sg, enum dma_data_direction direction) { - if (!vq->use_dma_api) + if (!vq->use_dma_api) { + /* + * If DMA is not used, KMSAN doesn't know that the scatterlist + * is initialized by the hardware. Explicitly check/unpoison it + * depending on the direction. + */ + kmsan_handle_dma(sg_page(sg), sg->offset, sg->length, direction); return (dma_addr_t)sg_phys(sg); + } /* * We can't use dma_map_sg, because we don't use scatterlists in From patchwork Fri Jul 1 14:22:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33C9AC43334 for ; Fri, 1 Jul 2022 14:24:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3B0A6B007B; Fri, 1 Jul 2022 10:24:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BEB8B6B0080; Fri, 1 Jul 2022 10:24:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A64CE6B0081; Fri, 1 Jul 2022 10:24:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 96B506B007B for ; Fri, 1 Jul 2022 10:24:24 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 619C332E61 for ; Fri, 1 Jul 2022 14:24:24 +0000 (UTC) X-FDA: 79638751248.23.DF0903D Received: from mail-lf1-f74.google.com (mail-lf1-f74.google.com [209.85.167.74]) by imf09.hostedemail.com (Postfix) with ESMTP id 0266014004A for ; Fri, 1 Jul 2022 14:24:23 +0000 (UTC) Received: by mail-lf1-f74.google.com with SMTP id cf10-20020a056512280a00b0047f5a295656so1179054lfb.15 for ; Fri, 01 Jul 2022 07:24:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zhMMZRuLDF6Pq/3lj4DVrVpRjyIDnZPpkOwtCQdBukQ=; b=lkUxcVIUd6qYJWLZjSG/W4rDF4tKMzJAIsWUfl6gMo2TbAtriVTJcbmnvCG0fDmLtR DSxHoi2jn1iZ6FTySbjTu0l2dKeJKYobkKMLVr3syT3UqAPU7Gicf7xbtJFBAnO8MUY4 vTjs9vt2d4hTA5DzWjclXDwG6pCooj2Y6P8QdUCwb/eQteblrOr5loBK0JOs3GeXIjiT bTlvokzLiEC8wCntWUwPevBsZ4yMr8XgzW+9JpT56ZFCmuIv1st0kOtDttQvmEkHDh8L 94SedVfvAPCYp0ve6D8T2GRRbwpPBMgE9SCNcHcKSge6nGUJRq/FSujqYTyhaNQGmAwj kbcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zhMMZRuLDF6Pq/3lj4DVrVpRjyIDnZPpkOwtCQdBukQ=; b=vzmgVgwGLXEMnWoorgu5XS2IuimuGmc3EqeOLVUpVmKfKWD1qpdoMYt7J5GfGFNAgS jGUWCjdZuUnoUyg8WTAks6HQ1BtuwCFVCUUm4d8Kr+vFMtRMHo+hdD3id8+DwONGVU23 d6IU0UIbMuC2iWqb3daNDYTIlojSs2ra4JUjLolqavBmtUv1fgZAAV9chSonCsbZjbWG /tOHF1bmQgL0HV1X04T5OBjSk4U403KME8LCsXImOo7pkyjccQNCEY2/C9w/tuKWnycI 56pfa38bO69nlyb77uAD9RL4e0iDRl1AGMbIM8gn0cn2yRKXR7wtwYUy9DQz1/pAxk5b Dq6A== X-Gm-Message-State: AJIora/di7/rd91R2aba26XKclO+Pd2mAbEZhDew7ZyGxasYdvVup5oz 1iRe9ybsJ861Xu/FZlJ4hn5zuCgd1oU= X-Google-Smtp-Source: AGRyM1ujhhEkPAeMzba+3rjROrqZ1qk7eONuMn4gI/gNE3ySm/CnDDnQvH3rDtZnfoMmy4wbS0DjU1LDOLQ= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6512:1291:b0:47f:6ece:310e with SMTP id u17-20020a056512129100b0047f6ece310emr9097403lfs.389.1656685462649; Fri, 01 Jul 2022 07:24:22 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:49 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-25-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 24/45] kmsan: handle memory sent to/from USB From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685464; a=rsa-sha256; cv=none; b=bZ2mZMpaWWgHV9fcNHPEiYOZXnaymniuHmZnwzCJfOluc4xXY/adzvFy0HIzPMbu0HEFt9 dh8ofvhZXj2Swy7SdcvvMdTgT9FxOINkM++OlY6fqpJnu06pDaSaHsoRBblotr8THcIid3 n8dSArRKAvFURWsmIHF4nQUtWT5TPBI= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lkUxcVIU; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3lgO_YgYKCLUbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com designates 209.85.167.74 as permitted sender) smtp.mailfrom=3lgO_YgYKCLUbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685464; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zhMMZRuLDF6Pq/3lj4DVrVpRjyIDnZPpkOwtCQdBukQ=; b=z1BpUM4VIegOpVa50T52F/2YkhtRUyHVLGmkvBOvGupeyc/VXR39cC9ls/yNWLqTdxiKFu 23prLU98usBivemD1xB3qzacIdOrNTWj7yv11TFGIKAAtJUmVmNWILFrVv2zv/kQdjB0py jakoP+Bds1WKPo22SbfpGYN0N8ZpnH4= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0266014004A Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lkUxcVIU; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3lgO_YgYKCLUbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com designates 209.85.167.74 as permitted sender) smtp.mailfrom=3lgO_YgYKCLUbgdYZmbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: siaqzb9dqmtce5sd3gbecndr4z3sux6m X-HE-Tag: 1656685463-920020 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Depending on the value of is_out kmsan_handle_urb() KMSAN either marks the data copied to the kernel from a USB device as initialized, or checks the data sent to the device for being initialized. Signed-off-by: Alexander Potapenko --- v2: -- move kmsan_handle_urb() implementation to this patch Link: https://linux-review.googlesource.com/id/Ifa67fb72015d4de14c30e971556f99fc8b2ee506 --- drivers/usb/core/urb.c | 2 ++ include/linux/kmsan.h | 15 +++++++++++++++ mm/kmsan/hooks.c | 17 +++++++++++++++++ 3 files changed, 34 insertions(+) diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c index 33d62d7e3929f..1fe3f23205624 100644 --- a/drivers/usb/core/urb.c +++ b/drivers/usb/core/urb.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -426,6 +427,7 @@ int usb_submit_urb(struct urb *urb, gfp_t mem_flags) URB_SETUP_MAP_SINGLE | URB_SETUP_MAP_LOCAL | URB_DMA_SG_COMBINED); urb->transfer_flags |= (is_out ? URB_DIR_OUT : URB_DIR_IN); + kmsan_handle_urb(urb, is_out); if (xfertype != USB_ENDPOINT_XFER_CONTROL && dev->state < USB_STATE_CONFIGURED) diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index 55fe673ee1e84..e8b5c306c4aa1 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -19,6 +19,7 @@ struct page; struct kmem_cache; struct task_struct; struct scatterlist; +struct urb; #ifdef CONFIG_KMSAN @@ -235,6 +236,16 @@ void kmsan_handle_dma(struct page *page, size_t offset, size_t size, void kmsan_handle_dma_sg(struct scatterlist *sg, int nents, enum dma_data_direction dir); +/** + * kmsan_handle_urb() - Handle a USB data transfer. + * @urb: struct urb pointer. + * @is_out: data transfer direction (true means output to hardware). + * + * If @is_out is true, KMSAN checks the transfer buffer of @urb. Otherwise, + * KMSAN initializes the transfer buffer. + */ +void kmsan_handle_urb(const struct urb *urb, bool is_out); + #else static inline void kmsan_init_shadow(void) @@ -327,6 +338,10 @@ static inline void kmsan_handle_dma_sg(struct scatterlist *sg, int nents, { } +static inline void kmsan_handle_urb(const struct urb *urb, bool is_out) +{ +} + #endif #endif /* _LINUX_KMSAN_H */ diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 8a6947a2a2f22..9aecbf2825837 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -17,6 +17,7 @@ #include #include #include +#include #include "../internal.h" #include "../slab.h" @@ -252,6 +253,22 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy, } EXPORT_SYMBOL(kmsan_copy_to_user); +/* Helper function to check an URB. */ +void kmsan_handle_urb(const struct urb *urb, bool is_out) +{ + if (!urb) + return; + if (is_out) + kmsan_internal_check_memory(urb->transfer_buffer, + urb->transfer_buffer_length, + /*user_addr*/ 0, REASON_SUBMIT_URB); + else + kmsan_internal_unpoison_memory(urb->transfer_buffer, + urb->transfer_buffer_length, + /*checked*/ false); +} +EXPORT_SYMBOL(kmsan_handle_urb); + static void kmsan_handle_dma_page(const void *addr, size_t size, enum dma_data_direction dir) { From patchwork Fri Jul 1 14:22:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 202A8C43334 for ; Fri, 1 Jul 2022 14:24:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B029D6B0081; Fri, 1 Jul 2022 10:24:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A8B8B6B0082; Fri, 1 Jul 2022 10:24:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92DB66B0083; Fri, 1 Jul 2022 10:24:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 809EA6B0081 for ; Fri, 1 Jul 2022 10:24:27 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 445941201F8 for ; Fri, 1 Jul 2022 14:24:27 +0000 (UTC) X-FDA: 79638751374.21.F53054A Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf26.hostedemail.com (Postfix) with ESMTP id BD55A140037 for ; Fri, 1 Jul 2022 14:24:26 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id i9-20020a05640242c900b004373cd1c4d5so1894208edc.2 for ; Fri, 01 Jul 2022 07:24:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=p+ekp6Bf711YmRVy7UlZIiEYvCGAcUihcnJb8tTtmyA=; b=mkSh1163luV3qxoZHS0Bk8vVP+BHGhIagKEyh4eZVHeswcwGF3268uhJLQLJZ0gSFe GXgi0jjMjXk5V+4vUPo95n7IJ8PbXouY5AihiAaC+x1cKiW6wLvmVmRnimBOQKxSc1hh T3Oy87SHL1yiBzALqFF8g9naQh/+nBIWBcgsyOwlW8wWtKIIb+Q0UbMOr22eq7mTgDHG eBEuABIZ9f9u9wa5ZQ5VKLXKD+bE90Ev6sQyGPGbiKkxauq+b9E+hkXZWmEkW00MFFOu 7LdC/zaLIIp7Tw9CgVQs3xopaG77kApUE4oUAdu8XDe+bmAQTUADvMiMqYHx0+Z0kM/g cUKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=p+ekp6Bf711YmRVy7UlZIiEYvCGAcUihcnJb8tTtmyA=; b=D240SfQFxdRL464vGD1/aJXy6BXAyHGGCHMA5+RLIMjcgCVMzk+FiSCk0wbsZevV7s 83DWYfoHbEBPRAetJRKz2rpGStLdyTcS0UuejfqKiFZCqJA60TS2Byn9JrKmuIgtcyDV TmgnCGqBshOMsvslMa61L8S1JP02SbELEH5k1e8J4iyu1ydJdqXF5u26So1kK9bvGHRL 9Gj0017wE2X3UMUX1Ck22Pl/4mug5RhE3OpJvCVN4Bu+ViyEsPEhCFBpzZbocd06CxMd PMM0yLR5JpjTtpmkzqlUWZJXp4+9T6HWML3CDjIo9pFI+ew9K4ejzlw0lg7JwBEc4RnI CiUw== X-Gm-Message-State: AJIora/rxTiMu5KXqPuH+2+HcnhJQ4GkSPdATW+pnC+TzZBQ3pEESJAl iy9touu3JyUG+/NKrohRl1YItutOeLE= X-Google-Smtp-Source: AGRyM1tXfndK7GJsdZI8E9P7ZV+hdnR7ngoGASYuwpqPPulNPdFiICHdMHM6lDwrtqj7fz1NF9PZtzd94pk= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:50f:b0:435:7996:e90f with SMTP id m15-20020a056402050f00b004357996e90fmr19229751edv.110.1656685465511; Fri, 01 Jul 2022 07:24:25 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:50 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-26-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 25/45] kmsan: add tests for KMSAN From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=mkSh1163; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 3mQO_YgYKCLgejgbcpemmejc.amkjglsv-kkitYai.mpe@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3mQO_YgYKCLgejgbcpemmejc.amkjglsv-kkitYai.mpe@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685466; a=rsa-sha256; cv=none; b=R3yUrl4ethpIZmDWZ9mpXzzh80OaCH+X/W4CECoBChQ/aaQ5jBewdlpCEEc4buH/7DhgNn VTgJ4ODIHep2wLKTANKMsqrHhAINXx2OO16rb2eWFEgYQJbOZQiOtiXRpLiRLXt3btxMlb t3hoefQz7cUcDzpwgxdWnyc1cM13bVY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685466; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p+ekp6Bf711YmRVy7UlZIiEYvCGAcUihcnJb8tTtmyA=; b=6v6C/KQUQNpFMLemHq7fhyTzijngD1+q8v97Uwr4s85BkgJ+Hf8N3CTR8pb3Q0XmDLOvVA FaHmmRwu5zEgCnOubv9uuYaHc1qnxQIROzj7o5gdvfP/gmFpRHUp/2HHIzXy3K2iEHBGVH XAwHlDpfCqfNJ3VZESCicmZSCMKpCDM= X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=mkSh1163; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 3mQO_YgYKCLgejgbcpemmejc.amkjglsv-kkitYai.mpe@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3mQO_YgYKCLgejgbcpemmejc.amkjglsv-kkitYai.mpe@flex--glider.bounces.google.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: BD55A140037 X-Stat-Signature: pcc6edecu7zmwxcuda5z33kd5if18f3n X-HE-Tag: 1656685466-975256 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The testing module triggers KMSAN warnings in different cases and checks that the errors are properly reported, using console probes to capture the tool's output. Signed-off-by: Alexander Potapenko --- v2: -- add memcpy tests v4: -- change sizeof(type) to sizeof(*ptr) -- add test expectations for CONFIG_KMSAN_CHECK_PARAM_RETVAL Link: https://linux-review.googlesource.com/id/I49c3f59014cc37fd13541c80beb0b75a75244650 --- lib/Kconfig.kmsan | 12 + mm/kmsan/Makefile | 4 + mm/kmsan/kmsan_test.c | 552 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 568 insertions(+) create mode 100644 mm/kmsan/kmsan_test.c diff --git a/lib/Kconfig.kmsan b/lib/Kconfig.kmsan index 8f768d4034e3c..f56ed7f7c7090 100644 --- a/lib/Kconfig.kmsan +++ b/lib/Kconfig.kmsan @@ -47,4 +47,16 @@ config KMSAN_CHECK_PARAM_RETVAL may potentially report errors in corner cases when non-instrumented functions call instrumented ones. +config KMSAN_KUNIT_TEST + tristate "KMSAN integration test suite" if !KUNIT_ALL_TESTS + default KUNIT_ALL_TESTS + depends on TRACEPOINTS && KUNIT + help + Test suite for KMSAN, testing various error detection scenarios, + and checking that reports are correctly output to console. + + Say Y here if you want the test to be built into the kernel and run + during boot; say M if you want the test to build as a module; say N + if you are unsure. + endif diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile index 401acb1a491ce..98eab2856626f 100644 --- a/mm/kmsan/Makefile +++ b/mm/kmsan/Makefile @@ -22,3 +22,7 @@ CFLAGS_init.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_instrumentation.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_report.o := $(CC_FLAGS_KMSAN_RUNTIME) CFLAGS_shadow.o := $(CC_FLAGS_KMSAN_RUNTIME) + +obj-$(CONFIG_KMSAN_KUNIT_TEST) += kmsan_test.o +KMSAN_SANITIZE_kmsan_test.o := y +CFLAGS_kmsan_test.o += $(call cc-disable-warning, uninitialized) diff --git a/mm/kmsan/kmsan_test.c b/mm/kmsan/kmsan_test.c new file mode 100644 index 0000000000000..1b8da71ae0d4f --- /dev/null +++ b/mm/kmsan/kmsan_test.c @@ -0,0 +1,552 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test cases for KMSAN. + * For each test case checks the presence (or absence) of generated reports. + * Relies on 'console' tracepoint to capture reports as they appear in the + * kernel log. + * + * Copyright (C) 2021-2022, Google LLC. + * Author: Alexander Potapenko + * + */ + +#include +#include "kmsan.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static DEFINE_PER_CPU(int, per_cpu_var); + +/* Report as observed from console. */ +static struct { + spinlock_t lock; + bool available; + bool ignore; /* Stop console output collection. */ + char header[256]; +} observed = { + .lock = __SPIN_LOCK_UNLOCKED(observed.lock), +}; + +/* Probe for console output: obtains observed lines of interest. */ +static void probe_console(void *ignore, const char *buf, size_t len) +{ + unsigned long flags; + + if (observed.ignore) + return; + spin_lock_irqsave(&observed.lock, flags); + + if (strnstr(buf, "BUG: KMSAN: ", len)) { + /* + * KMSAN report and related to the test. + * + * The provided @buf is not NUL-terminated; copy no more than + * @len bytes and let strscpy() add the missing NUL-terminator. + */ + strscpy(observed.header, buf, + min(len + 1, sizeof(observed.header))); + WRITE_ONCE(observed.available, true); + observed.ignore = true; + } + spin_unlock_irqrestore(&observed.lock, flags); +} + +/* Check if a report related to the test exists. */ +static bool report_available(void) +{ + return READ_ONCE(observed.available); +} + +/* Information we expect in a report. */ +struct expect_report { + const char *error_type; /* Error type. */ + /* + * Kernel symbol from the error header, or NULL if no report is + * expected. + */ + const char *symbol; +}; + +/* Check observed report matches information in @r. */ +static bool report_matches(const struct expect_report *r) +{ + typeof(observed.header) expected_header; + unsigned long flags; + bool ret = false; + const char *end; + char *cur; + + /* Doubled-checked locking. */ + if (!report_available() || !r->symbol) + return (!report_available() && !r->symbol); + + /* Generate expected report contents. */ + + /* Title */ + cur = expected_header; + end = &expected_header[sizeof(expected_header) - 1]; + + cur += scnprintf(cur, end - cur, "BUG: KMSAN: %s", r->error_type); + + scnprintf(cur, end - cur, " in %s", r->symbol); + /* The exact offset won't match, remove it; also strip module name. */ + cur = strchr(expected_header, '+'); + if (cur) + *cur = '\0'; + + spin_lock_irqsave(&observed.lock, flags); + if (!report_available()) + goto out; /* A new report is being captured. */ + + /* Finally match expected output to what we actually observed. */ + ret = strstr(observed.header, expected_header); +out: + spin_unlock_irqrestore(&observed.lock, flags); + + return ret; +} + +/* ===== Test cases ===== */ + +/* Prevent replacing branch with select in LLVM. */ +static noinline void check_true(char *arg) +{ + pr_info("%s is true\n", arg); +} + +static noinline void check_false(char *arg) +{ + pr_info("%s is false\n", arg); +} + +#define USE(x) \ + do { \ + if (x) \ + check_true(#x); \ + else \ + check_false(#x); \ + } while (0) + +#define EXPECTATION_ETYPE_FN(e, reason, fn) \ + struct expect_report e = { \ + .error_type = reason, \ + .symbol = fn, \ + } + +#define EXPECTATION_NO_REPORT(e) EXPECTATION_ETYPE_FN(e, NULL, NULL) +#define EXPECTATION_UNINIT_VALUE_FN(e, fn) \ + EXPECTATION_ETYPE_FN(e, "uninit-value", fn) +#define EXPECTATION_UNINIT_VALUE(e) EXPECTATION_UNINIT_VALUE_FN(e, __func__) +#define EXPECTATION_USE_AFTER_FREE(e) \ + EXPECTATION_ETYPE_FN(e, "use-after-free", __func__) + +/* Test case: ensure that kmalloc() returns uninitialized memory. */ +static void test_uninit_kmalloc(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE(expect); + int *ptr; + + kunit_info(test, "uninitialized kmalloc test (UMR report)\n"); + ptr = kmalloc(sizeof(*ptr), GFP_KERNEL); + USE(*ptr); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that kmalloc'ed memory becomes initialized after memset(). + */ +static void test_init_kmalloc(struct kunit *test) +{ + EXPECTATION_NO_REPORT(expect); + int *ptr; + + kunit_info(test, "initialized kmalloc test (no reports)\n"); + ptr = kmalloc(sizeof(*ptr), GFP_KERNEL); + memset(ptr, 0, sizeof(*ptr)); + USE(*ptr); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* Test case: ensure that kzalloc() returns initialized memory. */ +static void test_init_kzalloc(struct kunit *test) +{ + EXPECTATION_NO_REPORT(expect); + int *ptr; + + kunit_info(test, "initialized kzalloc test (no reports)\n"); + ptr = kzalloc(sizeof(*ptr), GFP_KERNEL); + USE(*ptr); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* Test case: ensure that local variables are uninitialized by default. */ +static void test_uninit_stack_var(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE(expect); + volatile int cond; + + kunit_info(test, "uninitialized stack variable (UMR report)\n"); + USE(cond); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* Test case: ensure that local variables with initializers are initialized. */ +static void test_init_stack_var(struct kunit *test) +{ + EXPECTATION_NO_REPORT(expect); + volatile int cond = 1; + + kunit_info(test, "initialized stack variable (no reports)\n"); + USE(cond); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +static noinline void two_param_fn_2(int arg1, int arg2) +{ + USE(arg1); + USE(arg2); +} + +static noinline void one_param_fn(int arg) +{ + two_param_fn_2(arg, arg); + USE(arg); +} + +static noinline void two_param_fn(int arg1, int arg2) +{ + int init = 0; + + one_param_fn(init); + USE(arg1); + USE(arg2); +} + +static void test_params(struct kunit *test) +{ +#ifdef CONFIG_KMSAN_CHECK_PARAM_RETVAL + /* + * With eager param/retval checking enabled, KMSAN will report an error + * before the call to two_param_fn(). + */ + EXPECTATION_UNINIT_VALUE_FN(expect, "test_params"); +#else + EXPECTATION_UNINIT_VALUE_FN(expect, "two_param_fn"); +#endif + volatile int uninit, init = 1; + + kunit_info(test, + "uninit passed through a function parameter (UMR report)\n"); + two_param_fn(uninit, init); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +static int signed_sum3(int a, int b, int c) +{ + return a + b + c; +} + +/* + * Test case: ensure that uninitialized values are tracked through function + * arguments. + */ +static void test_uninit_multiple_params(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE(expect); + volatile char b = 3, c; + volatile int a; + + kunit_info(test, "uninitialized local passed to fn (UMR report)\n"); + USE(signed_sum3(a, b, c)); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* Helper function to make an array uninitialized. */ +static noinline void do_uninit_local_array(char *array, int start, int stop) +{ + volatile char uninit; + int i; + + for (i = start; i < stop; i++) + array[i] = uninit; +} + +/* + * Test case: ensure kmsan_check_memory() reports an error when checking + * uninitialized memory. + */ +static void test_uninit_kmsan_check_memory(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE_FN(expect, "test_uninit_kmsan_check_memory"); + volatile char local_array[8]; + + kunit_info( + test, + "kmsan_check_memory() called on uninit local (UMR report)\n"); + do_uninit_local_array((char *)local_array, 5, 7); + + kmsan_check_memory((char *)local_array, 8); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: check that a virtual memory range created with vmap() from + * initialized pages is still considered as initialized. + */ +static void test_init_kmsan_vmap_vunmap(struct kunit *test) +{ + EXPECTATION_NO_REPORT(expect); + const int npages = 2; + struct page **pages; + void *vbuf; + int i; + + kunit_info(test, "pages initialized via vmap (no reports)\n"); + + pages = kmalloc_array(npages, sizeof(*pages), GFP_KERNEL); + for (i = 0; i < npages; i++) + pages[i] = alloc_page(GFP_KERNEL); + vbuf = vmap(pages, npages, VM_MAP, PAGE_KERNEL); + memset(vbuf, 0xfe, npages * PAGE_SIZE); + for (i = 0; i < npages; i++) + kmsan_check_memory(page_address(pages[i]), PAGE_SIZE); + + if (vbuf) + vunmap(vbuf); + for (i = 0; i < npages; i++) + if (pages[i]) + __free_page(pages[i]); + kfree(pages); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that memset() can initialize a buffer allocated via + * vmalloc(). + */ +static void test_init_vmalloc(struct kunit *test) +{ + EXPECTATION_NO_REPORT(expect); + int npages = 8, i; + char *buf; + + kunit_info(test, "vmalloc buffer can be initialized (no reports)\n"); + buf = vmalloc(PAGE_SIZE * npages); + buf[0] = 1; + memset(buf, 0xfe, PAGE_SIZE * npages); + USE(buf[0]); + for (i = 0; i < npages; i++) + kmsan_check_memory(&buf[PAGE_SIZE * i], PAGE_SIZE); + vfree(buf); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* Test case: ensure that use-after-free reporting works. */ +static void test_uaf(struct kunit *test) +{ + EXPECTATION_USE_AFTER_FREE(expect); + volatile int value; + volatile int *var; + + kunit_info(test, "use-after-free in kmalloc-ed buffer (UMR report)\n"); + var = kmalloc(80, GFP_KERNEL); + var[3] = 0xfeedface; + kfree((int *)var); + /* Copy the invalid value before checking it. */ + value = var[3]; + USE(value); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that uninitialized values are propagated through per-CPU + * memory. + */ +static void test_percpu_propagate(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE(expect); + volatile int uninit, check; + + kunit_info(test, + "uninit local stored to per_cpu memory (UMR report)\n"); + + this_cpu_write(per_cpu_var, uninit); + check = this_cpu_read(per_cpu_var); + USE(check); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that passing uninitialized values to printk() leads to an + * error report. + */ +static void test_printk(struct kunit *test) +{ +#ifdef CONFIG_KMSAN_CHECK_PARAM_RETVAL + /* + * With eager param/retval checking enabled, KMSAN will report an error + * before the call to pr_info(). + */ + EXPECTATION_UNINIT_VALUE_FN(expect, "test_printk"); +#else + EXPECTATION_UNINIT_VALUE_FN(expect, "number"); +#endif + volatile int uninit; + + kunit_info(test, "uninit local passed to pr_info() (UMR report)\n"); + pr_info("%px contains %d\n", &uninit, uninit); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that memcpy() correctly copies uninitialized values between + * aligned `src` and `dst`. + */ +static void test_memcpy_aligned_to_aligned(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE_FN(expect, "test_memcpy_aligned_to_aligned"); + volatile int uninit_src; + volatile int dst = 0; + + kunit_info(test, "memcpy()ing aligned uninit src to aligned dst (UMR report)\n"); + memcpy((void *)&dst, (void *)&uninit_src, sizeof(uninit_src)); + kmsan_check_memory((void *)&dst, sizeof(dst)); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that memcpy() correctly copies uninitialized values between + * aligned `src` and unaligned `dst`. + * + * Copying aligned 4-byte value to an unaligned one leads to touching two + * aligned 4-byte values. This test case checks that KMSAN correctly reports an + * error on the first of the two values. + */ +static void test_memcpy_aligned_to_unaligned(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE_FN(expect, "test_memcpy_aligned_to_unaligned"); + volatile int uninit_src; + volatile char dst[8] = {0}; + + kunit_info(test, "memcpy()ing aligned uninit src to unaligned dst (UMR report)\n"); + memcpy((void *)&dst[1], (void *)&uninit_src, sizeof(uninit_src)); + kmsan_check_memory((void *)dst, 4); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +/* + * Test case: ensure that memcpy() correctly copies uninitialized values between + * aligned `src` and unaligned `dst`. + * + * Copying aligned 4-byte value to an unaligned one leads to touching two + * aligned 4-byte values. This test case checks that KMSAN correctly reports an + * error on the second of the two values. + */ +static void test_memcpy_aligned_to_unaligned2(struct kunit *test) +{ + EXPECTATION_UNINIT_VALUE_FN(expect, "test_memcpy_aligned_to_unaligned2"); + volatile int uninit_src; + volatile char dst[8] = {0}; + + kunit_info(test, "memcpy()ing aligned uninit src to unaligned dst - part 2 (UMR report)\n"); + memcpy((void *)&dst[1], (void *)&uninit_src, sizeof(uninit_src)); + kmsan_check_memory((void *)&dst[4], sizeof(uninit_src)); + KUNIT_EXPECT_TRUE(test, report_matches(&expect)); +} + +static struct kunit_case kmsan_test_cases[] = { + KUNIT_CASE(test_uninit_kmalloc), + KUNIT_CASE(test_init_kmalloc), + KUNIT_CASE(test_init_kzalloc), + KUNIT_CASE(test_uninit_stack_var), + KUNIT_CASE(test_init_stack_var), + KUNIT_CASE(test_params), + KUNIT_CASE(test_uninit_multiple_params), + KUNIT_CASE(test_uninit_kmsan_check_memory), + KUNIT_CASE(test_init_kmsan_vmap_vunmap), + KUNIT_CASE(test_init_vmalloc), + KUNIT_CASE(test_uaf), + KUNIT_CASE(test_percpu_propagate), + KUNIT_CASE(test_printk), + KUNIT_CASE(test_memcpy_aligned_to_aligned), + KUNIT_CASE(test_memcpy_aligned_to_unaligned), + KUNIT_CASE(test_memcpy_aligned_to_unaligned2), + {}, +}; + +/* ===== End test cases ===== */ + +static int test_init(struct kunit *test) +{ + unsigned long flags; + + spin_lock_irqsave(&observed.lock, flags); + observed.header[0] = '\0'; + observed.ignore = false; + observed.available = false; + spin_unlock_irqrestore(&observed.lock, flags); + + return 0; +} + +static void test_exit(struct kunit *test) +{ +} + +static struct kunit_suite kmsan_test_suite = { + .name = "kmsan", + .test_cases = kmsan_test_cases, + .init = test_init, + .exit = test_exit, +}; +static struct kunit_suite *kmsan_test_suites[] = { &kmsan_test_suite, NULL }; + +static void register_tracepoints(struct tracepoint *tp, void *ignore) +{ + check_trace_callback_type_console(probe_console); + if (!strcmp(tp->name, "console")) + WARN_ON(tracepoint_probe_register(tp, probe_console, NULL)); +} + +static void unregister_tracepoints(struct tracepoint *tp, void *ignore) +{ + if (!strcmp(tp->name, "console")) + tracepoint_probe_unregister(tp, probe_console, NULL); +} + +/* + * We only want to do tracepoints setup and teardown once, therefore we have to + * customize the init and exit functions and cannot rely on kunit_test_suite(). + */ +static int __init kmsan_test_init(void) +{ + /* + * Because we want to be able to build the test as a module, we need to + * iterate through all known tracepoints, since the static registration + * won't work here. + */ + for_each_kernel_tracepoint(register_tracepoints, NULL); + return __kunit_test_suites_init(kmsan_test_suites); +} + +static void kmsan_test_exit(void) +{ + __kunit_test_suites_exit(kmsan_test_suites); + for_each_kernel_tracepoint(unregister_tracepoints, NULL); + tracepoint_synchronize_unregister(); +} + +late_initcall_sync(kmsan_test_init); +module_exit(kmsan_test_exit); + +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Alexander Potapenko "); From patchwork Fri Jul 1 14:22:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85D3DC433EF for ; Fri, 1 Jul 2022 14:24:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 166C76B0071; Fri, 1 Jul 2022 10:24:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1171C6B007D; Fri, 1 Jul 2022 10:24:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EABE56B0082; Fri, 1 Jul 2022 10:24:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DBF2B6B0071 for ; Fri, 1 Jul 2022 10:24:29 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B6CBD6D5 for ; Fri, 1 Jul 2022 14:24:29 +0000 (UTC) X-FDA: 79638751458.09.89054E4 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf16.hostedemail.com (Postfix) with ESMTP id 46282180009 for ; Fri, 1 Jul 2022 14:24:29 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id hd35-20020a17090796a300b0072a707cfac4so840869ejc.8 for ; Fri, 01 Jul 2022 07:24:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=BanD6+8XMfAM/imhDA9kzvs2Y65ZtmV55Antiu9BWiI=; b=mRTIF6L84JAlg2BdRm6GzX0JAROwRk/sEAnHEyfcCrrVzpgYQDP4s07FuADnqgr6wB ksMNaoreBTcwNzLVqW2CLQ5LB7q3aAIulZLYB9nuQCnWIkiiRq/Q6VQMDSS0zlOVULIz Ix5FQ2wgYo7L210aM/fKxFuU2tFPKcIZsO4Bd7Rml1N7o3/Qm7KUm5NB65g2Iem/FeNl 60bw+n3Gge8W8wvYFmfBqE0m7x4Rd1oXGrEp9Fu06BG/VFY3yN6zDkc7HdcrNddl9RFZ bKRR73XyhRvmnmBNBCirP4336bKCGzJJU7II1NZsV//Tb+lcEnJtFA9n/GAdR4WOxrCR uhTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BanD6+8XMfAM/imhDA9kzvs2Y65ZtmV55Antiu9BWiI=; b=Yk67brTUYCMKg9DM1d8Ps852GXqzM47EVY37VmuFKmau8I6zcUdUaBu7IQaJ4FN3ru 5C7oIDJ1gQwrPnRuKiXvxOB1gKcBsvAwsbPF7EF7+8O73YpvZIEyKI1+42K+pQGa8DI0 YMZ55d5UEUmn33THPBn28DuXGaXAp4QGUK6+AvhkM2/JFOmFbqZBCxQgqnX243NRo1DV 8YxNRfuZ52yx1Yixc+kwOAzfK6cybvj+p21dlZABzpUb/Q+lSlUqUCzK9lS8iTPthhDx 8h566RpI4I19JCGFwfuRfW4BZoe6/BfAihmLUHJlNOnW8YIWI+m8hRu09q52EQbUik9N vp5w== X-Gm-Message-State: AJIora8LctF+SSwXiqsXfhSOc2F6uBo2VW2xmaEc9PVa3Wy1H2pG0qDC 711dWrUC++spxwF6ky6vp22Rz7y0Tfs= X-Google-Smtp-Source: AGRyM1siOEbFYB7M2t/Uxp+hgW7r6Fq8QtqzrawcUEK1N9TlhUHD1diimHPgmIHCv/uA9yMN9zjsG0VRP+I= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a50:fe0c:0:b0:435:510a:9f1f with SMTP id f12-20020a50fe0c000000b00435510a9f1fmr19625057edt.297.1656685468052; Fri, 01 Jul 2022 07:24:28 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:51 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-27-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 26/45] kmsan: disable strscpy() optimization under KMSAN From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685469; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BanD6+8XMfAM/imhDA9kzvs2Y65ZtmV55Antiu9BWiI=; b=sNS3F9Fa05oa71wjr2AqCaq5HyMtUhikUNGD4yiVCsVg1P4qVZ6hfZQdDiXJXMGgphqVZB Ttac+AFqfLkQ2QCYPh7mhgue9sm0kGtHb2Tc5kV9hOhkLS778OfqLYBhW0sxjKeHvEd7ws E03aOryGYwKUrK17REfWE7S9ROnUY3k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685469; a=rsa-sha256; cv=none; b=u0SHMJ7bPUG6ciC8I2s49x+SAc7HW1zk83hmnanZROSUhBSkYMhGN/NoAjwOTiKWb+vCKZ yh/0eoYyr54wH8NozTApYz4n6WzaXXL34YIRdPUmf6iKzckJSyfgj3UQY7mAFHICqiLX5D LdVzzqTsJhZgpt5rS66Oabx4hoGi8Co= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=mRTIF6L8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3nAO_YgYKCLshmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3nAO_YgYKCLshmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com X-Stat-Signature: 9umt614uoj5qc4kjgzq9ojmj9otk4mbe X-Rspam-User: Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=mRTIF6L8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3nAO_YgYKCLshmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3nAO_YgYKCLshmjefshpphmf.dpnmjovy-nnlwbdl.psh@flex--glider.bounces.google.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 46282180009 X-HE-Tag: 1656685469-596652 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Disable the efficient 8-byte reading under KMSAN to avoid false positives. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Iffd8336965e88fce915db2e6a9d6524422975f69 --- lib/string.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/string.c b/lib/string.c index 6f334420f6871..3371d26a0e390 100644 --- a/lib/string.c +++ b/lib/string.c @@ -197,6 +197,14 @@ ssize_t strscpy(char *dest, const char *src, size_t count) max = 0; #endif + /* + * read_word_at_a_time() below may read uninitialized bytes after the + * trailing zero and use them in comparisons. Disable this optimization + * under KMSAN to prevent false positive reports. + */ + if (IS_ENABLED(CONFIG_KMSAN)) + max = 0; + while (max >= sizeof(unsigned long)) { unsigned long c, data; From patchwork Fri Jul 1 14:22:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B729C43334 for ; Fri, 1 Jul 2022 14:24:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 18D706B0078; Fri, 1 Jul 2022 10:24:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 13E5F6B007D; Fri, 1 Jul 2022 10:24:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFA626B0082; Fri, 1 Jul 2022 10:24:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DFBA96B0078 for ; Fri, 1 Jul 2022 10:24:32 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B5E1060EE1 for ; Fri, 1 Jul 2022 14:24:32 +0000 (UTC) X-FDA: 79638751584.27.CD12A04 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf24.hostedemail.com (Postfix) with ESMTP id 6D798180031 for ; Fri, 1 Jul 2022 14:24:32 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id w5-20020a056402268500b0043980311a5fso1407529edd.3 for ; Fri, 01 Jul 2022 07:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MSiZEh5HPppja7KB08TDPcjfgNFKAqe3wmJe9MrM4uA=; b=OazEMcVKP9raBwVMDG4T/fiEmrGUA04GYu29RTnWJdtBSg6FoU0yqzIOuDULnzsRUn 8PPCp0TkvH4G1NW5jz9sCSb1x4fervgZrRnUh4zD2Ge4kApkvUdhcWf1E+rjd9R3m8mb ZDmOcMZLAfOiPi3XgOETstZSQsP+r1Z7gMxbF4T882dKLQUuV8WXai0XVqUAW+BMvT0i DBHkIfJC8/ImBHdqym7+g0HPfvfoUGin1pPMSYwOBrt3JhW9x9OPidVC5kjlcGbHn1El ux2kJtoEvX/mjg7Updri4OqLLdpPEY/gN0nO17vVqG/JRZUyj5twFDs8SbiCu45b6WPz hMwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MSiZEh5HPppja7KB08TDPcjfgNFKAqe3wmJe9MrM4uA=; b=ETBTSxTnRildLKKr8QsQ1N5migLC+HCII3TOs7kH4e2AMYmVybxe9YaGAXMoyUw9Dl MHjC6QfgDPEOTblop5hySt8E+kqM8SfwFX9YT9Id1XpstfRWyKgW76TX+O3d0bmazqmw ZxmoAUE6tSBRSmgD9CwrraxSWaNs0FjBjYrbboj75w7IDKIBw8tX6YiwNFTxTyWIeaAy sa0xyii50XHcWuDVNTlfd0ZbVdwBpWO6aoscRIGI0JPXAi0kE+FdU+pHQPNiLAyqpv6U gfKb5GNMWcXCNHo+UOFwTMG9F+ezY6cX9eyRICAe+RtrdKQ10tjhcSJprwojfdIA+fcP Fqgw== X-Gm-Message-State: AJIora/8XchUnBk8y7Thjl7bZj5bWaqJFKBy4peisWFwS4r/FI9RJ8cm v22lxjg+sJCiaPX5SZ5FdSeBOs1CF8s= X-Google-Smtp-Source: AGRyM1t9McEyOdYfX8BH5gIjAASU9uHW8kvFWE4Fy4sHSu26c83KCf+RwktbhP60YuMs6NU1rY2DohKt3V4= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:906:6c82:b0:709:f868:97f6 with SMTP id s2-20020a1709066c8200b00709f86897f6mr14592696ejr.555.1656685471024; Fri, 01 Jul 2022 07:24:31 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:52 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-28-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 27/45] crypto: kmsan: disable accelerated configs under KMSAN From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685472; a=rsa-sha256; cv=none; b=Opn89st5KCSKY3QpZbmLOyFORB/Jz8I8sYCret23U1T1W8gYa9nBaWGFmaJavfU9G5VKhh h2/wn/APE6tJ6aqVAfdyB5QXZT3o88whMoG6KpJAgeITqYOExUJprsW8gEDVcQmEWFh4v+ fDrPn+PumF+SG3XfrHYVlqJYJPmbDI8= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OazEMcVK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3nwO_YgYKCL4kpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3nwO_YgYKCL4kpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685472; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MSiZEh5HPppja7KB08TDPcjfgNFKAqe3wmJe9MrM4uA=; b=IMugTuZ/fsTTCmuvzz8y849NfQz1DsKido7lyJ/+Ww+9lNbVWwLowtxXPmoy7liuYvDEjF DX2TSMv+JYC+CPjPbdFWgL6FuyplwacgTa/jXp7XjX8NHJysPFN7jZqjpYTzEAyXnuplIY 04/WnrEunEbk59hMD+VR9EAn4Sc+g/g= X-Stat-Signature: 1mn7skja59aiowcdw841oorj69dk1x3i X-Rspamd-Queue-Id: 6D798180031 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=OazEMcVK; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3nwO_YgYKCL4kpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3nwO_YgYKCL4kpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1656685472-180096 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN is unable to understand when initialized values come from assembly. Disable accelerated configs in KMSAN builds to prevent false positive reports. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Idb2334bf3a1b68b31b399709baefaa763038cc50 --- crypto/Kconfig | 30 ++++++++++++++++++++++++++++++ drivers/net/Kconfig | 1 + 2 files changed, 31 insertions(+) diff --git a/crypto/Kconfig b/crypto/Kconfig index 1d44893a997ba..7ddda6072ef35 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -298,6 +298,7 @@ config CRYPTO_CURVE25519 config CRYPTO_CURVE25519_X86 tristate "x86_64 accelerated Curve25519 scalar multiplication library" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_LIB_CURVE25519_GENERIC select CRYPTO_ARCH_HAVE_LIB_CURVE25519 @@ -346,11 +347,13 @@ config CRYPTO_AEGIS128 config CRYPTO_AEGIS128_SIMD bool "Support SIMD acceleration for AEGIS-128" depends on CRYPTO_AEGIS128 && ((ARM || ARM64) && KERNEL_MODE_NEON) + depends on !KMSAN # avoid false positives from assembly default y config CRYPTO_AEGIS128_AESNI_SSE2 tristate "AEGIS-128 AEAD algorithm (x86_64 AESNI+SSE2 implementation)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_AEAD select CRYPTO_SIMD help @@ -487,6 +490,7 @@ config CRYPTO_NHPOLY1305 config CRYPTO_NHPOLY1305_SSE2 tristate "NHPoly1305 hash function (x86_64 SSE2 implementation)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_NHPOLY1305 help SSE2 optimized implementation of the hash function used by the @@ -495,6 +499,7 @@ config CRYPTO_NHPOLY1305_SSE2 config CRYPTO_NHPOLY1305_AVX2 tristate "NHPoly1305 hash function (x86_64 AVX2 implementation)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_NHPOLY1305 help AVX2 optimized implementation of the hash function used by the @@ -608,6 +613,7 @@ config CRYPTO_CRC32C config CRYPTO_CRC32C_INTEL tristate "CRC32c INTEL hardware acceleration" depends on X86 + depends on !KMSAN # avoid false positives from assembly select CRYPTO_HASH help In Intel processor with SSE4.2 supported, the processor will @@ -648,6 +654,7 @@ config CRYPTO_CRC32 config CRYPTO_CRC32_PCLMUL tristate "CRC32 PCLMULQDQ hardware acceleration" depends on X86 + depends on !KMSAN # avoid false positives from assembly select CRYPTO_HASH select CRC32 help @@ -713,6 +720,7 @@ config CRYPTO_BLAKE2S config CRYPTO_BLAKE2S_X86 tristate "BLAKE2s digest algorithm (x86 accelerated version)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_LIB_BLAKE2S_GENERIC select CRYPTO_ARCH_HAVE_LIB_BLAKE2S @@ -727,6 +735,7 @@ config CRYPTO_CRCT10DIF config CRYPTO_CRCT10DIF_PCLMUL tristate "CRCT10DIF PCLMULQDQ hardware acceleration" depends on X86 && 64BIT && CRC_T10DIF + depends on !KMSAN # avoid false positives from assembly select CRYPTO_HASH help For x86_64 processors with SSE4.2 and PCLMULQDQ supported, @@ -779,6 +788,7 @@ config CRYPTO_POLY1305 config CRYPTO_POLY1305_X86_64 tristate "Poly1305 authenticator algorithm (x86_64/SSE2/AVX2)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_LIB_POLY1305_GENERIC select CRYPTO_ARCH_HAVE_LIB_POLY1305 help @@ -867,6 +877,7 @@ config CRYPTO_SHA1 config CRYPTO_SHA1_SSSE3 tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SHA1 select CRYPTO_HASH help @@ -878,6 +889,7 @@ config CRYPTO_SHA1_SSSE3 config CRYPTO_SHA256_SSSE3 tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SHA256 select CRYPTO_HASH help @@ -890,6 +902,7 @@ config CRYPTO_SHA256_SSSE3 config CRYPTO_SHA512_SSSE3 tristate "SHA512 digest algorithm (SSSE3/AVX/AVX2)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SHA512 select CRYPTO_HASH help @@ -1065,6 +1078,7 @@ config CRYPTO_WP512 config CRYPTO_GHASH_CLMUL_NI_INTEL tristate "GHASH hash function (CLMUL-NI accelerated)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_CRYPTD help This is the x86_64 CLMUL-NI accelerated implementation of @@ -1115,6 +1129,7 @@ config CRYPTO_AES_TI config CRYPTO_AES_NI_INTEL tristate "AES cipher algorithms (AES-NI)" depends on X86 + depends on !KMSAN # avoid false positives from assembly select CRYPTO_AEAD select CRYPTO_LIB_AES select CRYPTO_ALGAPI @@ -1239,6 +1254,7 @@ config CRYPTO_BLOWFISH_COMMON config CRYPTO_BLOWFISH_X86_64 tristate "Blowfish cipher algorithm (x86_64)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_BLOWFISH_COMMON imply CRYPTO_CTR @@ -1269,6 +1285,7 @@ config CRYPTO_CAMELLIA config CRYPTO_CAMELLIA_X86_64 tristate "Camellia cipher algorithm (x86_64)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER imply CRYPTO_CTR help @@ -1285,6 +1302,7 @@ config CRYPTO_CAMELLIA_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_CAMELLIA_X86_64 select CRYPTO_SIMD @@ -1303,6 +1321,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help Camellia cipher algorithm module (x86_64/AES-NI/AVX2). @@ -1348,6 +1367,7 @@ config CRYPTO_CAST5 config CRYPTO_CAST5_AVX_X86_64 tristate "CAST5 (CAST-128) cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_CAST5 select CRYPTO_CAST_COMMON @@ -1371,6 +1391,7 @@ config CRYPTO_CAST6 config CRYPTO_CAST6_AVX_X86_64 tristate "CAST6 (CAST-256) cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_CAST6 select CRYPTO_CAST_COMMON @@ -1404,6 +1425,7 @@ config CRYPTO_DES_SPARC64 config CRYPTO_DES3_EDE_X86_64 tristate "Triple DES EDE cipher algorithm (x86-64)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_LIB_DES imply CRYPTO_CTR @@ -1461,6 +1483,7 @@ config CRYPTO_CHACHA20 config CRYPTO_CHACHA20_X86_64 tristate "ChaCha stream cipher algorithms (x86_64/SSSE3/AVX2/AVX-512VL)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_LIB_CHACHA_GENERIC select CRYPTO_ARCH_HAVE_LIB_CHACHA @@ -1504,6 +1527,7 @@ config CRYPTO_SERPENT config CRYPTO_SERPENT_SSE2_X86_64 tristate "Serpent cipher algorithm (x86_64/SSE2)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_SERPENT select CRYPTO_SIMD @@ -1523,6 +1547,7 @@ config CRYPTO_SERPENT_SSE2_X86_64 config CRYPTO_SERPENT_SSE2_586 tristate "Serpent cipher algorithm (i586/SSE2)" depends on X86 && !64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_SERPENT select CRYPTO_SIMD @@ -1542,6 +1567,7 @@ config CRYPTO_SERPENT_SSE2_586 config CRYPTO_SERPENT_AVX_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_SERPENT select CRYPTO_SIMD @@ -1562,6 +1588,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen. @@ -1706,6 +1733,7 @@ config CRYPTO_TWOFISH_586 config CRYPTO_TWOFISH_X86_64 tristate "Twofish cipher algorithm (x86_64)" depends on (X86 || UML_X86) && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_ALGAPI select CRYPTO_TWOFISH_COMMON imply CRYPTO_CTR @@ -1723,6 +1751,7 @@ config CRYPTO_TWOFISH_X86_64 config CRYPTO_TWOFISH_X86_64_3WAY tristate "Twofish cipher algorithm (x86_64, 3-way parallel)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_TWOFISH_COMMON select CRYPTO_TWOFISH_X86_64 @@ -1743,6 +1772,7 @@ config CRYPTO_TWOFISH_X86_64_3WAY config CRYPTO_TWOFISH_AVX_X86_64 tristate "Twofish cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN # avoid false positives from assembly select CRYPTO_SKCIPHER select CRYPTO_SIMD select CRYPTO_TWOFISH_COMMON diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig index b2a4f998c180e..fed89b6981759 100644 --- a/drivers/net/Kconfig +++ b/drivers/net/Kconfig @@ -76,6 +76,7 @@ config WIREGUARD tristate "WireGuard secure network tunnel" depends on NET && INET depends on IPV6 || !IPV6 + depends on !KMSAN # KMSAN doesn't support the crypto configs below select NET_UDP_TUNNEL select DST_CACHE select CRYPTO From patchwork Fri Jul 1 14:22:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30E85C43334 for ; Fri, 1 Jul 2022 14:24:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA0126B007D; Fri, 1 Jul 2022 10:24:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B28876B0082; Fri, 1 Jul 2022 10:24:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C99B6B0083; Fri, 1 Jul 2022 10:24:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 88D5B6B007D for ; Fri, 1 Jul 2022 10:24:35 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 6A9F6603A7 for ; Fri, 1 Jul 2022 14:24:35 +0000 (UTC) X-FDA: 79638751710.02.562B3DC Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf22.hostedemail.com (Postfix) with ESMTP id 21204C0009 for ; Fri, 1 Jul 2022 14:24:34 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id z13-20020a056402274d00b004357fcdd51fso1870463edd.17 for ; Fri, 01 Jul 2022 07:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tSPSuwfCBJYINsR4GH3IgbP5patOgmYnzcC5xbF5jrk=; b=fBpJ5slqI8D0FOXlutf00Q01e2/qkKGZMnWH0QFqV3SMTrnSjxCWOU4Pqh88ufv3NQ IfRJI3UQIhuhwdip4JvVQMQ0GmgbHCP23qV+TLI5z5xixX0f7GhYWJ0KQ1im+NF7L5qc Pkq7nAOUgfMsrQqnSYEqcjeYwLR5EOBvpP0TGKGBvfsM2Tv2Ix3OAW1CAWsOCQoKqQ9c em6kSS3mBnEDJKreuud9sUY6SkRHrCFbsdOQdD/6Y3m3J12I9+Y2wd+Aam71SXnA0koY C8Wt7O96xHpDNeJE2AG92PtC1SizglmTVKICFmvWSO765naM8cDe6vlSu0NadfhbBuA2 mElQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tSPSuwfCBJYINsR4GH3IgbP5patOgmYnzcC5xbF5jrk=; b=YqdQQc85lrk4DSDJFFfNPn1HTF71QsVnI/rHf92xfLxd9KsyNImfuwAoVhxXt6R8nL qwDobsVCAeGtzrcm/Iz8chRC6X5kRQwRSAuw9UNg9tlR7AAzR9GHi7tR1yDsKP193Kwv C8Oo2HBXz355U/B31Rgf1GX9eI0BMMT2mpnO1IydpEbkRHWmLL2+vOfTzG9bcqONtKhB FyZtDgyaQ2aOn8P3u4jUAygDjBmx95Pe1gSB/1G8hpnvsZxEwysBr9cpR07KcxPYvb4N o1lJZkhudRaAXfIcvZqzXYoXG/vF/amb6/EAuLZXLBkZ0LRHUtWGncdnSsGIvGcxQ9S+ CfeQ== X-Gm-Message-State: AJIora8HXzQFJ7n0jVKdkmsWirxUgic30ZmpZY1gAEqY9x0OPkRSMKew FU6+hS9ohh2bkmMv2GViF//ZmgCNdZU= X-Google-Smtp-Source: AGRyM1vbbcq8j11xYn5VJA0O5txvAR43cAqjEC618P5YqGVKcV6s6WBbHTsxKY+3oiObUtk4x14dSKdPgts= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:1f15:b0:435:8a5a:e69c with SMTP id b21-20020a0564021f1500b004358a5ae69cmr19068159edb.90.1656685473944; Fri, 01 Jul 2022 07:24:33 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:53 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-29-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 28/45] kmsan: disable physical page merging in biovec From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685475; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tSPSuwfCBJYINsR4GH3IgbP5patOgmYnzcC5xbF5jrk=; b=3npuWybUI+lqarXYy03yyomSuJAIcN1WIwPDQoybXQMIKKUueiGXX4lW5pAOuZHwyCpIGw yxayMBabYUlyutXzQkqxs1HMnyIm+PJ96CQ8TwjeKgvRYCtXX8b4hoQNkyS91vkZsJMTMQ NLpWpSACWpcPkRGAxg6LbHHgYbuS1ks= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685475; a=rsa-sha256; cv=none; b=IN/5SOJ19OF0KsBdEs0zfjIqCmTJ9STRuli8K4QdqRxK+ZN7i+9zSexI6M3dGQmvYpVJT4 LpCZj0kNvcnWVyp6k+YzuHypH8nnra8gAOOT8fZUZMVIBSHy5N4s890USxRDNuzv8WDACf fNkHZljYugx0Yg1R6Acq5M3weRKMUPQ= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=fBpJ5slq; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3oQO_YgYKCMAmrojkxmuumrk.iusrot03-ssq1giq.uxm@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3oQO_YgYKCMAmrojkxmuumrk.iusrot03-ssq1giq.uxm@flex--glider.bounces.google.com Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=fBpJ5slq; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3oQO_YgYKCMAmrojkxmuumrk.iusrot03-ssq1giq.uxm@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3oQO_YgYKCMAmrojkxmuumrk.iusrot03-ssq1giq.uxm@flex--glider.bounces.google.com X-Rspamd-Server: rspam03 X-Rspam-User: X-Stat-Signature: kgtgo9soc59iewn83xijk9k8zr885juj X-Rspamd-Queue-Id: 21204C0009 X-HE-Tag: 1656685474-158063 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN metadata for adjacent physical pages may not be adjacent, therefore accessing such pages together may lead to metadata corruption. We disable merging pages in biovec to prevent such corruptions. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Iece16041be5ee47904fbc98121b105e5be5fea5c --- block/blk.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/block/blk.h b/block/blk.h index 434017701403f..96309a98a60e3 100644 --- a/block/blk.h +++ b/block/blk.h @@ -93,6 +93,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q, phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset; phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset; + /* + * Merging adjacent physical pages may not work correctly under KMSAN + * if their metadata pages aren't adjacent. Just disable merging. + */ + if (IS_ENABLED(CONFIG_KMSAN)) + return false; + if (addr1 + vec1->bv_len != addr2) return false; if (xen_domain() && !xen_biovec_phys_mergeable(vec1, vec2->bv_page)) From patchwork Fri Jul 1 14:22:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C592FC433EF for ; Fri, 1 Jul 2022 14:24:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FA4B6B0082; Fri, 1 Jul 2022 10:24:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D1436B0083; Fri, 1 Jul 2022 10:24:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 471906B0085; Fri, 1 Jul 2022 10:24:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 35D306B0082 for ; Fri, 1 Jul 2022 10:24:38 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F2A6B967 for ; Fri, 1 Jul 2022 14:24:37 +0000 (UTC) X-FDA: 79638751794.20.E4D6D55 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf28.hostedemail.com (Postfix) with ESMTP id 911BBC004A for ; Fri, 1 Jul 2022 14:24:37 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id x2-20020a1709065ac200b006d9b316257fso840605ejs.12 for ; Fri, 01 Jul 2022 07:24:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ssAyKoViePAlR4LG6grDIIfDn5PGY84PY+YiOkKWqdU=; b=G6KzvVOzqo6QJFZMWm6nHQe5vJR+zroKwVscg+jN/JKqdskPZJ/1aqhWFYtC+2bwxo NXP1bfPuLlXUIIgvbs9p8xHqK+UMJ7rwAgUxvQozkeEjqAnLE/RFvNQVwvJnt+81yd+v HpfzN1hhNUR3zRliOOqHbAvewRvY5bCwm21yXKVpqWRCePcOpIkqnSd6sMD2Wkw4/xTQ KhUqVjXvWdacoQCUWPYOlEcJPWnDk+z27yXN8YpIbfxkOh5QZGnMGQLocZsfvItYsGD6 QbDu599Ew6o6dwINWsW0S8a6FHYXse2CJVkRD/6pUzIL5NVKDDIcBkPGOvIc1AplaBNk xrxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ssAyKoViePAlR4LG6grDIIfDn5PGY84PY+YiOkKWqdU=; b=d5turYzM4qmiSGp/0vgCqR/gZqIgjyXhxNNuOOyAPiW/v6LH2tFhYEmSm2cSNw27hg WlqDhoD7mp2KCYD1/2XCULxk4W0kl+zfjUsoV2AmKpdxAUEFrKlyDnLBLpt+TloPUEPA DxZMDWodJpIVJpcVTwlZoeYqKTHm83GM04Q+0k64UnEBIPLPA7ztwAwD4COG/CmEGNP6 6EzzxtbH7xYVGf2RpjCWkL1K8Sx1akzkIAffVCAsnfymTlQN8t9WP41gLMXigbosFv8n JH+oPr/vLU56L6jKmTcMje6vGMTUvvcMZ9DdM1C/ks9/P57xpuqqV5j6iYoBBFC3MSRp zWrA== X-Gm-Message-State: AJIora/Brv4Rx7t5SwE0zXWIa1OiK2LQMMzCvgIBdUI7qxhA3RDZGwP7 KxJegzMK96E9QYj1w+SD7UGAHRydRDE= X-Google-Smtp-Source: AGRyM1tVCF+sV/LeuqKsrQABp8XqaJxrk3E/8bbS/UOAygvrbBIURmws57k41Fha3AzhybLg/iCTnOg5f0Y= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:2077:b0:435:a428:76e4 with SMTP id bd23-20020a056402207700b00435a42876e4mr19161928edb.367.1656685476397; Fri, 01 Jul 2022 07:24:36 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:54 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-30-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 29/45] block: kmsan: skip bio block merging logic for KMSAN From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Eric Biggers ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685477; a=rsa-sha256; cv=none; b=jjIFuWcdeg6zGrQLum/6rcVrGpPrMXfktcotkqxCeGNrN/B8FyW32XoJ38LrxSWW1WxSgl BpHJCJ/5uk/gMzb+a8PVI/2hLUTWEhoWzN1/XiVH6jiOOoPTz9ItHMt91znJoPCFNV+iFr j0H601PA5UZkaMdnatWLKQcfQJLz680= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=G6KzvVOz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 3pAO_YgYKCMMpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3pAO_YgYKCMMpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ssAyKoViePAlR4LG6grDIIfDn5PGY84PY+YiOkKWqdU=; b=k9tHqmGDEmnokMIXIXMUivx2zbCFgJmBObmWjpRZnB1vA3fptiwXDdhRi6eAMqQvZuBpME YFVWX1/RbDd826jW+NXyZbmnnYZQ1WyASv6IuX+ucGKjGRRgnA7ej99C39DMgRCwgOvhN0 xEoJnj4wrKCplTMIWYEPjUmEE4Dxq8M= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 911BBC004A Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=G6KzvVOz; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 3pAO_YgYKCMMpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3pAO_YgYKCMMpurmn0pxxpun.lxvurw36-vvt4jlt.x0p@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: 19ejg368354uj3ixg9y9jem3cz9w3sps X-HE-Tag: 1656685477-183631 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN doesn't allow treating adjacent memory pages as such, if they were allocated by different alloc_pages() calls. The block layer however does so: adjacent pages end up being used together. To prevent this, make page_is_mergeable() return false under KMSAN. Suggested-by: Eric Biggers Signed-off-by: Alexander Potapenko --- v4: -- swap block: and kmsan: in the subject Link: https://linux-review.googlesource.com/id/Ie29cc2464c70032347c32ab2a22e1e7a0b37b905 --- block/bio.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/bio.c b/block/bio.c index 51c99f2c5c908..ce6b3c82159a6 100644 --- a/block/bio.c +++ b/block/bio.c @@ -867,6 +867,8 @@ static inline bool page_is_mergeable(const struct bio_vec *bv, return false; *same_page = ((vec_end_addr & PAGE_MASK) == page_addr); + if (!*same_page && IS_ENABLED(CONFIG_KMSAN)) + return false; if (*same_page) return true; return (bv->bv_page + bv_end / PAGE_SIZE) == (page + off / PAGE_SIZE); From patchwork Fri Jul 1 14:22:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 970B2C43334 for ; Fri, 1 Jul 2022 14:24:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33ACC6B0083; Fri, 1 Jul 2022 10:24:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EA8C6B0085; Fri, 1 Jul 2022 10:24:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B2AF6B0087; Fri, 1 Jul 2022 10:24:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0C45C6B0083 for ; Fri, 1 Jul 2022 10:24:41 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D716633B75 for ; Fri, 1 Jul 2022 14:24:40 +0000 (UTC) X-FDA: 79638751920.14.91E4D69 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf25.hostedemail.com (Postfix) with ESMTP id 61CD2A0041 for ; Fri, 1 Jul 2022 14:24:40 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id 7-20020a170906310700b007263068d531so845864ejx.15 for ; Fri, 01 Jul 2022 07:24:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5N9rqr7HITf8FFSE4yH2hCrYnonc5u0Mk2BxyWMh4Wc=; b=ReMSGptb1gOevOQA5KIOUI2r4Y7KHZlx+U+473Qz+0IYhBm7Zh60skPZL94IWrvGHe OD/b/T8E/9DRBnHYkCik0EMtmERYvMYLmNm2a9PZ3CXqtQvv3NgmYs9l7lSp7JKC2JNu asOUoRmUrLf+MBvhhA5FucNbpwjzToLRIKdYTiOAjk7d5eN8Pd3e+soWyP6PDlWUzRey 6eKr6HZEm15XrcXNJMk7zw70qhG9SX/ZUCNS5t5YyEGHANeY85at4PO2xF7Ipa5N6imk 72H7eRaKJCimZpAza45clzrOO1tWWs5oG5q/zZogSjW/5CpnZ1YyjjTkf+gFTDMx7GOA kGOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5N9rqr7HITf8FFSE4yH2hCrYnonc5u0Mk2BxyWMh4Wc=; b=ogso0OTqO5CccEP/taL5MPJ5T+lFgbP3ZfPG+TEDlgiM8k18sqxoIW+0/OwETPg/cb cH8MY70zG5GpsX1oyN4toolEgmBK/QwTtmWzFCq5b8i6P4uuvxShAd7oAP2ygMo2IJ5g JnO0MA+ztzPjp1rJIB2ibkVTVx8+egho9Vut25HA4BBurw05Bk5j4C5Y/EbapWVdEVEV KJFAegA/7xo0My6hFAjclXki3MiwB8Gjxn1QPpVWshFaF8eXVUOvczR3itcNFF92k/0u QCO1MzkQzDJfOUB9mGUDyEqQ3E4oOMn1BdtONmuU6ssF99+TyrPQpDjaGQqst9wd2boo SmGQ== X-Gm-Message-State: AJIora8o7/G8ict8WPoLrHKMLzzDMXUwT1Q3Qe/M9deKUOFBJKpMswnI CI4XmlchfR/hHO7Xng0eEqQwXq+hpJg= X-Google-Smtp-Source: AGRyM1uKM3hcXWEB+X983N8ZUuPx4ycBhxuOiVX+On/gdGCeZVb6tOlhn3CoZm88ilqSQUTNpJWyQUtAfdc= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:906:8501:b0:711:bf65:2a47 with SMTP id i1-20020a170906850100b00711bf652a47mr14797955ejx.150.1656685479201; Fri, 01 Jul 2022 07:24:39 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:55 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-31-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 30/45] kcov: kmsan: unpoison area->list in kcov_remote_area_put() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685480; a=rsa-sha256; cv=none; b=TeJWEqnl8nrIdRo8bNOvMriz4B0MGHUNlktyT9CWTRaxnlirwINp/KRbQQdRVGvJ5B6UeT PFdGp7v7/Y0VMbmYlpfxBwsnHdPtsT2v1Wzo81/tE+TmmdWQtSJCkMKuJ6KAei+VDlWPac CsOSpGCxAP7i85O1NTrx69OVIR52mmo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ReMSGptb; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3pwO_YgYKCMYsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3pwO_YgYKCMYsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685480; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5N9rqr7HITf8FFSE4yH2hCrYnonc5u0Mk2BxyWMh4Wc=; b=JXozZlBmCpeS9GFFh8bKOLCcq+DmBht33jMXbgXvUGgllptPWNVOhMo4S84bOlRZaxA7kz NShyAQPpPYg2Ighf7slAA5lYu0IqIXpxiCcDJenz0Wdb9nPoOE+y8tTcb1x+7aJueTKcHg TL2j9SUjwKoGDVyonq5zblBqbHJPC14= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 61CD2A0041 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ReMSGptb; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3pwO_YgYKCMYsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3pwO_YgYKCMYsxupq3s00sxq.o0yxuz69-yyw7mow.03s@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: ke7n8w355r7e9wzufsterzkjc5immi3z X-HE-Tag: 1656685480-252209 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN does not instrument kernel/kcov.c for performance reasons (with CONFIG_KCOV=y virtually every place in the kernel invokes kcov instrumentation). Therefore the tool may miss writes from kcov.c that initialize memory. When CONFIG_DEBUG_LIST is enabled, list pointers from kernel/kcov.c are passed to instrumented helpers in lib/list_debug.c, resulting in false positives. To work around these reports, we unpoison the contents of area->list after initializing it. Signed-off-by: Alexander Potapenko --- v4: -- change sizeof(type) to sizeof(*ptr) -- swap kcov: and kmsan: in the subject Link: https://linux-review.googlesource.com/id/Ie17f2ee47a7af58f5cdf716d585ebf0769348a5a --- kernel/kcov.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/kernel/kcov.c b/kernel/kcov.c index e19c84b02452e..e5cd09fd8a050 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -152,6 +153,12 @@ static void kcov_remote_area_put(struct kcov_remote_area *area, INIT_LIST_HEAD(&area->list); area->size = size; list_add(&area->list, &kcov_remote_areas); + /* + * KMSAN doesn't instrument this file, so it may not know area->list + * is initialized. Unpoison it explicitly to avoid reports in + * kcov_remote_area_get(). + */ + kmsan_unpoison_memory(&area->list, sizeof(area->list)); } static notrace bool check_kcov_mode(enum kcov_mode needed_mode, struct task_struct *t) From patchwork Fri Jul 1 14:22:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAF6EC433EF for ; Fri, 1 Jul 2022 14:24:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66E7D6B0085; Fri, 1 Jul 2022 10:24:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61EBE6B0087; Fri, 1 Jul 2022 10:24:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4992C6B0088; Fri, 1 Jul 2022 10:24:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3A5666B0085 for ; Fri, 1 Jul 2022 10:24:44 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 21A778054E for ; Fri, 1 Jul 2022 14:24:44 +0000 (UTC) X-FDA: 79638752088.21.0ACF71F Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf28.hostedemail.com (Postfix) with ESMTP id BB16AC0038 for ; Fri, 1 Jul 2022 14:24:43 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id z17-20020a05640235d100b0043762b1e1e3so1878359edc.21 for ; Fri, 01 Jul 2022 07:24:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=j/MMQ3wL35Ri2ZE1mUjRi1/flHRltTweDkn7/EP9qns=; b=A0D6vxcEEYdG93cr58BEviQfu68kGt3tEP1iTrR6OH1MbSU/Ast73yd8+4vsdcPmvE F7ExVCd/cNsiCCMmB9SmZydBoWkU9FpwlqoBDrui/aoc2LQ1w9cfWSsiQe9fPTxl7POg NM5QDGCVrMNlDXqg+DOl6IQHgm2+D05loxFr7+rCszfrFODm8mVgsHCH0aSVcLkJQe7T kky823TWf9SifhaporOMyAr0TbyadwLpD+GGYZT4yExEPXSeAb92kGyVv5kZ7CV8ymNw QQyd/Be2e/kfhOwudlyywf9K4wuV9loKYMofj2XGSkbkbiUtOPenyfcrmjx9kiJdXq0B iddA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=j/MMQ3wL35Ri2ZE1mUjRi1/flHRltTweDkn7/EP9qns=; b=be5wZT5OF0tGq3xC64I6+y09IY6Un5oeud50ZMMrKbH/KvJAXDoa8ZU75XsNCB4e42 x23I0y18mjiAO/RNgbQ6WRwOyRMGUkBhTFdAEjPi31ntRIpz0zZqqQcDjhjat9Q46tUf RkOizK3XU9djhNZcBw3wwY+bUt7dObY2iHzxk6SanEJSIvD/0RfXqDJWuDpfRkhGwLOo ni+eBf07R/atNPU3IZHId+74h6QDYz2jOwF3yeI+S9xYyY8qyphF1UKFuegZ69Xb+TYQ vbjoth0r+fFFa6CCcN+BVu9k140TCBiE+eX+eDw2QC3UgTU7sTFAFSqK4DE99GWQhIZ4 M/Wg== X-Gm-Message-State: AJIora8RGF5Nhlbm3LZuddoKRW15ZqMPGo16zvqZzKgVL0oc7U6Q1uEJ gHa9uRlzymlyjwzFyVSFg0m+p8EZVKo= X-Google-Smtp-Source: AGRyM1tW+YFwdIUQ0n4BqBxiWtFhSDA/V1RYi8OfYlcWw4L6Ky6njb01crp74wyk5/Ql1YeiQo29PS5mWtc= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:906:9b86:b0:6fe:d37f:b29d with SMTP id dd6-20020a1709069b8600b006fed37fb29dmr14586176ejc.327.1656685482242; Fri, 01 Jul 2022 07:24:42 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:56 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-32-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 31/45] security: kmsan: fix interoperability with auto-initialization From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=A0D6vxcE; spf=pass (imf28.hostedemail.com: domain of 3qgO_YgYKCMkv0xst6v33v0t.r310x29C-11zAprz.36v@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3qgO_YgYKCMkv0xst6v33v0t.r310x29C-11zAprz.36v@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685483; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j/MMQ3wL35Ri2ZE1mUjRi1/flHRltTweDkn7/EP9qns=; b=C7/ODWx9dpbOZimGvmh+XO/4KEhbLIR+Fv6TfWahQirZdVYws51ubwXLpcR86e/7cjEx3P Z2j0K1TQtAUnjYpB/o2jhKf20CgWvnWY1mhTIqyKxcAl63yCC5/jxR/yxIESOv8PE4CTOf E7kQAHLRTaY+3UROjfM3mDHzlE1WKFw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685483; a=rsa-sha256; cv=none; b=5wf1XynCcKziqkYbO+0ubjF6rJ6FxU2HofSB1FyelbPhhBKSx/Mk573BHbpUhIQ+M0RcAx 43SdCqFD0ey+guIp25DGj2SnvfhXSAzYLJVBGem3BrYOxgz8hXCg7GqlenkZgPBbfI8ldS EnshHLHjpY3rxHXRullodou3tMH7uJM= X-Stat-Signature: 1je4mwbpf6965dmbmqtw1fambr4h8otz X-Rspamd-Queue-Id: BB16AC0038 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=A0D6vxcE; spf=pass (imf28.hostedemail.com: domain of 3qgO_YgYKCMkv0xst6v33v0t.r310x29C-11zAprz.36v@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3qgO_YgYKCMkv0xst6v33v0t.r310x29C-11zAprz.36v@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-HE-Tag: 1656685483-103415 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Heap and stack initialization is great, but not when we are trying uses of uninitialized memory. When the kernel is built with KMSAN, having kernel memory initialization enabled may introduce false negatives. We disable CONFIG_INIT_STACK_ALL_PATTERN and CONFIG_INIT_STACK_ALL_ZERO under CONFIG_KMSAN, making it impossible to auto-initialize stack variables in KMSAN builds. We also disable CONFIG_INIT_ON_ALLOC_DEFAULT_ON and CONFIG_INIT_ON_FREE_DEFAULT_ON to prevent accidental use of heap auto-initialization. We however still let the users enable heap auto-initialization at boot-time (by setting init_on_alloc=1 or init_on_free=1), in which case a warning is printed. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I86608dd867018683a14ae1870f1928ad925f42e9 --- mm/page_alloc.c | 4 ++++ security/Kconfig.hardening | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e8d5a0b2a3264..3a0a5e204df7a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -854,6 +854,10 @@ void init_mem_debugging_and_hardening(void) else static_branch_disable(&init_on_free); + if (IS_ENABLED(CONFIG_KMSAN) && + (_init_on_alloc_enabled_early || _init_on_free_enabled_early)) + pr_info("mem auto-init: please make sure init_on_alloc and init_on_free are disabled when running KMSAN\n"); + #ifdef CONFIG_DEBUG_PAGEALLOC if (!debug_pagealloc_enabled()) return; diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index bd2aabb2c60f9..2739a6776454e 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -106,6 +106,7 @@ choice config INIT_STACK_ALL_PATTERN bool "pattern-init everything (strongest)" depends on CC_HAS_AUTO_VAR_INIT_PATTERN + depends on !KMSAN help Initializes everything on the stack (including padding) with a specific debug value. This is intended to eliminate @@ -124,6 +125,7 @@ choice config INIT_STACK_ALL_ZERO bool "zero-init everything (strongest and safest)" depends on CC_HAS_AUTO_VAR_INIT_ZERO + depends on !KMSAN help Initializes everything on the stack (including padding) with a zero value. This is intended to eliminate all @@ -218,6 +220,7 @@ config STACKLEAK_RUNTIME_DISABLE config INIT_ON_ALLOC_DEFAULT_ON bool "Enable heap memory zeroing on allocation by default" + depends on !KMSAN help This has the effect of setting "init_on_alloc=1" on the kernel command line. This can be disabled with "init_on_alloc=0". @@ -230,6 +233,7 @@ config INIT_ON_ALLOC_DEFAULT_ON config INIT_ON_FREE_DEFAULT_ON bool "Enable heap memory zeroing on free by default" + depends on !KMSAN help This has the effect of setting "init_on_free=1" on the kernel command line. This can be disabled with "init_on_free=0". From patchwork Fri Jul 1 14:22:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8947BC43334 for ; Fri, 1 Jul 2022 14:24:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C5E86B0087; Fri, 1 Jul 2022 10:24:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2278B6B0088; Fri, 1 Jul 2022 10:24:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07C976B0089; Fri, 1 Jul 2022 10:24:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EC79E6B0087 for ; Fri, 1 Jul 2022 10:24:46 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CE960615CF for ; Fri, 1 Jul 2022 14:24:46 +0000 (UTC) X-FDA: 79638752172.14.F9C1B7B Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf17.hostedemail.com (Postfix) with ESMTP id 72B7B4003F for ; Fri, 1 Jul 2022 14:24:46 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id y5-20020a056402358500b0043592ac3961so1867498edc.6 for ; Fri, 01 Jul 2022 07:24:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=z3bEqNJSKGgtE+5cRSN2oSZAKY283RHsG2GOYMoI00Y=; b=qnQCWjKq5yG4F3jPwXLMHLCDtYd3ls2GTNlYs9o6Wm5n9ZlXM2kcHIEFbE0tYS3KKn HmTASR9qhhcfDdychxG1CfugY/2EWvcCDInodoMg7RU0zMb9HYdN78o9NjljMlNc8rMd hIPR0zujO1oAwrMaZfThO3gQ6658HJ1jfgOwDhl9ZusUhjpUAC4OrXaLRqEtr0Uzsch/ DsH42O9o1nqIfH3L8cjBMMq0dkqxJiiaM9JZTR6pQdrxlRJnL8p1AzkkSc/He2avvllk IPUugX4/4fHGoOL5a8zYmVOjP0tM9Ig0OyWFD1/fv11wQ/OCoEsmw4o/l++EGmFeYma8 W7cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=z3bEqNJSKGgtE+5cRSN2oSZAKY283RHsG2GOYMoI00Y=; b=6Eomon3Tj0WT7tHS4aXKwu+brD/I4k/cwX2z+qSPVT57gq9iXzDfg/uKnMPtN73og1 XpMkcOFO6n2PIl07bp3UOtMPryHkjEfnJMqpSjjgJI7qHbpOGs5UskiHEvS9eI0s+umK mAi9SVeQVJh3ASeGZcAOA4B7ohCZNxfjU7vt8GkfDrItPBd0ifsKXZB/GMO/BahCZ05r b96z2DcfzW+jr3MHvkY0qEMehkIjWdYlRbs4cadcTvceEzELeWujXtbz+0AQW6zE/JqT elM+omWbGKLz7kUVmn+H8Ntgd8ByjX1n8rlndLt2bW938kKWWCUgpMg5ax6Fhm/ePRY3 LplQ== X-Gm-Message-State: AJIora+tkXyZcP3VsJXh8QuQXxFxNgS0Op0sZJ5XIhauriLj4cQqsE4P u5Q1d8skGLZyfQr9kthauLPddzMDQGo= X-Google-Smtp-Source: AGRyM1v3S88h3I8S3PNOLZJhMmyQhBUBhKM4ToewpYwCFFWW8Gw6EHSPC9ywT4YZ1DjScWDrCrHuxSVSgLc= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:906:2086:b0:717:4e91:f1db with SMTP id 6-20020a170906208600b007174e91f1dbmr14229924ejq.345.1656685485180; Fri, 01 Jul 2022 07:24:45 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:57 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-33-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 32/45] objtool: kmsan: list KMSAN API functions as uaccess-safe From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qnQCWjKq; spf=pass (imf17.hostedemail.com: domain of 3rQO_YgYKCMwy30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3rQO_YgYKCMwy30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685486; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z3bEqNJSKGgtE+5cRSN2oSZAKY283RHsG2GOYMoI00Y=; b=AYZhD8n7hbj+vRbacPVTYubiZYYESsbE/1K/ztqQpQJbCIWTyUeY9ksDG3Xs0PrlcyXouA sjEiu4KEDtgb1OdXHB6i2E9qZ4ISW7L+Q7IncRuk06g/4Xy1VEgvQWz2OIvuX3Dk4wpoTN v8jKcXa+cV8j8XOyYOtSAcFJcfzhnzw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685486; a=rsa-sha256; cv=none; b=UYeYTwnZKx+1rrCChK7DZEarIGtI/FNHu0VbM3Ucauhh8Cq8Wul+JtSTxpGYyzJNgfdlYl kUbAa+0/3rmJVHNvUo5zEBvxVqOGd1SoVA+qsDQuGgRTA8QmGajz5sWeUwA3h+Bi7HGgkv jqeNm0UkbNP6Fh5cOOercDHXZqj2w2k= X-Stat-Signature: dkjadbhetmi9ufmigxk8gq98ybrhxx3y X-Rspamd-Queue-Id: 72B7B4003F Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qnQCWjKq; spf=pass (imf17.hostedemail.com: domain of 3rQO_YgYKCMwy30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3rQO_YgYKCMwy30vw9y66y3w.u64305CF-442Dsu2.69y@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-HE-Tag: 1656685486-12814 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN inserts API function calls in a lot of places (function entries and exits, local variables, memory accesses), so they may get called from the uaccess regions as well. KMSAN API functions are used to update the metadata (shadow/origin pages) for kernel memory accesses. The metadata pages for kernel pointers are also located in the kernel memory, so touching them is not a problem. For userspace pointers, no metadata is allocated. If an API function is supposed to read or modify the metadata, it does so for kernel pointers and ignores userspace pointers. If an API function is supposed to return a pair of metadata pointers for the instrumentation to use (like all __msan_metadata_ptr_for_TYPE_SIZE() functions do), it returns the allocated metadata for kernel pointers and special dummy buffers residing in the kernel memory for userspace pointers. As a result, none of KMSAN API functions perform userspace accesses, but since they might be called from UACCESS regions they use user_access_save/restore(). Signed-off-by: Alexander Potapenko --- v3: -- updated the patch description v4: -- add kmsan_unpoison_entry_regs() Link: https://linux-review.googlesource.com/id/I242bc9816273fecad4ea3d977393784396bb3c35 --- tools/objtool/check.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 864bb9dd35845..1cf260c966441 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -1013,6 +1013,26 @@ static const char *uaccess_safe_builtin[] = { "__sanitizer_cov_trace_cmp4", "__sanitizer_cov_trace_cmp8", "__sanitizer_cov_trace_switch", + /* KMSAN */ + "kmsan_copy_to_user", + "kmsan_report", + "kmsan_unpoison_entry_regs", + "kmsan_unpoison_memory", + "__msan_chain_origin", + "__msan_get_context_state", + "__msan_instrument_asm_store", + "__msan_metadata_ptr_for_load_1", + "__msan_metadata_ptr_for_load_2", + "__msan_metadata_ptr_for_load_4", + "__msan_metadata_ptr_for_load_8", + "__msan_metadata_ptr_for_load_n", + "__msan_metadata_ptr_for_store_1", + "__msan_metadata_ptr_for_store_2", + "__msan_metadata_ptr_for_store_4", + "__msan_metadata_ptr_for_store_8", + "__msan_metadata_ptr_for_store_n", + "__msan_poison_alloca", + "__msan_warning", /* UBSAN */ "ubsan_type_mismatch_common", "__ubsan_handle_type_mismatch", From patchwork Fri Jul 1 14:22:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AF39C43334 for ; Fri, 1 Jul 2022 14:24:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9F526B0078; Fri, 1 Jul 2022 10:24:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4FCA6B007D; Fri, 1 Jul 2022 10:24:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F0766B0088; Fri, 1 Jul 2022 10:24:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 906D36B0078 for ; Fri, 1 Jul 2022 10:24:49 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6AC0E3BD for ; Fri, 1 Jul 2022 14:24:49 +0000 (UTC) X-FDA: 79638752298.02.D3ECD64 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf16.hostedemail.com (Postfix) with ESMTP id 17A5018006B for ; Fri, 1 Jul 2022 14:24:48 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id r12-20020a05640251cc00b00435afb01d7fso1871779edd.18 for ; Fri, 01 Jul 2022 07:24:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=iJzH9xEnS9/oOC6mna7qxAN7hxj/kW7q3+xewUipeVQ=; b=RVivBlTR5bofeeHvx5RVBIPcMBigWOGTJx99w3m0vNDpNl5jHHs78XLLZXJxwe/+5i Us8Tqq85fBL1Q5tr4+EcRexPCYSh2izwsFNvThnyX0KBGJJYkcgwY30jVffqFmHDyD67 iZwEMX65hRenyyZFN+5/nfXdDGFpZ/LxSRWWqwwJ98IqgGy6Kzh5LAUVX4FbaxvP4YUk hrM1kwNT2keDEX5k5YiHsQtL6cxITuwX4ilkbVNOdYayhHQrA0ShA+8wI4qSIzydqax0 A0kabSTooD72wkvYqsGaqrmOnGcagimMwdacJ5Qqil/EEyGsz4qLQDSwcoHCbI3XLzmR q9Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=iJzH9xEnS9/oOC6mna7qxAN7hxj/kW7q3+xewUipeVQ=; b=ES+miafHXwXbgfQm+HBnJlO0r6xCcPVadeR7X/J+XczCyNaE+U0PTojK17XLU/zT4A pFSN37+orogofvhBsx9m+t1dmTinSdVVjg0bH1+bq+EHY7ni+V9/iWEwBgFsn/ENp47D zs4hf40eoPhQnNLzKWlbuh+/BfsonT+J5wlid1lAdmMG6yfxQ+liKr85N6jc3pc73xcs 7QnHX23H/AoDiHE9SSqz5Pq2ehZN37GglTx2TU7R7W8BUr5QWQjFU7FYascW1sd2B9ur Wu8+Q5ihR89DMm/mD1BlEbxzo4hLGzFSYc8tqKJJbXr3ww8BO2Jx7pFwH9gyY6urlt/j 1nwA== X-Gm-Message-State: AJIora8Rg1c6Zf8ZlnF1ju3wfecOLHSdKufK9REh1K3qs/23iKOzq2Ya Zlo+4S7xjQxJimWoXdjb5M8WmU5ZcxM= X-Google-Smtp-Source: AGRyM1tdwiFJ3qF9MS38GxgbW52KC3MGvTZOZgXwa4g5wlNolLgvT8yqZo+xIYJSOE8/WPEzc6X2ND/yzmI= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:3e13:b0:726:eebc:3461 with SMTP id hp19-20020a1709073e1300b00726eebc3461mr14278594ejc.528.1656685487802; Fri, 01 Jul 2022 07:24:47 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:58 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-34-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 33/45] x86: kmsan: disable instrumentation of unsupported code From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685489; a=rsa-sha256; cv=none; b=O2AVj3HQjs/0vwUdeO3VuZ9ddHq96uEApnaLYryBSVHWmqsOoc/gu0Qaqww+uAycBWBltv 5wf9q/lhBZoNmpwf2/4ictGAtY9Yz57E0EB3zUt6emjyv+t8wlVvVYnjYmIb5rMFgttAsY cTDN05vuX822uEYOqKVvi+tNboRWYno= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=RVivBlTR; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3rwO_YgYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3rwO_YgYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685489; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iJzH9xEnS9/oOC6mna7qxAN7hxj/kW7q3+xewUipeVQ=; b=WP6VFP9l4Jakn6g8yhfgJLaJzE5CQd6thRLWFCmU9Kabnl3oOz9XE0EL5knFnchujzbozm wPgH+S4WBHwX3pUfNFM0Wjd+Hpnqff7EuoM+KQi7i0aVviNlTxV1gu08nQnYU7Hpdgrkbz Tc6ewibX1D/mCiEk45C1KRtJuT98gOw= Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=RVivBlTR; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 3rwO_YgYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3rwO_YgYKCM4052xyB08805y.w86527EH-664Fuw4.8B0@flex--glider.bounces.google.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 17A5018006B X-Stat-Signature: mu6ytzg6ayjwwp7sn9134pcww3qn96wj X-Rspam-User: X-HE-Tag: 1656685488-501037 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instrumenting some files with KMSAN will result in kernel being unable to link, boot or crashing at runtime for various reasons (e.g. infinite recursion caused by instrumentation hooks calling instrumented code again). Completely omit KMSAN instrumentation in the following places: - arch/x86/boot and arch/x86/realmode/rm, as KMSAN doesn't work for i386; - arch/x86/entry/vdso, which isn't linked with KMSAN runtime; - three files in arch/x86/kernel - boot problems; - arch/x86/mm/cpu_entry_area.c - recursion. Signed-off-by: Alexander Potapenko --- v2: -- moved the patch earlier in the series so that KMSAN can compile -- split off the non-x86 part into a separate patch v3: -- added a comment to lib/Makefile Link: https://linux-review.googlesource.com/id/Id5e5c4a9f9d53c24a35ebb633b814c414628d81b --- arch/x86/boot/Makefile | 1 + arch/x86/boot/compressed/Makefile | 1 + arch/x86/entry/vdso/Makefile | 3 +++ arch/x86/kernel/Makefile | 2 ++ arch/x86/kernel/cpu/Makefile | 1 + arch/x86/mm/Makefile | 2 ++ arch/x86/realmode/rm/Makefile | 1 + lib/Makefile | 2 ++ 8 files changed, 13 insertions(+) diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile index b5aecb524a8aa..d5623232b763f 100644 --- a/arch/x86/boot/Makefile +++ b/arch/x86/boot/Makefile @@ -12,6 +12,7 @@ # Sanitizer runtimes are unavailable and cannot be linked for early boot code. KASAN_SANITIZE := n KCSAN_SANITIZE := n +KMSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y # Kernel does not boot with kcov instrumentation here. diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 19e1905dcbf6f..8d0d4d89a00ae 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -20,6 +20,7 @@ # Sanitizer runtimes are unavailable and cannot be linked for early boot code. KASAN_SANITIZE := n KCSAN_SANITIZE := n +KMSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in. diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile index c2a8b76ae0bce..645bd919f9845 100644 --- a/arch/x86/entry/vdso/Makefile +++ b/arch/x86/entry/vdso/Makefile @@ -11,6 +11,9 @@ include $(srctree)/lib/vdso/Makefile # Sanitizer runtimes are unavailable and cannot be linked here. KASAN_SANITIZE := n +KMSAN_SANITIZE_vclock_gettime.o := n +KMSAN_SANITIZE_vgetcpu.o := n + UBSAN_SANITIZE := n KCSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 4c8b6ae802ac3..4f2617721d3dc 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -33,6 +33,8 @@ KASAN_SANITIZE_sev.o := n # With some compiler versions the generated code results in boot hangs, caused # by several compilation units. To be safe, disable all instrumentation. KCSAN_SANITIZE := n +KMSAN_SANITIZE_head$(BITS).o := n +KMSAN_SANITIZE_nmi.o := n OBJECT_FILES_NON_STANDARD_test_nx.o := y diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile index 9661e3e802be5..f10a921ee7565 100644 --- a/arch/x86/kernel/cpu/Makefile +++ b/arch/x86/kernel/cpu/Makefile @@ -12,6 +12,7 @@ endif # If these files are instrumented, boot hangs during the first second. KCOV_INSTRUMENT_common.o := n KCOV_INSTRUMENT_perf_event.o := n +KMSAN_SANITIZE_common.o := n # As above, instrumenting secondary CPU boot code causes boot hangs. KCSAN_SANITIZE_common.o := n diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index f8220fd2c169a..39c0700c9955c 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -12,6 +12,8 @@ KASAN_SANITIZE_mem_encrypt_identity.o := n # Disable KCSAN entirely, because otherwise we get warnings that some functions # reference __initdata sections. KCSAN_SANITIZE := n +# Avoid recursion by not calling KMSAN hooks for CEA code. +KMSAN_SANITIZE_cpu_entry_area.o := n ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_mem_encrypt.o = -pg diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile index 83f1b6a56449f..f614009d3e4e2 100644 --- a/arch/x86/realmode/rm/Makefile +++ b/arch/x86/realmode/rm/Makefile @@ -10,6 +10,7 @@ # Sanitizer runtimes are unavailable and cannot be linked here. KASAN_SANITIZE := n KCSAN_SANITIZE := n +KMSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in. diff --git a/lib/Makefile b/lib/Makefile index 5056769d00bb6..73fea85b76365 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -272,6 +272,8 @@ obj-$(CONFIG_POLYNOMIAL) += polynomial.o CFLAGS_stackdepot.o += -fno-builtin obj-$(CONFIG_STACKDEPOT) += stackdepot.o KASAN_SANITIZE_stackdepot.o := n +# In particular, instrumenting stackdepot.c with KMSAN will result in infinite +# recursion. KMSAN_SANITIZE_stackdepot.o := n KCOV_INSTRUMENT_stackdepot.o := n From patchwork Fri Jul 1 14:22:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6346C433EF for ; Fri, 1 Jul 2022 14:24:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 620886B007D; Fri, 1 Jul 2022 10:24:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D0686B0082; Fri, 1 Jul 2022 10:24:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44AE06B0088; Fri, 1 Jul 2022 10:24:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 322866B007D for ; Fri, 1 Jul 2022 10:24:52 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 091AB60AF6 for ; Fri, 1 Jul 2022 14:24:52 +0000 (UTC) X-FDA: 79638752424.30.FB5CCC1 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf07.hostedemail.com (Postfix) with ESMTP id A4E7B40052 for ; Fri, 1 Jul 2022 14:24:51 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id hq41-20020a1709073f2900b00722e5ad076cso854405ejc.20 for ; Fri, 01 Jul 2022 07:24:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=J/zoX+mhuBfgzG0v9rZx6dvFzp5UyG4GyQqVH6m9IHs=; b=ReTKlob9Qp/FLxAV1DvlHkuUYXTYWPxzPkj5a7irIhWUye1sNVXC5MtER1Ls2z92mp 8Wo25eNrT33XToY0e8nz5feDu3mHb7QFF4VgDm2uDBb8l81TfyIVkddhJcjSsD9ufnM1 Rv9mKSVb0IZG3XVbqV42jTmV3T2Pq2cwTiXOWbvBXBTxw7dePWe5s9bn8qm8wQ8Lt42Z FJrd9KclhBhbHdq39ThpfJZtYbwUVP4vwFJvGlzf7CSyuOSqxyBQ2Qn2zplQeikVadq4 zV+wZJlGCyvmNqBwk6OsqxzwzJFz8qkWXt5LmJX5amCREhfg4UTcaupAIQUr4KVgADMB h0og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=J/zoX+mhuBfgzG0v9rZx6dvFzp5UyG4GyQqVH6m9IHs=; b=qmcEXCzNTKj5owOHQI73Co0ufwZVrpvkzli2DvngWL5dlsaFsgdqIzdP3BumSM9nJ0 KDUSJOWb1u9NFcgPt85o78BVX4flFtfHDFiW7Ec1bBBr8ZXcpoivcHKkrHLQpdc21YtH +XM+Ddocvm60+9qyxh09Q2wc+LVgduYK7UfEoetv/OlalL3yUl+MjhBYbcROzdk78mqt L+I9/PFUgmkb4qByLDyFgEydtG5jNPUqEti2W0ZR7TiK1wQgYtoDofDq/ofbhGGhihRO uxzPmQWnnnLBtrYtyeHT3kBNOqqBUi1dURId6URp/8TXjsRLc7K7nbJk8vYiTwOvAlew OXjQ== X-Gm-Message-State: AJIora+3LWkfiUGSiFJRSanzb2XzhqTtPvd0GzB973WoDn/0HVm+lBzi Ho6b6JdZhaUufZLQPYaY3JiJOV/YBws= X-Google-Smtp-Source: AGRyM1txu7w54o9DaeQXovmZ5qxDTEHTCqPpNip+hr7dMrgvfv9CAiY8ULpNYkP1mTZY+VlfmYuBRFUAtEM= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:a05c:b0:72a:3959:43db with SMTP id gz28-20020a170907a05c00b0072a395943dbmr13506354ejc.359.1656685490466; Fri, 01 Jul 2022 07:24:50 -0700 (PDT) Date: Fri, 1 Jul 2022 16:22:59 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-35-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 34/45] x86: kmsan: skip shadow checks in __switch_to() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685491; a=rsa-sha256; cv=none; b=jA9F96RE3t0bvyI+2CbiEJOhhpFzPvaIinyjbNsZPOShu6tODn4U+P4Ku5qB+FqJNRwgvm 9OnVj+cPYwDowviwX9NVD5CZIN/lItdQPM1SQnnF3mD2ZyilpF80whn7Sd7eeUrY0qbWCe pt27ovlsNqDNdjbrzUpxQqo22tlKSWU= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ReTKlob9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3sgO_YgYKCNE38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3sgO_YgYKCNE38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=J/zoX+mhuBfgzG0v9rZx6dvFzp5UyG4GyQqVH6m9IHs=; b=x4kJ1c4gm314IulI5OSM+Hhq6Di8y5B6VMwqJ9qHsiWosTYOlRDLZLY+QgzZNqomBxxEUB ShVOhF80oOl2pKzcMXzgwmuUN9+25Oj42UUlo4Fh4wfelFO3je96JgTBsDWPsHSSpsCgmx b1wHFZ6lM3f+vvvza1RkIRxqC4gxdgw= X-Stat-Signature: cgyg7ouq3ftykgrmn93hn1sswzb1tiza X-Rspamd-Queue-Id: A4E7B40052 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ReTKlob9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3sgO_YgYKCNE38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3sgO_YgYKCNE38501E3BB381.zB985AHK-997Ixz7.BE3@flex--glider.bounces.google.com X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1656685491-83313 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When instrumenting functions, KMSAN obtains the per-task state (mostly pointers to metadata for function arguments and return values) once per function at its beginning, using the `current` pointer. Every time the instrumented function calls another function, this state (`struct kmsan_context_state`) is updated with shadow/origin data of the passed and returned values. When `current` changes in the low-level arch code, instrumented code can not notice that, and will still refer to the old state, possibly corrupting it or using stale data. This may result in false positive reports. To deal with that, we need to apply __no_kmsan_checks to the functions performing context switching - this will result in skipping all KMSAN shadow checks and marking newly created values as initialized, preventing all false positive reports in those functions. False negatives are still possible, but we expect them to be rare and impersistent. Suggested-by: Marco Elver Signed-off-by: Alexander Potapenko Link: https://linux-review.googlesource.com/id/Ib7d4d70946f08128ade207519c1ee405fd812839 --- v2: -- This patch was previously called "kmsan: skip shadow checks in files doing context switches". Per Mark Rutland's suggestion, we now only skip checks in low-level arch-specific code, as context switches in common code should be invisible to KMSAN. We also apply the checks to precisely the functions performing the context switch instead of the whole file. v4: -- Replace KMSAN_ENABLE_CHECKS_process_64.o with __no_kmsan_checks Link: https://linux-review.googlesource.com/id/I45e3ed9c5f66ee79b0409d1673d66ae419029bcb --- arch/x86/kernel/process_64.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 1962008fe7437..6b3418bff3261 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -553,6 +553,7 @@ void compat_start_thread(struct pt_regs *regs, u32 new_ip, u32 new_sp, bool x32) * Kprobes not supported here. Set the probe on schedule instead. * Function graph tracer not supported too. */ +__no_kmsan_checks __visible __notrace_funcgraph struct task_struct * __switch_to(struct task_struct *prev_p, struct task_struct *next_p) { From patchwork Fri Jul 1 14:23:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903392 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC133C433EF for ; Fri, 1 Jul 2022 14:24:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E4446B0075; Fri, 1 Jul 2022 10:24:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 694606B0082; Fri, 1 Jul 2022 10:24:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55D856B0083; Fri, 1 Jul 2022 10:24:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 45A406B0075 for ; Fri, 1 Jul 2022 10:24:55 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 1482F8041E for ; Fri, 1 Jul 2022 14:24:55 +0000 (UTC) X-FDA: 79638752550.18.3AC60AC Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf09.hostedemail.com (Postfix) with ESMTP id 93CB0140047 for ; Fri, 1 Jul 2022 14:24:54 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id z13-20020a056402274d00b004357fcdd51fso1871048edd.17 for ; Fri, 01 Jul 2022 07:24:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=CG3pZwK286Wp+stKkliNeTwrORF17cOHhcnhVfCJeQE=; b=bYQ2u2U8tj3aHjd9MCtMixUrYfpSQwTg4WPOIoCPEeukhh64yxMHafM5vs/LGNKnzQ WbZCO/BXLkzB63X6UKAseam1QIfdrrDcHBVOGpbspRYW0ejgy0BAvw5OVpGh3iBOeZEX aaVkXTg3A0tuwa1e18PtKkEW1bZ16MVMyhI4LHrOLoqfrXkdK5JLNMCCXCzP+UqlS+7Y jMHKRvPORQSh2gGeBXFyCiEXw+TmFxCdnaNG5nGomGgf7TAWqOwR/NSipZV8JXpF91j0 T7T+oh1yqE70PVoxhXBFXshJGT1hPLtyEjj27zgL+pSYm9TNfYDnuWL3u/qLN7pkNGWn fRFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=CG3pZwK286Wp+stKkliNeTwrORF17cOHhcnhVfCJeQE=; b=eGg0uBrUd4DO++x6CALsCLjsi1ji01PDLDfGkIQDX61MmHXyLZ2b+M1mOlg5oNaOpv pHnPnJppr+7Yeg8ivE3UZomeldIvfFK8RNV14rHB/KiIYTrLaJjOARtNYG6p5jc6run/ Ke8Y+MNiCQYq902j1EqpzvtoUfl4Q3DUR3dYhun3Wp6GgOEvPhkeo+o6hr7+7kNmKgUY y2+zn6U3RwUMiNMSp7M2veJsZYbyraIAtVtgc90m0JV3fS19fhTcaWr9/jgx6qzzF6bk 4OkWxCoFGR+fPUmNpfcqDB4Qcvmw9pxZ+bxjJcsWEjbCIDAhkTMbYeXaNsZHveQ/yQda ysog== X-Gm-Message-State: AJIora9jLTTSQpPn6y8OrTbpPqTx+K6qUfZlmPb8Xt1XCmsvoI2Lcwk3 el+uOI0H6JKZd6LfqwmTrBPIsH3F0VU= X-Google-Smtp-Source: AGRyM1vuhrwmXAXJ9wJid3cKYDBtVUPAQNXpJsIZnZORqGteH/3hS1Qh3dFSI+7P2W3d1z7T02mluBgbb3c= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:2403:b0:439:682f:d12c with SMTP id t3-20020a056402240300b00439682fd12cmr7312146eda.301.1656685493399; Fri, 01 Jul 2022 07:24:53 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:00 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-36-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 35/45] x86: kmsan: handle open-coded assembly in lib/iomem.c From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=bYQ2u2U8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3tQO_YgYKCNQ6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3tQO_YgYKCNQ6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685494; a=rsa-sha256; cv=none; b=qgYrFJKw0sR40n/bTrBOyTgM9k500Pqf7zE5besGIqfuXMZkN6G/avf0etlYkH3SE/mB5U lWxjdOH5B9pusEXv1oE+KJSVR8QPVGOPcXyp5RNyHrWpg9f94CEM5RCF5ZT8gzSrtfAonC XZJVUmsyxtR8VQddvrrtiyw7QcdtMRE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685494; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CG3pZwK286Wp+stKkliNeTwrORF17cOHhcnhVfCJeQE=; b=v4emhrPSxlKUb95XMGCCwS7hFNK6kKNAgYCM0pxNBGkyqlWJRhfftsxtEOUhETL1U8UD/f Iiepm22auNdpYC2g/7HJNuwWdhzq7ymHe1l8o6CKi5ZwvAiH/CUZg3HkAevb2iX6hXxtXQ Xmeu4xTdwmwjgj/90AbBh+H4I2c1p/Y= X-Stat-Signature: 61by4qnrga5g64iwkpyk61qrcddan5ty X-Rspamd-Queue-Id: 93CB0140047 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=bYQ2u2U8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3tQO_YgYKCNQ6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3tQO_YgYKCNQ6B834H6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--glider.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1656685494-92151 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN cannot intercept memory accesses within asm() statements. That's why we add kmsan_unpoison_memory() and kmsan_check_memory() to hint it how to handle memory copied from/to I/O memory. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Icb16bf17269087e475debf07a7fe7d4bebc3df23 --- arch/x86/lib/iomem.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/lib/iomem.c b/arch/x86/lib/iomem.c index 3e2f33fc33de2..e0411a3774d49 100644 --- a/arch/x86/lib/iomem.c +++ b/arch/x86/lib/iomem.c @@ -1,6 +1,7 @@ #include #include #include +#include #define movs(type,to,from) \ asm volatile("movs" type:"=&D" (to), "=&S" (from):"0" (to), "1" (from):"memory") @@ -37,6 +38,8 @@ static void string_memcpy_fromio(void *to, const volatile void __iomem *from, si n-=2; } rep_movs(to, (const void *)from, n); + /* KMSAN must treat values read from devices as initialized. */ + kmsan_unpoison_memory(to, n); } static void string_memcpy_toio(volatile void __iomem *to, const void *from, size_t n) @@ -44,6 +47,8 @@ static void string_memcpy_toio(volatile void __iomem *to, const void *from, size if (unlikely(!n)) return; + /* Make sure uninitialized memory isn't copied to devices. */ + kmsan_check_memory(from, n); /* Align any unaligned destination IO */ if (unlikely(1 & (unsigned long)to)) { movs("b", to, from); From patchwork Fri Jul 1 14:23:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4C1FCCA479 for ; Fri, 1 Jul 2022 14:24:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 536836B0081; Fri, 1 Jul 2022 10:24:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EB736B0082; Fri, 1 Jul 2022 10:24:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 360A36B0083; Fri, 1 Jul 2022 10:24:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2559F6B0081 for ; Fri, 1 Jul 2022 10:24:58 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0686D20E0B for ; Fri, 1 Jul 2022 14:24:58 +0000 (UTC) X-FDA: 79638752676.02.AA1774A Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf08.hostedemail.com (Postfix) with ESMTP id A5F7816003A for ; Fri, 1 Jul 2022 14:24:57 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id w22-20020a05640234d600b00435ba41dbaaso1884167edc.12 for ; Fri, 01 Jul 2022 07:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=y2vNDbHfnK38Naadc3+Fwqq63OR/g0K1wBY1TqEXyAM=; b=JFjDqoVxXFGtm201IbbG357skezLd+T04/kWKBQGEughGhugLaopnebvFnawR0Kdsr 27ps95W+5qwzJFISc09B33+ERrKVXpOpqoMtrCKT/HtJS/mc+Y2T4vZtazhtmZt5dEW/ 6YcV3sK0Awn2rROg1rzXOx4DymiaFx9MNHURtldWEnOEz01PiUXsvdxOa9SJzM6FKaQu heiTguJRDqBIlOpunqu4UGMuMlcF9OwwspcMFPk4tB7coqypU57aVRBKufAmVL56Ntdn YavSj16XkJUWu5JYiDjJH4XZH6E6lfbc716eCpEJho13ZlQOC3VfHu610QO29KVA6ujZ pKyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=y2vNDbHfnK38Naadc3+Fwqq63OR/g0K1wBY1TqEXyAM=; b=J9LcSltgRJD+YHcsqCWpbEjh4qXVIGOg6d0sau0QjPi3xNAm5hIYUANJ94Enr87i88 bBoK1+yalTLB3WY+6Dg5LzDi+YMjKRcMOAmjJiF5Wc2U5B1Q4ZvK6n5g/1vEAAJg5Xhf qSrKyEMg464dVBK/Syy/45nASdfAlSwJc1ZXSc0ZfKoffao3iYQCzf7ySyMhy0wrvl6Z 9Hv/0dHqnGv9XuVv8Rz7UH1MzYqPoE3309we3qItg3Oe4/iUHmlCiO4yHPdHl6t118vR 71sxYsYxmujAGj6LQUSLQYQjL3iRtXafE/meLvmVOQEL1JmsHn9cWEQpTqMvgU+7UMzM e1EQ== X-Gm-Message-State: AJIora/CmuNL+FB4rGhFJCZawqJ+Kk9JVYR4z2pvgcnU+wxpjVba4PeI WNghW3EVzN9SV68O5b3lRX6T79a1A9k= X-Google-Smtp-Source: AGRyM1uuNnnM3qssrOrgZOqetYyyDgTLXfrXa0K/82jjlreLIfrHkHYk4kBhmEAowU1mOBxY98uQNMpqi60= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:aa7:c9c9:0:b0:431:962f:f61e with SMTP id i9-20020aa7c9c9000000b00431962ff61emr19491774edt.189.1656685496470; Fri, 01 Jul 2022 07:24:56 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:01 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-37-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 36/45] x86: kmsan: use __msan_ string functions where possible From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685497; a=rsa-sha256; cv=none; b=by29czc9H0QZmVxHUo1/3Kd+ZH9HBzXgRcsvFHH8piyqkF+JHZj8NUfaJedMqHHo5mXj/2 21aP6T84Q1qTqConv22o7xV/2G9dWfOTm6LBXQQdy1LJsiGAyfvWA8Q2zvVw4MjszTH9OX 8uqc7c/eW0JjN5e+oBam0zDrt5Zd6Mw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=JFjDqoVx; spf=pass (imf08.hostedemail.com: domain of 3uAO_YgYKCNc9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3uAO_YgYKCNc9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685497; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y2vNDbHfnK38Naadc3+Fwqq63OR/g0K1wBY1TqEXyAM=; b=OlsTxwcwokxWdbB5DhzWz4psV3vjqTrMjr8JUd/i65sAdZiBRzvuDVLluci+27i4Fz7/tZ 8tF3Sto76aSzyUFk6jFzv8fxFz6oU3/XaMmLx/Zn3B9tmRskvfrC+RpA/zi4e+6+CMPfGT 6Nw+dmDQEuyi6LR4stA5VXG0bfUAgow= X-Rspam-User: X-Rspamd-Server: rspam04 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=JFjDqoVx; spf=pass (imf08.hostedemail.com: domain of 3uAO_YgYKCNc9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3uAO_YgYKCNc9EB67K9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: 1njrzkjrrq1osmgt1numoxps4mmwp78o X-Rspamd-Queue-Id: A5F7816003A X-HE-Tag: 1656685497-23212 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Unless stated otherwise (by explicitly calling __memcpy(), __memset() or __memmove()) we want all string functions to call their __msan_ versions (e.g. __msan_memcpy() instead of memcpy()), so that shadow and origin values are updated accordingly. Bootloader must still use the default string functions to avoid crashes. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I7ca9bd6b4f5c9b9816404862ae87ca7984395f33 --- arch/x86/include/asm/string_64.h | 23 +++++++++++++++++++++-- include/linux/fortify-string.h | 2 ++ 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/string_64.h b/arch/x86/include/asm/string_64.h index 6e450827f677a..3b87d889b6e16 100644 --- a/arch/x86/include/asm/string_64.h +++ b/arch/x86/include/asm/string_64.h @@ -11,11 +11,23 @@ function. */ #define __HAVE_ARCH_MEMCPY 1 +#if defined(__SANITIZE_MEMORY__) +#undef memcpy +void *__msan_memcpy(void *dst, const void *src, size_t size); +#define memcpy __msan_memcpy +#else extern void *memcpy(void *to, const void *from, size_t len); +#endif extern void *__memcpy(void *to, const void *from, size_t len); #define __HAVE_ARCH_MEMSET +#if defined(__SANITIZE_MEMORY__) +extern void *__msan_memset(void *s, int c, size_t n); +#undef memset +#define memset __msan_memset +#else void *memset(void *s, int c, size_t n); +#endif void *__memset(void *s, int c, size_t n); #define __HAVE_ARCH_MEMSET16 @@ -55,7 +67,13 @@ static inline void *memset64(uint64_t *s, uint64_t v, size_t n) } #define __HAVE_ARCH_MEMMOVE +#if defined(__SANITIZE_MEMORY__) +#undef memmove +void *__msan_memmove(void *dest, const void *src, size_t len); +#define memmove __msan_memmove +#else void *memmove(void *dest, const void *src, size_t count); +#endif void *__memmove(void *dest, const void *src, size_t count); int memcmp(const void *cs, const void *ct, size_t count); @@ -64,8 +82,7 @@ char *strcpy(char *dest, const char *src); char *strcat(char *dest, const char *src); int strcmp(const char *cs, const char *ct); -#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__) - +#if (defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)) /* * For files that not instrumented (e.g. mm/slub.c) we * should use not instrumented version of mem* functions. @@ -73,7 +90,9 @@ int strcmp(const char *cs, const char *ct); #undef memcpy #define memcpy(dst, src, len) __memcpy(dst, src, len) +#undef memmove #define memmove(dst, src, len) __memmove(dst, src, len) +#undef memset #define memset(s, c, n) __memset(s, c, n) #ifndef __NO_FORTIFY diff --git a/include/linux/fortify-string.h b/include/linux/fortify-string.h index 3b401fa0f3746..6c8a1a29d0b63 100644 --- a/include/linux/fortify-string.h +++ b/include/linux/fortify-string.h @@ -285,8 +285,10 @@ __FORTIFY_INLINE void fortify_memset_chk(__kernel_size_t size, * __builtin_object_size() must be captured here to avoid evaluating argument * side-effects further into the macro layers. */ +#ifndef CONFIG_KMSAN #define memset(p, c, s) __fortify_memset_chk(p, c, s, \ __builtin_object_size(p, 0), __builtin_object_size(p, 1)) +#endif /* * To make sure the compiler can enforce protection against buffer overflows, From patchwork Fri Jul 1 14:23:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903394 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91F2BCCA47B for ; Fri, 1 Jul 2022 14:25:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 313B16B0082; Fri, 1 Jul 2022 10:25:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C3D76B0083; Fri, 1 Jul 2022 10:25:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18B516B0085; Fri, 1 Jul 2022 10:25:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 03E5A6B0082 for ; Fri, 1 Jul 2022 10:25:01 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D963621648 for ; Fri, 1 Jul 2022 14:25:00 +0000 (UTC) X-FDA: 79638752760.29.138F2FB Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf31.hostedemail.com (Postfix) with ESMTP id 8021C2003E for ; Fri, 1 Jul 2022 14:25:00 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id f13-20020a0564021e8d00b00437a2acb543so1884537edf.7 for ; Fri, 01 Jul 2022 07:25:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Hs1LG4gEZrL61QQbW1yHjuo38dVFxHr6aTHqGRd2nIU=; b=R1eUqKBbN6WzWIjXxRI2PDbx29b9tpJSiig3/YaznmdOIZ/neK78PvIj5Cl/JjgB/w 9SfdvW2BtOiP89nS8ThzmS35SU1A6VUk9hiJALgX6yvIEpuomqcW5bDLzgsEQ013f6rk rFf3VUovNXoIf4KVP4MMfWUh7tsgq1WO5Pa9II9uYW0yZdgX2/mSo5GXmSL2H/ZwJwsd uxM4RIZS+4NPzi2RYwgUcpKcS16h49QlGgoZ05LwFfkprgCDz5UPO4MlYHJZ5DPrqB+x 6AHtkpJzEpLuNlHTmvU9jllU2HKPHg+40cmHHxveZn6WEswHaMpjdduncEJv7l8ri3sA Qcyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Hs1LG4gEZrL61QQbW1yHjuo38dVFxHr6aTHqGRd2nIU=; b=scqNkCWmk2COssdn3dwtz/MqA395Y7i19SK2nDjzIGt61PjPTV6oEg/YLGq+I3padL 8mqLdE8FbbkTJMXXCCgIX5L3ccNISkznydw2kIJTSf0Xgg3QRXTUmJKdAfMAMYTXfrGH gB4BlFXqVJpwtLH+iluglxGwHxU/ER8FUKV7BMHII2TuLLLLBET87a4nUFxIevH7HOri 1oU7olAvC/Rd9QuFY1A3i3Mo8ROoMBUGUa5uO9upOPW/4gNaQHZR/W03ejrXJCX75enA u2Q84rFDCjsRj0BHI9s7+x0MjvTxi+75b8+Lu4e3k8s7kjuOaU6LwsO5qB26iHYWy7yB 5bXg== X-Gm-Message-State: AJIora9nVZDfSYatFZTobTh5kEwj82D8nUCSvTWPDn6AVSBx9I2dDJUI s2YkwDneFRdebv76Tkwhzyun4mR3ENI= X-Google-Smtp-Source: AGRyM1sHhTsQ0hcpXEAMUmnlfkHECIs68+d9wOkDzkI+VoyBaJw2lW70YlMf+nBUJARdbRgd+2PdvDTFpog= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:906:846c:b0:72a:4b4f:b1b1 with SMTP id hx12-20020a170906846c00b0072a4b4fb1b1mr11506007ejc.255.1656685499239; Fri, 01 Jul 2022 07:24:59 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:02 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-38-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 37/45] x86: kmsan: sync metadata pages on page fault From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Authentication-Results: i=1; imf31.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=R1eUqKBb; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf31.hostedemail.com: domain of 3uwO_YgYKCNoCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3uwO_YgYKCNoCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685500; a=rsa-sha256; cv=none; b=UTBMifYSv9dlZKPJMHFgpKlK0xf9D9iuMms5QfaFLdp/GLvqBQCa+Y4ulx+meQW/gnnZ7V bJOoa6+mTR/xIBbIpyP/jTPxVOuk5R9D4zyKXpxfawG04OuzK1+xAakVW2envy+EsyOuHs qTc9ZhKA3NaAOig0eEgxamU7biaJBI0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685500; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Hs1LG4gEZrL61QQbW1yHjuo38dVFxHr6aTHqGRd2nIU=; b=sf77JPrGhIxNfSk2BynJxnecmmhp1humo1FzS5sOsz2+RkEW4d8+53Oqv2+FOENfaTUK8a w1JJ/Xb/reWHl3Z5wwHYM8MP1ZdyfvmCuJPYBtswk9ZKUrILjWhjInjiE/cXcABV5CpLYN hsa5fIveRU+mEPrnpKge8ZWll4g0+j8= X-Stat-Signature: 1hyxbmwhq9dz78hjzjqpp9z4qntoxpch X-Rspamd-Queue-Id: 8021C2003E Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=R1eUqKBb; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf31.hostedemail.com: domain of 3uwO_YgYKCNoCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com designates 209.85.208.74 as permitted sender) smtp.mailfrom=3uwO_YgYKCNoCHE9ANCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--glider.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1656685500-727386 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN assumes shadow and origin pages for every allocated page are accessible. For pages between [VMALLOC_START, VMALLOC_END] those metadata pages start at KMSAN_VMALLOC_SHADOW_START and KMSAN_VMALLOC_ORIGIN_START, therefore we must sync a bigger memory region. Signed-off-by: Alexander Potapenko --- v2: -- addressed reports from kernel test robot Link: https://linux-review.googlesource.com/id/Ia5bd541e54f1ecc11b86666c3ec87c62ac0bdfb8 --- arch/x86/mm/fault.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index fad8faa29d042..d07fe0801f203 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -260,7 +260,7 @@ static noinline int vmalloc_fault(unsigned long address) } NOKPROBE_SYMBOL(vmalloc_fault); -void arch_sync_kernel_mappings(unsigned long start, unsigned long end) +static void __arch_sync_kernel_mappings(unsigned long start, unsigned long end) { unsigned long addr; @@ -284,6 +284,27 @@ void arch_sync_kernel_mappings(unsigned long start, unsigned long end) } } +void arch_sync_kernel_mappings(unsigned long start, unsigned long end) +{ + __arch_sync_kernel_mappings(start, end); +#ifdef CONFIG_KMSAN + /* + * KMSAN maintains two additional metadata page mappings for the + * [VMALLOC_START, VMALLOC_END) range. These mappings start at + * KMSAN_VMALLOC_SHADOW_START and KMSAN_VMALLOC_ORIGIN_START and + * have to be synced together with the vmalloc memory mapping. + */ + if (start >= VMALLOC_START && end < VMALLOC_END) { + __arch_sync_kernel_mappings( + start - VMALLOC_START + KMSAN_VMALLOC_SHADOW_START, + end - VMALLOC_START + KMSAN_VMALLOC_SHADOW_START); + __arch_sync_kernel_mappings( + start - VMALLOC_START + KMSAN_VMALLOC_ORIGIN_START, + end - VMALLOC_START + KMSAN_VMALLOC_ORIGIN_START); + } +#endif +} + static bool low_pfn(unsigned long pfn) { return pfn < max_low_pfn; From patchwork Fri Jul 1 14:23:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6D98C433EF for ; Fri, 1 Jul 2022 14:25:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 584EF6B0083; Fri, 1 Jul 2022 10:25:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 534696B0085; Fri, 1 Jul 2022 10:25:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FC7D6B0088; Fri, 1 Jul 2022 10:25:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2E2726B0083 for ; Fri, 1 Jul 2022 10:25:04 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 003F530638 for ; Fri, 1 Jul 2022 14:25:03 +0000 (UTC) X-FDA: 79638752928.22.08A8F37 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf05.hostedemail.com (Postfix) with ESMTP id 9EDB110004C for ; Fri, 1 Jul 2022 14:25:03 +0000 (UTC) Received: by mail-ej1-f73.google.com with SMTP id e20-20020a170906315400b007262bd0111eso843086eje.9 for ; Fri, 01 Jul 2022 07:25:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7Mhpta8bleV/UMALfZOJOInTXQD6eorZ40ixMEZLH18=; b=SK4zuM/LqN9V22hVvhABvQyziEAxlHrFJzT62CJ+8Gr5z1csKhH0NIvPZy7HXwJZFS 0o2Dz9vScER5v/m3+Ov0b57zCiiCbrCApAZdn0z0ihUpL5uyP1/bXdMsnOidMKiSUJDa BkpEhnOP5g9I2ipIkJ/00hmAFD/og1SK1bdSIzNn7w1oXdItFTnvjA3O0QGS+joj1qPX vu0mr88M+r8S11fh0jNIUSEtNfYY0tXFLftByqdopzCcL3vRLSr276xum9GnJaDchFRq uhPrpu558FoA7vusBo7CKw0yXyOXaTlW7kFEpBeSn/VATqyQvBlleRzR7PZQtdIROe8G gmaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7Mhpta8bleV/UMALfZOJOInTXQD6eorZ40ixMEZLH18=; b=O2ejW6coLAboV1Mv0xZb0TKqU2mjJ9m/6I0gMfZrwrCRQwt2/EN2vBbUwvwz9XpQa9 ZRxvy/cTrfAusigrNre4raoH28r56OD2hlHhZFFjY+OBULjjmu5LjqOLXP8xYikCm6v4 Ke/t/iIgnF5Aap1+jfq0TTr8KcSx5lCNMKAFH+kjoO050Oifw7L51AQyLHVV8AWO7zu4 DcVGZFD7GhULfxv9rvSJBmSd3MvXnA56Y0UeLDENeGOQLV8VjLB/CbejTaVo1E1HNH66 GBuzodMkZCf8fn7AEmFWfk5I9lY2j1i5kcRli1zbjNmGzH6gkvxIVDvSyZPfbfseSqQy BrLw== X-Gm-Message-State: AJIora8aliHMukefHMV72fhzmvwU9YLoaffA9SgWKx1nAtVw2QKh0MrB S5pF9BB/1A2uEg+mWy37oNcUJ6dZyxA= X-Google-Smtp-Source: AGRyM1ukHF/75hMCkE1ceqIKqAwme//w8kYLgX2Q1lXtK73q2uRxhmeeO4GeXUnmM9oAotsl2fJIw5hKglI= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:f29:b0:435:c108:58f2 with SMTP id i41-20020a0564020f2900b00435c10858f2mr19006559eda.401.1656685502496; Fri, 01 Jul 2022 07:25:02 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:03 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-39-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 38/45] x86: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685503; a=rsa-sha256; cv=none; b=LqliZmjfZ20z/SGSPQlxre6GIfkB7bFv2Gi65fIDxxF+3PTI3QCrOUAVselz5FMwYxQvmt KjTgg0/+1VFpGhhouK2L4lMkmHrn51zG10BViefFVVA/ptKEZEZy/VwN9m5fjChMtkYV8k TI+JMbf+k8ROqbYCQ0b688e9XzdwQDs= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="SK4zuM/L"; spf=pass (imf05.hostedemail.com: domain of 3vgO_YgYKCN0FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3vgO_YgYKCN0FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685503; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7Mhpta8bleV/UMALfZOJOInTXQD6eorZ40ixMEZLH18=; b=mQ5p6wqDS2W9T30ZXby/QQyQ8G2lZ1JVd8BNLvDM7VDBk8kaHGYPEP4acsrIPK8jjtKZlU /LfNkFj4lxoCDDMsZM3wE+lfnA5ZmaC2uI/x6pohs9i9lgYrLMBEhJaldaVN96kH9M+pfI 8tzBnbUxg9mcrp2J/NgVXxsnzBuOWwQ= X-Rspam-User: X-Rspamd-Server: rspam04 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="SK4zuM/L"; spf=pass (imf05.hostedemail.com: domain of 3vgO_YgYKCN0FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3vgO_YgYKCN0FKHCDQFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: ptuy16zy3s5ya511u8orcqd36yh1kosh X-Rspamd-Queue-Id: 9EDB110004C X-HE-Tag: 1656685503-1334 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is needed to allow memory tools like KASAN and KMSAN see the memory accesses from the checksum code. Without CONFIG_GENERIC_CSUM the tools can't see memory accesses originating from handwritten assembly code. For KASAN it's a question of detecting more bugs, for KMSAN using the C implementation also helps avoid false positives originating from seemingly uninitialized checksum values. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I3e95247be55b1112af59dbba07e8cbf34e50a581 --- arch/x86/Kconfig | 4 ++++ arch/x86/include/asm/checksum.h | 16 ++++++++++------ arch/x86/lib/Makefile | 2 ++ 3 files changed, 16 insertions(+), 6 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index be0b95e51df66..4a5d0a0f54dea 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -324,6 +324,10 @@ config GENERIC_ISA_DMA def_bool y depends on ISA_DMA_API +config GENERIC_CSUM + bool + default y if KMSAN || KASAN + config GENERIC_BUG def_bool y depends on BUG diff --git a/arch/x86/include/asm/checksum.h b/arch/x86/include/asm/checksum.h index bca625a60186c..6df6ece8a28ec 100644 --- a/arch/x86/include/asm/checksum.h +++ b/arch/x86/include/asm/checksum.h @@ -1,9 +1,13 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#define _HAVE_ARCH_COPY_AND_CSUM_FROM_USER 1 -#define HAVE_CSUM_COPY_USER -#define _HAVE_ARCH_CSUM_AND_COPY -#ifdef CONFIG_X86_32 -# include +#ifdef CONFIG_GENERIC_CSUM +# include #else -# include +# define _HAVE_ARCH_COPY_AND_CSUM_FROM_USER 1 +# define HAVE_CSUM_COPY_USER +# define _HAVE_ARCH_CSUM_AND_COPY +# ifdef CONFIG_X86_32 +# include +# else +# include +# endif #endif diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile index f76747862bd2e..7ba5f61d72735 100644 --- a/arch/x86/lib/Makefile +++ b/arch/x86/lib/Makefile @@ -65,7 +65,9 @@ ifneq ($(CONFIG_X86_CMPXCHG64),y) endif else obj-y += iomap_copy_64.o +ifneq ($(CONFIG_GENERIC_CSUM),y) lib-y += csum-partial_64.o csum-copy_64.o csum-wrappers_64.o +endif lib-y += clear_page_64.o copy_page_64.o lib-y += memmove_64.o memset_64.o lib-y += copy_user_64.o From patchwork Fri Jul 1 14:23:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC2E5C433EF for ; Fri, 1 Jul 2022 14:25:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44DE76B007D; Fri, 1 Jul 2022 10:25:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D53B6B007E; Fri, 1 Jul 2022 10:25:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29D106B0085; Fri, 1 Jul 2022 10:25:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1A9006B007D for ; Fri, 1 Jul 2022 10:25:07 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EAB2333AC8 for ; Fri, 1 Jul 2022 14:25:06 +0000 (UTC) X-FDA: 79638753012.26.5219171 Received: from mail-lf1-f73.google.com (mail-lf1-f73.google.com [209.85.167.73]) by imf24.hostedemail.com (Postfix) with ESMTP id 7E1FB180047 for ; Fri, 1 Jul 2022 14:25:06 +0000 (UTC) Received: by mail-lf1-f73.google.com with SMTP id b2-20020a0565120b8200b00477a4532448so1188074lfv.22 for ; Fri, 01 Jul 2022 07:25:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MLo3LhenoRrRqNTI2r4mtWD3J0vGsm9bvjleadBpMkY=; b=anOUkKWtulGEKBkArfZgJyNh0cGJx9d7Cq23k1uQwOBaeBk8BsWD249jvt+waaUJ0W wFq6nFfeaRMqhTZBfdHnekYcU9Y6DlOKwhd7zz8Pwv5Mf7POcnXxXlRO9IdWwc7sHxG8 EJehSvYRYGgoI66twoBlGoYEACzjwkAkNmneLnfsKkPQFvcZRfWhgQAodIu6D09PjFk8 Icdf6H3+rBIz7HcFSJ3Z9tX02BV8EOgwwt+6yOVuoCYfJOPeNjzAXzZ5xayTIEyTNe+u 12rzA9yQKhB/zdr6mRJsJWTzWXROIdNbnWlGX7VZEuLugd3XmFTzxEYhZNDwatZ7SBn9 IL1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MLo3LhenoRrRqNTI2r4mtWD3J0vGsm9bvjleadBpMkY=; b=vro+b7/dWbtMDEUQXEXM1xz6sghV1SBjbpzs2XX7PJ7qMBb/12vcKkgwUo16PUOPs6 1J2mjTx58y7LzGgtdUxXf1/ebgEjeKcDLgvXBCluilq9ki4NHMfN6avvuCSkwilMzr6P k2CbtOrdGNRKNnhc0hNimrGsYqgiyRjD7CgYCSp+nqzNvQyxb7LXYmtM5oNbH0lollD8 EsAHXVg9tcH7doveou/5xX6R0h/xIlD1FRbzOttgycntIHOcXIfBhaRiZ+zJJ5brl30X d+uNlcQ10u8dtg8KhC7nzGoSxUElyfebU7HvFmS7Ve0hn4S4fexjAMTEumTQVpJS9SKP 9LOw== X-Gm-Message-State: AJIora+hKQo9hzPbtUnqKgNARhDM8yiCaUxxhTPP89y2tKkDUIT+hL3m AVoS2Q0VLwM7L17/HC9UT/T9OYF5yzI= X-Google-Smtp-Source: AGRyM1vhkmL+KohFz0MszVF5zMXVYF2SuHbeJwCxM549Y5UN3wtICSWeskVNj4IhdECyposUmY2NkdReg2A= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6512:e83:b0:47f:635c:3369 with SMTP id bi3-20020a0565120e8300b0047f635c3369mr8918326lfb.659.1656685504982; Fri, 01 Jul 2022 07:25:04 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:04 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-40-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 39/45] x86: fs: kmsan: disable CONFIG_DCACHE_WORD_ACCESS From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Andrey Konovalov ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=anOUkKWt; spf=pass (imf24.hostedemail.com: domain of 3wAO_YgYKCN8HMJEFSHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--glider.bounces.google.com designates 209.85.167.73 as permitted sender) smtp.mailfrom=3wAO_YgYKCN8HMJEFSHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685506; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MLo3LhenoRrRqNTI2r4mtWD3J0vGsm9bvjleadBpMkY=; b=4hLODbWvLGyflZCLkxEjYRtxQlkDRxpAIYWR9upGm2IuORv8aZ8ayw/eSxiRrN9mTXYcqN 8QL3yHIeUiRXlQaF5lJJtx9XoCT+YN13ZkLLvMlFk1l5hBh5q+5ElprXJHBwPyiXKMkIER la3jVs2Blr2uT1xBRB8xSyQmQ5HDnfA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685506; a=rsa-sha256; cv=none; b=qEsUKrWKCVwpx0rPIIf15eVVNsfQB/NiydbCvu1hffLI5luPdeWs1ucoZoVuBTJ+fHtvjF TpTjKBiz16vWZmzfSlO9s30puyMZL5Zoa71zcPSlbGLPhTWIpuF0csbL0umYni0gpOz95t 3tUMOKU0ln7dd/xRtpSoIiAbWGErCsg= X-Stat-Signature: tn61cbfg1yr3qaebi5ngm37id8abm44n X-Rspamd-Queue-Id: 7E1FB180047 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=anOUkKWt; spf=pass (imf24.hostedemail.com: domain of 3wAO_YgYKCN8HMJEFSHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--glider.bounces.google.com designates 209.85.167.73 as permitted sender) smtp.mailfrom=3wAO_YgYKCN8HMJEFSHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--glider.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-HE-Tag: 1656685506-518721 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: dentry_string_cmp() calls read_word_at_a_time(), which might read uninitialized bytes to optimize string comparisons. Disabling CONFIG_DCACHE_WORD_ACCESS should prohibit this optimization, as well as (probably) similar ones. Suggested-by: Andrey Konovalov Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I4c0073224ac2897cafb8c037362c49dda9cfa133 --- arch/x86/Kconfig | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 4a5d0a0f54dea..aadbb16a59f01 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -129,7 +129,9 @@ config X86 select CLKEVT_I8253 select CLOCKSOURCE_VALIDATE_LAST_CYCLE select CLOCKSOURCE_WATCHDOG - select DCACHE_WORD_ACCESS + # Word-size accesses may read uninitialized data past the trailing \0 + # in strings and cause false KMSAN reports. + select DCACHE_WORD_ACCESS if !KMSAN select DYNAMIC_SIGFRAME select EDAC_ATOMIC_SCRUB select EDAC_SUPPORT From patchwork Fri Jul 1 14:23:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AF0FCCA481 for ; Fri, 1 Jul 2022 14:25:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 202766B0075; Fri, 1 Jul 2022 10:25:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 18C036B007E; Fri, 1 Jul 2022 10:25:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02BE06B0085; Fri, 1 Jul 2022 10:25:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E7AA06B0075 for ; Fri, 1 Jul 2022 10:25:09 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id BA0BD120472 for ; Fri, 1 Jul 2022 14:25:09 +0000 (UTC) X-FDA: 79638753138.21.13F1895 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf13.hostedemail.com (Postfix) with ESMTP id 2A63020012 for ; Fri, 1 Jul 2022 14:25:08 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id e24-20020a05600c219800b003a0471b1904so3125482wme.1 for ; Fri, 01 Jul 2022 07:25:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=l9HW4hvq2CQJApDpi6peUok9IhcNK11lvxcg04+BYhI=; b=m0zQVBcwGsjkgBeyzji8njbsYom2zyjvffvZ6f53ygK/a73Sef45vu34Ym/f3GwiY6 24n4n5Tlgq2qKw2wFpgxSipNYxkrLnqPxJkpsZM0TLgQ2V+ISia+unOFzoAz0KadaQq4 AvfYtgG93hXBLXmtu/JrK/Ct51BvQc3+jnECwKhDga5fUg0X6Ao4l5Ye7I/3vzo4ftFo GeOl93a20QvxkwE/6EI17QYzXpZVp27n/+oALgi5SCFfQ+HUJc4i0Oa+cDwovsuOxHTR +8xmXrHToewGWfcV26T9eZU1Ic9Ws7op5A1NuDcoULUqufGrU4oSyUyGYH/L6tZRdLvq YiQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=l9HW4hvq2CQJApDpi6peUok9IhcNK11lvxcg04+BYhI=; b=4vkstlaWGfLXEI/AScLXyaFyzSt4B3GjWAYztR1H4D+gEmw4j8McfvVqhxzuXP1Qwu 0oKHRon+YBGqNSm/6a6FH7fnsdlcQd2pszbgcyDLllqFDuvpaf/UOu4PN7PbFwPdxGHy rv9eMpCsfp+h3QBuEQ3FO/oMeeMDhANIBbJokpSi9qjDITG4HaUzC7VNO+IVvv3utUtX Jees8kwKqjrK5qvvXKhsXyd2T7V0KiPGyfElBej+ccrjZSsZDIPnQY6BCssIrqklvdPl SeaAC0PeB19GfaM4bQCSfMKdZtnWxz6MdyLVGavCwYq3Z/jpP45kIDmMriLiWvC/idBa ItRw== X-Gm-Message-State: AJIora+k2aZiDrnDY6SNTToZIBDZByrLTetBjlajKDs0kqhFm1G/p6uN g3f7SgO+38HE8qTZlFXe2Nzgi/P/mDU= X-Google-Smtp-Source: AGRyM1u2TNc3xlar4igzHbbMT3NVlXpY+dQRzXKImVUAgX/bRcfEAIkZdgH1TPrbfWv83OzrtLLw5K1VVl4= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6000:10c4:b0:21b:8ea4:a27a with SMTP id b4-20020a05600010c400b0021b8ea4a27amr14086444wrx.575.1656685507805; Fri, 01 Jul 2022 07:25:07 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:05 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-41-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 40/45] x86: kmsan: don't instrument stack walking functions From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685509; a=rsa-sha256; cv=none; b=O7oDTdbjzutw1+V87MbRlGkcPfCtx5eovJLFoRkH2/dyWsUNi1qumPNphScIdb6VkHkACT 1s8sSiiAsc2mYOUiR2ZkFZuk4DiqHMrcax8gc4ahmd2noL7GdWUvJMm4jEYGFwES0KPWzs SjnfWU/xoKQjNhqWVsq4eBhDkXWRWH4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=m0zQVBcw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3wwO_YgYKCOIKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3wwO_YgYKCOIKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685509; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l9HW4hvq2CQJApDpi6peUok9IhcNK11lvxcg04+BYhI=; b=TZWXpsZYE534pwPkGhlfezbk7JvFjWJYASvx7wMTgPANTRXOI2CnA/t9NzU1BhJv95Uzy0 pXYrDSfo6HjZg981JAHJR0fV6IX6/JmzxnKeD60vqK1+B+QX8vdu5qJcc/Mk3hA/hLRUD0 Xb8r4yYrXM3VPSgpEMyhmmFuoL+oC7s= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 2A63020012 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=m0zQVBcw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3wwO_YgYKCOIKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3wwO_YgYKCOIKPMHIVKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: oczmruz9sspt43fb8ritg5brg7gixje3 X-HE-Tag: 1656685508-450617 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Upon function exit, KMSAN marks local variables as uninitialized. Further function calls may result in the compiler creating the stack frame where these local variables resided. This results in frame pointers being marked as uninitialized data, which is normally correct, because they are not stack-allocated. However stack unwinding functions are supposed to read and dereference the frame pointers, in which case KMSAN might be reporting uses of uninitialized values. To work around that, we mark update_stack_state(), unwind_next_frame() and show_trace_log_lvl() with __no_kmsan_checks, preventing all KMSAN reports inside those functions and making them return initialized values. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I7001eaed630277e8d2ddaff1d6f223d54e997a6f --- arch/x86/kernel/dumpstack.c | 6 ++++++ arch/x86/kernel/unwind_frame.c | 11 +++++++++++ 2 files changed, 17 insertions(+) diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c index afae4dd774951..476eb504084e4 100644 --- a/arch/x86/kernel/dumpstack.c +++ b/arch/x86/kernel/dumpstack.c @@ -177,6 +177,12 @@ static void show_regs_if_on_stack(struct stack_info *info, struct pt_regs *regs, } } +/* + * This function reads pointers from the stack and dereferences them. The + * pointers may not have their KMSAN shadow set up properly, which may result + * in false positive reports. Disable instrumentation to avoid those. + */ +__no_kmsan_checks static void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs, unsigned long *stack, const char *log_lvl) { diff --git a/arch/x86/kernel/unwind_frame.c b/arch/x86/kernel/unwind_frame.c index 8e1c50c86e5db..d8ba93778ae32 100644 --- a/arch/x86/kernel/unwind_frame.c +++ b/arch/x86/kernel/unwind_frame.c @@ -183,6 +183,16 @@ static struct pt_regs *decode_frame_pointer(unsigned long *bp) } #endif +/* + * While walking the stack, KMSAN may stomp on stale locals from other + * functions that were marked as uninitialized upon function exit, and + * now hold the call frame information for the current function (e.g. the frame + * pointer). Because KMSAN does not specifically mark call frames as + * initialized, false positive reports are possible. To prevent such reports, + * we mark the functions scanning the stack (here and below) with + * __no_kmsan_checks. + */ +__no_kmsan_checks static bool update_stack_state(struct unwind_state *state, unsigned long *next_bp) { @@ -250,6 +260,7 @@ static bool update_stack_state(struct unwind_state *state, return true; } +__no_kmsan_checks bool unwind_next_frame(struct unwind_state *state) { struct pt_regs *regs; From patchwork Fri Jul 1 14:23:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58CD5C43334 for ; Fri, 1 Jul 2022 14:25:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E38796B0073; Fri, 1 Jul 2022 10:25:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0E756B0074; Fri, 1 Jul 2022 10:25:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3C776B007E; Fri, 1 Jul 2022 10:25:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B3A4F6B0073 for ; Fri, 1 Jul 2022 10:25:11 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 926A8804AF for ; Fri, 1 Jul 2022 14:25:11 +0000 (UTC) X-FDA: 79638753222.13.D413762 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf24.hostedemail.com (Postfix) with ESMTP id 30FCA180007 for ; Fri, 1 Jul 2022 14:25:11 +0000 (UTC) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-31ba792c793so20271477b3.20 for ; Fri, 01 Jul 2022 07:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=kErHaO8c2VDyOovKMTTPjLnffjCar7lX0B4oxY8ZYJM=; b=QikpwS8HSBicv3aiX5l18DFFItV5cAfTWljYQPRMj2W7svuU9foj5CYwNhPW5g1+1u bB4glo+uMZCRum5mXol+jHpQv/lgXI3n7dJV/CM+AV/P8K/r9Gy2E3HtcxQY9ID9PHSC wVw80I52aTXRRhG6Ci/za3zw0SD1eIdJQitn35q5f/DsJHwTzK0rEhzr2LJtogI15qw/ PNVnInhl+jBMMMMjNB1XrOY7EpC2Lh+bCIf6lfRBYGnQduk/EEoxldI491nxYD8nls9J Xv49bFK0wQhLo1KkR4mkWE/FmlIpB/UN96BNNMS/ZsSyzjXx8nx8GEKrJFs5/vreJQ9A EF9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=kErHaO8c2VDyOovKMTTPjLnffjCar7lX0B4oxY8ZYJM=; b=1dROglq/WqsvojUEIFnKJYmR2H8orDAWRq8ZMh+/asJ4CPxMGQR0WIJOZMttaDqy4U clSAKhm/AfT2XpRAUOs2RppDzsctbviuEUO5dIYogV+8tRcESl57vtB+YdLVodghYd2E 5LYI+emUZUQ6B98oARVrc9SJ6ObI50K2nAWDtSnJF3+bZdjKevZpd2Em8JOtnsKskgEs HQzyIv1XpeM/oL+R2X9HzxFmvBYKuJhCt2XrtWuES2UAsPfFbQGXGT9kRUw32MLUS0iI JfI1scyUP6gbNMhp5y/oxL5oYSNV5N8zEa06KUkxRgkF8uIVhiATWqo4Hdcnk5F8HCRS dPdg== X-Gm-Message-State: AJIora+vq9JkgOf/24B+Og5wOFj5lBn5lXYKyd+U4ojVu0WebRyX4yM9 svYqMyIwPEvdXvo+oCb7K2lt8pMRpHA= X-Google-Smtp-Source: AGRyM1sNDBrS8JsvaLtaQ5Nuve5a/xxYqM8vAs8mLC9hMfdeY1tED78m34uqRhiFnJFi3MqtOhAJlCD8K7A= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a81:d82:0:b0:31b:fd6f:9005 with SMTP id 124-20020a810d82000000b0031bfd6f9005mr16667693ywn.389.1656685510371; Fri, 01 Jul 2022 07:25:10 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:06 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-42-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 41/45] entry: kmsan: introduce kmsan_unpoison_entry_regs() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685511; a=rsa-sha256; cv=none; b=vO4KrHPfT7vl/32+AQvfergNmSO8yd2TUEG7OtPKLwxsEmkkwLvvFI3gaZBOScjE7BiO8B jkh+FKB7Qmh3qSsocvGxttCpEv6lbd8DOJ56SieCCWnVV4zJDFVmNY2BKbWBPPBF2BTI2h CNWXSaB0M1xdIN7CFcZkP+Xqu0V/VZw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=QikpwS8H; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3xgO_YgYKCOUNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3xgO_YgYKCOUNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685511; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kErHaO8c2VDyOovKMTTPjLnffjCar7lX0B4oxY8ZYJM=; b=cD/pG8taKelEBMQnTCW4/Y/aUQin4gJYA+YOVlv69ZiS7u6nEblpU9ITKsUzUSxpKlAgls /pxjTpUtE7In59DUfpQwYF4xJYkicEZYBFj8QmGKjcNVfyeZM/27vHnLmE2+s5bvs6n5ad dsRNZ7KQ2pAbavnaz3rJD14Oz7SNwD0= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 30FCA180007 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=QikpwS8H; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3xgO_YgYKCOUNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3xgO_YgYKCOUNSPKLYNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: 361duz6rhho7y99ho5k16eg6w7qbc14s X-HE-Tag: 1656685511-155763 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: struct pt_regs passed into IRQ entry code is set up by uninstrumented asm functions, therefore KMSAN may not notice the registers are initialized. kmsan_unpoison_entry_regs() unpoisons the contents of struct pt_regs, preventing potential false positives. Unlike kmsan_unpoison_memory(), it can be called under kmsan_in_runtime(), which is often the case in IRQ entry code. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/Ibfd7018ac847fd8e5491681f508ba5d14e4669cf --- include/linux/kmsan.h | 15 +++++++++++++++ kernel/entry/common.c | 5 +++++ mm/kmsan/hooks.c | 27 +++++++++++++++++++++++++++ 3 files changed, 47 insertions(+) diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h index e8b5c306c4aa1..c4412622b9a78 100644 --- a/include/linux/kmsan.h +++ b/include/linux/kmsan.h @@ -246,6 +246,17 @@ void kmsan_handle_dma_sg(struct scatterlist *sg, int nents, */ void kmsan_handle_urb(const struct urb *urb, bool is_out); +/** + * kmsan_unpoison_entry_regs() - Handle pt_regs in low-level entry code. + * @regs: struct pt_regs pointer received from assembly code. + * + * KMSAN unpoisons the contents of the passed pt_regs, preventing potential + * false positive reports. Unlike kmsan_unpoison_memory(), + * kmsan_unpoison_entry_regs() can be called from the regions where + * kmsan_in_runtime() returns true, which is the case in early entry code. + */ +void kmsan_unpoison_entry_regs(const struct pt_regs *regs); + #else static inline void kmsan_init_shadow(void) @@ -342,6 +353,10 @@ static inline void kmsan_handle_urb(const struct urb *urb, bool is_out) { } +static inline void kmsan_unpoison_entry_regs(const struct pt_regs *regs) +{ +} + #endif #endif /* _LINUX_KMSAN_H */ diff --git a/kernel/entry/common.c b/kernel/entry/common.c index 032f164abe7ce..055d3bdb0442c 100644 --- a/kernel/entry/common.c +++ b/kernel/entry/common.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -24,6 +25,7 @@ static __always_inline void __enter_from_user_mode(struct pt_regs *regs) user_exit_irqoff(); instrumentation_begin(); + kmsan_unpoison_entry_regs(regs); trace_hardirqs_off_finish(); instrumentation_end(); } @@ -352,6 +354,7 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs) lockdep_hardirqs_off(CALLER_ADDR0); rcu_irq_enter(); instrumentation_begin(); + kmsan_unpoison_entry_regs(regs); trace_hardirqs_off_finish(); instrumentation_end(); @@ -367,6 +370,7 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs) */ lockdep_hardirqs_off(CALLER_ADDR0); instrumentation_begin(); + kmsan_unpoison_entry_regs(regs); rcu_irq_enter_check_tick(); trace_hardirqs_off_finish(); instrumentation_end(); @@ -452,6 +456,7 @@ irqentry_state_t noinstr irqentry_nmi_enter(struct pt_regs *regs) rcu_nmi_enter(); instrumentation_begin(); + kmsan_unpoison_entry_regs(regs); trace_hardirqs_off_finish(); ftrace_nmi_enter(); instrumentation_end(); diff --git a/mm/kmsan/hooks.c b/mm/kmsan/hooks.c index 9aecbf2825837..c7528bcbb2f91 100644 --- a/mm/kmsan/hooks.c +++ b/mm/kmsan/hooks.c @@ -358,6 +358,33 @@ void kmsan_unpoison_memory(const void *address, size_t size) } EXPORT_SYMBOL(kmsan_unpoison_memory); +/* + * Version of kmsan_unpoison_memory() that can be called from within the KMSAN + * runtime. + * + * Non-instrumented IRQ entry functions receive struct pt_regs from assembly + * code. Those regs need to be unpoisoned, otherwise using them will result in + * false positives. + * Using kmsan_unpoison_memory() is not an option in entry code, because the + * return value of in_task() is inconsistent - as a result, certain calls to + * kmsan_unpoison_memory() are ignored. kmsan_unpoison_entry_regs() ensures that + * the registers are unpoisoned even if kmsan_in_runtime() is true in the early + * entry code. + */ +void kmsan_unpoison_entry_regs(const struct pt_regs *regs) +{ + unsigned long ua_flags; + + if (!kmsan_enabled) + return; + + ua_flags = user_access_save(); + kmsan_internal_unpoison_memory((void *)regs, sizeof(*regs), + KMSAN_POISON_NOCHECK); + user_access_restore(ua_flags); +} +EXPORT_SYMBOL(kmsan_unpoison_entry_regs); + void kmsan_check_memory(const void *addr, size_t size) { if (!kmsan_enabled) From patchwork Fri Jul 1 14:23:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39A8AC43334 for ; Fri, 1 Jul 2022 14:25:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8BDB6B0074; Fri, 1 Jul 2022 10:25:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C3B076B007E; Fri, 1 Jul 2022 10:25:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B01656B0081; Fri, 1 Jul 2022 10:25:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A09946B0074 for ; Fri, 1 Jul 2022 10:25:14 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 81B873455B for ; Fri, 1 Jul 2022 14:25:14 +0000 (UTC) X-FDA: 79638753348.04.1BDB6C1 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by imf14.hostedemail.com (Postfix) with ESMTP id 1CF86100037 for ; Fri, 1 Jul 2022 14:25:13 +0000 (UTC) Received: by mail-ej1-f74.google.com with SMTP id sd14-20020a1709076e0e00b0072a7c5a08f4so843007ejc.21 for ; Fri, 01 Jul 2022 07:25:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=NuZxaTt6SuBtelFew5LXxi86pENufXvsmy2qQXyoE6c=; b=tgfjQq+iQ/ysonFckb3Z2u6TJ9CEvDDKEzA5gkb+C4EKFnWD9TLdpPFWxQRtHD88Jg RYLB2Yb0k9id72G7IVJ8H91BQ7wXcKdXZ0Azu48hd+fSPpcrnVbABx59b2la2dgB1ZDR AAPr3HcC1/Yh4R+BkF5CNapSwZyhy3/pHIFwHkw79AVAJZ22Y8th2w7xgwft5pumnLsD 2R5Dmthk3JjdCZyOh+gBs/AieCejbEIH0zIdbihuM0GsaPffCE/o0zvm99bCYmyOaQgR TMh+X88DvOG6cOtWBuy2Dm2xKw9NnmVgVGZuSBeVLfUcMKHQAVfKgpF+8OdRVLd3hmjI vXVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=NuZxaTt6SuBtelFew5LXxi86pENufXvsmy2qQXyoE6c=; b=6Q11sWVWObCB17ifRKFnmIlPk2fh7dEv8AQ5ltDw8WHsZtjBNyHirlcOKKI3Pd6Nqd TQgweSNdPkssBYUYKV4NxZcPvLBkly60X4LNtC1Mtj6iBbNYsReWNrZViRtqO+pBj0zC L57ZiPwdUba4LebI9Up/m3X9kdh7C0dx690WZ4Hyy6RCdxLg+6X1HS5puMJGJTS7pYWr 3yIfRCpz/cwRRHeXB5PIjOfJD/9rUZSMCJeRTLmwwyWDjMyNSVXOPKNgKvRq27Y5RPyI slfOd9jGk532Wblkh/82jtfbSLGCx631uaYK0lWDP0MNIdoXXMFLAP+WWrKqAoUq9rre zn0w== X-Gm-Message-State: AJIora8y9P62Eu+ANp0iNV1TeRUXgMut10VmX0sGS6LjEZyXAySfBhSZ 89KaQ/h99B1FUuie29zC6WdAJR/JCgU= X-Google-Smtp-Source: AGRyM1spcb+a3miQij6KHvEFDixhz2AvZj0AcfEskbi1pdPvcSJdu4nWWmlOh1NQQeaAeMBr5KVjUAW0uyU= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a17:907:2704:b0:72a:596f:8b9f with SMTP id w4-20020a170907270400b0072a596f8b9fmr9611410ejk.761.1656685512839; Fri, 01 Jul 2022 07:25:12 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:07 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-43-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 42/45] bpf: kmsan: initialize BPF registers with zeroes From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685514; a=rsa-sha256; cv=none; b=EUO/7KT/TLDHJR1u6W1HWyDLG9aJDPLjDvMLbXJ0fuhgwAR8OemKRKkweDfmezXDf0t9F2 kutRuh5fy/rwAUsDSMlUJeiqiJ7/dMPOxMyzym0rFNFuW7+fdQBGtgXoeOdAbop7hVQGZi hF49W/hE1N5O8nkRNZozf76Us0BCjdY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=tgfjQq+i; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3yAO_YgYKCOcPURMNaPXXPUN.LXVURWdg-VVTeJLT.XaP@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3yAO_YgYKCOcPURMNaPXXPUN.LXVURWdg-VVTeJLT.XaP@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685514; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NuZxaTt6SuBtelFew5LXxi86pENufXvsmy2qQXyoE6c=; b=VNxA698C4WL8voi9hpfeI21F/zpdrGBnzRPk31WkVCmDYo2SGj7KgUmIilNsvhAlZKQhjq V/uQ41Nz+bFi+zUWqoWes2IqDadau5wuA2lUMj8JQKa74ObhTRW6SSix9ojGV+Bk+wr+NE vbsEiPiepNrpknpvMYaSCVrLlqYi3vE= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1CF86100037 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=tgfjQq+i; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3yAO_YgYKCOcPURMNaPXXPUN.LXVURWdg-VVTeJLT.XaP@flex--glider.bounces.google.com designates 209.85.218.74 as permitted sender) smtp.mailfrom=3yAO_YgYKCOcPURMNaPXXPUN.LXVURWdg-VVTeJLT.XaP@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: 9wtnd1bgozob1wmza1i4i85p79p9e3m4 X-HE-Tag: 1656685513-752900 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When executing BPF programs, certain registers may get passed uninitialized to helper functions. E.g. when performing a JMP_CALL, registers BPF_R1-BPF_R5 are always passed to the helper, no matter how many of them are actually used. Passing uninitialized values as function parameters is technically undefined behavior, so we work around it by always initializing the registers. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I40f39d26232b14816c14ba64a0ea4a8f336f2675 --- kernel/bpf/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 5f6f3f829b368..0ba7dd90a2ab3 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -2039,7 +2039,7 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn) static unsigned int PROG_NAME(stack_size)(const void *ctx, const struct bpf_insn *insn) \ { \ u64 stack[stack_size / sizeof(u64)]; \ - u64 regs[MAX_BPF_EXT_REG]; \ + u64 regs[MAX_BPF_EXT_REG] = {}; \ \ FP = (u64) (unsigned long) &stack[ARRAY_SIZE(stack)]; \ ARG1 = (u64) (unsigned long) ctx; \ From patchwork Fri Jul 1 14:23:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FD4BC43334 for ; Fri, 1 Jul 2022 14:25:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC6726B0071; Fri, 1 Jul 2022 10:25:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B76696B007E; Fri, 1 Jul 2022 10:25:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A174C6B0081; Fri, 1 Jul 2022 10:25:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9159F6B0071 for ; Fri, 1 Jul 2022 10:25:17 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 65758205D4 for ; Fri, 1 Jul 2022 14:25:17 +0000 (UTC) X-FDA: 79638753474.15.C8B654C Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf05.hostedemail.com (Postfix) with ESMTP id 0B75F100054 for ; Fri, 1 Jul 2022 14:25:16 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id x8-20020a056402414800b0042d8498f50aso1888171eda.23 for ; Fri, 01 Jul 2022 07:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=/n5egAWhgjqHJF7YploSYMAs3TFFsaM1JiTS0Bn58vA=; b=Q64KAy7Oa2nU5QEGyiPc+WuoNDDnAlGeuwj9Gnp4CxbUOEi4OI3mY3DVAxjpBiKgfy oa7qyOyW4RhD6IyTiejiC7YtXEPcWRYAw5gxMRCHiOP9br49u9DBYFU0wT8izXo989yb SsJ683z3889b7saADkdB2NmfOQkA9W3PR5kSJXdTK6rlAIAn/matyXusT0WxYIYR1PdC nsJAkTFlORzo0vstvDMrGlx6qd2DiIZ3JP81lA81CBvxEItVe4LQpcoX5OTFruunfhnn 6Il7stXu45y2U2bIykk0AFttgTTdTXs+YT4Idf4maBbQ8qhqqM+TVsUW/CeEtqqLOi3k x6Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=/n5egAWhgjqHJF7YploSYMAs3TFFsaM1JiTS0Bn58vA=; b=jU+u4VhMhUNjmBSSuwdcy2raG/7gotzV78jwOA+8KB3KOt7HVH2ZUX+2/QyL+yTB33 kOw/eD+nZUKruL6MSmhgrepdknn8m7WnY/OIQwv30McmawZkuzckg7E/IBwlOOUln0SM VitF5rIYkzRBkM+Dnvj2/TS+QV6tyT1fULFHFpXETHRkN/MfbzSXF46HrrLXND/dRBYF Oz7jJ2+F68LaCqeII17RggjMCC7WM+8nCd65b9GG8D+nOCUvKddBZo5hxKwQkrweHoEH 8XW/DAzNWdrD6a5JU5qFKNvUdIGc2c1Jb0JSMommKkFqUq7SZQdzXdiYzqFOrGVV5ui8 ztYw== X-Gm-Message-State: AJIora8IZTk2t75so2Ik0SkpXtmpAL/v2Xpm3UIilM+aMLRU0/YW7dJ4 znmOk9/HXQ+RBGJ9X5SRoBkULPCScyA= X-Google-Smtp-Source: AGRyM1vUh8ZooMqVrhXc2xeGCFSNfOoXJqVl3ZPA1oCNI4qUloUIWKz76DRGg3RXhdunivVizxxEE/eITwE= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:51ca:b0:437:79a9:4dd with SMTP id r10-20020a05640251ca00b0043779a904ddmr19173589edd.319.1656685515729; Fri, 01 Jul 2022 07:25:15 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:08 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-44-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 43/45] namei: initialize parameters passed to step_into() From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Evgenii Stepanov , Linus Torvalds , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685517; a=rsa-sha256; cv=none; b=DJq32Jw0iMhTBAItyr6pPKvdy5b977l7bj08mJDOyNKv3b/saVgOer294gpX1l5Zt2HZ/j hSqri981kQJaWpbKJ3M5wh6n/8MAhaY/7jYAqJ00Du+G9g6G+vbLqeu3oEy9YSa39/V9oY TRHFerfsATzyvqasAM5+doXUPC9+rGU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Q64KAy7O; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3ywO_YgYKCOoSXUPQdSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3ywO_YgYKCOoSXUPQdSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/n5egAWhgjqHJF7YploSYMAs3TFFsaM1JiTS0Bn58vA=; b=AYq8GtWH7wgKUcyON11X/Pm+goQXrTWCo5D6rxQiYVbpKAFu2tVtgr4qoDPaxaT0eQIWQ4 5RBK9i6jbBVCyhS1R3QWGcV5Nid9/ZHz3Ev+0OPhjCAujObM2i1z5bzZQ0IxTUtXJkQrHv k6uTNCM2MKD2hRwImf5E/h2e/U8jZPY= X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 0B75F100054 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Q64KAy7O; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3ywO_YgYKCOoSXUPQdSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=3ywO_YgYKCOoSXUPQdSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--glider.bounces.google.com X-Rspam-User: X-Stat-Signature: k35qk97jkw1ci51tpietwz7joj1eu68f X-HE-Tag: 1656685516-166560 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Under certain circumstances initialization of `unsigned seq` and `struct inode *inode` passed into step_into() may be skipped. In particular, if the call to lookup_fast() in walk_component() returns NULL, and lookup_slow() returns a valid dentry, then the `seq` and `inode` will remain uninitialized until the call to step_into() (see [1] for more info). Right now step_into() does not use these uninitialized values, yet passing uninitialized values to functions is considered undefined behavior (see [2]). To fix that, we initialize `seq` and `inode` at definition. [1] https://github.com/ClangBuiltLinux/linux/issues/1648#issuecomment-1146608063 [2] https://lore.kernel.org/linux-toolchains/CAHk-=whjz3wO8zD+itoerphWem+JZz4uS3myf6u1Wd6epGRgmQ@mail.gmail.com/ Cc: Evgenii Stepanov Cc: Kees Cook Cc: Linus Torvalds Cc: Marco Elver Cc: Nathan Chancellor Cc: Nick Desaulniers Cc: Peter Zijlstra Cc: Segher Boessenkool Cc: Thomas Gleixner Cc: Vitaly Buka Cc: linux-kernel@vger.kernel.org Cc: linux-toolchains@vger.kernel.org Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I94d4e8cc1f0ecc7174659e9506ce96aaf2201d0a --- fs/namei.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index 1f28d3f463c3b..6b39dfd3b41bc 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1995,8 +1995,8 @@ static const char *handle_dots(struct nameidata *nd, int type) static const char *walk_component(struct nameidata *nd, int flags) { struct dentry *dentry; - struct inode *inode; - unsigned seq; + struct inode *inode = NULL; + unsigned seq = 0; /* * "." and ".." are special - ".." especially so because it has * to be able to know about the current root directory and @@ -3393,8 +3393,8 @@ static const char *open_last_lookups(struct nameidata *nd, struct dentry *dir = nd->path.dentry; int open_flag = op->open_flag; bool got_write = false; - unsigned seq; - struct inode *inode; + unsigned seq = 0; + struct inode *inode = NULL; struct dentry *dentry; const char *res; From patchwork Fri Jul 1 14:23:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D585FCCA479 for ; Fri, 1 Jul 2022 14:25:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7803E6B007D; Fri, 1 Jul 2022 10:25:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 730886B007E; Fri, 1 Jul 2022 10:25:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D1546B0081; Fri, 1 Jul 2022 10:25:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4C3306B007E for ; Fri, 1 Jul 2022 10:25:20 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2CB5420592 for ; Fri, 1 Jul 2022 14:25:20 +0000 (UTC) X-FDA: 79638753600.12.86E98B9 Received: from mail-lf1-f74.google.com (mail-lf1-f74.google.com [209.85.167.74]) by imf17.hostedemail.com (Postfix) with ESMTP id C39564003F for ; Fri, 1 Jul 2022 14:25:19 +0000 (UTC) Received: by mail-lf1-f74.google.com with SMTP id p36-20020a05651213a400b004779d806c13so1178712lfa.10 for ; Fri, 01 Jul 2022 07:25:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=iGvD4mfjvjSWX+nv6535woTSPAyHTTcU/sEMimyHpxI=; b=P7QKDp3hV6WoKSuU7RWtHYNnUzKsYPMuoVFYGC5CAEPmJ0nl1tQiRY+9OYfXLY42wB nFSOJB8BBflVDmdDuWSkOCm55g6bwX9dQkpXj4xCR6s6JCEls/jHSMNPkv+lVDZh5nKs hUARMca5KeTe+Ir0fpc6ZOi4Uq5KQZl/WLZTZOatCEOeYcao1NRgQlKKrm6iBVPu3X1Z JYhJ2HBgIVmRPYmPgcVuYdtvK/4rJXCJLuWe7FZ63qgHZC3DxwnGi+wOpBYOkbuzJh/f yrpShYElychLxpVWGpPbFszkIP0ovqs29G1ENpHftNUWpa2XkYBhGLzu4Zi+nYeVFrno cQCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=iGvD4mfjvjSWX+nv6535woTSPAyHTTcU/sEMimyHpxI=; b=S7K4L3i+42iqfBjfM2mOmhH2SB3hDlwBwhODcvikjfRD3xGQSCIkbuIRIi/onPyBy4 NrUUWx+InqAzbTLz7QGSDNVteuEK0CVYzkrjvEe0+NrOWHmGOwQ45rbgTT+JlS+jWx8b 31QPCLTVc2ruAGCTmgjkHGC+PD/QYGblKQU50qfnpkJXZYm6D8hJzk5S8HVbverTZ+OY w5ZFDYpI6oj6QJooGZGW7AdrYBfJAWRW75DGmnbefpaApdkr6gG7e3HxlQs9nCnKDE06 UPPr7zD5830bsR7up/PtfIRPiYxYgrf+26utGqKkijvJJsDO3oUrZ9sBcnh3i5fIeoH5 lnpw== X-Gm-Message-State: AJIora+lUe6vg6pVjR8s1RzgT7AQVgbtdz/tuq92xHuA9r064nK76xdR Qq/I4ro4IYCwikSdP1Wyx7lrxuKSXJI= X-Google-Smtp-Source: AGRyM1s7QtTcOviesKYVgedtf1csh4E3jTJYd2DFNUsU1Lb1euL9e2M1gyfvpy9SPS5aVSID3ksrqFaDH6I= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6512:4c3:b0:47f:6f6e:a7e7 with SMTP id w3-20020a05651204c300b0047f6f6ea7e7mr9859006lfq.674.1656685518270; Fri, 01 Jul 2022 07:25:18 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:09 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-45-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 44/45] mm: fs: initialize fsdata passed to write_begin/write_end interface From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iGvD4mfjvjSWX+nv6535woTSPAyHTTcU/sEMimyHpxI=; b=GWh18qXUx1JMLNE8ZakDimyCJQ0ZuK3uJpR+oo2b4jdCZcjgS5zKt9FWetsGyeuWmZRIGX FY0jRjtOK/m1STYhoJo8yiJS1me4CFLEN2DqRVjfMmJjVLvNhYpQhUZQUOl22mhQ5MsDHX ejKcU+ZwD/qw7y7kyxFHXoitwtgk+Y0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685519; a=rsa-sha256; cv=none; b=R8ILuMi57Zv3OlnK0muj3E+hBS3s865vDNXeWmBiWyM0Ww2SBMA8UeNAtDHrlORaJoGgTL qLwuVBr32anyH0iYIQbpv4gFpPXdrieaYnXjLNqXu6hkDV1tl8cZw1QRn3gOT2Wr0WZkxX y+xmlo8WzREB7wyNKKx7HKbbggofx14= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=P7QKDp3h; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3zgO_YgYKCO0VaXSTgVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--glider.bounces.google.com designates 209.85.167.74 as permitted sender) smtp.mailfrom=3zgO_YgYKCO0VaXSTgVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--glider.bounces.google.com X-Stat-Signature: fb68xdp74cdqes7kxpgj59jwysnoxg7e X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=P7QKDp3h; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3zgO_YgYKCO0VaXSTgVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--glider.bounces.google.com designates 209.85.167.74 as permitted sender) smtp.mailfrom=3zgO_YgYKCO0VaXSTgVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--glider.bounces.google.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C39564003F X-HE-Tag: 1656685519-964464 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Functions implementing the a_ops->write_end() interface accept the `void *fsdata` parameter that is supposed to be initialized by the corresponding a_ops->write_begin() (which accepts `void **fsdata`). However not all a_ops->write_begin() implementations initialize `fsdata` unconditionally, so it may get passed uninitialized to a_ops->write_end(), resulting in undefined behavior. Fix this by initializing fsdata with NULL before the call to write_begin(), rather than doing so in all possible a_ops implementations. This patch covers only the following cases found by running x86 KMSAN under syzkaller: - generic_perform_write() - cont_expand_zero() and generic_cont_expand_simple() - page_symlink() Other cases of passing uninitialized fsdata may persist in the codebase. Signed-off-by: Alexander Potapenko --- Link: https://linux-review.googlesource.com/id/I414f0ee3a164c9c335d91d82ce4558f6f2841471 --- fs/buffer.c | 4 ++-- fs/namei.c | 2 +- mm/filemap.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 898c7f301b1b9..d014009cff941 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2349,7 +2349,7 @@ int generic_cont_expand_simple(struct inode *inode, loff_t size) struct address_space *mapping = inode->i_mapping; const struct address_space_operations *aops = mapping->a_ops; struct page *page; - void *fsdata; + void *fsdata = NULL; int err; err = inode_newsize_ok(inode, size); @@ -2375,7 +2375,7 @@ static int cont_expand_zero(struct file *file, struct address_space *mapping, const struct address_space_operations *aops = mapping->a_ops; unsigned int blocksize = i_blocksize(inode); struct page *page; - void *fsdata; + void *fsdata = NULL; pgoff_t index, curidx; loff_t curpos; unsigned zerofrom, offset, len; diff --git a/fs/namei.c b/fs/namei.c index 6b39dfd3b41bc..5e3ff9d65f502 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -5051,7 +5051,7 @@ int page_symlink(struct inode *inode, const char *symname, int len) const struct address_space_operations *aops = mapping->a_ops; bool nofs = !mapping_gfp_constraint(mapping, __GFP_FS); struct page *page; - void *fsdata; + void *fsdata = NULL; int err; unsigned int flags; diff --git a/mm/filemap.c b/mm/filemap.c index ffdfbc8b0e3ca..72467f00f1916 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3753,7 +3753,7 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) unsigned long offset; /* Offset into pagecache page */ unsigned long bytes; /* Bytes to write to page */ size_t copied; /* Bytes copied from user */ - void *fsdata; + void *fsdata = NULL; offset = (pos & (PAGE_SIZE - 1)); bytes = min_t(unsigned long, PAGE_SIZE - offset, From patchwork Fri Jul 1 14:23:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 12903402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6B6EC43334 for ; Fri, 1 Jul 2022 14:25:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44C5C6B0075; Fri, 1 Jul 2022 10:25:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FDEB6B007E; Fri, 1 Jul 2022 10:25:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C4D26B0082; Fri, 1 Jul 2022 10:25:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1EC4F6B007E for ; Fri, 1 Jul 2022 10:25:23 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E4A0A34B2F for ; Fri, 1 Jul 2022 14:25:22 +0000 (UTC) X-FDA: 79638753684.17.7984213 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf16.hostedemail.com (Postfix) with ESMTP id 8EF9318006A for ; Fri, 1 Jul 2022 14:25:22 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id g8-20020a056402090800b00433940d207eso1899121edz.1 for ; Fri, 01 Jul 2022 07:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=auy4Z9/QbdBH93W7clb5ShO/bvtQNUkdCMoxr4yxtk4=; b=TJFxAn5Ss516YvoMDGVEwjUVJwMCvGA1uys6DFajPPpbZIMmu2QzyGWPOotDWBpn+r rgIkzih/ujs5bTKljPLwUBtpQa/dNgrfp0MBxT1kb+eKYfRAVyELcJtbxVB7958xF9rm Zk0fIJMREitsgbyi1gy837q4R1mHa+3JGBT6oi1Hs5AApXttsWy/csNFxBANvSau0uZu me+bQ66jrcYrfHdDel7oJAdmULEw4S6O18cufBNY46EF+SYvAbZhuw7DOE9ygaIc+DUH epOsUEkbVsljZQLqnxiEbG1wf8XSFU+0Q/2K/o1oN5f4fsTtQ+6GgHHmMLvBksi3ftZb I9uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=auy4Z9/QbdBH93W7clb5ShO/bvtQNUkdCMoxr4yxtk4=; b=c1HM/CR9iFRmGMHCFaWZDebenlHX8FKQlMLFnU+g/TRqjKW8SRKOi/IBoBpFbKymIp 6vabwWbtCa9HQuevOVn0F+0fct9dzE9fNFQ8MPxO0Ep2PcCy8eV78686p0Ka2Yv0mZyn P0FAK3/Lt6uDw9ityIICVamZbxNCYVsC7ZIVS2SMBZpT/UTXqcuKxC1a9MndLTCZ4N3L 6xGtHbV05e1Y4UYdgTHATQYguUDnbePTQ8ZFYg0oD0h8eXXeX5YcfI1rN3vum+NBsawo Y9bpZ2PUAOMJ+QaD78A6JRtYjwWrQX11wkeP1VT3pjqKJIe8sz4a1dIPr0LLFWtSIwNw dzag== X-Gm-Message-State: AJIora9oYDnOVqdFvsPEdjle1Abhjefuj6/oOru5s7YP1aGpfkzem/SS YhRA3u/2KuFwupNwWkDnv+s4Xpx9FcQ= X-Google-Smtp-Source: AGRyM1sPr6BbVcj2G+ZB+lPBZAJjGton9fbZSyRJE7DwNhWSxx5v5nK5h5ZIpUkHajs/dx3CGzAw9kjasBQ= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:a6f5:f713:759c:abb6]) (user=glider job=sendgmr) by 2002:a05:6402:4408:b0:435:9ed2:9be with SMTP id y8-20020a056402440800b004359ed209bemr18990092eda.81.1656685521229; Fri, 01 Jul 2022 07:25:21 -0700 (PDT) Date: Fri, 1 Jul 2022 16:23:10 +0200 In-Reply-To: <20220701142310.2188015-1-glider@google.com> Message-Id: <20220701142310.2188015-46-glider@google.com> Mime-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v4 45/45] x86: kmsan: enable KMSAN builds for x86 From: Alexander Potapenko To: glider@google.com Cc: Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656685522; a=rsa-sha256; cv=none; b=66QjF4DqhjWfWAiHl32dgn/rDSNyyU4dhctZg5TCOf4vd9fy4xlzv+webtowM1+YJCIYSl hpvjr+tbev9c4Q9ErgSmJspSbk1iQ7LxsQiJJ5LSkEynPrfqDsrQxzbbJEYlEk5udxO1pd 3JDNodsj1MEpimNKKQ/4kazqGspTrrE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TJFxAn5S; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 30QO_YgYKCPAYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=30QO_YgYKCPAYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656685522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=auy4Z9/QbdBH93W7clb5ShO/bvtQNUkdCMoxr4yxtk4=; b=OR2LQE0cIdC1/fbY/hY8lP7EFJunreAEEFnd85c5oN7bFB8gH7Qxo7Y/WMbyfX5BoDA2SO rqc8J+J5Cq+yPE0apBff/D6Vy0vipM0ROy9Sj9TML1UYSuDXSGQUA9Z/zTKC0EjR0FaWB8 0yknn2nEFvv/+MUlgm+//GzBKRbvvAA= Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TJFxAn5S; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of 30QO_YgYKCPAYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com designates 209.85.208.73 as permitted sender) smtp.mailfrom=30QO_YgYKCPAYdaVWjYggYdW.Ugedafmp-eecnSUc.gjY@flex--glider.bounces.google.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 8EF9318006A X-Stat-Signature: 6pjewequed4utq5agubhrcd9i75s5oxi X-Rspam-User: X-HE-Tag: 1656685522-261221 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make KMSAN usable by adding the necessary Kconfig bits. Also declare x86-specific functions checking address validity in arch/x86/include/asm/kmsan.h. Signed-off-by: Alexander Potapenko --- v4: -- per Marco Elver's request, create arch/x86/include/asm/kmsan.h and move arch-specific inline functions there. Link: https://linux-review.googlesource.com/id/I1d295ce8159ce15faa496d20089d953a919c125e --- arch/x86/Kconfig | 1 + arch/x86/include/asm/kmsan.h | 55 ++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 arch/x86/include/asm/kmsan.h diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index aadbb16a59f01..d1a601111b277 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -169,6 +169,7 @@ config X86 select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 select HAVE_ARCH_KFENCE + select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT diff --git a/arch/x86/include/asm/kmsan.h b/arch/x86/include/asm/kmsan.h new file mode 100644 index 0000000000000..a790b865d0a68 --- /dev/null +++ b/arch/x86/include/asm/kmsan.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * x86 KMSAN support. + * + * Copyright (C) 2022, Google LLC + * Author: Alexander Potapenko + */ + +#ifndef _ASM_X86_KMSAN_H +#define _ASM_X86_KMSAN_H + +#ifndef MODULE + +#include +#include + +/* + * Taken from arch/x86/mm/physaddr.h to avoid using an instrumented version. + */ +static inline bool kmsan_phys_addr_valid(unsigned long addr) +{ + if (IS_ENABLED(CONFIG_PHYS_ADDR_T_64BIT)) + return !(addr >> boot_cpu_data.x86_phys_bits); + else + return true; +} + +/* + * Taken from arch/x86/mm/physaddr.c to avoid using an instrumented version. + */ +static inline bool kmsan_virt_addr_valid(void *addr) +{ + unsigned long x = (unsigned long)addr; + unsigned long y = x - __START_KERNEL_map; + + /* use the carry flag to determine if x was < __START_KERNEL_map */ + if (unlikely(x > y)) { + x = y + phys_base; + + if (y >= KERNEL_IMAGE_SIZE) + return false; + } else { + x = y + (__START_KERNEL_map - PAGE_OFFSET); + + /* carry flag will be set if starting x was >= PAGE_OFFSET */ + if ((x > y) || !kmsan_phys_addr_valid(x)) + return false; + } + + return pfn_valid(x >> PAGE_SHIFT); +} + +#endif /* !MODULE */ + +#endif /* _ASM_X86_KMSAN_H */