From patchwork Wed Oct 30 14:22:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A771E1668 for ; Wed, 30 Oct 2019 14:22:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 73FA12087E for ; Wed, 30 Oct 2019 14:22:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tuSSH/RX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73FA12087E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3E3E26B0006; Wed, 30 Oct 2019 10:22:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 397656B000A; Wed, 30 Oct 2019 10:22:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25AB76B000C; Wed, 30 Oct 2019 10:22:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 028DE6B0006 for ; Wed, 30 Oct 2019 10:22:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id A6E261F10 for ; Wed, 30 Oct 2019 14:22:47 +0000 (UTC) X-FDA: 76100667174.26.roll29_356b1125ab15f X-Spam-Summary: 2,0,0,50d9235414b14d03,d41d8cd98f00b204,3tzy5xqykcmguzwrs5u22uzs.q20zw18b-00y9oqy.25u@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: roll29_356b1125ab15f X-Filterd-Recvd-Size: 4694 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:22:46 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id s9so1386678wrw.23 for ; Wed, 30 Oct 2019 07:22:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wepTYO8w6YbsFcNGVBbCvFTC7ov3ermwyIo9NfCllcM=; b=tuSSH/RXaOYvoaCX05Hm4ebL/1iuObTQEqwJVyYhTco2kzaPuGDZQJi5zHsS5aJRbW 7AqUYnjppQRVWp+rg4utFpxSSqVXDMy5FWmqFti3G5cBo15GdUHpoLkWu25xVOIMdhEM tQbG5unhrg+jLDMxQya0Yv0SY83bYZ0LOfRkepC0ldUl0OqeTWHf3iU7uufqhxemi6he y45n8QmE72CKBqHplqqY2qR/ye6uPq85OaYOIXV24yZhP0WJJ7YUzO/NW7XcnCc6B88s 9lRADRPxbf8pBdaaubR4i8mRLTZT9vStGE6HgxE8T8gU3MK/9bhGLKz48qv7qJY36FNQ uzMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wepTYO8w6YbsFcNGVBbCvFTC7ov3ermwyIo9NfCllcM=; b=IB9RRTB8en8gqy2Nsy2MNcv03tlSMtthXT4IoiDnXxdcHFctWA5Be2Olexsh2ly0i6 Rhqap+GkljKfF3RzouK/G5l6h1pzfDhsI3jF6qt5caRl37jeg2jh/0TrfJLIyzCfqGyo HeCDXuzpSqhI23sZtEzvGyS6s6YAbuV4t/JUnSxYnqXzBxrfEoNasezzEbpfhjQEXtUU 1FMLmnNp0+A+wtAR24m0d+PXfB2iXmIYVhBmT650vNeLpLnKDhKLQ3xCsIRVWKyGNytH R31iBI7wLexBKgjRIoly0Rrk3HxUjluP7bAQv+36U29zoQNAgQJCJK3lkK7HGhMfQnz9 S2yg== X-Gm-Message-State: APjAAAValLw9UsORoM/Z5PnZe2Bi0HLW7pbMvt1Ic9zLl8wR3OHOwx0C 27cGdPCtAV4HA/Be7w4fZGwYX2YjE60= X-Google-Smtp-Source: APXvYqxy8q0yVVe+377nMfdxrg0sNlwz5C3Wg71IQuaMfaeytnFg8++wVxM1NxCI4tyFBA8J1obBsBYqUSo= X-Received: by 2002:a05:6000:14a:: with SMTP id r10mr118378wrx.310.1572445365361; Wed, 30 Oct 2019 07:22:45 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:13 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-2-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 01/25] stackdepot: check depot_index before accessing the stack slab From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Avoid crashes on corrupted stack ids. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I0a0b38ed5057090696a2c6ff0be7cfcc24ae6738 --- lib/stackdepot.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index ed717dd08ff3..0d00d2ac0c4b 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -198,9 +198,22 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries) { union handle_parts parts = { .handle = handle }; - void *slab = stack_slabs[parts.slabindex]; + void *slab; size_t offset = parts.offset << STACK_ALLOC_ALIGN; - struct stack_record *stack = slab + offset; + struct stack_record *stack; + + if (parts.slabindex > depot_index) { + WARN(1, "slab index %d out of bounds (%d) for stack id %08x\n", + parts.slabindex, depot_index, handle); + __memset(trace, 0, sizeof(*trace)); + return; + } + slab = stack_slabs[parts.slabindex]; + stack = slab + offset; + if (!stack) { + entries = NULL; + return 0; + } *entries = stack->entries; return stack->size; From patchwork Wed Oct 30 14:22:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 39F3E1668 for ; Wed, 30 Oct 2019 14:22:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F061A208C0 for ; Wed, 30 Oct 2019 14:22:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kdRaR3sd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F061A208C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C260C6B000A; Wed, 30 Oct 2019 10:22:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AE5EB6B000C; Wed, 30 Oct 2019 10:22:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B21D6B000D; Wed, 30 Oct 2019 10:22:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 771B56B000A for ; Wed, 30 Oct 2019 10:22:50 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 293F08249980 for ; Wed, 30 Oct 2019 14:22:50 +0000 (UTC) X-FDA: 76100667300.13.gun41_35cff63b97d47 X-Spam-Summary: 2,0,0,6cd9ffa039ee03c8,d41d8cd98f00b204,3ujy5xqykcmsx2zuv8x55x2v.t532z4be-331crt1.58x@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: gun41_35cff63b97d47 X-Filterd-Recvd-Size: 4361 Received: from mail-vs1-f74.google.com (mail-vs1-f74.google.com [209.85.217.74]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:22:49 +0000 (UTC) Received: by mail-vs1-f74.google.com with SMTP id d75so285388vsc.11 for ; Wed, 30 Oct 2019 07:22:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=plzgBrf4XRluLUj/PzB2ncSNnS5pEmW2k0rQhwkYJ7A=; b=kdRaR3sdQBRFSQzy8k+qX2Ad2lD40/rMbvRzTdqm9P0URN3NYQg8QiPJkdvVq10Z72 iZIdcPHXl/lnyYKfVdLSy9U4VNql/q0GSpGrhyI0csAsua40PuWZPyI4A0Q8zYz+5qoC CYtHhRJCBzaFae0g5j8/GrlLWUMYUKNDYM+E08bUR68QFUtfj0S12Gbjf2m3BdcO3rry Nx5WbnJuEJyxenjM3aOT6z9OuE18Yjxi/jXPF0uGkY+zC47BQS3hoim1SGGu9pmjNBQD RvXezbSzZHc2MGYQVffqEj1NUTs1rZq65WEoykDs7jq5L11k17S1PpGD1h701mdLUBfg 4MXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=plzgBrf4XRluLUj/PzB2ncSNnS5pEmW2k0rQhwkYJ7A=; b=a45T+bNw4q0XjCXgiDtjOU9O3ZDKW8F3VVkNnoCZxZs1N2Gh8TkxzSJkddnyNnu9Wq 7qXOHg4QisXrQu+aOxCqs9e/5bpLChIGOymW7JY1DwjkqIZMjz//RGiqhWEzfq1SWfmr 54g6LERdTY0rFEg06eQZE2k8NRle6VK/bx+DVbW3A8s9GOJhgaueI9rIIVosh4zhJtUo 5FR9d7DR7X/WykDV9PEseahI8Pg2Mvknze1+hZ372bA1ZviE2RlLArnfvr2Xn2WouWS6 nel/WjaQs9g7Grj4WcicwmyAoqVy/EzXaO7fPw+GgjW4/2lD0Lg+mcM0c/W/X6o31Z2w FKNw== X-Gm-Message-State: APjAAAU38oZYUDpofaZTkgBP8hyYI9ASQug8PhRdVn/0TmO+ZBY17qUm N5f6zwg5R/YkY8/x6X8FEwWciSjQTEE= X-Google-Smtp-Source: APXvYqwBK0+xQJstq3eyLZqF11CvHzgFRdSrO1R/vDHGepI3fxsD68olJqtwEeqpCRTtHQT9uyYRhgEyfIU= X-Received: by 2002:a1f:c441:: with SMTP id u62mr14905093vkf.88.1572445368591; Wed, 30 Oct 2019 07:22:48 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:14 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-3-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 02/25] stackdepot: prevent Clang from optimizing away stackdepot_memcmp() From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Clang may replace stackdepot_memcmp() with a call to instrumented bcmp(), which is exactly what we wanted to avoid creating stackdepot_memcmp(). Add a compiler barrier() to prevent optimizations. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I4495b617b15c0ab003a61c1f0d54d0026fa8b144 --- lib/stackdepot.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 0d00d2ac0c4b..785839298e08 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -163,6 +163,11 @@ int stackdepot_memcmp(const unsigned long *u1, const unsigned long *u2, unsigned int n) { for ( ; n-- ; u1++, u2++) { + /* + * Prevent Clang from replacing this function with a bcmp() + * call. + */ + barrier(); if (*u1 != *u2) return 1; } From patchwork Wed Oct 30 14:22:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219611 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 404BB112B for ; Wed, 30 Oct 2019 14:22:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F38C22087E for ; Wed, 30 Oct 2019 14:22:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IDkLsl0H" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F38C22087E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CBB796B000C; Wed, 30 Oct 2019 10:22:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C463C6B000D; Wed, 30 Oct 2019 10:22:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B35386B000E; Wed, 30 Oct 2019 10:22:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0142.hostedemail.com [216.40.44.142]) by kanga.kvack.org (Postfix) with ESMTP id 88D016B000C for ; Wed, 30 Oct 2019 10:22:54 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id F00394DA3 for ; Wed, 30 Oct 2019 14:22:53 +0000 (UTC) X-FDA: 76100667426.25.bears49_3650a6d34452d X-Spam-Summary: 2,0,0,822ba077cada1bc3,d41d8cd98f00b204,3u5y5xqykcm4052xyb08805y.w86527eh-664fuw4.8b0@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com:aryabinin@virtuozzo.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:69:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1543:1593:1594:1711:1 730:1747 X-HE-Tag: bears49_3650a6d34452d X-Filterd-Recvd-Size: 6761 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:22:53 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id 4so1391104wrf.19 for ; Wed, 30 Oct 2019 07:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4iI5KVL9+jQTau4pHL/7v6XfJMmusyma1xyeyxplPt4=; b=IDkLsl0HBsPt63OF+cCGz8SicUYJsNuGkRDLKiePNuepxOyKxJan0SklgcUa0ASURb cAZj4zrvDxQs1shKbaSXHpTUos+VqjaqmDitnSNPXHESAcgo5CtKuLQ2WzYu1ke839AD mQeU2WLQFHTpQ72DvlHYKjVyLeHDaXxU6eKJRgk0719aKmM7ozF1aIywsoHQb/F9alrn Bj1izZaIxfA/JoBnQf6u9rLpR3HlTrNmrOpouoTmYdVGRnWEUDcryCaMRGap0YIUsRIg urq334gP/ONmjOUEXYI+ZGCjpkH9jl2hu8e2u1jA3+946NCzhkxGY0HUi9zMs6P0cyNM L3Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4iI5KVL9+jQTau4pHL/7v6XfJMmusyma1xyeyxplPt4=; b=U8hZO6TBu5kq/uahUXyly9nKxD1LVkf2oxfBluh5G5qEoQVO0ntED4oZe9ah9yptDy Eb/8skjqKDw8T3gZ+4CjU0/6qgbp6pEK53S/zaOVCNQhTOjeFyPlRT8T/EwuJjWvxHC9 G2Kh5NixmykzdFTDOLpYj3XUxQ1txT8q7OvBM3CrE7tpsqywgX23ofjqnryooFm/EexO aAk14H9UaBUhhcEGS4D9C5hMsHsJ2F9NHtSW1Me+ShQZZKS+xMBQAMVRwJx+b/AcW/im /8kDZcOF9ZCM8ucdS51J0xfJGLjOX8qJxNHUqN2v3cdboJDgwE8vf3E0Q4HEFYl8ImYu 4zKA== X-Gm-Message-State: APjAAAWYlB4RvL+UuyLLTGgLsZekJkLmjWgv9hcf89ps4gCZ3EIwdgoU RkCPRaDyJDFGa8i4PWF+cBLOmnj14Dc= X-Google-Smtp-Source: APXvYqzf5VWF/vghfT5EXuODI75zBnydtoKmea0SagtgsSyNLxdP+fQoMSyR2QciHaRW14/naviVW83ehOY= X-Received: by 2002:a5d:55c2:: with SMTP id i2mr134672wrw.176.1572445371706; Wed, 30 Oct 2019 07:22:51 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:15 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-4-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 03/25] kasan: stackdepot: move filter_irq_stacks() to stackdepot.c From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , Andrey Ryabinin , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: filter_irq_stacks() can be used by other tools (e.g. KMSAN), so it needs to be moved to a common location. lib/stackdepot.c seems a good place, as filter_irq_stacks() is usually applied to the output of stack_trace_save(). Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Andrey Ryabinin Cc: linux-mm@kvack.org --- Change-Id: I65acecf64930a3658e8c2aa7423801082ded8602 --- include/linux/stackdepot.h | 2 ++ lib/stackdepot.c | 23 +++++++++++++++++++++++ mm/kasan/common.c | 23 ----------------------- 3 files changed, 25 insertions(+), 23 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 3efa97d482cb..24d49c732341 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -19,4 +19,6 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries); +unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries); + #endif diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 785839298e08..decb1e073b58 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -20,6 +20,7 @@ */ #include +#include #include #include #include @@ -319,3 +320,25 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, return retval; } EXPORT_SYMBOL_GPL(stack_depot_save); + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +unsigned int filter_irq_stacks(unsigned long *entries, + unsigned int nr_entries) +{ + unsigned int i; + + for (i = 0; i < nr_entries; i++) { + if (in_irqentry_text(entries[i])) { + /* Include the irqentry function into the stack. */ + return i + 1; + } + } + return nr_entries; +} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 6814d6d6a023..154eba5700d8 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -15,7 +15,6 @@ */ #include -#include #include #include #include @@ -39,28 +38,6 @@ #include "kasan.h" #include "../slab.h" -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline unsigned int filter_irq_stacks(unsigned long *entries, - unsigned int nr_entries) -{ - unsigned int i; - - for (i = 0; i < nr_entries; i++) { - if (in_irqentry_text(entries[i])) { - /* Include the irqentry function into the stack. */ - return i + 1; - } - } - return nr_entries; -} - static inline depot_stack_handle_t save_stack(gfp_t flags) { unsigned long entries[KASAN_STACK_DEPTH]; From patchwork Wed Oct 30 14:22:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219613 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47DE6112B for ; Wed, 30 Oct 2019 14:22:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F1CA620656 for ; Wed, 30 Oct 2019 14:22:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="USt262gJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1CA620656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7DB556B000D; Wed, 30 Oct 2019 10:22:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 78EC76B000E; Wed, 30 Oct 2019 10:22:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62F506B0010; Wed, 30 Oct 2019 10:22:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id 3A4896B000D for ; Wed, 30 Oct 2019 10:22:57 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id DAF68180AD822 for ; Wed, 30 Oct 2019 14:22:56 +0000 (UTC) X-FDA: 76100667552.16.fight50_36c498c16c83a X-Spam-Summary: 2,0,0,77fd3856d548b5b1,d41d8cd98f00b204,3vpy5xqykcne38501e3bb381.zb985ahk-997ixz7.be3@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: fight50_36c498c16c83a X-Filterd-Recvd-Size: 6793 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:22:56 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id i8so934187wmd.5 for ; Wed, 30 Oct 2019 07:22:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=H2Tf4p5InaykhPw6XaGzFla8deeHdIWiA5FFmW92s6o=; b=USt262gJUs3sxkakNKIDk1sJCInJe7jM93ofq2HHPyZqQO8rqNAr0h1ZxFRpymmW1m 4hrf9HVmM0p4aNYjDDvOuSUPuF1qYVsfA2GWW1oTPZrKqzT5MRE/kJ8VlWwVf93N+6fB 9fTBuwjcH+AKdrjZ1bqJ/1kcs4++pSNA8sQMBjEvZ3P4lEm7U702t9yjDkfCL7rq7c/q QcN2Y8gkrOriukCJEMv28Zv91gAKhdZaCiHn6+H0nXSUdLa9lzzbxeCi6vk/zy00jluD RdjofBROq2RIuXIAXXa7kzp+ZaaI7u7+fUhBwYvrjBSTLZXFtrCpgplOTY7xaL7EOPka UvQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=H2Tf4p5InaykhPw6XaGzFla8deeHdIWiA5FFmW92s6o=; b=sauz+0cDPij2RLvTXmkNeZLGqBNxIFcNfhiMcJ3OYDDdSVaHVfjwkj0TA93dpkm8Xe VuPociNzYnrs4rZNlKgnOxK0g0d3B+dBmlEpnQUA/KkTaUVkKZ3HpwTHLaOohv6s6r4s 2PsyDjG/rlwJeBtgoZ7YMXlqTERT24E5i1cdj4qZDR3d1tnf/RnqDhGRA7w44TFermTP E2Dv+WfQXHzLfKTFoAWB8/2yn6Zer9a793Dz7NvSOW6HTS3/KQb7G2Hu8wD2BjeEtbLP J+AiUAvpuuqZIwJb1qJMFU9qbr6/e+8ZRaSDx4KwmvgohAgYrk4qIX2JdjV7th6mS2KH WegA== X-Gm-Message-State: APjAAAXUSJOlSEZ7znuXcpoBiJmynmrBFEjeHiJMZWGIzwW35xghymFN z8F2JYIHvnGz6gcruKG+3KlLI2H8av0= X-Google-Smtp-Source: APXvYqzi6OWpVZPnRxIo10s3uHlOFkOUGoOhcEV4MArIStTlyn9CbtijEN7m0qRPKCZ6rYGRNn4Xy5CcEA4= X-Received: by 2002:adf:e942:: with SMTP id m2mr155171wrn.26.1572445374905; Wed, 30 Oct 2019 07:22:54 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:16 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-5-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 04/25] stackdepot: reserve 5 extra bits in depot_stack_handle_t From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some users (currently only KMSAN) may want to use spare bits in depot_stack_handle_t. Let them do so and provide get_dsh_extra_bits() and set_dsh_extra_bits() to access those bits. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I23580dbde85908eeda0bdd8f83a8c3882ab3e012 --- include/linux/stackdepot.h | 8 ++++++++ lib/stackdepot.c | 24 +++++++++++++++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 24d49c732341..ac1b5a78d7f6 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -12,6 +12,11 @@ #define _LINUX_STACKDEPOT_H typedef u32 depot_stack_handle_t; +/* + * Number of bits in the handle that stack depot doesn't use. Users may store + * information in them. + */ +#define STACK_DEPOT_EXTRA_BITS 5 depot_stack_handle_t stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t gfp_flags); @@ -20,5 +25,8 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, unsigned long **entries); unsigned int filter_irq_stacks(unsigned long *entries, unsigned int nr_entries); +depot_stack_handle_t set_dsh_extra_bits(depot_stack_handle_t handle, + unsigned int bits); +unsigned int get_dsh_extra_bits(depot_stack_handle_t handle); #endif diff --git a/lib/stackdepot.c b/lib/stackdepot.c index decb1e073b58..460efd3ef742 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -40,8 +40,10 @@ #define STACK_ALLOC_ALIGN 4 #define STACK_ALLOC_OFFSET_BITS (STACK_ALLOC_ORDER + PAGE_SHIFT - \ STACK_ALLOC_ALIGN) + #define STACK_ALLOC_INDEX_BITS (DEPOT_STACK_BITS - \ - STACK_ALLOC_NULL_PROTECTION_BITS - STACK_ALLOC_OFFSET_BITS) + STACK_ALLOC_NULL_PROTECTION_BITS - \ + STACK_ALLOC_OFFSET_BITS - STACK_DEPOT_EXTRA_BITS) #define STACK_ALLOC_SLABS_CAP 8192 #define STACK_ALLOC_MAX_SLABS \ (((1LL << (STACK_ALLOC_INDEX_BITS)) < STACK_ALLOC_SLABS_CAP) ? \ @@ -54,6 +56,7 @@ union handle_parts { u32 slabindex : STACK_ALLOC_INDEX_BITS; u32 offset : STACK_ALLOC_OFFSET_BITS; u32 valid : STACK_ALLOC_NULL_PROTECTION_BITS; + u32 extra : STACK_DEPOT_EXTRA_BITS; }; }; @@ -72,6 +75,24 @@ static int next_slab_inited; static size_t depot_offset; static DEFINE_SPINLOCK(depot_lock); +depot_stack_handle_t set_dsh_extra_bits(depot_stack_handle_t handle, + u32 bits) +{ + union handle_parts parts = { .handle = handle }; + + parts.extra = bits & ((1U << STACK_DEPOT_EXTRA_BITS) - 1); + return parts.handle; +} +EXPORT_SYMBOL_GPL(set_dsh_extra_bits); + +u32 get_dsh_extra_bits(depot_stack_handle_t handle) +{ + union handle_parts parts = { .handle = handle }; + + return parts.extra; +} +EXPORT_SYMBOL_GPL(get_dsh_extra_bits); + static bool init_stack_slab(void **prealloc) { if (!*prealloc) @@ -132,6 +153,7 @@ static struct stack_record *depot_alloc_stack(unsigned long *entries, int size, stack->handle.slabindex = depot_index; stack->handle.offset = depot_offset >> STACK_ALLOC_ALIGN; stack->handle.valid = 1; + stack->handle.extra = 0; memcpy(stack->entries, entries, size * sizeof(unsigned long)); depot_offset += required_size; From patchwork Wed Oct 30 14:22:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23DD41668 for ; Wed, 30 Oct 2019 14:23:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C946420656 for ; Wed, 30 Oct 2019 14:23:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ecTIvnoM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C946420656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 605946B000E; Wed, 30 Oct 2019 10:23:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 58F776B0010; Wed, 30 Oct 2019 10:23:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 458176B0266; Wed, 30 Oct 2019 10:23:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 0B4256B000E for ; Wed, 30 Oct 2019 10:23:01 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 9FCC3180AD822 for ; Wed, 30 Oct 2019 14:23:00 +0000 (UTC) X-FDA: 76100667720.07.slave11_3748f6acd0620 X-Spam-Summary: 50,0,0,da1853dce7c23e03,d41d8cd98f00b204,3wpy5xqykcnu7c945i7ff7c5.3fdc9elo-ddbm13b.fi7@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:69:152:327:355:379:541:800:960:966:967:968:973:982:988:989:1260:1263:1277:1313:1314:1345:1359:1437:1516:1518:1593: 1594:160 X-HE-Tag: slave11_3748f6acd0620 X-Filterd-Recvd-Size: 21426 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:22:59 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id t2so1388191wri.18 for ; Wed, 30 Oct 2019 07:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=kC4fHddMlzrW3H+GQD1LapxofFrWtpYFYDWwAFlzg/I=; b=ecTIvnoMfs5vGHbKoX1zLDqg+33Jz/M6k9xcrhGedUxKKIYEEWMk1Jp83d7tCiQYM8 OgFrCpNxnuG0mJ8VOUfA7ARXyCx9f3ur6gJ7psrvl5QiUrx/6dfolS3Wi5cI5qCPn5ZZ +m229o1R6PMGs51aW1GvQ9fdzy6lR6vhggaL/qmT8pH5viI20X76TgTXr1p9HrZJvqFB EP2Z/dEeOYdLYZog23OKdj3DYmaaRVaL0dJigUG9LZ534hXaSUpj4ktxcJyqobh09Xb4 MbM0iIZF8po2536sJgUx3I6LngIGYqaB7AUjv9/EHbS1jVPb/ZQR83IpeUOvGvsKJ3pc GLHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=kC4fHddMlzrW3H+GQD1LapxofFrWtpYFYDWwAFlzg/I=; b=PdE+zro50KILmgO7DzZ7YeKenjy7kKEjI1vBXHC5Zr0eL9xrqI1s8cpuZKtB9je2e1 ZyVKTu0swLhSzO/qyi3ypk09jeA1sRYaKD2tPM4yHVhXYpDj2/2oBNDqCZsKSJgMRYNM 0VqBQTWLEmtsT+kNxdFrE9LUDa226nyLoMbattff+TnNaTiFjsjYZ4jNU9kbw+OB9rc6 0bMu2xfmstx/PxfUPoCIV0SFdjn3nIlSex7BkRGzziW3gK9jIXJAyPtjc19wGlOTfan3 4jCLgyONNlOnBF4D4ngqDxyNnlIvr/Ws/EiVHSTSqfSOXkTBMyjxmG1yh8ndKIAbGQtR 13SA== X-Gm-Message-State: APjAAAXvAWHNdEiogNVX77aoPlQON0v8R3ZyX8v9Kvmw2VpjVRkZ4m5D YlvM2uIe8Mq1+Dfa6+aMhgY9PYjBB0E= X-Google-Smtp-Source: APXvYqykSzlkcXE6zFNul/+PiyRpvdnTXjZBgtyg7XLn8E2I4OThbJDTJ1wfwPjmuhXlBbr0JAVYMdzsMHk= X-Received: by 2002:adf:9f08:: with SMTP id l8mr78847wrf.325.1572445378000; Wed, 30 Oct 2019 07:22:58 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:17 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-6-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 05/25] kmsan: add ReST documentation From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add Documentation/dev-tools/kmsan.rst and reference it in the dev-tools index. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: Iac6345065e6804ef811f1124fdf779c67ff1530e --- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kmsan.rst | 418 ++++++++++++++++++++++++++++++ 2 files changed, 419 insertions(+) create mode 100644 Documentation/dev-tools/kmsan.rst diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index b0522a4dd107..bc5e3fd87efa 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -19,6 +19,7 @@ whole; patches welcome! kcov gcov kasan + kmsan ubsan kmemleak gdb-kernel-debugging diff --git a/Documentation/dev-tools/kmsan.rst b/Documentation/dev-tools/kmsan.rst new file mode 100644 index 000000000000..51f9c207cc2c --- /dev/null +++ b/Documentation/dev-tools/kmsan.rst @@ -0,0 +1,418 @@ +============================= +KernelMemorySanitizer (KMSAN) +============================= + +KMSAN is a dynamic memory error detector aimed at finding uses of uninitialized +memory. +It is based on compiler instrumentation, and is quite similar to the userspace +MemorySanitizer tool (http://clang.llvm.org/docs/MemorySanitizer.html). + +KMSAN and Clang +=============== + +In order for KMSAN to work the kernel must be +built with Clang, which is so far the only compiler that has KMSAN support. +The kernel instrumentation pass is based on the userspace MemorySanitizer tool +(http://clang.llvm.org/docs/MemorySanitizer.html). Because of the +instrumentation complexity it's unlikely that any other compiler will support +KMSAN soon. + +Right now the instrumentation pass supports x86_64 only. + +How to build +============ + +In order to build a kernel with KMSAN you'll need a fresh Clang (10.0.0+, trunk +version r365008 or greater). Please refer to +https://llvm.org/docs/GettingStarted.html for the instructions on how to build +Clang:: + + export KMSAN_CLANG_PATH=/path/to/clang + # Now configure and build the kernel with CONFIG_KMSAN enabled. + make CC=$KMSAN_CLANG_PATH -j64 + +How KMSAN works +=============== + +KMSAN shadow memory +------------------- + +KMSAN associates a so-called shadow byte with every byte of kernel memory. +A bit in the shadow byte is set iff the corresponding bit of the kernel memory +byte is uninitialized. +Marking the memory uninitialized (i.e. setting its shadow bytes to 0xff) is +called poisoning, marking it initialized (setting the shadow bytes to 0x00) is +called unpoisoning. + +When a new variable is allocated on the stack, it's poisoned by default by +instrumentation code inserted by the compiler (unless it's a stack variable that +is immediately initialized). Any new heap allocation done without ``__GFP_ZERO`` +is also poisoned. + +Compiler instrumentation also tracks the shadow values with the help from the +runtime library in ``mm/kmsan/``. + +The shadow value of a basic or compound type is an array of bytes of the same +length. +When a constant value is written into memory, that memory is unpoisoned. +When a value is read from memory, its shadow memory is also obtained and +propagated into all the operations which use that value. For every instruction +that takes one or more values the compiler generates code that calculates the +shadow of the result depending on those values and their shadows. + +Example:: + + int a = 0xff; + int b; + int c = a | b; + +In this case the shadow of ``a`` is ``0``, shadow of ``b`` is ``0xffffffff``, +shadow of ``c`` is ``0xffffff00``. This means that the upper three bytes of +``c`` are uninitialized, while the lower byte is initialized. + + +Origin tracking +--------------- + +Every four bytes of kernel memory also have a so-called origin assigned to +them. +This origin describes the point in program execution at which the uninitialized +value was created. Every origin is associated with a creation stack, which lets +the user figure out what's going on. + +When an uninitialized variable is allocated on stack or heap, a new origin +value is created, and that variable's origin is filled with that value. +When a value is read from memory, its origin is also read and kept together +with the shadow. For every instruction that takes one or more values the origin +of the result is one of the origins corresponding to any of the uninitialized +inputs. +If a poisoned value is written into memory, its origin is written to the +corresponding storage as well. + +Example 1:: + + int a = 0; + int b; + int c = a + b; + +In this case the origin of ``b`` is generated upon function entry, and is +stored to the origin of ``c`` right before the addition result is written into +memory. + +Several variables may share the same origin address, if they are stored in the +same four-byte chunk. +In this case every write to either variable updates the origin for all of them. + +Example 2:: + + int combine(short a, short b) { + union ret_t { + int i; + short s[2]; + } ret; + ret.s[0] = a; + ret.s[1] = b; + return ret.i; + } + +If ``a`` is initialized and ``b`` is not, the shadow of the result would be +0xffff0000, and the origin of the result would be the origin of ``b``. +``ret.s[0]`` would have the same origin, but it will be never used, because +that variable is initialized. + +If both function arguments are uninitialized, only the origin of the second +argument is preserved. + +Origin chaining +~~~~~~~~~~~~~~~ +To ease the debugging, KMSAN creates a new origin for every memory store. +The new origin references both its creation stack and the previous origin the +memory location had. +This may cause increased memory consumption, so we limit the length of origin +chains in the runtime. + +Clang instrumentation API +------------------------- + +Clang instrumentation pass inserts calls to functions defined in +``mm/kmsan/kmsan_instr.c`` into the kernel code. + +Shadow manipulation +~~~~~~~~~~~~~~~~~~~ +For every memory access the compiler emits a call to a function that returns a +pair of pointers to the shadow and origin addresses of the given memory:: + + typedef struct { + void *s, *o; + } shadow_origin_ptr_t + + shadow_origin_ptr_t __msan_metadata_ptr_for_load_{1,2,4,8}(void *addr) + shadow_origin_ptr_t __msan_metadata_ptr_for_store_{1,2,4,8}(void *addr) + shadow_origin_ptr_t __msan_metadata_ptr_for_load_n(void *addr, u64 size) + shadow_origin_ptr_t __msan_metadata_ptr_for_store_n(void *addr, u64 size) + +The function name depends on the memory access size. +Each such function also checks if the shadow of the memory in the range +[``addr``, ``addr + n``) is contiguous and reports an error otherwise. + +The compiler makes sure that for every loaded value its shadow and origin +values are read from memory. +When a value is stored to memory, its shadow and origin are also stored using +the metadata pointers. + +Origin tracking +~~~~~~~~~~~~~~~ +A special function is used to create a new origin value for a local variable +and set the origin of that variable to that value:: + + void __msan_poison_alloca(u64 address, u64 size, char *descr) + +Access to per-task data +~~~~~~~~~~~~~~~~~~~~~~~~~ + +At the beginning of every instrumented function KMSAN inserts a call to +``__msan_get_context_state()``:: + + kmsan_context_state *__msan_get_context_state(void) + +``kmsan_context_state`` is declared in ``include/linux/kmsan.h``:: + + struct kmsan_context_s { + char param_tls[KMSAN_PARAM_SIZE]; + char retval_tls[RETVAL_SIZE]; + char va_arg_tls[KMSAN_PARAM_SIZE]; + char va_arg_origin_tls[KMSAN_PARAM_SIZE]; + u64 va_arg_overflow_size_tls; + depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE]; + depot_stack_handle_t retval_origin_tls; + depot_stack_handle_t origin_tls; + }; + +This structure is used by KMSAN to pass parameter shadows and origins between +instrumented functions. + +String functions +~~~~~~~~~~~~~~~~ + +The compiler replaces calls to ``memcpy()``/``memmove()``/``memset()`` with the +following functions. These functions are also called when data structures are +initialized or copied, making sure shadow and origin values are copied alongside +with the data:: + + void *__msan_memcpy(void *dst, void *src, u64 n) + void *__msan_memmove(void *dst, void *src, u64 n) + void *__msan_memset(void *dst, int c, size_t n) + +Error reporting +~~~~~~~~~~~~~~~ + +For each pointer dereference and each condition the compiler emits a shadow +check that calls ``__msan_warning()`` in the case a poisoned value is being +used:: + + void __msan_warning(u32 origin) + +``__msan_warning()`` causes KMSAN runtime to print an error report. + +Inline assembly instrumentation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +KMSAN instruments every inline assembly output with a call to:: + + void __msan_instrument_asm_store(u64 addr, u64 size) + +, which unpoisons the memory region. + +This approach may mask certain errors, but it also helps to avoid a lot of +false positives in bitwise operations, atomics etc. + +Sometimes the pointers passed into inline assembly don't point to valid memory. +In such cases they are ignored at runtime. + +Disabling the instrumentation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +A function can be marked with ``__no_sanitize_memory``. +Doing so doesn't remove KMSAN instrumentation from it, however it makes the +compiler ignore the uninitialized values coming from the function's inputs, +and initialize the function's outputs. +The compiler won't inline functions marked with this attribute into functions +not marked with it, and vice versa. + +It's also possible to disable KMSAN for a single file (e.g. main.o):: + + KMSAN_SANITIZE_main.o := n + +or for the whole directory:: + + KMSAN_SANITIZE := n + +in the Makefile. This comes at a cost however: stack allocations from such files +and parameters of instrumented functions called from them will have incorrect +shadow/origin values. As a rule of thumb, avoid using KMSAN_SANITIZE. + +Runtime library +--------------- +The code is located in ``mm/kmsan/``. + +Per-task KMSAN state +~~~~~~~~~~~~~~~~~~~~ + +Every task_struct has an associated KMSAN task state that holds the KMSAN +context (see above) and a per-task flag disallowing KMSAN reports:: + + struct kmsan_task_state { + ... + bool allow_reporting; + struct kmsan_context_state cstate; + ... + } + + struct task_struct { + ... + struct kmsan_task_state kmsan; + ... + } + + +KMSAN contexts +~~~~~~~~~~~~~~ + +When running in a kernel task context, KMSAN uses ``current->kmsan.cstate`` to +hold the metadata for function parameters and return values. + +But in the case the kernel is running in the interrupt, softirq or NMI context, +where ``current`` is unavailable, KMSAN switches to per-cpu interrupt state:: + + DEFINE_PER_CPU(kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX], + kmsan_percpu_cstate); + +Metadata allocation +~~~~~~~~~~~~~~~~~~~ +There are several places in the kernel for which the metadata is stored. + +1. Each ``struct page`` instance contains two pointers to its shadow and +origin pages:: + + struct page { + ... + struct page *shadow, *origin; + ... + }; + +Every time a ``struct page`` is allocated, the runtime library allocates two +additional pages to hold its shadow and origins. This is done by adding hooks +to ``alloc_pages()``/``free_pages()`` in ``mm/page_alloc.c``. +To avoid allocating the metadata for non-interesting pages (right now only the +shadow/origin page themselves and stackdepot storage) the +``__GFP_NO_KMSAN_SHADOW`` flag is used. + +There is a problem related to this allocation algorithm: when two contiguous +memory blocks are allocated with two different ``alloc_pages()`` calls, their +shadow pages may not be contiguous. So, if a memory access crosses the boundary +of a memory block, accesses to shadow/origin memory may potentially corrupt +other pages or read incorrect values from them. + +As a workaround, we check the access size in +``__msan_metadata_ptr_for_XXX_YYY()`` and return a pointer to a fake shadow +region in the case of an error:: + + char dummy_load_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); + char dummy_store_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); + +``dummy_load_page`` is zero-initialized, so reads from it always yield zeroes. +All stores to ``dummy_store_page`` are ignored. + +Unfortunately at boot time we need to allocate shadow and origin pages for the +kernel data (``.data``, ``.bss`` etc.) and percpu memory regions, the size of +which is not a power of 2. As a result, we have to allocate the metadata page by +page, so that it is also non-contiguous, although it may be perfectly valid to +access the corresponding kernel memory across page boundaries. +This can be probably fixed by allocating 1<] __dump_stack lib/dump_stack.c:15 + [] dump_stack+0x238/0x290 lib/dump_stack.c:51 + [] kmsan_report+0x276/0x2e0 mm/kmsan/kmsan.c:1003 + [] __msan_warning+0x5b/0xb0 mm/kmsan/kmsan_instr.c:424 + [< inline >] strlen lib/string.c:484 + [] strlcpy+0x9d/0x200 lib/string.c:144 + [] packet_bind_spkt+0x144/0x230 net/packet/af_packet.c:3132 + [] SYSC_bind+0x40d/0x5f0 net/socket.c:1370 + [] SyS_bind+0x82/0xa0 net/socket.c:1356 + [] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:? + chained origin: + [] save_stack_trace+0x27/0x50 arch/x86/kernel/stacktrace.c:67 + [< inline >] kmsan_save_stack_with_flags mm/kmsan/kmsan.c:322 + [< inline >] kmsan_save_stack mm/kmsan/kmsan.c:334 + [] kmsan_internal_chain_origin+0x118/0x1e0 mm/kmsan/kmsan.c:527 + [] __msan_set_alloca_origin4+0xc3/0x130 mm/kmsan/kmsan_instr.c:380 + [] SYSC_bind+0x129/0x5f0 net/socket.c:1356 + [] SyS_bind+0x82/0xa0 net/socket.c:1356 + [] entry_SYSCALL_64_fastpath+0x13/0x8f arch/x86/entry/entry_64.o:? + origin description: ----address@SYSC_bind (origin=00000000eb400911) + ================================================================== + +The report tells that the local variable ``address`` was created uninitialized +in ``SYSC_bind()`` (the ``bind`` system call implementation). The lower stack +trace corresponds to the place where this variable was created. + +The upper stack shows where the uninit value was used - in ``strlen()``. +It turned out that the contents of ``address`` were partially copied from the +userspace, but the buffer wasn't zero-terminated and contained some trailing +uninitialized bytes. +``packet_bind_spkt()`` didn't check the length of the buffer, but called +``strlcpy()`` on it, which called ``strlen()``, which started reading the +buffer byte by byte till it hit the uninitialized memory. + + +References +========== + +E. Stepanov, K. Serebryany. MemorySanitizer: fast detector of uninitialized +memory use in C++. +In Proceedings of CGO 2015. From patchwork Wed Oct 30 14:22:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219617 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 135981668 for ; Wed, 30 Oct 2019 14:23:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CA49120656 for ; Wed, 30 Oct 2019 14:23:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UQrJdKCX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA49120656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 172A56B0010; Wed, 30 Oct 2019 10:23:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0FBA26B0266; Wed, 30 Oct 2019 10:23:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EDE626B0269; Wed, 30 Oct 2019 10:23:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0065.hostedemail.com [216.40.44.65]) by kanga.kvack.org (Postfix) with ESMTP id C95946B0010 for ; Wed, 30 Oct 2019 10:23:03 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 70D9D8249980 for ; Wed, 30 Oct 2019 14:23:03 +0000 (UTC) X-FDA: 76100667846.12.alley70_37bce58ae9e48 X-Spam-Summary: 2,0,0,655843d74928547f,d41d8cd98f00b204,3xzy5xqykcngafc78laiiaf8.6igfchor-ggep46e.ila@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:akpm@linux-foundation.org:dvyukov@google.com::viro@zeniv.linux.org.uk:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1541:1593:1594:1711:1730 :1747:17 X-HE-Tag: alley70_37bce58ae9e48 X-Filterd-Recvd-Size: 5103 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:02 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id j17so1207524wru.13 for ; Wed, 30 Oct 2019 07:23:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zjqLA8icycEaF+lV+J+rI33Voloa3PXA51lTTa+kIJM=; b=UQrJdKCXvrvp6fRE1XmTTIky6U0f1mhLPtOAw+zJcnr66aTyQjWPDjJleHmzEjkDAC 9BUQsG4GuXHiJ2W/PG99IP/kxV1PeIwRLyF5ORySqF46pGFITVrmINlZ8o2C/Po9uEEr 3fGxi3sn1ftUW6lo9kBurbZ/NAO+eI8eQRLZB6hth6FrmCzDlauPkluZbycW/ZS7IlGN hJ369CWZu22JA0A7BsKwXmlIOksVwtjjJzqQy/LZkcApsS0enPpKOHTo9XayVZmqdQaH OB9PyRz9ATcg0RlrdXbit4+uNli8mur89u7bcYq4Hy0hGv688Hmd+IE58CaTohBGbLxq 00Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zjqLA8icycEaF+lV+J+rI33Voloa3PXA51lTTa+kIJM=; b=DcrawsS0ramuRjSYNy1Gq79rC9QRG4clIT54mAul+f8FH8Qy1z7iPvU6Wcng3ByyZK vv4gPgDSYk3h12V4nQJs1AEWSTxYSrGmEDpEfs4+5BeQU9tEMH6PRvKRhhcbaE2EL30A rzz2+PCxuDX8fi9bzBQWh6qzFTdiDmqkWtGsxAdQBPfmydQcExw2hXfSHeDxcLk58ume dc0eYoasaoCGOCQiv9X6NLJwKyg+23YHY5jVCEgjht3UYivHtllzsV1tnM8K3D8IvP4Z KVpBhbuqSBvPM+V5GWe1tXaxDW+M0gm4f6uqeG1eAowu1hFAJarV+bafoFBeKoSpGE/Y TU2g== X-Gm-Message-State: APjAAAVDlQeUXiKbCAiokR1kERvGZe3a/qooB9pVZVtpT+83NSNUbFGj d2TvkMBHSWpTDdWpucKm7YK721jgzt0= X-Google-Smtp-Source: APXvYqy2jBLEa5gUxeoY8U+T1rJGeAelsDsZfYh7vgXH4qK2uiM6fP1EncUZd9wtfzDsH0zoBWEYupxUPWg= X-Received: by 2002:a5d:6104:: with SMTP id v4mr140850wrt.36.1572445381411; Wed, 30 Oct 2019 07:23:01 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:18 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-7-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 06/25] kmsan: gfp: introduce __GFP_NO_KMSAN_SHADOW From: glider@google.com To: Vegard Nossum , Andrew Morton , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This flag is to be used by KMSAN runtime to mark that newly created memory pages don't need KMSAN metadata backing them. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Andrew Morton Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- We can't decide what to do here: - do we need to conditionally define ___GFP_NO_KMSAN_SHADOW depending on CONFIG_KMSAN like LOCKDEP does? - if KMSAN is defined, and LOCKDEP is not, do we want to "compactify" the GFP bits? Change-Id: If5d0352fd5711ad103328e2c185eb885e826423a --- include/linux/gfp.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index fb07b503dc45..b4e7963cd94b 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -44,6 +44,7 @@ struct vm_area_struct; #else #define ___GFP_NOLOCKDEP 0 #endif +#define ___GFP_NO_KMSAN_SHADOW 0x1000000u /* If the above are modified, __GFP_BITS_SHIFT may need updating */ /* @@ -212,12 +213,13 @@ struct vm_area_struct; #define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN) #define __GFP_COMP ((__force gfp_t)___GFP_COMP) #define __GFP_ZERO ((__force gfp_t)___GFP_ZERO) +#define __GFP_NO_KMSAN_SHADOW ((__force gfp_t)___GFP_NO_KMSAN_SHADOW) /* Disable lockdep for GFP context tracking */ #define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP) /* Room for N __GFP_FOO bits */ -#define __GFP_BITS_SHIFT (23 + IS_ENABLED(CONFIG_LOCKDEP)) +#define __GFP_BITS_SHIFT (25) #define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1)) /** From patchwork Wed Oct 30 14:22:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219619 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D1978112B for ; Wed, 30 Oct 2019 14:23:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9B76B20656 for ; Wed, 30 Oct 2019 14:23:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KroV8hCy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B76B20656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0605A6B0266; Wed, 30 Oct 2019 10:23:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F2B906B0269; Wed, 30 Oct 2019 10:23:06 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D56096B026A; Wed, 30 Oct 2019 10:23:06 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id B29596B0266 for ; Wed, 30 Oct 2019 10:23:06 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 5EC026132 for ; Wed, 30 Oct 2019 14:23:06 +0000 (UTC) X-FDA: 76100667972.21.rule92_382864bb1cf07 X-Spam-Summary: 2,0,0,3844b352f5f41132,d41d8cd98f00b204,3yjy5xqykcnsdifabodlldib.9ljifkru-jjhs79h.lod@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1535:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: rule92_382864bb1cf07 X-Filterd-Recvd-Size: 5319 Received: from mail-qk1-f201.google.com (mail-qk1-f201.google.com [209.85.222.201]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:05 +0000 (UTC) Received: by mail-qk1-f201.google.com with SMTP id s3so2207078qkd.6 for ; Wed, 30 Oct 2019 07:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=akfNCofcaplWYSusBCm6RIllOookGu0Nmi5nMbilfDI=; b=KroV8hCy5YvvDtHUgx8njZkV1VcMmzh9xhYv8fymY4CEv7/3v6B+m+FjaT0xVyCdtb 2WtwsEGERBoxS6ONsaN+dHi0TpiGRFlwNEPULn4h+7jsTmgJiN4+zFEjliCPKUQCm8wf xYl8XxX1Ngb+MHHxJnFLWNmH8nIn7oBPaVRqGMjWMdJ+2buJkMQIffAB3hAGkLNQmanm 5mTSfh+zImgIeegNZBxPJw2hs1ZhiQpU0dc/xbuCuNgptsrvSehz83ra4B7IH4j1xUwC PwmxrWjzp7AM6SDrZAD2sTmzq6SfQuXoAkllRQDwgMQarYGLtk6hcL3y0qRLzgU9QsX8 Q0fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=akfNCofcaplWYSusBCm6RIllOookGu0Nmi5nMbilfDI=; b=N8qlse1ZNP4CtA3CQYk+9eprKDgIjZdswmcu8xB1Yhmy2vh1oyB/LwSlz4PdO9PNIs /Rq4bDy+DOdwY3C7dIU5EZH5lxaLNuavurAvDc/PVyGYvqXkjUFpjK6K0s8E8XTzbxEB lgZrgZPOoA5/VJtDil8BEgS9zlIg01aJT6SggcGmCG7/grDXDtlmYVFhr72oSdCOXVn0 +NOI9chYsKJT13pp0137DhQjRQC5y8Par6Jt2uqy7IRJnTViv2HD0Nq1lHjNZdGMICF1 DycXS5EVfsiGuF+w/TEREQ05ovXcfa5Y30F8LXxjC3qrs6I4HH+fPc4W1e8ObnfHuMQC A/Zw== X-Gm-Message-State: APjAAAUtx8KO5xx87bbsV9WtPoyjwl/sF7EsJ1Vcys79lgOTtIBl6Bv4 mdJf5Y2y/swAQ5gRST7+BvHajKicMdo= X-Google-Smtp-Source: APXvYqyLqV3WHpqCjY+K5TwB3sVeBWtRq4al0aD//TqSvfceUEzrB7eOGPFFt+OUfhxDNO9Du9fosiYyVZs= X-Received: by 2002:a05:6214:8c6:: with SMTP id da6mr22415815qvb.1.1572445384776; Wed, 30 Oct 2019 07:23:04 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:19 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-8-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 07/25] kmsan: introduce __no_sanitize_memory and __SANITIZE_MEMORY__ From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __no_sanitize_memory is a function attribute that makes KMSAN ignore the uninitialized values coming from the function's inputs, and initialize the function's outputs. Functions marked with this attribute can't be inlined into functions not marked with it, and vice versa. __SANITIZE_MEMORY__ is a macro that's defined iff the file is instrumented with KMSAN. This is not the same as CONFIG_KMSAN, which is defined for every file. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I1f1672652c8392f15f7ca8ac26cd4e71f9cc1e4b --- include/linux/compiler-clang.h | 8 ++++++++ include/linux/compiler-gcc.h | 5 +++++ 2 files changed, 13 insertions(+) diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 333a6695a918..edba13a069a6 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -24,6 +24,14 @@ #define __no_sanitize_address #endif +/* KMSAN is a Clang-only tool, thus putting the defines here */ +#if __has_feature(memory_sanitizer) +# define __SANITIZE_MEMORY__ +# define __no_sanitize_memory __attribute__((no_sanitize("kernel-memory"))) +#else +# define __no_sanitize_memory +#endif + /* * Not all versions of clang implement the the type-generic versions * of the builtin overflow checkers. Fortunately, clang implements diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index d7ee4c6bad48..e5ebc788dde4 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -145,6 +145,11 @@ #define __no_sanitize_address #endif +/* + * GCC doesn't support KMSAN. + */ +#define __no_sanitize_memory + #if GCC_VERSION >= 50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1 #endif From patchwork Wed Oct 30 14:22:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219621 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B64041668 for ; Wed, 30 Oct 2019 14:23:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7812C20656 for ; Wed, 30 Oct 2019 14:23:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tKtPfAyz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7812C20656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A4E2E6B0269; Wed, 30 Oct 2019 10:23:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 987356B026A; Wed, 30 Oct 2019 10:23:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84E216B026B; Wed, 30 Oct 2019 10:23:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id 61A266B0269 for ; Wed, 30 Oct 2019 10:23:10 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 0CF0B180AD822 for ; Wed, 30 Oct 2019 14:23:10 +0000 (UTC) X-FDA: 76100668140.16.wood06_38b1013cc404b X-Spam-Summary: 2,0,0,d105e5b01584d5aa,d41d8cd98f00b204,3y5y5xqykcn4glidergoogle.comlinux-mmkvack.org@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:akpm@linux-foundation.org:dvyukov@google.com::viro@zeniv.linux.org.uk:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1541:1593:1594:1711:1730 :1747:17 X-HE-Tag: wood06_38b1013cc404b X-Filterd-Recvd-Size: 5211 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:09 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id g17so936460wmc.4 for ; Wed, 30 Oct 2019 07:23:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=B0/OKDvoBq/9N8RtHxhB5aWl4cTZk6XthTMcJiiHzTI=; b=tKtPfAyzDF7M0HlVHNOTLefC5+yn/DJWCxSSC1pxLA27y6INfzRL+b3TtGY1iIOidn rs3/7akrRCdR5bDT30QWT3l+Ssytz+TBffGgMSktvTcZta5vNsJtvaM5yHwaxnNfdoFh DSXcx0Ge87v10iQF+YGto1tNgbTbzVO5/aH5Su5ei0xaNEV/ahh0Ayyjlzj2vRKrijSW bWlbxJYS2BACp20+fR2nuPZ/RR3PbugUQFn/q4dWPBev4NwiZ3Yg3V4eY+KjS09XFzXQ G8LgC3U3QhlldABOiEEDc9dietKpWkpVF0smNcCG061OiWtBfId5QO7WC/2X4NynhXRt VHCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=B0/OKDvoBq/9N8RtHxhB5aWl4cTZk6XthTMcJiiHzTI=; b=XV1+cZZ7DKGHfwu5Se0ThpiqQ2A0dVRZkccj772tmwvmb1lwzVTw/Jws6cY4M+scd7 6dlwD4owL/4vfgqxqMVzbFW+ghd/HZ/Eu7YqmO7RbtDnx8pnSvy3UpHDDmYKj7NrkToq fSXhhiS3aCBYwT8QRWZRcRsoLR4wHmoEmd++gukTHoK3TU39CTb4XoNsaL6AU9dwBPHc 7R4PaMTH35W3sJ14IHXRIfAACpL2ZCIiG5CRculwWnxaw2Pn4AnWEoNrQjrVfnPZxZwF ctqqhKATMj2zMxpBzSngvKpSZ0fAY/rNrXJ9e3dZ1wSfxTMsUh9Ia1Jn7x+EIl5kSn3+ 86Wg== X-Gm-Message-State: APjAAAUpyJ15vuiSUiLVLOHlOVmoeu68yyoyi09S5fWbyb0Oq+5z52Jd Y66JwkqLRHod1hPaJb13fVEDURp79Mk= X-Google-Smtp-Source: APXvYqxkTmPJltC//1rsh6xv6iKs4b8BP9HXBLViVnYMxTyNX4DgzBVyhUbmT2e69paLzsbYIFVjfrrdFRU= X-Received: by 2002:adf:dbd2:: with SMTP id e18mr125693wrj.268.1572445387900; Wed, 30 Oct 2019 07:23:07 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:20 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-9-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 08/25] kmsan: reduce vmalloc space From: glider@google.com To: Vegard Nossum , Andrew Morton , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN is going to use 3/4 of existing vmalloc space to hold the metadata, therefore we lower VMALLOC_END to make sure vmalloc() doesn't allocate past the first 1/4. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Andrew Morton Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: Iaa5e8e0fc2aa66c956f937f5a1de6e5ef40d57cc --- arch/x86/include/asm/pgtable_64_types.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h index 52e5f5f2240d..586629e20436 100644 --- a/arch/x86/include/asm/pgtable_64_types.h +++ b/arch/x86/include/asm/pgtable_64_types.h @@ -139,7 +139,22 @@ extern unsigned int ptrs_per_p4d; # define VMEMMAP_START __VMEMMAP_BASE_L4 #endif /* CONFIG_DYNAMIC_MEMORY_LAYOUT */ +#ifndef CONFIG_KMSAN #define VMALLOC_END (VMALLOC_START + (VMALLOC_SIZE_TB << 40) - 1) +#else +/* + * In KMSAN builds vmalloc area is four times smaller, and the remaining 3/4 + * are used to keep the metadata for virtual pages. + */ +#define VMALLOC_QUARTER_SIZE ((VMALLOC_SIZE_TB << 40) >> 2) +#define VMALLOC_END (VMALLOC_START + VMALLOC_QUARTER_SIZE - 1) +#define VMALLOC_SHADOW_OFFSET VMALLOC_QUARTER_SIZE +#define VMALLOC_ORIGIN_OFFSET (VMALLOC_QUARTER_SIZE * 2) +#define VMALLOC_META_END (VMALLOC_END + VMALLOC_ORIGIN_OFFSET) +#define MODULES_SHADOW_START (VMALLOC_META_END + 1) +#define MODULES_ORIGIN_START (MODULES_SHADOW_START + MODULES_LEN) +#define MODULES_ORIGIN_END (MODULES_ORIGIN_START + MODULES_LEN) +#endif #define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE) /* The module sections ends with the start of the fixmap */ From patchwork Wed Oct 30 14:22:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219623 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D80561668 for ; Wed, 30 Oct 2019 14:23:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5EA6720656 for ; Wed, 30 Oct 2019 14:23:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="p2jfFEQx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EA6720656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8027B6B026A; Wed, 30 Oct 2019 10:23:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7B0026B026B; Wed, 30 Oct 2019 10:23:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DD316B026C; Wed, 30 Oct 2019 10:23:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 18E236B026A for ; Wed, 30 Oct 2019 10:23:15 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 91CEE582A for ; Wed, 30 Oct 2019 14:23:14 +0000 (UTC) X-FDA: 76100668308.22.dad97_392b98cf3c62b X-Spam-Summary: 2,0,0,83f9f69ebb24aaae,d41d8cd98f00b204,3z5y5xqykcoikpmhivksskpi.gsqpmryb-qqozego.svk@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:152:355:379:541:960:966:967:973:982:988:989:1042:1260:1277:1311:1313:1314:1345:1359:1431:1437:1513:1515:1516:1518:1521 :1593:15 X-HE-Tag: dad97_392b98cf3c62b X-Filterd-Recvd-Size: 90311 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:12 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id j14so1409128wrm.6 for ; Wed, 30 Oct 2019 07:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=YrDN24N/WY4R1KLB0/1P51BwJDG2PjdreclFhfDNsA0=; b=p2jfFEQx87Ba0MAuO6OwoUwnZw2z9XnuUCcwIQtQEBY66q77RGwuSkx2BYID8m6VkC BDYcdlXxi3BSf/9QKb3Y+cFQBvuaGkDOWZnrfquWUjHYcEzO8A2JToVvN0hrG4ZVG7al HVqEJPMJYzGHWtT+FYKIIfJsFYlW+h68MLN2dKVBrC7OyZuO7JFsHy4p9/zR0Y13EUHH pyO1dgJQ4277ni2eZSH8QnlaUSlg+BFN5dwqhj+eMctzrCTlctSPZK+OY4kYf8UVM+4K mppjRteDB4rKoVYmR6EZiKzBh/3MqmdNfD73mL5v+FF2fiwZsXp5yMntuwEsYM1KskKK Oa1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=YrDN24N/WY4R1KLB0/1P51BwJDG2PjdreclFhfDNsA0=; b=Ad8EiwmRJvj3Aeg1xLZVaqmUqYjjwpL6nX4MgWXvUdsLah0cF4nj5TXj8aHt9bNQfr Nxwzx4+qrsni4LCO8V2KdkqAhLueAzccZZSa0TpUbRRLhTC5kDDqolXAD2+MwQ+fBYjC iRjc02Q4C1yXwCE1osOX/dDWsqIF7CdUNHGjXgGwPFp0XKosBhM6BDyJOtPBnZ9fQk+B oCJ00NBBcTZ9jVEW2qjBejmLPxsX/gAF9HQwrwzCL9TyQaciKAVv3DXwYnVsh7GLs/wp u8W8RUEorNtSgyuPCHGVpJuI+hGQIW8Ty49M4ybynECQSZoI/1T4ytfsXsp6PRB0KF7e jIxg== X-Gm-Message-State: APjAAAV0PUXXXCqSysFfJ2DjmN0RRoEXzhRVgX7azL21QbvgD99Fxa8f NiX2xpA6989a5G3XZ5jU0Ff60VIvk10= X-Google-Smtp-Source: APXvYqzdreH6r0EwP+YJD08T6OMJnrHBb/x3uatMy81DF1QdMmIjAqfuu3/MKPn3en3+Y3PEMuJ2k9zNSZc= X-Received: by 2002:a05:6000:14a:: with SMTP id r10mr119959wrx.310.1572445391211; Wed, 30 Oct 2019 07:23:11 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:21 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-10-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 09/25] kmsan: add KMSAN runtime From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch adds the KernelMemorySanitizer runtime and associated files: - arch/x86/include/asm/kmsan.h: assembly definitions for hooking interrupt handlers; - include/linux/kmsan-checks.h: user API to enable/disable KMSAN, poison/unpoison memory etc. - include/linux/kmsan.h: declarations of KMSAN memory hooks to be referenced outside KMSAN runtime - lib/Kconfig.kmsan: declarations for CONFIG_KMSAN and CONFIG_TEST_KMSAN - mm/kmsan/Makefile: boilerplate Makefile - mm/kmsan/kmsan.h: internal KMSAN declarations - mm/kmsan/kmsan.c: core functions that operate with shadow and origin memory and perform checks, utility functions - mm/kmsan/kmsan_entry.c: KMSAN hooks for entry_64.S - mm/kmsan/kmsan_hooks.c: KMSAN hooks for kernel subsystems - mm/kmsan/kmsan_init.c: KMSAN initialization routines - mm/kmsan/kmsan_instr.c: functions called by KMSAN instrumentation - scripts/Makefile.kmsan: CFLAGS_KMSAN Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- v2: - dropped kmsan_handle_vprintk() - use locking for single kmsan_pr_err() calls - don't try to understand we're inside printk() Change-Id: I4b3a7aba6d5804afac4f5f7274cadf8675b6e119 --- arch/x86/Kconfig | 1 + arch/x86/include/asm/kmsan.h | 129 ++++++++ include/linux/kmsan-checks.h | 121 ++++++++ include/linux/kmsan.h | 143 +++++++++ lib/Kconfig.debug | 2 + lib/Kconfig.kmsan | 22 ++ mm/kmsan/Makefile | 4 + mm/kmsan/kmsan.c | 570 +++++++++++++++++++++++++++++++++++ mm/kmsan/kmsan.h | 149 +++++++++ mm/kmsan/kmsan_entry.c | 130 ++++++++ mm/kmsan/kmsan_hooks.c | 393 ++++++++++++++++++++++++ mm/kmsan/kmsan_init.c | 88 ++++++ mm/kmsan/kmsan_instr.c | 259 ++++++++++++++++ mm/kmsan/kmsan_report.c | 133 ++++++++ mm/kmsan/kmsan_shadow.c | 543 +++++++++++++++++++++++++++++++++ mm/kmsan/kmsan_shadow.h | 30 ++ scripts/Makefile.kmsan | 12 + 17 files changed, 2729 insertions(+) create mode 100644 arch/x86/include/asm/kmsan.h create mode 100644 include/linux/kmsan-checks.h create mode 100644 include/linux/kmsan.h create mode 100644 lib/Kconfig.kmsan create mode 100644 mm/kmsan/Makefile create mode 100644 mm/kmsan/kmsan.c create mode 100644 mm/kmsan/kmsan.h create mode 100644 mm/kmsan/kmsan_entry.c create mode 100644 mm/kmsan/kmsan_hooks.c create mode 100644 mm/kmsan/kmsan_init.c create mode 100644 mm/kmsan/kmsan_instr.c create mode 100644 mm/kmsan/kmsan_report.c create mode 100644 mm/kmsan/kmsan_shadow.c create mode 100644 mm/kmsan/kmsan_shadow.h create mode 100644 scripts/Makefile.kmsan diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index d6e1faa28c58..3f83a5c53808 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -135,6 +135,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 + select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT diff --git a/arch/x86/include/asm/kmsan.h b/arch/x86/include/asm/kmsan.h new file mode 100644 index 000000000000..22322904102b --- /dev/null +++ b/arch/x86/include/asm/kmsan.h @@ -0,0 +1,129 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Assembly bits to safely invoke KMSAN hooks from .S files. + * + * Adopted from KTSAN assembly hooks implementation by Dmitry Vyukov: + * https://github.com/google/ktsan/blob/ktsan/arch/x86/include/asm/ktsan.h + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ +#ifndef _ASM_X86_KMSAN_H +#define _ASM_X86_KMSAN_H + +#ifdef CONFIG_KMSAN + +#define KMSAN_PUSH_REGS \ + pushq %rax; \ + pushq %rcx; \ + pushq %rdx; \ + pushq %rdi; \ + pushq %rsi; \ + pushq %r8; \ + pushq %r9; \ + pushq %r10; \ + pushq %r11; \ +/**/ + +#define KMSAN_POP_REGS \ + popq %r11; \ + popq %r10; \ + popq %r9; \ + popq %r8; \ + popq %rsi; \ + popq %rdi; \ + popq %rdx; \ + popq %rcx; \ + popq %rax; \ +/**/ + +#define KMSAN_INTERRUPT_ENTER \ + KMSAN_PUSH_REGS \ + call kmsan_interrupt_enter; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_INTERRUPT_EXIT \ + KMSAN_PUSH_REGS \ + call kmsan_interrupt_exit; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_SOFTIRQ_ENTER \ + KMSAN_PUSH_REGS \ + call kmsan_softirq_enter; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_SOFTIRQ_EXIT \ + KMSAN_PUSH_REGS \ + call kmsan_softirq_exit; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_NMI_ENTER \ + KMSAN_PUSH_REGS \ + call kmsan_nmi_enter; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_NMI_EXIT \ + KMSAN_PUSH_REGS \ + call kmsan_nmi_exit; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_SYSCALL_ENTER \ + KMSAN_PUSH_REGS \ + call kmsan_syscall_enter; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_SYSCALL_EXIT \ + KMSAN_PUSH_REGS \ + call kmsan_syscall_exit; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_IST_ENTER(shift_ist) \ + KMSAN_PUSH_REGS \ + movq $shift_ist, %rdi; \ + call kmsan_ist_enter; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_IST_EXIT(shift_ist) \ + KMSAN_PUSH_REGS \ + movq $shift_ist, %rdi; \ + call kmsan_ist_exit; \ + KMSAN_POP_REGS \ +/**/ + +#define KMSAN_UNPOISON_PT_REGS \ + KMSAN_PUSH_REGS \ + call kmsan_unpoison_pt_regs; \ + KMSAN_POP_REGS \ +/**/ + + +#else /* ifdef CONFIG_KMSAN */ + +#define KMSAN_INTERRUPT_ENTER +#define KMSAN_INTERRUPT_EXIT +#define KMSAN_SOFTIRQ_ENTER +#define KMSAN_SOFTIRQ_EXIT +#define KMSAN_NMI_ENTER +#define KMSAN_NMI_EXIT +#define KMSAN_SYSCALL_ENTER +#define KMSAN_SYSCALL_EXIT +#define KMSAN_IST_ENTER(shift_ist) +#define KMSAN_IST_EXIT(shift_ist) +#define KMSAN_UNPOISON_PT_REGS + +#endif /* ifdef CONFIG_KMSAN */ +#endif /* ifndef _ASM_X86_KMSAN_H */ diff --git a/include/linux/kmsan-checks.h b/include/linux/kmsan-checks.h new file mode 100644 index 000000000000..5c60540ba324 --- /dev/null +++ b/include/linux/kmsan-checks.h @@ -0,0 +1,121 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KMSAN checks. + * TODO(glider): unite with kmsan.h? + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef _LINUX_KMSAN_CHECKS_H +#define _LINUX_KMSAN_CHECKS_H + +#include +#include + +struct i2c_msg; +struct page; +struct sk_buff; +struct urb; + +#ifdef CONFIG_KMSAN + +/* + * Helper functions that mark the return value initialized. + * Note that Clang ignores the inline attribute in the cases when a no_sanitize + * function is called from an instrumented one. + */ + +__no_sanitize_memory +static inline unsigned char KMSAN_INIT_1(unsigned char value) +{ + return value; +} + +__no_sanitize_memory +static inline unsigned short KMSAN_INIT_2(unsigned short value) +{ + return value; +} + +__no_sanitize_memory +static inline unsigned int KMSAN_INIT_4(unsigned int value) +{ + return value; +} + +__no_sanitize_memory +static inline unsigned long KMSAN_INIT_8(unsigned long value) +{ + return value; +} + +#define KMSAN_INIT_VALUE(val) \ + ({ \ + typeof(val) __ret; \ + switch (sizeof(val)) { \ + case 1: \ + *(unsigned char *)&__ret = KMSAN_INIT_1( \ + (unsigned char)val); \ + break; \ + case 2: \ + *(unsigned short *)&__ret = KMSAN_INIT_2( \ + (unsigned short)val); \ + break; \ + case 4: \ + *(unsigned int *)&__ret = KMSAN_INIT_4( \ + (unsigned int)val); \ + break; \ + case 8: \ + *(unsigned long *)&__ret = KMSAN_INIT_8( \ + (unsigned long)val); \ + break; \ + default: \ + BUILD_BUG_ON(1); \ + } \ + __ret; \ + }) /**/ + +void kmsan_ignore_page(struct page *page, int order); +void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags); +void kmsan_unpoison_shadow(const void *address, size_t size); +void kmsan_check_memory(const void *address, size_t size); +void kmsan_check_skb(const struct sk_buff *skb); +void kmsan_handle_urb(const struct urb *urb, bool is_out); +void kmsan_handle_i2c_transfer(struct i2c_msg *msgs, int num); +void kmsan_copy_to_user(const void *to, const void *from, size_t to_copy, + size_t left); +void *__msan_memcpy(void *dst, const void *src, u64 n); +void kmsan_enter_runtime(unsigned long *flags); +void kmsan_leave_runtime(unsigned long *flags); + +#else + +#define KMSAN_INIT_VALUE(value) (value) + +static inline void kmsan_ignore_page(struct page *page, int order) {} +static inline void kmsan_poison_shadow(const void *address, size_t size, + gfp_t flags) {} +static inline void kmsan_unpoison_shadow(const void *address, size_t size) {} +static inline void kmsan_check_memory(const void *address, size_t size) {} +static inline void kmsan_check_skb(const struct sk_buff *skb) {} +static inline void kmsan_handle_urb(const struct urb *urb, bool is_out) {} +static inline void kmsan_handle_i2c_transfer(struct i2c_msg *msgs, int num) {} +static inline void kmsan_copy_to_user( + const void *to, const void *from, size_t to_copy, size_t left) {} +static inline void *__msan_memcpy(void *dst, const void *src, size_t n) +{ + return NULL; +} + +static inline void kmsan_enter_runtime(unsigned long *flags) {} +static inline void kmsan_leave_runtime(unsigned long *flags) {} + +#endif + +#endif /* _LINUX_KMSAN_CHECKS_H */ diff --git a/include/linux/kmsan.h b/include/linux/kmsan.h new file mode 100644 index 000000000000..f5638bac368e --- /dev/null +++ b/include/linux/kmsan.h @@ -0,0 +1,143 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KMSAN API for subsystems. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ +#ifndef LINUX_KMSAN_H +#define LINUX_KMSAN_H + +#include +#include +#include +#include + +struct page; +struct kmem_cache; +struct task_struct; +struct vm_struct; + + +extern bool kmsan_ready; + +#ifdef CONFIG_KMSAN +void __init kmsan_initialize_shadow(void); +void __init kmsan_initialize(void); + +/* These constants are defined in the MSan LLVM instrumentation pass. */ +#define RETVAL_SIZE 800 +#define KMSAN_PARAM_SIZE 800 + +#define PARAM_ARRAY_SIZE (KMSAN_PARAM_SIZE / sizeof(depot_stack_handle_t)) + +struct kmsan_context_state { + char param_tls[KMSAN_PARAM_SIZE]; + char retval_tls[RETVAL_SIZE]; + char va_arg_tls[KMSAN_PARAM_SIZE]; + char va_arg_origin_tls[KMSAN_PARAM_SIZE]; + u64 va_arg_overflow_size_tls; + depot_stack_handle_t param_origin_tls[PARAM_ARRAY_SIZE]; + depot_stack_handle_t retval_origin_tls; + depot_stack_handle_t origin_tls; +}; + +struct kmsan_task_state { + bool allow_reporting; + struct kmsan_context_state cstate; +}; + +void kmsan_task_create(struct task_struct *task); +void kmsan_task_exit(struct task_struct *task); +void kmsan_alloc_shadow_for_region(void *start, size_t size); +int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags); +void kmsan_gup_pgd_range(struct page **pages, int nr); +void kmsan_free_page(struct page *page, unsigned int order); +void kmsan_split_page(struct page *page, unsigned int order); +void kmsan_copy_page_meta(struct page *dst, struct page *src); + +void kmsan_poison_slab(struct page *page, gfp_t flags); +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void kmsan_kfree_large(const void *ptr); +void kmsan_kmalloc(struct kmem_cache *s, const void *object, size_t size, + gfp_t flags); +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void kmsan_slab_free(struct kmem_cache *s, void *object); + +void kmsan_slab_setup_object(struct kmem_cache *s, void *object); +void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags, + size_t size, void *object); + +/* vmap */ +void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end, + pgprot_t prot, struct page **pages); +void kmsan_vunmap_page_range(unsigned long addr, unsigned long end); + +/* ioremap */ +void kmsan_ioremap_page_range(unsigned long addr, unsigned long end, + phys_addr_t phys_addr, pgprot_t prot); +void kmsan_iounmap_page_range(unsigned long start, unsigned long end); + +void kmsan_softirq_enter(void); +void kmsan_softirq_exit(void); + +void kmsan_clear_page(void *page_addr); + +#else + +static inline void __init kmsan_initialize_shadow(void) { } +static inline void __init kmsan_initialize(void) { } + +static inline void kmsan_task_create(struct task_struct *task) {} +static inline void kmsan_task_exit(struct task_struct *task) {} +static inline void kmsan_alloc_shadow_for_region(void *start, size_t size) {} +static inline int kmsan_alloc_page(struct page *page, unsigned int order, + gfp_t flags) +{ + return 0; +} +static inline void kmsan_gup_pgd_range(struct page **pages, int nr) {} +static inline void kmsan_free_page(struct page *page, unsigned int order) {} +static inline void kmsan_split_page(struct page *page, unsigned int order) {} +static inline void kmsan_copy_page_meta(struct page *dst, struct page *src) {} + +static inline void kmsan_poison_slab(struct page *page, gfp_t flags) {} +static inline void kmsan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags) {} +static inline void kmsan_kfree_large(const void *ptr) {} +static inline void kmsan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) {} +static inline void kmsan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) {} +static inline void kmsan_slab_free(struct kmem_cache *s, void *object) {} + +static inline void kmsan_slab_setup_object(struct kmem_cache *s, + void *object) {} +static inline void kmsan_post_alloc_hook(struct kmem_cache *s, gfp_t flags, + size_t size, void *object) {} + +static inline void kmsan_vmap_page_range_noflush(unsigned long start, + unsigned long end, + pgprot_t prot, + struct page **pages) {} +static inline void kmsan_vunmap_page_range(unsigned long start, + unsigned long end) {} + +static inline void kmsan_ioremap_page_range(unsigned long start, + unsigned long end, + phys_addr_t phys_addr, + pgprot_t prot) {} +static inline void kmsan_iounmap_page_range(unsigned long start, + unsigned long end) {} +static inline void kmsan_softirq_enter(void) {} +static inline void kmsan_softirq_exit(void) {} + +static inline void kmsan_clear_page(void *page_addr) {} +#endif + +#endif /* LINUX_KMSAN_H */ diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 93d97f9b0157..75c36318943d 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -756,6 +756,8 @@ config DEBUG_STACKOVERFLOW source "lib/Kconfig.kasan" +source "lib/Kconfig.kmsan" + endmenu # "Memory Debugging" config ARCH_HAS_KCOV diff --git a/lib/Kconfig.kmsan b/lib/Kconfig.kmsan new file mode 100644 index 000000000000..187dddfcf220 --- /dev/null +++ b/lib/Kconfig.kmsan @@ -0,0 +1,22 @@ +config HAVE_ARCH_KMSAN + bool + +if HAVE_ARCH_KMSAN + +config KMSAN + bool "KMSAN: detector of uninitialized memory use" + depends on SLUB && !KASAN + select STACKDEPOT + help + KMSAN is a dynamic detector of uses of uninitialized memory in the + kernel. It is based on compiler instrumentation provided by Clang + and thus requires Clang 10.0.0+ to build. + +config TEST_KMSAN + tristate "Module for testing KMSAN for bug detection" + depends on m && KMSAN + help + Test module that can trigger various uses of uninitialized memory + detectable by KMSAN. + +endif diff --git a/mm/kmsan/Makefile b/mm/kmsan/Makefile new file mode 100644 index 000000000000..ccf6d2d00a7a --- /dev/null +++ b/mm/kmsan/Makefile @@ -0,0 +1,4 @@ +obj-y := kmsan.o kmsan_instr.o kmsan_init.o kmsan_entry.o kmsan_hooks.o kmsan_report.o kmsan_shadow.o + +KMSAN_SANITIZE := n +KCOV_INSTRUMENT := n diff --git a/mm/kmsan/kmsan.c b/mm/kmsan/kmsan.c new file mode 100644 index 000000000000..fecb82dc5f4c --- /dev/null +++ b/mm/kmsan/kmsan.c @@ -0,0 +1,570 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN runtime library. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "../slab.h" +#include "kmsan.h" + +/* + * Some kernel asm() calls mention the non-existing |__force_order| variable + * in the asm constraints to preserve the order of accesses to control + * registers. KMSAN turns those mentions into actual memory accesses, therefore + * the variable is now required to link the kernel. + */ +unsigned long __force_order; + +bool kmsan_ready; +#define KMSAN_STACK_DEPTH 64 +#define MAX_CHAIN_DEPTH 7 + +/* + * According to Documentation/x86/kernel-stacks, kernel code can run on the + * following stacks: + * - regular task stack - when executing the task code + * - interrupt stack - when handling external hardware interrupts and softirqs + * - NMI stack + * 0 is for regular interrupts, 1 for softirqs, 2 for NMI. + * Because interrupts may nest, trying to use a new context for every new + * interrupt. + */ +/* [0] for dummy per-CPU context. */ +DEFINE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX], + kmsan_percpu_cstate); +/* 0 for task context, |i>0| for kmsan_context_state[i]. */ +DEFINE_PER_CPU(int, kmsan_context_level); +DEFINE_PER_CPU(int, kmsan_in_interrupt); +DEFINE_PER_CPU(bool, kmsan_in_softirq); +DEFINE_PER_CPU(bool, kmsan_in_nmi); +DEFINE_PER_CPU(int, kmsan_in_runtime); +/* TODO(glider): debug-only. */ +DEFINE_PER_CPU(unsigned long, kmsan_runtime_last_caller); + +struct kmsan_context_state *task_kmsan_context_state(void) +{ + int cpu = smp_processor_id(); + int level = this_cpu_read(kmsan_context_level); + struct kmsan_context_state *ret; + + if (!kmsan_ready || IN_RUNTIME()) { + ret = &per_cpu(kmsan_percpu_cstate[0], cpu); + __memset(ret, 0, sizeof(struct kmsan_context_state)); + return ret; + } + + if (!level) + ret = ¤t->kmsan.cstate; + else + ret = &per_cpu(kmsan_percpu_cstate[level], cpu); + return ret; +} + +void kmsan_internal_task_create(struct task_struct *task) +{ + struct kmsan_task_state *state = &task->kmsan; + + __memset(state, 0, sizeof(struct kmsan_task_state)); + state->allow_reporting = true; +} + +void kmsan_internal_memset_shadow(void *addr, int b, size_t size, + bool checked) +{ + void *shadow_start; + u64 page_offset, address = (u64)addr; + size_t to_fill; + + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW)); + while (size) { + page_offset = address % PAGE_SIZE; + to_fill = min(PAGE_SIZE - page_offset, (u64)size); + shadow_start = kmsan_get_metadata((void *)address, to_fill, + META_SHADOW); + if (!shadow_start) { + if (checked) { + kmsan_pr_locked("WARNING: not memsetting %d bytes starting at %px, because the shadow is NULL\n", to_fill, address); + BUG(); + } + /* Otherwise just move on. */ + } else { + __memset(shadow_start, b, to_fill); + } + address += to_fill; + size -= to_fill; + } +} + +void kmsan_internal_poison_shadow(void *address, size_t size, + gfp_t flags, unsigned int poison_flags) +{ + bool checked = poison_flags & KMSAN_POISON_CHECK; + depot_stack_handle_t handle; + u32 extra_bits = 0; + + if (poison_flags & KMSAN_POISON_FREE) + extra_bits = 1; + kmsan_internal_memset_shadow(address, -1, size, checked); + handle = kmsan_save_stack_with_flags(flags, extra_bits); + kmsan_set_origin_checked(address, size, handle, checked); +} + +void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked) +{ + kmsan_internal_memset_shadow(address, 0, size, checked); + kmsan_set_origin_checked(address, size, 0, checked); +} + +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, + unsigned int reserved) +{ + depot_stack_handle_t handle; + unsigned long entries[KMSAN_STACK_DEPTH]; + unsigned int nr_entries; + + nr_entries = stack_trace_save(entries, KMSAN_STACK_DEPTH, 0); + filter_irq_stacks(entries, nr_entries); + + /* Don't sleep (see might_sleep_if() in __alloc_pages_nodemask()). */ + flags &= ~__GFP_DIRECT_RECLAIM; + + handle = stack_depot_save(entries, nr_entries, flags); + return set_dsh_extra_bits(handle, reserved); +} + +/* + * Depending on the value of is_memmove, this serves as both a memcpy and a + * memmove implementation. + * + * As with the regular memmove, do the following: + * - if src and dst don't overlap, use memcpy(); + * - if src and dst overlap: + * - if src > dst, use memcpy(); + * - if src < dst, use reverse-memcpy. + * Why this is correct: + * - problems may arise if for some part of the overlapping region we + * overwrite its shadow with a new value before copying it somewhere. + * But there's a 1:1 mapping between the kernel memory and its shadow, + * therefore if this doesn't happen with the kernel memory it can't happen + * with the shadow. + */ +void kmsan_memcpy_memmove_metadata(void *dst, void *src, size_t n, + bool is_memmove) +{ + void *shadow_src, *shadow_dst; + depot_stack_handle_t *origin_src, *origin_dst; + int src_slots, dst_slots, i, iter, step, skip_bits; + depot_stack_handle_t old_origin = 0, chain_origin, new_origin = 0; + u32 *align_shadow_src, shadow; + bool backwards; + + BUG_ON(!metadata_is_contiguous(dst, n, META_SHADOW)); + BUG_ON(!metadata_is_contiguous(src, n, META_SHADOW)); + + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW); + if (!shadow_dst) + return; + + shadow_src = kmsan_get_metadata(src, n, META_SHADOW); + if (!shadow_src) { + /* + * |src| is untracked: zero out destination shadow, ignore the + * origins, we're done. + */ + __memset(shadow_dst, 0, n); + return; + } + if (is_memmove) + __memmove(shadow_dst, shadow_src, n); + else + __memcpy(shadow_dst, shadow_src, n); + + origin_dst = kmsan_get_metadata(dst, n, META_ORIGIN); + origin_src = kmsan_get_metadata(src, n, META_ORIGIN); + BUG_ON(!origin_dst || !origin_src); + BUG_ON(!metadata_is_contiguous(dst, n, META_ORIGIN)); + BUG_ON(!metadata_is_contiguous(src, n, META_ORIGIN)); + src_slots = (ALIGN((u64)src + n, ORIGIN_SIZE) - + ALIGN_DOWN((u64)src, ORIGIN_SIZE)) / ORIGIN_SIZE; + dst_slots = (ALIGN((u64)dst + n, ORIGIN_SIZE) - + ALIGN_DOWN((u64)dst, ORIGIN_SIZE)) / ORIGIN_SIZE; + BUG_ON(!src_slots || !dst_slots); + BUG_ON((src_slots < 1) || (dst_slots < 1)); + BUG_ON((src_slots - dst_slots > 1) || (dst_slots - src_slots < -1)); + + backwards = is_memmove && (dst > src); + i = backwards ? min(src_slots, dst_slots) - 1 : 0; + iter = backwards ? -1 : 1; + + align_shadow_src = (u32 *)ALIGN_DOWN((u64)shadow_src, ORIGIN_SIZE); + for (step = 0; step < min(src_slots, dst_slots); step++, i += iter) { + BUG_ON(i < 0); + shadow = align_shadow_src[i]; + if (i == 0) { + /* + * If |src| isn't aligned on ORIGIN_SIZE, don't + * look at the first |src % ORIGIN_SIZE| bytes + * of the first shadow slot. + */ + skip_bits = ((u64)src % ORIGIN_SIZE) * 8; + shadow = (shadow << skip_bits) >> skip_bits; + } + if (i == src_slots - 1) { + /* + * If |src + n| isn't aligned on + * ORIGIN_SIZE, don't look at the last + * |(src + n) % ORIGIN_SIZE| bytes of the + * last shadow slot. + */ + skip_bits = (((u64)src + n) % ORIGIN_SIZE) * 8; + shadow = (shadow >> skip_bits) << skip_bits; + } + /* + * Overwrite the origin only if the corresponding + * shadow is nonempty. + */ + if (origin_src[i] && (origin_src[i] != old_origin) && shadow) { + old_origin = origin_src[i]; + chain_origin = kmsan_internal_chain_origin(old_origin); + /* + * kmsan_internal_chain_origin() may return + * NULL, but we don't want to lose the previous + * origin value. + */ + if (chain_origin) + new_origin = chain_origin; + else + new_origin = old_origin; + } + if (shadow) + origin_dst[i] = new_origin; + else + origin_dst[i] = 0; + } +} + +void kmsan_memcpy_metadata(void *dst, void *src, size_t n) +{ + kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/false); +} + +void kmsan_memmove_metadata(void *dst, void *src, size_t n) +{ + kmsan_memcpy_memmove_metadata(dst, src, n, /*is_memmove*/true); +} + +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id) +{ + depot_stack_handle_t handle; + unsigned long entries[3]; + u64 magic = KMSAN_CHAIN_MAGIC_ORIGIN_FULL; + int depth = 0; + static int skipped; + u32 extra_bits; + + if (!kmsan_ready) + return 0; + + if (!id) + return id; + /* + * Make sure we have enough spare bits in |id| to hold the UAF bit and + * the chain depth. + */ + BUILD_BUG_ON((1 << STACK_DEPOT_EXTRA_BITS) <= (MAX_CHAIN_DEPTH << 1)); + + extra_bits = get_dsh_extra_bits(id); + + depth = extra_bits >> 1; + if (depth >= MAX_CHAIN_DEPTH) { + skipped++; + if (skipped % 10000 == 0) { + kmsan_pr_locked("not chained %d origins\n", skipped); + dump_stack(); + kmsan_print_origin(id); + } + return id; + } + depth++; + /* Lowest bit is the UAF flag, higher bits hold the depth. */ + extra_bits = (depth << 1) | (extra_bits & 1); + /* TODO(glider): how do we figure out we've dropped some frames? */ + entries[0] = magic + depth; + entries[1] = kmsan_save_stack_with_flags(GFP_ATOMIC, extra_bits); + entries[2] = id; + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC); + return set_dsh_extra_bits(handle, extra_bits); +} + +void kmsan_write_aligned_origin(void *var, size_t size, u32 origin) +{ + u32 *var_cast = (u32 *)var; + int i; + + BUG_ON((u64)var_cast % ORIGIN_SIZE); + BUG_ON(size % ORIGIN_SIZE); + for (i = 0; i < size / ORIGIN_SIZE; i++) + var_cast[i] = origin; +} + +/* + * TODO(glider): writing an initialized byte shouldn't zero out the origin, if + * the remaining three bytes are uninitialized. + */ +void kmsan_internal_set_origin(void *addr, int size, u32 origin) +{ + void *origin_start; + u64 address = (u64)addr, page_offset; + size_t to_fill, pad = 0; + + if (!IS_ALIGNED(address, ORIGIN_SIZE)) { + pad = address % ORIGIN_SIZE; + address -= pad; + size += pad; + } + + while (size > 0) { + page_offset = address % PAGE_SIZE; + to_fill = min(PAGE_SIZE - page_offset, (u64)size); + /* write at least ORIGIN_SIZE bytes */ + to_fill = ALIGN(to_fill, ORIGIN_SIZE); + BUG_ON(!to_fill); + origin_start = kmsan_get_metadata((void *)address, to_fill, + META_ORIGIN); + address += to_fill; + size -= to_fill; + if (!origin_start) + /* Can happen e.g. if the memory is untracked. */ + continue; + kmsan_write_aligned_origin(origin_start, to_fill, origin); + } +} + +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked) +{ + if (checked && !metadata_is_contiguous(addr, size, META_ORIGIN)) { + kmsan_pr_locked("WARNING: not setting origin for %d bytes starting at %px, because the metadata is incontiguous\n", size, addr); + BUG(); + } + kmsan_internal_set_origin(addr, size, origin); +} + +struct page *vmalloc_to_page_or_null(void *vaddr) +{ + struct page *page; + + if (!kmsan_internal_is_vmalloc_addr(vaddr) && + !kmsan_internal_is_module_addr(vaddr)) + return NULL; + page = vmalloc_to_page(vaddr); + if (pfn_valid(page_to_pfn(page))) + return page; + else + return NULL; +} + +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr, + int reason) +{ + unsigned long irq_flags; + unsigned long addr64 = (unsigned long)addr; + unsigned char *shadow = NULL; + depot_stack_handle_t *origin = NULL; + depot_stack_handle_t cur_origin = 0, new_origin = 0; + int cur_off_start = -1; + int i, chunk_size; + size_t pos = 0; + + BUG_ON(!metadata_is_contiguous(addr, size, META_SHADOW)); + if (size <= 0) + return; + while (pos < size) { + chunk_size = min(size - pos, + PAGE_SIZE - ((addr64 + pos) % PAGE_SIZE)); + shadow = kmsan_get_metadata((void *)(addr64 + pos), chunk_size, + META_SHADOW); + if (!shadow) { + /* + * This page is untracked. If there were uninitialized + * bytes before, report them. + */ + if (cur_origin) { + ENTER_RUNTIME(irq_flags); + kmsan_report(cur_origin, addr, size, + cur_off_start, pos - 1, user_addr, + reason); + LEAVE_RUNTIME(irq_flags); + } + cur_origin = 0; + cur_off_start = -1; + pos += chunk_size; + continue; + } + for (i = 0; i < chunk_size; i++) { + if (!shadow[i]) { + /* + * This byte is unpoisoned. If there were + * poisoned bytes before, report them. + */ + if (cur_origin) { + ENTER_RUNTIME(irq_flags); + kmsan_report(cur_origin, addr, size, + cur_off_start, pos + i - 1, + user_addr, reason); + LEAVE_RUNTIME(irq_flags); + } + cur_origin = 0; + cur_off_start = -1; + continue; + } + origin = kmsan_get_metadata((void *)(addr64 + pos + i), + chunk_size - i, META_ORIGIN); + BUG_ON(!origin); + new_origin = *origin; + /* + * Encountered new origin - report the previous + * uninitialized range. + */ + if (cur_origin != new_origin) { + if (cur_origin) { + ENTER_RUNTIME(irq_flags); + kmsan_report(cur_origin, addr, size, + cur_off_start, pos + i - 1, + user_addr, reason); + LEAVE_RUNTIME(irq_flags); + } + cur_origin = new_origin; + cur_off_start = pos + i; + } + } + pos += chunk_size; + } + BUG_ON(pos != size); + if (cur_origin) { + ENTER_RUNTIME(irq_flags); + kmsan_report(cur_origin, addr, size, cur_off_start, pos - 1, + user_addr, reason); + LEAVE_RUNTIME(irq_flags); + } +} + +/* + * TODO(glider): this check shouldn't be performed for origin pages, because + * they're always accessed after the shadow pages. + */ +bool metadata_is_contiguous(void *addr, size_t size, bool is_origin) +{ + u64 cur_addr = (u64)addr, next_addr; + char *cur_meta = NULL, *next_meta = NULL; + depot_stack_handle_t *origin_p; + bool all_untracked = false; + const char *fname = is_origin ? "origin" : "shadow"; + + if (!size) + return true; + + /* The whole range belongs to the same page. */ + if (ALIGN_DOWN(cur_addr + size - 1, PAGE_SIZE) == + ALIGN_DOWN(cur_addr, PAGE_SIZE)) + return true; + cur_meta = kmsan_get_metadata((void *)cur_addr, 1, is_origin); + if (!cur_meta) + all_untracked = true; + for (next_addr = cur_addr + PAGE_SIZE; next_addr < (u64)addr + size; + cur_addr = next_addr, + cur_meta = next_meta, + next_addr += PAGE_SIZE) { + next_meta = kmsan_get_metadata((void *)next_addr, 1, is_origin); + if (!next_meta) { + if (!all_untracked) + goto report; + continue; + } + if ((u64)cur_meta == ((u64)next_meta - PAGE_SIZE)) + continue; + goto report; + } + return true; + +report: + kmsan_pr_locked("BUG: attempting to access two shadow page ranges.\n"); + dump_stack(); + kmsan_pr_locked("\n"); + kmsan_pr_locked("Access of size %d at %px.\n", size, addr); + kmsan_pr_locked("Addresses belonging to different ranges: %px and %px\n", + cur_addr, next_addr); + kmsan_pr_locked("page[0].%s: %px, page[1].%s: %px\n", + fname, cur_meta, fname, next_meta); + origin_p = kmsan_get_metadata(addr, 1, META_ORIGIN); + if (origin_p) { + kmsan_pr_locked("Origin: %08x\n", *origin_p); + kmsan_print_origin(*origin_p); + } else { + kmsan_pr_locked("Origin: unavailable\n"); + } + return false; +} + +/* + * Dummy replacement for __builtin_return_address() which may crash without + * frame pointers. + */ +void *kmsan_internal_return_address(int arg) +{ +#ifdef CONFIG_UNWINDER_FRAME_POINTER + switch (arg) { + case 1: + return __builtin_return_address(1); + case 2: + return __builtin_return_address(2); + default: + BUG(); + } +#else + unsigned long entries[1]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = 1, + .skip = arg + }; + save_stack_trace(&trace); + return entries[0]; +#endif +} + +bool kmsan_internal_is_module_addr(void *vaddr) +{ + return ((u64)vaddr >= MODULES_VADDR) && ((u64)vaddr < MODULES_END); +} + +bool kmsan_internal_is_vmalloc_addr(void *addr) +{ + return ((u64)addr >= VMALLOC_START) && ((u64)addr < VMALLOC_END); +} diff --git a/mm/kmsan/kmsan.h b/mm/kmsan/kmsan.h new file mode 100644 index 000000000000..4cb3723e2d76 --- /dev/null +++ b/mm/kmsan/kmsan.h @@ -0,0 +1,149 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KMSAN internal declarations. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __MM_KMSAN_KMSAN_H +#define __MM_KMSAN_KMSAN_H + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kmsan_shadow.h" + +#define KMSAN_MAGIC_MASK 0xffffffffff00 +#define KMSAN_ALLOCA_MAGIC_ORIGIN 0x4110c4071900 +#define KMSAN_CHAIN_MAGIC_ORIGIN_FULL 0xd419170cba00 + +#define KMSAN_POISON_NOCHECK 0x0 +#define KMSAN_POISON_CHECK 0x1 +#define KMSAN_POISON_FREE 0x2 + +#define ORIGIN_SIZE 4 + +#define META_SHADOW (false) +#define META_ORIGIN (true) + +#define KMSAN_NESTED_CONTEXT_MAX (8) +/* [0] for dummy per-CPU context */ +DECLARE_PER_CPU(struct kmsan_context_state[KMSAN_NESTED_CONTEXT_MAX], + kmsan_percpu_cstate); +/* 0 for task context, |i>0| for kmsan_context_state[i]. */ +DECLARE_PER_CPU(int, kmsan_context_level); +DECLARE_PER_CPU(int, kmsan_in_interrupt); +DECLARE_PER_CPU(bool, kmsan_in_softirq); +DECLARE_PER_CPU(bool, kmsan_in_nmi); + +extern spinlock_t report_lock; + +/* Stolen from kernel/printk/internal.h */ +#define PRINTK_SAFE_CONTEXT_MASK 0x3fffffff + +/* Called by kmsan_report.c under a lock. */ +#define kmsan_pr_err(...) pr_err(__VA_ARGS__) + +/* Used in other places - doesn't require a lock. */ +#define kmsan_pr_locked(...) \ + do { \ + unsigned long flags; \ + spin_lock_irqsave(&report_lock, flags); \ + pr_err(__VA_ARGS__); \ + spin_unlock_irqrestore(&report_lock, flags); \ + } while (0) + +void kmsan_print_origin(depot_stack_handle_t origin); +void kmsan_report(depot_stack_handle_t origin, + void *address, int size, int off_first, int off_last, + const void *user_addr, int reason); + + +enum KMSAN_BUG_REASON { + REASON_ANY = 0, + REASON_COPY_TO_USER = 1, + REASON_USE_AFTER_FREE = 2, + REASON_SUBMIT_URB = 3, +}; + +/* + * When a compiler hook is invoked, it may make a call to instrumented code + * and eventually call itself recursively. To avoid that, we protect the + * runtime entry points with ENTER_RUNTIME()/LEAVE_RUNTIME() macros and exit + * the hook if IN_RUNTIME() is true. But when an interrupt occurs inside the + * runtime, the hooks won’t run either, which may lead to errors. + * Therefore we have to disable interrupts inside the runtime. + */ +DECLARE_PER_CPU(int, kmsan_in_runtime); +DECLARE_PER_CPU(unsigned long, kmsan_runtime_last_caller); +#define IN_RUNTIME() (this_cpu_read(kmsan_in_runtime)) +#define ENTER_RUNTIME(irq_flags) \ + do { \ + preempt_disable(); \ + local_irq_save(irq_flags); \ + stop_nmi(); \ + this_cpu_inc(kmsan_in_runtime); \ + this_cpu_write(kmsan_runtime_last_caller, _THIS_IP_); \ + BUG_ON(this_cpu_read(kmsan_in_runtime) > 1); \ + } while (0) +#define LEAVE_RUNTIME(irq_flags) \ + do { \ + this_cpu_dec(kmsan_in_runtime); \ + if (this_cpu_read(kmsan_in_runtime)) { \ + kmsan_pr_err("kmsan_in_runtime: %d, last_caller: %pS\n", \ + this_cpu_read(kmsan_in_runtime), \ + this_cpu_read(kmsan_runtime_last_caller)); \ + BUG(); \ + } \ + restart_nmi(); \ + local_irq_restore(irq_flags); \ + preempt_enable(); } while (0) + +void kmsan_memcpy_metadata(void *dst, void *src, size_t n); +void kmsan_memmove_metadata(void *dst, void *src, size_t n); + +depot_stack_handle_t kmsan_save_stack(void); +depot_stack_handle_t kmsan_save_stack_with_flags(gfp_t flags, + unsigned int extra_bits); +void kmsan_internal_poison_shadow(void *address, size_t size, gfp_t flags, + unsigned int poison_flags); +void kmsan_internal_unpoison_shadow(void *address, size_t size, bool checked); +void kmsan_internal_memset_shadow(void *address, int b, size_t size, + bool checked); +depot_stack_handle_t kmsan_internal_chain_origin(depot_stack_handle_t id); +void kmsan_write_aligned_origin(void *var, size_t size, u32 origin); + +void kmsan_internal_task_create(struct task_struct *task); +void kmsan_internal_set_origin(void *addr, int size, u32 origin); +void kmsan_set_origin_checked(void *addr, int size, u32 origin, bool checked); + +struct kmsan_context_state *task_kmsan_context_state(void); + +bool metadata_is_contiguous(void *addr, size_t size, bool is_origin); +void kmsan_internal_check_memory(void *addr, size_t size, const void *user_addr, + int reason); + +struct page *vmalloc_to_page_or_null(void *vaddr); + +/* Declared in mm/vmalloc.c */ +void __vunmap_page_range(unsigned long addr, unsigned long end); +int __vmap_page_range_noflush(unsigned long start, unsigned long end, + pgprot_t prot, struct page **pages); + +void *kmsan_internal_return_address(int arg); +bool kmsan_internal_is_module_addr(void *vaddr); +bool kmsan_internal_is_vmalloc_addr(void *addr); + +#endif /* __MM_KMSAN_KMSAN_H */ diff --git a/mm/kmsan/kmsan_entry.c b/mm/kmsan/kmsan_entry.c new file mode 100644 index 000000000000..9511a7dad541 --- /dev/null +++ b/mm/kmsan/kmsan_entry.c @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN hooks for entry_64.S + * + * Copyright (C) 2018-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include "kmsan.h" + +static void kmsan_context_enter(void) +{ + int level = this_cpu_read(kmsan_context_level) + 1; + + BUG_ON(level >= KMSAN_NESTED_CONTEXT_MAX); + this_cpu_write(kmsan_context_level, level); +} + +static void kmsan_context_exit(void) +{ + int level = this_cpu_read(kmsan_context_level) - 1; + + BUG_ON(level < 0); + this_cpu_write(kmsan_context_level, level); +} + +void kmsan_interrupt_enter(void) +{ + int in_interrupt = this_cpu_read(kmsan_in_interrupt); + + /* Turns out it's possible for in_interrupt to be >0 here. */ + kmsan_context_enter(); + BUG_ON(in_interrupt > 1); + /* Can't check preempt_count() here, it may be zero. */ + this_cpu_write(kmsan_in_interrupt, in_interrupt + 1); +} +EXPORT_SYMBOL(kmsan_interrupt_enter); + +void kmsan_interrupt_exit(void) +{ + int in_interrupt = this_cpu_read(kmsan_in_interrupt); + + BUG_ON(!in_interrupt); + kmsan_context_exit(); + /* Can't check preempt_count() here, it may be zero. */ + this_cpu_write(kmsan_in_interrupt, in_interrupt - 1); +} +EXPORT_SYMBOL(kmsan_interrupt_exit); + +void kmsan_softirq_enter(void) +{ + bool in_softirq = this_cpu_read(kmsan_in_softirq); + + BUG_ON(in_softirq); + kmsan_context_enter(); + /* Can't check preempt_count() here, it may be zero. */ + this_cpu_write(kmsan_in_softirq, true); +} +EXPORT_SYMBOL(kmsan_softirq_enter); + +void kmsan_softirq_exit(void) +{ + bool in_softirq = this_cpu_read(kmsan_in_softirq); + + BUG_ON(!in_softirq); + kmsan_context_exit(); + /* Can't check preempt_count() here, it may be zero. */ + this_cpu_write(kmsan_in_softirq, false); +} +EXPORT_SYMBOL(kmsan_softirq_exit); + +void kmsan_nmi_enter(void) +{ + bool in_nmi = this_cpu_read(kmsan_in_nmi); + + BUG_ON(in_nmi); + BUG_ON(preempt_count() & NMI_MASK); + kmsan_context_enter(); + this_cpu_write(kmsan_in_nmi, true); +} +EXPORT_SYMBOL(kmsan_nmi_enter); + +void kmsan_nmi_exit(void) +{ + bool in_nmi = this_cpu_read(kmsan_in_nmi); + + BUG_ON(!in_nmi); + BUG_ON(preempt_count() & NMI_MASK); + kmsan_context_exit(); + this_cpu_write(kmsan_in_nmi, false); + +} +EXPORT_SYMBOL(kmsan_nmi_exit); + +void kmsan_syscall_enter(void) +{ + +} +EXPORT_SYMBOL(kmsan_syscall_enter); + +void kmsan_syscall_exit(void) +{ + +} +EXPORT_SYMBOL(kmsan_syscall_exit); + +void kmsan_ist_enter(u64 shift_ist) +{ + kmsan_context_enter(); +} +EXPORT_SYMBOL(kmsan_ist_enter); + +void kmsan_ist_exit(u64 shift_ist) +{ + kmsan_context_exit(); +} +EXPORT_SYMBOL(kmsan_ist_exit); + +void kmsan_unpoison_pt_regs(struct pt_regs *regs) +{ + if (!kmsan_ready || IN_RUNTIME()) + return; + kmsan_internal_unpoison_shadow(regs, sizeof(*regs), /*checked*/true); +} +EXPORT_SYMBOL(kmsan_unpoison_pt_regs); diff --git a/mm/kmsan/kmsan_hooks.c b/mm/kmsan/kmsan_hooks.c new file mode 100644 index 000000000000..37b362d0cea9 --- /dev/null +++ b/mm/kmsan/kmsan_hooks.c @@ -0,0 +1,393 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN hooks for kernel subsystems. + * + * These functions handle creation of KMSAN metadata for memory allocations. + * + * Copyright (C) 2018-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../slab.h" +#include "kmsan.h" + +/* TODO(glider): do we need to export these symbols? */ + +/* + * The functions may call back to instrumented code, which, in turn, may call + * these hooks again. To avoid re-entrancy, we use __GFP_NO_KMSAN_SHADOW. + * Instrumented functions shouldn't be called under + * ENTER_RUNTIME()/LEAVE_RUNTIME(), because this will lead to skipping + * effects of functions like memset() inside instrumented code. + */ +/* Called from kernel/kthread.c, kernel/fork.c */ +void kmsan_task_create(struct task_struct *task) +{ + unsigned long irq_flags; + + if (!task) + return; + ENTER_RUNTIME(irq_flags); + kmsan_internal_task_create(task); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_task_create); + + +/* Called from kernel/exit.c */ +void kmsan_task_exit(struct task_struct *task) +{ + unsigned long irq_flags; + struct kmsan_task_state *state = &task->kmsan; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + ENTER_RUNTIME(irq_flags); + state->allow_reporting = false; + + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_task_exit); + +/* Called from mm/slub.c */ +void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) +{ + unsigned long irq_flags; + + if (unlikely(object == NULL)) + return; + if (!kmsan_ready || IN_RUNTIME()) + return; + /* + * There's a ctor or this is an RCU cache - do nothing. The memory + * status hasn't changed since last use. + */ + if (s->ctor || (s->flags & SLAB_TYPESAFE_BY_RCU)) + return; + + ENTER_RUNTIME(irq_flags); + if (flags & __GFP_ZERO) { + kmsan_internal_unpoison_shadow(object, s->object_size, + KMSAN_POISON_CHECK); + } else { + kmsan_internal_poison_shadow(object, s->object_size, flags, + KMSAN_POISON_CHECK); + } + LEAVE_RUNTIME(irq_flags); +} + +/* Called from mm/slub.c */ +void kmsan_slab_free(struct kmem_cache *s, void *object) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + ENTER_RUNTIME(irq_flags); + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON))) + goto leave; + if (s->ctor) + goto leave; + kmsan_internal_poison_shadow(object, s->object_size, + GFP_KERNEL, + KMSAN_POISON_CHECK | KMSAN_POISON_FREE); +leave: + LEAVE_RUNTIME(irq_flags); +} + +/* Called from mm/slub.c */ +void kmsan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + unsigned long irq_flags; + + if (unlikely(ptr == NULL)) + return; + if (!kmsan_ready || IN_RUNTIME()) + return; + ENTER_RUNTIME(irq_flags); + if (flags & __GFP_ZERO) { + kmsan_internal_unpoison_shadow((void *)ptr, size, + /*checked*/true); + } else { + kmsan_internal_poison_shadow((void *)ptr, size, flags, + KMSAN_POISON_CHECK); + } + LEAVE_RUNTIME(irq_flags); +} + +/* Called from mm/slub.c */ +void kmsan_kfree_large(const void *ptr) +{ + struct page *page; + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + ENTER_RUNTIME(irq_flags); + page = virt_to_head_page((void *)ptr); + BUG_ON(ptr != page_address(page)); + kmsan_internal_poison_shadow( + (void *)ptr, PAGE_SIZE << compound_order(page), GFP_KERNEL, + KMSAN_POISON_CHECK | KMSAN_POISON_FREE); + LEAVE_RUNTIME(irq_flags); +} + + +static unsigned long vmalloc_shadow(unsigned long addr) +{ + return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_SHADOW); +} + +static unsigned long vmalloc_origin(unsigned long addr) +{ + return (unsigned long)kmsan_get_metadata((void *)addr, 1, META_ORIGIN); +} + +/* Called from mm/vmalloc.c */ +void kmsan_vunmap_page_range(unsigned long start, unsigned long end) +{ + __vunmap_page_range(vmalloc_shadow(start), vmalloc_shadow(end)); + __vunmap_page_range(vmalloc_origin(start), vmalloc_origin(end)); +} + +/* Called from lib/ioremap.c */ +/* + * This function creates new shadow/origin pages for the physical pages mapped + * into the virtual memory. If those physical pages already had shadow/origin, + * those are ignored. + */ +void kmsan_ioremap_page_range(unsigned long start, unsigned long end, + phys_addr_t phys_addr, pgprot_t prot) +{ + unsigned long irq_flags; + struct page *shadow, *origin; + int i, nr; + unsigned long off = 0; + gfp_t gfp_mask = GFP_KERNEL | __GFP_ZERO | __GFP_NO_KMSAN_SHADOW; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + nr = (end - start) / PAGE_SIZE; + ENTER_RUNTIME(irq_flags); + for (i = 0; i < nr; i++, off += PAGE_SIZE) { + shadow = alloc_pages(gfp_mask, 1); + origin = alloc_pages(gfp_mask, 1); + __vmap_page_range_noflush(vmalloc_shadow(start + off), + vmalloc_shadow(start + off + PAGE_SIZE), + prot, &shadow); + __vmap_page_range_noflush(vmalloc_origin(start + off), + vmalloc_origin(start + off + PAGE_SIZE), + prot, &origin); + } + flush_cache_vmap(vmalloc_shadow(start), vmalloc_shadow(end)); + flush_cache_vmap(vmalloc_origin(start), vmalloc_origin(end)); + LEAVE_RUNTIME(irq_flags); +} + +void kmsan_iounmap_page_range(unsigned long start, unsigned long end) +{ + int i, nr; + struct page *shadow, *origin; + unsigned long v_shadow, v_origin; + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + nr = (end - start) / PAGE_SIZE; + ENTER_RUNTIME(irq_flags); + v_shadow = (unsigned long)vmalloc_shadow(start); + v_origin = (unsigned long)vmalloc_origin(start); + for (i = 0; i < nr; i++, v_shadow += PAGE_SIZE, v_origin += PAGE_SIZE) { + shadow = vmalloc_to_page_or_null((void *)v_shadow); + origin = vmalloc_to_page_or_null((void *)v_origin); + __vunmap_page_range(v_shadow, v_shadow + PAGE_SIZE); + __vunmap_page_range(v_origin, v_origin + PAGE_SIZE); + if (shadow) + __free_pages(shadow, 1); + if (origin) + __free_pages(origin, 1); + } + LEAVE_RUNTIME(irq_flags); +} + +/* Called from include/linux/uaccess.h, include/linux/uaccess.h */ +void kmsan_copy_to_user(const void *to, const void *from, + size_t to_copy, size_t left) +{ + void *shadow; + + if (!kmsan_ready || IN_RUNTIME()) + return; + /* + * At this point we've copied the memory already. It's hard to check it + * before copying, as the size of actually copied buffer is unknown. + */ + + /* copy_to_user() may copy zero bytes. No need to check. */ + if (!to_copy) + return; + /* Or maybe copy_to_user() failed to copy anything. */ + if (to_copy == left) + return; + if ((u64)to < TASK_SIZE) { + /* This is a user memory access, check it. */ + kmsan_internal_check_memory((void *)from, to_copy - left, to, + REASON_COPY_TO_USER); + return; + } + /* Otherwise this is a kernel memory access. This happens when a compat + * syscall passes an argument allocated on the kernel stack to a real + * syscall. + * Don't check anything, just copy the shadow of the copied bytes. + */ + shadow = kmsan_get_metadata((void *)to, to_copy - left, META_SHADOW); + if (shadow) + kmsan_memcpy_metadata((void *)to, (void *)from, to_copy - left); +} +EXPORT_SYMBOL(kmsan_copy_to_user); + +void kmsan_poison_shadow(const void *address, size_t size, gfp_t flags) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + ENTER_RUNTIME(irq_flags); + /* The users may want to poison/unpoison random memory. */ + kmsan_internal_poison_shadow((void *)address, size, flags, + KMSAN_POISON_NOCHECK); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_poison_shadow); + +void kmsan_unpoison_shadow(const void *address, size_t size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + ENTER_RUNTIME(irq_flags); + /* The users may want to poison/unpoison random memory. */ + kmsan_internal_unpoison_shadow((void *)address, size, + KMSAN_POISON_NOCHECK); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_unpoison_shadow); + +void kmsan_check_memory(const void *addr, size_t size) +{ + return kmsan_internal_check_memory((void *)addr, size, /*user_addr*/ 0, + REASON_ANY); +} +EXPORT_SYMBOL(kmsan_check_memory); + +void kmsan_gup_pgd_range(struct page **pages, int nr) +{ + int i; + void *page_addr; + + /* + * gup_pgd_range() has just created a number of new pages that KMSAN + * treats as uninitialized. In the case they belong to the userspace + * memory, unpoison the corresponding kernel pages. + */ + for (i = 0; i < nr; i++) { + page_addr = page_address(pages[i]); + if (((u64)page_addr < TASK_SIZE) && + ((u64)page_addr + PAGE_SIZE < TASK_SIZE)) + kmsan_unpoison_shadow(page_addr, PAGE_SIZE); + } + +} +EXPORT_SYMBOL(kmsan_gup_pgd_range); + +/* Helper function to check an SKB. */ +void kmsan_check_skb(const struct sk_buff *skb) +{ + int start = skb_headlen(skb); + struct sk_buff *frag_iter; + int i, copy = 0; + skb_frag_t *f; + u32 p_off, p_len, copied; + struct page *p; + u8 *vaddr; + + if (!skb || !skb->len) + return; + + kmsan_internal_check_memory(skb->data, skb_headlen(skb), 0, REASON_ANY); + if (skb_is_nonlinear(skb)) { + for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) { + f = &skb_shinfo(skb)->frags[i]; + + skb_frag_foreach_page(f, + skb_frag_off(f) - start, + copy, p, p_off, p_len, copied) { + + vaddr = kmap_atomic(p); + kmsan_internal_check_memory(vaddr + p_off, + p_len, /*user_addr*/ 0, + REASON_ANY); + kunmap_atomic(vaddr); + } + } + } + skb_walk_frags(skb, frag_iter) + kmsan_check_skb(frag_iter); +} +EXPORT_SYMBOL(kmsan_check_skb); + +/* Helper function to check an URB. */ +void kmsan_handle_urb(const struct urb *urb, bool is_out) +{ + if (!urb) + return; + if (is_out) + kmsan_internal_check_memory(urb->transfer_buffer, + urb->transfer_buffer_length, + /*user_addr*/ 0, REASON_SUBMIT_URB); + else + kmsan_internal_unpoison_shadow(urb->transfer_buffer, + urb->transfer_buffer_length, + /*checked*/false); +} +EXPORT_SYMBOL(kmsan_handle_urb); + +/* Helper function to check I2C-transferred data. */ +void kmsan_handle_i2c_transfer(struct i2c_msg *msgs, int num) +{ + int i; + + if (!msgs) + return; + for (i = 0; i < num; i++) { + if (msgs[i].flags & I2C_M_RD) + kmsan_internal_unpoison_shadow(msgs[i].buf, + msgs[i].len, + /*checked*/false); + else + kmsan_internal_check_memory(msgs[i].buf, msgs[i].len, + /*user_addr*/0, + REASON_ANY); + } +} diff --git a/mm/kmsan/kmsan_init.c b/mm/kmsan/kmsan_init.c new file mode 100644 index 000000000000..2816e7075a30 --- /dev/null +++ b/mm/kmsan/kmsan_init.c @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN initialization routines. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include "kmsan.h" + +#include +#include +#include + +#define NUM_FUTURE_RANGES 128 +struct start_end_pair { + void *start, *end; +}; + +static struct start_end_pair start_end_pairs[NUM_FUTURE_RANGES] __initdata; +static int future_index __initdata; + +/* + * Record a range of memory for which the metadata pages will be created once + * the page allocator becomes available. + * TODO(glider): squash together ranges belonging to the same page. + */ +static void __init kmsan_record_future_shadow_range(void *start, void *end) +{ + BUG_ON(future_index == NUM_FUTURE_RANGES); + BUG_ON((start >= end) || !start || !end); + start_end_pairs[future_index].start = start; + start_end_pairs[future_index].end = end; + future_index++; +} + +extern char _sdata[], _edata[]; + + + +/* + * Initialize the shadow for existing mappings during kernel initialization. + * These include kernel text/data sections, NODE_DATA and future ranges + * registered while creating other data (e.g. percpu). + * + * Allocations via memblock can be only done before slab is initialized. + */ +void __init kmsan_initialize_shadow(void) +{ + int nid; + u64 i; + const size_t nd_size = roundup(sizeof(pg_data_t), PAGE_SIZE); + phys_addr_t p_start, p_end; + + for_each_reserved_mem_region(i, &p_start, &p_end) { + kmsan_record_future_shadow_range(phys_to_virt(p_start), + phys_to_virt(p_end+1)); + } + /* Allocate shadow for .data */ + kmsan_record_future_shadow_range(_sdata, _edata); + + /* + * TODO(glider): alloc_node_data() in arch/x86/mm/numa.c uses + * sizeof(pg_data_t). + */ + for_each_online_node(nid) + kmsan_record_future_shadow_range( + NODE_DATA(nid), (char *)NODE_DATA(nid) + nd_size); + + for (i = 0; i < future_index; i++) + kmsan_init_alloc_meta_for_range(start_end_pairs[i].start, + start_end_pairs[i].end); +} +EXPORT_SYMBOL(kmsan_initialize_shadow); + +void __init kmsan_initialize(void) +{ + /* Assuming current is init_task */ + kmsan_internal_task_create(current); + kmsan_pr_locked("Starting KernelMemorySanitizer\n"); + kmsan_ready = true; +} +EXPORT_SYMBOL(kmsan_initialize); diff --git a/mm/kmsan/kmsan_instr.c b/mm/kmsan/kmsan_instr.c new file mode 100644 index 000000000000..74cb7cee7f70 --- /dev/null +++ b/mm/kmsan/kmsan_instr.c @@ -0,0 +1,259 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN compiler API. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include "kmsan.h" +#include +#include + +static bool is_bad_asm_addr(void *addr, u64 size, bool is_store) +{ + if ((u64)addr < TASK_SIZE) + return true; + if (!kmsan_get_metadata(addr, size, META_SHADOW)) + return true; + return false; +} + +struct shadow_origin_ptr __msan_metadata_ptr_for_load_n(void *addr, u64 size) +{ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_n); + +struct shadow_origin_ptr __msan_metadata_ptr_for_store_n(void *addr, u64 size) +{ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); +} +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_n); + +#define DECLARE_METADATA_PTR_GETTER(size) \ +struct shadow_origin_ptr __msan_metadata_ptr_for_load_##size(void *addr) \ +{ \ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/false); \ +} \ +EXPORT_SYMBOL(__msan_metadata_ptr_for_load_##size); \ + \ +struct shadow_origin_ptr __msan_metadata_ptr_for_store_##size(void *addr) \ +{ \ + return kmsan_get_shadow_origin_ptr(addr, size, /*store*/true); \ +} \ +EXPORT_SYMBOL(__msan_metadata_ptr_for_store_##size) + +DECLARE_METADATA_PTR_GETTER(1); +DECLARE_METADATA_PTR_GETTER(2); +DECLARE_METADATA_PTR_GETTER(4); +DECLARE_METADATA_PTR_GETTER(8); + +void __msan_instrument_asm_store(void *addr, u64 size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + /* + * Most of the accesses are below 32 bytes. The two exceptions so far + * are clwb() (64 bytes) and FPU state (512 bytes). + * It's unlikely that the assembly will touch more than 512 bytes. + */ + if (size > 512) + size = 8; + if (is_bad_asm_addr(addr, size, /*is_store*/true)) + return; + ENTER_RUNTIME(irq_flags); + /* Unpoisoning the memory on best effort. */ + kmsan_internal_unpoison_shadow(addr, size, /*checked*/false); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(__msan_instrument_asm_store); + +void *__msan_memmove(void *dst, void *src, u64 n) +{ + void *result; + void *shadow_dst; + + result = __memmove(dst, src, n); + if (!n) + /* Some people call memmove() with zero length. */ + return result; + if (!kmsan_ready || IN_RUNTIME()) + return result; + + /* Ok to skip address check here, we'll do it later. */ + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW); + + if (!shadow_dst) + /* Can happen e.g. if the memory is untracked. */ + return result; + + kmsan_memmove_metadata(dst, src, n); + + return result; +} +EXPORT_SYMBOL(__msan_memmove); + +void *__msan_memmove_nosanitize(void *dst, void *src, u64 n) +{ + return __memmove(dst, src, n); +} +EXPORT_SYMBOL(__msan_memmove_nosanitize); + +void *__msan_memcpy(void *dst, const void *src, u64 n) +{ + void *result; + void *shadow_dst; + + result = __memcpy(dst, src, n); + if (!n) + /* Some people call memcpy() with zero length. */ + return result; + + if (!kmsan_ready || IN_RUNTIME()) + return result; + + /* Ok to skip address check here, we'll do it later. */ + shadow_dst = kmsan_get_metadata(dst, n, META_SHADOW); + if (!shadow_dst) + /* Can happen e.g. if the memory is untracked. */ + return result; + + kmsan_memcpy_metadata(dst, (void *)src, n); + + return result; +} +EXPORT_SYMBOL(__msan_memcpy); + +void *__msan_memcpy_nosanitize(void *dst, void *src, u64 n) +{ + return __memcpy(dst, src, n); +} +EXPORT_SYMBOL(__msan_memcpy_nosanitize); + +void *__msan_memset(void *dst, int c, size_t n) +{ + void *result; + unsigned long irq_flags; + depot_stack_handle_t new_origin; + unsigned int shadow; + + result = __memset(dst, c, n); + if (!kmsan_ready || IN_RUNTIME()) + return result; + + ENTER_RUNTIME(irq_flags); + shadow = 0; + kmsan_internal_memset_shadow(dst, shadow, n, /*checked*/false); + new_origin = 0; + kmsan_internal_set_origin(dst, n, new_origin); + LEAVE_RUNTIME(irq_flags); + + return result; +} +EXPORT_SYMBOL(__msan_memset); + +void *__msan_memset_nosanitize(void *dst, int c, size_t n) +{ + return __memset(dst, c, n); +} +EXPORT_SYMBOL(__msan_memset_nosanitize); + +depot_stack_handle_t __msan_chain_origin(depot_stack_handle_t origin) +{ + depot_stack_handle_t ret = 0; + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return ret; + + /* Creating new origins may allocate memory. */ + ENTER_RUNTIME(irq_flags); + ret = kmsan_internal_chain_origin(origin); + LEAVE_RUNTIME(irq_flags); + return ret; +} +EXPORT_SYMBOL(__msan_chain_origin); + +void __msan_poison_alloca(void *address, u64 size, char *descr) +{ + depot_stack_handle_t handle; + unsigned long entries[4]; + unsigned long irq_flags; + u64 size_copy = size, to_fill; + u64 addr_copy = (u64)address; + u64 page_offset; + void *shadow_start; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + while (size_copy) { + page_offset = addr_copy % PAGE_SIZE; + to_fill = min(PAGE_SIZE - page_offset, size_copy); + shadow_start = kmsan_get_metadata((void *)addr_copy, to_fill, + META_SHADOW); + if (!shadow_start) + /* Can happen e.g. if the memory is untracked. */ + continue; + __memset(shadow_start, -1, to_fill); + addr_copy += to_fill; + size_copy -= to_fill; + } + + entries[0] = KMSAN_ALLOCA_MAGIC_ORIGIN; + entries[1] = (u64)descr; + entries[2] = (u64)__builtin_return_address(0); + entries[3] = (u64)kmsan_internal_return_address(1); + + /* stack_depot_save() may allocate memory. */ + ENTER_RUNTIME(irq_flags); + handle = stack_depot_save(entries, ARRAY_SIZE(entries), GFP_ATOMIC); + LEAVE_RUNTIME(irq_flags); + kmsan_internal_set_origin(address, size, handle); +} +EXPORT_SYMBOL(__msan_poison_alloca); + +void __msan_unpoison_alloca(void *address, u64 size) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + ENTER_RUNTIME(irq_flags); + /* Assuming the shadow exists. */ + kmsan_internal_unpoison_shadow(address, size, /*checked*/true); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(__msan_unpoison_alloca); + +void __msan_warning(u32 origin) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + ENTER_RUNTIME(irq_flags); + kmsan_report(origin, /*address*/0, /*size*/0, + /*off_first*/0, /*off_last*/0, /*user_addr*/0, REASON_ANY); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(__msan_warning); + +struct kmsan_context_state *__msan_get_context_state(void) +{ + struct kmsan_context_state *ret; + + ret = task_kmsan_context_state(); + BUG_ON(!ret); + return ret; +} +EXPORT_SYMBOL(__msan_get_context_state); diff --git a/mm/kmsan/kmsan_report.c b/mm/kmsan/kmsan_report.c new file mode 100644 index 000000000000..443ab9c1e8bf --- /dev/null +++ b/mm/kmsan/kmsan_report.c @@ -0,0 +1,133 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN error reporting routines. + * + * Copyright (C) 2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include + +#include "kmsan.h" + +DEFINE_SPINLOCK(report_lock); + +void kmsan_print_origin(depot_stack_handle_t origin) +{ + unsigned long *entries = NULL, *chained_entries = NULL; + unsigned long nr_entries, chained_nr_entries, magic; + char *descr = NULL; + void *pc1 = NULL, *pc2 = NULL; + depot_stack_handle_t head; + + if (!origin) { + kmsan_pr_err("Origin not found, presumably a false report.\n"); + return; + } + + while (true) { + nr_entries = stack_depot_fetch(origin, &entries); + magic = nr_entries ? (entries[0] & KMSAN_MAGIC_MASK) : 0; + if ((nr_entries == 4) && (magic == KMSAN_ALLOCA_MAGIC_ORIGIN)) { + descr = (char *)entries[1]; + pc1 = (void *)entries[2]; + pc2 = (void *)entries[3]; + kmsan_pr_err("Local variable description: %s\n", descr); + kmsan_pr_err("Variable was created at:\n"); + kmsan_pr_err(" %pS\n", pc1); + kmsan_pr_err(" %pS\n", pc2); + break; + } + if ((nr_entries == 3) && + (magic == KMSAN_CHAIN_MAGIC_ORIGIN_FULL)) { + head = entries[1]; + origin = entries[2]; + kmsan_pr_err("Uninit was stored to memory at:\n"); + chained_nr_entries = + stack_depot_fetch(head, &chained_entries); + stack_trace_print(chained_entries, chained_nr_entries, + 0); + kmsan_pr_err("\n"); + continue; + } + kmsan_pr_err("Uninit was created at:\n"); + if (entries) + stack_trace_print(entries, nr_entries, 0); + else + kmsan_pr_err("No stack\n"); + break; + } +} + +void kmsan_report(depot_stack_handle_t origin, + void *address, int size, int off_first, int off_last, + const void *user_addr, int reason) +{ + unsigned long flags; + unsigned long *entries; + unsigned int nr_entries; + bool is_uaf = false; + char *bug_type = NULL; + + if (!kmsan_ready) + return; + if (!current->kmsan.allow_reporting) + return; + if (!origin) + return; + + nr_entries = stack_depot_fetch(origin, &entries); + + current->kmsan.allow_reporting = false; + spin_lock_irqsave(&report_lock, flags); + kmsan_pr_err("=====================================================\n"); + if (get_dsh_extra_bits(origin) & 1) + is_uaf = true; + switch (reason) { + case REASON_ANY: + bug_type = is_uaf ? "use-after-free" : "uninit-value"; + break; + case REASON_COPY_TO_USER: + bug_type = is_uaf ? "kernel-infoleak-after-free" : + "kernel-infoleak"; + break; + case REASON_SUBMIT_URB: + bug_type = is_uaf ? "kernel-usb-infoleak-after-free" : + "kernel-usb-infoleak"; + break; + } + kmsan_pr_err("BUG: KMSAN: %s in %pS\n", + bug_type, kmsan_internal_return_address(2)); + dump_stack(); + kmsan_pr_err("\n"); + + kmsan_print_origin(origin); + + if (size) { + kmsan_pr_err("\n"); + if (off_first == off_last) + kmsan_pr_err("Byte %d of %d is uninitialized\n", + off_first, size); + else + kmsan_pr_err("Bytes %d-%d of %d are uninitialized\n", + off_first, off_last, size); + } + if (address) + kmsan_pr_err("Memory access of size %d starts at %px\n", + size, address); + if (user_addr && reason == REASON_COPY_TO_USER) + kmsan_pr_err("Data copied to user address %px\n", user_addr); + kmsan_pr_err("=====================================================\n"); + add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); + spin_unlock_irqrestore(&report_lock, flags); + if (panic_on_warn) + panic("panic_on_warn set ...\n"); + current->kmsan.allow_reporting = true; +} diff --git a/mm/kmsan/kmsan_shadow.c b/mm/kmsan/kmsan_shadow.c new file mode 100644 index 000000000000..06801d76e6b8 --- /dev/null +++ b/mm/kmsan/kmsan_shadow.c @@ -0,0 +1,543 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KMSAN shadow implementation. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kmsan.h" +#include "kmsan_shadow.h" + +#define shadow_page_for(page) \ + ((page)->shadow) + +#define origin_page_for(page) \ + ((page)->origin) + +#define shadow_ptr_for(page) \ + (page_address((page)->shadow)) + +#define origin_ptr_for(page) \ + (page_address((page)->origin)) + +#define has_shadow_page(page) \ + (!!((page)->shadow)) + +#define has_origin_page(page) \ + (!!((page)->origin)) + +#define set_no_shadow_origin_page(page) \ + do { \ + (page)->shadow = NULL; \ + (page)->origin = NULL; \ + } while (0) /**/ + +#define is_ignored_page(page) \ + (!!(((u64)((page)->shadow)) % 2)) + +#define ignore_page(pg) \ + ((pg)->shadow = (struct page *)((u64)((pg)->shadow) | 1)) \ + +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_shadow); +DEFINE_PER_CPU(char[CPU_ENTRY_AREA_SIZE], cpu_entry_area_origin); + +/* + * Dummy load and store pages to be used when the real metadata is unavailable. + * There are separate pages for loads and stores, so that every load returns a + * zero, and every store doesn't affect other stores. + */ +char dummy_load_page[PAGE_SIZE] __aligned(PAGE_SIZE); +char dummy_store_page[PAGE_SIZE] __aligned(PAGE_SIZE); + +/* + * Taken from arch/x86/mm/physaddr.h to avoid using an instrumented version. + */ +static int kmsan_phys_addr_valid(unsigned long addr) +{ +#ifdef CONFIG_PHYS_ADDR_T_64BIT + return !(addr >> boot_cpu_data.x86_phys_bits); +#else + return 1; +#endif +} + +/* + * Taken from arch/x86/mm/physaddr.c to avoid using an instrumented version. + */ +static bool kmsan_virt_addr_valid(void *addr) +{ + unsigned long x = (unsigned long)addr; + unsigned long y = x - __START_KERNEL_map; + + /* use the carry flag to determine if x was < __START_KERNEL_map */ + if (unlikely(x > y)) { + x = y + phys_base; + + if (y >= KERNEL_IMAGE_SIZE) + return false; + } else { + x = y + (__START_KERNEL_map - PAGE_OFFSET); + + /* carry flag will be set if starting x was >= PAGE_OFFSET */ + if ((x > y) || !kmsan_phys_addr_valid(x)) + return false; + } + + return pfn_valid(x >> PAGE_SHIFT); +} + +static unsigned long vmalloc_meta(void *addr, bool is_origin) +{ + unsigned long addr64 = (unsigned long)addr, off; + + BUG_ON(is_origin && !IS_ALIGNED(addr64, ORIGIN_SIZE)); + if (kmsan_internal_is_vmalloc_addr(addr)) { + return addr64 + (is_origin ? VMALLOC_ORIGIN_OFFSET + : VMALLOC_SHADOW_OFFSET); + } + if (kmsan_internal_is_module_addr(addr)) { + off = addr64 - MODULES_VADDR; + return off + (is_origin ? MODULES_ORIGIN_START + : MODULES_SHADOW_START); + } + return 0; +} + +static void *get_cea_meta_or_null(void *addr, bool is_origin) +{ + int cpu = smp_processor_id(); + int off; + char *metadata_array; + + if (((u64)addr < CPU_ENTRY_AREA_BASE) || + ((u64)addr >= (CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE))) + return NULL; + off = (char *)addr - (char *)get_cpu_entry_area(cpu); + if ((off < 0) || (off >= CPU_ENTRY_AREA_SIZE)) + return NULL; + metadata_array = is_origin ? cpu_entry_area_origin : + cpu_entry_area_shadow; + return &per_cpu(metadata_array[off], cpu); +} + +static struct page *virt_to_page_or_null(void *vaddr) +{ + if (kmsan_virt_addr_valid(vaddr)) + return virt_to_page(vaddr); + else + return NULL; +} + +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *address, u64 size, + bool store) +{ + struct shadow_origin_ptr ret; + struct page *page; + u64 pad, offset, o_offset; + const u64 addr64 = (u64)address; + u64 o_addr64 = (u64)address; + void *shadow; + + if (size > PAGE_SIZE) { + WARN(1, "size too big in %s(%px, %d, %d)\n", + __func__, address, size, store); + BUG(); + } + if (store) { + ret.s = dummy_store_page; + ret.o = dummy_store_page; + } else { + ret.s = dummy_load_page; + ret.o = dummy_load_page; + } + if (!kmsan_ready || IN_RUNTIME()) + return ret; + BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW)); + + if (!IS_ALIGNED(addr64, ORIGIN_SIZE)) { + pad = addr64 % ORIGIN_SIZE; + o_addr64 -= pad; + } + + if (kmsan_internal_is_vmalloc_addr(address) || + kmsan_internal_is_module_addr(address)) { + ret.s = (void *)vmalloc_meta(address, META_SHADOW); + ret.o = (void *)vmalloc_meta((void *)o_addr64, META_ORIGIN); + return ret; + } + + if (!kmsan_virt_addr_valid(address)) { + page = vmalloc_to_page_or_null(address); + if (page) + goto next; + shadow = get_cea_meta_or_null(address, META_SHADOW); + if (shadow) { + ret.s = shadow; + ret.o = get_cea_meta_or_null((void *)o_addr64, + META_ORIGIN); + return ret; + } + } + page = virt_to_page_or_null(address); + if (!page) + return ret; +next: + if (is_ignored_page(page)) + return ret; + + if (!has_shadow_page(page) || !has_origin_page(page)) + return ret; + offset = addr64 % PAGE_SIZE; + o_offset = o_addr64 % PAGE_SIZE; + + if (offset + size - 1 > PAGE_SIZE) { + /* + * The access overflows the current page and touches the + * subsequent ones. Make sure the shadow/origin pages are also + * consequent. + */ + BUG_ON(!metadata_is_contiguous(address, size, META_SHADOW)); + } + + ret.s = shadow_ptr_for(page) + offset; + ret.o = origin_ptr_for(page) + o_offset; + return ret; +} + +/* + * Obtain the shadow or origin pointer for the given address, or NULL if there's + * none. The caller must check the return value for being non-NULL if needed. + * The return value of this function should not depend on whether we're in the + * runtime or not. + */ +void *kmsan_get_metadata(void *address, size_t size, bool is_origin) +{ + struct page *page; + void *ret; + u64 addr = (u64)address, pad, off; + + if (is_origin && !IS_ALIGNED(addr, ORIGIN_SIZE)) { + pad = addr % ORIGIN_SIZE; + addr -= pad; + size += pad; + } + address = (void *)addr; + if (kmsan_internal_is_vmalloc_addr(address) || + kmsan_internal_is_module_addr(address)) { + return (void *)vmalloc_meta(address, is_origin); + } + + if (!kmsan_virt_addr_valid(address)) { + page = vmalloc_to_page_or_null(address); + if (page) + goto next; + ret = get_cea_meta_or_null(address, is_origin); + if (ret) + return ret; + } + page = virt_to_page_or_null(address); + if (!page) + return NULL; +next: + if (is_ignored_page(page)) + return NULL; + if (!has_shadow_page(page) || !has_origin_page(page)) + return NULL; + off = addr % PAGE_SIZE; + + ret = (is_origin ? origin_ptr_for(page) : shadow_ptr_for(page)) + off; + return ret; +} + +void __init kmsan_init_alloc_meta_for_range(void *start, void *end) +{ + u64 addr, size; + struct page *page; + void *shadow, *origin; + struct page *shadow_p, *origin_p; + + start = (void *)ALIGN_DOWN((u64)start, PAGE_SIZE); + size = ALIGN((u64)end - (u64)start, PAGE_SIZE); + shadow = memblock_alloc(size, PAGE_SIZE); + origin = memblock_alloc(size, PAGE_SIZE); + for (addr = 0; addr < size; addr += PAGE_SIZE) { + page = virt_to_page_or_null((char *)start + addr); + shadow_p = virt_to_page_or_null((char *)shadow + addr); + set_no_shadow_origin_page(shadow_p); + shadow_page_for(page) = shadow_p; + origin_p = virt_to_page_or_null((char *)origin + addr); + set_no_shadow_origin_page(origin_p); + origin_page_for(page) = origin_p; + } +} + +/* Called from mm/memory.c */ +void kmsan_copy_page_meta(struct page *dst, struct page *src) +{ + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + if (!has_shadow_page(src)) { + /* TODO(glider): are we leaking pages here? */ + set_no_shadow_origin_page(dst); + return; + } + if (!has_shadow_page(dst)) + return; + if (is_ignored_page(src)) { + ignore_page(dst); + return; + } + + ENTER_RUNTIME(irq_flags); + __memcpy(shadow_ptr_for(dst), shadow_ptr_for(src), + PAGE_SIZE); + BUG_ON(!has_origin_page(src) || !has_origin_page(dst)); + __memcpy(origin_ptr_for(dst), origin_ptr_for(src), + PAGE_SIZE); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_copy_page_meta); + +/* Helper function to allocate page metadata. */ +static int kmsan_internal_alloc_meta_for_pages(struct page *page, + unsigned int order, + gfp_t flags, int node) +{ + struct page *shadow, *origin; + int pages = 1 << order; + int i; + bool initialized = (flags & __GFP_ZERO) || !kmsan_ready; + depot_stack_handle_t handle; + + if (flags & __GFP_NO_KMSAN_SHADOW) { + for (i = 0; i < pages; i++) + set_no_shadow_origin_page(&page[i]); + return 0; + } + + /* TODO(glider): must we override the flags? */ + flags = GFP_ATOMIC; + if (initialized) + flags |= __GFP_ZERO; + shadow = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order); + if (!shadow) { + for (i = 0; i < pages; i++) { + set_no_shadow_origin_page(&page[i]); + set_no_shadow_origin_page(&page[i]); + } + return -ENOMEM; + } + if (!initialized) + __memset(page_address(shadow), -1, PAGE_SIZE * pages); + + origin = alloc_pages_node(node, flags | __GFP_NO_KMSAN_SHADOW, order); + /* Assume we've allocated the origin. */ + if (!origin) { + __free_pages(shadow, order); + for (i = 0; i < pages; i++) + set_no_shadow_origin_page(&page[i]); + return -ENOMEM; + } + + if (!initialized) { + handle = kmsan_save_stack_with_flags(flags, /*extra_bits*/0); + /* + * Addresses are page-aligned, pages are contiguous, so it's ok + * to just fill the origin pages with |handle|. + */ + for (i = 0; i < PAGE_SIZE * pages / sizeof(handle); i++) { + ((depot_stack_handle_t *)page_address(origin))[i] = + handle; + } + } + + for (i = 0; i < pages; i++) { + shadow_page_for(&page[i]) = &shadow[i]; + set_no_shadow_origin_page(shadow_page_for(&page[i])); + origin_page_for(&page[i]) = &origin[i]; + set_no_shadow_origin_page(origin_page_for(&page[i])); + } + return 0; +} + +/* Called from mm/page_alloc.c */ +int kmsan_alloc_page(struct page *page, unsigned int order, gfp_t flags) +{ + unsigned long irq_flags; + int ret; + + if (IN_RUNTIME()) + return 0; + ENTER_RUNTIME(irq_flags); + ret = kmsan_internal_alloc_meta_for_pages(page, order, flags, -1); + LEAVE_RUNTIME(irq_flags); + return ret; +} + +/* Called from mm/page_alloc.c */ +void kmsan_free_page(struct page *page, unsigned int order) +{ + struct page *shadow, *origin, *cur_page; + int pages = 1 << order; + int i; + unsigned long irq_flags; + + if (!shadow_page_for(page)) { + for (i = 0; i < pages; i++) { + cur_page = &page[i]; + BUG_ON(shadow_page_for(cur_page)); + } + return; + } + + if (!kmsan_ready) { + for (i = 0; i < pages; i++) { + cur_page = &page[i]; + set_no_shadow_origin_page(cur_page); + } + return; + } + + if (IN_RUNTIME()) { + /* + * TODO(glider): looks legit. depot_save_stack() may call + * free_pages(). + */ + return; + } + + ENTER_RUNTIME(irq_flags); + shadow = shadow_page_for(&page[0]); + origin = origin_page_for(&page[0]); + + /* TODO(glider): this is racy. */ + for (i = 0; i < pages; i++) { + BUG_ON(has_shadow_page(shadow_page_for(&page[i]))); + BUG_ON(has_shadow_page(origin_page_for(&page[i]))); + set_no_shadow_origin_page(&page[i]); + } + BUG_ON(has_shadow_page(shadow)); + __free_pages(shadow, order); + + BUG_ON(has_shadow_page(origin)); + __free_pages(origin, order); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_free_page); + +/* Called from mm/page_alloc.c */ +void kmsan_split_page(struct page *page, unsigned int order) +{ + struct page *shadow, *origin; + unsigned long irq_flags; + + if (!kmsan_ready || IN_RUNTIME()) + return; + + ENTER_RUNTIME(irq_flags); + if (!has_shadow_page(&page[0])) { + BUG_ON(has_origin_page(&page[0])); + LEAVE_RUNTIME(irq_flags); + return; + } + shadow = shadow_page_for(&page[0]); + split_page(shadow, order); + + origin = origin_page_for(&page[0]); + split_page(origin, order); + LEAVE_RUNTIME(irq_flags); +} +EXPORT_SYMBOL(kmsan_split_page); + +/* Called from include/linux/highmem.h */ +void kmsan_clear_page(void *page_addr) +{ + struct page *page; + + if (!kmsan_ready || IN_RUNTIME()) + return; + BUG_ON(!IS_ALIGNED((u64)page_addr, PAGE_SIZE)); + page = vmalloc_to_page_or_null(page_addr); + if (!page) + page = virt_to_page_or_null(page_addr); + if (!page || !has_shadow_page(page)) + return; + __memset(shadow_ptr_for(page), 0, PAGE_SIZE); + BUG_ON(!has_origin_page(page)); + __memset(origin_ptr_for(page), 0, PAGE_SIZE); +} +EXPORT_SYMBOL(kmsan_clear_page); + +/* Called from mm/vmalloc.c */ +void kmsan_vmap_page_range_noflush(unsigned long start, unsigned long end, + pgprot_t prot, struct page **pages) +{ + int nr, i, mapped; + struct page **s_pages, **o_pages; + unsigned long shadow_start, shadow_end, origin_start, origin_end; + + if (!kmsan_ready || IN_RUNTIME()) + return; + shadow_start = vmalloc_meta((void *)start, META_SHADOW); + if (!shadow_start) + return; + + BUG_ON(start >= end); + nr = (end - start) / PAGE_SIZE; + s_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL); + o_pages = kcalloc(nr, sizeof(struct page *), GFP_KERNEL); + if (!s_pages || !o_pages) + goto ret; + for (i = 0; i < nr; i++) { + s_pages[i] = shadow_page_for(pages[i]); + o_pages[i] = origin_page_for(pages[i]); + } + prot = __pgprot(pgprot_val(prot) | _PAGE_NX); + prot = PAGE_KERNEL; + + shadow_end = vmalloc_meta((void *)end, META_SHADOW); + origin_start = vmalloc_meta((void *)start, META_ORIGIN); + origin_end = vmalloc_meta((void *)end, META_ORIGIN); + mapped = __vmap_page_range_noflush(shadow_start, shadow_end, + prot, s_pages); + BUG_ON(mapped != nr); + flush_tlb_kernel_range(shadow_start, shadow_end); + mapped = __vmap_page_range_noflush(origin_start, origin_end, + prot, o_pages); + BUG_ON(mapped != nr); + flush_tlb_kernel_range(origin_start, origin_end); +ret: + kfree(s_pages); + kfree(o_pages); +} + +void kmsan_ignore_page(struct page *page, int order) +{ + int pages = 1 << order; + int i; + struct page *cp; + + for (i = 0; i < pages; i++) { + cp = &page[i]; + ignore_page(cp); + } +} diff --git a/mm/kmsan/kmsan_shadow.h b/mm/kmsan/kmsan_shadow.h new file mode 100644 index 000000000000..eaa7f771b6a5 --- /dev/null +++ b/mm/kmsan/kmsan_shadow.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KMSAN shadow API. + * + * This should be agnostic to shadow implementation details. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#ifndef __MM_KMSAN_KMSAN_SHADOW_H +#define __MM_KMSAN_KMSAN_SHADOW_H + +#include /* for CPU_ENTRY_AREA_MAP_SIZE */ + +struct shadow_origin_ptr { + void *s, *o; +}; + +struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size, + bool store); +void *kmsan_get_metadata(void *addr, size_t size, bool is_origin); +void __init kmsan_init_alloc_meta_for_range(void *start, void *end); + +#endif /* __MM_KMSAN_KMSAN_SHADOW_H */ diff --git a/scripts/Makefile.kmsan b/scripts/Makefile.kmsan new file mode 100644 index 000000000000..8b3844b66b22 --- /dev/null +++ b/scripts/Makefile.kmsan @@ -0,0 +1,12 @@ +ifdef CONFIG_KMSAN + +CFLAGS_KMSAN := -fsanitize=kernel-memory + +ifeq ($(call cc-option, $(CFLAGS_KMSAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KMSAN: \ + -fsanitize=kernel-memory is not supported by compiler) + endif +endif + +endif From patchwork Wed Oct 30 14:22:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219625 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE148112B for ; Wed, 30 Oct 2019 14:23:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8A20720656 for ; Wed, 30 Oct 2019 14:23:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EVCxrwaV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A20720656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 608356B026B; Wed, 30 Oct 2019 10:23:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 568596B026C; Wed, 30 Oct 2019 10:23:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 455C36B026D; Wed, 30 Oct 2019 10:23:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0160.hostedemail.com [216.40.44.160]) by kanga.kvack.org (Postfix) with ESMTP id 21E8A6B026B for ; Wed, 30 Oct 2019 10:23:16 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id C0BFD180ACEE4 for ; Wed, 30 Oct 2019 14:23:15 +0000 (UTC) X-FDA: 76100668350.16.beam51_398b4fc5b5a35 X-Spam-Summary: 2,0,0,8e0b06f70b6492bb,d41d8cd98f00b204,30py5xqykcounspklynvvnsl.jvtspube-ttrchjr.vyn@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: beam51_398b4fc5b5a35 X-Filterd-Recvd-Size: 4640 Received: from mail-vs1-f74.google.com (mail-vs1-f74.google.com [209.85.217.74]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:15 +0000 (UTC) Received: by mail-vs1-f74.google.com with SMTP id z20so286335vsf.1 for ; Wed, 30 Oct 2019 07:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EGPzaL+niyswOznwvIuwrUyCjjQExKUEjV+KnYXXwaQ=; b=EVCxrwaV/qTVAvzAb/1nrTFltIfxcpwtmokBEcUO8dcStmv4nNRQ/QQ5XIYnLJvgHr HETCNvWv6divvbLP07WbqseTy+rRM0gz0HOpHyFGKykBsvtKsx7WhH5D8qHBqJr605M4 zZSOfd556o4FVy4wMe8+BK+Nw9sUIdfE6Mb82j6VILOY0Hycbdc/123Ton8ZEVvrbzaX 17RaepM1xxGYPJer7Ojx282mmw09A43IsdYPLqUuGqCiOCo/f7usgp1tT2TcFHsOtrc/ toib/+DQQtT2rh2bi5NEy3bI2bGbBYFXd6YDASTat4uENRA172hFLRUSvVnpzpSeAUvW 5Igw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EGPzaL+niyswOznwvIuwrUyCjjQExKUEjV+KnYXXwaQ=; b=hqInchzUDqoTXihsXDA0czldF93qu6biypIFRyNOdJ15vz74JUHoEdhU0u55JqWZpx AFcTVbEGpl8MCiX84wdFgXAJT/E6IXMTgGbgGwzBB1ZuSZnfvr6cJsseT1ZJBxpYYeaB V7579BqUzSW2AOgXo8n9ex8FJ3Lr/JF8yAShtuEF81tk0EnJuccG9Qpsnpqow0b6OmtV GhMPGPOp1eXO0xjpOiDI9EBF8PeoXzzoowsCjioiOKHVlINL+3IABlclY34ESQu2+KYT 15CXlHsTxvoCbAcjWgAMPLkavI1sbdkdTTeEeXgbdnnSH3PER1wreL9rOweO2bEuMLq3 GbjQ== X-Gm-Message-State: APjAAAUaranw1kutoTf/rcoH5loxoAZVfNw9MQ0WVKxYpo6gpcymtMkc PWy/BGvxiaKuzWrg27rPYUaTaxqXdUQ= X-Google-Smtp-Source: APXvYqwNLWD3F36CmMBJqVa4XZ6mG+R5g5iwr7WEW+oS0TijyHTuSxkWJ+7a/Bzxl79P2EdyTRHbc3LThsA= X-Received: by 2002:a67:dd97:: with SMTP id i23mr5176139vsk.79.1572445394336; Wed, 30 Oct 2019 07:23:14 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:22 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-11-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 10/25] kmsan: define READ_ONCE_NOCHECK() From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: READ_ONCE_NOCHECK() is already used by KASAN to ignore memory accesses from e.g. stack unwinders. Define READ_ONCE_NOCHECK() for KMSAN so that it returns initialized values. This helps defeat false positives from leftover stack contents. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: Ib38369ba038ab3b581d8e45b81036c3304fb79cb --- include/linux/compiler.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 5e88e7e33abe..e8c86debdb2b 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -270,9 +270,13 @@ static __always_inline void __write_once_size(volatile void *p, void *res, int s /* * Use READ_ONCE_NOCHECK() instead of READ_ONCE() if you need - * to hide memory access from KASAN. + * to hide memory access from KASAN or KMSAN. */ +#ifndef CONFIG_KMSAN #define READ_ONCE_NOCHECK(x) __READ_ONCE(x, 0) +#else +#define READ_ONCE_NOCHECK(x) KMSAN_INIT_VALUE(__READ_ONCE(x, 0)) +#endif static __no_kasan_or_inline unsigned long read_word_at_a_time(const void *addr) From patchwork Wed Oct 30 14:22:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219627 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F6DE112B for ; Wed, 30 Oct 2019 14:23:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5BF1720656 for ; Wed, 30 Oct 2019 14:23:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kDMiVf15" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5BF1720656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BAB726B026C; Wed, 30 Oct 2019 10:23:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B38266B026D; Wed, 30 Oct 2019 10:23:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 963276B026E; Wed, 30 Oct 2019 10:23:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0076.hostedemail.com [216.40.44.76]) by kanga.kvack.org (Postfix) with ESMTP id 6A0E56B026C for ; Wed, 30 Oct 2019 10:23:19 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 192306D9B for ; Wed, 30 Oct 2019 14:23:19 +0000 (UTC) X-FDA: 76100668518.07.fact52_3a050dafbb842 X-Spam-Summary: 2,0,0,a9d8c2354d4df402,d41d8cd98f00b204,31zy5xqykcogqvsnobqyyqvo.mywvsxeh-wwufkmu.ybq@flex--glider.bounces.google.com,:mingo@elte.hu:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1535:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: fact52_3a050dafbb842 X-Filterd-Recvd-Size: 5142 Received: from mail-vs1-f73.google.com (mail-vs1-f73.google.com [209.85.217.73]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:18 +0000 (UTC) Received: by mail-vs1-f73.google.com with SMTP id b3so287187vsh.0 for ; Wed, 30 Oct 2019 07:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=DxGu6whVYQ2z6BOovYk+cqzdGAM1gRpyILsCGDV4umw=; b=kDMiVf15bd1LThcGiH2zcYr7aozapZf2Bp0hagBRakdbX5s8GxktNyedFlakJdLfA1 gmV83OXEjOPVP9dG0Mayg7o7vWpvG19FfuCsh7//uGg9RF7esqYJabivqeX8fIETpIZa rTLNE3Qyal3at6XDvHKRDYU1hzz0nyU/QBtFgSqMenk2qsYWDXfxjv3Nvskm5Eh4N5Ul MVl83mTIYtTTf107HhPJMWurbd6dcMu9PqZBdHRrU0nBierJjUkt8U1b6VeiUo7KFlEL le6P79D35xn+2FIJA1eB8WYoENCPckNHXNCRbXgNhQ1Y7g0YyH0zBTKz1IK5oChiqrE5 AfpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DxGu6whVYQ2z6BOovYk+cqzdGAM1gRpyILsCGDV4umw=; b=LWn2FuutlzWkRec1EGJpIMC8tSrSy0Wu4yLn4d4d5rerQwrOzDrR5Mo+e+1YIq6kow dWY0Zi7i8ZL1ZNRPQlKctTWemJ6Yb4GIL7Nav3S6J74Z7BO0jDCbqfRMcYa8HJNqfZHT ke77wU8A4l0xHM2h2ecy0pCzMLh0aXXZ/J64CiI9forqK+ih6YEnTbxQuq/Nm9uvgSxC iWjY+R10kEpODJoaN+SmoNHCIpb6q3m/LMSp2rQxGWWqlPNpqC6fytGBxBaywvVZm/w/ v7kWyObxiqB4AIUsIsR0YCiMlPoC7ZK7rsPZpSPcVSfae+EXnLpezF5Fei213CWsoN61 YG0A== X-Gm-Message-State: APjAAAXjTaORqbdzl0oy1qwjH70/yEUFC9wQrYD5DF9gcgonygnRkiH0 HlCNgaMJML6a5MQ6TMZYlfBsKAIlzMk= X-Google-Smtp-Source: APXvYqxsSbb0WCtWoYis7OjMA5IHFW+O7xo5waLBGlFQD3ygjOHDew+ijgj8fsjzYgHAnCQ08j9QWupbKnQ= X-Received: by 2002:a1f:3a15:: with SMTP id h21mr14986807vka.17.1572445397665; Wed, 30 Oct 2019 07:23:17 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:23 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-12-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 11/25] kmsan: x86: sync metadata pages on page fault From: glider@google.com To: Ingo Molnar , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KMSAN assumes shadow and origin pages for every allocated page are accessible. For pages in vmalloc region those metadata pages reside in [VMALLOC_END, VMALLOC_META_END), therefore we must sync a bigger memory region. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Ingo Molnar Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I0d54855489870ef1180b37fe2120b601da464bf7 --- arch/x86/mm/fault.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 9ceacd1156db..d582337ba45d 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -320,7 +320,17 @@ static void dump_pagetable(unsigned long address) void vmalloc_sync_all(void) { +#ifdef CONFIG_KMSAN + /* + * For KMSAN, make sure metadata pages for vmalloc area and modules are + * also synced. + */ + sync_global_pgds(VMALLOC_START & PGDIR_MASK, VMALLOC_META_END); + sync_global_pgds(MODULES_SHADOW_START & PGDIR_MASK, + MODULES_ORIGIN_END); +#else sync_global_pgds(VMALLOC_START & PGDIR_MASK, VMALLOC_END); +#endif } /* @@ -337,7 +347,17 @@ static noinline int vmalloc_fault(unsigned long address) pte_t *pte; /* Make sure we are in vmalloc area: */ +#ifdef CONFIG_KMSAN + /* + * For KMSAN, make sure metadata pages for vmalloc area and modules are + * also synced. + */ + if (!(address >= VMALLOC_START && address < VMALLOC_META_END) && + !(address >= MODULES_SHADOW_START && + address < MODULES_ORIGIN_END)) +#else if (!(address >= VMALLOC_START && address < VMALLOC_END)) +#endif return -1; /* From patchwork Wed Oct 30 14:22:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219629 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAB25112B for ; Wed, 30 Oct 2019 14:23:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5C74C21734 for ; Wed, 30 Oct 2019 14:23:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OqrP5uZl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C74C21734 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 93FA86B026D; Wed, 30 Oct 2019 10:23:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 89E7E6B026E; Wed, 30 Oct 2019 10:23:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71A6C6B026F; Wed, 30 Oct 2019 10:23:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id 48B7D6B026D for ; Wed, 30 Oct 2019 10:23:23 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id DD45A181AEF31 for ; Wed, 30 Oct 2019 14:23:22 +0000 (UTC) X-FDA: 76100668644.02.uncle16_3a896ecf44c1c X-Spam-Summary: 2,0,0,215871f4de3ef88b,d41d8cd98f00b204,32jy5xqykcostyvqretbbtyr.pbzyvahk-zzxinpx.bet@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:1:2:41:69:152:355:379:541:800:960:966:973:988:989:1042:1260:1277:1313:1314:1345:1359:1437:1516:1518:1593:1594:1605:173 0:1747:1 X-HE-Tag: uncle16_3a896ecf44c1c X-Filterd-Recvd-Size: 10039 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:22 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id j17so1208013wru.13 for ; Wed, 30 Oct 2019 07:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=yhkanGx5vAdMfrYLCnIBr9tRyX0+t2wO7Ce/6R5gBR4=; b=OqrP5uZld+tlI/lLLqDRbf4maIu6jpR+Nvw3KZESTpQuf8Vv781G7MdO9v/D4Rfqxy Hsrc50e1iK3iAITv0BiUCzhsKNMR6E8DS6wQXpvYcvydrPYHpSDVwB2gnmdbVUlwjh4v YSydGDz+XVLgGZqRLlKL1S0X2iTEFet9Qy55RlfeiPb1YJMtresY/TDMDVesS6lOyJbR SSg+QuWxaxuprorL60rh9oRiNShptIr8Ui9Fxz5QZJRxOSHKV99Zh16DPWh6SbjbVGho XsGrNvjaIL6JcNoJDlCwdH4e59bM0QMu7Cr9iBRD6ymzAdqkYw9m6fA0h7zXkWP1KQsD s4yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=yhkanGx5vAdMfrYLCnIBr9tRyX0+t2wO7Ce/6R5gBR4=; b=RhFWvIHcoeyk9Dhkbt5OOSQRQbDsyMCLxFBxuuHv/u5cnqS514osB0oK+gaf25hlxr fYCnidOhpiFdjcH+fZd+6MHtinocEolBbd9F4xmdM0+Rv9+74h4EuYRT5dnSEu6wIo04 X/D3hgi5MYGalhD0G3+mIoarHRl1ovuC+znNpxkQI0ZtBhemmuzq9P0n7gLAL9he6DY+ RtkioEB4LoA9eqeNJ6iu8OFZY0yXqu/qSOhMSQhnr0Ya6FobM+eMDqBTufciLxjDxRkH TRyeVn9CujS9ysMWWb0BLmH8AFeLvQtaDb5rS/vBTFGZDc1JXV2Ov8R2mdiPMqTEMk0C hGGw== X-Gm-Message-State: APjAAAVXHZ3tiZOMS0urlSrsD1yJ6L0zfMfU7D8CUoIdfQ5i713ixmiS jcMlJ22Fuse2b4CZR33Hdtcff8QQZFo= X-Google-Smtp-Source: APXvYqwHZughLhxZA+LuafiFB2L3fLGaieXVwqYtr7hZ8/mE0teeTLGbjbissOJHc9gZnX8NfY6Kn9foJLs= X-Received: by 2002:adf:9044:: with SMTP id h62mr110687wrh.91.1572445400720; Wed, 30 Oct 2019 07:23:20 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:24 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-13-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 12/25] kmsan: add tests for KMSAN From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The initial commit adds several tests that trigger KMSAN warnings in simple cases. To use, build the kernel with CONFIG_TEST_KMSAN and do `insmod test_kmsan.ko` Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- v2: - added printk_test() Change-Id: I287e86ae83a82b770f2baa46e5bbdce1dfa65195 --- lib/Makefile | 1 + lib/test_kmsan.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 232 insertions(+) create mode 100644 lib/test_kmsan.c diff --git a/lib/Makefile b/lib/Makefile index c5892807e06f..cb44262c38ee 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -65,6 +65,7 @@ CFLAGS_test_kasan.o += $(call cc-disable-warning, vla) obj-$(CONFIG_TEST_UBSAN) += test_ubsan.o CFLAGS_test_ubsan.o += $(call cc-disable-warning, vla) UBSAN_SANITIZE_test_ubsan.o := y +obj-$(CONFIG_TEST_KMSAN) += test_kmsan.o obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o obj-$(CONFIG_TEST_LIST_SORT) += test_list_sort.o obj-$(CONFIG_TEST_LKM) += test_module.o diff --git a/lib/test_kmsan.c b/lib/test_kmsan.c new file mode 100644 index 000000000000..dcbe02adbdb0 --- /dev/null +++ b/lib/test_kmsan.c @@ -0,0 +1,231 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Module for testing KMSAN. + * + * Copyright (C) 2017-2019 Google LLC + * Author: Alexander Potapenko + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +/* + * Tests below use noinline and volatile to work around compiler optimizations + * that may mask KMSAN bugs. + */ +#define pr_fmt(fmt) "kmsan test: %s : " fmt, __func__ + +#include +#include +#include +#include +#include + +#define CHECK(x) \ + do { \ + if (x) \ + pr_info(#x " is true\n"); \ + else \ + pr_info(#x " is false\n"); \ + } while (0) + +noinline void use_integer(int cond) +{ + CHECK(cond); +} + +int signed_sum3(int a, int b, int c) +{ + return a + b + c; +} + +noinline void uninit_kmalloc_test(void) +{ + int *ptr; + + pr_info("-----------------------------\n"); + pr_info("uninitialized kmalloc test (UMR report)\n"); + ptr = kmalloc(sizeof(int), GFP_KERNEL); + pr_info("kmalloc returned %p\n", ptr); + CHECK(*ptr); +} +noinline void init_kmalloc_test(void) +{ + int *ptr; + + pr_info("-----------------------------\n"); + pr_info("initialized kmalloc test (no reports)\n"); + ptr = kmalloc(sizeof(int), GFP_KERNEL); + memset(ptr, 0, sizeof(int)); + pr_info("kmalloc returned %p\n", ptr); + CHECK(*ptr); +} + +noinline void init_kzalloc_test(void) +{ + int *ptr; + + pr_info("-----------------------------\n"); + pr_info("initialized kzalloc test (no reports)\n"); + ptr = kzalloc(sizeof(int), GFP_KERNEL); + pr_info("kzalloc returned %p\n", ptr); + CHECK(*ptr); +} + +noinline void uninit_multiple_args_test(void) +{ + volatile int a; + volatile char b = 3, c; + + pr_info("-----------------------------\n"); + pr_info("uninitialized local passed to fn (UMR report)\n"); + CHECK(signed_sum3(a, b, c)); +} + +noinline void uninit_stack_var_test(void) +{ + int cond; + + pr_info("-----------------------------\n"); + pr_info("uninitialized stack variable (UMR report)\n"); + CHECK(cond); +} + +noinline void init_stack_var_test(void) +{ + volatile int cond = 1; + + pr_info("-----------------------------\n"); + pr_info("initialized stack variable (no reports)\n"); + CHECK(cond); +} + +noinline void two_param_fn_2(int arg1, int arg2) +{ + CHECK(arg1); + CHECK(arg2); +} + +noinline void one_param_fn(int arg) +{ + two_param_fn_2(arg, arg); + CHECK(arg); +} + +noinline void two_param_fn(int arg1, int arg2) +{ + int init = 0; + + one_param_fn(init); + CHECK(arg1); + CHECK(arg2); +} + +void params_test(void) +{ + int uninit, init = 1; + + two_param_fn(uninit, init); +} + +noinline void do_uninit_local_array(char *array, int start, int stop) +{ + int i; + volatile char uninit; + + for (i = start; i < stop; i++) + array[i] = uninit; +} + +noinline void uninit_kmsan_check_memory_test(void) +{ + volatile char local_array[8]; + + pr_info("-----------------------------\n"); + pr_info("kmsan_check_memory() called on uninit local (UMR report)\n"); + do_uninit_local_array((char *)local_array, 5, 7); + + kmsan_check_memory((char *)local_array, 8); +} + +noinline void init_kmsan_vmap_vunmap_test(void) +{ + const int npages = 2; + struct page *pages[npages]; + void *vbuf; + int i; + + pr_info("-----------------------------\n"); + pr_info("pages initialized via vmap (no reports)\n"); + + for (i = 0; i < npages; i++) + pages[i] = alloc_page(GFP_KERNEL); + vbuf = vmap(pages, npages, VM_MAP, PAGE_KERNEL); + memset(vbuf, 0xfe, npages * PAGE_SIZE); + for (i = 0; i < npages; i++) + kmsan_check_memory(page_address(pages[i]), PAGE_SIZE); + + if (vbuf) + vunmap(vbuf); + for (i = 0; i < npages; i++) + if (pages[i]) + __free_page(pages[i]); +} + +noinline void init_vmalloc(void) +{ + char *buf; + int npages = 8, i; + + pr_info("-----------------------------\n"); + pr_info("pages initialized via vmap (no reports)\n"); + buf = vmalloc(PAGE_SIZE * npages); + buf[0] = 1; + memset(buf, 0xfe, PAGE_SIZE * npages); + CHECK(buf[0]); + for (i = 0; i < npages; i++) + kmsan_check_memory(&buf[PAGE_SIZE * i], PAGE_SIZE); + vfree(buf); +} + +noinline void uaf_test(void) +{ + volatile int *var; + + pr_info("-----------------------------\n"); + pr_info("use-after-free in kmalloc-ed buffer (UMR report)\n"); + var = kmalloc(80, GFP_KERNEL); + var[3] = 0xfeedface; + kfree((int *)var); + CHECK(var[3]); +} + +noinline void printk_test(void) +{ + volatile int uninit; + + pr_info("-----------------------------\n"); + pr_info("uninit local passed to pr_info() (UMR report)\n"); + pr_info("%px contains %d\n", &uninit, uninit); +} + +static noinline int __init kmsan_tests_init(void) +{ + uninit_kmalloc_test(); + init_kmalloc_test(); + init_kzalloc_test(); + uninit_multiple_args_test(); + uninit_stack_var_test(); + init_stack_var_test(); + uninit_kmsan_check_memory_test(); + init_kmsan_vmap_vunmap_test(); + init_vmalloc(); + uaf_test(); + printk_test(); + return -EAGAIN; +} + +module_init(kmsan_tests_init); +MODULE_LICENSE("GPL"); From patchwork Wed Oct 30 14:22:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219631 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35542112B for ; Wed, 30 Oct 2019 14:23:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0171B2087E for ; Wed, 30 Oct 2019 14:23:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="J9iWz8o1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0171B2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 51FF66B026E; Wed, 30 Oct 2019 10:23:27 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4A61E6B026F; Wed, 30 Oct 2019 10:23:27 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3710C6B0270; Wed, 30 Oct 2019 10:23:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 121D16B026E for ; Wed, 30 Oct 2019 10:23:27 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id A8F21180AD822 for ; Wed, 30 Oct 2019 14:23:26 +0000 (UTC) X-FDA: 76100668812.04.spark76_3b05309ae8319 X-Spam-Summary: 2,0,0,85d0922c67f83d17,d41d8cd98f00b204,325y5xqykco4wbytuhweewbu.secbydkn-ccalqsa.ehw@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1535:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: spark76_3b05309ae8319 X-Filterd-Recvd-Size: 5041 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:25 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id z5so679785wma.5 for ; Wed, 30 Oct 2019 07:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=r1rimWPSVZnlscUh2KjkHO3RzEDqzDMgK4H1V1YaUMg=; b=J9iWz8o1jeNdl2+zd6zF5q8TbNep85mHubqBUtf3WoIbxeqFJmXUKTTL3oEwTXe2y9 8GajzIDy0wCtHuDus67jkdgDgpq4wWPVeBcdZYOPEjvOZshejxE5kzeoNKrILQmR5aX8 IaRmrpJ6gz3rKG2Ukbc5kEdnOTNU2YQz24Blr5UclSmLu33xFHLxbP0uvuXqzpyvTXwo aTnCBd5gvdT8hSpB73FySxQcEoCz5tdjD/Yn/u+ukv2ULYbIsW0KXNNCUgVMU4xRPAAP 9plJ70P+caOZLFVXK/dVIIQ5JtxC1eWOQrozoEWjWeGHtNT73srdEQ46qGNq8PpFb2c3 +FKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=r1rimWPSVZnlscUh2KjkHO3RzEDqzDMgK4H1V1YaUMg=; b=eCwyaOyd2IlD4ttCD5JOBhILMkRekBKO26ojuvAiO0n8hmCv8bD82R1o04cvGm/abx cskt8xgsp3nDg/UCmAzVJByHbDEsg/L/BeBcEDfICipzUoqo4qxDxd4SHxk4eMW73we4 EPXNzu0dYitK3i6WupmE3eLaWnSS9ug2oN0GET4BHQzE1QWmGF23LqzDeomawmKKZtJq IlZ2mYL2VLbdZAttxnEl9hBNNBbGK3d75/gToqltjA2p08UnHG6o8nWUNSxi/xR0HmKY fhVKbKqX/eAT2/sAjC8dNj8pJIBVoNy98eseWAY3wuWc6bGq5NlxoLZFcDbLjrGbXg8i lLkA== X-Gm-Message-State: APjAAAUJR+2lW6GSSa8VDuNIpXFbWag4FKDXxKgts8h2jI5YJfME9pBF G3JxsxVXUgEBKz/RpAyBCjJfhk3EadY= X-Google-Smtp-Source: APXvYqw4oTuQi8o6uJ4O3XSt/8G4qv52mS+ZInpBFLCgSkxJaM92QnVojPia+CtT2ixPtB0UafE27JuNbAk= X-Received: by 2002:adf:dd10:: with SMTP id a16mr103618wrm.213.1572445403925; Wed, 30 Oct 2019 07:23:23 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:25 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-14-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 13/25] kmsan: make READ_ONCE_TASK_STACK() return initialized values From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To avoid false positives, assume that reading from the task stack always produces initialized values. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: Ie73e5a41fdc8195699928e65f5cbe0d3d3c9e2fa --- arch/x86/include/asm/unwind.h | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/unwind.h b/arch/x86/include/asm/unwind.h index 499578f7e6d7..f60c2bd1ddf2 100644 --- a/arch/x86/include/asm/unwind.h +++ b/arch/x86/include/asm/unwind.h @@ -100,9 +100,10 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size, #endif /* - * This disables KASAN checking when reading a value from another task's stack, - * since the other task could be running on another CPU and could have poisoned - * the stack in the meantime. + * This disables KASAN/KMSAN checking when reading a value from another task's + * stack, since the other task could be running on another CPU and could have + * poisoned the stack in the meantime. Frame pointers are uninitialized by + * default, so for KMSAN we mark the return value initialized unconditionally. */ #define READ_ONCE_TASK_STACK(task, x) \ ({ \ @@ -111,7 +112,7 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size, val = READ_ONCE(x); \ else \ val = READ_ONCE_NOCHECK(x); \ - val; \ + KMSAN_INIT_VALUE(val); \ }) static inline bool task_on_another_cpu(struct task_struct *task) From patchwork Wed Oct 30 14:22:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219633 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6FB3112B for ; Wed, 30 Oct 2019 14:23:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 677EC20656 for ; Wed, 30 Oct 2019 14:23:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bhjh6Wp+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 677EC20656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 03BF96B026F; Wed, 30 Oct 2019 10:23:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F2AB36B0270; Wed, 30 Oct 2019 10:23:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCF016B0271; Wed, 30 Oct 2019 10:23:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id AE5E56B026F for ; Wed, 30 Oct 2019 10:23:29 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 5EF2F5DD6 for ; Wed, 30 Oct 2019 14:23:29 +0000 (UTC) X-FDA: 76100668938.22.space68_3b77ab4aa1f3c X-Spam-Summary: 2,0,0,e34431b36dccb390,d41d8cd98f00b204,335y5xqykcpiafcxylaiiafy.wigfchor-ggepuwe.ila@flex--glider.bounces.google.com,:herbert@gondor.apana.org.au:harry.wentland@amd.com:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:1:41:152:355:379:541:800:960:967:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1593:1594:1605:1730:1747:177 7:1792:1 X-HE-Tag: space68_3b77ab4aa1f3c X-Filterd-Recvd-Size: 13673 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:28 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id 6so929764wmj.9 for ; Wed, 30 Oct 2019 07:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=2lI8fAPEidWHUX0XryAFdz3FKCOshz4KWP6nGISgPPY=; b=bhjh6Wp+jCeKDuLGFcLCeSZr1KAgdmjJZwznQUDKH1l3SPQrfF9HiYNaFG9MfWD1xz vb0oR0ZVo2vvXMrV8wHGbyfCetv9Hy6YBTh1h2ExfQ53mEECd53jTsmwqRc0DH/uPLDC gTPn3KYTXFGVWaocOHgwBr3BiK7m9xiWh/btp+ms5sFmlKpM4ElbC/qEfItvpltNz6pM qK5jKsU6/w3fyWwul0E8UWeM9HOKQA3F6y0itRufqh/h523qtLtkL01h/NmyXFlyholF bhkoA9qjNEBggLyFDMUMpA1fDyDy57nWacV19cCcyxNDcoU7TtQSSzDEddi6TEXHzCfO hDlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2lI8fAPEidWHUX0XryAFdz3FKCOshz4KWP6nGISgPPY=; b=LavdYiSaGGrAoMEvR8hHXoa+RAy06J3dmfVd89sBTkhkEVnG8QiuhsGKdeZdjQ74cx m2lcZGxpDai1s3ueWvXXznYIBhWLv5qtPe/gW6M02XSXUcCR2BaH/KQR6D+d67RlrsXW pu2YrXIvVMyftunEIOkfaMQnbJILWHEMOeZH2A6PNkBZ+RUbwUjHXLYvUt6U+S6l9FwC yBcVCqfqVAT/XODLjvODPWQiHCXncf8DYVUIcQZFI1LToPcT8bxzK/WewP4KISEpexRw 0RZc2moP3WLlJK+dv8SBmzU3Fyftp8TTQR0F09Aoa1K7CaYJscxmJX/bwQB3bDWCGvkX WWFQ== X-Gm-Message-State: APjAAAXLGqTeQOsD47ewRNvPogWDj3sK+iB1HLOeW03Q39laTVgzu/4Q zyJzZdC6rcEOw6m0XG9RppNidJwmowk= X-Google-Smtp-Source: APXvYqxlxyBPhg+J42yPUH6nJ3bZz14Tdr+mGoyFGQNkPmnNrMXY8EfF4yDq9xIN0xWD/XSwzdPVXUVLj/U= X-Received: by 2002:a05:6000:101:: with SMTP id o1mr117809wrx.394.1572445407056; Wed, 30 Oct 2019 07:23:27 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:26 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-15-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 14/25] kmsan: Kconfig changes to disable options incompatible with KMSAN From: glider@google.com To: Herbert Xu , Harry Wentland , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Certain config options are currently incompatible with KMSAN, disable them. Signed-off-by: Alexander Potapenko Acked-by: Harry Wentland To: Alexander Potapenko Cc: Herbert Xu Cc: Harry Wentland Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I4521f5c764cfaeba2fae79c954925c302c5e3bc0 --- arch/Kconfig | 2 +- arch/x86/Kconfig | 1 + arch/x86/Kconfig.debug | 3 +++ crypto/Kconfig | 26 ++++++++++++++++++++++++++ drivers/gpu/drm/amd/display/Kconfig | 2 +- lib/Kconfig.debug | 3 +++ 6 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 5f8a5d84dbbe..d5ca838ae8ad 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -843,7 +843,7 @@ config HAVE_ARCH_VMAP_STACK config VMAP_STACK default y bool "Use a virtually-mapped stack" - depends on HAVE_ARCH_VMAP_STACK && !KASAN + depends on HAVE_ARCH_VMAP_STACK && !KASAN && !KMSAN ---help--- Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 3f83a5c53808..6d2b0fb3a871 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2143,6 +2143,7 @@ config RELOCATABLE config RANDOMIZE_BASE bool "Randomize the address of the kernel image (KASLR)" depends on RELOCATABLE + depends on !KMSAN default y ---help--- In support of Kernel Address Space Layout Randomization (KASLR), diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug index bf9cd83de777..db3cd6147829 100644 --- a/arch/x86/Kconfig.debug +++ b/arch/x86/Kconfig.debug @@ -292,6 +292,9 @@ choice config UNWINDER_ORC bool "ORC unwinder" depends on X86_64 + # KMSAN doesn't support UNWINDER_ORC yet, + # see https://github.com/google/kmsan/issues/48. + depends on !KMSAN select STACK_VALIDATION ---help--- This option enables the ORC (Oops Rewind Capability) unwinder for diff --git a/crypto/Kconfig b/crypto/Kconfig index 9e524044d312..37a22711012c 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -309,11 +309,13 @@ config CRYPTO_AEGIS128 config CRYPTO_AEGIS128_SIMD bool "Support SIMD acceleration for AEGIS-128" depends on CRYPTO_AEGIS128 && ((ARM || ARM64) && KERNEL_MODE_NEON) + depends on !KMSAN default y config CRYPTO_AEGIS128_AESNI_SSE2 tristate "AEGIS-128 AEAD algorithm (x86_64 AESNI+SSE2 implementation)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_AEAD select CRYPTO_SIMD help @@ -571,6 +573,7 @@ config CRYPTO_CRC32C config CRYPTO_CRC32C_INTEL tristate "CRC32c INTEL hardware acceleration" depends on X86 + depends on !KMSAN select CRYPTO_HASH help In Intel processor with SSE4.2 supported, the processor will @@ -611,6 +614,7 @@ config CRYPTO_CRC32 config CRYPTO_CRC32_PCLMUL tristate "CRC32 PCLMULQDQ hardware acceleration" depends on X86 + depends on !KMSAN select CRYPTO_HASH select CRC32 help @@ -649,6 +653,7 @@ config CRYPTO_CRCT10DIF config CRYPTO_CRCT10DIF_PCLMUL tristate "CRCT10DIF PCLMULQDQ hardware acceleration" depends on X86 && 64BIT && CRC_T10DIF + depends on !KMSAN select CRYPTO_HASH help For x86_64 processors with SSE4.2 and PCLMULQDQ supported, @@ -695,6 +700,7 @@ config CRYPTO_POLY1305 config CRYPTO_POLY1305_X86_64 tristate "Poly1305 authenticator algorithm (x86_64/SSE2/AVX2)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_POLY1305 help Poly1305 authenticator algorithm, RFC7539. @@ -814,6 +820,7 @@ config CRYPTO_SHA1 config CRYPTO_SHA1_SSSE3 tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_SHA1 select CRYPTO_HASH help @@ -825,6 +832,7 @@ config CRYPTO_SHA1_SSSE3 config CRYPTO_SHA256_SSSE3 tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2/SHA-NI)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_SHA256 select CRYPTO_HASH help @@ -837,6 +845,7 @@ config CRYPTO_SHA256_SSSE3 config CRYPTO_SHA512_SSSE3 tristate "SHA512 digest algorithm (SSSE3/AVX/AVX2)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_SHA512 select CRYPTO_HASH help @@ -1011,6 +1020,7 @@ config CRYPTO_WP512 config CRYPTO_GHASH_CLMUL_NI_INTEL tristate "GHASH hash function (CLMUL-NI accelerated)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_CRYPTD help This is the x86_64 CLMUL-NI accelerated implementation of @@ -1064,6 +1074,7 @@ config CRYPTO_AES_TI config CRYPTO_AES_NI_INTEL tristate "AES cipher algorithms (AES-NI)" depends on X86 + depends on !KMSAN select CRYPTO_AEAD select CRYPTO_LIB_AES select CRYPTO_ALGAPI @@ -1190,6 +1201,7 @@ config CRYPTO_BLOWFISH_COMMON config CRYPTO_BLOWFISH_X86_64 tristate "Blowfish cipher algorithm (x86_64)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_BLOWFISH_COMMON help @@ -1221,6 +1233,7 @@ config CRYPTO_CAMELLIA_X86_64 tristate "Camellia cipher algorithm (x86_64)" depends on X86 && 64BIT depends on CRYPTO + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 help @@ -1238,6 +1251,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX)" depends on X86 && 64BIT depends on CRYPTO + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_CAMELLIA_X86_64 select CRYPTO_GLUE_HELPER_X86 @@ -1258,6 +1272,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" depends on X86 && 64BIT depends on CRYPTO + depends on !KMSAN select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help Camellia cipher algorithm module (x86_64/AES-NI/AVX2). @@ -1303,6 +1318,7 @@ config CRYPTO_CAST5 config CRYPTO_CAST5_AVX_X86_64 tristate "CAST5 (CAST-128) cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_CAST5 select CRYPTO_CAST_COMMON @@ -1325,6 +1341,7 @@ config CRYPTO_CAST6 config CRYPTO_CAST6_AVX_X86_64 tristate "CAST6 (CAST-256) cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_CAST6 select CRYPTO_CAST_COMMON @@ -1360,6 +1377,7 @@ config CRYPTO_DES_SPARC64 config CRYPTO_DES3_EDE_X86_64 tristate "Triple DES EDE cipher algorithm (x86-64)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_LIB_DES help @@ -1426,6 +1444,7 @@ config CRYPTO_CHACHA20 config CRYPTO_CHACHA20_X86_64 tristate "ChaCha stream cipher algorithms (x86_64/SSSE3/AVX2/AVX-512VL)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_CHACHA20 help @@ -1462,6 +1481,7 @@ config CRYPTO_SERPENT config CRYPTO_SERPENT_SSE2_X86_64 tristate "Serpent cipher algorithm (x86_64/SSE2)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 select CRYPTO_SERPENT @@ -1481,6 +1501,7 @@ config CRYPTO_SERPENT_SSE2_X86_64 config CRYPTO_SERPENT_SSE2_586 tristate "Serpent cipher algorithm (i586/SSE2)" depends on X86 && !64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 select CRYPTO_SERPENT @@ -1500,6 +1521,7 @@ config CRYPTO_SERPENT_SSE2_586 config CRYPTO_SERPENT_AVX_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 select CRYPTO_SERPENT @@ -1520,6 +1542,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen. @@ -1615,6 +1638,7 @@ config CRYPTO_TWOFISH_586 config CRYPTO_TWOFISH_X86_64 tristate "Twofish cipher algorithm (x86_64)" depends on (X86 || UML_X86) && 64BIT + depends on !KMSAN select CRYPTO_ALGAPI select CRYPTO_TWOFISH_COMMON help @@ -1631,6 +1655,7 @@ config CRYPTO_TWOFISH_X86_64 config CRYPTO_TWOFISH_X86_64_3WAY tristate "Twofish cipher algorithm (x86_64, 3-way parallel)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_TWOFISH_COMMON select CRYPTO_TWOFISH_X86_64 @@ -1652,6 +1677,7 @@ config CRYPTO_TWOFISH_X86_64_3WAY config CRYPTO_TWOFISH_AVX_X86_64 tristate "Twofish cipher algorithm (x86_64/AVX)" depends on X86 && 64BIT + depends on !KMSAN select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 select CRYPTO_SIMD diff --git a/drivers/gpu/drm/amd/display/Kconfig b/drivers/gpu/drm/amd/display/Kconfig index 71991a28a775..99585eaf4dd0 100644 --- a/drivers/gpu/drm/amd/display/Kconfig +++ b/drivers/gpu/drm/amd/display/Kconfig @@ -6,7 +6,7 @@ config DRM_AMD_DC bool "AMD DC - Enable new display engine" default y select SND_HDA_COMPONENT if SND_HDA_CORE - select DRM_AMD_DC_DCN1_0 if X86 && !(KCOV_INSTRUMENT_ALL && KCOV_ENABLE_COMPARISONS) + select DRM_AMD_DC_DCN1_0 if X86 && !(KCOV_INSTRUMENT_ALL && KCOV_ENABLE_COMPARISONS) && !KMSAN help Choose this option if you want to use the new display engine support for AMDGPU. This adds required support for Vega and diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 75c36318943d..a3f6f5d68593 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1068,6 +1068,9 @@ menu "Lock Debugging (spinlocks, mutexes, etc...)" config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT + # KMSAN is incompatible with lockdep, + # see https://github.com/google/kmsan/issues/57. + depends on !KMSAN default y config PROVE_LOCKING From patchwork Wed Oct 30 14:22:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219635 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DDAC1709 for ; Wed, 30 Oct 2019 14:23:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2FCA520656 for ; Wed, 30 Oct 2019 14:23:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FMHtD4So" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2FCA520656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 47F806B0270; Wed, 30 Oct 2019 10:23:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4097C6B0271; Wed, 30 Oct 2019 10:23:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20B836B0272; Wed, 30 Oct 2019 10:23:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id E78856B0270 for ; Wed, 30 Oct 2019 10:23:32 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 95906181AEF31 for ; Wed, 30 Oct 2019 14:23:32 +0000 (UTC) X-FDA: 76100669064.28.house14_3bed5790e235d X-Spam-Summary: 10,1,0,dca7397e1f1873ec,d41d8cd98f00b204,34py5xqykcpudifabodlldib.zljifkru-jjhsxzh.lod@flex--glider.bounces.google.com,:axboe@kernel.dk:luto@kernel.org:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:1:41:152:334:355:379:541:617:800:960:973:982:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605 :1690:17 X-HE-Tag: house14_3bed5790e235d X-Filterd-Recvd-Size: 14491 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:31 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id j14so1409499wrm.6 for ; Wed, 30 Oct 2019 07:23:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Qn9uGyCi8LQXEYobYHMlSmj0M7rPOYDf1BFHM1dbAQA=; b=FMHtD4Sotg86acgtSnsa2e/TOGt7pMsxNIvAdUfCFCiSIw3grhYPVmcBmYcCxFfRev 4sgygEcWAj4ifhsFpYihoECyEQ0kS2LUEl+7cEv2fecVajApB38dOTZXo+5zaEr+SNvR I7ijOzlauNwYDuL07bSSpeodNzACQeO3MUhGNhYMwT+Dt+MYAKsxMIw7iymWcLrqbVgc 4W4Jqmc2Rhkr18kOTPLme4bng7Pm/fPBRDkzbBLd9Hr9ymFxFMPTLFvdD6hJDLWUS26x K3wIc/m7vh6dvxQ7C6pIP9jUKuqrYPbgYLfouU+YACpHC05a6sXjstvkXRDpZA9mNQAU 884A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Qn9uGyCi8LQXEYobYHMlSmj0M7rPOYDf1BFHM1dbAQA=; b=d7kCuLg1GQWrMuvCfTRP0QzcGsL4fpgStugxPbT98lAPGSGDfnkl2uj5teMhFa8tOF a92C/aEz3ZaRLwRDecbA5P5XcPOXklN93F9hJzQIq/4dnhR5HQ6IfcOr1+9W9qTbXktK UPOxKYS0GunaLfY7MdwKfU+x1nlkOLaVt/sAc7IUA+1jY1LdqzCeMCKr4Unp0BlRyilu XIJp9IBHaBCPy1LloKygGU8XmmNwdfOF5oDXWkmUtgU9Zv4lRaAaPtotawRIncIIfxKr xAvMTAk1f97htrrf04Zt1g9887+GULF6GufEcr7c8PS5jFBC0OdWacXxaHrT/0d6vPNB WERA== X-Gm-Message-State: APjAAAX1zAFiElr1ngXMRjQsR2ARBb9XUdCpR49q7hG6v/IFN9l0cxfs /ehzSkP7rNnHo1o/LC5U5MR8wlcFx5I= X-Google-Smtp-Source: APXvYqy07Mg2nnumIPWlIyodUuh3ebh+5mTPYmdCFAAYOWHiBBn4e0Nek1ivfL1bBAcKSU24EYrshIKN6lU= X-Received: by 2002:a5d:50c2:: with SMTP id f2mr105308wrt.147.1572445410331; Wed, 30 Oct 2019 07:23:30 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:27 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-16-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 15/25] kmsan: Changing existing files to enable KMSAN builds From: glider@google.com To: Jens Axboe , Andy Lutomirski , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do the following to make KMSAN usable: - add KMSAN declarations to struct page and struct task_struct; - add Makefile.kmsan to top-level Makefile; - call KMSAN initialization from init/main.c - add asm hooks to arch/x86/entry/entry_64.S; - increase task stack size under KMSAN; - disable page merging in block/blk.h; - disable CONFIG_KMSAN in arch/x86/boot/compressed/misc.h to use default string functions instead of KMSAN ones. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Jens Axboe Cc: Andy Lutomirski Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I37e0b7f2d2f2b0aeac5753ff9d6b411485fc374e --- Makefile | 3 ++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/entry/entry_64.S | 18 ++++++++++++++++++ arch/x86/include/asm/page_64_types.h | 12 +++++++----- block/blk.h | 7 +++++++ include/linux/mm_types.h | 9 +++++++++ include/linux/sched.h | 5 +++++ init/main.c | 3 +++ lib/stackdepot.c | 7 ++++--- mm/Makefile | 1 + scripts/Makefile.lib | 6 ++++++ 11 files changed, 63 insertions(+), 9 deletions(-) diff --git a/Makefile b/Makefile index 79be70bf2899..181ae2dac415 100644 --- a/Makefile +++ b/Makefile @@ -478,7 +478,7 @@ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE -export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN +export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KMSAN export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL @@ -898,6 +898,7 @@ KBUILD_CFLAGS += $(call cc-option,-fcf-protection=none) endif include scripts/Makefile.kasan +include scripts/Makefile.kmsan include scripts/Makefile.extrawarn include scripts/Makefile.ubsan diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h index c8181392f70d..dd4bd8c5d97a 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -12,6 +12,7 @@ #undef CONFIG_PARAVIRT_XXL #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN +#undef CONFIG_KMSAN /* cpu_feature_enabled() cannot be used this early */ #define USE_EARLY_PGTABLE_L5 diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index b7c3ea4cb19d..1218672bad85 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -168,12 +169,14 @@ GLOBAL(entry_SYSCALL_64_after_hwframe) PUSH_AND_CLEAR_REGS rax=$-ENOSYS TRACE_IRQS_OFF + KMSAN_SYSCALL_ENTER /* IRQs are off. */ movq %rax, %rdi movq %rsp, %rsi call do_syscall_64 /* returns with IRQs disabled */ + KMSAN_SYSCALL_EXIT TRACE_IRQS_IRETQ /* we're about to change IF */ /* @@ -575,6 +578,7 @@ ENTRY(interrupt_entry) 1: ENTER_IRQ_STACK old_rsp=%rdi save_ret=1 + KMSAN_INTERRUPT_ENTER /* We entered an interrupt context - irqs are off: */ TRACE_IRQS_OFF @@ -604,12 +608,14 @@ common_interrupt: addq $-0x80, (%rsp) /* Adjust vector to [-256, -1] range */ call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call do_IRQ /* rdi points to pt_regs */ /* 0(%rsp): old RSP */ ret_from_intr: DISABLE_INTERRUPTS(CLBR_ANY) TRACE_IRQS_OFF + KMSAN_INTERRUPT_EXIT LEAVE_IRQ_STACK testb $3, CS(%rsp) @@ -802,6 +808,7 @@ ENTRY(\sym) .Lcommon_\sym: call interrupt_entry UNWIND_HINT_REGS indirect=1 + KMSAN_UNPOISON_PT_REGS call \do_sym /* rdi points to pt_regs */ jmp ret_from_intr END(\sym) @@ -909,15 +916,18 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt .if \shift_ist != -1 subq $\ist_offset, CPU_TSS_IST(\shift_ist) + KMSAN_IST_ENTER(\shift_ist) .endif .if \read_cr2 movq %r12, %rdx /* Move CR2 into 3rd argument */ .endif + KMSAN_UNPOISON_PT_REGS call \do_sym .if \shift_ist != -1 + KMSAN_IST_EXIT(\shift_ist) addq $\ist_offset, CPU_TSS_IST(\shift_ist) .endif @@ -1079,7 +1089,9 @@ ENTRY(do_softirq_own_stack) pushq %rbp mov %rsp, %rbp ENTER_IRQ_STACK regs=0 old_rsp=%r11 + KMSAN_SOFTIRQ_ENTER call __do_softirq + KMSAN_SOFTIRQ_EXIT LEAVE_IRQ_STACK regs=0 leaveq ret @@ -1466,9 +1478,12 @@ ENTRY(nmi) * done with the NMI stack. */ + KMSAN_NMI_ENTER movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* * Return back to user mode. We must *not* do the normal exit @@ -1678,10 +1693,13 @@ end_repeat_nmi: call paranoid_entry UNWIND_HINT_REGS + KMSAN_NMI_ENTER /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */ movq %rsp, %rdi movq $-1, %rsi + KMSAN_UNPOISON_PT_REGS call do_nmi + KMSAN_NMI_EXIT /* Always restore stashed CR3 value (see paranoid_entry) */ RESTORE_CR3 scratch_reg=%r15 save_reg=%r14 diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h index 288b065955b7..ea9fbf09f43b 100644 --- a/arch/x86/include/asm/page_64_types.h +++ b/arch/x86/include/asm/page_64_types.h @@ -7,18 +7,20 @@ #endif #ifdef CONFIG_KASAN -#define KASAN_STACK_ORDER 1 +#define EXTRA_STACK_ORDER 1 +#elif defined(CONFIG_KMSAN) +#define EXTRA_STACK_ORDER 2 #else -#define KASAN_STACK_ORDER 0 +#define EXTRA_STACK_ORDER 0 #endif -#define THREAD_SIZE_ORDER (2 + KASAN_STACK_ORDER) +#define THREAD_SIZE_ORDER (2 + EXTRA_STACK_ORDER) #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) -#define EXCEPTION_STACK_ORDER (0 + KASAN_STACK_ORDER) +#define EXCEPTION_STACK_ORDER (0 + EXTRA_STACK_ORDER) #define EXCEPTION_STKSZ (PAGE_SIZE << EXCEPTION_STACK_ORDER) -#define IRQ_STACK_ORDER (2 + KASAN_STACK_ORDER) +#define IRQ_STACK_ORDER (2 + EXTRA_STACK_ORDER) #define IRQ_STACK_SIZE (PAGE_SIZE << IRQ_STACK_ORDER) /* diff --git a/block/blk.h b/block/blk.h index 47fba9362e60..9ee271a22423 100644 --- a/block/blk.h +++ b/block/blk.h @@ -78,6 +78,13 @@ static inline bool biovec_phys_mergeable(struct request_queue *q, phys_addr_t addr1 = page_to_phys(vec1->bv_page) + vec1->bv_offset; phys_addr_t addr2 = page_to_phys(vec2->bv_page) + vec2->bv_offset; +#ifdef CONFIG_KMSAN + /* + * Merging consequent physical pages may not work correctly under KMSAN + * if their metadata pages aren't consequent. Just disable merging. + */ + return false; +#endif if (addr1 + vec1->bv_len != addr2) return false; if (xen_domain() && !xen_biovec_phys_mergeable(vec1, vec2->bv_page)) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 2222fa795284..c87c5416a802 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -216,6 +216,15 @@ struct page { not kmapped, ie. highmem) */ #endif /* WANT_PAGE_VIRTUAL */ +#ifdef CONFIG_KMSAN + /* + * Bits in struct page are scarce, so the LSB in *shadow is used to + * indicate whether the page should be ignored by KMSAN or not. + */ + struct page *shadow; + struct page *origin; +#endif + #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS int _last_cpupid; #endif diff --git a/include/linux/sched.h b/include/linux/sched.h index 72b20f33c56e..ba705f66f78c 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -1173,6 +1174,10 @@ struct task_struct { unsigned int kasan_depth; #endif +#ifdef CONFIG_KMSAN + struct kmsan_task_state kmsan; +#endif + #ifdef CONFIG_FUNCTION_GRAPH_TRACER /* Index of current stored address in ret_stack: */ int curr_ret_stack; diff --git a/init/main.c b/init/main.c index 91f6ebb30ef0..afcca2a38348 100644 --- a/init/main.c +++ b/init/main.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include #include @@ -554,6 +555,7 @@ static void __init mm_init(void) */ page_ext_init_flatmem(); report_meminit(); + kmsan_initialize_shadow(); mem_init(); kmem_cache_init(); kmemleak_init(); @@ -625,6 +627,7 @@ asmlinkage __visible void __init start_kernel(void) sort_main_extable(); trap_init(); mm_init(); + kmsan_initialize(); ftrace_init(); diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 460efd3ef742..529a86eb1dd1 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -233,8 +233,8 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (parts.slabindex > depot_index) { WARN(1, "slab index %d out of bounds (%d) for stack id %08x\n", parts.slabindex, depot_index, handle); - __memset(trace, 0, sizeof(*trace)); - return; + *entries = NULL; + return 0; } slab = stack_slabs[parts.slabindex]; stack = slab + offset; @@ -299,7 +299,8 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, * contexts and I/O. */ alloc_flags &= ~GFP_ZONEMASK; - alloc_flags &= (GFP_ATOMIC | GFP_KERNEL); + alloc_flags &= (GFP_ATOMIC | GFP_KERNEL | + __GFP_NO_KMSAN_SHADOW); alloc_flags |= __GFP_NOWARN; page = alloc_pages(alloc_flags, STACK_ALLOC_ORDER); if (page) diff --git a/mm/Makefile b/mm/Makefile index d996846697ef..419e6e02dfaf 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -71,6 +71,7 @@ obj-$(CONFIG_PAGE_POISONING) += page_poison.o obj-$(CONFIG_SLAB) += slab.o obj-$(CONFIG_SLUB) += slub.o obj-$(CONFIG_KASAN) += kasan/ +obj-$(CONFIG_KMSAN) += kmsan/ obj-$(CONFIG_FAILSLAB) += failslab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_MEMTEST) += memtest.o diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 179d55af5852..f9f38f7c1cd4 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -140,6 +140,12 @@ _c_flags += $(if $(patsubst n%,, \ $(CFLAGS_KASAN), $(CFLAGS_KASAN_NOSANITIZE)) endif +ifeq ($(CONFIG_KMSAN),y) +_c_flags += $(if $(patsubst n%,, \ + $(KMSAN_SANITIZE_$(basetarget).o)$(KMSAN_SANITIZE)y), \ + $(CFLAGS_KMSAN)) +endif + ifeq ($(CONFIG_UBSAN),y) _c_flags += $(if $(patsubst n%,, \ $(UBSAN_SANITIZE_$(basetarget).o)$(UBSAN_SANITIZE)$(CONFIG_UBSAN_SANITIZE_ALL)), \ From patchwork Wed Oct 30 14:22:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219637 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 372BD1668 for ; Wed, 30 Oct 2019 14:23:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E8C8F20656 for ; Wed, 30 Oct 2019 14:23:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NjzwrgXu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8C8F20656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 199646B0272; Wed, 30 Oct 2019 10:23:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 149876B0273; Wed, 30 Oct 2019 10:23:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0125A6B0274; Wed, 30 Oct 2019 10:23:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id C63B86B0272 for ; Wed, 30 Oct 2019 10:23:35 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 5B9C44DC0 for ; Wed, 30 Oct 2019 14:23:35 +0000 (UTC) X-FDA: 76100669190.06.hand68_3c5f9d22b7b0e X-Spam-Summary: 2,0,0,d283d895bea4d5ca,d41d8cd98f00b204,35zy5xqykcpglqnijwlttlqj.htrqnsz2-rrp0fhp.twl@flex--glider.bounces.google.com,:ard.biesheuvel@linaro.org:tglx@linutronix.de:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:2:41:152:355:379:541:800:960:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1593:1594:1605:173 0:1747:1 X-HE-Tag: hand68_3c5f9d22b7b0e X-Filterd-Recvd-Size: 9386 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:34 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id b4so1403960wrn.8 for ; Wed, 30 Oct 2019 07:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xi9D7JDIAq+0yP5j9PqxctLN53PCCc/3qM0t4vepucw=; b=NjzwrgXu8Zmb59lDjCi5qFmrFyVkUcobEDlAYDjzDqDmstKIuMhSC7KpW/UAP2Vrb9 5wiVDVg51LEaz3VvjdVGw2ZcTYnedx1jWOPIZQPjnTgtrjG6FX5VxUMJQxwOHCvjRXHt 4kqEL/nbunJ75S3f6WU4Ch2rXMldaNUONuf4ygXElw44yzCccgo75M0HEEkSzfhSAw4l WaSxS432VmnQR2LyYePM8tRLLGt15A8kBpLFz/GLqEVIOETmn1rFLk0ceSh/QJHN/+62 +/R204TjPc1RbkA8dGxzlGTkuS4LZtYd8ljChEG5ok3j+LhigO+XdGpC/EDuTgUtlW6X oJnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xi9D7JDIAq+0yP5j9PqxctLN53PCCc/3qM0t4vepucw=; b=BFyjOjlzJjOISO6tr2ssWjktCH1RVF8QPgff06CcLo4GWVrP2QhQkcC20Ldp3s1RC4 qNEjMGVTWxtWmegk4YaApV4iZt1lq7OX9UizAsYV6vF919UP5EGqGs26qOpFBG3OkIWl JUpj6j5GRiYDU2YYXZlJVdMviU2rdNoEF5t2gp6vzlslGGKN6nLSTWAfY+teX/Jn8Gb8 B0tKyoPmpVDz1y/Lvh5hLPk2vM/NUZC3ewg8ABmPjxqJSooXOzAsyUmbUMXpa9kOAWA0 S3ciRgRzfK7vrK+htAqAB2boTDQEjUZPZyozW9qyqMBevKUUFII/KDVh44LV/+0Bm+va qXIg== X-Gm-Message-State: APjAAAXxT94tKN9ngZws0YNF3oJh2H3VDJD5IXgV7cgfYjuDtWoeY5tn 5NBpaWPYqwbCoOOG21pDcZy7UBEduPQ= X-Google-Smtp-Source: APXvYqxQQtN17t2Is9s6ytWEJPnn8N/GMar9nap3JRKFnByx4myc5XBEJfYgPiRlyvOW392DUWGf9xRjeRY= X-Received: by 2002:a05:6000:14a:: with SMTP id r10mr121305wrx.310.1572445413362; Wed, 30 Oct 2019 07:23:33 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:28 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-17-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 16/25] kmsan: disable KMSAN instrumentation for certain kernel parts From: glider@google.com To: Ard Biesheuvel , Thomas Gleixner , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instrumenting some files with KMSAN will result in kernel being unable to link, boot or crashing at runtime for various reasons (e.g. infinite recursion caused by instrumentation hooks calling instrumented code again). Disable KMSAN in the following places: - arch/x86/boot and arch/x86/realmode/rm, as KMSAN doesn't work for i386; - arch/x86/entry/vdso, which isn't linked with KMSAN runtime; - three files in arch/x86/kernel - boot problems; - arch/x86/mm/cpu_entry_area.c - recursion; - EFI stub - build failures; - kcov, stackdepot - recursion. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Ard Biesheuvel Cc: Thomas Gleixner Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I90961eabf2dcb9ae992aed259088953bad5e4d6d --- arch/x86/boot/Makefile | 2 ++ arch/x86/boot/compressed/Makefile | 2 ++ arch/x86/entry/vdso/Makefile | 3 +++ arch/x86/kernel/Makefile | 4 ++++ arch/x86/kernel/cpu/Makefile | 1 + arch/x86/mm/Makefile | 2 ++ arch/x86/realmode/rm/Makefile | 2 ++ drivers/firmware/efi/libstub/Makefile | 1 + kernel/Makefile | 1 + lib/Makefile | 1 + 10 files changed, 19 insertions(+) diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile index e2839b5c246c..c039abd4c81f 100644 --- a/arch/x86/boot/Makefile +++ b/arch/x86/boot/Makefile @@ -10,6 +10,8 @@ # KASAN_SANITIZE := n +# KMSAN doesn't work for i386 +KMSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y # Kernel does not boot with kcov instrumentation here. diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/Makefile index 6b84afdd7538..9efe2d9fca4c 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -18,6 +18,8 @@ # compressed vmlinux.bin.all + u32 size of vmlinux.bin.all KASAN_SANITIZE := n +# KMSAN doesn't work for i386 +KMSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in. diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile index 0f2154106d01..000467a1a4f2 100644 --- a/arch/x86/entry/vdso/Makefile +++ b/arch/x86/entry/vdso/Makefile @@ -11,6 +11,9 @@ include $(srctree)/lib/vdso/Makefile KBUILD_CFLAGS += $(DISABLE_LTO) KASAN_SANITIZE := n +# Undefined references to KMSAN hooks. +KMSAN_SANITIZE_vclock_gettime.o := n +KMSAN_SANITIZE_vgetcpu.o := n UBSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 3578ad248bc9..ce39972a7edf 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -28,6 +28,10 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n KASAN_SANITIZE_stacktrace.o := n KASAN_SANITIZE_paravirt.o := n +# Work around reboot loop. +KMSAN_SANITIZE_head$(BITS).o := n +KMSAN_SANITIZE_nmi.o := n + OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o := y OBJECT_FILES_NON_STANDARD_test_nx.o := y OBJECT_FILES_NON_STANDARD_paravirt_patch.o := y diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile index d7a1e5a9331c..41f4f8f2f2f0 100644 --- a/arch/x86/kernel/cpu/Makefile +++ b/arch/x86/kernel/cpu/Makefile @@ -12,6 +12,7 @@ endif # If these files are instrumented, boot hangs during the first second. KCOV_INSTRUMENT_common.o := n KCOV_INSTRUMENT_perf_event.o := n +KMSAN_SANITIZE_common.o := n # Make sure load_percpu_segment has no stackprotector nostackp := $(call cc-option, -fno-stack-protector) diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index 84373dc9b341..42cb3a6409b0 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -7,6 +7,8 @@ KCOV_INSTRUMENT_mem_encrypt_identity.o := n KASAN_SANITIZE_mem_encrypt.o := n KASAN_SANITIZE_mem_encrypt_identity.o := n +KMSAN_SANITIZE_cpu_entry_area.o := n + ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_mem_encrypt.o = -pg CFLAGS_REMOVE_mem_encrypt_identity.o = -pg diff --git a/arch/x86/realmode/rm/Makefile b/arch/x86/realmode/rm/Makefile index f60501a384f9..27e7bc0bbdde 100644 --- a/arch/x86/realmode/rm/Makefile +++ b/arch/x86/realmode/rm/Makefile @@ -7,6 +7,8 @@ # # KASAN_SANITIZE := n +# KMSAN doesn't work for i386 +KMSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y # Prevents link failures: __sanitizer_cov_trace_pc() is not linked in. diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index 0460c7581220..11869c17a64c 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -32,6 +32,7 @@ KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \ GCOV_PROFILE := n KASAN_SANITIZE := n +KMSAN_SANITIZE := n UBSAN_SANITIZE := n OBJECT_FILES_NON_STANDARD := y diff --git a/kernel/Makefile b/kernel/Makefile index daad787fb795..5fd6fbca2592 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -30,6 +30,7 @@ KCOV_INSTRUMENT_extable.o := n # Don't self-instrument. KCOV_INSTRUMENT_kcov.o := n KASAN_SANITIZE_kcov.o := n +KMSAN_SANITIZE_kcov.o := n CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) # cond_syscall is currently not LTO compatible diff --git a/lib/Makefile b/lib/Makefile index cb44262c38ee..a96d1d6e6a34 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -218,6 +218,7 @@ obj-$(CONFIG_IRQ_POLL) += irq_poll.o obj-$(CONFIG_STACKDEPOT) += stackdepot.o KASAN_SANITIZE_stackdepot.o := n +KMSAN_SANITIZE_stackdepot.o := n KCOV_INSTRUMENT_stackdepot.o := n libfdt_files = fdt.o fdt_ro.o fdt_wip.o fdt_rw.o fdt_sw.o fdt_strerror.o \ From patchwork Wed Oct 30 14:22:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219639 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 757A31668 for ; Wed, 30 Oct 2019 14:23:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34C1E2087E for ; Wed, 30 Oct 2019 14:23:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Enj5rdnS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34C1E2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9AE826B0274; Wed, 30 Oct 2019 10:23:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 960786B0275; Wed, 30 Oct 2019 10:23:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 800366B0276; Wed, 30 Oct 2019 10:23:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 523156B0274 for ; Wed, 30 Oct 2019 10:23:39 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id EA131181AEF31 for ; Wed, 30 Oct 2019 14:23:38 +0000 (UTC) X-FDA: 76100669316.19.blood79_3ce060332e947 X-Spam-Summary: 2,0,0,06767967a12d4169,d41d8cd98f00b204,36jy5xqykcpsjolghujrrjoh.frpolqx0-ppnydfn.ruj@flex--glider.bounces.google.com,:akpm@linux-foundation.org:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:2:41:152:355:379:541:800:960:966:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1593:1594:1605 :1606:17 X-HE-Tag: blood79_3ce060332e947 X-Filterd-Recvd-Size: 9258 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:38 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id j17so1208301wru.13 for ; Wed, 30 Oct 2019 07:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=RzDCqX7fVteJS3E4fZ9bQiryP3pTaR7mMOYJU/v7Dvc=; b=Enj5rdnSmxKCHUa2MGe3XLvYYbthefD0qIMthtdUf3p7jt4q6J1KlYRxvvaeJbw25K b389zApkQVqVpOyF+WBV0hvK+Of+uAW+6w+M6n9B+xkQ/YQ8LW48oKBm2DMXDQ/LMPFW IvPONzwfed8iqQCbXnIUL2IG5t45eMEY9Eg645L0QXRnNI0Jqqn9eNV+/Zb7jra5dvcV P0/xaXjiH7c27EvtBaB7Ux6WcLahyF5CUbTwQ0r1fILaOceX7Hd6Xu1tBnQh2NAEK7/F VfR7YWPf76n1yhc7mftyEFHr0dw038yrukIdhSQWMny2zMXmuOOLRQ6DNldgmDQ0MCeU WRSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=RzDCqX7fVteJS3E4fZ9bQiryP3pTaR7mMOYJU/v7Dvc=; b=r8Fx5tMwIrTeum2/xqlr/lT+8QLSiwolRspeY6CyRo0sFzdn6Ii94pucC1XlxWWmL8 kbljIr6LdPGhon0nmo80jDHACmhgB0Ga3QEs9BxODDk7NR3v2sf4tYt5TocUQFJAGfBf JAUq/v2H4060w6t1cnfveFt78oTqx3MO8YmvUsmQfTdzYUNKAFkIt7KrCAvbgnjD3aGP 9c9GQThZ7hUW9xS48XfT/mOfdS7z0pWEWx7O9C4W8m/yp3+g5wMkiJiB3BDHrmtppO4M M6Qj571TZzhnNa2PYLIZEjZTV67kKDE5OmyXyMkTP3aWlw2dd2H2aIG+ermOLyGVwbdc WDXw== X-Gm-Message-State: APjAAAV7cZCfWXQAmsZs314MJNk1bN/cAl9avQ0Gtbe27MrmBOmaCKtF M1uID6l0nhF6Hhpw1DqwuFhO/8TMlqM= X-Google-Smtp-Source: APXvYqxODVnetcMyi6dC7YdP/TzwKi6vgzBTkf8NjuMxEP2/5wJhjhj8Q3h4YpSCkA5ozl37OqsUWgLcV5c= X-Received: by 2002:a5d:424f:: with SMTP id s15mr113158wrr.368.1572445416536; Wed, 30 Oct 2019 07:23:36 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:29 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-18-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 17/25] kmsan: mm: call KMSAN hooks from SLUB code From: glider@google.com To: Andrew Morton , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In order to report uninitialized memory coming from heap allocations KMSAN has to poison them unless they're created with __GFP_ZERO. It's handy that we need KMSAN hooks in the places where init_on_alloc/init_on_free initialization is performed. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Andrew Morton Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I51103b7981d3aabed747d0c85cbdc85568665871 --- mm/slub.c | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index b25c807a111f..8b7069812801 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -21,6 +21,8 @@ #include #include #include +#include +#include /* KMSAN_INIT_VALUE */ #include #include #include @@ -285,17 +287,27 @@ static void prefetch_freepointer(const struct kmem_cache *s, void *object) prefetch(object + s->offset); } +/* + * When running under KMSAN, get_freepointer_safe() may return an uninitialized + * pointer value in the case the current thread loses the race for the next + * memory chunk in the freelist. In that case this_cpu_cmpxchg_double() in + * slab_alloc_node() will fail, so the uninitialized value won't be used, but + * KMSAN will still check all arguments of cmpxchg because of imperfect + * handling of inline assembly. + * To work around this problem, use KMSAN_INIT_VALUE() to force initialize the + * return value of get_freepointer_safe(). + */ static inline void *get_freepointer_safe(struct kmem_cache *s, void *object) { unsigned long freepointer_addr; void *p; if (!debug_pagealloc_enabled()) - return get_freepointer(s, object); + return KMSAN_INIT_VALUE(get_freepointer(s, object)); freepointer_addr = (unsigned long)object + s->offset; probe_kernel_read(&p, (void **)freepointer_addr, sizeof(p)); - return freelist_ptr(s, p, freepointer_addr); + return KMSAN_INIT_VALUE(freelist_ptr(s, p, freepointer_addr)); } static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) @@ -1390,6 +1402,7 @@ static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) ptr = kasan_kmalloc_large(ptr, size, flags); /* As ptr might get tagged, call kmemleak hook after KASAN. */ kmemleak_alloc(ptr, size, 1, flags); + kmsan_kmalloc_large(ptr, size, flags); return ptr; } @@ -1397,6 +1410,7 @@ static __always_inline void kfree_hook(void *x) { kmemleak_free(x); kasan_kfree_large(x, _RET_IP_); + kmsan_kfree_large(x); } static __always_inline bool slab_free_hook(struct kmem_cache *s, void *x) @@ -1453,6 +1467,12 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, } while (object != old_tail); } + do { + object = next; + next = get_freepointer(s, object); + kmsan_slab_free(s, object); + } while (object != old_tail); + /* * Compiler cannot detect this function can be removed if slab_free_hook() * evaluates to nothing. Thus, catch all relevant config debug options here. @@ -2776,6 +2796,7 @@ static __always_inline void *slab_alloc_node(struct kmem_cache *s, if (unlikely(slab_want_init_on_alloc(gfpflags, s)) && object) memset(object, 0, s->object_size); + kmsan_slab_alloc(s, object, gfpflags); slab_post_alloc_hook(s, gfpflags, 1, &object); return object; @@ -2804,6 +2825,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); ret = kasan_kmalloc(s, ret, size, gfpflags); + return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2816,7 +2838,6 @@ void *kmem_cache_alloc_node(struct kmem_cache *s, gfp_t gfpflags, int node) trace_kmem_cache_alloc_node(_RET_IP_, ret, s->object_size, s->size, gfpflags, node); - return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node); @@ -2832,6 +2853,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, size, s->size, gfpflags, node); ret = kasan_kmalloc(s, ret, size, gfpflags); + return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3157,7 +3179,7 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size, void **p) { struct kmem_cache_cpu *c; - int i; + int i, j; /* memcg and kmem_cache debug support */ s = slab_pre_alloc_hook(s, flags); @@ -3198,11 +3220,11 @@ int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size, /* Clear memory outside IRQ disabled fastpath loop */ if (unlikely(slab_want_init_on_alloc(flags, s))) { - int j; - for (j = 0; j < i; j++) memset(p[j], 0, s->object_size); } + for (j = 0; j < i; j++) + kmsan_slab_alloc(s, p[j], flags); /* memcg and kmem_cache debug support */ slab_post_alloc_hook(s, flags, size, p); @@ -3803,6 +3825,7 @@ static int __init setup_slub_min_objects(char *str) __setup("slub_min_objects=", setup_slub_min_objects); +__no_sanitize_memory void *__kmalloc(size_t size, gfp_t flags) { struct kmem_cache *s; @@ -5717,6 +5740,7 @@ static char *create_unique_id(struct kmem_cache *s) p += sprintf(p, "%07u", s->size); BUG_ON(p > name + ID_STR_LENGTH - 1); + kmsan_unpoison_shadow(name, p - name); return name; } @@ -5866,6 +5890,7 @@ static int sysfs_slab_alias(struct kmem_cache *s, const char *name) al->name = name; al->next = alias_list; alias_list = al; + kmsan_unpoison_shadow(al, sizeof(struct saved_alias)); return 0; } From patchwork Wed Oct 30 14:22:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219641 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D993C112B for ; Wed, 30 Oct 2019 14:23:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8BA972173E for ; Wed, 30 Oct 2019 14:23:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RJBEDx47" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8BA972173E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F06D76B0276; Wed, 30 Oct 2019 10:23:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E67E36B0277; Wed, 30 Oct 2019 10:23:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D09546B0278; Wed, 30 Oct 2019 10:23:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0241.hostedemail.com [216.40.44.241]) by kanga.kvack.org (Postfix) with ESMTP id A16846B0276 for ; Wed, 30 Oct 2019 10:23:42 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 4E38F181AEF31 for ; Wed, 30 Oct 2019 14:23:42 +0000 (UTC) X-FDA: 76100669484.29.wheel01_3d588eececc2b X-Spam-Summary: 2,0,0,4bf5850416632889,d41d8cd98f00b204,365y5xqykcp4mrojkxmuumrk.iusrot03-ssq1giq.uxm@flex--glider.bounces.google.com,:akpm@linux-foundation.org:gregkh@linuxfoundation.org:edumazet@google.com:wsa@the-dreams.de:pmladek@suse.com:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:ericvh@gmail.com:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:305:327:355:379:541:800:960:966:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:160 5:1730:1 X-HE-Tag: wheel01_3d588eececc2b X-Filterd-Recvd-Size: 20580 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:41 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id 14so677874wmk.4 for ; Wed, 30 Oct 2019 07:23:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5uQc/iunKaF8S87eaeamjDv6eJFfG470j2IHwC4WUGo=; b=RJBEDx47xws0o/Nco3Adwb3dwgOMS6+hNMwTkkkTs+52NKXZPNeDmvUTwHQCDLwWnm WI9l32zr6cWW+rubRoOzzLowPHJegaEn5kuAJyjj15SyrpdI2760OHs8FgIwGZp5pEiF ATHJyKryj3rKeLblr8qesYi9/hDoWOI5JlFDiLVf4xP0RrfgrrVwIMUGCcNbvc7uI9fv 3dXtpl6I5tqXnzkkHH394gWxmMVnk+GTq7Q/YfbP81uMOR9M4SJfwOMEFHTKySAlIHtF +Xh0NfGKH34AJPNouIQ/SFOujNbHqkaKoSfueNmoPov9Dr2qWlSS6HkjXPe4+NgWxcxb R2Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5uQc/iunKaF8S87eaeamjDv6eJFfG470j2IHwC4WUGo=; b=j5we6g7stTJrjfWQn5uGVWAcWRQrKsVJGarkv5p2Ze3DEfDdJb12j+4+Ux4YRM1uAX 1gdyeSXKP8r2EuNVjwC1FQ0Itzz/umg35e8FfpnQmojvkzfa+SU0mS4xQV/aiNcjwY3F kqsIK871gPyhgFlgen9aRo/QsDvMTUxcZiYl6QinPEUNdFVHrIPIISeTQYLKpaY8pdbw 34SC3nlIdaU8rvx0/jy3UULOGIPiK4YfWjRcJhHi9P06wCoBoEvT81NxGvvBIMztyMoY L9XGIE4/8m9u7xepz/KOSXYc9ni/Vj/vmKEUeY4DA64BmSMynH7D+gSqOEIOc5wTNw+o 7Pew== X-Gm-Message-State: APjAAAUmZmYxLHjlb7qDrvYN+oTec7Z5tZ7kldAyaPvd0dO3AunynnPI nfUCEWJ6ebxAwvOQMoJoy3UOhnIMhvU= X-Google-Smtp-Source: APXvYqzZSL2w68VuEUwPAGHanGAsV8Sy+yzCz/mHenMdOfUEZwktY/bucxYhNBor4p/geKYOaB1yVsM0XXI= X-Received: by 2002:adf:e64f:: with SMTP id b15mr112572wrn.372.1572445419937; Wed, 30 Oct 2019 07:23:39 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:30 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-19-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 18/25] kmsan: call KMSAN hooks where needed From: glider@google.com To: Andrew Morton , Greg Kroah-Hartman , Eric Dumazet , Wolfram Sang , Petr Mladek , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, ericvh@gmail.com, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Insert KMSAN hooks that check for potential memory errors and/or make necessary bookkeeping changes: - allocate/split/deallocate metadata pages in alloc_pages()/split_page()/free_page(); - clear page shadow and origins in clear_page(), copy_user_highpage(); - copy page metadata in copy_highpage(), wp_page_copy(); - handle vmap()/vunmap()/iounmap(); - handle task creation and deletion; - initialize result of vscnprintf() in vprintk_store(); - call softirq entry/exit hooks in kernel/softirq.c; - check/initialize memory sent to/read from USB, I2C, and network Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Andrew Morton Cc: Greg Kroah-Hartman Cc: Eric Dumazet Cc: Wolfram Sang Cc: Petr Mladek Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org Acked-by: Petr Mladek # printk part --- v2: - dropped call to kmsan_handle_vprintk, updated comment in printk.c Change-Id: I1250a928d9263bf71fdaa067a070bdee686ef47b --- arch/x86/include/asm/page_64.h | 13 +++++++++++++ arch/x86/mm/ioremap.c | 3 +++ drivers/i2c/i2c-core-base.c | 2 ++ drivers/usb/core/urb.c | 2 ++ include/linux/highmem.h | 4 ++++ kernel/exit.c | 2 ++ kernel/fork.c | 2 ++ kernel/kthread.c | 2 ++ kernel/printk/printk.c | 8 +++++++- kernel/softirq.c | 5 +++++ lib/ioremap.c | 5 +++++ mm/compaction.c | 9 +++++++++ mm/gup.c | 3 +++ mm/memory.c | 2 ++ mm/page_alloc.c | 16 ++++++++++++++++ mm/vmalloc.c | 23 +++++++++++++++++++++-- net/sched/sch_generic.c | 2 ++ 17 files changed, 100 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index 939b1cff4a7b..0ba43d93414f 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -44,14 +44,27 @@ void clear_page_orig(void *page); void clear_page_rep(void *page); void clear_page_erms(void *page); +/* This is an assembly header, avoid including too much of kmsan.h */ +#ifdef CONFIG_KMSAN +void kmsan_clear_page(void *page_addr); +#endif +__no_sanitize_memory static inline void clear_page(void *page) { +#ifdef CONFIG_KMSAN + /* alternative_call_2() changes |page|. */ + void *page_copy = page; +#endif alternative_call_2(clear_page_orig, clear_page_rep, X86_FEATURE_REP_GOOD, clear_page_erms, X86_FEATURE_ERMS, "=D" (page), "0" (page) : "cc", "memory", "rax", "rcx"); +#ifdef CONFIG_KMSAN + /* Clear KMSAN shadow for the pages that have it. */ + kmsan_clear_page(page_copy); +#endif } void copy_page(void *to, void *from); diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index a39dcdb5ae34..fdb2abc11a82 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -7,6 +7,7 @@ * (C) Copyright 1995 1996 Linus Torvalds */ +#include #include #include #include @@ -451,6 +452,8 @@ void iounmap(volatile void __iomem *addr) return; } + kmsan_iounmap_page_range((unsigned long)addr, + (unsigned long)addr + get_vm_area_size(p)); free_memtype(p->phys_addr, p->phys_addr + get_vm_area_size(p)); /* Finally remove it */ diff --git a/drivers/i2c/i2c-core-base.c b/drivers/i2c/i2c-core-base.c index 5f6a4985f2bc..9685d3399c79 100644 --- a/drivers/i2c/i2c-core-base.c +++ b/drivers/i2c/i2c-core-base.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -1975,6 +1976,7 @@ int __i2c_transfer(struct i2c_adapter *adap, struct i2c_msg *msgs, int num) trace_i2c_write(adap, &msgs[i], i); } + kmsan_handle_i2c_transfer(msgs, num); /* Retry automatically on arbitration loss */ orig_jiffies = jiffies; for (ret = 0, try = 0; try <= adap->retries; try++) { diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c index 0eab79f82ce4..5bdb54d71c2e 100644 --- a/drivers/usb/core/urb.c +++ b/drivers/usb/core/urb.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -401,6 +402,7 @@ int usb_submit_urb(struct urb *urb, gfp_t mem_flags) URB_SETUP_MAP_SINGLE | URB_SETUP_MAP_LOCAL | URB_DMA_SG_COMBINED); urb->transfer_flags |= (is_out ? URB_DIR_OUT : URB_DIR_IN); + kmsan_handle_urb(urb, is_out); if (xfertype != USB_ENDPOINT_XFER_CONTROL && dev->state < USB_STATE_CONFIGURED) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index ea5cdbd8c2c3..623b56f48685 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -255,6 +256,8 @@ static inline void copy_user_highpage(struct page *to, struct page *from, vfrom = kmap_atomic(from); vto = kmap_atomic(to); copy_user_page(vto, vfrom, vaddr, to); + /* User pages don't have shadow, just clear the destination. */ + kmsan_clear_page(page_address(to)); kunmap_atomic(vto); kunmap_atomic(vfrom); } @@ -270,6 +273,7 @@ static inline void copy_highpage(struct page *to, struct page *from) vfrom = kmap_atomic(from); vto = kmap_atomic(to); copy_page(vto, vfrom); + kmsan_copy_page_meta(to, from); kunmap_atomic(vto); kunmap_atomic(vfrom); } diff --git a/kernel/exit.c b/kernel/exit.c index a46a50d67002..9e3ce929110b 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -60,6 +60,7 @@ #include #include #include +#include #include #include #include @@ -719,6 +720,7 @@ void __noreturn do_exit(long code) profile_task_exit(tsk); kcov_task_exit(tsk); + kmsan_task_exit(tsk); WARN_ON(blk_needs_flush_plug(tsk)); diff --git a/kernel/fork.c b/kernel/fork.c index bcdf53125210..0f08952a42dc 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -931,6 +932,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) account_kernel_stack(tsk, 1); kcov_task_init(tsk); + kmsan_task_create(tsk); #ifdef CONFIG_FAULT_INJECTION tsk->fail_nth = 0; diff --git a/kernel/kthread.c b/kernel/kthread.c index b262f47046ca..33ca743ca8b5 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -350,6 +351,7 @@ struct task_struct *__kthread_create_on_node(int (*threadfn)(void *data), set_cpus_allowed_ptr(task, cpu_all_mask); } kfree(create); + kmsan_task_create(task); return task; } diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index ca65327a6de8..4b0dbed0333a 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -1914,7 +1914,13 @@ int vprintk_store(int facility, int level, * The printf needs to come first; we need the syslog * prefix which might be passed-in as a parameter. */ - text_len = vscnprintf(text, sizeof(textbuf), fmt, args); + /* + * If any of vscnprintf() arguments is uninitialized, KMSAN will report + * one or more errors and also probably mark text_len as uninitialized. + * Initialize |text_len| to prevent the errors from spreading further. + */ + text_len = KMSAN_INIT_VALUE(vscnprintf(text, sizeof(textbuf), fmt, + args)); /* mark and strip a trailing newline */ if (text_len && text[text_len-1] == '\n') { diff --git a/kernel/softirq.c b/kernel/softirq.c index 0427a86743a4..6d566dd68b35 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -11,6 +11,7 @@ #include #include +#include #include #include #include @@ -370,7 +371,9 @@ static inline void invoke_softirq(void) * it is the irq stack, because it should be near empty * at this stage. */ + kmsan_softirq_enter(); __do_softirq(); + kmsan_softirq_exit(); #else /* * Otherwise, irq_exit() is called on the task stack that can @@ -600,7 +603,9 @@ static void run_ksoftirqd(unsigned int cpu) * We can safely run softirq on inline stack, as we are not deep * in the task stack here. */ + kmsan_softirq_enter(); __do_softirq(); + kmsan_softirq_exit(); local_irq_enable(); cond_resched(); return; diff --git a/lib/ioremap.c b/lib/ioremap.c index 0a2ffadc6d71..5f830cee5bfc 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -6,6 +6,7 @@ * * (C) Copyright 1995 1996 Linus Torvalds */ +#include #include #include #include @@ -214,6 +215,8 @@ int ioremap_page_range(unsigned long addr, unsigned long start; unsigned long next; int err; + unsigned long old_addr = addr; + phys_addr_t old_phys_addr = phys_addr; might_sleep(); BUG_ON(addr >= end); @@ -228,6 +231,8 @@ int ioremap_page_range(unsigned long addr, } while (pgd++, phys_addr += (next - addr), addr = next, addr != end); flush_cache_vmap(start, end); + if (!err) + kmsan_ioremap_page_range(old_addr, end, old_phys_addr, prot); return err; } diff --git a/mm/compaction.c b/mm/compaction.c index 672d3c78c6ab..720a8a4dafec 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -84,6 +84,15 @@ static void split_map_pages(struct list_head *list) for (i = 0; i < nr_pages; i++) { list_add(&page->lru, &tmp_list); +#ifdef CONFIG_KMSAN + /* + * TODO(glider): we may lose the metadata when copying + * something to these pages. Need to allocate shadow + * and origin pages here instead. + */ + page->shadow = NULL; + page->origin = NULL; +#endif page++; } } diff --git a/mm/gup.c b/mm/gup.c index 8f236a335ae9..8f5f99772278 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -4,6 +4,7 @@ #include #include +#include #include #include #include @@ -2349,6 +2350,7 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write, gup_fast_permitted(start, end)) { local_irq_save(flags); gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr); + kmsan_gup_pgd_range(pages, nr); local_irq_restore(flags); } @@ -2418,6 +2420,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, gup_fast_permitted(start, end)) { local_irq_disable(); gup_pgd_range(addr, end, gup_flags, pages, &nr); + kmsan_gup_pgd_range(pages, nr); local_irq_enable(); ret = nr; } diff --git a/mm/memory.c b/mm/memory.c index b1ca51a079f2..48ceacc06e2d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -51,6 +51,7 @@ #include #include #include +#include #include #include #include @@ -2328,6 +2329,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (!new_page) goto oom; cow_user_page(new_page, old_page, vmf->address, vma); + kmsan_copy_page_meta(new_page, old_page); } if (mem_cgroup_try_charge_delay(new_page, mm, GFP_KERNEL, &memcg, false)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ecc3dbad606b..c98e4441c7c0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -26,6 +26,8 @@ #include #include #include +#include +#include #include #include #include @@ -1133,6 +1135,7 @@ static __always_inline bool free_pages_prepare(struct page *page, VM_BUG_ON_PAGE(PageTail(page), page); trace_mm_page_free(page, order); + kmsan_free_page(page, order); /* * Check tail pages before head page information is cleared to @@ -3121,6 +3124,7 @@ void split_page(struct page *page, unsigned int order) VM_BUG_ON_PAGE(PageCompound(page), page); VM_BUG_ON_PAGE(!page_count(page), page); + kmsan_split_page(page, order); for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); split_page_owner(page, order); @@ -3253,6 +3257,13 @@ static struct page *rmqueue_pcplist(struct zone *preferred_zone, /* * Allocate a page from the given zone. Use pcplists for order-0 allocations. */ +/* + * TODO(glider): rmqueue() may call __msan_poison_alloca() through a call to + * set_pfnblock_flags_mask(). If __msan_poison_alloca() attempts to allocate + * pages for the stack depot, it may call rmqueue() again, which will result + * in a deadlock. + */ +__no_sanitize_memory static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, @@ -4781,6 +4792,11 @@ __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid, trace_mm_page_alloc(page, order, alloc_mask, ac.migratetype); + if (page) + if (kmsan_alloc_page(page, order, gfp_mask)) { + __free_pages(page, order); + page = NULL; + } return page; } EXPORT_SYMBOL(__alloc_pages_nodemask); diff --git a/mm/vmalloc.c b/mm/vmalloc.c index a3c70e275f4e..bdf66ffcf02c 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -119,7 +120,8 @@ static void vunmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end) } while (p4d++, addr = next, addr != end); } -static void vunmap_page_range(unsigned long addr, unsigned long end) +/* Exported for KMSAN, visible in mm/kmsan/kmsan.h only. */ +void __vunmap_page_range(unsigned long addr, unsigned long end) { pgd_t *pgd; unsigned long next; @@ -133,6 +135,12 @@ static void vunmap_page_range(unsigned long addr, unsigned long end) vunmap_p4d_range(pgd, addr, next); } while (pgd++, addr = next, addr != end); } +EXPORT_SYMBOL(__vunmap_page_range); +static void vunmap_page_range(unsigned long addr, unsigned long end) +{ + kmsan_vunmap_page_range(addr, end); + __vunmap_page_range(addr, end); +} static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr) @@ -216,8 +224,11 @@ static int vmap_p4d_range(pgd_t *pgd, unsigned long addr, * will have pfns corresponding to the "pages" array. * * Ie. pte at addr+N*PAGE_SIZE shall point to pfn corresponding to pages[N] + * + * This function is exported for use in KMSAN, but is only declared in KMSAN + * headers. */ -static int vmap_page_range_noflush(unsigned long start, unsigned long end, +int __vmap_page_range_noflush(unsigned long start, unsigned long end, pgprot_t prot, struct page **pages) { pgd_t *pgd; @@ -237,6 +248,14 @@ static int vmap_page_range_noflush(unsigned long start, unsigned long end, return nr; } +EXPORT_SYMBOL(__vmap_page_range_noflush); + +static int vmap_page_range_noflush(unsigned long start, unsigned long end, + pgprot_t prot, struct page **pages) +{ + kmsan_vmap_page_range_noflush(start, end, prot, pages); + return __vmap_page_range_noflush(start, end, prot, pages); +} static int vmap_page_range(unsigned long start, unsigned long end, pgprot_t prot, struct page **pages) diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 17bd8f539bc7..fd22c4a4ba42 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -659,6 +660,7 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc) } else { qdisc->empty = true; } + kmsan_check_skb(skb); return skb; } From patchwork Wed Oct 30 14:22:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219643 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83C97112B for ; Wed, 30 Oct 2019 14:23:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34BD620656 for ; Wed, 30 Oct 2019 14:23:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="HHKb6zBK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34BD620656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 013816B0277; Wed, 30 Oct 2019 10:23:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EDE836B0279; Wed, 30 Oct 2019 10:23:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA6DA6B027A; Wed, 30 Oct 2019 10:23:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id ADB676B0277 for ; Wed, 30 Oct 2019 10:23:45 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 291E68249980 for ; Wed, 30 Oct 2019 14:23:45 +0000 (UTC) X-FDA: 76100669610.03.knot70_3dc51ee4dae0a X-Spam-Summary: 2,0,0,f337da719a4c2695,d41d8cd98f00b204,375y5xqykcaqkpmhivksskpi.gsqpmry1-qqozego.svk@flex--glider.bounces.google.com,:tglx@linutronix.de:akpm@linux-foundation.org:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:1:2:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747: 1777:179 X-HE-Tag: knot70_3dc51ee4dae0a X-Filterd-Recvd-Size: 12893 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:44 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id g17so937028wmc.4 for ; Wed, 30 Oct 2019 07:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=yVafNoSuKaWqJmOqG3sfMlqPbvP5HezJZtUVQQWagt4=; b=HHKb6zBKgiK7ekEBUiVlDbev8PdfmgkqJpB0gg9w096aaQC2quZx4kefCqHMMiCp5x 258D14MxLKrmxLD1sa/rVSh9ZjJlbzUZgRNlfHgH8zV5hYW5A3z9sh9EA3FO6LMOz1Bp 0EWwmPn/m1hYZn6vUdJ/HGQSRwbfCJGgcZ8jnjhxh2KIQpbloJkNLi0kUL4+8paNuBXI icS0fy3fLxZk6tGkj0o1vQErzmmzvJJmUoqrFUOHWduARLTx686upGlMnb/9XJRcMpo0 8JRXpG22etMdujY60LhdtlmE6LZsr7e91nw0WQuWSQ8QBgKP9wfKwYBAGcEY9/5vG5AV yaAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=yVafNoSuKaWqJmOqG3sfMlqPbvP5HezJZtUVQQWagt4=; b=Mg+qtwSgvVmCX1/A2znJ542Vdl7ufyFWMw7yo7w2eG0omHX4mWJeDsGXCr9kwHg3nD gW/P2nTLFxIxMppAYHOHgfAYC06J6VzX31DBJAZHvS0dBW/wItKhhkW6OO7194WzT9VG oEyOp8uKJI6IaWx0t7tsv8k2hGXgYvF5YV257l2KVQatsj7jca+la7boGr04Wb4akTBS O+UTl+ivn05RDl7iI9l3OgrENg+n5LHIb8R4XdisnbRChS5vMq+U6hAt9E/IADaqY8Q3 jAAYvxT7WsLHkhpwdysfv+w5O9NHadHXgov0tI/rOays6w1I9xRhitbckx0ALre63Lb0 kxRw== X-Gm-Message-State: APjAAAUJCSQNYYd6I4Nj7CXb7Vfb1pnqJKKUBD2m127HjJ2hRC13vhSS 92nuL9Xd+zOGDLNQajZG8kCYRSuI3es= X-Google-Smtp-Source: APXvYqzql1EshXL6Cl0oupOdonaBxLZarexvB11pbfsfAao4AQATOnKGquTZVZ+nYc7Ash5JBJSRlHGGTzU= X-Received: by 2002:a5d:4283:: with SMTP id k3mr131463wrq.236.1572445423040; Wed, 30 Oct 2019 07:23:43 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:31 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-20-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 19/25] kmsan: disable instrumentation of certain functions From: glider@google.com To: Thomas Gleixner , Andrew Morton , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some functions are called from handwritten assembly, and therefore don't have their arguments' metadata fully set up by the instrumentation code. Mark them with __no_sanitize_memory to avoid false positives from spreading further. Certain functions perform task switching, so that the value of |current| is different as they proceed. Because KMSAN state pointer is only read once at the beginning of the function, touching it after |current| has changed may be dangerous. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Thomas Gleixner Cc: Andrew Morton Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I684d23dac5a22eb0a4cea71993cb934302b17cea --- arch/x86/entry/common.c | 1 + arch/x86/include/asm/irq_regs.h | 1 + arch/x86/include/asm/syscall_wrapper.h | 1 + arch/x86/kernel/apic/apic.c | 2 ++ arch/x86/kernel/dumpstack_64.c | 1 + arch/x86/kernel/process_64.c | 5 +++++ arch/x86/kernel/traps.c | 12 ++++++++++-- arch/x86/kernel/uprobes.c | 7 ++++++- kernel/profile.c | 1 + kernel/sched/core.c | 11 +++++++++++ 10 files changed, 39 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 3f8e22615812..0dd5b2acb355 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -275,6 +275,7 @@ __visible inline void syscall_return_slowpath(struct pt_regs *regs) } #ifdef CONFIG_X86_64 +__no_sanitize_memory __visible void do_syscall_64(unsigned long nr, struct pt_regs *regs) { struct thread_info *ti; diff --git a/arch/x86/include/asm/irq_regs.h b/arch/x86/include/asm/irq_regs.h index 187ce59aea28..d65a00bd6f02 100644 --- a/arch/x86/include/asm/irq_regs.h +++ b/arch/x86/include/asm/irq_regs.h @@ -14,6 +14,7 @@ DECLARE_PER_CPU(struct pt_regs *, irq_regs); +__no_sanitize_memory static inline struct pt_regs *get_irq_regs(void) { return __this_cpu_read(irq_regs); diff --git a/arch/x86/include/asm/syscall_wrapper.h b/arch/x86/include/asm/syscall_wrapper.h index e046a405743d..43910ce1b53b 100644 --- a/arch/x86/include/asm/syscall_wrapper.h +++ b/arch/x86/include/asm/syscall_wrapper.h @@ -159,6 +159,7 @@ ALLOW_ERROR_INJECTION(__x64_sys##name, ERRNO); \ static long __se_sys##name(__MAP(x,__SC_LONG,__VA_ARGS__)); \ static inline long __do_sys##name(__MAP(x,__SC_DECL,__VA_ARGS__));\ + __no_sanitize_memory \ asmlinkage long __x64_sys##name(const struct pt_regs *regs) \ { \ return __se_sys##name(SC_X86_64_REGS_TO_ARGS(x,__VA_ARGS__));\ diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c index 9e2dd2b296cd..43a1edd919ea 100644 --- a/arch/x86/kernel/apic/apic.c +++ b/arch/x86/kernel/apic/apic.c @@ -1118,6 +1118,8 @@ static void local_apic_timer_interrupt(void) * [ if a single-CPU system runs an SMP kernel then we call the local * interrupt as well. Thus we cannot inline the local irq ... ] */ +/* TODO(glider): |regs| is uninitialized, so is |*regs|. */ +__no_sanitize_memory __visible void __irq_entry smp_apic_timer_interrupt(struct pt_regs *regs) { struct pt_regs *old_regs = set_irq_regs(regs); diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c index 753b8cfe8b8a..ba883d282a43 100644 --- a/arch/x86/kernel/dumpstack_64.c +++ b/arch/x86/kernel/dumpstack_64.c @@ -143,6 +143,7 @@ static bool in_irq_stack(unsigned long *stack, struct stack_info *info) return true; } +__no_sanitize_memory int get_stack_info(unsigned long *stack, struct task_struct *task, struct stack_info *info, unsigned long *visit_mask) { diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index af64519b2695..27649ee3dbf8 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -500,6 +500,11 @@ void compat_start_thread(struct pt_regs *regs, u32 new_ip, u32 new_sp) * Kprobes not supported here. Set the probe on schedule instead. * Function graph tracer not supported too. */ +/* + * TODO(glider): __switch_to() does weird things with tasks, don't report + * anything here (also avoid touching the KMSAN state). + */ +__no_sanitize_memory __visible __notrace_funcgraph struct task_struct * __switch_to(struct task_struct *prev_p, struct task_struct *next_p) { diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 4bb0f8447112..a94282d1f60b 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -618,7 +618,10 @@ NOKPROBE_SYMBOL(do_int3); * Help handler running on a per-cpu (IST or entry trampoline) stack * to switch to the normal thread stack if the interrupted code was in * user mode. The actual stack switch is done in entry_64.S + * + * This function switches the registers - don't instrument it with KMSAN! */ +__no_sanitize_memory asmlinkage __visible notrace struct pt_regs *sync_regs(struct pt_regs *eregs) { struct pt_regs *regs = (struct pt_regs *)this_cpu_read(cpu_current_top_of_stack) - 1; @@ -634,6 +637,11 @@ struct bad_iret_stack { }; asmlinkage __visible notrace +/* + * Dark magic happening here, let's not instrument this function. + * Also avoid copying any metadata by using raw __memmove(). + */ +__no_sanitize_memory struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s) { /* @@ -648,10 +656,10 @@ struct bad_iret_stack *fixup_bad_iret(struct bad_iret_stack *s) (struct bad_iret_stack *)this_cpu_read(cpu_tss_rw.x86_tss.sp0) - 1; /* Copy the IRET target to the new stack. */ - memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8); + __memmove(&new_stack->regs.ip, (void *)s->regs.sp, 5*8); /* Copy the remainder of the stack from the current stack. */ - memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip)); + __memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip)); BUG_ON(!user_mode(&new_stack->regs)); return new_stack; diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c index 8cd745ef8c7b..bcd4bf5a909f 100644 --- a/arch/x86/kernel/uprobes.c +++ b/arch/x86/kernel/uprobes.c @@ -8,6 +8,7 @@ * Jim Keniston */ #include +#include #include #include #include @@ -997,9 +998,13 @@ int arch_uprobe_post_xol(struct arch_uprobe *auprobe, struct pt_regs *regs) int arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data) { struct die_args *args = data; - struct pt_regs *regs = args->regs; + struct pt_regs *regs; int ret = NOTIFY_DONE; + kmsan_unpoison_shadow(args, sizeof(*args)); + regs = args->regs; + if (regs) + kmsan_unpoison_shadow(regs, sizeof(*regs)); /* We are only interested in userspace traps */ if (regs && !user_mode(regs)) return NOTIFY_DONE; diff --git a/kernel/profile.c b/kernel/profile.c index af7c94bf5fa1..835a5b66d1a4 100644 --- a/kernel/profile.c +++ b/kernel/profile.c @@ -399,6 +399,7 @@ void profile_hits(int type, void *__pc, unsigned int nr_hits) } EXPORT_SYMBOL_GPL(profile_hits); +__no_sanitize_memory void profile_tick(int type) { struct pt_regs *regs = get_irq_regs(); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index dd05a378631a..951d19d217d7 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -475,6 +475,8 @@ void wake_q_add_safe(struct wake_q_head *head, struct task_struct *task) put_task_struct(task); } +/* TODO(glider): context switching here. */ +__no_sanitize_memory void wake_up_q(struct wake_q_head *head) { struct wake_q_node *node = head->first; @@ -3180,6 +3182,8 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev, * past. prev == current is still correct but we need to recalculate this_rq * because prev may have moved to another CPU. */ +/* TODO(glider): |current| changes here. */ +__no_sanitize_memory static struct rq *finish_task_switch(struct task_struct *prev) __releases(rq->lock) { @@ -3986,6 +3990,8 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) * * WARNING: must be called with preemption disabled! */ +/* TODO(glider): |current| changes here. */ +__no_sanitize_memory static void __sched notrace __schedule(bool preempt) { struct task_struct *prev, *next; @@ -4605,6 +4611,8 @@ int task_prio(const struct task_struct *p) * * Return: 1 if the CPU is currently idle. 0 otherwise. */ +/* TODO(glider): nothing to report here. */ +__no_sanitize_memory int idle_cpu(int cpu) { struct rq *rq = cpu_rq(cpu); @@ -6544,6 +6552,7 @@ static struct kmem_cache *task_group_cache __read_mostly; DECLARE_PER_CPU(cpumask_var_t, load_balance_mask); DECLARE_PER_CPU(cpumask_var_t, select_idle_mask); +__no_sanitize_memory void __init sched_init(void) { unsigned long ptr = 0; @@ -6716,6 +6725,8 @@ static inline int preempt_count_equals(int preempt_offset) return (nested == preempt_offset); } +/* TODO(glider): the args are most certainly initialized. */ +__no_sanitize_memory void __might_sleep(const char *file, int line, int preempt_offset) { /* From patchwork Wed Oct 30 14:22:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219645 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A62331668 for ; Wed, 30 Oct 2019 14:23:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7268D20656 for ; Wed, 30 Oct 2019 14:23:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iFw5x4jE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7268D20656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D5F5B6B0003; Wed, 30 Oct 2019 10:23:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C1EC66B0279; Wed, 30 Oct 2019 10:23:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE6376B027B; Wed, 30 Oct 2019 10:23:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 86F456B0279 for ; Wed, 30 Oct 2019 10:23:48 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 332E4824999B for ; Wed, 30 Oct 2019 14:23:48 +0000 (UTC) X-FDA: 76100669736.07.pets22_3e41da9c06b3e X-Spam-Summary: 2,0,0,57f7382b6d85fe27,d41d8cd98f00b204,38py5xqykcacnspklynvvnsl.jvtspu14-ttr2hjr.vyn@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: pets22_3e41da9c06b3e X-Filterd-Recvd-Size: 4838 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:47 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id n15so679593wmc.3 for ; Wed, 30 Oct 2019 07:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=i0Ie74SM2sNjD7EzrzgEhVZsFkM1WnCABTXFFFbssjQ=; b=iFw5x4jEIXJpnPCObOp9edi0dyNJiDmJvPifiAbS2Iqg9E2Jtv0rSthNxerRIfaGvB MixCUaYd1S+eGC/3QGBaDFT8m0I5Rb6bEnyF9i8mSKWZk+8WOuBpjhDLOVue5iqVfVNB U4lhLBnT7MPyfTJwAjGwE6ZE2U2VfPgd6aJT4i0BN7Vh+hR2qQJ+Ds8/uRbTTTO8fqo4 kD1B0DhGdMy5k1/4p4OcIM4C7AnEudaq5wMG1MHtRa1KN6tpA7C1qXS4T9fzAgeFIHj5 Rn9QswCIwHx2tRDCG3W/PbOK2foCrQTd1WMX+KbG5dcv2Dn6zy7GmIzqFj0BvuWbazJp lVLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=i0Ie74SM2sNjD7EzrzgEhVZsFkM1WnCABTXFFFbssjQ=; b=pdW1OrFc1yuiaPVvU9grGlMOdrNM0MpuYMZrz6mD/NZulIUH1scexIJhVgSb6/JbmX XDZ1clo06HqQ60rG+/I7bi0NtY6iGEevkbUkeJbFGevly+sGWahtHGC6hIi5DR6d9Jc2 5Y7bBFaCiLhvfTWx5ku8zf7Z35wPz3rvfUfk2dec1pBGDfcbdkKcByhB9xVOpjV+PLxc p/247aoM8Z/8vpT2v6pl8sDHwdJuRHJpWyD+xrr8SLkLtx9rb8nqkSj9XtHT03R28KIf k3bS4pIWwVdoH4np52L4ItTbH24zxenBZVwn4CB5dMb/HQeZTL7HiZp8SHDRyohAbz6p +iHw== X-Gm-Message-State: APjAAAVvYYdH0zju+UgMn4DnfAFYa1kNzYOleZwFgrbVKeBp48VwEuqZ aWmQdJDgQmgILZTyzbfmEbTkCIEtoHQ= X-Google-Smtp-Source: APXvYqwwTRTjwGk13hTEDvRXOJxRzzHPyaLs8Lvew0WBVDXcPCCJRdF8xGK4/GFNZfluDlaXizFbiKvBC2s= X-Received: by 2002:adf:ee03:: with SMTP id y3mr141216wrn.116.1572445426114; Wed, 30 Oct 2019 07:23:46 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:32 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-21-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 20/25] kmsan: unpoison |tlb| in arch_tlb_gather_mmu() From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is a hack to reduce stackdepot pressure. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I22a201e7e4f67ed74f8129072f12e5351b26103a --- mm/mmu_gather.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c index 7d70e5c78f97..8c5ea2d2e7d5 100644 --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -1,6 +1,7 @@ #include #include #include +#include #include #include #include @@ -206,6 +207,15 @@ void tlb_flush_mmu(struct mmu_gather *tlb) void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, unsigned long start, unsigned long end) { + /* + * TODO(glider): struct mmu_gather contains 7 1-bit fields packed into a + * 32-bit unsigned int value. The remaining 25 bits remain uninitialized + * and are never used, but KMSAN updates the origin for them in + * zap_pXX_range() in mm/memory.c, thus creating very long origin + * chains. This is technically correct, but consumes too much memory. + * Unpoisoning the whole structure will prevent creating such chains. + */ + kmsan_unpoison_shadow(tlb, sizeof(*tlb)); tlb->mm = mm; /* Is it from 0 to ~0? */ From patchwork Wed Oct 30 14:22:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219647 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DFEE51668 for ; Wed, 30 Oct 2019 14:24:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9E3EF2087E for ; Wed, 30 Oct 2019 14:24:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JVF10oD2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E3EF2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D0A7C6B0006; Wed, 30 Oct 2019 10:23:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C93886B0284; Wed, 30 Oct 2019 10:23:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0D5B6B0285; Wed, 30 Oct 2019 10:23:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0186.hostedemail.com [216.40.44.186]) by kanga.kvack.org (Postfix) with ESMTP id 85ACC6B0006 for ; Wed, 30 Oct 2019 10:23:51 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 2C2A2180AD83A for ; Wed, 30 Oct 2019 14:23:51 +0000 (UTC) X-FDA: 76100669862.28.aunt41_3eaea32feaf32 X-Spam-Summary: 2,0,0,8fbeca7167404135,d41d8cd98f00b204,39zy5xqykcaoqvsno1qyyqvo.mywvsx47-wwu5kmu.y1q@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: aunt41_3eaea32feaf32 X-Filterd-Recvd-Size: 6690 Received: from mail-vk1-f202.google.com (mail-vk1-f202.google.com [209.85.221.202]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:50 +0000 (UTC) Received: by mail-vk1-f202.google.com with SMTP id a188so983058vkc.7 for ; Wed, 30 Oct 2019 07:23:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=2Vd4w0UYILTts21BfSGtWCVgdj+rkFRA9ehXw906emw=; b=JVF10oD2qnr/thY+y6Y8aue3Kqz2kyscSfvG0hUWAGTWoNCue5lm85pHNIOsLdJLao JPCoqVf9DpbzvsNWHYNT+N7SkYUvZEM4jGd2R+wKhVLRqtC5cpgQRRjl4T7Lyv2IFFi9 DlkB23z2dISPlo6ZTf9VKCC3L9jVHhTqJyylIXani8NqlO7azuR6YMtYxHqHjMnQG3dj r383kzGb3Jjhlm/3sQWxfGJ91NG7RNJlhKL5m8biP6zaHiZXg4TbVAUNanP7oGpZUBGz sjsKZo7GRqqrXfVbKFeFNGJWQ7JcNy7uj+deGRLmxDqfFm+vPmrEuPzZRDC4PpGl7lC0 0p4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2Vd4w0UYILTts21BfSGtWCVgdj+rkFRA9ehXw906emw=; b=GUiyaCz7MtfqRUcwJ7gfYmxEOp5YsPpjVfPRJXJz6C2DXbj1UH/tsSkI/6ygCSCrGY 0YasDnpX7DUbxA5EMIFqTQCeELptnB37fA4bmltHlsYZb5z29kH8DgeDodZtS0qMwoVi /gcjsvNPJ/EjWa/oPg9+obLpT8bxC6j1/thPLB/QntOJ1684OtCP4VQjIaXZKQXuTzWy agErTGnEm2IfHijrS2Gsci0QDZYnj/QUvesat3DpvGTSy0jf7RPVAV+oCCCI+M2HnheA KXNsGwi9ZAeAcjA4By5SKGx+kwRAyooyTMIJYCz5AuHizMr8qvZj1xj37BWDpTgu65A9 1L1Q== X-Gm-Message-State: APjAAAUKF57KO2cSB6qExLSdMyMINC7pKQBekL4KbjorlL7uSmhtHqTH a7SZQfO5+nGHXFm/q+9s8maV6btaWTM= X-Google-Smtp-Source: APXvYqwk/LlRXDyhW1sP49hhsXZyvlnmjuCQNpsv+1z+N3CsLIMaz7THDfgdNfgHf7kxqnd4/PDNKm3dlpA= X-Received: by 2002:a1f:1897:: with SMTP id 145mr4780581vky.47.1572445429651; Wed, 30 Oct 2019 07:23:49 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:33 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-22-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 21/25] kmsan: use __msan_memcpy() where possible. From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Unless stated otherwise (by explicitly calling __memcpy()) we want all memcpy() calls to call __msan_memcpy() so that shadow and origin values are updated accordingly. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: Ib2512ce5aa8d457453dd38caa12f58f002166813 --- arch/x86/include/asm/string_64.h | 9 ++++++++- include/linux/compiler.h | 9 ++++++++- include/linux/string.h | 2 ++ 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/string_64.h b/arch/x86/include/asm/string_64.h index 75314c3dbe47..d3c76d910c23 100644 --- a/arch/x86/include/asm/string_64.h +++ b/arch/x86/include/asm/string_64.h @@ -11,7 +11,13 @@ function. */ #define __HAVE_ARCH_MEMCPY 1 +#if defined(CONFIG_KMSAN) +#undef memcpy +/* __msan_memcpy() is defined in compiler.h */ +#define memcpy(dst, src, len) __msan_memcpy(dst, src, len) +#else extern void *memcpy(void *to, const void *from, size_t len); +#endif extern void *__memcpy(void *to, const void *from, size_t len); #define __HAVE_ARCH_MEMSET @@ -64,7 +70,8 @@ char *strcpy(char *dest, const char *src); char *strcat(char *dest, const char *src); int strcmp(const char *cs, const char *ct); -#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__) +#if (defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)) || \ + (defined(CONFIG_KMSAN) && !defined(__SANITIZE_MEMORY__)) /* * For files that not instrumented (e.g. mm/slub.c) we diff --git a/include/linux/compiler.h b/include/linux/compiler.h index e8c86debdb2b..d0bc367e9164 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -179,6 +179,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, #include +#ifdef CONFIG_KMSAN +void *__msan_memcpy(void *dst, const void *src, u64 size); +#define __DO_MEMCPY(res, p, size) __msan_memcpy(res, p, size) +#else +#define __DO_MEMCPY(res, p, size) __builtin_memcpy(res, p, size) +#endif + #define __READ_ONCE_SIZE \ ({ \ switch (size) { \ @@ -188,7 +195,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, case 8: *(__u64 *)res = *(volatile __u64 *)p; break; \ default: \ barrier(); \ - __builtin_memcpy((void *)res, (const void *)p, size); \ + __DO_MEMCPY((void *)res, (const void *)p, size); \ barrier(); \ } \ }) diff --git a/include/linux/string.h b/include/linux/string.h index b6ccdc2c7f02..5d8ce09cba2e 100644 --- a/include/linux/string.h +++ b/include/linux/string.h @@ -363,6 +363,7 @@ __FORTIFY_INLINE void *memset(void *p, int c, __kernel_size_t size) return __builtin_memset(p, c, size); } +#ifndef CONFIG_KMSAN __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size) { size_t p_size = __builtin_object_size(p, 0); @@ -377,6 +378,7 @@ __FORTIFY_INLINE void *memcpy(void *p, const void *q, __kernel_size_t size) fortify_panic(__func__); return __builtin_memcpy(p, q, size); } +#endif __FORTIFY_INLINE void *memmove(void *p, const void *q, __kernel_size_t size) { From patchwork Wed Oct 30 14:22:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219649 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A22E112B for ; Wed, 30 Oct 2019 14:24:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3C1352087E for ; Wed, 30 Oct 2019 14:24:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MyV1qrIq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C1352087E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C74816B0284; Wed, 30 Oct 2019 10:23:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BD7146B0286; Wed, 30 Oct 2019 10:23:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4EED6B0287; Wed, 30 Oct 2019 10:23:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0052.hostedemail.com [216.40.44.52]) by kanga.kvack.org (Postfix) with ESMTP id 6F75C6B0284 for ; Wed, 30 Oct 2019 10:23:55 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 1E47C181AF5CA for ; Wed, 30 Oct 2019 14:23:55 +0000 (UTC) X-FDA: 76100670030.29.crowd02_3f384ae44d918 X-Spam-Summary: 2,0,0,a60ac088bd6d8778,d41d8cd98f00b204,3-jy5xqykca0tyvqr4t11tyr.p1zyv07a-zzx8npx.14t@flex--glider.bounces.google.com,:akpm@linux-foundation.org:axboe@kernel.dk:tytso@mit.edu:dmitry.torokhov@gmail.com:martin.petersen@oracle.com:mst@redhat.com:hch@lst.de:edumazet@google.com:ericvh@gmail.com:tiwai@suse.com:vegard.nossum@oracle.com:dvyukov@google.com:willy@infradead.org::viro@zeniv.linux.org.uk:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:schwidefsky@de.ibm.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:4:41:152:355:371:372:379:541:800:960:966:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605: 1730:174 X-HE-Tag: crowd02_3f384ae44d918 X-Filterd-Recvd-Size: 17717 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:54 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id 14so678088wmk.4 for ; Wed, 30 Oct 2019 07:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=IavD1+EX+I7T0EwpbC93cXpZGYX8OPnszZrjBjNNMHU=; b=MyV1qrIqIEnLApiqnTSeQG2mtCnWwWwNRSwuaueWHZaBnyi3357xtKCbMd+L6HrpCv QAHfMOOcTm4trVEW/19PGNNXcGpGitZfW0ahEXf7ewiwYVtv5wgjcZCuZtfk74hb0F8l ERPFipu3/9iQ2NYmzbfDPzEY+0SEb418OnRi5T4T66yOHdVAU7YdnsC1tzHL//w76BiV LkTMxKLUp1LkANFa0iJGpNQukCGpDWeEzcrl4bV4aRuFPbBVt0ZrOUquHa1O70H70pY+ 8nxva/L84PPBR5ISbwAGFiym6OY5KAEUqirkWeMERTGtfxCliGCVjJY3Nxuq1XM35g1z /kXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=IavD1+EX+I7T0EwpbC93cXpZGYX8OPnszZrjBjNNMHU=; b=O7QGgWn5lwzFwOHnk2HUtF2y1IrUYVSi9Grv5U5OJ5aAX74TZsc3qWCkYpg0RT5WOF 6Bgv/PZMYXRRUPgTxdrdSPU7mt4YyAmCnYNsaIibSN20IskjLCPZ06EgRnO9rDwpC6pl vlj+UCKCsOMoYyIker19jgE/LTT+3B38+5OkNSo1xMsyAOTGFk2X7BFqICXrWhnNv4+w P1OSFsm+zhSfBW5F7UPgkGsTLqnHu6EEZIhWvnyAGWrcK2ODZ1M6C/JlPT9EOu9KfUfP mbi+LyCcbsjhMH/4wDMPQ8GTF7wTUm5xxQbveACpiEs6DmfZVC1cPkMyPKZrNJVXodKJ WSqg== X-Gm-Message-State: APjAAAVmoADvvs6zEVJrDOqEa8JTWnl5mcuBzWrS6rklnG/TGs2n0y2H nsffzTV/wsfZ/D5pLdbzVY0jGhJhzTc= X-Google-Smtp-Source: APXvYqyt73dIX9ev53FXV4TgOm/Db+XnpAVqZ1JnWnJ/N7VoejI/5wPjBHAA+C6vOl3MNQ7kHqRyTBQ1HPc= X-Received: by 2002:adf:e74c:: with SMTP id c12mr105526wrn.133.1572445432829; Wed, 30 Oct 2019 07:23:52 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:34 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-23-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 22/25] kmsan: unpoisoning buffers from devices etc. From: glider@google.com To: Andrew Morton , Jens Axboe , "Theodore Ts'o" , Dmitry Torokhov , "Martin K. Petersen" , "Michael S. Tsirkin" , Christoph Hellwig , Eric Dumazet , Eric Van Hensbergen , Takashi Iwai , Vegard Nossum , Dmitry Vyukov , Matthew Wilcox , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, schwidefsky@de.ibm.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When data is copied to memory from a device KMSAN should treat it as initialized. In most cases it's enough to just unpoison the buffer that is known to come from a device. In the case with __do_page_cache_readahead() and bio_copy_user_iov() we have to mark the whole pages as ignored by KMSAN, as it's not obvious where these pages are read again. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Andrew Morton Cc: Jens Axboe Cc: "Theodore Ts'o" Cc: Dmitry Torokhov Cc: Martin K. Petersen Cc: "Michael S. Tsirkin" Cc: Christoph Hellwig Cc: Eric Dumazet Cc: Eric Van Hensbergen Cc: Takashi Iwai Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Matthew Wilcox Cc: linux-mm@kvack.org --- Suggestions on how to avoid scattering too many annotations over the code are welcome. Perhaps annotating DMA reads/writes is enough to get rid of most of these annotations. v2: - dropped a call to kmsan_unpoison_shadow() in do_read_cache_page() Change-Id: Id460e7a86ce564f1357469f53d0c7410ca08f0e9 --- block/bio.c | 20 ++++++++++++++++++++ block/partition-generic.c | 9 ++++++++- drivers/char/random.c | 2 ++ drivers/input/serio/libps2.c | 6 +++++- drivers/scsi/scsi_lib.c | 4 ++++ drivers/usb/core/message.c | 6 +++++- drivers/virtio/virtio_ring.c | 14 ++++++++++++++ fs/buffer.c | 7 ++++++- include/linux/dma-mapping.h | 2 ++ include/linux/skbuff.h | 5 ++++- mm/filemap.c | 1 + mm/readahead.c | 6 ++++++ net/9p/protocol.c | 2 ++ sound/core/oss/pcm_oss.c | 7 +++++++ 14 files changed, 86 insertions(+), 5 deletions(-) diff --git a/block/bio.c b/block/bio.c index 8f0ed6228fc5..40773cf4b50c 100644 --- a/block/bio.c +++ b/block/bio.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -900,6 +901,13 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) __bio_add_page(bio, page, len, offset); } offset = 0; + /* + * TODO(glider): these pages will be soon passed to block + * device for reading, so consider them initialized. + */ + if (iov_iter_rw(iter) == READ) + kmsan_unpoison_shadow(page_address(page), + PAGE_SIZE); } iov_iter_advance(iter, size); @@ -1274,11 +1282,16 @@ struct bio *bio_copy_user_iov(struct request_queue *q, i++; } else { + /* + * TODO(glider): KMSAN doesn't track the pages + * allocated for bio here. + */ page = alloc_page(q->bounce_gfp | gfp_mask); if (!page) { ret = -ENOMEM; break; } + kmsan_ignore_page(page, /*order*/0); } if (bio_add_pc_page(q, bio, page, bytes, offset) < bytes) { @@ -1574,6 +1587,13 @@ struct bio *bio_copy_kern(struct request_queue *q, void *data, unsigned int len, if (!page) goto cleanup; + /* + * TODO(glider): if we're about to read data from a SCSI device, + * assume the page allocated for that is already initialized. + */ + if (reading) + kmsan_unpoison_shadow(page_address(page), PAGE_SIZE); + if (!reading) memcpy(page_address(page), p, bytes); diff --git a/block/partition-generic.c b/block/partition-generic.c index aee643ce13d1..87b3ae9e7727 100644 --- a/block/partition-generic.c +++ b/block/partition-generic.c @@ -663,13 +663,20 @@ unsigned char *read_dev_sector(struct block_device *bdev, sector_t n, Sector *p) { struct address_space *mapping = bdev->bd_inode->i_mapping; struct page *page; + unsigned char *retval; page = read_mapping_page(mapping, (pgoff_t)(n >> (PAGE_SHIFT-9)), NULL); if (!IS_ERR(page)) { if (PageError(page)) goto fail; p->v = page; - return (unsigned char *)page_address(page) + ((n & ((1 << (PAGE_SHIFT - 9)) - 1)) << 9); + retval = (unsigned char *)page_address(page) + + ((n & ((1 << (PAGE_SHIFT - 9)) - 1)) << 9); + /* + * Unpoison sector-sized chunk of memory coming from the device. + */ + kmsan_unpoison_shadow(retval, SECTOR_SIZE); + return retval; fail: put_page(page); } diff --git a/drivers/char/random.c b/drivers/char/random.c index de434feb873a..cc4afc0f1039 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -320,6 +320,7 @@ #include #include #include +#include #include #include #include @@ -1061,6 +1062,7 @@ static void _extract_crng(struct crng_state *crng, spin_lock_irqsave(&crng->lock, flags); if (arch_get_random_long(&v)) crng->state[14] ^= v; + kmsan_unpoison_shadow(crng->state, sizeof(crng->state)); chacha20_block(&crng->state[0], out); if (crng->state[12] == 0) crng->state[13]++; diff --git a/drivers/input/serio/libps2.c b/drivers/input/serio/libps2.c index a8c94a940a79..80a9e0a9d3c3 100644 --- a/drivers/input/serio/libps2.c +++ b/drivers/input/serio/libps2.c @@ -8,6 +8,7 @@ #include +#include #include #include #include @@ -30,6 +31,7 @@ static int ps2_do_sendbyte(struct ps2dev *ps2dev, u8 byte, int error; lockdep_assert_held(&ps2dev->serio->lock); + kmsan_check_memory(&byte, 1); do { ps2dev->nak = 1; @@ -294,9 +296,11 @@ int __ps2_command(struct ps2dev *ps2dev, u8 *param, unsigned int command) serio_pause_rx(ps2dev->serio); - if (param) + if (param) { for (i = 0; i < receive; i++) param[i] = ps2dev->cmdbuf[(receive - 1) - i]; + kmsan_unpoison_shadow(param, receive); + } if (ps2dev->cmdcnt && (command != PS2_CMD_RESET_BAT || ps2dev->cmdcnt != 1)) { diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 5447738906ac..a6d6a0eed8ec 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -296,6 +297,9 @@ int __scsi_execute(struct scsi_device *sdev, const unsigned char *cmd, ret = rq->result; out: blk_put_request(req); + /* TODO(glider): this is a bit rough. */ + if (data_direction == DMA_FROM_DEVICE) + kmsan_unpoison_shadow(buffer, bufflen); return ret; } diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c index 5adf489428aa..f1c3f8b3cfae 100644 --- a/drivers/usb/core/message.c +++ b/drivers/usb/core/message.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -101,8 +102,11 @@ static int usb_internal_control_msg(struct usb_device *usb_dev, retv = usb_start_wait_urb(urb, timeout, &length); if (retv < 0) return retv; - else + else { + /* TODO(glider): USB initializes |length| bytes? */ + kmsan_unpoison_shadow(data, length); return length; + } } /** diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index a8041e451e9e..19d35744e3e1 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #ifdef DEBUG @@ -387,6 +388,12 @@ static void vring_unmap_one_split(const struct vring_virtqueue *vq, (flags & VRING_DESC_F_WRITE) ? DMA_FROM_DEVICE : DMA_TO_DEVICE); } + /* + * Unmapping DMA memory after a transfer from device requires this + * memory to be unpoisoned. + */ + if (flags & VRING_DESC_F_WRITE) + kmsan_unpoison_shadow((const void *)desc->addr, desc->len); } static struct vring_desc *alloc_indirect_split(struct virtqueue *_vq, @@ -500,6 +507,13 @@ static inline int virtqueue_add_split(struct virtqueue *_vq, desc[i].flags = cpu_to_virtio16(_vq->vdev, VRING_DESC_F_NEXT | VRING_DESC_F_WRITE); desc[i].addr = cpu_to_virtio64(_vq->vdev, addr); desc[i].len = cpu_to_virtio32(_vq->vdev, sg->length); + /* + * It's hard to figure out the buffer's address upon + * receive. Instead we unpoison it once, when exposing + * it to the device, and hope nobody else will write to + * it. + */ + kmsan_unpoison_shadow(sg_virt(sg), sg->length); prev = i; i = virtio16_to_cpu(_vq->vdev, desc[i].next); } diff --git a/fs/buffer.c b/fs/buffer.c index 86a38b979323..c08fb1a77fa1 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -1170,14 +1170,17 @@ static struct buffer_head *__bread_slow(struct buffer_head *bh) lock_buffer(bh); if (buffer_uptodate(bh)) { unlock_buffer(bh); + kmsan_unpoison_shadow(bh->b_data, bh->b_size); return bh; } else { get_bh(bh); bh->b_end_io = end_buffer_read_sync; submit_bh(REQ_OP_READ, 0, bh); wait_on_buffer(bh); - if (buffer_uptodate(bh)) + if (buffer_uptodate(bh)) { + kmsan_unpoison_shadow(bh->b_data, bh->b_size); return bh; + } } brelse(bh); return NULL; @@ -1320,6 +1323,8 @@ __getblk_gfp(struct block_device *bdev, sector_t block, might_sleep(); if (bh == NULL) bh = __getblk_slow(bdev, block, size, gfp); + if (bh) + kmsan_unpoison_shadow(bh->b_data, bh->b_size); return bh; } EXPORT_SYMBOL(__getblk_gfp); diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 4a1c4fca475a..cc7b05eb024f 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -281,6 +282,7 @@ static inline dma_addr_t dma_map_page_attrs(struct device *dev, const struct dma_map_ops *ops = get_dma_ops(dev); dma_addr_t addr; + kmsan_unpoison_shadow(page_address(page), size); BUG_ON(!valid_dma_direction(dir)); if (dma_is_direct(ops)) addr = dma_direct_map_page(dev, page, offset, size, dir, attrs); diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 7914fdaf4226..1efe136250c9 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -11,6 +11,7 @@ #define _LINUX_SKBUFF_H #include +#include #include #include #include @@ -2214,6 +2215,8 @@ static inline void *skb_put_data(struct sk_buff *skb, const void *data, { void *tmp = skb_put(skb, len); + /* Unpoison the data received from the network device. */ + kmsan_unpoison_shadow(data, len); memcpy(tmp, data, len); return tmp; @@ -2221,7 +2224,7 @@ static inline void *skb_put_data(struct sk_buff *skb, const void *data, static inline void skb_put_u8(struct sk_buff *skb, u8 val) { - *(u8 *)skb_put(skb, 1) = val; + *(u8 *)skb_put(skb, 1) = KMSAN_INIT_VALUE(val); } void *skb_push(struct sk_buff *skb, unsigned int len); diff --git a/mm/filemap.c b/mm/filemap.c index 85b7d087eb45..05b930e9f484 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include diff --git a/mm/readahead.c b/mm/readahead.c index 2fe72cd29b47..1fe61128095b 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -193,7 +193,13 @@ unsigned int __do_page_cache_readahead(struct address_space *mapping, continue; } + /* + * The easiest way to handle these pages is to mark them + * untracked by KMSAN, assuming they're never used by anything + * else. + */ page = __page_cache_alloc(gfp_mask); + kmsan_ignore_page(page, /*order*/0); if (!page) break; page->index = page_offset; diff --git a/net/9p/protocol.c b/net/9p/protocol.c index 03593eb240d8..5d88ed181422 100644 --- a/net/9p/protocol.c +++ b/net/9p/protocol.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -46,6 +47,7 @@ EXPORT_SYMBOL(p9stat_free); size_t pdu_read(struct p9_fcall *pdu, void *data, size_t size) { size_t len = min(pdu->size - pdu->offset, size); + kmsan_unpoison_shadow(&pdu->sdata[pdu->offset], len); memcpy(data, &pdu->sdata[pdu->offset], len); pdu->offset += len; return size - len; diff --git a/sound/core/oss/pcm_oss.c b/sound/core/oss/pcm_oss.c index f57c610d7523..c3795e495196 100644 --- a/sound/core/oss/pcm_oss.c +++ b/sound/core/oss/pcm_oss.c @@ -12,6 +12,7 @@ #endif #include +#include /* for kmsan_unpoison_shadow() */ #include #include #include @@ -1054,6 +1055,12 @@ static int snd_pcm_oss_change_params_locked(struct snd_pcm_substream *substream) err = -ENOMEM; goto failure; } + /* + * Unpoison the freshly created buffer to prevent KMSAN from reporting + * uninit errors. + * TODO(glider): unpoison it only when it's actually initialized. + */ + kmsan_unpoison_shadow(runtime->oss.buffer, runtime->oss.period_bytes); runtime->oss.params = 0; runtime->oss.prepare = 1; From patchwork Wed Oct 30 14:22:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 813701668 for ; Wed, 30 Oct 2019 14:24:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 31D3420656 for ; Wed, 30 Oct 2019 14:24:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fByZ06Zd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 31D3420656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 03AFB6B0286; Wed, 30 Oct 2019 10:23:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F08A46B0288; Wed, 30 Oct 2019 10:23:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D66546B0289; Wed, 30 Oct 2019 10:23:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id A50A76B0286 for ; Wed, 30 Oct 2019 10:23:58 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 54610282A for ; Wed, 30 Oct 2019 14:23:58 +0000 (UTC) X-FDA: 76100670156.15.name18_3fadffc48cb10 X-Spam-Summary: 2,0,0,994159b97af9e968,d41d8cd98f00b204,3_jy5xqykcbex2zuv8x55x2v.t532z4be-331crt1.58x@flex--glider.bounces.google.com,:viro@zeniv.linux.org.uk:vegard.nossum@oracle.com:dvyukov@google.com::akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:1:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:17 77:1792: X-HE-Tag: name18_3fadffc48cb10 X-Filterd-Recvd-Size: 13623 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:23:57 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id c6so1423547wrp.3 for ; Wed, 30 Oct 2019 07:23:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=SaCxedWrWmYuu3heks9wmzJiGaxK2x9+HevM/+wrMlg=; b=fByZ06ZdmpBPPyKDS93WRP7wtYRjCUF3BTLO0dUzzFfkSMt1/GTFclHIO1oxa5m8I0 xSzOVolux65lqa0hvUNqNC+Kh9lI3cdt33sII2hbYqaWMCd38qiWBO1d+g61YNr3x3Eq ps5ZPGIS5B8yFTbAQJcKsnPNpzcQdPFEleB1LCJi8pebnPnBq9p/A/L4q2CcwRM5G2kt WS3k+YgKe4g21JsKBxlqPN5FDqC2K2moIfgYZlzBfRxTlWcTRo8C9cW7gqMFENOtUIgf LrPDBCAKUAFvedFYw9IGPTgYnA56HTqiVJKq2Zx1FBOHcitedjNmzvDw1oySa65VHMeX bpcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SaCxedWrWmYuu3heks9wmzJiGaxK2x9+HevM/+wrMlg=; b=JHo53m9Oftuu8m6+qOtmA+vQMZhShFBTawZAKiMSn87xYP6sPvZ9G6S9WdyGFXxW7A 4o8IXD2QKwQ+fhzo+dFTeify5olB/+F25MQznRY6rroATgWuBcXc2VFHqhIKRTyZCJu1 p5Hy57bC2NkuVltxGfYX4Xnan05DOmWl5KqXTypW8WVvtVlveF2NPs5M2+4W7SKp9DxY 6PJJyPps6mcGp40zyN3HOxMjE3BQSDEZKcqvyvETDu728eTtdf7f5FSg02TGJVoPbsK4 xMPriV8gV/7QBFjvDj8QKOR/x1zizBxOnSzxh7f67tIgFZi2jw9U6sdP4XhxbZI3VEV0 YZLg== X-Gm-Message-State: APjAAAWRnx15gtM2FS+jSKmeS2zgdUkDsa65TCKt8e5EOLxs3T/tTx1m 4cIuYJ7XMayjVRNly5pdfaQ5GpbIIwk= X-Google-Smtp-Source: APXvYqxCysyUV+v2+9MK8wdK9rrokFHhjaLVxCcPjJwbfYuuLvYgX1shJOL4CwRWAHZBO79i/X0CfPkwm10= X-Received: by 2002:adf:ea8d:: with SMTP id s13mr120277wrm.366.1572445436019; Wed, 30 Oct 2019 07:23:56 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:35 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-24-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 23/25] kmsan: hooks for copy_to_user() and friends From: glider@google.com To: Alexander Viro , Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Memory that is copied from userspace must be unpoisoned. Before copying memory to userspace, check it and report an error if it contains uninitialized bits. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Alexander Viro Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I38428b9c7d1909b8441dcec1749b080494a7af99 --- arch/x86/include/asm/uaccess.h | 12 ++++++++++++ include/asm-generic/cacheflush.h | 7 ++++++- include/asm-generic/uaccess.h | 12 ++++++++++-- include/linux/uaccess.h | 32 +++++++++++++++++++++++++++----- lib/iov_iter.c | 6 ++++++ lib/usercopy.c | 6 +++++- 6 files changed, 66 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h index 61d93f062a36..ac4b26583f7c 100644 --- a/arch/x86/include/asm/uaccess.h +++ b/arch/x86/include/asm/uaccess.h @@ -6,6 +6,7 @@ */ #include #include +#include #include #include #include @@ -174,6 +175,7 @@ __typeof__(__builtin_choose_expr(sizeof(x) > sizeof(0UL), 0ULL, 0UL)) ASM_CALL_CONSTRAINT \ : "0" (ptr), "i" (sizeof(*(ptr)))); \ (x) = (__force __typeof__(*(ptr))) __val_gu; \ + kmsan_unpoison_shadow(&(x), sizeof(*(ptr))); \ __builtin_expect(__ret_gu, 0); \ }) @@ -248,6 +250,7 @@ extern void __put_user_8(void); __chk_user_ptr(ptr); \ might_fault(); \ __pu_val = x; \ + kmsan_check_memory(&(__pu_val), sizeof(*(ptr))); \ switch (sizeof(*(ptr))) { \ case 1: \ __put_user_x(1, __pu_val, ptr, __ret_pu); \ @@ -270,7 +273,9 @@ extern void __put_user_8(void); #define __put_user_size(x, ptr, size, label) \ do { \ + __typeof__(*(ptr)) __pus_val = x; \ __chk_user_ptr(ptr); \ + kmsan_check_memory(&(__pus_val), size); \ switch (size) { \ case 1: \ __put_user_goto(x, ptr, "b", "b", "iq", label); \ @@ -295,7 +300,10 @@ do { \ */ #define __put_user_size_ex(x, ptr, size) \ do { \ + __typeof__(*(ptr)) __puse_val; \ __chk_user_ptr(ptr); \ + __puse_val = x; \ + kmsan_check_memory(&(__puse_val), size); \ switch (size) { \ case 1: \ __put_user_asm_ex(x, ptr, "b", "b", "iq"); \ @@ -363,6 +371,7 @@ do { \ default: \ (x) = __get_user_bad(); \ } \ + kmsan_unpoison_shadow(&(x), size); \ } while (0) #define __get_user_asm(x, addr, err, itype, rtype, ltype, errret) \ @@ -413,6 +422,7 @@ do { \ default: \ (x) = __get_user_bad(); \ } \ + kmsan_unpoison_shadow(&(x), size); \ } while (0) #define __get_user_asm_ex(x, addr, itype, rtype, ltype) \ @@ -428,11 +438,13 @@ do { \ #define __put_user_nocheck(x, ptr, size) \ ({ \ __label__ __pu_label; \ + __typeof__(*(ptr)) __pun_val = x; \ int __pu_err = -EFAULT; \ __typeof__(*(ptr)) __pu_val = (x); \ __typeof__(ptr) __pu_ptr = (ptr); \ __typeof__(size) __pu_size = (size); \ __uaccess_begin(); \ + kmsan_check_memory(&(__pun_val), size); \ __put_user_size(__pu_val, __pu_ptr, __pu_size, __pu_label); \ __pu_err = 0; \ __pu_label: \ diff --git a/include/asm-generic/cacheflush.h b/include/asm-generic/cacheflush.h index a950a22c4890..707531dccf5e 100644 --- a/include/asm-generic/cacheflush.h +++ b/include/asm-generic/cacheflush.h @@ -4,6 +4,7 @@ /* Keep includes the same across arches. */ #include +#include #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0 @@ -72,10 +73,14 @@ static inline void flush_cache_vunmap(unsigned long start, unsigned long end) #define copy_to_user_page(vma, page, vaddr, dst, src, len) \ do { \ + kmsan_check_memory(src, len); \ memcpy(dst, src, len); \ flush_icache_user_range(vma, page, vaddr, len); \ } while (0) #define copy_from_user_page(vma, page, vaddr, dst, src, len) \ - memcpy(dst, src, len) + do { \ + memcpy(dst, src, len); \ + kmsan_unpoison_shadow(dst, len); \ + } while (0) #endif /* __ASM_CACHEFLUSH_H */ diff --git a/include/asm-generic/uaccess.h b/include/asm-generic/uaccess.h index e935318804f8..18a50333ffc0 100644 --- a/include/asm-generic/uaccess.h +++ b/include/asm-generic/uaccess.h @@ -142,7 +142,11 @@ static inline int __access_ok(unsigned long addr, unsigned long size) static inline int __put_user_fn(size_t size, void __user *ptr, void *x) { - return unlikely(raw_copy_to_user(ptr, x, size)) ? -EFAULT : 0; + int n; + + n = raw_copy_to_user(ptr, x, size); + kmsan_copy_to_user(ptr, x, size, n); + return unlikely(n) ? -EFAULT : 0; } #define __put_user_fn(sz, u, k) __put_user_fn(sz, u, k) @@ -203,7 +207,11 @@ extern int __put_user_bad(void) __attribute__((noreturn)); #ifndef __get_user_fn static inline int __get_user_fn(size_t size, const void __user *ptr, void *x) { - return unlikely(raw_copy_from_user(x, ptr, size)) ? -EFAULT : 0; + int copied, to_copy = size; + + copied = raw_copy_from_user(x, ptr, size); + kmsan_unpoison_shadow(to, to_copy - res); + return unlikely(copied) ? -EFAULT : 0; } #define __get_user_fn(sz, u, k) __get_user_fn(sz, u, k) diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index d4ee6e942562..4257b5f80689 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -5,6 +5,7 @@ #include #include #include +#include #define uaccess_kernel() segment_eq(get_fs(), KERNEL_DS) @@ -58,18 +59,26 @@ static __always_inline __must_check unsigned long __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n) { + unsigned long to_copy = n; + kasan_check_write(to, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + n = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, to_copy - n); + return n; } static __always_inline __must_check unsigned long __copy_from_user(void *to, const void __user *from, unsigned long n) { + unsigned long to_copy = n; + might_fault(); kasan_check_write(to, n); check_object_size(to, n, false); - return raw_copy_from_user(to, from, n); + n = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, to_copy - n); + return n; } /** @@ -88,29 +97,39 @@ __copy_from_user(void *to, const void __user *from, unsigned long n) static __always_inline __must_check unsigned long __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n) { + unsigned long to_copy = n; + kasan_check_read(from, n); check_object_size(from, n, true); - return raw_copy_to_user(to, from, n); + n = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, to_copy, n); + return n; } static __always_inline __must_check unsigned long __copy_to_user(void __user *to, const void *from, unsigned long n) { + unsigned long to_copy = n; + might_fault(); kasan_check_read(from, n); check_object_size(from, n, true); - return raw_copy_to_user(to, from, n); + n = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, to_copy, n); + return n; } #ifdef INLINE_COPY_FROM_USER static inline __must_check unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n) { - unsigned long res = n; + unsigned long res = n, to_copy = n; + might_fault(); if (likely(access_ok(from, n))) { kasan_check_write(to, n); res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, to_copy - res); } if (unlikely(res)) memset(to + (n - res), 0, res); @@ -125,10 +144,13 @@ _copy_from_user(void *, const void __user *, unsigned long); static inline __must_check unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n) { + unsigned long to_copy = n; + might_fault(); if (access_ok(to, n)) { kasan_check_read(from, n); n = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, to_copy, n); } return n; } diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 639d5e7014c1..f038676068b2 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -137,18 +137,24 @@ static int copyout(void __user *to, const void *from, size_t n) { + size_t to_copy = n; + if (access_ok(to, n)) { kasan_check_read(from, n); n = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, to_copy, n); } return n; } static int copyin(void *to, const void __user *from, size_t n) { + size_t to_copy = n; + if (access_ok(from, n)) { kasan_check_write(to, n); n = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, to_copy - n); } return n; } diff --git a/lib/usercopy.c b/lib/usercopy.c index cbb4d9ec00f2..abfd93edecba 100644 --- a/lib/usercopy.c +++ b/lib/usercopy.c @@ -1,4 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include @@ -7,11 +8,12 @@ #ifndef INLINE_COPY_FROM_USER unsigned long _copy_from_user(void *to, const void __user *from, unsigned long n) { - unsigned long res = n; + unsigned long res = n, to_copy = n; might_fault(); if (likely(access_ok(from, n))) { kasan_check_write(to, n); res = raw_copy_from_user(to, from, n); + kmsan_unpoison_shadow(to, to_copy - res); } if (unlikely(res)) memset(to + (n - res), 0, res); @@ -23,10 +25,12 @@ EXPORT_SYMBOL(_copy_from_user); #ifndef INLINE_COPY_TO_USER unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n) { + unsigned long to_copy = n; might_fault(); if (likely(access_ok(to, n))) { kasan_check_read(from, n); n = raw_copy_to_user(to, from, n); + kmsan_copy_to_user(to, from, to_copy, n); } return n; } From patchwork Wed Oct 30 14:22:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219653 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA0111668 for ; Wed, 30 Oct 2019 14:24:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8BD9920656 for ; Wed, 30 Oct 2019 14:24:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="G564xCi5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8BD9920656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 961296B000D; Wed, 30 Oct 2019 10:24:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8C7AE6B0288; Wed, 30 Oct 2019 10:24:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6553F6B028A; Wed, 30 Oct 2019 10:24:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id 373406B000D for ; Wed, 30 Oct 2019 10:24:01 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id C5532180AD83A for ; Wed, 30 Oct 2019 14:24:00 +0000 (UTC) X-FDA: 76100670240.12.owl79_4018919f97b19 X-Spam-Summary: 2,0,0,5ce3d9f4711eb963,d41d8cd98f00b204,3_5y5xqykcbq052xyb08805y.w86527eh-664fuw4.8b0@flex--glider.bounces.google.com,:vegard.nossum@oracle.com:dvyukov@google.com::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:aryabinin@virtuozzo.com:luto@kernel.org:ard.biesheuvel@linaro.org:arnd@arndb.de:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:monstr@monstr.eu:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:rdunlap@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747 :1777:17 X-HE-Tag: owl79_4018919f97b19 X-Filterd-Recvd-Size: 4411 Received: from mail-vs1-f74.google.com (mail-vs1-f74.google.com [209.85.217.74]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:24:00 +0000 (UTC) Received: by mail-vs1-f74.google.com with SMTP id q8so284338vso.16 for ; Wed, 30 Oct 2019 07:24:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=DjLgBhOxImo8wyDx7ZIRDkwVUdutczZPc5fAA6NR3ag=; b=G564xCi5mTolvYVdU13Btorjpx/PdDy5Re2jioFL9zjQxw3Agd8i+sdbIg6pNEXsuX w5FnEVQNVmMHYaC2OzhrCwrJBzl6NxG1PNt6kw2yxsGD8Agb/SW0l2nxSndS1KS+au4J EyQkG0Bx8KAbYi8IBoWZ7GDhfeoYnyOwqM4NnPpU/Okbk/xX0uizPPT1L2G0HsEVwlrX S540xqiKQOEc95AvsXWoPpragSnr85yke9H88Wb81+nbG7y72L7z/YKPCtMP0FecEtRV U70EEu7QA47tBLgcOe0XryJX/pEjynahyZ7NlT/GP2tO+9kPkO/4y77mEyTd6w8SXftK rbxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DjLgBhOxImo8wyDx7ZIRDkwVUdutczZPc5fAA6NR3ag=; b=UAq0NJlUXXv6SSRtt6bgPuug97UxlJzyRxzzoQp94ZnpmbWQvMCdVIB2DHK5l9aE3W i9fHLBrF3iwB38QrfibPhB4qIMkFQt/vfaVSGU1fIbDQLgvlVdAsFw4BN1JaT1dLFa9B asTS2bf3EKN9p6NHYnU6LgAqDZIWoVt7cWdBo+sAJzzGwdZurqqlZQTLa89ty/Dem6X1 2ZBSU3TMTpH1BwnBn29FC5A5JzC+5cDja6tElhaW3iZMht+DAh0rDI6pDoVcYszqezX0 0yMZA95lxAK7zfiNQbx/mIrs+P1ilwxRV5uEb3G5mMTngwdBTT4WVy0LDNOju2gT3ESj W6Ew== X-Gm-Message-State: APjAAAWq864ZdRpOxh3Bo7lj1YpVcJ/DHdF47nEVQjDpRsUMQsbktpCx KTjElC/u/oksb1jfAwPbr1xQoRe6Xds= X-Google-Smtp-Source: APXvYqz+xokYBX/34Rf8S5KcM5PzTHarp8HHmxpOSPPjWyj6uxiE0qX+RutkQdyowaVjvP29YCvdrlXkzn8= X-Received: by 2002:a9f:31a9:: with SMTP id v38mr7042326uad.33.1572445439435; Wed, 30 Oct 2019 07:23:59 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:36 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-25-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 24/25] kmsan: disable strscpy() optimization under KMSAN From: glider@google.com To: Vegard Nossum , Dmitry Vyukov , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, aryabinin@virtuozzo.com, luto@kernel.org, ard.biesheuvel@linaro.org, arnd@arndb.de, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, monstr@monstr.eu, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, rdunlap@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Disable the efficient 8-byte reading under KMSAN to avoid false positives. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: linux-mm@kvack.org --- Change-Id: I25d1acf5c3df6eff85894cd94f5ddbe93308271c --- lib/string.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/string.c b/lib/string.c index 08ec58cc673b..15efdc51bda6 100644 --- a/lib/string.c +++ b/lib/string.c @@ -186,7 +186,10 @@ ssize_t strscpy(char *dest, const char *src, size_t count) if (count == 0 || WARN_ON_ONCE(count > INT_MAX)) return -E2BIG; -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS +/** + * Disable the efficient 8-byte reading under KMSAN to avoid false positives. + */ +#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && !defined(CONFIG_KMSAN) /* * If src is unaligned, don't cross a page boundary, * since we don't know if the next page is mapped. From patchwork Wed Oct 30 14:22:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 11219655 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21710112B for ; Wed, 30 Oct 2019 14:24:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D482D20656 for ; Wed, 30 Oct 2019 14:24:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="foinEXrn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D482D20656 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6527A6B0288; Wed, 30 Oct 2019 10:24:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5DB7A6B028B; Wed, 30 Oct 2019 10:24:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A5086B028C; Wed, 30 Oct 2019 10:24:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 233DA6B0288 for ; Wed, 30 Oct 2019 10:24:05 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id C9CA2181AEF31 for ; Wed, 30 Oct 2019 14:24:04 +0000 (UTC) X-FDA: 76100670408.03.pies67_40a4fa2e90048 X-Spam-Summary: 2,0,0,acf6be10b1cd2312,d41d8cd98f00b204,3ap25xqykcbc38501e3bb381.zb985ahk-997ixz7.be3@flex--glider.bounces.google.com,:arnd@arndb.de:monstr@monstr.eu:aryabinin@virtuozzo.com:vegard.nossum@oracle.com:dvyukov@google.com:rdunlap@infradead.org::viro@zeniv.linux.org.uk:akpm@linux-foundation.org:luto@kernel.org:ard.biesheuvel@linaro.org:hch@lst.de:dmitry.torokhov@gmail.com:edumazet@google.com:ericvh@gmail.com:gregkh@linuxfoundation.org:harry.wentland@amd.com:herbert@gondor.apana.org.au:mingo@elte.hu:axboe@kernel.dk:martin.petersen@oracle.com:schwidefsky@de.ibm.com:mst@redhat.com:pmladek@suse.com:sergey.senozhatsky@gmail.com:rostedt@goodmis.org:tiwai@suse.com:tytso@mit.edu:tglx@linutronix.de:wsa@the-dreams.de:gor@linux.ibm.com:iii@linux.ibm.com:mark.rutland@arm.com:willy@infradead.org:andreyknvl@google.com:elver@google.com:glider@google.com,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730 :1747:17 X-HE-Tag: pies67_40a4fa2e90048 X-Filterd-Recvd-Size: 5958 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Oct 2019 14:24:04 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id t203so933913wmt.7 for ; Wed, 30 Oct 2019 07:24:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=aEKImc3GyS6ol208UKSp/0U/bMelxB/cN8IdhVf0ZLU=; b=foinEXrn5sSIcguYqbpItfdFbBSQwUGgw7ZWYtog9jhutmhpGn/CP20f9ZRoWZU5d8 +in3hmNdMZEf4jSAuirGS+hY9tqzg1aVax2lbvIqeLiBqsGMEMdH9a9vcYskbzaPr7n5 0+ytUPoN8C2UEZ0hOblVYRxgFfIen19EYBdqUK3ez/e9Rzl1oTByL5487bav/5ems123 NH4RWdYkBeqbcq6hLED1Fgx8+K8umFXUdmvWx8rpVN6zsbng7V0RJvx/LQHjUDfCht4e 6dPFkmL0FHA6t6S/vpsgDeOlp8nlq9MFfNnuEII7akWB7J/Up1fAKKYBlo6lAYs3uvMY qTAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=aEKImc3GyS6ol208UKSp/0U/bMelxB/cN8IdhVf0ZLU=; b=ot7XmQu5Pg29Hmjtw6Z/mubQRNL4DrxpqDOsGpAYZL5Aww4jyl87se859NgQ36USLR w98qzJIXsE4vE6jWJvBls+v7du0lZjbjW7ViGwfUEB93gFtAe8DGAhnuYsHc2xPCVbkg T27I3JA8GBh+pVbCqIEWVeyL7pr5ycyTniXFKHU0La78FnjL5eaWQsaqc+/tU7/G0NTa Zf+a139RcRTidkBuh1UYLegX8wpdtM6+rT6nvhBEKJa07m6zN5cHP1D7Z7l0v8A+c43P nrICyZtt0wZxrEVZj7RN0iLM8LaxXd5qRAQEHVVEleCTCV/pNp7lGL8o7s0ueRmXdWlL QZpQ== X-Gm-Message-State: APjAAAXVjSjygDx8tUPcJWsz+pGZUmlthvmTwI9Qqqie8t0QApGXoGEE 4jZ61peZg+2nGNQQzoxzf51EhiM9BP8= X-Google-Smtp-Source: APXvYqwjPpcpgUP0Z97icCyzivmA96mht8GU52uWCfM2g0JpQRjExKg2Vw12ZjrmMQ96STLfojortXs+6R4= X-Received: by 2002:adf:ea88:: with SMTP id s8mr144262wrm.120.1572445442607; Wed, 30 Oct 2019 07:24:02 -0700 (PDT) Date: Wed, 30 Oct 2019 15:22:37 +0100 In-Reply-To: <20191030142237.249532-1-glider@google.com> Message-Id: <20191030142237.249532-26-glider@google.com> Mime-Version: 1.0 References: <20191030142237.249532-1-glider@google.com> X-Mailer: git-send-email 2.24.0.rc0.303.g954a862665-goog Subject: [PATCH RFC v2 25/25] net: kasan: kmsan: support CONFIG_GENERIC_CSUM on x86, enable it for KASAN/KMSAN From: glider@google.com To: Arnd Bergmann , Michal Simek , Andrey Ryabinin , Vegard Nossum , Dmitry Vyukov , Randy Dunlap , linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, akpm@linux-foundation.org, luto@kernel.org, ard.biesheuvel@linaro.org, hch@lst.de, dmitry.torokhov@gmail.com, edumazet@google.com, ericvh@gmail.com, gregkh@linuxfoundation.org, harry.wentland@amd.com, herbert@gondor.apana.org.au, mingo@elte.hu, axboe@kernel.dk, martin.petersen@oracle.com, schwidefsky@de.ibm.com, mst@redhat.com, pmladek@suse.com, sergey.senozhatsky@gmail.com, rostedt@goodmis.org, tiwai@suse.com, tytso@mit.edu, tglx@linutronix.de, wsa@the-dreams.de, gor@linux.ibm.com, iii@linux.ibm.com, mark.rutland@arm.com, willy@infradead.org, andreyknvl@google.com, elver@google.com, Alexander Potapenko X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is needed to allow memory tools like KASAN and KMSAN see the memory accesses from the checksum code. Without CONFIG_GENERIC_CSUM the tools can't see memory accesses originating from handwritten assembly code. For KASAN it's a question of detecting more bugs, for KMSAN using the C implementation also helps avoid false positives originating from seemingly uninitialized checksum values. Signed-off-by: Alexander Potapenko To: Alexander Potapenko Cc: Arnd Bergmann Cc: Michal Simek Cc: Andrey Ryabinin Cc: Vegard Nossum Cc: Dmitry Vyukov Cc: Randy Dunlap Cc: linux-mm@kvack.org --- v2: - dropped the "default n" (as requested by Randy Dunlap) Change-Id: I645e2c097253a8d5717ad87e2e2df6f6f67251f3 --- arch/x86/Kconfig | 4 ++++ arch/x86/include/asm/checksum.h | 10 +++++++--- arch/x86/lib/Makefile | 2 ++ 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 6d2b0fb3a871..9ea394f6513a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -272,6 +272,10 @@ config GENERIC_ISA_DMA def_bool y depends on ISA_DMA_API +config GENERIC_CSUM + bool + default y if KMSAN || KASAN + config GENERIC_BUG def_bool y depends on BUG diff --git a/arch/x86/include/asm/checksum.h b/arch/x86/include/asm/checksum.h index d79d1e622dcf..ab3464cbce26 100644 --- a/arch/x86/include/asm/checksum.h +++ b/arch/x86/include/asm/checksum.h @@ -1,6 +1,10 @@ /* SPDX-License-Identifier: GPL-2.0 */ -#ifdef CONFIG_X86_32 -# include +#ifdef CONFIG_GENERIC_CSUM +# include #else -# include +# ifdef CONFIG_X86_32 +# include +# else +# include +# endif #endif diff --git a/arch/x86/lib/Makefile b/arch/x86/lib/Makefile index 5246db42de45..bca9031de9ff 100644 --- a/arch/x86/lib/Makefile +++ b/arch/x86/lib/Makefile @@ -55,7 +55,9 @@ endif lib-$(CONFIG_X86_USE_3DNOW) += mmx_32.o else obj-y += iomap_copy_64.o +ifneq ($(CONFIG_GENERIC_CSUM),y) lib-y += csum-partial_64.o csum-copy_64.o csum-wrappers_64.o +endif lib-y += clear_page_64.o copy_page_64.o lib-y += memmove_64.o memset_64.o lib-y += copy_user_64.o