From patchwork Mon Nov 19 17:26:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689001 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E11614DB for ; Mon, 19 Nov 2018 17:26:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31CFE298B4 for ; Mon, 19 Nov 2018 17:26:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25131299F9; Mon, 19 Nov 2018 17:26:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 36F2429C87 for ; Mon, 19 Nov 2018 17:26:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 528AB6B1B00; Mon, 19 Nov 2018 12:26:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4B12C6B1B01; Mon, 19 Nov 2018 12:26:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B8BC6B1B23; Mon, 19 Nov 2018 12:26:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id ACCCA6B1B00 for ; Mon, 19 Nov 2018 12:26:52 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id r200-v6so13764678wmg.1 for ; Mon, 19 Nov 2018 09:26:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ERgHrl82HJcuSUKaxUm1GBhCqPu/YglHxJKcF5dZ9/Y=; b=Hsn/hOUowj6LVmJ1O0+FBP/U0nmqmzFdTaUdi3Ak5PMYsRsvRshedOnHlYMP2d/k2A L1IVdu7nn8yOCqkVv0ffz6NazSG/+QnnvKlCgd5/a63CEIrhOHdOpG9nmDD84zZV/vMf j4NIUTAWUtWGgR6g/VFozwnJKij9mlrjnYyJdK6l8eKSd9Xd6xYycMdry7xFKpUFfbVU +TPJu5+JcY8GLljqqaHGc8UoKLh6Rck4abpYir2+5YDKhtHJykEfNLpePD9nvNz1AjJ1 82GLMa4BReJ6gSbUTNwqj0TZWMQ9bNAaN+vXn9BuDSbCX3t3re5BRCRtruFOAt5D5fDz 2l3A== X-Gm-Message-State: AA+aEWbJ799OP8FGY4iyyHh0zM0WUe/2JPSsLs7N0OB2VdVdq812xKQT yoB/VtN8Es4gD0Owd/Ox2rVFOETO6nOQVUt5ac7u2F/uUijCxTZQT2fC1k4Cbviz8njabhonzKC L+5i/nscMJnlH/cCQp8HCJgOeh0fONMXfJLDnyPk4HmJnzwha/8LY6H2QaM8GRf0XfvLd7VkPec Hbi12oWAGeR5Bshkoz2BxDeAXxpt7cOIo608KkeF/2WPc8D3ZKxr26Eaxr9EdULHmoOPKeKqi2O pgpTCFWZ7P3kf4Wk9T+Zu7pUEDQ/8SF5cUuinrMsetLcRzJRX+2peWvLpqpFZEuOpyGj5q5vGoz zT1bwBA8Hw6bZYVrF8+B/gS5n0a86CwPv3GlArbCrRiK3YYCnSxZj4qiIAxV+0ypEFsJn1ZPao5 Q X-Received: by 2002:a1c:1109:: with SMTP id 9-v6mr7779814wmr.146.1542648412195; Mon, 19 Nov 2018 09:26:52 -0800 (PST) X-Received: by 2002:a1c:1109:: with SMTP id 9-v6mr7779741wmr.146.1542648410825; Mon, 19 Nov 2018 09:26:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648410; cv=none; d=google.com; s=arc-20160816; b=Qn07Xvk5xm+/BiSs8YJJEk1dLe35RnD2lDV81APb89ughBXx6stluVa8PP6RT5mQZ7 NdCh+oYMs7V6VpY4vRRwg4ZY7ekPYpBTnPFPC9KIhY1DV4e8lXt20NeWAB46qSExtul5 IpP3ySoYuKmAGqKLk8l1frYHKtfFs0MTKEo7f84aIteWBvJ8B7+ql5KG0drKShxoTwQj A2wt0N6fFpnmhV5OndWXX2syOnYJHxeU8AkgBmrkVpJAkmYZmJwGEguUTJ+C/8/jR9bJ zPsJP4mCGEW0t/AFzJSCZz74TL5wVdPoA/SCo2a40GE9G4InQOJ6cyyiJQWn3Rf6/D2+ pONQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=ERgHrl82HJcuSUKaxUm1GBhCqPu/YglHxJKcF5dZ9/Y=; b=FVB14Hg1WYAl9htClg6vbVM+Pr5wsrIfa8ndUxo60NqEsd34jW2TPTHtecZioDu/Uk 9vhm605TnRTnHl3gibmWm5O303G5afZdjZXk0o8mlHHGdakIWlxrIyqxmqIT7I1xX1G1 h8SdqdqkvGAum56x4jdJMsyz48gxEiH/LRk9jm79fRTnIT+Scx7FYadXdwXOWk9catdC w+mi/3DOiu+BIQ9BvDns9TVpqSzCPQxjwlh1uIa5upgz7NOcc0E9sWlD8M3ELdzb61IL 4wRqfNOSCvzt7UEX1L1i8Hacm7ac7To53BmBdjImUuDqv9uFg2MMS5yrONw8c8ndTICP 4aDg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VaXoqbGg; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i125-v6sor19363713wma.15.2018.11.19.09.26.50 for (Google Transport Security); Mon, 19 Nov 2018 09:26:50 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VaXoqbGg; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ERgHrl82HJcuSUKaxUm1GBhCqPu/YglHxJKcF5dZ9/Y=; b=VaXoqbGgUtrMKl5XLeHM0lwBEJoBfMizsn3Pxp2sskPU0GM9CK0ySsXwFV49yDKNhb vVwDc8H8G9RsFrdmxq+EPSz/IK1BUmirEe1dDE7htWcnMmUNMVIKz5wsCRZ81xQ4+4Ct XGpyr1oTuJih6U+wqw+TE1XsFPJft+1iHLL8+6p/a9V8Nks2hO4/DP0EiI9IresL43fK CFPFxfsY+TCI0tZqqbiKdg83gRPsMHgxO5VonlN1Dv4aw1Z1rDlx4xuxXAG1R9m6X+Vg QzcTfxE4OVdt/CEYJ8srd9JgKMbAJkrP7YBVZbxe9mNLQbU8H0oIbXlVInxkxAURH+Rb xpJg== X-Google-Smtp-Source: AFSGD/XfUS6pF7mxzIVJSuHOZEzzMoPiQ+v+GLgkZJC81kA+Cm6mNMj6yyZrfws/xU5sarsTUtsp9A== X-Received: by 2002:a1c:8c4d:: with SMTP id o74mr2765320wmd.134.1542648410108; Mon, 19 Nov 2018 09:26:50 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.26.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:26:49 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 01/24] kasan, mm: change hooks signatures Date: Mon, 19 Nov 2018 18:26:17 +0100 Message-Id: <61ec4c761463ef428a9182815e4b65c9285544d4.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN changes the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 43 +++++++++++++++++++++++++++++-------------- include/linux/slab.h | 4 ++-- mm/kasan/kasan.c | 30 ++++++++++++++++++------------ mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 7 files changed, 65 insertions(+), 45 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 46aae129917c..52c86a568a4e 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -51,16 +51,16 @@ void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -105,19 +105,34 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) {} -static inline void kasan_init_slab_obj(struct kmem_cache *cache, - const void *object) {} +static inline void *kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + return (void *)object; +} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/include/linux/slab.h b/include/linux/slab.h index 918f374e7156..351ac48dabc4 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -444,7 +444,7 @@ static __always_inline void *kmem_cache_alloc_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc(s, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -455,7 +455,7 @@ kmem_cache_alloc_node_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc_node(s, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } #endif /* CONFIG_TRACING */ diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..55deff17a4d9 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -474,20 +474,22 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, return (void *)object + cache->kasan_info.free_meta_offset; } -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; if (!(cache->flags & SLAB_KASAN)) - return; + return (void *)object; alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +530,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +540,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +553,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +568,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +578,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index 2a5654bb3b3f..26f60a22e5e0 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3681,7 +3681,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3719,7 +3719,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 58c6c1c2a78e..4190c24ef0e9 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7eb8dc136c1c..5f3504e26d4c 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1204,7 +1204,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1482,7 +1482,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index e3629cd7aff1..fdd4a86aa882 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1372,10 +1372,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2768,7 +2768,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2796,7 +2796,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3784,7 +3784,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3801,8 +3801,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3829,7 +3828,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } From patchwork Mon Nov 19 17:26:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689003 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A18B14DB for ; Mon, 19 Nov 2018 17:26:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7DFDC298B4 for ; Mon, 19 Nov 2018 17:26:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 71A282A2AF; Mon, 19 Nov 2018 17:26:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1098F298B4 for ; Mon, 19 Nov 2018 17:26:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 91F3F6B1B01; Mon, 19 Nov 2018 12:26:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8AA656B1B69; Mon, 19 Nov 2018 12:26:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 725166B1B6A; Mon, 19 Nov 2018 12:26:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 1B8A46B1B01 for ; Mon, 19 Nov 2018 12:26:54 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id z14-v6so33639315wrh.23 for ; Mon, 19 Nov 2018 09:26:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=jyRw+rMzdi1alRfDuiZVB0ZJPKSze4LAdpqwyD2ClBE=; b=cBrCfRWyvWv1cm77GNj+QS0sx3PNFT5KA/iUzFlpPee4vLwKzHRLBw4bOBfdhDtzPP WGdNii7jtWSSQDrZs0omqIz6/6zwphX6GgcYFjA6Li4OFJwep8aeqblWB2Fdl9aSA8Zg faZvjfM8XQ8KtFfRd4NR91rWL4okQAWFhcK9k3oUlkzAF4AHtbFoHz3NZht/ejeiN+VB eT1aVcC3wJM1Y6SW3DAZsucBOCoF2z2RrWrdzqpoQaIi/dvogNWV+cFrIK9Uey2SvjwH 7h4GAiw0GizHNfaQz4V/E/XjxvdJIZRtrb0TH7D+k4pu2i/OjT2m5Hm5/cRiJxNgApOJ SYDg== X-Gm-Message-State: AA+aEWbgLj+kPsyL4yq8DZAsb+OHTysIlJi0YsToVGSyHUxiSEAUb9J8 DYIAynDsST/NSoAOcpN79l7DCci2Fj9y7Xa+iKIVNwCLqDghFUd8dfpHP/yZdDc15bq2L1vXMBe L96ppX3e9vZgmcIsvLUsZ5fpBrMQ3JA+mSqEBEUWA+nvJpwfwkVIsWysoNDL15Xm5gaD+H9E9A+ Cxm0WKBI3iDPSnBsW1KBjuTcacYtvXPVnsfY/vxvGKNVz1Pc5T9CtvtXePATPbQ5vz3FBXdMiGG GBILa69yKACmVBJzsqukYKXy1uxxXRUTdeKUdR8U4FB8P5V6JXlmC/YIgH0v47GQDRVWhc/Cqr3 23yZ/p+2bLnNkTMFyZ2m+d9LvE9mHhBlaFilGLjBSJ4dDpnmSoCcFAkkj1eKz4Q15wPx5XeXqnq G X-Received: by 2002:adf:a50c:: with SMTP id i12mr9208568wrb.220.1542648413650; Mon, 19 Nov 2018 09:26:53 -0800 (PST) X-Received: by 2002:adf:a50c:: with SMTP id i12mr9208531wrb.220.1542648412834; Mon, 19 Nov 2018 09:26:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648412; cv=none; d=google.com; s=arc-20160816; b=BqVfbrqN1wP4Szs7ZXJIc8QmejNIUtn3L+5/X9CYcumdBRYpmMxpIRcoBpG8rxTqtI KEGs/fpKFWlW1rH/YL03TZxcToiECkKVyeM13n4rFX3/7IfJo4NoG3DSJRVM02GFvhYS iyWYH5M1HS0wbY93bTxJerlZx9Lkg6SjICN4rMoSqS6gNgHqgYKsjiAvrE3RXYKJNtkR XzWE10VkSxz9wShH4RT9CaT0uZWnFYIy80rHkVIEsSNLX0Rho/vrJLSSgQAZeumHy8nb bGZND12mVDZwQS1/JflqHaIxZPTVIabeEXgnNAUS3yZWrVJmgKel0bsLHs3Y8rHzjGEF aqHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=jyRw+rMzdi1alRfDuiZVB0ZJPKSze4LAdpqwyD2ClBE=; b=CwduKVgnd6NbUQWU1lur6Rw1K1pCufaEXdMrt0iT5k53bnw+iyWQ9UtwkZJOJGIgDf rqQBoXtmNF2HetcnXuRKOlB3W7pLR/+x8r/fG0/Qk5/YSZMRx61QeoTfh5hne7uyVitL 92WOxtLbzHEaGSRRJIr8yetpojcNr7eGXvHh2FZAgi/nE3Gy9hp2HOwIMViQQ/nPyWW8 IOuKrcmxOblfGHl5154tLf2NJLMI4YqCoaCp0sHE2Vvg8pmSUuVMPL3AUM439fFp7SUm Z9ILcaxje9vGjfqFW28JUMIuJBtKHZUuRDCHuxclKPQqkR2e4dZPk+I2NSPYDKpQR/Yd QqKw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=mix8fNuo; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 59-v6sor25733094wro.34.2018.11.19.09.26.52 for (Google Transport Security); Mon, 19 Nov 2018 09:26:52 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=mix8fNuo; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jyRw+rMzdi1alRfDuiZVB0ZJPKSze4LAdpqwyD2ClBE=; b=mix8fNuo76mUcOPQpFGJWa+t7fphMwTFH0Yv/elxeFlZH0rFcGIhZSYZDdFmCW/Yc2 gdVTJ1G8VrAf3ZTQFOBWXHTHQML1qLB4+ArqqGrvFMpu/52VezVzttvjQGeCn/ppJiAz 2M36LqSQ0j3VewPVv18rf6O38zK9OStZG8MZbQrOrkxapZyL8dadP1feNdsZh9E4ibWH 89x1sGZMqHA0DyrvQ1nSWJHqp+wx8b0Yhaml0yUS5oH0Mo3uo7NPWGOy5iePmASMlyX4 Cx1OsEIjkZD2mMsnpK0cA4AcUmPsh33slFpcSzjdJFEbCyh8yVbEh6cMGs7uDR1BzfR+ XpMQ== X-Google-Smtp-Source: AJdET5ePNjMSgad+LbaZrgUmfsb6OSab/vVHZ5XlZ7W2Ovv+StAw8FD1z8DRCPskgWhsuE36zAqtCg== X-Received: by 2002:adf:a409:: with SMTP id d9-v6mr19710119wra.190.1542648412325; Mon, 19 Nov 2018 09:26:52 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.26.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:26:51 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 02/24] kasan, slub: handle pointer tags in early_kmem_cache_node_alloc Date: Mon, 19 Nov 2018 18:26:18 +0100 Message-Id: <14a68774683e15692615600c47bba53506a1c8ef.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The previous patch updated KASAN hooks signatures and their usage in SLAB and SLUB code, except for the early_kmem_cache_node_alloc function. This patch handles that function separately, as it requires to reorder some of the initialization code to correctly propagate a tagged pointer in case a tag is assigned by kasan_kmalloc. Signed-off-by: Andrey Konovalov --- mm/slub.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index fdd4a86aa882..8561a32910dd 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3364,16 +3364,16 @@ static void early_kmem_cache_node_alloc(int node) n = page->freelist; BUG_ON(!n); - page->freelist = get_freepointer(kmem_cache_node, n); - page->inuse = 1; - page->frozen = 0; - kmem_cache_node->node[node] = n; #ifdef CONFIG_SLUB_DEBUG init_object(kmem_cache_node, n, SLUB_RED_ACTIVE); init_tracking(kmem_cache_node, n); #endif - kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node), + n = kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node), GFP_KERNEL); + page->freelist = get_freepointer(kmem_cache_node, n); + page->inuse = 1; + page->frozen = 0; + kmem_cache_node->node[node] = n; init_kmem_cache_node(n); inc_slabs_node(kmem_cache_node, node, page->objects); From patchwork Mon Nov 19 17:26:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689011 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5822014DB for ; Mon, 19 Nov 2018 17:27:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38E6B298B4 for ; Mon, 19 Nov 2018 17:27:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2C85F29C87; Mon, 19 Nov 2018 17:27:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 164572A2AF for ; Mon, 19 Nov 2018 17:27:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06C6B6B1B6C; Mon, 19 Nov 2018 12:26:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F3C4F6B1B6D; Mon, 19 Nov 2018 12:26:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC9866B1B71; Mon, 19 Nov 2018 12:26:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 4A5BF6B1B6C for ; Mon, 19 Nov 2018 12:26:58 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id t62-v6so13675379wmg.6 for ; Mon, 19 Nov 2018 09:26:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9UVYbkZUo66eijj9PNZwdH++RpJ8usmi08tQvlIl+dk=; b=LSrvKUAytr+roO7Xdbbd38j46ljkAbVB7iqtivJToE+L+gSzI4JTwMoYWU3r7XZPes KCO8aWZEF49NAArgh5TLoblZOQ40U0yAhNCZiqJ7S71DGWRmZP4dUWntaWb/NPEOgATv 3qveNjkG7cjKGNu4hh8rfRUbk0THDgBca/Y7/XgoYJZ746YIXbeo5U3hUNAoEyDiDyDb cNtFHktLcEh6UWQ/lVEQaiSsdIgTVh0iU8kGbuxmGJ/Kiok5OLU55w/z9HcDZyX3mw89 uwpLgMl6dbvFb9BG1IDMgnN2anZkvbW0N9tBhQsbXuxd0r+SxiFp6S6uXZI6t9Z95G5S zhtg== X-Gm-Message-State: AA+aEWZmqDxTeLMPRadkB9t3gTUi1B+UAc1CGcxJcjqk6vu7Ot8prYzu 1WJGg0vLaNS+juBGwj15uKClrZpgXpPXf9wIiSEnnSe5YXdBai/+jPc/xNUibG9MY6OMwgeu067 h8rhz2vysZhJaFDHCLmKB+rXRknKgkDAnw+uP0hJFe5IElx4pDc2V5UDCmQkZPBYkfYFuOTx3yL 2YbUzNiWdU3CEEmOeJz+qEpptqUVM6QINcGC27xA8jrzIBAN+zQZdQ1q2AvgE1zsu3Y7EU4wMzE Dl8DJxgf2VMu5e+HZO/+a8Z07yDnGakTb0KIVZ+XZu7OKm4iHS9VA8w7u0gx60g0Tsvhkyr70yN 3ndkFgDlUyY8dqS10agQDQc1mHR4nAeD5wB2SriVKi/ld3B5c3UyMCYhlUPfBRes2WIWzzB9p8z j X-Received: by 2002:a5d:444a:: with SMTP id x10mr9780887wrr.162.1542648417690; Mon, 19 Nov 2018 09:26:57 -0800 (PST) X-Received: by 2002:a5d:444a:: with SMTP id x10mr9780769wrr.162.1542648415225; Mon, 19 Nov 2018 09:26:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648415; cv=none; d=google.com; s=arc-20160816; b=enrrENYuSnmXFpR9GFRRapOPaPZ31WRIRBFpYfPvExJVlhfOWo9eXZ98L+tTiyjo6V AHUlhO7IfKBThqKQh9iejscBJRUMXlz5YqOMIJIz5Lc7psGTZ78teSawc9BNefy7chs4 8vyv6YpO54Nyev5rYB9QMSLkmfdckqFEEvELL9/rJF7tc/UhMQBQDd8+mkOXbEBSS+Z3 GsSbx+kBuNJxQdr9O+1WaPOxMfOV1EhrkJatgv1W352OQ0M2bC0gJLtxpUZlS8E7L8Ig 6GynmFxxNezQGoApQHoS1OVQ9dwxAot+0t9C2mgRCCfAkHtGmG/wHOU4+aBnr4hhnqiO b5bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=9UVYbkZUo66eijj9PNZwdH++RpJ8usmi08tQvlIl+dk=; b=IdCHFwIpiIXDfRnyLRB6sqQ3TSY+cb9lt8Yba8G+kognSCK+2rmhP4MxO9bpChJYpg tbmV0V9IX2c+gmFPH48EajpBh1sAsbMA/ArYdn/irIyVOEACzJRSayWStvoibKXl1K57 FdzSGgOGft0w5jFmUSDb9CmCG/CQeeKX39efUrvIzvwUYzRtP+f0yzFz8RqFvfebDPF3 rpnN5jV8f0vG+LFLMovv4ZtAuruW6yn/H69zyc0Nm6+fCiBXOKHd6BVx8jrRZK0f1u6i A9qbg7RUdNDVKMSgNGRBwgYWOAgkLYXxTVvxsocta2ozb7lwhBVvuiWdsZ9yS75QafL0 K1fQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=h0UX91l1; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r2-v6sor20244419wrq.43.2018.11.19.09.26.55 for (Google Transport Security); Mon, 19 Nov 2018 09:26:55 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=h0UX91l1; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9UVYbkZUo66eijj9PNZwdH++RpJ8usmi08tQvlIl+dk=; b=h0UX91l1E2x8wu1JBiUW1Z/VUdO21TxTtu/xbkUajCc5rQX9a2Eazp/fP45lfTzS8R pQwCynpK9B83ZVp7RDe9Q2wjeBw283xuWyu7+PPuemOjrPj3hpzzLv23mraZ8WTlT4Sd w+gffOQTfxqsrym3K063sT243IR/zrPZCzGhGgDBVPR59fKzyOscapUY1qBbiihN6pE3 iK9M5jCm/8eSQw9Y1c9fPG93GYi1c5UqPcCMOFxNNKUtqv8xcMV/JHzGf1Cj0skE90wQ +crBL2b3zG2XDEWMruNaV08h4YpIaFuzekkDIKokf++DTJ5yJFCBd3p0KQf2svsincq4 ZByQ== X-Google-Smtp-Source: AFSGD/UmMJSnP5nyokRkJcCtIL6miS8+wuNdckwQJo0lO7O7SZBpUlFzcd60+hobcGji/75GdyDiZA== X-Received: by 2002:adf:f449:: with SMTP id f9mr12068654wrp.40.1542648414216; Mon, 19 Nov 2018 09:26:54 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.26.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:26:53 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 03/24] kasan: move common generic and tag-based code to common.c Date: Mon, 19 Nov 2018 18:26:19 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN reuses a significant part of the generic KASAN code, so move the common parts to common.c without any functional changes. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 603 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 570 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 614 insertions(+), 569 deletions(-) create mode 100644 mm/kasan/common.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 3289db38bc87..a6df14bffb6b 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o diff --git a/mm/kasan/common.c b/mm/kasan/common.c new file mode 100644 index 000000000000..5f68c93734ba --- /dev/null +++ b/mm/kasan/common.c @@ -0,0 +1,603 @@ +/* + * This file contains common generic and tag-based KASAN code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +static inline void filter_irq_stacks(struct stack_trace *trace) +{ + int i; + + if (!trace->nr_entries) + return; + for (i = 0; i < trace->nr_entries; i++) + if (in_irqentry_text(trace->entries[i])) { + /* Include the irqentry function into the stack. */ + trace->nr_entries = i + 1; + break; + } +} + +static inline depot_stack_handle_t save_stack(gfp_t flags) +{ + unsigned long entries[KASAN_STACK_DEPTH]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = KASAN_STACK_DEPTH, + .skip = 0 + }; + + save_stack_trace(&trace); + filter_irq_stacks(&trace); + if (trace.nr_entries != 0 && + trace.entries[trace.nr_entries-1] == ULONG_MAX) + trace.nr_entries--; + + return depot_save_stack(&trace, flags); +} + +static inline void set_track(struct kasan_track *track, gfp_t flags) +{ + track->pid = current->pid; + track->stack = save_stack(flags); +} + +void kasan_enable_current(void) +{ + current->kasan_depth++; +} + +void kasan_disable_current(void) +{ + current->kasan_depth--; +} + +void kasan_check_read(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_read); + +void kasan_check_write(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + check_memory_region((unsigned long)addr, len, true, _RET_IP_); + + return __memset(addr, c, len); +} + +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memmove(dest, src, len); +} + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + */ +void kasan_poison_shadow(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void kasan_unpoison_shadow(const void *address, size_t size) +{ + kasan_poison_shadow(address, size, 0); + + if (size & KASAN_SHADOW_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + *shadow = size & KASAN_SHADOW_MASK; + } +} + +static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) +{ + void *base = task_stack_page(task); + size_t size = sp - base; + + kasan_unpoison_shadow(base, size); +} + +/* Unpoison the entire stack for a task. */ +void kasan_unpoison_task_stack(struct task_struct *task) +{ + __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); +} + +/* Unpoison the stack for the current task beyond a watermark sp value. */ +asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) +{ + /* + * Calculate the task stack base address. Avoid using 'current' + * because this function is called by early resume code which hasn't + * yet set up the percpu register (%gs). + */ + void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); + + kasan_unpoison_shadow(base, watermark - base); +} + +/* + * Clear all poison for the region between the current SP and a provided + * watermark value, as is sometimes required prior to hand-crafted asm function + * returns in the middle of functions. + */ +void kasan_unpoison_stack_above_sp_to(const void *watermark) +{ + const void *sp = __builtin_frame_address(0); + size_t size = watermark - sp; + + if (WARN_ON(sp > watermark)) + return; + kasan_unpoison_shadow(sp, size); +} + +void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); +} + +void kasan_free_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_poison_shadow(page_address(page), + PAGE_SIZE << order, + KASAN_FREE_PAGE); +} + +/* + * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. + * For larger allocations larger redzones are used. + */ +static inline unsigned int optimal_redzone(unsigned int object_size) +{ + return + object_size <= 64 - 16 ? 16 : + object_size <= 128 - 32 ? 32 : + object_size <= 512 - 64 ? 64 : + object_size <= 4096 - 128 ? 128 : + object_size <= (1 << 14) - 256 ? 256 : + object_size <= (1 << 15) - 512 ? 512 : + object_size <= (1 << 16) - 1024 ? 1024 : 2048; +} + +void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) +{ + unsigned int orig_size = *size; + int redzone_adjust; + + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } + redzone_adjust = optimal_redzone(cache->object_size) - + (*size - cache->object_size); + + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + + optimal_redzone(cache->object_size))); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } + + *flags |= SLAB_KASAN; +} + +size_t kasan_metadata_size(struct kmem_cache *cache) +{ + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); +} + +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); + return (void *)object + cache->kasan_info.alloc_meta_offset; +} + +struct kasan_free_meta *get_free_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + return (void *)object + cache->kasan_info.free_meta_offset; +} + +void kasan_poison_slab(struct page *page) +{ + kasan_poison_shadow(page_address(page), + PAGE_SIZE << compound_order(page), + KASAN_KMALLOC_REDZONE); +} + +void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_unpoison_shadow(object, cache->object_size); +} + +void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_poison_shadow(object, + round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + KASAN_KMALLOC_REDZONE); +} + +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; +} + +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +{ + return kasan_kmalloc(cache, object, cache->object_size, flags); +} + +static bool __kasan_slab_free(struct kmem_cache *cache, void *object, + unsigned long ip, bool quarantine) +{ + s8 shadow_byte; + unsigned long rounded_up_size; + + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != + object)) { + kasan_report_invalid_free(object, ip); + return true; + } + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + kasan_report_invalid_free(object, ip); + return true; + } + + rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + + if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; +} + +bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +{ + return __kasan_slab_free(cache, object, ip, true); +} + +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, + gfp_t flags) +{ + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(object == NULL)) + return NULL; + + redzone_start = round_up((unsigned long)(object + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = round_up((unsigned long)object + cache->object_size, + KASAN_SHADOW_SCALE_SIZE); + + kasan_unpoison_shadow(object, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); + + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; +} +EXPORT_SYMBOL(kasan_kmalloc); + +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + struct page *page; + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(ptr == NULL)) + return NULL; + + page = virt_to_page(ptr); + redzone_start = round_up((unsigned long)(ptr + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); + + kasan_unpoison_shadow(ptr, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); + + return (void *)ptr; +} + +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +{ + struct page *page; + + if (unlikely(object == ZERO_SIZE_PTR)) + return (void *)object; + + page = virt_to_head_page(object); + + if (unlikely(!PageSlab(page))) + return kasan_kmalloc_large(object, size, flags); + else + return kasan_kmalloc(page->slab_cache, object, size, flags); +} + +void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), + KASAN_FREE_PAGE); + } else { + __kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + +void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (ptr != page_address(virt_to_head_page(ptr))) + kasan_report_invalid_free(ptr, ip); + /* The object will be poisoned by page_alloc. */ +} + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL | __GFP_ZERO, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || + WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 55deff17a4d9..44ec228de0a2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,5 +1,5 @@ /* - * This file contains shadow memory manipulation code. + * This file contains core KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -40,82 +40,6 @@ #include "kasan.h" #include "../slab.h" -void kasan_enable_current(void) -{ - current->kasan_depth++; -} - -void kasan_disable_current(void) -{ - current->kasan_depth--; -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. - */ -static void kasan_poison_shadow(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - memset(shadow_start, value, shadow_end - shadow_start); -} - -void kasan_unpoison_shadow(const void *address, size_t size) -{ - kasan_poison_shadow(address, size, 0); - - if (size & KASAN_SHADOW_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; - } -} - -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_shadow(base, size); -} - -/* Unpoison the entire stack for a task. */ -void kasan_unpoison_task_stack(struct task_struct *task) -{ - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); -} - -/* Unpoison the stack for the current task beyond a watermark sp value. */ -asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) -{ - /* - * Calculate the task stack base address. Avoid using 'current' - * because this function is called by early resume code which hasn't - * yet set up the percpu register (%gs). - */ - void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - - kasan_unpoison_shadow(base, watermark - base); -} - -/* - * Clear all poison for the region between the current SP and a provided - * watermark value, as is sometimes required prior to hand-crafted asm function - * returns in the middle of functions. - */ -void kasan_unpoison_stack_above_sp_to(const void *watermark) -{ - const void *sp = __builtin_frame_address(0); - size_t size = watermark - sp; - - if (WARN_ON(sp > watermark)) - return; - kasan_unpoison_shadow(sp, size); -} - /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -260,121 +184,12 @@ static __always_inline void check_memory_region_inline(unsigned long addr, kasan_report(addr, size, write, ret_ip); } -static void check_memory_region(unsigned long addr, - size_t size, bool write, +void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { check_memory_region_inline(addr, size, write, ret_ip); } -void kasan_check_read(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_read); - -void kasan_check_write(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - check_memory_region((unsigned long)addr, len, true, _RET_IP_); - - return __memset(addr, c, len); -} - -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memmove(dest, src, len); -} - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memcpy(dest, src, len); -} - -void kasan_alloc_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); -} - -void kasan_free_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); -} - -/* - * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. - * For larger allocations larger redzones are used. - */ -static unsigned int optimal_redzone(unsigned int object_size) -{ - return - object_size <= 64 - 16 ? 16 : - object_size <= 128 - 32 ? 32 : - object_size <= 512 - 64 ? 64 : - object_size <= 4096 - 128 ? 128 : - object_size <= (1 << 14) - 256 ? 256 : - object_size <= (1 << 15) - 512 ? 512 : - object_size <= (1 << 16) - 1024 ? 1024 : 2048; -} - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) -{ - unsigned int orig_size = *size; - int redzone_adjust; - - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); - - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); - - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; - } - - *flags |= SLAB_KASAN; -} - void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); @@ -386,277 +201,6 @@ void kasan_cache_shutdown(struct kmem_cache *cache) quarantine_remove_cache(cache); } -size_t kasan_metadata_size(struct kmem_cache *cache) -{ - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); -} - -void kasan_poison_slab(struct page *page) -{ - kasan_poison_shadow(page_address(page), - PAGE_SIZE << compound_order(page), - KASAN_KMALLOC_REDZONE); -} - -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_unpoison_shadow(object, cache->object_size); -} - -void kasan_poison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_poison_shadow(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), - KASAN_KMALLOC_REDZONE); -} - -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline void filter_irq_stacks(struct stack_trace *trace) -{ - int i; - - if (!trace->nr_entries) - return; - for (i = 0; i < trace->nr_entries; i++) - if (in_irqentry_text(trace->entries[i])) { - /* Include the irqentry function into the stack. */ - trace->nr_entries = i + 1; - break; - } -} - -static inline depot_stack_handle_t save_stack(gfp_t flags) -{ - unsigned long entries[KASAN_STACK_DEPTH]; - struct stack_trace trace = { - .nr_entries = 0, - .entries = entries, - .max_entries = KASAN_STACK_DEPTH, - .skip = 0 - }; - - save_stack_trace(&trace); - filter_irq_stacks(&trace); - if (trace.nr_entries != 0 && - trace.entries[trace.nr_entries-1] == ULONG_MAX) - trace.nr_entries--; - - return depot_save_stack(&trace, flags); -} - -static inline void set_track(struct kasan_track *track, gfp_t flags) -{ - track->pid = current->pid; - track->stack = save_stack(flags); -} - -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); - return (void *)object + cache->kasan_info.alloc_meta_offset; -} - -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; -} - -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) -{ - struct kasan_alloc_meta *alloc_info; - - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); - - return (void *)object; -} - -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) -{ - return kasan_kmalloc(cache, object, cache->object_size, flags); -} - -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, - unsigned long ip, bool quarantine) -{ - s8 shadow_byte; - unsigned long rounded_up_size; - - if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != - object)) { - kasan_report_invalid_free(object, ip); - return true; - } - - /* RCU slabs could be legally used after free within the RCU period */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); - return true; - } - - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) - return false; - - set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); - quarantine_put(get_free_info(cache, object), cache); - return true; -} - -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) -{ - return __kasan_slab_free(cache, object, ip, true); -} - -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) -{ - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(object == NULL)) - return NULL; - - redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); - - kasan_unpoison_shadow(object, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); - - if (cache->flags & SLAB_KASAN) - set_track(&get_alloc_info(cache, object)->alloc_track, flags); - - return (void *)object; -} -EXPORT_SYMBOL(kasan_kmalloc); - -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) -{ - struct page *page; - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(ptr == NULL)) - return NULL; - - page = virt_to_page(ptr); - redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); - - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); - - return (void *)ptr; -} - -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) -{ - struct page *page; - - if (unlikely(object == ZERO_SIZE_PTR)) - return ZERO_SIZE_PTR; - - page = virt_to_head_page(object); - - if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); - else - return kasan_kmalloc(page->slab_cache, object, size, flags); -} - -void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), - KASAN_FREE_PAGE); - } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - -void kasan_kfree_large(void *ptr, unsigned long ip) -{ - if (ptr != page_address(virt_to_head_page(ptr))) - kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ -} - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); @@ -797,113 +341,3 @@ DEFINE_ASAN_SET_SHADOW(f2); DEFINE_ASAN_SET_SHADOW(f3); DEFINE_ASAN_SET_SHADOW(f5); DEFINE_ASAN_SET_SHADOW(f8); - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c12dcfde2ebd..659463800f10 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow(const void *address, size_t size, u8 value); + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); From patchwork Mon Nov 19 17:26:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689005 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B94314DB for ; Mon, 19 Nov 2018 17:27:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F255298B4 for ; Mon, 19 Nov 2018 17:27:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 22B9329C87; Mon, 19 Nov 2018 17:27:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7FEE3298B4 for ; Mon, 19 Nov 2018 17:27:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D4566B1B6A; Mon, 19 Nov 2018 12:26:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6AE146B1B71; Mon, 19 Nov 2018 12:26:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 530306B1B6A; Mon, 19 Nov 2018 12:26:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id ED2B86B1B6A for ; Mon, 19 Nov 2018 12:26:57 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id v3-v6so38758091wrw.8 for ; Mon, 19 Nov 2018 09:26:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MOxiGtbnzhqpDBspVzWEzY99H8AFgGg0mLPeJFn5o/U=; b=LOW/XYzn1r4U8x8H3lCxD8DvvTapTlNE2rCWnZP3t1WhF8wiMsxKisgXCNmoZyHktK 71Umek/uvda1ZZqsDBI9kPGObCsj+GHu5scziDJyebr3/4L8nHLOVRau2hPDpv2kXBwQ UmsavcrsKCEZn2KnsX+N4PsF8ZVwBK6S+F7yNsnrKRAlO8rsRiy644e3JuW9J9T2r43P vQHfQG7/zeZpEhV/NjfCRDR6/aISe4CWKzq1ZsFuMHGyitydbFwX4GlcJPMAcBzNl8WN njvQA5WKYGiID+dxyAJgyaLgRCMF6KV2r7/RTGKHlqJjeP0SbEgRfgViacVlSb54pt10 KM7w== X-Gm-Message-State: AA+aEWbohuNFl8d2b74N/h73+IcUa9STGlUiyX6f++V/ZfvZCA4krHHO GY8wz4Ca4wvD9r4+JoiRhQpaoATS7ItViHRahJMShVGuQBWV89md+bFX5vO1IY6YWJgRgked9PG OCnj3hFNM5Md25t1+7SQIG8kcyWotFbvSnJAwWDxNVyrl6s1crIEHJZpmNiu97Y1Ty129E8ollS gGtc7Rye51Kzu3UDJmr+ursVV9E4cACMVSWGUf4yPuIyLqCcRVLCPaxEtif+hRMeiueKKvnAzmD SVFMMbcgzUmx5OWWstooRAvjjxBXEzL1i1hMNO1/Aqrq7g5pMSa/BYoXq0Vr1ID2VL3tY7avu1C nUf2PYH4J61Bi3vTapRy4EqPnz+aL1aOP90R0WK4F0qfXhJh/6rU5SW7YPzv1jye06OI8UwlDYE l X-Received: by 2002:adf:e707:: with SMTP id c7mr4961721wrm.196.1542648417464; Mon, 19 Nov 2018 09:26:57 -0800 (PST) X-Received: by 2002:adf:e707:: with SMTP id c7mr4961690wrm.196.1542648416712; Mon, 19 Nov 2018 09:26:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648416; cv=none; d=google.com; s=arc-20160816; b=eaTEuorTQRdzBOqOStZXa+UhRbCVUSG979QZKQ9ekjNx1AGTv5NfrgYZgHQW9nXZfX +f6mADgBUnZ5So6Qb5HDb126V1jIf8zYGBouUdYWglBt7BnboQvPQYajI35FziQ/rAaI 4IZhI8vKVusVtzbYRk0IfknrE4sSgiT6E/asjIHmSm8f9OqwShc86ZuJsbhK0oOqMDsn 0ZgrjmUoPiDlMDzmpNJzlLKlUEM8lCNHBX8BKLtmUmgYT/rXIY4MTfoNJQkVo0pKLmSo 8eRfYtKJCnwmoaqP7kH+Fh79CZ4TxDdvCJQIlAPHDpkVFKbQZ8kUYKnHBkL+9DcP+EBP 6uOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=MOxiGtbnzhqpDBspVzWEzY99H8AFgGg0mLPeJFn5o/U=; b=z1GVuZwkoQm6Q54t0ASk+ztPd03dMGDvaMGzG1unvd/PNilkUod/iSPqmCq3BfAupX PK6wsdIP3JLpgxM1jFhqnA4g1A4As+f/6/LCVbabzIJRITmsWBg0IBBF1g8aU9Va3ghf Ms9+lwdiLWpIJdbtM/mGynQC8rfX0lxIHT6dJvuAKUNA2d+/6Ecb86YfjTvVAdBxQrEF WAN3iIFLGIIFz1E4xTQWx8IplqXV9zP7mLz0BArqjiTVbHeeLO1jo26seQJrqXjC2DQv FHe6LtAkl2IfHbSndsPc4cp1bts4Px1XTlF5okJrdMoXTULo7kDM3bSDIdKoL2M0sij8 0P+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=URLmOXZK; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l13sor1797941wre.39.2018.11.19.09.26.56 for (Google Transport Security); Mon, 19 Nov 2018 09:26:56 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=URLmOXZK; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MOxiGtbnzhqpDBspVzWEzY99H8AFgGg0mLPeJFn5o/U=; b=URLmOXZKwOvDj+SSbXLXhbvfDTH5VmJxVPJ5zUuwGtDuhF/01XSIZ7lprwchOG+Z5y 98y5fkI5hwCXwPSPOA+uJ707d7eQjlwq/jf3dez9mnFfDgJoYKmND/whcJC2DGnKV+RE raTHNIG9TznaSyAkHxebdbUg7KNiKJoCMrC58JqtVQsQ9lKhbRoKM4miUZQA8DIQA3/4 75le7lOrBXtx4KGu6GBSHjzSpm0XMpaCk8qP0cwQnuhgj1SPCCkQLirUSYrEFKCwSZuG ZduFHeRdT7g25y9PVclAGkstBd9FD9Oh2hhgP2ca/Wwjg8sizSdCqQv6wA8kq0gnm50v hxLA== X-Google-Smtp-Source: AFSGD/U8sgq6gI0uk28IKx47rH6VRYWmJDqB5I8lWxiIoUIosbabdvsEgnEhuL/jWxx1BbIZHGP1pw== X-Received: by 2002:a5d:63c3:: with SMTP id c3mr2726578wrw.215.1542648416246; Mon, 19 Nov 2018 09:26:56 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.26.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:26:55 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 04/24] kasan: rename source files to reflect the new naming scheme Date: Mon, 19 Nov 2018 18:26:20 +0100 Message-Id: <9cd2f3cc890932e8d3ae5eb2ac24458d39d87960.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We now have two KASAN modes: generic KASAN and tag-based KASAN. Rename kasan.c to generic.c to reflect that. Also rename kasan_init.c to init.c as it contains initialization code for both KASAN modes. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 8 ++++---- mm/kasan/{kasan.c => generic.c} | 0 mm/kasan/{kasan_init.c => init.c} | 0 3 files changed, 4 insertions(+), 4 deletions(-) rename mm/kasan/{kasan.c => generic.c} (100%) rename mm/kasan/{kasan_init.c => init.c} (100%) diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index a6df14bffb6b..d643530b24aa 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,14 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n -UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_generic.o := n KCOV_INSTRUMENT := n -CFLAGS_REMOVE_kasan.o = -pg +CFLAGS_REMOVE_generic.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o generic.o report.o init.o quarantine.o diff --git a/mm/kasan/kasan.c b/mm/kasan/generic.c similarity index 100% rename from mm/kasan/kasan.c rename to mm/kasan/generic.c diff --git a/mm/kasan/kasan_init.c b/mm/kasan/init.c similarity index 100% rename from mm/kasan/kasan_init.c rename to mm/kasan/init.c From patchwork Mon Nov 19 17:26:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689013 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 239A714E2 for ; Mon, 19 Nov 2018 17:27:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 05337298B4 for ; Mon, 19 Nov 2018 17:27:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E737029C87; Mon, 19 Nov 2018 17:27:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8DFD298B4 for ; Mon, 19 Nov 2018 17:27:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E0966B1B80; Mon, 19 Nov 2018 12:27:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 095626B1B82; Mon, 19 Nov 2018 12:27:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4E596B1B83; Mon, 19 Nov 2018 12:27:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id 716C46B1B80 for ; Mon, 19 Nov 2018 12:27:02 -0500 (EST) Received: by mail-wm1-f71.google.com with SMTP id o63-v6so13721165wma.2 for ; Mon, 19 Nov 2018 09:27:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=TKgQ9ZhuDJuekqCIiYoGtFV6Mq3lYEbLPXM5e4CNk7I=; b=HHytzmT+HoZ0RDs7cP8ir5/1CJ5zO3VjDu7e6ote+AcLkPu97ht6T8I05D9KNiZljo tQAlXAR+T6IGYNsqL024yqvhpfGu6D6myNypR2HZFCwa54d7oHR5nkH9C2ZoQX+RgrKn SQdV+ZzVV2vYtF1C5jwVrlTUoJfynO9RP/6qyX60+apOYGuTGmfaHxzmj1TiqbgyVZ1i mik6l9wsPdVaj44T+1JlC/eRHdN9S+Or5k7QFaD+Iw2513QrUxrl8bxmKjZNierq7pgu wZOx2MtQpZHGMrw75WaoxpcSvXhqlze5EPqJQTE8YPJDeyfRzi06uMwkKFxrzLj1FwPu EQqg== X-Gm-Message-State: AA+aEWapx0r83o0lLvi0aSfEvOdes4ij9xZ3TB5aOc3mYi6Jyj9fBrq8 uMxR39hZ73aD3UuvEoUVOl/bLpfOFZSEywROm+XAmmni9IGinZj5PEEfLEqzXqMdlsQpvTC8HUT sQFIlGuaqrx3JzAqM2vzMzi7IxaSTBcYrvujkiwixoy6tV+hI2B0ZrRZIT0ErI/B1I3UU1Q+CjI HOCkqLn0IsSc3cAyzS/rvmwd5eZ6PLsK02c4+R8ZhJhTjX/vDCNleKpc6Eiels2bOcI+YtA9LaC SPe85zz1DchKF7FXPbZz9vJRgEcfEEahHX4STeDl2DwrcQ3eIqHljzMuHq/GcSg+CobWcllJLKA 6gqNXwP9ozVstzhkibpk7PAN0QYjDZ1DDFc+fkKT5JnzEQVs8bd6U7kCTvIV5KYVKGu8dRqZc5R F X-Received: by 2002:a1c:e90d:: with SMTP id q13mr8321051wmc.106.1542648421811; Mon, 19 Nov 2018 09:27:01 -0800 (PST) X-Received: by 2002:a1c:e90d:: with SMTP id q13mr8320959wmc.106.1542648420162; Mon, 19 Nov 2018 09:27:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648420; cv=none; d=google.com; s=arc-20160816; b=wWSXg37bAL4tiMuq6Drl9n38j78eJmFyKAQc0dAPsI26UrtieRh0rkCGNlDAd7oNWy D61WUgN+tK2XRHRRFzSyxaW4uRrtLdMJcDp25CMmWc+bw+XFiwFtiNRIfaRY29Y7E2yU NnO153dwfumFW5R0x8ejB/4OpHD4zA7rKVHfrAW9p1zhW9oW3HxtasTSSCP9d+tOMuc1 HjtSs3IMQkl9D+K7yorNOleVjdTp2GsCWqvWS42fFQNvuVO4cKdZKtSmYyOCQM4wGczs V1ZBhN3GrbCY0OqxZvQyIXxeOlzgHQzTn05tM2p8TVasrpWiFtNhTv+GnK/j+GafYBP5 jqXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=TKgQ9ZhuDJuekqCIiYoGtFV6Mq3lYEbLPXM5e4CNk7I=; b=Uylo/ryibtPuedqUds9aOv4gxW4MAFZh6RflTzJRSV9Oncgw9FQP9ddcuySeKy+a5o GO5J44GnOOFFt3EFLEB6vfePreOeXzc5trLFvme4gLNaSJNYf9ByJcAvc7AILJFp/cRQ RXDnlglxv/Ues0N4ivpdBg9Gr7ahoF2UNdL2wfW1zo7fRa5/7TjibbJp0vqlwg5Wgyyu 1o249HUdz6WP+3Hlj+Qjk2UN1OUfs79mwQ2Cmv84d023dSVAELMUyoUDHiRNlxAvTkyk un1xcsXOy0FYEwzrPmJQB7KABGw2XDR24hQyTlx6n+0cjXRdVFubju7TX6Qwoba84Hk7 vNRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ggHJE6gS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n184-v6sor19342589wma.14.2018.11.19.09.26.59 for (Google Transport Security); Mon, 19 Nov 2018 09:27:00 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ggHJE6gS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TKgQ9ZhuDJuekqCIiYoGtFV6Mq3lYEbLPXM5e4CNk7I=; b=ggHJE6gSWaiWuqp9N+HKhZdM1Fsrkd6CiNCad0RNwvLJXvnVmgzoqL8XQlvHAutabd oNuzfj2TyLVn6LqlCS6L+PvmBxBoWSIggcqqVlVtKntNzDNjMEf65oCX/S3b5FuFHLpV j4zS/dBtPrImFP9mrzbU+3l4koLdhquxYa1E5Sqs7qA6Sz1vggXRScNeuLTyGfIf/udj kOJackkB5SnCPgZsuWllFpTAv/z3tX5L5RPnpnBeDBzU0bppDWDeR2p9FqS4IXtTaptJ aHfiXNcU72wdcuHF6Y4xN8IYstNZOLGbqcxMRJUdJIgXvo626HyMoGMl/PlbI6dAiuFV ItAw== X-Google-Smtp-Source: AJdET5frJHo4MEVgSCHyncsGCcbirJeliiLA0NE/zo0ORuLf+mZYkqsZUQ1xILF11fRbYVmShBkNEQ== X-Received: by 2002:a7b:ca44:: with SMTP id m4mr7741040wml.76.1542648419232; Mon, 19 Nov 2018 09:26:59 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.26.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:26:57 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 05/24] kasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS Date: Mon, 19 Nov 2018 18:26:21 +0100 Message-Id: <356c34c9a2ae8348a6cbd1de53135a28187fa120.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit splits the current CONFIG_KASAN config option into two: 1. CONFIG_KASAN_GENERIC, that enables the generic KASAN mode (the one that exists now); 2. CONFIG_KASAN_SW_TAGS, that enables the software tag-based KASAN mode. The name CONFIG_KASAN_SW_TAGS is chosen as in the future we will have another hardware tag-based KASAN mode, that will rely on hardware memory tagging support in arm64. With CONFIG_KASAN_SW_TAGS enabled, compiler options are changed to instrument kernel files with -fsantize=kernel-hwaddress (except the ones for which KASAN_SANITIZE := n is set). Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes. This commit also adds empty placeholder (for now) implementation of tag-based KASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 5 +- include/linux/compiler-gcc.h | 6 ++ include/linux/compiler_attributes.h | 13 ---- include/linux/kasan.h | 16 +++-- lib/Kconfig.kasan | 96 +++++++++++++++++++++++------ mm/kasan/Makefile | 6 +- mm/kasan/generic.c | 2 +- mm/kasan/kasan.h | 3 +- mm/kasan/tags.c | 75 ++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 53 +++++++++------- 12 files changed, 216 insertions(+), 62 deletions(-) create mode 100644 mm/kasan/tags.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 787d7850e064..8b331dcfb48e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -111,6 +111,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_SW_TAGS if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 3e7dafb3ea80..59db75f03218 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -16,9 +16,12 @@ /* all clang versions usable with the kernel support KASAN ABI version 5 */ #define KASAN_ABI_VERSION 5 +#if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) /* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature(address_sanitizer) #define __SANITIZE_ADDRESS__ +#define __no_sanitize_address __attribute__((no_sanitize("address", "hwaddress"))) +#else +#define __no_sanitize_address #endif /* diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 2010493e1040..5776da43da97 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -143,6 +143,12 @@ #define KASAN_ABI_VERSION 3 #endif +#if __has_attribute(__no_sanitize_address__) +#define __no_sanitize_address __attribute__((no_sanitize_address)) +#else +#define __no_sanitize_address +#endif + #if GCC_VERSION >= 50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1 #endif diff --git a/include/linux/compiler_attributes.h b/include/linux/compiler_attributes.h index f8c400ba1929..7bceb9469197 100644 --- a/include/linux/compiler_attributes.h +++ b/include/linux/compiler_attributes.h @@ -206,19 +206,6 @@ */ #define __noreturn __attribute__((__noreturn__)) -/* - * Optional: only supported since gcc >= 4.8 - * Optional: not supported by icc - * - * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute - * clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis - */ -#if __has_attribute(__no_sanitize_address__) -# define __no_sanitize_address __attribute__((__no_sanitize_address__)) -#else -# define __no_sanitize_address -#endif - /* * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Type-Attributes.html#index-packed-type-attribute * clang: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#index-packed-variable-attribute diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 52c86a568a4e..b66fdf5ea7ab 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); -void kasan_cache_shrink(struct kmem_cache *cache); -void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); @@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} -static inline void kasan_cache_shrink(struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, @@ -155,4 +151,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink(struct kmem_cache *cache); +void kasan_cache_shutdown(struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink(struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ + #endif /* LINUX_KASAN_H */ diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index d0bad1bd9a2b..b5d0cbfce4a1 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -1,35 +1,95 @@ +# This config refers to the generic KASAN mode. config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_SW_TAGS + bool + +config CC_HAS_KASAN_GENERIC + def_bool $(cc-option, -fsanitize=kernel-address) + +config CC_HAS_KASAN_SW_TAGS + def_bool $(cc-option, -fsanitize=kernel-hwaddress) + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs. + See Documentation/dev-tools/kasan.rst for details. + +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: generic KASAN (similar to userspace ASan, + x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and + software tag-based KASAN (a version based on software memory + tagging, arm64 only, similar to userspace HWASan, enabled with + CONFIG_KASAN_SW_TAGS). + Both generic and tag-based KASAN are strictly debugging features. + +config KASAN_GENERIC + bool "Generic mode" + depends on CC_HAS_KASAN_GENERIC depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables generic KASAN mode. + Supported in both GCC and Clang. With GCC it requires version 4.9.2 + or later for basic support and version 5.0 or later for detection of + out-of-bounds accesses for stack and global variables and for inline + instrumentation mode (CONFIG_KASAN_INLINE). With Clang it requires + version 3.7.0 or later and it doesn't support detection of + out-of-bounds accesses for global variables yet. + This mode consumes about 1/8th of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +if HAVE_ARCH_KASAN_SW_TAGS + +config KASAN_SW_TAGS + bool "Software tag-based mode" + depends on CC_HAS_KASAN_SW_TAGS + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enables software tag-based KASAN mode. + This mode requires Top Byte Ignore support by the CPU and therefore + is only supported for arm64. + This mode requires Clang version 7.0.0 or later. + This mode consumes about 1/16th of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + This mode may potentially introduce problems relating to pointer + casting and comparison, as it embeds tags into the top byte of each + pointer. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer + This enables further checks in generic KASAN, for now it only + includes the address-use-after-scope check that can lead to + excessive kernel stack usage, frame size warnings and longer compile time. - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more + See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 choice @@ -53,7 +113,7 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice @@ -67,11 +127,11 @@ config KASAN_S390_4_LEVEL_PAGING 4-level paging instead. config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endif diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index d643530b24aa..68ba1822f003 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_generic.o := n +UBSAN_SANITIZE_tags.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_generic.o = -pg @@ -10,5 +11,8 @@ CFLAGS_REMOVE_generic.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o generic.o report.o init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 44ec228de0a2..b8de6d33c55c 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,5 +1,5 @@ /* - * This file contains core KASAN code. + * This file contains core generic KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 659463800f10..19b950eaccff 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB) +#if defined(CONFIG_KASAN_GENERIC) && \ + (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c new file mode 100644 index 000000000000..04194923c543 --- /dev/null +++ b/mm/kasan/tags.c @@ -0,0 +1,75 @@ +/* + * This file contains core tag-based KASAN code. + * + * Copyright (c) 2018 Google, Inc. + * Author: Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE(size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE(1); +DEFINE_HWASAN_LOAD_STORE(2); +DEFINE_HWASAN_LOAD_STORE(4); +DEFINE_HWASAN_LOAD_STORE(8); +DEFINE_HWASAN_LOAD_STORE(16); + +void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_storeN_noabort); + +void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_tag_memory); diff --git a/mm/slub.c b/mm/slub.c index 8561a32910dd..e739d46600b9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2992,7 +2992,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page, do_slab_free(s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr) { do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr); diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 69552a39951d..25c259df8ffa 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC + ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else @@ -12,36 +13,44 @@ CFLAGS_KASAN_MINIMAL := -fsanitize=kernel-address cc-param = $(call cc-option, -mllvm -$(1), $(call cc-option, --param $(1))) -ifeq ($(call cc-option, $(CFLAGS_KASAN_MINIMAL) -Werror),) - ifneq ($(CONFIG_COMPILE_TEST),y) - $(warning Cannot use CONFIG_KASAN: \ - -fsanitize=kernel-address is not supported by compiler) - endif -else - # -fasan-shadow-offset fails without -fsanitize - CFLAGS_KASAN_SHADOW := $(call cc-option, -fsanitize=kernel-address \ +# -fasan-shadow-offset fails without -fsanitize +CFLAGS_KASAN_SHADOW := $(call cc-option, -fsanitize=kernel-address \ -fasan-shadow-offset=$(KASAN_SHADOW_OFFSET), \ $(call cc-option, -fsanitize=kernel-address \ -mllvm -asan-mapping-offset=$(KASAN_SHADOW_OFFSET))) - ifeq ($(strip $(CFLAGS_KASAN_SHADOW)),) - CFLAGS_KASAN := $(CFLAGS_KASAN_MINIMAL) - else - # Now add all the compiler specific options that are valid standalone - CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ - $(call cc-param,asan-globals=1) \ - $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ - $(call cc-param,asan-stack=1) \ - $(call cc-param,asan-use-after-scope=1) \ - $(call cc-param,asan-instrument-allocas=1) - endif - +ifeq ($(strip $(CFLAGS_KASAN_SHADOW)),) + CFLAGS_KASAN := $(CFLAGS_KASAN_MINIMAL) +else + # Now add all the compiler specific options that are valid standalone + CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ + $(call cc-param,asan-globals=1) \ + $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ + $(call cc-param,asan-stack=1) \ + $(call cc-param,asan-use-after-scope=1) \ + $(call cc-param,asan-instrument-allocas=1) endif ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif # CONFIG_KASAN_GENERIC +ifdef CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif + +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +endif # CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif From patchwork Mon Nov 19 17:26:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689017 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 850D214DB for ; Mon, 19 Nov 2018 17:27:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A35529A31 for ; Mon, 19 Nov 2018 17:27:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5BF892A2B5; Mon, 19 Nov 2018 17:27:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E45A629A31 for ; Mon, 19 Nov 2018 17:27:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 153DD6B1B82; Mon, 19 Nov 2018 12:27:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 10C196B1B84; Mon, 19 Nov 2018 12:27:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F136F6B1B85; Mon, 19 Nov 2018 12:27:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 9B8D36B1B82 for ; Mon, 19 Nov 2018 12:27:03 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id d11so30250572wrq.18 for ; Mon, 19 Nov 2018 09:27:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=dR8lcfOg/asfdxo/WfCVQzFad9LYpScILBiq6D3Qbc8=; b=UNni48IbSytFhKihM7px1WS/YYacT51Q/g5f4C8QSA827VZGbgoHk59z0f53gi0CwC tI4oEK6lV0g0dPBPLRHYvw1Ph5Vp5j6/kRVNhk6/cE9WOOsFWw6ijRYhbqbu0yQGMcGH XaRv7XsqdKeWtxCwnyfoyo1cadyRSdmBkLLNgwtK+HnKB0ylE936mCx/wzCOpO4ghFUs vYmiBY1vWqSCEu95slz47Bzy45pafryk+ZWdrxKNTFkMLBZpEnYMFuTmBMhol3TPumtk puOv6+onrCT6hINAOxab+iOx4IPGY98LgOB6ASz16Ibi8Y9hFXB56j9FH2vHsX81iBlz 29/A== X-Gm-Message-State: AGRZ1gKvLUgwquDS9MSJIq7xtulrbIyvSBT3olRmH0bzLDeju+nbE1J7 pjlCvj9c/qmv/ie7RFjaSq9arw3NwoSkJyYr/hg9z5bEcxITCdd4sVKH4N9JZsrLigW0z9/mwxQ 8prYK5y860yljTyTbwy2GLG1/of67/OyoVmXsRVzOp2Uv39hVOKh9UUPaBko4PjPpwuQplVpHSr uVVVtr8r41Sk7jfPQEayzSwNEdoFoyRFh65Snt3OqN8p12SP9SyLSk5lQvgKkF0xwgpw7KErJg5 VKDVU3r2vauyxhfKTrJHUwBVVbq+oF8NIbtNWIYXznEsXHb2u0y4TdtCNgCb3jrfp7QoXh0pjmM Xht1xOGTWR0ldRrjNsy3roV0d835ZJhCOrln13GSZ3oR8NUKCGskaUFDgxXdBEG80shc/u3kMFi H X-Received: by 2002:a1c:c483:: with SMTP id u125mr7755428wmf.14.1542648423161; Mon, 19 Nov 2018 09:27:03 -0800 (PST) X-Received: by 2002:a1c:c483:: with SMTP id u125mr7755387wmf.14.1542648422309; Mon, 19 Nov 2018 09:27:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648422; cv=none; d=google.com; s=arc-20160816; b=nuMqaX761VFZMU90PaAwFNvyixCNc6ahDxi1TYpFrZZGEQPBq349Q8wV6INtewrOON kLSY4OGqZQ9LfSCu2J1B7xdE8Q1abZuumNUwr2LDpqm6vrqDWbuhmlJSF8A98FInSyTr /JwJicIUiOUUUxaD+Crb/sCEG320v1tmhqXQ26xHlE7ikvTvPddP7QNvUQtkkcj386yn g2IO2klG3E6snAZPuaqaJ3UN01xNP13mGj8eWueJqae1xrI+BHUQYZojjUIRMxn4L+Ea K1UOILnt9Q9apbjHmr2LppCrD2lcztWeoOinlQrDe3Ist+Owcb64//tzH0lHZ44/LAVu Lc8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=dR8lcfOg/asfdxo/WfCVQzFad9LYpScILBiq6D3Qbc8=; b=aDRav+IZsBJpodemlomj/wLsN+70kVUBZ5NmsJSsuYR3OiJPpofrTTdEgtMmBqMrLu YPKZckJ6VfUAC+GkNDElUS5Cl4g4cZEyQoCuwNUErBtDLBh8WG3CGoxK5qi75w9UNhkf 59n2cF9Wmk7cc/ipzEACmL+Nc1EuQg9Hjh4QJKf5e8h+OmW39kJk3ADQo6V9vIiHlhXb cIAidPNWcrdswb9euYK8inqQuq8GzHDq5KwS8IoisElbnElSxN7ok5XsxP7tCJKTRDi3 3pJeuODCIvca02mViMjay7xmmWJZrMKZ22KE2CPclLPJQ/xe5a0VNv1MIqix1F3jyHjA s9CQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=b3lKnIQw; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v15-v6sor19561009wmd.6.2018.11.19.09.27.02 for (Google Transport Security); Mon, 19 Nov 2018 09:27:02 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=b3lKnIQw; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dR8lcfOg/asfdxo/WfCVQzFad9LYpScILBiq6D3Qbc8=; b=b3lKnIQw33I69347c6fVByjZWZr7rDfZ6pO/Th+7IH/z4NfXLoPk6gkZ6U+EuX8ViZ YvJr7csFPYS3irKKPwFBlIUVwi664iOL1+0Zl2X2madbWgValetXf8gjws1+oU9Tu1ye OEwGw5UlVlxIv/bPEHYGYnySoUH32sHcN6B14z0CVpKxTCSstFOwYILAL0z/aC1+bVaI 7qjaP5iEylpKToNC3dh4SGcPFIAaCiniXcIq9QFyoplJCiGTFhrdjEKkYbqYwX2beBSj Q+4Eaq6aFNUnLw4H9+FNi6Msuzomb85hS5dxEpUktBeo4ZKCl8DjGrDQOJc1a87JAIXi 16vQ== X-Google-Smtp-Source: AFSGD/WNR8o/WnEF0/GAJSPJLxjNxD3wmkq6whPNnsgmqmNKBmrpQx9bRReY6c/V+wChGBijG3HyMw== X-Received: by 2002:a1c:f916:: with SMTP id x22mr5632080wmh.87.1542648421670; Mon, 19 Nov 2018 09:27:01 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.26.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:00 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 06/24] kasan, arm64: adjust shadow size for tag-based mode Date: Mon, 19 Nov 2018 18:26:22 +0100 Message-Id: <8122d5bedc46bb3655d576c922db3d722a231bf0.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory. This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when the tag-based KASAN mode is enabled. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/Makefile | 11 ++++++++++- arch/arm64/include/asm/memory.h | 7 +++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 6cb9fc7e9382..99e7d08c6083 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -91,10 +91,19 @@ else TEXT_OFFSET := 0x00080000 endif +ifeq ($(CONFIG_KASAN_SW_TAGS), y) +KASAN_SHADOW_SCALE_SHIFT := 4 +else +KASAN_SHADOW_SCALE_SHIFT := 3 +endif + +KBUILD_CFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) +KBUILD_CPPFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) +KBUILD_AFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) + # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b96442960aea..05fbc7ffcd31 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -74,12 +74,11 @@ #define KERNEL_END _end /* - * KASAN requires 1/8th of the kernel virtual address space for the shadow - * region. KASAN can bloat the stack significantly, so double the (minimum) - * stack size when KASAN is in use. + * Generic and tag-based KASAN require 1/8th and 1/16th of the kernel virtual + * address space for the shadow region respectively. They can bloat the stack + * significantly, so double the (minimum) stack size when they are in use. */ #ifdef CONFIG_KASAN -#define KASAN_SHADOW_SCALE_SHIFT 3 #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_THREAD_SHIFT 1 #else From patchwork Mon Nov 19 17:26:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689023 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 062CC14E2 for ; Mon, 19 Nov 2018 17:27:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DDEEA298B4 for ; Mon, 19 Nov 2018 17:27:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D05CF29C87; Mon, 19 Nov 2018 17:27:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26B8829A31 for ; Mon, 19 Nov 2018 17:27:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7991A6B1B84; Mon, 19 Nov 2018 12:27:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 721B26B1B85; Mon, 19 Nov 2018 12:27:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5EECF6B1B86; Mon, 19 Nov 2018 12:27:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id D10386B1B84 for ; Mon, 19 Nov 2018 12:27:07 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id y85so13995227wmc.7 for ; Mon, 19 Nov 2018 09:27:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=LX2VpEunsATaNfvuRrhUTYQxy3Jf9F2v8rd4afQXov8=; b=QVDSKQvaS5Odbt94Edo4NiGHm60mKpsLeuGIkfSu6OQ+IQQxqUpwlD4FI3W4c3WRd+ +B4SA8VLe+z6bXSGLxHCQbmyieKQ/KDIomoH9Pnk8ydGYjfot5+xbExT1+4s/0QGmM4R wdsLwazVWL/0CIyBrsnin7qkPH4VEKn5r3JTiNX7B8xat/kbq8e7SKrXIpef4fzZh+pr FPrA92uok6xcyUF3xlkXNhLZm1nR2I9ZndBzjefM4x+LnD41cnQzPvZzNeTaG6X2X8+D QpuIYW2zLVWhqkv1BlDVDUlLpqqbGHFB4ZRj9KRXXV2OZXoQpkFmeTwyKdV5PbKAdB4U nacw== X-Gm-Message-State: AA+aEWbR7twriMhNwp4DepjZiP4+7TcDfBjpcbI9lBBD7C/tBMPeDyH/ wIMD6sU7Ca5agl0Fx11f49SnYjtxy1DmkNOv/Pgl2G0EzU0/88Q0CAZ70LuBMl6eqYQX73fJhe4 bqEb0AhzMbYHzNK+NDoC6nC75e1xhlDsPOwEDYz9MskuPejXtvphGkvu1iBpyfvPzZ4TJr2/Wfa jJBOLXbYdGdK/+Wtw7PCcpxksBTRO31OlS+XGWnbQ6YFSNwXmiasZ8l1hcg2MHk77oOWrGTPiEF 7tV3QJtDVhYK1oSAV4vSxlSSBdQ8Nn90dL6di0DmjvbFrqQTMAQMVqGzlX4RVMzV4aFtozkM1+E HRVfBDBkkNhvSP92oMQ7ZwyMKK4HE9cYC4Jo5yb5GqpWOoS8IqVr2/X0NcFJcHt+fx3am88yqal 4 X-Received: by 2002:adf:d0c9:: with SMTP id z9mr6725858wrh.317.1542648427295; Mon, 19 Nov 2018 09:27:07 -0800 (PST) X-Received: by 2002:adf:d0c9:: with SMTP id z9mr6725755wrh.317.1542648425249; Mon, 19 Nov 2018 09:27:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648425; cv=none; d=google.com; s=arc-20160816; b=ngLAJBGIl6OMr3fGykZOxAp42mO5+v1w46X5Smxqb8jW9ML2ve0p/Xd7Vpd8Hzq3RG lX6j8fIe2KYKYEYOdy4TP4udEzAx4JSFzA3G8Z0q70y2yAORgoANWOo7OXzBCXI8Xwpf MQMt/pRodf4AFXy5gjU0Yo1gz9xE1v60404OIGBsAlrY/enwoZ+2OAWXHsPGjOlD92ek tBOupmrZknrnQDxn8RhdmaHaGCoc61M7IOVq3TBHzFcAysYHFxuQTsJHPAdQJQe2tDxK gnV0xoetvGwoK/hmdN0H19Qh2BphkX/GCOY3y0/ldSOks4N0/fhUAUeYPKJiHrh/o4kk DtGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=LX2VpEunsATaNfvuRrhUTYQxy3Jf9F2v8rd4afQXov8=; b=NN1C+4+HBiKJ7HRbIi9JZUfEqh63lMgBNqX1rx0Oy5DTgYrjh39tvzey9MUNDvBZzu CNM2SxvQJVM4VaVTxFmadTPtouYKe61ze4YKbpis+uNytW22mA/NlIMb6pm23zZf078V 1C1x2CP+nZ/tur5cDffs4LDqWNO6bY7I9DABkOHJJu0ze7p959JQWRdQ1SQIPINZSFEA W/G/UajNYK9WRHP8f6m6IWcOlZ3t/OkDqi0U077nzzFDGS1r/5v5jnJUABqekcK3aBhi H6F2jW/2hfgawTz9Oy4+hM1YyfcotZ03Xj8DQlZc8CaxGQF92HgILDaMj7IEhkLyHljz cMYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hZf+7G3p; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h126sor5631784wmf.21.2018.11.19.09.27.05 for (Google Transport Security); Mon, 19 Nov 2018 09:27:05 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hZf+7G3p; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LX2VpEunsATaNfvuRrhUTYQxy3Jf9F2v8rd4afQXov8=; b=hZf+7G3pcnJ8oL0njOGBq3hZ8Yc35pFF5HjCsmF7ZAdPMknnTcYVJScoWKLU7ci0Po xe7r11QS+VeZ9m0yD9sfi3pWde94+IzeyqljYqugxqsr77QFES64/WYEocMunZ5ktLzy y4xFeR06ArpPvZLf0/wK2p0yzgSbgcl+rfSKei3vShEwCMLY4Eoa5RId4CE5g1bbgzqv VGGxj59IlS8GQiQwoEabVhVra8M4oUvBXhnSSMzp3DDej28OhPW6HYAhI+rwFUDnFJjL rSRGnzl1yDsvLeHN98Xm4cTi68Ntmi/ybJesp1soROXJkC2zzTEoYJZwhU02Ek+8mYMU gKkA== X-Google-Smtp-Source: AJdET5dttrK9XxRz6ajaobXQ2aEfrYQq2WvyzenQdBlFZpdhsJyTjgYY0obao6azFzDgGFWD+kFEEw== X-Received: by 2002:a1c:7a03:: with SMTP id v3-v6mr7719482wmc.108.1542648424455; Mon, 19 Nov 2018 09:27:04 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:02 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 07/24] kasan: rename kasan_zero_page to kasan_early_shadow_page Date: Mon, 19 Nov 2018 18:26:23 +0100 Message-Id: <76d5f82b50773a4e88402f77da132e40d14e301d.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With tag based KASAN mode the early shadow value is 0xff and not 0x00, so this patch renames kasan_zero_(page|pte|pmd|pud|p4d) to kasan_early_shadow_(page|pte|pmd|pud|p4d) to avoid confusion. Suggested-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 43 ++++++++++++--------- arch/s390/mm/dump_pagetables.c | 16 ++++---- arch/s390/mm/kasan_init.c | 33 +++++++++------- arch/x86/mm/dump_pagetables.c | 11 +++--- arch/x86/mm/kasan_init_64.c | 55 +++++++++++++------------- arch/xtensa/mm/kasan_init.c | 18 +++++---- include/linux/kasan.h | 12 +++--- mm/kasan/init.c | 70 +++++++++++++++++++--------------- 8 files changed, 144 insertions(+), 114 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 63527e585aac..4ebc19422931 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -47,8 +47,9 @@ static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { if (pmd_none(READ_ONCE(*pmdp))) { - phys_addr_t pte_phys = early ? __pa_symbol(kasan_zero_pte) - : kasan_alloc_zeroed_page(node); + phys_addr_t pte_phys = early ? + __pa_symbol(kasan_early_shadow_pte) + : kasan_alloc_zeroed_page(node); __pmd_populate(pmdp, pte_phys, PMD_TYPE_TABLE); } @@ -60,8 +61,9 @@ static pmd_t *__init kasan_pmd_offset(pud_t *pudp, unsigned long addr, int node, bool early) { if (pud_none(READ_ONCE(*pudp))) { - phys_addr_t pmd_phys = early ? __pa_symbol(kasan_zero_pmd) - : kasan_alloc_zeroed_page(node); + phys_addr_t pmd_phys = early ? + __pa_symbol(kasan_early_shadow_pmd) + : kasan_alloc_zeroed_page(node); __pud_populate(pudp, pmd_phys, PMD_TYPE_TABLE); } @@ -72,8 +74,9 @@ static pud_t *__init kasan_pud_offset(pgd_t *pgdp, unsigned long addr, int node, bool early) { if (pgd_none(READ_ONCE(*pgdp))) { - phys_addr_t pud_phys = early ? __pa_symbol(kasan_zero_pud) - : kasan_alloc_zeroed_page(node); + phys_addr_t pud_phys = early ? + __pa_symbol(kasan_early_shadow_pud) + : kasan_alloc_zeroed_page(node); __pgd_populate(pgdp, pud_phys, PMD_TYPE_TABLE); } @@ -87,8 +90,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, pte_t *ptep = kasan_pte_offset(pmdp, addr, node, early); do { - phys_addr_t page_phys = early ? __pa_symbol(kasan_zero_page) - : kasan_alloc_zeroed_page(node); + phys_addr_t page_phys = early ? + __pa_symbol(kasan_early_shadow_page) + : kasan_alloc_zeroed_page(node); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -205,14 +209,14 @@ void __init kasan_init(void) kasan_map_populate(kimg_shadow_start, kimg_shadow_end, early_pfn_to_nid(virt_to_pfn(lm_alias(_text)))); - kasan_populate_zero_shadow((void *)KASAN_SHADOW_START, - (void *)mod_shadow_start); - kasan_populate_zero_shadow((void *)kimg_shadow_end, - kasan_mem_to_shadow((void *)PAGE_OFFSET)); + kasan_populate_early_shadow((void *)KASAN_SHADOW_START, + (void *)mod_shadow_start); + kasan_populate_early_shadow((void *)kimg_shadow_end, + kasan_mem_to_shadow((void *)PAGE_OFFSET)); if (kimg_shadow_start > mod_shadow_end) - kasan_populate_zero_shadow((void *)mod_shadow_end, - (void *)kimg_shadow_start); + kasan_populate_early_shadow((void *)mod_shadow_end, + (void *)kimg_shadow_start); for_each_memblock(memory, reg) { void *start = (void *)__phys_to_virt(reg->base); @@ -227,14 +231,15 @@ void __init kasan_init(void) } /* - * KAsan may reuse the contents of kasan_zero_pte directly, so we - * should make sure that it maps the zero page read-only. + * KAsan may reuse the contents of kasan_early_shadow_pte directly, + * so we should make sure that it maps the zero page read-only. */ for (i = 0; i < PTRS_PER_PTE; i++) - set_pte(&kasan_zero_pte[i], - pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO)); + set_pte(&kasan_early_shadow_pte[i], + pfn_pte(sym_to_pfn(kasan_early_shadow_page), + PAGE_KERNEL_RO)); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/arch/s390/mm/dump_pagetables.c b/arch/s390/mm/dump_pagetables.c index 363f6470d742..43cdaf96f79f 100644 --- a/arch/s390/mm/dump_pagetables.c +++ b/arch/s390/mm/dump_pagetables.c @@ -111,11 +111,11 @@ static void note_page(struct seq_file *m, struct pg_state *st, } #ifdef CONFIG_KASAN -static void note_kasan_zero_page(struct seq_file *m, struct pg_state *st) +static void note_kasan_early_shadow_page(struct seq_file *m, struct pg_state *st) { unsigned int prot; - prot = pte_val(*kasan_zero_pte) & + prot = pte_val(*kasan_early_shadow_pte) & (_PAGE_PROTECT | _PAGE_INVALID | _PAGE_NOEXEC); note_page(m, st, prot, 4); } @@ -154,8 +154,8 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_zero_pmd)) { - note_kasan_zero_page(m, st); + if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_early_shadow_pmd)) { + note_kasan_early_shadow_page(m, st); return; } #endif @@ -185,8 +185,8 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_zero_pud)) { - note_kasan_zero_page(m, st); + if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_early_shadow_pud)) { + note_kasan_early_shadow_page(m, st); return; } #endif @@ -215,8 +215,8 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_zero_p4d)) { - note_kasan_zero_page(m, st); + if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_early_shadow_p4d)) { + note_kasan_early_shadow_page(m, st); return; } #endif diff --git a/arch/s390/mm/kasan_init.c b/arch/s390/mm/kasan_init.c index acb9645b762b..bac5c27d11fc 100644 --- a/arch/s390/mm/kasan_init.c +++ b/arch/s390/mm/kasan_init.c @@ -107,7 +107,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PGDIR_SIZE) && end - address >= PGDIR_SIZE) { - pgd_populate(&init_mm, pg_dir, kasan_zero_p4d); + pgd_populate(&init_mm, pg_dir, + kasan_early_shadow_p4d); address = (address + PGDIR_SIZE) & PGDIR_MASK; continue; } @@ -120,7 +121,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, P4D_SIZE) && end - address >= P4D_SIZE) { - p4d_populate(&init_mm, p4_dir, kasan_zero_pud); + p4d_populate(&init_mm, p4_dir, + kasan_early_shadow_pud); address = (address + P4D_SIZE) & P4D_MASK; continue; } @@ -133,7 +135,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PUD_SIZE) && end - address >= PUD_SIZE) { - pud_populate(&init_mm, pu_dir, kasan_zero_pmd); + pud_populate(&init_mm, pu_dir, + kasan_early_shadow_pmd); address = (address + PUD_SIZE) & PUD_MASK; continue; } @@ -146,7 +149,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PMD_SIZE) && end - address >= PMD_SIZE) { - pmd_populate(&init_mm, pm_dir, kasan_zero_pte); + pmd_populate(&init_mm, pm_dir, + kasan_early_shadow_pte); address = (address + PMD_SIZE) & PMD_MASK; continue; } @@ -188,7 +192,7 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, pte_val(*pt_dir) = __pa(page) | pgt_prot; break; case POPULATE_ZERO_SHADOW: - page = kasan_zero_page; + page = kasan_early_shadow_page; pte_val(*pt_dir) = __pa(page) | pgt_prot_zero; break; } @@ -256,14 +260,14 @@ void __init kasan_early_init(void) unsigned long vmax; unsigned long pgt_prot = pgprot_val(PAGE_KERNEL_RO); pte_t pte_z; - pmd_t pmd_z = __pmd(__pa(kasan_zero_pte) | _SEGMENT_ENTRY); - pud_t pud_z = __pud(__pa(kasan_zero_pmd) | _REGION3_ENTRY); - p4d_t p4d_z = __p4d(__pa(kasan_zero_pud) | _REGION2_ENTRY); + pmd_t pmd_z = __pmd(__pa(kasan_early_shadow_pte) | _SEGMENT_ENTRY); + pud_t pud_z = __pud(__pa(kasan_early_shadow_pmd) | _REGION3_ENTRY); + p4d_t p4d_z = __p4d(__pa(kasan_early_shadow_pud) | _REGION2_ENTRY); kasan_early_detect_facilities(); if (!has_nx) pgt_prot &= ~_PAGE_NOEXEC; - pte_z = __pte(__pa(kasan_zero_page) | pgt_prot); + pte_z = __pte(__pa(kasan_early_shadow_page) | pgt_prot); memsize = get_mem_detect_end(); if (!memsize) @@ -292,10 +296,13 @@ void __init kasan_early_init(void) } /* init kasan zero shadow */ - crst_table_init((unsigned long *)kasan_zero_p4d, p4d_val(p4d_z)); - crst_table_init((unsigned long *)kasan_zero_pud, pud_val(pud_z)); - crst_table_init((unsigned long *)kasan_zero_pmd, pmd_val(pmd_z)); - memset64((u64 *)kasan_zero_pte, pte_val(pte_z), PTRS_PER_PTE); + crst_table_init((unsigned long *)kasan_early_shadow_p4d, + p4d_val(p4d_z)); + crst_table_init((unsigned long *)kasan_early_shadow_pud, + pud_val(pud_z)); + crst_table_init((unsigned long *)kasan_early_shadow_pmd, + pmd_val(pmd_z)); + memset64((u64 *)kasan_early_shadow_pte, pte_val(pte_z), PTRS_PER_PTE); shadow_alloc_size = memsize >> KASAN_SHADOW_SCALE_SHIFT; pgalloc_low = round_up((unsigned long)_end, _SEGMENT_SIZE); diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index fc37bbd23eb8..c4696ab9a72b 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -380,7 +380,7 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, /* * This is an optimization for KASAN=y case. Since all kasan page tables - * eventually point to the kasan_zero_page we could call note_page() + * eventually point to the kasan_early_shadow_page we could call note_page() * right away without walking through lower level page tables. This saves * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. @@ -388,10 +388,11 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, void *pt) { - if (__pa(pt) == __pa(kasan_zero_pmd) || - (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_zero_p4d)) || - __pa(pt) == __pa(kasan_zero_pud)) { - pgprotval_t prot = pte_flags(kasan_zero_pte[0]); + if (__pa(pt) == __pa(kasan_early_shadow_pmd) || + (pgtable_l5_enabled() && + __pa(pt) == __pa(kasan_early_shadow_p4d)) || + __pa(pt) == __pa(kasan_early_shadow_pud)) { + pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); note_page(m, st, __pgprot(prot), 0, 5); return true; } diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 04a9cf6b034f..462fde83b515 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -211,7 +211,8 @@ static void __init kasan_early_p4d_populate(pgd_t *pgd, unsigned long next; if (pgd_none(*pgd)) { - pgd_entry = __pgd(_KERNPG_TABLE | __pa_nodebug(kasan_zero_p4d)); + pgd_entry = __pgd(_KERNPG_TABLE | + __pa_nodebug(kasan_early_shadow_p4d)); set_pgd(pgd, pgd_entry); } @@ -222,7 +223,8 @@ static void __init kasan_early_p4d_populate(pgd_t *pgd, if (!p4d_none(*p4d)) continue; - p4d_entry = __p4d(_KERNPG_TABLE | __pa_nodebug(kasan_zero_pud)); + p4d_entry = __p4d(_KERNPG_TABLE | + __pa_nodebug(kasan_early_shadow_pud)); set_p4d(p4d, p4d_entry); } while (p4d++, addr = next, addr != end && p4d_none(*p4d)); } @@ -261,10 +263,11 @@ static struct notifier_block kasan_die_notifier = { void __init kasan_early_init(void) { int i; - pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC; - pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE; - pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE; - p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE; + pteval_t pte_val = __pa_nodebug(kasan_early_shadow_page) | + __PAGE_KERNEL | _PAGE_ENC; + pmdval_t pmd_val = __pa_nodebug(kasan_early_shadow_pte) | _KERNPG_TABLE; + pudval_t pud_val = __pa_nodebug(kasan_early_shadow_pmd) | _KERNPG_TABLE; + p4dval_t p4d_val = __pa_nodebug(kasan_early_shadow_pud) | _KERNPG_TABLE; /* Mask out unsupported __PAGE_KERNEL bits: */ pte_val &= __default_kernel_pte_mask; @@ -273,16 +276,16 @@ void __init kasan_early_init(void) p4d_val &= __default_kernel_pte_mask; for (i = 0; i < PTRS_PER_PTE; i++) - kasan_zero_pte[i] = __pte(pte_val); + kasan_early_shadow_pte[i] = __pte(pte_val); for (i = 0; i < PTRS_PER_PMD; i++) - kasan_zero_pmd[i] = __pmd(pmd_val); + kasan_early_shadow_pmd[i] = __pmd(pmd_val); for (i = 0; i < PTRS_PER_PUD; i++) - kasan_zero_pud[i] = __pud(pud_val); + kasan_early_shadow_pud[i] = __pud(pud_val); for (i = 0; pgtable_l5_enabled() && i < PTRS_PER_P4D; i++) - kasan_zero_p4d[i] = __p4d(p4d_val); + kasan_early_shadow_p4d[i] = __p4d(p4d_val); kasan_map_early_shadow(early_top_pgt); kasan_map_early_shadow(init_top_pgt); @@ -326,7 +329,7 @@ void __init kasan_init(void) clear_pgds(KASAN_SHADOW_START & PGDIR_MASK, KASAN_SHADOW_END); - kasan_populate_zero_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK), + kasan_populate_early_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK), kasan_mem_to_shadow((void *)PAGE_OFFSET)); for (i = 0; i < E820_MAX_ENTRIES; i++) { @@ -338,41 +341,41 @@ void __init kasan_init(void) shadow_cpu_entry_begin = (void *)CPU_ENTRY_AREA_BASE; shadow_cpu_entry_begin = kasan_mem_to_shadow(shadow_cpu_entry_begin); - shadow_cpu_entry_begin = (void *)round_down((unsigned long)shadow_cpu_entry_begin, - PAGE_SIZE); + shadow_cpu_entry_begin = (void *)round_down( + (unsigned long)shadow_cpu_entry_begin, PAGE_SIZE); shadow_cpu_entry_end = (void *)(CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE); shadow_cpu_entry_end = kasan_mem_to_shadow(shadow_cpu_entry_end); - shadow_cpu_entry_end = (void *)round_up((unsigned long)shadow_cpu_entry_end, - PAGE_SIZE); + shadow_cpu_entry_end = (void *)round_up( + (unsigned long)shadow_cpu_entry_end, PAGE_SIZE); - kasan_populate_zero_shadow( + kasan_populate_early_shadow( kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM), shadow_cpu_entry_begin); kasan_populate_shadow((unsigned long)shadow_cpu_entry_begin, (unsigned long)shadow_cpu_entry_end, 0); - kasan_populate_zero_shadow(shadow_cpu_entry_end, - kasan_mem_to_shadow((void *)__START_KERNEL_map)); + kasan_populate_early_shadow(shadow_cpu_entry_end, + kasan_mem_to_shadow((void *)__START_KERNEL_map)); kasan_populate_shadow((unsigned long)kasan_mem_to_shadow(_stext), (unsigned long)kasan_mem_to_shadow(_end), early_pfn_to_nid(__pa(_stext))); - kasan_populate_zero_shadow(kasan_mem_to_shadow((void *)MODULES_END), - (void *)KASAN_SHADOW_END); + kasan_populate_early_shadow(kasan_mem_to_shadow((void *)MODULES_END), + (void *)KASAN_SHADOW_END); load_cr3(init_top_pgt); __flush_tlb_all(); /* - * kasan_zero_page has been used as early shadow memory, thus it may - * contain some garbage. Now we can clear and write protect it, since - * after the TLB flush no one should write to it. + * kasan_early_shadow_page has been used as early shadow memory, thus + * it may contain some garbage. Now we can clear and write protect it, + * since after the TLB flush no one should write to it. */ - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); for (i = 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; @@ -380,8 +383,8 @@ void __init kasan_init(void) prot = __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC); pgprot_val(prot) &= __default_kernel_pte_mask; - pte = __pte(__pa(kasan_zero_page) | pgprot_val(prot)); - set_pte(&kasan_zero_pte[i], pte); + pte = __pte(__pa(kasan_early_shadow_page) | pgprot_val(prot)); + set_pte(&kasan_early_shadow_pte[i], pte); } /* Flush TLBs again to be sure that write protection applied. */ __flush_tlb_all(); diff --git a/arch/xtensa/mm/kasan_init.c b/arch/xtensa/mm/kasan_init.c index 6b95ca43aec0..1734cda6bc4a 100644 --- a/arch/xtensa/mm/kasan_init.c +++ b/arch/xtensa/mm/kasan_init.c @@ -24,12 +24,13 @@ void __init kasan_early_init(void) int i; for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_zero_pte + i, - mk_pte(virt_to_page(kasan_zero_page), PAGE_KERNEL)); + set_pte(kasan_early_shadow_pte + i, + mk_pte(virt_to_page(kasan_early_shadow_page), + PAGE_KERNEL)); for (vaddr = 0; vaddr < KASAN_SHADOW_SIZE; vaddr += PMD_SIZE, ++pmd) { BUG_ON(!pmd_none(*pmd)); - set_pmd(pmd, __pmd((unsigned long)kasan_zero_pte)); + set_pmd(pmd, __pmd((unsigned long)kasan_early_shadow_pte)); } early_trap_init(); } @@ -80,13 +81,16 @@ void __init kasan_init(void) populate(kasan_mem_to_shadow((void *)VMALLOC_START), kasan_mem_to_shadow((void *)XCHAL_KSEG_BYPASS_VADDR)); - /* Write protect kasan_zero_page and zero-initialize it again. */ + /* + * Write protect kasan_early_shadow_page and zero-initialize it again. + */ for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_zero_pte + i, - mk_pte(virt_to_page(kasan_zero_page), PAGE_KERNEL_RO)); + set_pte(kasan_early_shadow_pte + i, + mk_pte(virt_to_page(kasan_early_shadow_page), + PAGE_KERNEL_RO)); local_flush_tlb_all(); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); /* At this point kasan is fully initialized. Enable error messages. */ current->kasan_depth = 0; diff --git a/include/linux/kasan.h b/include/linux/kasan.h index b66fdf5ea7ab..ec22d548d0d7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -14,13 +14,13 @@ struct task_struct; #include #include -extern unsigned char kasan_zero_page[PAGE_SIZE]; -extern pte_t kasan_zero_pte[PTRS_PER_PTE]; -extern pmd_t kasan_zero_pmd[PTRS_PER_PMD]; -extern pud_t kasan_zero_pud[PTRS_PER_PUD]; -extern p4d_t kasan_zero_p4d[MAX_PTRS_PER_P4D]; +extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; +extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE]; +extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD]; +extern pud_t kasan_early_shadow_pud[PTRS_PER_PUD]; +extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; -int kasan_populate_zero_shadow(const void *shadow_start, +int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); static inline void *kasan_mem_to_shadow(const void *addr) diff --git a/mm/kasan/init.c b/mm/kasan/init.c index c7550eb65922..2b21d3717d62 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -30,13 +30,13 @@ * - Latter it reused it as zero shadow to cover large ranges of memory * that allowed to access, but not handled by kasan (vmalloc/vmemmap ...). */ -unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; +unsigned char kasan_early_shadow_page[PAGE_SIZE] __page_aligned_bss; #if CONFIG_PGTABLE_LEVELS > 4 -p4d_t kasan_zero_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; +p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; static inline bool kasan_p4d_table(pgd_t pgd) { - return pgd_page(pgd) == virt_to_page(lm_alias(kasan_zero_p4d)); + return pgd_page(pgd) == virt_to_page(lm_alias(kasan_early_shadow_p4d)); } #else static inline bool kasan_p4d_table(pgd_t pgd) @@ -45,10 +45,10 @@ static inline bool kasan_p4d_table(pgd_t pgd) } #endif #if CONFIG_PGTABLE_LEVELS > 3 -pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; +pud_t kasan_early_shadow_pud[PTRS_PER_PUD] __page_aligned_bss; static inline bool kasan_pud_table(p4d_t p4d) { - return p4d_page(p4d) == virt_to_page(lm_alias(kasan_zero_pud)); + return p4d_page(p4d) == virt_to_page(lm_alias(kasan_early_shadow_pud)); } #else static inline bool kasan_pud_table(p4d_t p4d) @@ -57,10 +57,10 @@ static inline bool kasan_pud_table(p4d_t p4d) } #endif #if CONFIG_PGTABLE_LEVELS > 2 -pmd_t kasan_zero_pmd[PTRS_PER_PMD] __page_aligned_bss; +pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD] __page_aligned_bss; static inline bool kasan_pmd_table(pud_t pud) { - return pud_page(pud) == virt_to_page(lm_alias(kasan_zero_pmd)); + return pud_page(pud) == virt_to_page(lm_alias(kasan_early_shadow_pmd)); } #else static inline bool kasan_pmd_table(pud_t pud) @@ -68,16 +68,16 @@ static inline bool kasan_pmd_table(pud_t pud) return 0; } #endif -pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; +pte_t kasan_early_shadow_pte[PTRS_PER_PTE] __page_aligned_bss; static inline bool kasan_pte_table(pmd_t pmd) { - return pmd_page(pmd) == virt_to_page(lm_alias(kasan_zero_pte)); + return pmd_page(pmd) == virt_to_page(lm_alias(kasan_early_shadow_pte)); } -static inline bool kasan_zero_page_entry(pte_t pte) +static inline bool kasan_early_shadow_page_entry(pte_t pte) { - return pte_page(pte) == virt_to_page(lm_alias(kasan_zero_page)); + return pte_page(pte) == virt_to_page(lm_alias(kasan_early_shadow_page)); } static __init void *early_alloc(size_t size, int node) @@ -92,7 +92,8 @@ static void __ref zero_pte_populate(pmd_t *pmd, unsigned long addr, pte_t *pte = pte_offset_kernel(pmd, addr); pte_t zero_pte; - zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_zero_page)), PAGE_KERNEL); + zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_early_shadow_page)), + PAGE_KERNEL); zero_pte = pte_wrprotect(zero_pte); while (addr + PAGE_SIZE <= end) { @@ -112,7 +113,8 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr, next = pmd_addr_end(addr, end); if (IS_ALIGNED(addr, PMD_SIZE) && end - addr >= PMD_SIZE) { - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -145,9 +147,11 @@ static int __ref zero_pud_populate(p4d_t *p4d, unsigned long addr, if (IS_ALIGNED(addr, PUD_SIZE) && end - addr >= PUD_SIZE) { pmd_t *pmd; - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -181,12 +185,14 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, pud_t *pud; pmd_t *pmd; - p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); + p4d_populate(&init_mm, p4d, + lm_alias(kasan_early_shadow_pud)); pud = pud_offset(p4d, addr); - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); pmd_populate_kernel(&init_mm, pmd, - lm_alias(kasan_zero_pte)); + lm_alias(kasan_early_shadow_pte)); continue; } @@ -209,13 +215,13 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, } /** - * kasan_populate_zero_shadow - populate shadow memory region with - * kasan_zero_page + * kasan_populate_early_shadow - populate shadow memory region with + * kasan_early_shadow_page * @shadow_start - start of the memory range to populate * @shadow_end - end of the memory range to populate */ -int __ref kasan_populate_zero_shadow(const void *shadow_start, - const void *shadow_end) +int __ref kasan_populate_early_shadow(const void *shadow_start, + const void *shadow_end) { unsigned long addr = (unsigned long)shadow_start; unsigned long end = (unsigned long)shadow_end; @@ -231,7 +237,7 @@ int __ref kasan_populate_zero_shadow(const void *shadow_start, pmd_t *pmd; /* - * kasan_zero_pud should be populated with pmds + * kasan_early_shadow_pud should be populated with pmds * at this moment. * [pud,pmd]_populate*() below needed only for * 3,2 - level page tables where we don't have @@ -241,21 +247,25 @@ int __ref kasan_populate_zero_shadow(const void *shadow_start, * The ifndef is required to avoid build breakage. * * With 5level-fixup.h, pgd_populate() is not nop and - * we reference kasan_zero_p4d. It's not defined + * we reference kasan_early_shadow_p4d. It's not defined * unless 5-level paging enabled. * * The ifndef can be dropped once all KASAN-enabled * architectures will switch to pgtable-nop4d.h. */ #ifndef __ARCH_HAS_5LEVEL_HACK - pgd_populate(&init_mm, pgd, lm_alias(kasan_zero_p4d)); + pgd_populate(&init_mm, pgd, + lm_alias(kasan_early_shadow_p4d)); #endif p4d = p4d_offset(pgd, addr); - p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); + p4d_populate(&init_mm, p4d, + lm_alias(kasan_early_shadow_pud)); pud = pud_offset(p4d, addr); - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -350,7 +360,7 @@ static void kasan_remove_pte_table(pte_t *pte, unsigned long addr, if (!pte_present(*pte)) continue; - if (WARN_ON(!kasan_zero_page_entry(*pte))) + if (WARN_ON(!kasan_early_shadow_page_entry(*pte))) continue; pte_clear(&init_mm, addr, pte); } @@ -480,7 +490,7 @@ int kasan_add_zero_shadow(void *start, unsigned long size) WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE))) return -EINVAL; - ret = kasan_populate_zero_shadow(shadow_start, shadow_end); + ret = kasan_populate_early_shadow(shadow_start, shadow_end); if (ret) kasan_remove_zero_shadow(shadow_start, size >> KASAN_SHADOW_SCALE_SHIFT); From patchwork Mon Nov 19 17:26:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B73614DB for ; Mon, 19 Nov 2018 17:27:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10C5329C87 for ; Mon, 19 Nov 2018 17:27:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 03BEC2A2B5; Mon, 19 Nov 2018 17:27:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D05429C87 for ; Mon, 19 Nov 2018 17:27:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F1CC6B1B85; Mon, 19 Nov 2018 12:27:09 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1CAA06B1B87; Mon, 19 Nov 2018 12:27:09 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 044186B1B88; Mon, 19 Nov 2018 12:27:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id A05266B1B85 for ; Mon, 19 Nov 2018 12:27:08 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id z14-v6so33640376wrh.23 for ; Mon, 19 Nov 2018 09:27:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nn1bGI0lAecZVd/Nr8NjkoGpex3MA1HD9KRsJD9iBaE=; b=b3u2yQjnaZi+nXJy8PkQaS/Q0FOy6OR2F3lPRPMBcfbGBjrr9f510Y98XbSze1Hlu5 0GuK/Dc1r3+K/7mxRg6/tn8x44OAiD9AqKC9tb3sGVRP8QTMRGN7LNsZnC72BFio871W FQwwcvF0CeKD883NONrz5zL/HSOmB0/+IN5UjjjqrKHszSrT4PWagr7/i65+vRuZ//X9 IxWPolMaUsjD+sywhUTocrWTh/zxHicrXRHFoZ13feqlaFVGDcQevYr9M7DHXpI82ztg WkyM4bbOA6M5HiUUvWItYy7MCJrJSmLrDp7vvq3LCwJbZuAMn6iW9jEQmwMshAXZGyzl AFXQ== X-Gm-Message-State: AA+aEWbOKdOwoNBvCfVQBr9SOrrQ8DjOlB/jhCOCYXtkvu3TtMmWATIw wWu8MpkvyFIJJpo/pjTa3AaHNahn3sQmy9bGt40O5a79iCcsKK8w1RxhFHrAPxToYOP99neBGof 7vd8u3HMDJqQvJaGMYs1hvyjZ2Bt7vWapnDdJGtT80vhjzsH8chRaR/V2WXjSr9pIiqAbbpqbRp /b4ohw6vIosef6ifmKjBphdmxuWMc877MMkGUVpSzHnkJUJAGMwPBQfoR8wyfAD4YlAkh6DMEgz 1pwfBPO/Bh42RSqE056pZI6tb83Sv+124G5k9qdPijSMxyo8Q6eV2KjimtBOGSKmxrfWhVfFboW 4xp/1C3UmSP4WazoOtdNzChiADHUhw6Xo382j+wnghQlctxVZXCuq1iVchHtfKt1TsQwEbkn8BJ z X-Received: by 2002:a1c:3c83:: with SMTP id j125-v6mr7630256wma.65.1542648428158; Mon, 19 Nov 2018 09:27:08 -0800 (PST) X-Received: by 2002:a1c:3c83:: with SMTP id j125-v6mr7630194wma.65.1542648427182; Mon, 19 Nov 2018 09:27:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648427; cv=none; d=google.com; s=arc-20160816; b=VETTIp93QXRaoGKHkKfQSgS/QwCFLNtTA/CmSthfY73drOeBuafw/fTUhotdNAOfzN rWTH88M6if2GxA2ip6VXyYTRxtTuvt1esmlHeChWy+jLmsUToxxNe7WyHo+CJzTxtbag K92XOUmkxu62bgEmv2OmtuQYEPmhzThJ0SQ9db95cxmcnEgnOXUwKF+DVPlpoFl+uTor suodnEsIxCh03pto1al6P1RC+KGvUOnkaD0XiaIQqLzh5djkOhgk7fzvmhehzRO+DuU1 AYCJvhdTJiE9Hhp8Djr+nYpKwWy9UDABmHqxS3DuWVUM3MU78fUweLoNM/fGGBhE6wMn awFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=nn1bGI0lAecZVd/Nr8NjkoGpex3MA1HD9KRsJD9iBaE=; b=bESSR7AdMHc3bWLdRzaQJZMBB6WBAlF29CGLiOBcbGzjPvnFZa4vI+YQCLeDCXC/tw oYqtCVb8sqLi+GWr76ZVylT1tkVzt2raFpc4vrzelYVsTVuSgKwWmcWrSkBlyVvjcVu3 w7C5nBcz8Pt851U4XkaaIRHxOiXhTnx3+anglgj9uORh6DafYeNmWXydcevqFlYwt9au SPd4O8Orcm1JRn1Kglky0GsAoDBEmf+NolMAaBT6kHRzq2ZJ0BCc7DPDipggwqwgTQDp LudhFyzxtMX1000WaktBUoNyqP9A/ekQI3OlL0MHCJxZXx38UhTH3IvvNyeyBGTW+EzV QFaw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=al3YzVeC; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g5sor12797870wme.13.2018.11.19.09.27.07 for (Google Transport Security); Mon, 19 Nov 2018 09:27:07 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=al3YzVeC; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nn1bGI0lAecZVd/Nr8NjkoGpex3MA1HD9KRsJD9iBaE=; b=al3YzVeCj0YHhNT6cg0Cjcm8u0ncAy2aMGzZTXZmBtegGLPp2r2vS/AJZTwT7SL1V1 G7ymRvAYBgYbSCnWsgyJUCkbY4M087bjjhgwKwffeoOJZkm4H5MVbig26lZWKrrpyugY laWJcgG0cgDzgtN8+NEf/Iha4W6cl+INqnjWh0rK/vi5RvWuvaAN0qJwoj7nuwHz9FO8 PX7IhcrXscvAGA8VE5Nd0vgHVsO/z9IDKSL/xCCwkBoK1bB+u0M0vt2z9yHPeByUETUh +FkaBUwb8g+m4sfCg6fphwjcewjH/c/0YrgivC2giSPSBTQYyTpIfbyOo3rG2wmlx6FG qX6A== X-Google-Smtp-Source: AFSGD/WQMm489HnpI1eyiDD7CjXlAqXEJysYrdjjAsIWTnQ1RLMBImBnpeCFI1436TFPYvZWdfHBfw== X-Received: by 2002:a1c:7601:: with SMTP id r1mr7770386wmc.98.1542648426679; Mon, 19 Nov 2018 09:27:06 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:05 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 08/24] kasan: initialize shadow to 0xff for tag-based mode Date: Mon, 19 Nov 2018 18:26:24 +0100 Message-Id: <1a06035a7777dedcc93680ca739c481fb068073d.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A tag-based KASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with tag-based KASAN we need to initialize shadow memory to 0xff. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 15 +++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 4ebc19422931..7a4a0904cac8 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -43,6 +43,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) return __pa(p); } +static phys_addr_t __init kasan_alloc_raw_page(int node) +{ + void *p = memblock_alloc_try_nid_raw(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, + node); + return __pa(p); +} + static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { @@ -92,7 +101,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, do { phys_addr_t page_phys = early ? __pa_symbol(kasan_early_shadow_page) - : kasan_alloc_zeroed_page(node); + : kasan_alloc_raw_page(node); + if (!early) + memset(__va(page_phys), KASAN_SHADOW_INIT, PAGE_SIZE); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -239,7 +250,7 @@ void __init kasan_init(void) pfn_pte(sym_to_pfn(kasan_early_shadow_page), PAGE_KERNEL_RO)); - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index ec22d548d0d7..c56af24bd3e7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -153,6 +153,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 + void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); @@ -163,4 +165,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_SW_TAGS */ + #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 5f68c93734ba..7134e75447ff 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -473,11 +473,12 @@ int kasan_module_alloc(void *addr, size_t size) ret = __vmalloc_node_range(shadow_size, 1, shadow_start, shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, + GFP_KERNEL, PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, __builtin_return_address(0)); if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); find_vm_area(addr)->flags |= VM_KASAN; kmemleak_ignore(ret); return 0; From patchwork Mon Nov 19 17:26:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A39E14E2 for ; Mon, 19 Nov 2018 17:27:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F3BA2A2AF for ; Mon, 19 Nov 2018 17:27:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2FF0B2A2B9; Mon, 19 Nov 2018 17:27:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF1912A2AF for ; Mon, 19 Nov 2018 17:27:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BDD846B1B87; Mon, 19 Nov 2018 12:27:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BB9426B1B88; Mon, 19 Nov 2018 12:27:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A06946B1B89; Mon, 19 Nov 2018 12:27:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 4A48D6B1B87 for ; Mon, 19 Nov 2018 12:27:10 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id w1-v6so41394767wrr.3 for ; Mon, 19 Nov 2018 09:27:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FwKg6/9dBzGs42zHBzot5G3OJIBxXRkYl3JEBLj4VCI=; b=sLVho+DMzlHoydIrx6iaCE+FQBLTI6XqL+uA27NPEfNBqqqL7xu0P/kUJVLvAf7rAm +HF8stVv/mm2i99/osXBci/qlNqNIJ3yLfUsgzTcFtP8j9Wvv0gvffLyvcF70OT9g70m piGnHE9ni4gZnHSdGlBun4fYB/1cmBtiOK3wrtffPhaNeHX4tpeudqaPNtPti45U5Rev 1at/IKu2RPO1+T2hoN0F7mXSmj+CzqT+gB097z4nF/tL9qA9zF+qKSBypaS5PCk+qg4X yTZ89E8LjY62GItoKEtkqZBCljHg7BVM8+8wPoaaXlTaYzZljw/2gdJFg0SpaflcPCa7 lGNw== X-Gm-Message-State: AGRZ1gL+zGQb0eo4jKVd51WiZ/1dOULm3UzCvwdzwD0o24R2cr302rTN +nStLHinGjMPnap+rp870mAeelXaseTrSOtcsEqvr/ik+/MV/AYuCkmM+ZJLBFpgB9l3Ox2p7/5 Ypt+7ba9Sv6qy4JPCdi8uGtG2xm4rYBLwPiVXF+PUEl3gzR50t7sL7DiuOUQMXsYqnNyy3GTouc qLq21FekI5RnLvfAMmxd0fsRLBNOxFZF1PdyvkCpJL0slo0KKxmFHjpwSuQQ0iFivph0E8BtK7q 2y84wEJ2akYIZecE97GIdhjBr64JAGezLAxy4Rv0nWvSy3OXBThYlSLFl5RkbmI7G7pGJ3ndupR c/HLarZdIAUG2rEgfJxJhXhZpybwizNKdqx38ZTUhmXumLxqnH5tidyf6BopCUMRV4GsSHGA/AW X X-Received: by 2002:a7b:ca44:: with SMTP id m4mr7741598wml.76.1542648429845; Mon, 19 Nov 2018 09:27:09 -0800 (PST) X-Received: by 2002:a7b:ca44:: with SMTP id m4mr7741540wml.76.1542648429001; Mon, 19 Nov 2018 09:27:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648428; cv=none; d=google.com; s=arc-20160816; b=Rqh/vfPJ8QQG4cwCRxOhtXkRFuMaJQa6ydP/WDVtJm4vpwJro64N7fx6JxYaiLo2BM GiJdyGNTW99MqRx/C8mJeX8L3A9x/CD2sRs5oyj62wudWHxb7C9ZpSp6mVQ6o5Wg2u85 etBeoUHhvr0yvlp2Y8l+zz076Vi0cfcaL/jSUA46HmB9q5dqPYWEXG7cu6PJl8fq/WSA tdB8iSatHU3+1nw/dmUSTB3CGNlx+kCgvoio1N74aogBg0m2HvM1xSm8Sx1Sud6AgQmz V74/8LiYjF0HQ5vGN6z3l9dwizBIJKTp/usNER9LKsf1RCmAVYP/ztpqNx3yvnzUzOx4 fxzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=FwKg6/9dBzGs42zHBzot5G3OJIBxXRkYl3JEBLj4VCI=; b=oT85roBoEAaHz/S1UOnuFg29mgyNIdatjnWlvomhj37ajEkMqMoZL2UHGCLTx6qZft WMsnMfQysNLKIeAJbf4bawiFiyOsRkAbOPnMHKgKyklO3Ht0RBpPnSIe8DiTjynu9U2u Ty4Z4Rj9NUvKZ/MNf51XZ7oxeCdfgHFSufDSVZX9R5hM0mn3Gghb0SKNS6fLqpkAs/+7 9YugY3O57CNkC6gQT17W6XcMmMYdLVpwoL3fHTeXZqWGW5YWWpHXVOKJnFoVavW98dZK fVubAtewomcr63p2VNdAkcC9C0trUOTvxjCJgM8rnU5f9dS3XNoAJbiMO30LEmpuDhF2 YzJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XRgFtW78; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y11-v6sor26325308wrp.50.2018.11.19.09.27.08 for (Google Transport Security); Mon, 19 Nov 2018 09:27:08 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XRgFtW78; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FwKg6/9dBzGs42zHBzot5G3OJIBxXRkYl3JEBLj4VCI=; b=XRgFtW78jY8uE+DLOcRj01/ohwGwYRVTHOZm6kkIX3wG78z6h1mGRsYa7WH0kSmPHo AgJRvOnjXH+ryNrZV51t7wuW1nLUfyd7dVxMwdnHcE1gRgWDB5jR5a/noGVOeHj4w0B/ 8yuVzlMIRzVf2cnjqzvYbrpnoyo7e5Z1CXPyIYr53fB2+Pb1yBVNk+oDQlS/I+ZOm6do prUWxiNXpr1bUnZo7OoJTcsYxMaIWv4nn4nKD3LFFs3CROs/kb+6dwYChGBQKiN105SH 5TdFEGUtGrIXnGkfkToX+7nV2GhomixqpcmF0F6aS6F+opCc3zhUMsY2CplJVtJK3cYR otIg== X-Google-Smtp-Source: AFSGD/WHNP7mi0wPd7mMILK100bob8aXhZpnIiD+3cKXrZDRq6mGPKEPk5lZq9uwmj5IXWtJqjVhRQ== X-Received: by 2002:adf:a357:: with SMTP id d23mr5758176wrb.195.1542648428411; Mon, 19 Nov 2018 09:27:08 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:07 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 09/24] arm64: move untagged_addr macro from uaccess.h to memory.h Date: Mon, 19 Nov 2018 18:26:25 +0100 Message-Id: <0288334225edc99d98d70c896494e19c3bd9361a.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Move the untagged_addr() macro from arch/arm64/include/asm/uaccess.h to arch/arm64/include/asm/memory.h to be later reused by KASAN. Also make the untagged_addr() macro accept all kinds of address types (void *, unsigned long, etc.). This allows not to specify type casts in each place where the macro is used. This is done by using __typeof__. Signed-off-by: Andrey Konovalov Acked-by: Mark Rutland --- arch/arm64/include/asm/memory.h | 8 ++++++++ arch/arm64/include/asm/uaccess.h | 7 ------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 05fbc7ffcd31..deb95be44392 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -73,6 +73,14 @@ #define KERNEL_START _text #define KERNEL_END _end +/* + * When dealing with data aborts, watchpoints, or instruction traps we may end + * up with a tagged userland pointer. Clear the tag to get a sane pointer to + * pass on to access_ok(), for instance. + */ +#define untagged_addr(addr) \ + (__typeof__(addr))sign_extend64((__u64)(addr), 55) + /* * Generic and tag-based KASAN require 1/8th and 1/16th of the kernel virtual * address space for the shadow region respectively. They can bloat the stack diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 07c34087bd5e..281a1e47263d 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -96,13 +96,6 @@ static inline unsigned long __range_ok(const void __user *addr, unsigned long si return ret; } -/* - * When dealing with data aborts, watchpoints, or instruction traps we may end - * up with a tagged userland pointer. Clear the tag to get a sane pointer to - * pass on to access_ok(), for instance. - */ -#define untagged_addr(addr) sign_extend64(addr, 55) - #define access_ok(type, addr, size) __range_ok(addr, size) #define user_addr_max get_fs From patchwork Mon Nov 19 17:26:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689031 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E83B14DB for ; Mon, 19 Nov 2018 17:27:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3195E2A2B5 for ; Mon, 19 Nov 2018 17:27:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 24CC42A2BA; Mon, 19 Nov 2018 17:27:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 725592A2B5 for ; Mon, 19 Nov 2018 17:27:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 093936B1B89; Mon, 19 Nov 2018 12:27:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 020AA6B1B8A; Mon, 19 Nov 2018 12:27:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBA336B1B8B; Mon, 19 Nov 2018 12:27:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 846226B1B89 for ; Mon, 19 Nov 2018 12:27:12 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id v2-v6so44124531wrn.0 for ; Mon, 19 Nov 2018 09:27:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=36j1+BtpF6G5yltk+oqDjUfwUjm2MmQLog+fUAQOOBs=; b=XSxASlniRkt73ZOtMGVe7Xe4rYjBFtoTBnNuhFECA36hoNPBQmiPXJmBB9pbx+Nw7X trPXGIDyeMS+1k7exq5OME77tc6Z36q/ZWWb30TUyG9znL0u0INkeEmOiXq3sui25eI0 qLO4KTp8XfmmyY38gxWk3U9bAdpjG1XWHjsdG2/o3vGcYfcK3/wyUmOgD/x1QEjenvku m1RtLZhRbd9fDDseJC6Lm4KZQOlp8fuU3jWSmguR5ag1vqG1FcidXH/r4zRYt5P1PfZM 5RGKb0FCEUCMMnRWjz6/H5zE51n+A1NoaWdAMie8ZcZ+G3vurGYLOCmD3ZngGlofVLY7 Y/mg== X-Gm-Message-State: AA+aEWZi1+N1Ou6z6xTbUBeoQEH/tm+xKesgBhMufG/9F7/j8gFHzV+N 9qE+oudbTJ3JRsHXGfu0zJGQJho2iA9LqaZBR959hmEYvv4c6g47tASnTdgQFP0xkzwtsBj05i7 mhcScw4Zf8tPCjBDWyczFifHlGXjZP5jDS/h2DGSZoQG8pQmgiMLrZ4ndJWbTpMf0zbK8zeBIVY RVzQ2Ejg2GCDp0sC49iJ1Sz2O3v4c/D6KOX40zodG6K+nMiwjqa0dZqCIGbPT7XlWgxV7NUSblE WMXhwsmfvVuZh20zzRnwQMu50ReX9ng/Dcy1hiQZn3SPTneKkbsiIZkcrkl2fnU6Ok/RRWL4kaR t1NkZJMx1owJtCVTlFWLJG65oVq/IYFfE1YsyDh3L6YIgWD7kp0gTGLqtDoJUcUeO0HucdqdyfJ N X-Received: by 2002:a5d:4d11:: with SMTP id z17mr9377119wrt.209.1542648432036; Mon, 19 Nov 2018 09:27:12 -0800 (PST) X-Received: by 2002:a5d:4d11:: with SMTP id z17mr9377057wrt.209.1542648430825; Mon, 19 Nov 2018 09:27:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648430; cv=none; d=google.com; s=arc-20160816; b=iqa2onE9Tc0OBeAVONwur3bQm2T3O3lODbIIpnrEpv+QQGWfRoQw89dLtAigJpRHvu iHq10uQrbcVKM3aJNtntnIWKtAu6vq7ME5ftuArSYpYNlfWA8TBeGUKJMaIpUyyhI3+j TYeEUc75MYSb3G/iPyXWl04+e8Q7ZFzGlElwHScaJX/xMIacvp/iL54YVCsK9/ozthog llOpNYzEJFnZhqygepIZ/cU7LJ3okZT55cSbHAIfOCUD2yY0HVLrj3GfYBEfoKqDU600 pRpzj01r+hdjuQKAO4WiXTwr2XoplSynZPzPz6eInvKTZ/6eti1yteiv3+UYb7+4wsBN Z71w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=36j1+BtpF6G5yltk+oqDjUfwUjm2MmQLog+fUAQOOBs=; b=Gw+iohB3qEzSVNf0733WZuLaeALA8JNzDdZM3Xpf9n3MeG7hVwxX24/9X8uZMclBHD Ao/ocu0PXSplJcARvwNPeCxgK+Xkqwy10b4aq6ys20f04PCwLeH2EYBkL2/cEz2DMMtB MLvqeyTk+yBprK6tMn10ptZs9aTtB32MuvovQaa0LqZL2ZClaWu/OFW9xmt8HvjenOb5 S4D5S+MAfJckD5ZWnqOM4DpN5DMennu92kjyZlODkBEkv8qQFb7EjKiJ8qrYdy0seOQz dyYA7vMYcPuVYPnPTI5h6iegqV+M0/31ikMI4xVo8y86mfCuBcuFnAtYBnpjK70P4zj7 WYzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=NasRMxqy; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h7sor13717248wrv.20.2018.11.19.09.27.10 for (Google Transport Security); Mon, 19 Nov 2018 09:27:10 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=NasRMxqy; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=36j1+BtpF6G5yltk+oqDjUfwUjm2MmQLog+fUAQOOBs=; b=NasRMxqyha+Xd9iuibAb2YbfD+HoajSfdJqzco6qTgfSvXlNho6IfTWpkQlZfo4YWu 58chFaXtBsmRpi2wyQsdMFhXIjYrkhHzy1mutLq55EKMsJpzlvbUQKKBUz5jxXZl+RFs RkAjSyj5JpQ0LcmJBUc3FI/8evs2zK/OK0/7eYqo3tFklzRCqZOwN10grCQUD9vwGlpK OdKReepNlqyPJPBso8Tpv6O/X7jNBHvSCroShAKgsdJ/YmNcYdj2fGClRgVlTBoeBCi9 r7k2rvP2RbMekFDOf4TNxYBNMCiJJtoXoyxN2ceN+F9/LBd353I1pYMtV97KVONfpn+q /aLA== X-Google-Smtp-Source: AFSGD/VT7zJxk+nz+XGziv7S3QWnmqIb23NvbGIPrptEW3TSo7Rpwp5pKmLi/+1gj0a4bH2GfUAw7g== X-Received: by 2002:adf:8544:: with SMTP id 62mr9024122wrh.1.1542648430253; Mon, 19 Nov 2018 09:27:10 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:09 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 10/24] kasan: add tag related helper functions Date: Mon, 19 Nov 2018 18:26:26 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/kasan.h | 8 +++++-- arch/arm64/include/asm/memory.h | 12 +++++++++++ arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 13 ++++++++++++ mm/kasan/kasan.h | 31 +++++++++++++++++++++++++++ mm/kasan/tags.c | 37 +++++++++++++++++++++++++++++++++ 6 files changed, 101 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h index 8758bb008436..b52aacd2c526 100644 --- a/arch/arm64/include/asm/kasan.h +++ b/arch/arm64/include/asm/kasan.h @@ -4,12 +4,16 @@ #ifndef __ASSEMBLY__ -#ifdef CONFIG_KASAN - #include #include #include +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) + +#ifdef CONFIG_KASAN + /* * KASAN_SHADOW_START: beginning of the kernel virtual addresses. * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses, diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index deb95be44392..ae741827039c 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -94,6 +94,18 @@ #define KASAN_THREAD_SHIFT 0 #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define __tag_shifted(tag) ((__u64)(tag) << 56) +#define __tag_set(addr, tag) (__typeof__(addr))( \ + ((__u64)(addr) & ~__tag_shifted(0xff)) | __tag_shifted(tag)) +#define __tag_reset(addr) untagged_addr(addr) +#define __tag_get(addr) (__u8)((__u64)(addr) >> 56) +#else +#define __tag_set(addr, tag) (addr) +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif + #define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT) /* diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 7a4a0904cac8..1df536bdabcb 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -253,6 +253,8 @@ void __init kasan_init(void) memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); + kasan_init_tags(); + /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; pr_info("KernelAddressSanitizer initialized\n"); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index c56af24bd3e7..a477ce2abdc9 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -169,6 +169,19 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #define KASAN_SHADOW_INIT 0xFF +void kasan_init_tags(void); + +void *kasan_reset_tag(const void *addr); + +#else /* CONFIG_KASAN_SW_TAGS */ + +static inline void kasan_init_tags(void) { } + +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)addr; +} + #endif /* CONFIG_KASAN_SW_TAGS */ #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 19b950eaccff..6e42f46522c0 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -8,6 +8,10 @@ #define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ +#define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ +#define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ + #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ @@ -126,6 +130,33 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif +#ifdef CONFIG_KASAN_SW_TAGS + +u8 random_tag(void); + +#else + +static inline u8 random_tag(void) +{ + return 0; +} + +#endif + +#ifndef arch_kasan_set_tag +#define arch_kasan_set_tag(addr, tag) (void *)addr +#endif +#ifndef arch_kasan_reset_tag +#define arch_kasan_reset_tag(addr) (void *)addr +#endif +#ifndef arch_kasan_get_tag +#define arch_kasan_get_tag(addr) 0 +#endif + +#define set_tag(addr, tag) (void *)arch_kasan_set_tag(addr, tag) +#define reset_tag(addr) (void *)arch_kasan_reset_tag(addr) +#define get_tag(addr) arch_kasan_get_tag(addr) + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 04194923c543..1c4e7ce2e6fe 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -38,6 +38,43 @@ #include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void kasan_init_tags(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(prng_state, cpu) = get_random_u32(); +} + +/* + * If a preemption happens between this_cpu_read and this_cpu_write, the only + * side effect is that we'll give a few allocated in different contexts objects + * the same tag. Since tag-based KASAN is meant to be used a probabilistic + * bug-detection debug feature, this doesn't have significant negative impact. + * + * Ideally the tags use strong randomness to prevent any attempts to predict + * them during explicit exploit attempts. But strong randomness is expensive, + * and we did an intentional trade-off to use a PRNG. This non-atomic RMW + * sequence has in fact positive effect, since interrupts that randomly skew + * PRNG at unpredictable points do only good. + */ +u8 random_tag(void) +{ + u32 state = this_cpu_read(prng_state); + + state = 1664525 * state + 1013904223; + this_cpu_write(prng_state, state); + + return (u8)(state % (KASAN_TAG_MAX + 1)); +} + +void *kasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { From patchwork Mon Nov 19 17:26:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 896F114E2 for ; Mon, 19 Nov 2018 17:27:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C0FE2A2B5 for ; Mon, 19 Nov 2018 17:27:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5F17F2A2BA; Mon, 19 Nov 2018 17:27:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA9C52A2B5 for ; Mon, 19 Nov 2018 17:27:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96BC56B1B8B; Mon, 19 Nov 2018 12:27:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 946436B1B8C; Mon, 19 Nov 2018 12:27:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7499B6B1B8D; Mon, 19 Nov 2018 12:27:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 1C79F6B1B8B for ; Mon, 19 Nov 2018 12:27:14 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id x3so18443173wru.22 for ; Mon, 19 Nov 2018 09:27:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1YhKty0PhBPSl7VZmDwGA+CXyNSRUPKrtcyaCeImY3A=; b=c15DT+Wfo3eEqXbPzFCnGJslhu1k3Ai4iF2gelR7tdsQMSwiOxjo3RzdniSjMZaNcb RDsA6iQMBmKfD0tPBXHystVL5bdd3I5qygzoMErv5AiXPC2kJrK9Nd5ln/eo4ugJWI5Q tD3yBMCBljLFtdlS7nB+dRs/Qved/bIHXZ11qmwmXrbe0sMZjqZDaiQJJMkW2PrAbPvE RhIc+ZAKm6+yf7pwiYW3zaOHNr3IS2Eo+ubBJlsPakWz/TsHFDbc8KBhXX3aQknezYWZ 2JgZt4mkKvmwuHVmvPn8xq62x2RlXVHNOk+Xj/RJqdsf0lhxFXhTjBcOFErLaSbQc3nf IgnA== X-Gm-Message-State: AA+aEWY+7wdHEhnKPP34wYRUxGdkjZOrl/ZcAXP6tymZ57zqEG6qKEQY l6uPw/g3aIGdWNCpO+8gRbmnMlKpaYtWjE7P7DFeWZTfmRYTQp1G/QSC6IOMjcvEyC0/ywrESSd WOtsOjrkf+KxHenrrVGDg8xCaEvxGu5mBnXh235M/sm4yo7XqRwc27Bl670jO4qv/choNqgTmdM gVxKHFpDMfREpfNJJnOuBxtWai4gCttLsvOElOt16eHGix8GjAg+dLe8Jzn0Z/NPkv+poQIMoXI wzoyXQ50wqIoofwG/d+mztZBdmO6HoawWWegj1wJ3+G5cc4sTawaQx8l2iJL20tnGDLvKyw8cWW TjLKvYu8Qz75PXgDEmd/B/PpfBrd0A3xhqJV+SIJySJimd4OWXjvYEwgy9OGHlq7/WU7RRebypR f X-Received: by 2002:a5d:4609:: with SMTP id t9-v6mr9005467wrq.198.1542648433642; Mon, 19 Nov 2018 09:27:13 -0800 (PST) X-Received: by 2002:a5d:4609:: with SMTP id t9-v6mr9005430wrq.198.1542648432967; Mon, 19 Nov 2018 09:27:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648432; cv=none; d=google.com; s=arc-20160816; b=nJpFx0ulFgAFFPJvy4jxmDl4bMTo6mIeJT7GLTZZ2eSuCLnl+b5OJOWdDx3UNx5c2j kA+jzTWXXXfLGZIOE7bpBAj7i3Fz7L/uucEqlaA9DDXOWl+2/LpG1FfQsbQsLxhmORD4 A/cEaQgUrzW1pnAU5cnEKI+vMrZd8Ajz2Zp3Bcwea5Od/22a7EZUGX98PT3G3Pkq4tXa C+Lij6RsfWnsV0emH7ttB38W1pUNaYPa4YT5OzykgTDi72W6K4wbKRVxFemteGF7+L1a 6YSf/9a/iBQp+NGoZIQTFDv7VTD8MTUTOhFssbfK3Fo3NMnOj7P7gKauUp6oSMRsvdeQ Q71A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=1YhKty0PhBPSl7VZmDwGA+CXyNSRUPKrtcyaCeImY3A=; b=O67jw5/Z7+DRnzHkAPaMEjX7oZA0lhijXRrRDvEUXnqn0tNh/o0wj9s+8whvXnZVJu GfpaHSSkYRHbKeuavq0TZ6jBHzxzVGV/YI89XrGjZXCwd8f+K33/ajr3Ra49x4Cy3XoY u0/HRlmOvJiq9t+Eyz/xjPpK6lQvhkHMUJ1Qgo14ns++737PQowGn6jbVpHWKZtLFHVe gGzP/lzlCvsRJ7S8HmZlWjUOmr70AebpvBsA+ksF5QRFP/V5zSuHlsjFUgV7pjrtQ956 Kw43jLd8MrhqobEcWUyusmWHD+1NzVq7xTdlWsCBm+pz6+sHCYCQAcNusIN5KrAUrhC/ IMHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=DnG4e1wJ; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r2-v6sor18569538wma.5.2018.11.19.09.27.12 for (Google Transport Security); Mon, 19 Nov 2018 09:27:12 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=DnG4e1wJ; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1YhKty0PhBPSl7VZmDwGA+CXyNSRUPKrtcyaCeImY3A=; b=DnG4e1wJQP7UD6p6zIz57/Rv8pyRn6SQ8HJI0SVPFeiAkkFExKONMAX5F3DnTwyOah jlxMnrHVidMA0iBppEJreCsRSZSmTCkl/vQmtwv0zX6Sis02dF/q6YhQ0tuCy9MevN62 sQnySQ2k9tiaKs3VCMmHZXVu9sAg3dZd0ofuLPUfas+VFLIU3xWkBt67dN/j+gfvjNyn ADr69OpHpZ7rV9SYeeb6A5M6p0ilZbXkzmlesFgYR2iDng071A9Gvk8BwkHiV+uNzaQS iNToRyorfV5Ah8VewgqY0Tvh7DIQKHsYjgQbuZmxuHMlH5BYYFYK+ZXOLD1F3BODIn59 cHtQ== X-Google-Smtp-Source: AJdET5erzvm32b8ZF3d92FQfSjsREk6d9aLRHY/YmUcwUo+OxmSXKpVoDAiZpQ39XDdOFZWvSxgUJw== X-Received: by 2002:a7b:ca44:: with SMTP id m4mr7741735wml.76.1542648432514; Mon, 19 Nov 2018 09:27:12 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:11 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 11/24] kasan, arm64: untag address in _virt_addr_is_linear Date: Mon, 19 Nov 2018 18:26:27 +0100 Message-Id: <5cb565e6a6b17eceb6590a33d84208f725cf8dc2.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP virt_addr_is_linear (which is used by virt_addr_valid) assumes that the top byte of the address is 0xff, which isn't always the case with tag-based KASAN. This patch resets the tag in this macro. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index ae741827039c..ddad7df77027 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -322,9 +322,10 @@ static inline void *phys_to_virt(phys_addr_t x) #endif #endif -#define _virt_addr_is_linear(kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) -#define virt_addr_valid(kaddr) (_virt_addr_is_linear(kaddr) && \ - _virt_addr_valid(kaddr)) +#define _virt_addr_is_linear(kaddr) \ + (__tag_reset((u64)(kaddr)) >= PAGE_OFFSET) +#define virt_addr_valid(kaddr) \ + (_virt_addr_is_linear(kaddr) && _virt_addr_valid(kaddr)) #include From patchwork Mon Nov 19 17:26:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D59114DB for ; Mon, 19 Nov 2018 17:27:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F32D82A2B9 for ; Mon, 19 Nov 2018 17:27:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6B7A2A2C0; Mon, 19 Nov 2018 17:27:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 589C72A2BA for ; Mon, 19 Nov 2018 17:27:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C51FA6B1B8C; Mon, 19 Nov 2018 12:27:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C304C6B1B8D; Mon, 19 Nov 2018 12:27:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7C2F6B1B8E; Mon, 19 Nov 2018 12:27:16 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 4D5376B1B8C for ; Mon, 19 Nov 2018 12:27:16 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id z14-v6so33640882wrh.23 for ; Mon, 19 Nov 2018 09:27:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BnKWna4Fxza+RguSxuMKg2MKEZp1kGJKTvVJBWbpyy4=; b=cP09kk3UnsTQIEYCRU1P+e5D4Vgj/C0Nd+QrBz7EQwnWBBUOLwKROvQrEkVcBnCCH7 sUM4kyNl2gUnV601hpxPqE8z3F3nyt6MwP8CDnM013JhghgUahe6ZGAH+TnHdav5XTGR c3WBpkQkBJe0Fl8c8UFn3bj2dS/ZYFItsmxrVULNgU0q2ESKn1x+BCkbr2EJks2oUsyx XhVkfJTVmC59Z3LrSSqY6WHLvB73nHZZrLrPxHXWm+3yk3IFvrHTGwvmYqw0dxLh2Ch7 pDloZB2R0f4L9mmB8aX1q9gU8mv9gMtvVplOiySBFEkYMKhqIKrAXS3PFtt4kenFwf16 NRXQ== X-Gm-Message-State: AGRZ1gJDDUQKbEpulW/gDIgq10uTnRH3ozajZ+fl2DkVoRQX/4OstyZ4 kQKXTuTaFTxl7SK6XiQa541PsS2mODeFA1/9AnjfWcltPDGtc5vD93dgtM03LDt/banoyDv7RVA AK9XrWl2Yr8KkLBGP4QJaGRz+FY6bOyUz/QrPPt3C9VSVnm5ttIqJfb9OeL077tyLHVzxUBi62A x4l5SBKzUq+ANjokNYsqVzm03M0TToKQq1XlLrsL0DfsX+uhLvi1xRnnq9NJNuIJ2Er7X88oGyq l4RAuUpTAg5r/JB8ZTtw2xRH41tgT+UYz6RHc5erezQOadkfcfFWSb28lcooqVFmEYxb1yfrD2u zr89/6+JDjyLXvvO06CUEBCaSLiqonGOHnhyv+/bvnvTLTLVBKMQvYDIr80pxWsYmBmiyK7nRjF 1 X-Received: by 2002:adf:a78a:: with SMTP id j10-v6mr19703204wrc.286.1542648435803; Mon, 19 Nov 2018 09:27:15 -0800 (PST) X-Received: by 2002:adf:a78a:: with SMTP id j10-v6mr19703158wrc.286.1542648434969; Mon, 19 Nov 2018 09:27:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648434; cv=none; d=google.com; s=arc-20160816; b=RL6bX9waIWApt9mwkkQZ5VOX/JXvhR5nj830ThRSHAOC5I1ig36cmTk0Wa6NEEOGp7 LWz5OFAAaRUZ3xDGC6YksUNLzA8G2x03CVmF/rNklpihb4LvVKX9GDTCt4NT83TvfOHA RQh3C/TzxC+00KQvKgWwGrPtKZ3qXHc8rtRUeo5zAISjb414Mh6xMlL2Q4uqYWAKTSdg CrlZDKNvTl+In++RKuQ/nxA3taI+AQnL3w9ERxzPJHVwfKDfkiHZvQUOX9ADw9M1FalX 6ZVE427NQAhOynFnwFvGxzObhO1bmiNO+WVHXBP9fQAM0BWM0SVTpYNNbuMwPM1YFmSm arDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=BnKWna4Fxza+RguSxuMKg2MKEZp1kGJKTvVJBWbpyy4=; b=RUMI05zpmLFLosPWwLWRYX9aPfjLIAQkX3S/GhRVe84QARhM+OBSfYweiWUDzFyB2K MyI86hnQsQ3vbQhX70ShAIpK8KKQsuY+kbUPkhAcEwwWMNn0ALUP2X002qDEf1V1xxUB b2AB4Sv40JJLOotyR+Rs54OQsyQllb6kUVNFK2qWSliFP49zcYa7ZYxcjfVW/YJSmXGk Zj+NReYxNp3odS/luEAd8q/clVkuENgXy5hs1Hf1LE6Q1rYoFTZYrhH2Ko/f3ho0xj+r oedUQ6UOzE32TU6DxAiZZnXrYHAjyTbQKvKGFrmQndWb9rpqA3nz5ocPA+6wQOZWK5Ah bOuQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=oRAsbIGT; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j20-v6sor25571546wrd.49.2018.11.19.09.27.14 for (Google Transport Security); Mon, 19 Nov 2018 09:27:14 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=oRAsbIGT; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BnKWna4Fxza+RguSxuMKg2MKEZp1kGJKTvVJBWbpyy4=; b=oRAsbIGTG+DwdIgakoFqgS2wwrqiL6u/3qXf2aYV0B+JOFEAMq8PpX7k8RvwVAtj4P FUkJKTZgCXSTOUJZ96XPqNLsLArvFJV8e/2Mwfi5PzxQTl5c6Ij/T0z+jD8ds9xbc7OC Y6JM518vNZqALTyHZXgOsbMJ1wXH+v43rtH+RrV7GySWQIeo/fe27OKiy4I27AKb7P9B XACJCvq2frJxIebmbJZLXrXn+OFI4j40xJR99lfzw4S8BP0aKWfMrHSt6ND8ktpE/5O0 CZ/dPz+wp8hSeEJFMRKQPluNjPND6wZqrgAww9XaoKZqECi2zg4rdly+JhkCid9XqASH k+Fw== X-Google-Smtp-Source: AJdET5eOHsG65bvoFQR6qqSubK3f30AO88X+3m5RRAiWi3EigIHxArp4k3X75wmHUzj3gP5G1V3N1Q== X-Received: by 2002:adf:a409:: with SMTP id d9-v6mr19711186wra.190.1542648434312; Mon, 19 Nov 2018 09:27:14 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:13 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 12/24] kasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU Date: Mon, 19 Nov 2018 18:26:28 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor. The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it. Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/slab.c | 2 +- mm/slub.c | 24 ++++++++++++++---------- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index 26f60a22e5e0..27859fb39889 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2574,7 +2574,7 @@ static void cache_init_objs(struct kmem_cache *cachep, for (i = 0; i < cachep->num; i++) { objp = index_to_obj(cachep, page, i); - kasan_init_slab_obj(cachep, objp); + objp = kasan_init_slab_obj(cachep, objp); /* constructor could break poison info */ if (DEBUG == 0 && cachep->ctor) { diff --git a/mm/slub.c b/mm/slub.c index e739d46600b9..08740c3f3745 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1451,16 +1451,17 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, #endif } -static void setup_object(struct kmem_cache *s, struct page *page, +static void *setup_object(struct kmem_cache *s, struct page *page, void *object) { setup_object_debug(s, page, object); - kasan_init_slab_obj(s, object); + object = kasan_init_slab_obj(s, object); if (unlikely(s->ctor)) { kasan_unpoison_object_data(s, object); s->ctor(object); kasan_poison_object_data(s, object); } + return object; } /* @@ -1568,16 +1569,16 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page) /* First entry is used as the base of the freelist */ cur = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + cur = setup_object(s, page, cur); page->freelist = cur; for (idx = 1; idx < page->objects; idx++) { - setup_object(s, page, cur); next = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + next = setup_object(s, page, next); set_freepointer(s, cur, next); cur = next; } - setup_object(s, page, cur); set_freepointer(s, cur, NULL); return true; @@ -1599,7 +1600,7 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) struct page *page; struct kmem_cache_order_objects oo = s->oo; gfp_t alloc_gfp; - void *start, *p; + void *start, *p, *next; int idx, order; bool shuffle; @@ -1651,13 +1652,16 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) if (!shuffle) { for_each_object_idx(p, idx, s, start, page->objects) { - setup_object(s, page, p); - if (likely(idx < page->objects)) - set_freepointer(s, p, p + s->size); - else + if (likely(idx < page->objects)) { + next = p + s->size; + next = setup_object(s, page, next); + set_freepointer(s, p, next); + } else set_freepointer(s, p, NULL); } - page->freelist = fixup_red_left(s, start); + start = fixup_red_left(s, start); + start = setup_object(s, page, start); + page->freelist = start; } page->inuse = page->objects; From patchwork Mon Nov 19 17:26:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AEBD714DB for ; Mon, 19 Nov 2018 17:27:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 920452A2C0 for ; Mon, 19 Nov 2018 17:27:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 862DE2A2CD; Mon, 19 Nov 2018 17:27:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2C962A2C0 for ; Mon, 19 Nov 2018 17:27:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5ECBA6B1B8D; Mon, 19 Nov 2018 12:27:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5C41D6B1B8E; Mon, 19 Nov 2018 12:27:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 462E06B1B8F; Mon, 19 Nov 2018 12:27:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id E11E86B1B8D for ; Mon, 19 Nov 2018 12:27:17 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id w4so5598385wrt.21 for ; Mon, 19 Nov 2018 09:27:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4UxJd/TD5dhnni0EdJsiBbVNK+EVuSfLspufPfY8Op0=; b=nBn+0W+N2hI31XDt6wxYjaL2YecN9FahGgqsoqQqCC234AiAd/7FpyJ+PtyQ4GpmVC 45602JS0J5f4BRPvZKyyLFOEELz5NYX2q86Zhn8o7aJRfBuU6UY9bdVn/Ipmf1BFVfve coRsmeB0uq2I8YemOhu7xKI6+AH0vjbLKYVxJ6Oz6COXf94yOqSfIyGmG09+76ZGL9/O 9sfDA/HapoYx+j9t3ZN8LhkVp6CgNqHNS4svB6Mopkct3pkY/B8ESYthjxKepZUBC7cu qypLYLtME+6nTk1zCwCgeffHplgC3HTQ7ZSZUZvBsoXz3I4ffKt28B6JSXovyTi5B8tD FUJg== X-Gm-Message-State: AGRZ1gJljtsUz+YGEmI1qUhu+L/QHIIXcO22temhDrxXXf89OUUqog5B lwy6QASwmd8xnVP5bLXbPBLaZcdx79e7yqmCh1dkZZ38d+r09Yu6FxTxjb2Vie9BMC2GE8t6yo+ 2dg6ZOm7BF+ueRUAdHkM9E5EDVnSovG51zrUuOHTSFC9FaUz+oGQMsJAfnK5DmtIC2QJxqdmkvw b0L71GDnCmbEbp0fFto6HSzj3lVXneZMMwjXXv1g1PVJQKof+CecoGmlStqDzz2meNYqdkgMUd0 eqc05mPi2dwStU/Xb/zXPZ6zf6UK3IJEKWypw0qh+AcaLGGO8Y/74J1qEVBoboqjP+VTLL10Av+ ymGaNmzOcfoO8W6/WpszPTEWi59QItm8EYGz3TJQpgQhBEuue/9iph43ou92ROyRxoIDDVs5rc7 q X-Received: by 2002:a5d:4406:: with SMTP id z6-v6mr18982604wrq.294.1542648437424; Mon, 19 Nov 2018 09:27:17 -0800 (PST) X-Received: by 2002:a5d:4406:: with SMTP id z6-v6mr18982556wrq.294.1542648436521; Mon, 19 Nov 2018 09:27:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648436; cv=none; d=google.com; s=arc-20160816; b=Y2wz/yVVdd+j3nEvCUzSXZ63pI5bVtXunqaSSEhNb5JroWPZMYzeYzz9hiaKhg+vKx Z9BHzlETZUcGzminTj39a3/OgVGU4j/r/QR2gBNag+uSwYI/5QJEUm+xnyhrfI8AxOsr uarIWY/5iZxf4zv86kk535xMFfv/c6IdO0c1eIViImoqlYzd/ApDSP2APq9+3+sR5pix dWOupzJSWoIZcAh1Fz9gc8uMs2keHU3lijt3B8i7mF5Q4Zk7RsHH9t+JtK6lbzoMh9nf EDZ9u1IemWQ0sXsSjdnrxxP/KLV1Ry2DyCWWiULOSy/Ur2HotOc7BKayJEy+8ZDuleoT 03WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=4UxJd/TD5dhnni0EdJsiBbVNK+EVuSfLspufPfY8Op0=; b=DR2vChRtuQoz9fXIrejUAwnefZvxjqgWp+9p5XvqVKfFHusv1RY9tFqYdrEqbD66rN Z8yT5HVr1EF+FBAkslgnP81zgpPfJrh6x2oO0WxVmOnm/4gRV6+pW6e6TAz4H88bmrvm oMbe+SLfmFxCz84ZmMQ3yck+PRyYruwv+QuTCD7msTGVoEYz7Z5/RqNBYJBxw36oYKok 1+OTDvSb0VOnLsgBS+ctkJFZmAqEBPrODU4jJgwNIlxNeuaAuqRJqnzmWrzJXtSO7yVb GzIploN4ZRYeVPPalfM1CksOR6ucEnby+4ZwnFaQJCdBDqag+5gH4lQ98mnyy/pFs56a qFrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ezP0ESTG; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i74sor7048772wmd.20.2018.11.19.09.27.16 for (Google Transport Security); Mon, 19 Nov 2018 09:27:16 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ezP0ESTG; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4UxJd/TD5dhnni0EdJsiBbVNK+EVuSfLspufPfY8Op0=; b=ezP0ESTGIDPD6PIZXaeQ1HQvAtE12YKEKXk2IQhb30r0JyeclvTIkALJudtga1L/Q3 sfSqapZGnhPLM3BAb3PXOYLm/bZTb1MZ54ZRnv6h5/3IlejoIUHx/AaGd5qfhPuKxnpn GsnYAXIpMb9ohJGYhY0zD2xvR+T/Aku8msHLcEukSHopb9SGDZXGijaVWkpc9CKGUaiu VLTlA6dJNpc7t3wdP5GQdvZ/cbM0ca5EE+ZdL+ZEBNJjVa09dOGeUYw8b0oESjRUAScJ k2aoTr6qOxO96slc3aQpZ9kDtYvc9TSHsfhmZQgS17UlDgmj7Ll18teQgR09yy1V1Gpr QcAg== X-Google-Smtp-Source: AJdET5f/CWAArcyhbvqSSe7+ZirG+qS0xHVggXGTUB5W8Aw5Wyx6utJIbL2GhHlD46DxV8W80eGl3Q== X-Received: by 2002:a1c:f514:: with SMTP id t20-v6mr7673551wmh.129.1542648436015; Mon, 19 Nov 2018 09:27:16 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:15 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 13/24] kasan, arm64: fix up fault handling logic Date: Mon, 19 Nov 2018 18:26:29 +0100 Message-Id: <99c747edfba8ea4b93d9c70aac47ea86ef258b86.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Right now arm64 fault handling code removes pointer tags from addresses covered by TTBR0 in faults taken from both EL0 and EL1, but doesn't do that for pointers covered by TTBR1. This patch adds two helper functions is_ttbr0_addr() and is_ttbr1_addr(), where the latter one accounts for the fact that TTBR1 pointers might be tagged when tag-based KASAN is in use, and uses these helper functions to perform pointer checks in arch/arm64/mm/fault.c. Suggested-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/mm/fault.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 7d9571f4ae3d..6023d4752701 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -132,6 +133,18 @@ static void mem_abort_decode(unsigned int esr) data_abort_decode(esr); } +static inline bool is_ttbr0_addr(unsigned long addr) +{ + /* entry assembly clears tags for TTBR0 addrs */ + return addr < TASK_SIZE; +} + +static inline bool is_ttbr1_addr(unsigned long addr) +{ + /* TTBR1 addresses may have a tag if KASAN_SW_TAGS is in use */ + return arch_kasan_reset_tag(addr) >= VA_START; +} + /* * Dump out the page tables associated with 'addr' in the currently active mm. */ @@ -141,7 +154,7 @@ void show_pte(unsigned long addr) pgd_t *pgdp; pgd_t pgd; - if (addr < TASK_SIZE) { + if (is_ttbr0_addr(addr)) { /* TTBR0 */ mm = current->active_mm; if (mm == &init_mm) { @@ -149,7 +162,7 @@ void show_pte(unsigned long addr) addr); return; } - } else if (addr >= VA_START) { + } else if (is_ttbr1_addr(addr)) { /* TTBR1 */ mm = &init_mm; } else { @@ -254,7 +267,7 @@ static inline bool is_el1_permission_fault(unsigned long addr, unsigned int esr, if (fsc_type == ESR_ELx_FSC_PERM) return true; - if (addr < TASK_SIZE && system_uses_ttbr0_pan()) + if (is_ttbr0_addr(addr) && system_uses_ttbr0_pan()) return fsc_type == ESR_ELx_FSC_FAULT && (regs->pstate & PSR_PAN_BIT); @@ -319,7 +332,7 @@ static void set_thread_esr(unsigned long address, unsigned int esr) * type", so we ignore this wrinkle and just return the translation * fault.) */ - if (current->thread.fault_address >= TASK_SIZE) { + if (!is_ttbr0_addr(current->thread.fault_address)) { switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_DABT_LOW: /* @@ -455,7 +468,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (addr < TASK_SIZE && is_el1_permission_fault(addr, esr, regs)) { + if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die_kernel_fault("access to user memory with fs=KERNEL_DS", @@ -603,7 +616,7 @@ static int __kprobes do_translation_fault(unsigned long addr, unsigned int esr, struct pt_regs *regs) { - if (addr < TASK_SIZE) + if (is_ttbr0_addr(addr)) return do_page_fault(addr, esr, regs); do_bad_area(addr, esr, regs); @@ -758,7 +771,7 @@ asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, * re-enabled IRQs. If the address is a kernel address, apply * BP hardening prior to enabling IRQs and pre-emption. */ - if (addr > TASK_SIZE) + if (!is_ttbr0_addr(addr)) arm64_apply_bp_hardening(); local_daif_restore(DAIF_PROCCTX); @@ -771,7 +784,7 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr, struct pt_regs *regs) { if (user_mode(regs)) { - if (instruction_pointer(regs) > TASK_SIZE) + if (!is_ttbr0_addr(instruction_pointer(regs))) arm64_apply_bp_hardening(); local_daif_restore(DAIF_PROCCTX); } @@ -825,7 +838,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr, if (interrupts_enabled(regs)) trace_hardirqs_off(); - if (user_mode(regs) && instruction_pointer(regs) > TASK_SIZE) + if (user_mode(regs) && !is_ttbr0_addr(instruction_pointer(regs))) arm64_apply_bp_hardening(); if (!inf->fn(addr, esr, regs)) { From patchwork Mon Nov 19 17:26:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6909714DB for ; Mon, 19 Nov 2018 17:27:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4EA112A2C0 for ; Mon, 19 Nov 2018 17:27:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 416E42A2D1; Mon, 19 Nov 2018 17:27:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E6332A2CD for ; Mon, 19 Nov 2018 17:27:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE8D96B1B8F; Mon, 19 Nov 2018 12:27:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EC92B6B1B90; Mon, 19 Nov 2018 12:27:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D162E6B1B91; Mon, 19 Nov 2018 12:27:19 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 7A6906B1B8F for ; Mon, 19 Nov 2018 12:27:19 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id v2-v6so44125064wrn.0 for ; Mon, 19 Nov 2018 09:27:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=h+5rV8i8oHAlZCp9BrjdSXzDHw5vrl+aIvlpSNxpC04=; b=S8oWTwpkpy1EK2r/GawK707KIzaFHKP3Xkwu1NzhRagFsANfblktF7IMNm+SmMM/6i PHKRwETSmzLl5dqINBMvjWgg9InLxVicYPmhbqrHszHMGKnEpH6emxNfS31LnzlWpz7b dplrAgWHRL0vADwyLLaBOOmDwaI3Jz0fGcUV1AmdIjZsXomC1aJ9W9+MlI3K4qaoWmEd G4DkkyhCK8NWNPdRw0OsHPDreOX7fiKQ993/x6KHW9SqtzG25rL2BV0wR2kmQla3d6BJ AYdkJ96zOQrODoQDTg/xPT3yQaUOtSaLz7VnPAg4Q/C0Gy0aeye0cJ6QqYqE+sB6oiiA qAHA== X-Gm-Message-State: AA+aEWaoPqX2f0UKWDDK5qx5nhap8Si+WsQFryhvHAzTA/YO513qB1ki f5qjMuzRfrqlYdFVgU3L5Gyzd+Twfz0NF/4cWYJoenHWJtTnV9UpycHMCRxE94dhl9LYxnnddAk DOr7hlE2vD1PTglMhwLfGe+OZiXHYbE8qWegUqt2J86cm+VZVAF8lhekDsxz7SO5i9r/piYkEhy tAzwYOnC0joBPyBpGyhZHcS+kNQprNkNJKUHkeB9f58u/91qbQdSCHEQ17Q8thpNLCiTxNkxsfu F1XcdCU22SBL/GwZZFkO4w/2dbAF4gliZfMHKKpJjEIkqG6rnz1zRtBG94ie9BPlG5CfNylz4k5 0ceivzL0iWbIsrqTCeIrY437Y1EhBQakPvif8j5XC/7hD9IV14486XfdFrEPLEMi+LsNWfeaxJa C X-Received: by 2002:adf:fa51:: with SMTP id y17-v6mr5711933wrr.155.1542648439037; Mon, 19 Nov 2018 09:27:19 -0800 (PST) X-Received: by 2002:adf:fa51:: with SMTP id y17-v6mr5711899wrr.155.1542648438277; Mon, 19 Nov 2018 09:27:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648438; cv=none; d=google.com; s=arc-20160816; b=n6H+6C9BYUIL1UvgFUXzCdO+EtXdVMoLXm+DGaZ0kU58Nl23MupJs3dk8bVOACF3LS jCq2LJrIY0kDziFpOjCMkrHvkxzFDTyERP7RnjQbQxp+1dRmHDbu2bMVSoWYO2Ls8iLr NOFn0DGTPKcvXNVuQUhKLq/8M0jiyNFWqIQPQYbAg5Lm7/nSQkKOKZqZcaD/tUiPBHqE wLbGoU8ze3cXx1P5DB3oXtBUJyffHGsctL6DauG20mFKkzJimhqd9oxXvhYIU7TgmqRp 6HWaM07VMm0rrBF0VwgqNCGzT7WVpCUrNxIovs7+4n1CgSjxFaX5bbZLyjPlaLzT6kU+ 6kLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=h+5rV8i8oHAlZCp9BrjdSXzDHw5vrl+aIvlpSNxpC04=; b=B8ozK6cXdQE/zfH6LGhFTwBvjckVPQ34FDNt+8HifJqyA4FXEpe0IEFoZdOyCPeYMV dHyhvTK8QB1uQ8zVqlkULYpdPLGuhQPZw8z/R8Se1ajA49F5qpKSA/sGT7ArSgBs/oCI ozZLAu/zhy6es4BQs7fzcwhgtT5thV95nsBg6A9PS/LYlZKYN2WCCJKeWk9wCuzLn6qu ugodfVTtH0+argwpdaSdga+ZKEaD1Cbaoa722XYFezSiTK37PpZDeSyHdLH4njRp+f6i Qux8FtmxhJ8FufJoX3ZbX4zq682olKZNy4R54GUgF818Z6ZPZFcmcn6gZ+U7P8uA6olg Lp2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sv45BNyF; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r7sor16612093wru.16.2018.11.19.09.27.18 for (Google Transport Security); Mon, 19 Nov 2018 09:27:18 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sv45BNyF; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=h+5rV8i8oHAlZCp9BrjdSXzDHw5vrl+aIvlpSNxpC04=; b=sv45BNyFXG98rVEUXSuOI+lgHZgjUg5xwoOUc83v5acd0OsW6+U+9c6hkGjllywz+Z +mYa8Pp6dMD6Dj3z132SayAzcPiFUezDhHFiQ6b9GZ1+HbO+gPvk7eJdLli6M6Pm7aDp yvTkTWg4GyPVv5+cToj21xhgO7l/fld8I0Po+a35SVrHXei9pZ/00mwpxSiNVxot6cr4 FbpSvNbS+Z8JySxL4rI2xopNeM8Kdax53vP1n8fNMR9QF+k5d/onDhRA6WKqOK7/U9+n FjPjGg+UKfsu7LexwgYtUDMbONwR2COyPbx03Ej7NiqCb4FbRj7Zs9j1nefbnhf2ePV2 qQ6g== X-Google-Smtp-Source: AFSGD/UOnJKpBQZNcwhfKdJD/c2SfY50D4CePkI6Yh7W+ALQWMH0I5DjVzv2hGxFtShtidsjDNfgcg== X-Received: by 2002:adf:891a:: with SMTP id s26mr944808wrs.44.1542648437776; Mon, 19 Nov 2018 09:27:17 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:16 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 14/24] kasan, arm64: enable top byte ignore for the kernel Date: Mon, 19 Nov 2018 18:26:30 +0100 Message-Id: <7194a50384cb755e389ccecda66942e15b7b76ad.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when tag-based KASAN is used. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 1d7d8da2ef9b..d43b870c39b3 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -291,6 +291,7 @@ #define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 2c75b0b903ae..d861f208eeb1 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_SW_TAGS +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /* @@ -445,7 +451,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /* From patchwork Mon Nov 19 17:26:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 93E8114E2 for ; Mon, 19 Nov 2018 17:27:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76B222A2CD for ; Mon, 19 Nov 2018 17:27:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A8E02A2D5; Mon, 19 Nov 2018 17:27:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFA892A2CD for ; Mon, 19 Nov 2018 17:27:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 91DB96B1B90; Mon, 19 Nov 2018 12:27:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8F5D86B1B91; Mon, 19 Nov 2018 12:27:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 748546B1B92; Mon, 19 Nov 2018 12:27:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 17BF36B1B90 for ; Mon, 19 Nov 2018 12:27:21 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id d11so30251864wrq.18 for ; Mon, 19 Nov 2018 09:27:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=f7KSm+jFXQkCQXEc5rnrCA4r7eJPDm1tJhN4hvjkMsI=; b=hY+TkYYZdmrzdCGI/FdgMfj14v0NEP9m4eFreU/JWikPaq+MteTwdGApCNvmIyDBrZ FgcFdgfbemgmcpF5rTy0n2GUVqDgP3Ap8CMGL0fXK+dJhg8eECXRf9crbezusC+wUixq yJW/mfh65SM3dnTx4hC+dPfkPGc1sKHxf28ZONyWJUX/Dsm9weu3yVkdWght2KB7vspK xlSV31I0hs1Ekk7SGgR53J4GqHrWvc+B9PX0FF2tXUE/SEU9IkusCeu2QGIoPa406SwI RZ9M1kuoCTioj+VAhP4SmSgl2SC5mM6F6MntVhX3uAmhETGCr7+bsZAh8j8QJzbEPEIP 0yhA== X-Gm-Message-State: AA+aEWa4AiFCXAePlX0UQevvWZOfWNh6HjGX2zzPaxNOTEQLbiSs/GVz xATmEN12ceDIjnoLU3v74vkX0N1tcbGZcKb8x49TzZT58veBY+sziqWmX56cNZ5ssfx33Ustkdt Ja6VVrdm9oZco06IAQgEi4x0bFOHZ7S8PVYO00xzjtSKbLDep7mNBLf39OW1Zu6feKxSgS8dvzK wok82uqn/DEQQz5hIhazlyXoRXWOuEWHb98kbVAlsMQBzLoqR/ExN7rBdFLoXFP0s5ccuMyCghs Aj+2mj/DRSiJ6HrFTEu5FABMWw0WUVCKhuXUqi4hnjccz3WycJqFMzLUQVAdPreLa2mfHJiWTPv ZAq9JtB5iWWKMfXDbmr1Qppn7nE7Av8j6IMUxI2y6RaVH4x5MDJpUkzmt38/E7zTTxZO6E6pCVw 3 X-Received: by 2002:a5d:5089:: with SMTP id a9mr17514739wrt.327.1542648440633; Mon, 19 Nov 2018 09:27:20 -0800 (PST) X-Received: by 2002:a5d:5089:: with SMTP id a9mr17514696wrt.327.1542648439887; Mon, 19 Nov 2018 09:27:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648439; cv=none; d=google.com; s=arc-20160816; b=bvmvK50yCVvUTi35HIiE1RvDz08XsRldlxGjs9GlVWVVHEm49z57wZ6Fb+Zns/Z4Mh GauUV40Hk7ByNQunvibYV4qRbcLtXz0uxwIdM1R0T8ZuHw9Ry3cN59Y4Mh3wQ0L26uPJ inUg5gDY/kYssWyl/yXd30B0QU9jDFS/x1bPUJ+KGILD+NDZnYB7BOjXiAInBl+MJjAn jWWj72bRBAANf/XZow6mAX0ldXKOIWK/PDhODnq5wCTcmvtEwsUKiYSTnRaeevsja3w4 b7PWM3IUN+U5IAOKDJIff+Dg1GICYvR6A5VeXta4uWgQXTegI4FGjlZrp0uiuWrRFSFd 8Rlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=f7KSm+jFXQkCQXEc5rnrCA4r7eJPDm1tJhN4hvjkMsI=; b=QGKgLktbWmIy03pmBgEGh2n7i65JPASDgpGLQCPRWCuHikC8lSgQun4pTK5PGgxEKO ZvIMUwtbfrBRdYklcyVqqxFtNyWa5TGlwQqpEb3F0Sp8WQjFM2vvkqn67Ycp6yZlc5Mz ucLEam0T1y7ceI4kxXaPxwKspxqGHIkPFpwWeaAvTMl3yVipRKOyefVceDRjfcDKHgyU T3QBcNIHHTmKuK0l1vKJwIkbNVyA0ib6lLMknevhxslzORvSqoPLP7p3qlluGd7Pqw1s H/L8ieAglvSHQnzmooX0Q30eTKQVBhIavPfwS+U/2ubILBW5IzQ4SMWigWvcZhbZB8/3 /i9g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=pxIEoLBV; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g18sor1811229wrw.3.2018.11.19.09.27.19 for (Google Transport Security); Mon, 19 Nov 2018 09:27:19 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=pxIEoLBV; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f7KSm+jFXQkCQXEc5rnrCA4r7eJPDm1tJhN4hvjkMsI=; b=pxIEoLBVS/155thk/jlirYN+v21XOcVr2GW9d9IBt1OVBarBjRyqIHbmLLhe8psyyq 5Ug5UTJ5X7VrMdCeKBD9K2dkVd+nc0AzF32lJLmucUFRU/GgpdqaUcMhykaBbosPF6dk KIu1FcKLwylt2oS/TycDYqUiERABjULH2Gwy1yv1Mw7ZhgGAthWgeAmCkVzYSlPWpzD8 5Aa+fhgB4AEe7eG4R0/QPBHYc+/htnw7UG7ePdKyWf6uffMpCJytw1h3Wnkeg1RRvRGs 9OTHbUPdKOxvcS/eCi6mPSgOtTdigpIKY3SU9AHkvrAMtJbrq82+to13FudtfZ995L4C EmVw== X-Google-Smtp-Source: AFSGD/XZpgLAg7rezu/mjkHy6lW0RJIb0uV/wXKhtOG3pV2Pb/tXon7urXALPBJFf4VkMPoh4EQ6nA== X-Received: by 2002:a5d:63c3:: with SMTP id c3mr2727742wrw.215.1542648439441; Mon, 19 Nov 2018 09:27:19 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:18 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 15/24] kasan, mm: perform untagged pointers comparison in krealloc Date: Mon, 19 Nov 2018 18:26:31 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. Tag-based KASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 5f3504e26d4c..5aabcbd32d82 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1534,7 +1534,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret = __do_krealloc(p, new_size, flags); - if (ret && p != ret) + if (ret && kasan_reset_tag(p) != kasan_reset_tag(ret)) kfree(p); return ret; From patchwork Mon Nov 19 17:26:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689049 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2AA314E2 for ; Mon, 19 Nov 2018 17:27:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C545E2A2CD for ; Mon, 19 Nov 2018 17:27:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B8F372A2D5; Mon, 19 Nov 2018 17:27:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8579B2A2CD for ; Mon, 19 Nov 2018 17:27:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC7436B1B92; Mon, 19 Nov 2018 12:27:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C9D606B1B93; Mon, 19 Nov 2018 12:27:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B19566B1B94; Mon, 19 Nov 2018 12:27:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 3CA476B1B92 for ; Mon, 19 Nov 2018 12:27:24 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id x13so17136517wro.9 for ; Mon, 19 Nov 2018 09:27:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ozydNqGoiMkcM4ZuiUgQ16xLVdXv8H81BMf2LOK9ZRc=; b=D7WBp3PmzGWrHvqnjN//58sgxYi++Z3QzZawrnMdWdNfke8wvESfE0li8Dvnwvt3RP G206g4TBta4GsWng2AjPakdCZXNzMjG9bnuX51d0bvtT/BXATZyeCWN8CzZTDdtLpptg MOW2aiR2Wniaetw6EUDtUrUSXjGtzotsu36yudUql/zY5vAxg9EvimwkC3vXuMd/o36H e/p9Lc9Ar9rMX0f18Nfu9zEA4FJK0ZfpraYZc0NmJ6DDEr93NZ4LWN6RrI835uqlvhuT RR+WWBv1ie9GQkr3GT+0n3yUZVTWl4IJ7hFXBvIdkUhZXMiTL2VwWJlSUwGF5DGsFcv6 /G8A== X-Gm-Message-State: AA+aEWbxNDuA+CohU88bR8RK0hL7dhiMEKliPLM61laKiREExMvjJIRq kSPcY2A9x4MSECFp//xFMUY91Ve4HRMfYBP3DgyojNX5J/anW51ng9T5ByjgztlLRc8IZBHCWXt ZwSFHOfklWt5Zne0JFX8vIZ34LndWjginZf+9dQNBoJbujcYmrxMCxSkhHFNz/UzTO095xeGcH+ hL3T6H3djCQLtTmFEkUd5WhVnbAteW+ThDPw7ssOkcQ7ya9oe7zp/i24+IWyniExRfxpo1F4rbj c38nUAhZNR8DjQhtEQulOy2dhulPnUakiEOZgcN0ccwJFjAhO1gfueIwHP8hanWGeeS02bcYTLh E8P8pJ+64yW+xmtva4uhHetx21zKvoIGtWdju8NzU7IPgd7I1SS8jlkS/COreTwt4Mgzrlxu25e P X-Received: by 2002:a1c:3a8d:: with SMTP id h135-v6mr7897946wma.92.1542648443685; Mon, 19 Nov 2018 09:27:23 -0800 (PST) X-Received: by 2002:a1c:3a8d:: with SMTP id h135-v6mr7897850wma.92.1542648441945; Mon, 19 Nov 2018 09:27:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648441; cv=none; d=google.com; s=arc-20160816; b=v19jEFgGbJPGDU1Pt3f+UF7ih0HZOFKbTUlMJjFo4X+uPZueZD2IF9U6DGhyFN1MdC hKVVp+O4KzaGQ0LSsxQtjoNjunDOIGCck+ZyH1RHmd53xpN3Xd/QHOArMLF8RO5XktKl 0rtEcJAqV1M1Xi9Y5SGyy4EUa6UdqLeRlAynHuQJg0iIFjYIavmQ2oHBh43Pt+yf4oQ+ gybzQ3b1SwnS6CcmdOsqg1dzuHtgcbVL8OAV+7OJSQmnLmSgvnhzoX5NjIzZF1T+aB/p ablQQU3wN2Xt4OVngA1DHHF3zkKG8fS+Yba6Tqg1MM4AAFTMpWx/mHNqMMx4DaXQ+Kd+ ZMMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=ozydNqGoiMkcM4ZuiUgQ16xLVdXv8H81BMf2LOK9ZRc=; b=rfIkVNp++Dp0t2/cPYf7FGecyiJS65aAlQDf8iXjfDVMUyQOncrQmAIPeRpSbCtubj M6+8XX3hDI8pmTdcXN7aim5HMop0EfZaONlx4GkLt/FDYZThjfo5kJfleVJGWhqK28vB SzDqolFzdpmGjxhFWspIZYRp9yIfYq2uS5ODzzib2Y0F4T6WNTj5V9YfBJD5LLAPf5bd O9HeC+IK3DUvovjmthzDY0yodO6egOKQ1kBw2y5RV6IQ/et/G7AX55k6/sp3DX5g9qv6 SIIG5o5oaZJ3Md192VcTq1Hb88HT/y+Rdau961g0bxTl7UAqk+70wfn2LO38WH3sW90o 4ysw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="cLq/qyJ4"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a14sor3668529wrm.46.2018.11.19.09.27.21 for (Google Transport Security); Mon, 19 Nov 2018 09:27:21 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="cLq/qyJ4"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ozydNqGoiMkcM4ZuiUgQ16xLVdXv8H81BMf2LOK9ZRc=; b=cLq/qyJ47kAMOjFdtVUoB0TygN6Bx15eawVb3dtUyCfm7zV6LcV+2aGs4ipMTZLVKy wCk1WpU7wfbVDyiOTSTZfGBYS3e8BIaG6p8DzLmlxzXwDcTp/+UbD3riSLw2EJ4/PjVx ZlIu+PlCrRFTtuoPyeylID12tkzHH99tB02A0qkUhVrKvKCNPEjOUDnPslAFZgtNEX6T WtI0f7grP7BCwD1f2BclXDDpNyWEy9U575AGtRGB6optNSGornU3Gkqd+fvLyhLcAKeE wV8/6E0yRFRMcrD/6SDKyXjY8l3bC5lK8mrNnRtWqzWz8LKBCB0Lv5YVepb3DnQxyTGF 74jA== X-Google-Smtp-Source: AJdET5eqlVgZmLOK6avfjqbOZEazsn/uzTEi48aBhvGWvnmHCuqZ9kDXPouYzTgGIhpw0D7pDWtsKA== X-Received: by 2002:adf:eec9:: with SMTP id a9mr18635509wrp.242.1542648441278; Mon, 19 Nov 2018 09:27:21 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:20 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 16/24] kasan: split out generic_report.c from report.c Date: Mon, 19 Nov 2018 18:26:32 +0100 Message-Id: <81d57050bcb07539c49b4479e7ac61fdff623ec4.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch moves generic KASAN specific error reporting routines to generic_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by tag-based KASAN. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 4 +- mm/kasan/generic_report.c | 158 +++++++++++++++++++++++++ mm/kasan/kasan.h | 7 ++ mm/kasan/report.c | 234 +++++++++----------------------------- mm/kasan/tags_report.c | 39 +++++++ 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/generic_report.c create mode 100644 mm/kasan/tags_report.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 68ba1822f003..0a14fcff70ed 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -14,5 +14,5 @@ CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += tags.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c new file mode 100644 index 000000000000..5201d1770700 --- /dev/null +++ b/mm/kasan/generic_report.c @@ -0,0 +1,158 @@ +/* + * This file contains generic KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +static const void *find_first_bad_addr(const void *addr, size_t size) +{ + u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); + const void *first_bad_addr = addr; + + while (!shadow_val && first_bad_addr < addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +static const char *get_shadow_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + u8 *shadow_addr; + + info->first_bad_addr = find_first_bad_addr(info->access_addr, + info->access_size); + + shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); + + /* + * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * at the next shadow byte to determine the type of the bad access. + */ + if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + shadow_addr++; + + switch (*shadow_addr) { + case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + /* + * In theory it's still possible to see these shadow values + * due to a data race in the kernel code. + */ + bug_type = "out-of-bounds"; + break; + case KASAN_PAGE_REDZONE: + case KASAN_KMALLOC_REDZONE: + bug_type = "slab-out-of-bounds"; + break; + case KASAN_GLOBAL_REDZONE: + bug_type = "global-out-of-bounds"; + break; + case KASAN_STACK_LEFT: + case KASAN_STACK_MID: + case KASAN_STACK_RIGHT: + case KASAN_STACK_PARTIAL: + bug_type = "stack-out-of-bounds"; + break; + case KASAN_FREE_PAGE: + case KASAN_KMALLOC_FREE: + bug_type = "use-after-free"; + break; + case KASAN_USE_AFTER_SCOPE: + bug_type = "use-after-scope"; + break; + case KASAN_ALLOCA_LEFT: + case KASAN_ALLOCA_RIGHT: + bug_type = "alloca-out-of-bounds"; + break; + } + + return bug_type; +} + +static const char *get_wild_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + + if ((unsigned long)info->access_addr < PAGE_SIZE) + bug_type = "null-ptr-deref"; + else if ((unsigned long)info->access_addr < TASK_SIZE) + bug_type = "user-memory-access"; + else + bug_type = "wild-memory-access"; + + return bug_type; +} + +const char *get_bug_type(struct kasan_access_info *info) +{ + if (addr_has_shadow(info->access_addr)) + return get_shadow_bug_type(info); + return get_wild_bug_type(info); +} + +#define DEFINE_ASAN_REPORT_LOAD(size) \ +void __asan_report_load##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, false, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_load##size##_noabort) + +#define DEFINE_ASAN_REPORT_STORE(size) \ +void __asan_report_store##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, true, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_store##size##_noabort) + +DEFINE_ASAN_REPORT_LOAD(1); +DEFINE_ASAN_REPORT_LOAD(2); +DEFINE_ASAN_REPORT_LOAD(4); +DEFINE_ASAN_REPORT_LOAD(8); +DEFINE_ASAN_REPORT_LOAD(16); +DEFINE_ASAN_REPORT_STORE(1); +DEFINE_ASAN_REPORT_STORE(2); +DEFINE_ASAN_REPORT_STORE(4); +DEFINE_ASAN_REPORT_STORE(8); +DEFINE_ASAN_REPORT_STORE(16); + +void __asan_report_load_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_load_n_noabort); + +void __asan_report_store_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 6e42f46522c0..6e808dbc31e8 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow(const void *addr) +{ + return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +} + void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +const char *get_bug_type(struct kasan_access_info *info); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 5c169aa688fd..64a74f334c45 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,5 +1,5 @@ /* - * This file contains error reporting code. + * This file contains common generic and tag-based KASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -39,103 +39,34 @@ #define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) #define SHADOW_ROWS_AROUND_ADDR 2 -static const void *find_first_bad_addr(const void *addr, size_t size) -{ - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; - - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; -} +static unsigned long kasan_flags; -static bool addr_has_shadow(struct kasan_access_info *info) -{ - return (info->access_addr >= - kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -} +#define KASAN_BIT_REPORTED 0 +#define KASAN_BIT_MULTI_SHOT 1 -static const char *get_shadow_bug_type(struct kasan_access_info *info) +bool kasan_save_enable_multi_shot(void) { - const char *bug_type = "unknown-crash"; - u8 *shadow_addr; - - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); - - /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look - * at the next shadow byte to determine the type of the bad access. - */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) - shadow_addr++; - - switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: - /* - * In theory it's still possible to see these shadow values - * due to a data race in the kernel code. - */ - bug_type = "out-of-bounds"; - break; - case KASAN_PAGE_REDZONE: - case KASAN_KMALLOC_REDZONE: - bug_type = "slab-out-of-bounds"; - break; - case KASAN_GLOBAL_REDZONE: - bug_type = "global-out-of-bounds"; - break; - case KASAN_STACK_LEFT: - case KASAN_STACK_MID: - case KASAN_STACK_RIGHT: - case KASAN_STACK_PARTIAL: - bug_type = "stack-out-of-bounds"; - break; - case KASAN_FREE_PAGE: - case KASAN_KMALLOC_FREE: - bug_type = "use-after-free"; - break; - case KASAN_USE_AFTER_SCOPE: - bug_type = "use-after-scope"; - break; - case KASAN_ALLOCA_LEFT: - case KASAN_ALLOCA_RIGHT: - bug_type = "alloca-out-of-bounds"; - break; - } - - return bug_type; + return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); -static const char *get_wild_bug_type(struct kasan_access_info *info) +void kasan_restore_multi_shot(bool enabled) { - const char *bug_type = "unknown-crash"; - - if ((unsigned long)info->access_addr < PAGE_SIZE) - bug_type = "null-ptr-deref"; - else if ((unsigned long)info->access_addr < TASK_SIZE) - bug_type = "user-memory-access"; - else - bug_type = "wild-memory-access"; - - return bug_type; + if (!enabled) + clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); -static const char *get_bug_type(struct kasan_access_info *info) +static int __init kasan_set_multi_shot(char *str) { - if (addr_has_shadow(info)) - return get_shadow_bug_type(info); - return get_wild_bug_type(info); + set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); + return 1; } +__setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info) +static void print_error_description(struct kasan_access_info *info, + const char *bug_type) { - const char *bug_type = get_bug_type(info); - pr_err("BUG: KASAN: %s in %pS\n", bug_type, (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", @@ -143,25 +74,9 @@ static void print_error_description(struct kasan_access_info *info) info->access_addr, current->comm, task_pid_nr(current)); } -static inline bool kernel_or_module_addr(const void *addr) -{ - if (addr >= (void *)_stext && addr < (void *)_end) - return true; - if (is_module_address((unsigned long)addr)) - return true; - return false; -} - -static inline bool init_task_stack_addr(const void *addr) -{ - return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); -} - static DEFINE_SPINLOCK(report_lock); -static void kasan_start_report(unsigned long *flags) +static void start_report(unsigned long *flags) { /* * Make sure we don't end up in loop. @@ -171,7 +86,7 @@ static void kasan_start_report(unsigned long *flags) pr_err("==================================================================\n"); } -static void kasan_end_report(unsigned long *flags) +static void end_report(unsigned long *flags) { pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); @@ -249,6 +164,22 @@ static void describe_object(struct kmem_cache *cache, void *object, describe_object_addr(cache, object, addr); } +static inline bool kernel_or_module_addr(const void *addr) +{ + if (addr >= (void *)_stext && addr < (void *)_end) + return true; + if (is_module_address((unsigned long)addr)) + return true; + return false; +} + +static inline bool init_task_stack_addr(const void *addr) +{ + return addr >= (void *)&init_thread_union.stack && + (addr <= (void *)&init_thread_union.stack + + sizeof(init_thread_union.stack)); +} + static void print_address_description(void *addr) { struct page *page = addr_to_page(addr); @@ -326,29 +257,38 @@ static void print_shadow_for_address(const void *addr) } } +static bool report_enabled(void) +{ + if (current->kasan_depth) + return false; + if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + return true; + return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); +} + void kasan_report_invalid_free(void *object, unsigned long ip) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); pr_err("\n"); print_address_description(object); pr_err("\n"); print_shadow_for_address(object); - kasan_end_report(&flags); + end_report(&flags); } static void kasan_report_error(struct kasan_access_info *info) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); - print_error_description(info); + print_error_description(info, get_bug_type(info)); pr_err("\n"); - if (!addr_has_shadow(info)) { + if (!addr_has_shadow(info->access_addr)) { dump_stack(); } else { print_address_description((void *)info->access_addr); @@ -356,41 +296,7 @@ static void kasan_report_error(struct kasan_access_info *info) print_shadow_for_address(info->first_bad_addr); } - kasan_end_report(&flags); -} - -static unsigned long kasan_flags; - -#define KASAN_BIT_REPORTED 0 -#define KASAN_BIT_MULTI_SHOT 1 - -bool kasan_save_enable_multi_shot(void) -{ - return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); - -void kasan_restore_multi_shot(bool enabled) -{ - if (!enabled) - clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); - -static int __init kasan_set_multi_shot(char *str) -{ - set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); - return 1; -} -__setup("kasan_multi_shot", kasan_set_multi_shot); - -static inline bool kasan_report_enabled(void) -{ - if (current->kasan_depth) - return false; - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) - return true; - return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); + end_report(&flags); } void kasan_report(unsigned long addr, size_t size, @@ -398,7 +304,7 @@ void kasan_report(unsigned long addr, size_t size, { struct kasan_access_info info; - if (likely(!kasan_report_enabled())) + if (likely(!report_enabled())) return; disable_trace_on_warning(); @@ -411,41 +317,3 @@ void kasan_report(unsigned long addr, size_t size, kasan_report_error(&info); } - - -#define DEFINE_ASAN_REPORT_LOAD(size) \ -void __asan_report_load##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, false, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_load##size##_noabort) - -#define DEFINE_ASAN_REPORT_STORE(size) \ -void __asan_report_store##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, true, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_store##size##_noabort) - -DEFINE_ASAN_REPORT_LOAD(1); -DEFINE_ASAN_REPORT_LOAD(2); -DEFINE_ASAN_REPORT_LOAD(4); -DEFINE_ASAN_REPORT_LOAD(8); -DEFINE_ASAN_REPORT_LOAD(16); -DEFINE_ASAN_REPORT_STORE(1); -DEFINE_ASAN_REPORT_STORE(2); -DEFINE_ASAN_REPORT_STORE(4); -DEFINE_ASAN_REPORT_STORE(8); -DEFINE_ASAN_REPORT_STORE(16); - -void __asan_report_load_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_load_n_noabort); - -void __asan_report_store_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c new file mode 100644 index 000000000000..8af15e87d3bc --- /dev/null +++ b/mm/kasan/tags_report.c @@ -0,0 +1,39 @@ +/* + * This file contains tag-based KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} From patchwork Mon Nov 19 17:26:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689053 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E821B14DB for ; Mon, 19 Nov 2018 17:27:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB16C2A2D1 for ; Mon, 19 Nov 2018 17:27:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE9392A2D7; Mon, 19 Nov 2018 17:27:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 285E22A2D1 for ; Mon, 19 Nov 2018 17:27:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9E8F6B1B93; Mon, 19 Nov 2018 12:27:25 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C79646B1B95; Mon, 19 Nov 2018 12:27:25 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DE8E6B1B96; Mon, 19 Nov 2018 12:27:25 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 474046B1B93 for ; Mon, 19 Nov 2018 12:27:25 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id j90-v6so43803474wrj.20 for ; Mon, 19 Nov 2018 09:27:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=VX+tkuNFpRNN5cATl0IWymxiZ86nGQ6AtLq/lZihWwQ=; b=VC6CiPFYoJMxE6cYWd52COalU5EJwiVYdnHCF35mWtbXj9TIY1EqDOVcbFms9LixG+ jXb555NyhCEV/ESu8y0l7AArgZwbZrAb0bGO4Jq1NlKkl64iUEu3XJ+F2WoS+YPCZWse qtAQCSSzI/6CiY+ruzkH+Tyr4bpRjZfpgSedr1XF80kHGUx3IEAgz1Oe0YqTQ3ZtDBkP JgYGD5M1Ou41tgjA2SMdy3q11HVX9plG712BPN6GS1UJCVGTkKTa+kRg6mpuahFnxSDO 7bKjCdOTeeSyFUCW1UjQnaVEWj1r+UYzScMM3JUMVUME8cN42AFufF6nASIfVWRwfllO /bxw== X-Gm-Message-State: AGRZ1gKe3HNMEvJPZiYia7HVs40FbAs/vgwVb81CGobR1cJhLY/zZ6qi F/+9GJYdc1bgWPs8psmbW7/Io5hvJwUKEccfMA3PE7jZgYt0PDerORlNLXkdsvHPkrn+1X3n9c4 Ts242NQ4y11xyWgDv5pZ6MW6YpMJ0XDRTnmbrdKJ06PIbhFIosFgNyJY3B/z3SpSpao9cniEnEQ edblhk20+rqNo9vOocNUhU3o3UTaF9AFR8UL9Pp8UyFRlsGe5tRtcVf4xEkGzcxeN2qd/PdiRxw dylSUOnNgO9XMhYVtYHHT6JdDV1NuJP4JLcHKi6lSk8zo1w1BOCbA2xoc/RLSjdgwBpgdCYRRF8 PCNXQezqPJiSo3jsdWcNJ5w3uzVjkfHfrYpEYenBO3BGQuYHZAuzj6tgN60g4vZZKvcHAMsWcdG w X-Received: by 2002:adf:9123:: with SMTP id j32-v6mr18495043wrj.0.1542648444780; Mon, 19 Nov 2018 09:27:24 -0800 (PST) X-Received: by 2002:adf:9123:: with SMTP id j32-v6mr18494983wrj.0.1542648443734; Mon, 19 Nov 2018 09:27:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648443; cv=none; d=google.com; s=arc-20160816; b=zf+7ct9jF9BqYiPfa0gvr0V2QXGXSiAwTRbYPYPIHjFN1iV2GsBfWJxBOUub//K6rp ZlqTWzbOGf7sDRmy/qGVXYXNNmzXJy0zZ58mEYDbME2fZfThrM4uS1mkadsdbxxdmmYf jt8srDoX7mMFYPv4vFshDdMoPdobCYmUIRIjxE9jrTPJrQ93SQ7uduS+hkXhcCx/c3VB okQ57pu/GUYuLv7J2gra/ADmY4pjImhwTu1yWtk3LLQBovGB0ihN/1SFTZXi9DkBS5Qd XyBgGhl3LzebHDzX1fr2+o2nW+IEqUi/B2jjWLqZsUXePPbBtDDOXMb7ThFMbCcT/lJE rtDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=VX+tkuNFpRNN5cATl0IWymxiZ86nGQ6AtLq/lZihWwQ=; b=mtyAbi47Gle75uwMFPrc164AI8H+y7RymJIdual3ztIXUdMMEphJjaFPunhTyckXJA T59x5Jtg8OI8xfR20C6Cfjm7aSSD5uu8eGOaFNeXSUJyvwwTvvUmmnP/TGan3XqrcwJq uwuDBQiq9eDBsnqp2mNkHD78lfNy6jwhMoAzyMKwyjHgxOQqBUWU5bT5lb4OEk7iGJjk 12UwlPDJbikjiSvuOxrJzYoS9fSkeUz4QDW0appuaTkxWYR7e/Hf6MaUUa0efZY/W6it 6acEna6y14z4b7b7QQ5ry/S8I2dNoeWXGrwDHI4cLxSr5bpbIHZ0vJdeEzEf4zMr+WTt c2jg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ehu+0LLh; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b77-v6sor19226193wme.9.2018.11.19.09.27.23 for (Google Transport Security); Mon, 19 Nov 2018 09:27:23 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ehu+0LLh; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VX+tkuNFpRNN5cATl0IWymxiZ86nGQ6AtLq/lZihWwQ=; b=ehu+0LLhsZ5FjQH32F9EPJrbb5oWs7Oh4XfQIu171E1unLIGpqWCjl2EOv69jykbOL Kyl6bUecMLVTtK/80cBk1c3ZsjKVg6eQGgUVf6UqiPCMz31/g6NsapL63DvkzL3rtvlZ tJeWSzjVJEUPur2gqSC4u3ZBrAnco+7gzIRukS1lDm6vNhFRWhYQNvPHzTnzj54PCdl3 e7oZzTUBSish/91T4LkzvewPuuyjfRT4F/VnxSzFuGRtrymR0+XzHM0OzhVxPGia/d3/ JjdU/07uaaZZRGiAlUYRvecGo16SPx0GO2Nn+MbbTQsaN4BpFH3bQj33zQatUhWtfU5t jTDA== X-Google-Smtp-Source: AFSGD/V5fAx6wyUMnyQPUxGU2gD2Pac2PAOQ2/SLJRv5DyiBMYEXUkUYmldpI3F2TVfLoy5yslGSXQ== X-Received: by 2002:a1c:34d1:: with SMTP id b200-v6mr8323551wma.115.1542648443165; Mon, 19 Nov 2018 09:27:23 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:22 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 17/24] kasan: add bug reporting routines for tag-based mode Date: Mon, 19 Nov 2018 18:26:33 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds rountines, that print tag-based KASAN error reports. Those are quite similar to generic KASAN, the difference is: 1. The way tag-based KASAN finds the first bad shadow cell (with a mismatching tag). Tag-based KASAN compares memory tags from the shadow memory to the pointer tag. 2. Tag-based KASAN reports all bugs with the "KASAN: invalid-access" header. Also simplify generic KASAN find_first_bad_addr. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/generic_report.c | 16 ++++------- mm/kasan/kasan.h | 5 ++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ mm/kasan/tags_report.c | 18 +++++++++++++ 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index 5201d1770700..a4604cceae59 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -33,16 +33,13 @@ #include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr(const void *addr, size_t size) +void *find_first_bad_addr(void *addr, size_t size) { - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; + void *p = addr; - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; + while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p))) + p += KASAN_SHADOW_SCALE_SIZE; + return p; } static const char *get_shadow_bug_type(struct kasan_access_info *info) @@ -50,9 +47,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info) const char *bug_type = "unknown-crash"; u8 *shadow_addr; - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 6e808dbc31e8..86fc18a21664 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -119,6 +119,7 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void kasan_report(unsigned long addr, size_t size, @@ -139,10 +140,14 @@ static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #ifdef CONFIG_KASAN_SW_TAGS +void print_tags(u8 addr_tag, const void *addr); + u8 random_tag(void); #else +static inline void print_tags(u8 addr_tag, const void *addr) { } + static inline u8 random_tag(void) { return 0; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 64a74f334c45..214d85035f99 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) } __setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info, - const char *bug_type) +static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - bug_type, (void *)info->ip); + get_bug_type(info), (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, info->access_addr, current->comm, task_pid_nr(current)); @@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip) start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); + print_tags(get_tag(object), reset_tag(object)); + object = reset_tag(object); pr_err("\n"); print_address_description(object); pr_err("\n"); @@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags); } -static void kasan_report_error(struct kasan_access_info *info) -{ - unsigned long flags; - - start_report(&flags); - - print_error_description(info, get_bug_type(info)); - pr_err("\n"); - - if (!addr_has_shadow(info->access_addr)) { - dump_stack(); - } else { - print_address_description((void *)info->access_addr); - pr_err("\n"); - print_shadow_for_address(info->first_bad_addr); - } - - end_report(&flags); -} - void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { struct kasan_access_info info; + void *tagged_addr; + void *untagged_addr; + unsigned long flags; if (likely(!report_enabled())) return; disable_trace_on_warning(); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr = reset_tag(tagged_addr); + + info.access_addr = tagged_addr; + if (addr_has_shadow(untagged_addr)) + info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; - kasan_report_error(&info); + start_report(&flags); + + print_error_description(&info); + if (addr_has_shadow(untagged_addr)) + print_tags(get_tag(tagged_addr), info.first_bad_addr); + pr_err("\n"); + + if (addr_has_shadow(untagged_addr)) { + print_address_description(untagged_addr); + pr_err("\n"); + print_shadow_for_address(info.first_bad_addr); + } else { + dump_stack(); + } + + end_report(&flags); } diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 8af15e87d3bc..573c51d20d09 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -37,3 +37,21 @@ const char *get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } + +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *p = reset_tag(addr); + void *end = p + size; + + while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) + p += KASAN_SHADOW_SCALE_SIZE; + return p; +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); +} From patchwork Mon Nov 19 17:26:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689055 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37D8414E2 for ; Mon, 19 Nov 2018 17:27:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B1942A2D1 for ; Mon, 19 Nov 2018 17:27:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0F2AD2A2D7; Mon, 19 Nov 2018 17:27:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71ED82A2D1 for ; Mon, 19 Nov 2018 17:27:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 302066B1B95; Mon, 19 Nov 2018 12:27:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2DEB66B1B96; Mon, 19 Nov 2018 12:27:27 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 154C66B1B97; Mon, 19 Nov 2018 12:27:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 9E49C6B1B95 for ; Mon, 19 Nov 2018 12:27:26 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id p16so13717794wmc.5 for ; Mon, 19 Nov 2018 09:27:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=EyBpjtdRieSeOMALerP208swINOjBw8fKlaLIeFHrqw=; b=TkMfXgsctpL13qdvZ+Y/cRVjgreKVGCD2rPfuWyTYBHVKQH4vmLvFz8SrU78jSzlwR mN69zdV8I4Uzb0HDZ9EpNEc8FtKpE6OJGHmnPUbn4ZIO1TlbPh+VC2TWMDfzYkdWAyMZ 5rj/TBteQHOxuYpN7StK3zlSO0xEuXnLfBjWFWbuiUQTPWRXoG1HISw1PmElPQoKuQvn AMVeG2ffJCW5Yq4EXDuiFnsqoWj0RHOodzbwFuboBZhEAKNaIi71bb1mlsSvTQOh9s4T Ig7+XrOTKdMaU+Ki5F6MXmdrqcyPskhOn3vVFYU3MXhAtfMMvA3ii44/HgeDSQ80RqKw eytg== X-Gm-Message-State: AA+aEWYwM/LZ6hNUMYJoO77N/DT+YYeaD8m/MW0dfl5zW615IQOl5fwb jPWuTwwPKHvexg1tZlQWMdTF1XoUFunvnKipXm70ZcylTV1kGwHWz3taXoN06WRPZsqyowO3fcy PRb637tjmwMzwArDyACdiVXvYqMAMd7lKGJA11Pdmdkv1tXyXvwEtM2bZ9eFTPDsjnaXlcwtqaI RMP3MtXTuxmPjLbIwawfCXdlyfF0WW49EJir2HV/g/QflcU2qFU0j7gOvN4ow7WcOLuLDifC7bg URuCpdCsMg1IJs7ingWbNnNvxk+oWJh/Ti/GAI4M5/eZGhoeryBTt9L6kNei4ssPd9u9wLuX066 JjcJFZ4BoyyT0s0A6KW/gQZWCqbzjqi9JZ6QfBWXcJIlHAqUCARZXfxjKmyh4FQgnnAbVRQ5cl9 X X-Received: by 2002:a05:6000:51:: with SMTP id k17mr15070190wrx.259.1542648446183; Mon, 19 Nov 2018 09:27:26 -0800 (PST) X-Received: by 2002:a05:6000:51:: with SMTP id k17mr15070161wrx.259.1542648445439; Mon, 19 Nov 2018 09:27:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648445; cv=none; d=google.com; s=arc-20160816; b=bZPB8hhLPmO+9mnM/GWncLaMZccfNtSgQtUwOoIdyBFDIDr9zNPxYwSXBYAiRSONGg 1muVqUBqM51L88/OtOOWisrxohYcOQwivIxAqqTgRY5GYPnOwJN1XXMtqjnJiPvq9ERD fCnG60LHRvkoGQ9OuldkTZQVeVZZHDBXV4biVkSLWW23mp81R9mZr4VPtOFKx2dcf9XT rL9bRAya3Kfg74Nnc8BzXEI6A3LYMmuzOSqgYK8v9arbBpVrr3mqP/K82zyI+v6NFDir koPDW/l0mf8CfoxMn5+9DA+ojgOVGgb6yOd9aP5v3eOn9GPG1J57pLGh8+xUF0WJcoLi 4jqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=EyBpjtdRieSeOMALerP208swINOjBw8fKlaLIeFHrqw=; b=Ouyjmtkg5zMfwbnUGXBloBJJyk38RiOMul4VEfyojGlqdvwChqLxoLkegA/coec4sD MWg05u5vqW79QYAMBMI/F2mYXc34VrkuCM9Itv9iljZAu+VeSjAJascIUHDTRBIbbO2v vy0D+n9GdZcGBXMixF2X3RKavNsJ0AfWHk7lYEj6eKRwmcwBNZJOCFPO9mEV18LU5K88 0rR4nNdstiazonhGDo8cCPsRTrPWXFydzeCBbnWPaIPhybXUAKQ9tzWt9i+4ckLPvPew 1ovGkABCD8feeeSKRU0fDtEAvs10rh2hJs8eWasZ/3vdP7u/3rQzwErOvoVxyd3YGlo1 xWSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XIyQ7LkU; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y131sor14074401wmc.15.2018.11.19.09.27.25 for (Google Transport Security); Mon, 19 Nov 2018 09:27:25 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XIyQ7LkU; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EyBpjtdRieSeOMALerP208swINOjBw8fKlaLIeFHrqw=; b=XIyQ7LkUD5KAD5LNVibegZo2YQL6sjY4shf+8EvtMS6lfNAIXlipRsrY9SlZEP7tD8 B5CRBSh/M3mEjQrRCV/ecd+9q4uglviLpKl/jQ0nz3N45zcqgiXrJG/PA/uAFzg8cPxN ZYrVcJ48b3UetIZqZ8rVQIQLIKBJk6ESdkoMs4NEBLtfVOB56wxkbDJBaySqw3JL8XNK xHsv7FoerSQTBV277w91lFXqeBB6vczxEDIlx2tGUMbQRzpBiq3hwY2CNya3WFAfzqeC HBL9OedMuAPzzcdHA2a0nv4EsScPU6M6n7KIYif+hal1VdNgGMers+rOlJaJDx7od8bA IMcg== X-Google-Smtp-Source: AJdET5em08xCL54dcPzAIIByw4HLT4EUANvkk0hgkMmdTHmVtMZ3YohRtB8Wp4xwH3qn/bCiwV77Tw== X-Received: by 2002:a1c:4a:: with SMTP id 71-v6mr7564782wma.140.1542648444928; Mon, 19 Nov 2018 09:27:24 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:24 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 18/24] mm: move obj_to_index to include/linux/slab_def.h Date: Mon, 19 Nov 2018 18:26:34 +0100 Message-Id: <10ca1e57fe2d5dc835e88fb9dc9ab6855ae618dc.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP While with SLUB we can actually preassign tags for caches with contructors and store them in pointers in the freelist, SLAB doesn't allow that since the freelist is stored as an array of indexes, so there are no pointers to store the tags. Instead we compute the tag twice, once when a slab is created before calling the constructor and then again each time when an object is allocated with kmalloc. Tag is computed simply by taking the lowest byte of the index that corresponds to the object. However in kasan_kmalloc we only have access to the objects pointer, so we need a way to find out which index this object corresponds to. This patch moves obj_to_index from slab.c to include/linux/slab_def.h to be reused by KASAN. Acked-by: Christoph Lameter Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/slab_def.h | 13 +++++++++++++ mm/slab.c | 13 ------------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h index 3485c58cfd1c..9a5eafb7145b 100644 --- a/include/linux/slab_def.h +++ b/include/linux/slab_def.h @@ -104,4 +104,17 @@ static inline void *nearest_obj(struct kmem_cache *cache, struct page *page, return object; } +/* + * We want to avoid an expensive divide : (offset / cache->size) + * Using the fact that size is a constant for a particular cache, + * we can replace (offset / cache->size) by + * reciprocal_divide(offset, cache->reciprocal_buffer_size) + */ +static inline unsigned int obj_to_index(const struct kmem_cache *cache, + const struct page *page, void *obj) +{ + u32 offset = (obj - page->s_mem); + return reciprocal_divide(offset, cache->reciprocal_buffer_size); +} + #endif /* _LINUX_SLAB_DEF_H */ diff --git a/mm/slab.c b/mm/slab.c index 27859fb39889..d2f827316dfc 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -406,19 +406,6 @@ static inline void *index_to_obj(struct kmem_cache *cache, struct page *page, return page->s_mem + cache->size * idx; } -/* - * We want to avoid an expensive divide : (offset / cache->size) - * Using the fact that size is a constant for a particular cache, - * we can replace (offset / cache->size) by - * reciprocal_divide(offset, cache->reciprocal_buffer_size) - */ -static inline unsigned int obj_to_index(const struct kmem_cache *cache, - const struct page *page, void *obj) -{ - u32 offset = (obj - page->s_mem); - return reciprocal_divide(offset, cache->reciprocal_buffer_size); -} - #define BOOT_CPUCACHE_ENTRIES 1 /* internal cache of cache description objs */ static struct kmem_cache kmem_cache_boot = { From patchwork Mon Nov 19 17:26:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689057 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 46FED14DB for ; Mon, 19 Nov 2018 17:27:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29B9D2A2D1 for ; Mon, 19 Nov 2018 17:27:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1CA082A2DA; Mon, 19 Nov 2018 17:27:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED5DE2A2D1 for ; Mon, 19 Nov 2018 17:27:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06A6B6B1B97; Mon, 19 Nov 2018 12:27:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 040926B1B98; Mon, 19 Nov 2018 12:27:29 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C9DB06B1B99; Mon, 19 Nov 2018 12:27:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 536C16B1B97 for ; Mon, 19 Nov 2018 12:27:29 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id d11so11442630wrw.4 for ; Mon, 19 Nov 2018 09:27:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=90CmP7GgztkoVs5eb+wdJbPAT39DZZS6nz5L6m01xVw=; b=PXgddXJjTodFeAK/JHv5O14V1Rk5JjqkFVZarrb49uLmBS9QF07DhidHQEKuwC7H0H 5GJfdK04SzIPLk6/HMKO1esfcLbWj2f8MtzY6u2P5QLlzG5VoXFtV6891cyS96Q5g7vZ Ekj8Tlmc1XJbK01QLTECdnE9gyeYwZu5CDGSEbtgEdr5+vhOFl5yoWmxA4pp8p6oQgHd mDNCyTpAlvjQKYnxA9mw9ZoApJadVORp6WzsZppNzXsAGysauYOl8HuHjXjiMW5TIBIK mSTe5ojv1OL2Dj2/67ozdhc2l94cDqroJYj9ov0pC6SPeNq+ZOmZA7DOYSC9m61KUZOg iDvA== X-Gm-Message-State: AA+aEWZoN7zDjQacK3oJ4VMGnNveszvrPXrBg/MTML4uD5bn+f9rHWQ9 HdVwIZjtzwRbPUgWUBqP75BxsOzKodoWdbcCz882yXgbMmi7EmVOt/o5VQn+dpvGp0vxHZH0Xtu MN44JMFBR5UFXQISVW2vQh7NL1WxpGJ8ueZgXCf2RWFixjf0cJeYFvgSpuF3Btb8q/jiewibpSk KaASE1G+80fU9sEvd7p/KA25lCowui8brBEKioWTBy3LYrq9uYRx7dwQLB28tewwNd+87FltBy8 xD3lSka8PAJg3iX9SIa57DSBD2VRGtcoeflZYTea+u4LsY057bRvqVGzZEUJWjrhr3c1+pwo4mr EdnVThaXvoEkbmNi6bgJr6mJCz9cHczoLqEQ7c+IRJC0UGgW3v68eOIvzr5DNUrCSonQsmgfYoC q X-Received: by 2002:a1c:ed18:: with SMTP id l24mr5369630wmh.12.1542648448824; Mon, 19 Nov 2018 09:27:28 -0800 (PST) X-Received: by 2002:a1c:ed18:: with SMTP id l24mr5369560wmh.12.1542648447431; Mon, 19 Nov 2018 09:27:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648447; cv=none; d=google.com; s=arc-20160816; b=J4ggvoWEtgN10aPuzQ6xCAOJYfbNGxfc6vmtCMhjj7JV/RQwOBZwfM3AE+SzGX1PZh tY3e7v3k9Qpc+R1px6hMYwNsyB6EiIROOR+cJIWOm26jc5r8mZB1MJ7uR2rRzwh7UHaq o8R4TwYHwf4XttkqjD8q9IuGKwBXq9wNW/NIVElvy+hhIC1jN7gi+g3R/eeTVnbcreAf IVa4ioUmmwgNVGi5zjZrdUK0LFmu2tEux1r9BOpw3awsfbXAIjJEXgk/HqV2vu5HGGwd O9m8PeV8mWTjgmi4QG7U2SzGZF3UQJ7R8VEK+4EmdSLHQHn7r8gGZti5u2JnosnUzpst GIDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=90CmP7GgztkoVs5eb+wdJbPAT39DZZS6nz5L6m01xVw=; b=tljtH6TKah/AAHB3v0k9w8X9QTBo0QyVXTlgVPzQHs8nk29T+7ybRw5TQtPVtp3Xc5 UPse9ANoDOq887UKQmZd+3O00MydFdKw4sgmb5shS+hT7VBM8Q5v99M5aZrUkrtnKgH8 vEgstw8X5zwgvYP0E4ad3vG0u8Kx4DKlG4/6XsluuE1BKwyIW5CQwvA+Mz4GizXuuWn5 6iB5di00JKWWTyQB6exn+Nf6KUNMkHsOQ6V0cqoZ7J63pqEoLWah+aNv3dIVLI4PX3A4 5R9wVpIKT1ocwGP0hsOoKk13tPlF3nDbzatlCI9MQAgpu0zkSrx1IhzzI3I4izaj9NkJ cnNA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Z829uwYy; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l127sor3953855wmb.7.2018.11.19.09.27.27 for (Google Transport Security); Mon, 19 Nov 2018 09:27:27 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Z829uwYy; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=90CmP7GgztkoVs5eb+wdJbPAT39DZZS6nz5L6m01xVw=; b=Z829uwYy8LaCaCtdMQBUQg7SxfzQDyhClQnUMCrMDKxxz+OKwYqLeuzWkwQ8ZgCp1j hpf5Vu9xmxJ043Kg4Zypj/n/ep9LAES/ZjMC/nMZendSLLYpZ0Hekt+qydKX8Q2bF6vL NnhmdR7T6LyPusZtKLEaM9i/y8gr2qKcXLi4GkQFwFRelzxa0IDkG1Hn2lAR9XIv/+aA RXcoCZduhW7QoMUppm3nhlHFivZkqRxA6JPhYD1uUgDrmHWwNK6JiMnyNfem/PWMQzjo MCujBSF0Z+ynHFG+e+b0q6pO+hHtVMEbax/tzzvZ0O4hVuK2zNif2BEz4dtBkWAj1hlq 2cNQ== X-Google-Smtp-Source: AJdET5ewESFxx4Uvzbq1mJPzAieXQcvckZ9FZ4+QHYEs9wqzAa/7ORYWdl16jLn9oBffgLx1cCVgUg== X-Received: by 2002:a1c:f514:: with SMTP id t20-v6mr7674084wmh.129.1542648446729; Mon, 19 Nov 2018 09:27:26 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:25 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 19/24] kasan: add hooks implementation for tag-based mode Date: Mon, 19 Nov 2018 18:26:35 +0100 Message-Id: <1ca66e212e466d2d8c2a16c2bbba981d0ab8ec9b.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds tag-based KASAN specific hooks implementation and adjusts common generic and tag-based KASAN ones. 1. When a new slab cache is created, tag-based KASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16). 2. On each kmalloc tag-based KASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer. 3. On each kfree tag-based KASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs. The rest of the logic of the hook implementation is very much similar to the one provided by generic KASAN. Tag-based KASAN saves allocation and free stack metadata to the slab object the same way generic KASAN does. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 116 ++++++++++++++++++++++++++++++++++++++-------- mm/kasan/kasan.h | 8 ++++ mm/kasan/tags.c | 48 +++++++++++++++++++ 3 files changed, 153 insertions(+), 19 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 7134e75447ff..27f0cae336c9 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -140,6 +140,13 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_poison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -148,11 +155,24 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) void kasan_unpoison_shadow(const void *address, size_t size) { - kasan_poison_shadow(address, size, 0); + u8 tag = get_tag(address); + + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_unpoison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + + kasan_poison_shadow(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; + + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + *shadow = tag; + else + *shadow = size & KASAN_SHADOW_MASK; } } @@ -200,8 +220,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + if (unlikely(PageHighMem(page))) + return; + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) @@ -218,6 +239,9 @@ void kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + return 0; + return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -232,6 +256,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { unsigned int orig_size = *size; + unsigned int redzone_size; int redzone_adjust; /* Add alloc meta. */ @@ -239,20 +264,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size += sizeof(struct kasan_alloc_meta); /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); if (redzone_adjust > 0) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); + max(*size, cache->object_size + redzone_size)); /* * If the metadata doesn't fit, don't enable KASAN at all. @@ -265,6 +290,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, return; } + cache->align = round_up(cache->align, KASAN_SHADOW_SCALE_SIZE); + *flags |= SLAB_KASAN; } @@ -309,6 +336,32 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object) KASAN_KMALLOC_REDZONE); } +/* + * Since it's desirable to only call object contructors once during slab + * allocation, we preassign tags to all such objects. Also preassign tags for + * SLAB_TYPESAFE_BY_RCU slabs to avoid use-after-free reports. + * For SLAB allocator we can't preassign tags randomly since the freelist is + * stored as an array of indexes instead of a linked list. Assign tags based + * on objects indexes, so that objects that are next to each other get + * different tags. + * After a tag is assigned, the object always gets allocated with the same tag. + * The reason is that we can't change tags for objects with constructors on + * reallocation (even for non-SLAB_TYPESAFE_BY_RCU), because the constructor + * code can save the pointer to the object somewhere (e.g. in the object + * itself). Then if we retag it, the old saved pointer will become invalid. + */ +static u8 assign_tag(struct kmem_cache *cache, const void *object, bool new) +{ + if (!cache->ctor && !(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return new ? KASAN_TAG_KERNEL : random_tag(); + +#ifdef CONFIG_SLAB + return (u8)obj_to_index(cache, virt_to_page(object), (void *)object); +#else + return new ? random_tag() : get_tag(object); +#endif +} + void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; @@ -319,6 +372,9 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + object = set_tag(object, assign_tag(cache, object, true)); + return (void *)object; } @@ -327,15 +383,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) return kasan_kmalloc(cache, object, cache->object_size, flags); } +static inline bool shadow_invalid(u8 tag, s8 shadow_byte) +{ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return shadow_byte < 0 || + shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + else + return tag != (u8)shadow_byte; +} + static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { s8 shadow_byte; + u8 tag; + void *tagged_object; unsigned long rounded_up_size; + tag = get_tag(object); + tagged_object = object; + object = reset_tag(object); + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { - kasan_report_invalid_free(object, ip); + kasan_report_invalid_free(tagged_object, ip); return true; } @@ -344,20 +415,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return false; shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); + if (shadow_invalid(tag, shadow_byte)) { + kasan_report_invalid_free(tagged_object, ip); return true; } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) return false; set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); quarantine_put(get_free_info(cache, object), cache); - return true; + + return IS_ENABLED(CONFIG_KASAN_GENERIC); } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -370,6 +443,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, { unsigned long redzone_start; unsigned long redzone_end; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -382,14 +456,18 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(object, size); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + tag = assign_tag(cache, object, false); + + /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ + kasan_unpoison_shadow(set_tag(object, tag), size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); - return (void *)object; + return set_tag(object, tag); } EXPORT_SYMBOL(kasan_kmalloc); @@ -439,7 +517,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { + if (reset_tag(ptr) != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -452,7 +530,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (ptr != page_address(virt_to_head_page(ptr))) + if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 86fc18a21664..5758af10ebe4 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,10 +12,18 @@ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KASAN_TAG_INVALID +#endif + #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ /* diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 1c4e7ce2e6fe..1d1b79350e28 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -78,15 +78,60 @@ void *kasan_reset_tag(const void *addr) void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { + u8 tag; + u8 *shadow_first, *shadow_last, *shadow; + void *untagged_addr; + + if (unlikely(size == 0)) + return; + + tag = get_tag((const void *)addr); + + /* + * Ignore accesses for pointers tagged with 0xff (native kernel + * pointer tag) to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KASAN_TAG_KERNEL (0xFF)). + */ + if (tag == KASAN_TAG_KERNEL) + return; + + untagged_addr = reset_tag((const void *)addr); + if (unlikely(untagged_addr < + kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { + kasan_report(addr, size, write, ret_ip); + return; + } + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + size - 1); + for (shadow = shadow_first; shadow <= shadow_last; shadow++) { + if (*shadow != tag) { + kasan_report(addr, size, write, ret_ip); + return; + } + } } #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -98,15 +143,18 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { + kasan_poison_shadow((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); From patchwork Mon Nov 19 17:26:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689059 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A378414E2 for ; Mon, 19 Nov 2018 17:28:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87ADB2A2D1 for ; Mon, 19 Nov 2018 17:28:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B8FE2A2E1; Mon, 19 Nov 2018 17:28:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C98D72A2D1 for ; Mon, 19 Nov 2018 17:28:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EC46B6B1B98; Mon, 19 Nov 2018 12:27:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E9AF86B1B99; Mon, 19 Nov 2018 12:27:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C52966B1B9A; Mon, 19 Nov 2018 12:27:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 6BD356B1B98 for ; Mon, 19 Nov 2018 12:27:30 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id r137-v6so13796889wmb.0 for ; Mon, 19 Nov 2018 09:27:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wBF5UF4+9Mv4z7HvvYvHo19YeCHdsuE1+in1R6vNAJ0=; b=qjm0ljBSC6/8yFRVvFiZwmPp3i+JTHUnqNBvLW8GN4EYSCi9rOp6wj6R4JxGkZV8h2 /Kf2FLO7Wl52TbcDHAhq+6GoBVcOvx6jGYS7uLQE8FpmDw0KSN2taI25Cm8+7I8mtxJM FVupc2ntGFHIyN4ayIYBoJxCGvuWlFfm/9ueClE0gDUcyNcD0OsgD/fvxHmGJpvdIyIJ YK/hmhDONFBIfdsoWdwyOJJo19VcSViTy0GwS/0ICHXSm+vdnLzD49IbpXETfIcCNzip M4CUmbGnEs+//GxSnD3NYQrOMfycoPqUQOL5FMRr77oEJCfhBk1FVywV0qKZfxyOJXmp rGlA== X-Gm-Message-State: AGRZ1gLXRXL7SeTEV5v+VBGGDwWYSvRBEnbtYKIkUjlNE2TiJdEdUWS0 B2NWFmq55JiW1skznJTjC/vvesJEz6+FoWivYHfpba8BCGT9ITdn1UdOSJv+VZZzA7aQbjv0s7B pRHAcXSBvrCWo27wF0ZecbeUNSG+nV8TDYIMURBJrc3isJAiygl1d48ALasD27Mfyv89EWuGwUr BtMhwT4tJa0D6/YdpU1oAxdvNBsohpu/HWWtvWvNF+vFN8m51Ba8rTeS7xGup/mY/wKZcoaBLFH A8JcyAdz/rjPUlNgsKX8O5Z7uBdmbdUxxlRm6PQ5IJ7xaYdee9+0iz4jsynwSfwG+EnsavfIFbs 98TPnVRr03w2KHTh8ofUbvDmY4b2Mzr2R0NrbD/Ji+MvOTSUqCwn+nz+TTOJIJiwMvEALbUC+VB q X-Received: by 2002:a1c:ae05:: with SMTP id x5mr2709282wme.137.1542648449943; Mon, 19 Nov 2018 09:27:29 -0800 (PST) X-Received: by 2002:a1c:ae05:: with SMTP id x5mr2709239wme.137.1542648448914; Mon, 19 Nov 2018 09:27:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648448; cv=none; d=google.com; s=arc-20160816; b=z3rHfDGCtItGkHvX/2SAH3yNNqBvATV4adZuoprcT/AKFSzDvsSpekg1VghjF47Tgx MU8WeLACvNGZIT4ztp9jQftMJ++WviECqKuW3r86dw0TiYtY2aTHuG28cDWlE6Bho5La 8tYliQEI6TpYRysxhyugpCO+F3wjgAcMIs580EEtFDQOvw1kxq2aR+zxIfr+TqcrNXsW LNKl/OZXcQh5xKEPnreUshXjpFgil6rZbcPBOcOKHVFOKiy/lYW7soBuFB37UiJQc0d8 Y9g9Im2j68ztQjocLWfenALgll2zk38vWrPhlDwgMWoLg1usTRZfwNw+lla9UqR2FvEi rRxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=wBF5UF4+9Mv4z7HvvYvHo19YeCHdsuE1+in1R6vNAJ0=; b=Z1M94SR8zWkDbUplFtomvvhOrDLMPyqoJE7QxIS1yeFOOknKQCZpT+l/MUFrN7AmUw QNejkPKrxz7Mqwk5bh2xNoFZZqUX159T3qElD2N7qMrKXfTsK1wY6abXNdw/0Bw06Cyd WNi9kk/Zmq4lbuo6eR24FUtE88TwcO38UP+ODiUA+PbI3ju7W0I9yZjZIMBflP9h7nVo O50Z6h3ifZMf9BDEnoN/h6Zy/cz0Ybe1HEayqn00TO1zn+6LHxL36TfUIVTfTLINUIC6 AAu1YG9OPfQqs+FgvxSHMFy+8KvZ10MH7TMifFbujKh96pPLbLjKzhK7PdZwV7XK+7y1 xxpQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hJw352I0; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n15-v6sor19244582wmd.2.2018.11.19.09.27.28 for (Google Transport Security); Mon, 19 Nov 2018 09:27:28 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hJw352I0; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wBF5UF4+9Mv4z7HvvYvHo19YeCHdsuE1+in1R6vNAJ0=; b=hJw352I0VKmInsCOkuELsOv+Pahph8JfJ73CQXAZKCXN65JnrmcZlGzURxPmgXvG3g QazJpRBIHupHzYPfP7MYb0PuKhph+X1s8OpuF8UpSC4mgWmjWvNRznIWGSNoJFEYfVEu 99xZ8gRRopTUIz+9Me9DLsMKuJYtjyjZOtrg6wqtXK0h63pL4szujO0UI0L0daJy1NDq mVnQVySQ+v+H1rd/xXcoegJ2Gs7uXsMvxU5XAJoxOrFeFq5jDJZUBIl3MB4bV1DisWdE 3BLNXZajk3syornY+EfBF1om3yJE12IbjYbNel6kelBqvJ7+A5HKRDy/6d8eAm5ykh3K zlng== X-Google-Smtp-Source: AFSGD/UnIiLKI4KNAdCg4BuOwdkMDxbO6AUrlCl42nJ+mPUKPiATHpNbFPe96DXL5euEhW1+kxSToA== X-Received: by 2002:a1c:8c4d:: with SMTP id o74mr2767337wmd.134.1542648448371; Mon, 19 Nov 2018 09:27:28 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:27 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 20/24] kasan, arm64: add brk handler for inline instrumentation Date: Mon, 19 Nov 2018 18:26:36 +0100 Message-Id: <33d91a4f2ea70b7c58cdfbedb8042118b05eca30.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit adds a tag-based KASAN specific brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 68 +++++++++++++++++++++++++++++++- include/linux/kasan.h | 3 ++ 3 files changed, 71 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/brk-imm.h b/arch/arm64/include/asm/brk-imm.h index ed693c5bcec0..2945fe6cd863 100644 --- a/arch/arm64/include/asm/brk-imm.h +++ b/arch/arm64/include/asm/brk-imm.h @@ -16,10 +16,12 @@ * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps + * 0x9xx: tag-based KASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 #define BUG_BRK_IMM 0x800 +#define KASAN_BRK_IMM 0x900 #endif diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 5f4d9acb32f5..04bdc53716ef 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -284,10 +285,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +void __arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +{ + __arm64_skip_faulting_instruction(regs, size); /* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -959,7 +964,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) } /* If thread survives, skip over the BUG instruction and continue: */ - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); return DBG_HOOK_HANDLED; } @@ -969,6 +974,58 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_ESR_RECOVER 0x20 +#define KASAN_ESR_WRITE 0x10 +#define KASAN_ESR_SIZE_MASK 0x0f +#define KASAN_ESR_SIZE(esr) (1 << ((esr) & KASAN_ESR_SIZE_MASK)) + +static int kasan_handler(struct pt_regs *regs, unsigned int esr) +{ + bool recover = esr & KASAN_ESR_RECOVER; + bool write = esr & KASAN_ESR_WRITE; + size_t size = KASAN_ESR_SIZE(esr); + u64 addr = regs->regs[0]; + u64 pc = regs->pc; + + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + kasan_report(addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; this is controlled by + * current->kasan_depth). All these accesses are detected by the tool, + * even though the reports for them are not printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KASAN", regs, 0); + + /* If thread survives, skip over the brk instruction and continue: */ + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + return DBG_HOOK_HANDLED; +} + +#define KASAN_ESR_VAL (0xf2000000 | KASAN_BRK_IMM) +#define KASAN_ESR_MASK 0xffffff00 + +static struct break_hook kasan_break_hook = { + .esr_val = KASAN_ESR_VAL, + .esr_mask = KASAN_ESR_MASK, + .fn = kasan_handler, +}; +#endif + /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -976,6 +1033,10 @@ static struct break_hook bug_break_hook = { int __init early_brk64(unsigned long addr, unsigned int esr, struct pt_regs *regs) { +#ifdef CONFIG_KASAN_SW_TAGS + if ((esr & KASAN_ESR_MASK) == KASAN_ESR_VAL) + return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; +#endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; } @@ -983,4 +1044,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr, void __init trap_init(void) { register_break_hook(&bug_break_hook); +#ifdef CONFIG_KASAN_SW_TAGS + register_break_hook(&kasan_break_hook); +#endif } diff --git a/include/linux/kasan.h b/include/linux/kasan.h index a477ce2abdc9..8da7b7a4397a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -173,6 +173,9 @@ void kasan_init_tags(void); void *kasan_reset_tag(const void *addr); +void kasan_report(unsigned long addr, size_t size, + bool is_write, unsigned long ip); + #else /* CONFIG_KASAN_SW_TAGS */ static inline void kasan_init_tags(void) { } From patchwork Mon Nov 19 17:26:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689065 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E532814E2 for ; Mon, 19 Nov 2018 17:28:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C87C62A2D7 for ; Mon, 19 Nov 2018 17:28:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB5DF2A2DF; Mon, 19 Nov 2018 17:28:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B09252A2D7 for ; Mon, 19 Nov 2018 17:28:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F378B6B1B99; Mon, 19 Nov 2018 12:27:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E9D716B1B9B; Mon, 19 Nov 2018 12:27:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3B406B1B9C; Mon, 19 Nov 2018 12:27:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 7CC116B1B99 for ; Mon, 19 Nov 2018 12:27:32 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id b186so13684067wmc.8 for ; Mon, 19 Nov 2018 09:27:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=m2gD6jG6UjcUCCzC3DB6fvQafuDy/Q0SvqB1cqo0FZQ=; b=ksi/cT+Lz3diZK0EG/XytczLCSfRmKeu6wjRUJ8k7zK7+RyKH3usJmw8wRV4lo3pvB Jc59hRm76lU1mk1WYtUfgFmlzS+e3TC0jrRl2p18ltiSqKOopmmaueOn9hvv9PSGrjTS EoMzkcCGO3yebjgujSPGQ8ESE3MMYhGWbHiZ1s0JwXntLqxEGeJKHcL7bAES5KnI2chG DHlvlP54xxlZDN0k2r+AZ9JLf4lnBAM65L7wxmspHUYUQppRd9B2/gJ+iwlKRHNo/1o6 CBc3B5lIZTm3VLb7a38W7JJRIrIA31upyrkgF8/JE6uqhoWQwPvs7NdzrfFmZt2o4uHR bQgQ== X-Gm-Message-State: AGRZ1gLkIy/ZHTcY9d95/sPR7svqYYLrq8YAyCD2jAGceoSogpECj5tV svyYgXokOj/usbU8loPZlBp7SJyx7KNSthIW5AMwSPPqdz0/ESoEWjCEtHCGv7XtNZ1u44Mvf8c YYfQsrSekvuppHhMJL7eG/+DIABPLkYtU4ikMBq4B17Kp1PQiFUNRop1fLNHOnoDrEVxeveuGR6 T579kNWP7HkgRj/4LGPFnGHIFuKouRVQOMu/yYbc2BI7BKU7B/BI3Y30irn13UrmsOODgtCbIso gfF32yKi/wqbgAaKdMRx6leK9o7miNjjyNtjOmsw8flC2eC93qQ1slnOGGtsQBYnF8TRBwUwQnv EfbAIayPAOk3HR2tnktYtgk2Ih8bPeGqeStrMuLYkIhzOtCHZo8tHageaYTyQ0lfAo43n/a8r4x Q X-Received: by 2002:a1c:1286:: with SMTP id 128mr8361207wms.70.1542648452025; Mon, 19 Nov 2018 09:27:32 -0800 (PST) X-Received: by 2002:a1c:1286:: with SMTP id 128mr8361151wms.70.1542648450853; Mon, 19 Nov 2018 09:27:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648450; cv=none; d=google.com; s=arc-20160816; b=majBfiyJUlFcvIHpFG76svT0okGRw0PqQ9QhLHGn36eXeY/ThJStXJvnHEb0UoX2wd 683EAtxzC05+dZ0k17kvkSIZdFrQOao4TZnZwiOhIi8d4njJUkY9ky9qEhA8yHbHpBe/ YmPbuhj0f20ZDdfq/q0I1TBq80r+3UuOzD4J1kWvvN2SrsaB+0jaHEJAuupghUTCkUaz 4WKkzr6GcKY1V1WnUCR4mStSTTTI9dwhip5Oc6Ppg0IaokWefNYe2QYWPwUdqS+aDmkD fgha1mXX+cxCBgzKONIuVXk+Fpf4Qhv7Xvxzxh+aSlrvnYJQ/r7AU3pSShBAuIHhwlMZ 4wxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=m2gD6jG6UjcUCCzC3DB6fvQafuDy/Q0SvqB1cqo0FZQ=; b=SdmsfCDAcvbwgvda+PG5i4I2UVXaMHAQO1YUgGMCO0P6ma3L7WwktVPP5auzVpGeYw hqcSa6NkT24CluZuHgKmaK73U5iLc8ShsKk94c+jAVjrA65yH3dQlGlM4FDKL/vvSF1b N3eDhVvrMGywJgtTdDjgiUtcG4g8+TAyEJGEyvNj6AQvdOEDOn5tDKE/4iCmAsOHkfN0 qLlFLodKTtoMiDHERzo/0iDDNXMoKD4te7FICZ17Lil96jdnb+a50kXnysJWcjCWOrlx 5LG/H2s3NhbMqX1iZwtirySA5/52a4hEa8jgsyE+srexTUVKOVgjXfNf28lZ0Yq598Y4 xLhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=pjDiSCVg; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j78sor6036246wmd.27.2018.11.19.09.27.30 for (Google Transport Security); Mon, 19 Nov 2018 09:27:30 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=pjDiSCVg; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m2gD6jG6UjcUCCzC3DB6fvQafuDy/Q0SvqB1cqo0FZQ=; b=pjDiSCVg//UnAnfAvsKfruC5ZkIFreYYJeUtUZKN8mx67ZJym3I89799zvgk2dXQIa 93CKN4RbFUBoPvqDEkHYQ29oz/AxgnCJ9JeWgi/i0Mevugpk1Tz0uDrSuxxwrR5buJt+ tGa6MGtOO+ElTz+ihvxG2zAqvnGZdxiZjfVbYqZfbKIkrFpn3n1Vr41gMvJaGVusHZgd T+L/h+Y9YgBqoozUSPi/1W9if0xjse3do7B0Ppb/74ph8MDS+GmjD5xI/II894nqTKnS x58MDmcHrGiLXiVyLyQaQt7wkVCCQwRQaYO/LzU63w4Y5jVQZ0kaQcU/+Wu4L0SXilVL Pk0A== X-Google-Smtp-Source: AFSGD/XqS4Vk0bGqcUDl0yvexiT2kVlI/ax3x3OuQ9q/UTCw61+Odyh94ddngYHjNWh3aII7RGg+gQ== X-Received: by 2002:a1c:d78b:: with SMTP id o133-v6mr8189136wmg.38.1542648450270; Mon, 19 Nov 2018 09:27:30 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:29 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 21/24] kasan, mm, arm64: tag non slab memory allocated via pagealloc Date: Mon, 19 Nov 2018 18:26:37 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently. For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag. This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 8 +++++++- include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 15 +++++++++++++-- mm/page_alloc.c | 1 + mm/slab.c | 2 +- 7 files changed, 72 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index ddad7df77027..48586d7c7edb 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -314,7 +314,13 @@ static inline void *phys_to_virt(phys_addr_t x) #define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page)) #define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) -#define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#define page_to_virt(page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = __tag_set(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) + #define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START)) #define _virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \ diff --git a/include/linux/mm.h b/include/linux/mm.h index 5411de93a363..b4d01969e700 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -804,6 +804,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH) #define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH) #define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) /* * Define the bit shifts to access each section. For non-existent @@ -814,6 +815,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0)) #define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0)) #define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0)) /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS @@ -836,6 +838,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) static inline enum zone_type page_zonenum(const struct page *page) @@ -1101,6 +1104,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) } #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_SW_TAGS +static inline u8 page_kasan_tag(const struct page *page) +{ + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; +} + +static inline void page_kasan_tag_reset(struct page *page) +{ + page_kasan_tag_set(page, 0xff); +} +#else +static inline u8 page_kasan_tag(const struct page *page) +{ + return 0xff; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) { } +static inline void page_kasan_tag_reset(struct page *page) { } +#endif + static inline struct zone *page_zone(const struct page *page) { return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)]; diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7ec86bf31ce4..1dda31825ec4 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -82,6 +82,16 @@ #define LAST_CPUPID_WIDTH 0 #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH+LAST_CPUPID_WIDTH+KASAN_TAG_WIDTH \ + > BITS_PER_LONG - NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif + /* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit. diff --git a/mm/cma.c b/mm/cma.c index 4cb76121a3ab..c7b39dd3b4f6 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + size_t i; struct page *page = NULL; int ret = -ENOMEM; @@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, trace_cma_alloc(pfn, page, count, align); + /* + * CMA can allocate multiple page blocks, which results in different + * blocks being marked with different tags. Reset the tags to ignore + * those page blocks. + */ + if (page) { + for (i = 0; i < count; i++) + page_kasan_tag_reset(page + i); + } + if (ret && !no_warn) { pr_err("%s: alloc failed, req-size: %zu pages, ret: %d\n", __func__, count, ret); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 27f0cae336c9..195ca385cf7a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -220,8 +220,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { + u8 tag; + unsigned long i; + if (unlikely(PageHighMem(page))) return; + + tag = random_tag(); + for (i = 0; i < (1 << order); i++) + page_kasan_tag_set(page + i, tag); kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } @@ -319,6 +326,10 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, void kasan_poison_slab(struct page *page) { + unsigned long i; + + for (i = 0; i < (1 << compound_order(page)); i++) + page_kasan_tag_reset(page + i); kasan_poison_shadow(page_address(page), PAGE_SIZE << compound_order(page), KASAN_KMALLOC_REDZONE); @@ -517,7 +528,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (reset_tag(ptr) != page_address(page)) { + if (ptr != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -530,7 +541,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) + if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6847177dc4a1..5b2a1b6ef1be 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1183,6 +1183,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, init_page_count(page); page_mapcount_reset(page); page_cpupid_reset_last(page); + page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL diff --git a/mm/slab.c b/mm/slab.c index d2f827316dfc..d747433ecdbb 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2357,7 +2357,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, void *freelist; void *addr = page_address(page); - page->s_mem = addr + colour_off; + page->s_mem = kasan_reset_tag(addr) + colour_off; page->active = 0; if (OBJFREELIST_SLAB(cachep)) From patchwork Mon Nov 19 17:26:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689071 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C89217FE for ; Mon, 19 Nov 2018 17:28:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E58482A2DE for ; Mon, 19 Nov 2018 17:28:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D7E762A2E1; Mon, 19 Nov 2018 17:28:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E2992A2E0 for ; Mon, 19 Nov 2018 17:28:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C63B6B1B9B; Mon, 19 Nov 2018 12:27:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3778D6B1B9C; Mon, 19 Nov 2018 12:27:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F31F6B1B9D; Mon, 19 Nov 2018 12:27:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id B74DD6B1B9B for ; Mon, 19 Nov 2018 12:27:33 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id v2-v6so44126266wrn.0 for ; Mon, 19 Nov 2018 09:27:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4Hsa8V5cyqurd6qNrrNh17vMgQf+VOFwFcgWWUblsJY=; b=pThbNSljf78ZfpM8NQDvX3ZM1cfnGM3QawrqLZsMlFKB6PjLVWQ5kyK00i+3/XAJYB iWOPare1b8uHmru3Sfg6MFMupnk6xYbGLGEWHJ+V/OcMk2voFo9H5A0lcAGGD9ctallt xq8vSX0mwYmGcJ0zodLRnf8AU+bahWhBlox3H6KCKZ06NtrQw8qR6pdOaJksCYjc+G7u wwbZtUzzXgQt+LxxXAnmEtZq++N0wbG/39AbIv32KENTp+MsYH6aWttsOZlIoMXIQdjl rW7yLiSGQ+z+WD6p05AxjEJh8Q9Wbk8zpU1wVZkfN1uesNfR2HEiVU1UK0tIzPRwcn/S zM3g== X-Gm-Message-State: AGRZ1gIzKcLxWg0hEca0gxK08RFXVY9OQMV+IOQY3CX9IaK+TAyNyn5b VI5VWm2YirKmYUhGjAg4PqN7urmaX7naDOpyIH+f7dqTwGp8bllJLaSCx52fjrFRxkBeW6by9xV jzqyTFMgx0Ctw6ajAYc034k5oEr9esQ7gL/M3drJEYWgciSpwKNL/MlOwpa5b/Z2K9/jBtYV1G0 L2R2QzfKx3/HGKFmXAcW9kWkBXMbn0NSSuwh0b5vLok67T9+UYTCfgVihc6Ez6027876FmMgfv+ JzYk0w7MzdaCrXM2akHO0g9fmnlNC8TmWAzkB94uTHoQ16ulM28XM1Rn0HeCPB2D27XDsj6O1dH osWH3KX0kAfDABLztKGp16e58DkOclBXKflXA/f1H3BcZlApBjM3o8b3JSiI9JDd+dlpJqd8pCk D X-Received: by 2002:adf:80ab:: with SMTP id 40-v6mr19731270wrl.158.1542648453283; Mon, 19 Nov 2018 09:27:33 -0800 (PST) X-Received: by 2002:adf:80ab:: with SMTP id 40-v6mr19731227wrl.158.1542648452384; Mon, 19 Nov 2018 09:27:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648452; cv=none; d=google.com; s=arc-20160816; b=LBe4Q1aRVcHeLgEcb8dPemWRR6NNFbFxKDqTYofWvVhAJ0edfk0aQvME/MV8dJhmBl 5uXyqH7TzH7Gbru2tgnsiKpWRPXd6NEJVgjFORThbfzLoh/1D9qghtTFpoy/OjvZ5A06 2ypRStLil7Pv3vobIH1eEBWG2zFEoz/R64u/qrkbfsTi/ajQXRmarGzAuR0mbNK4J/a2 vOw5xdibS33VqUQgLQ6Hv7zpUNXWTpySdmYM1HFFqfsQFz7qaJTayzTgBqnspQfv60xr OdfmAdQKN6ABOxTOAWJkDm6Vf3d4DdDqUDTLxae1qlkgb3Oz/6x1pycLYk80UdpfQu10 YfAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=4Hsa8V5cyqurd6qNrrNh17vMgQf+VOFwFcgWWUblsJY=; b=aZn9HOjg0V/zJ2fmrWmihBOjeUt6rUeZ4aBsv1XdJhAxRtj/X4lyP2Qd/Yt2eDtqX1 1PoiXk8yuvSSutoA7QnLqZG5cTd9ntjHPkHKbG8DMWC+/rJHEaSpUezd+WgHbHsABVsT z9olUbKNyJqWsAfT0JCHI6FK0vnKSAtY8/y8HACV73sI/7qJ7D0VtpTYC9vDGJG8pdYv YXCvwDTBcJnEmBY6YiOTjXjZif3byYq4LPo/C5juHaHVt/pb6VNZqskHFnNp0fbTM6As gw2rMDqfZHff/RibApp5BT/vWHpw52c/iW3dKxo7hxRW7YwRhJ1f5Fk24HswvgqWvaEO qkHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=eJUIqqcT; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id d70-v6sor19360358wme.3.2018.11.19.09.27.32 for (Google Transport Security); Mon, 19 Nov 2018 09:27:32 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=eJUIqqcT; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4Hsa8V5cyqurd6qNrrNh17vMgQf+VOFwFcgWWUblsJY=; b=eJUIqqcTCwdMK+Ze5U3tBDmP7zaYnnM7KjIf6MsdirCNEMcg85CW5ArsNDzJefv3Ow NxAFPAWc1lKeYz6XKKw/SNzpF6BbaxAPiCkfCF1fq0i8Df5tOVzMQY7L92FIapGg9vGi ewUGbGFHEdxAYMerulbTxKoMAJ0HKJdiC/m6cJTlDPNL8kr5xGlc0jCfEChyFnoXnp+B pOkxzznQGcKOfZ+Mro5UO/JZtO+069vPnr6TzWgUvL6om1WY/aDrURMVklzJlu9rxwva ZcDEm53f3Z8HN2tUnkRyIfYCMGWVZb1J6JE3obCk0EMXovqNXVcPqWj8byicTzfJWSE0 2ejw== X-Google-Smtp-Source: AJdET5fanlhsIp0g/xpGFrbxrhl1+EyGkJqzbcRiaBt+vYNbFkmdxUN2MTjJ3If/N5HOS7n3bvTZnw== X-Received: by 2002:a1c:110b:: with SMTP id 11mr5140713wmr.83.1542648451862; Mon, 19 Nov 2018 09:27:31 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:31 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 22/24] kasan: add __must_check annotations to kasan hooks Date: Mon, 19 Nov 2018 18:26:38 +0100 Message-Id: <0a0cabf188838f1d42b417f6aadc97b8fcf5c60b.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds __must_check annotations to kasan hooks that return a pointer to make sure that a tagged pointer always gets propagated. Suggested-by: Andrey Ryabinin Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 16 ++++++++++------ mm/kasan/common.c | 14 ++++++++------ 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 8da7b7a4397a..b40ea104dd36 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -49,16 +49,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, - gfp_t flags); -void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +void * __must_check kasan_krealloc(const void *object, size_t new_size, + gfp_t flags); -void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 195ca385cf7a..ba8e78eb0c67 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -373,7 +373,7 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, bool new) #endif } -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; @@ -389,7 +389,8 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) return (void *)object; } -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, + gfp_t flags) { return kasan_kmalloc(cache, object, cache->object_size, flags); } @@ -449,8 +450,8 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) +void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { unsigned long redzone_start; unsigned long redzone_end; @@ -482,7 +483,8 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, } EXPORT_SYMBOL(kasan_kmalloc); -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -506,7 +508,7 @@ void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) return (void *)ptr; } -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; From patchwork Mon Nov 19 17:26:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689075 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 586FE14E2 for ; Mon, 19 Nov 2018 17:28:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D0552A2DE for ; Mon, 19 Nov 2018 17:28:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2FA3E2A2E4; Mon, 19 Nov 2018 17:28:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1056D2A2E0 for ; Mon, 19 Nov 2018 17:28:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD7D26B1B9D; Mon, 19 Nov 2018 12:27:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D880C6B1B9E; Mon, 19 Nov 2018 12:27:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C28766B1B9F; Mon, 19 Nov 2018 12:27:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 562D36B1B9D for ; Mon, 19 Nov 2018 12:27:36 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id o76so8329422wmg.4 for ; Mon, 19 Nov 2018 09:27:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DA1vCDk3HizYxPCF2Izvx7BouF2QAzUDMdkLcLPOtIQ=; b=e3aaytXnWC7dsBrEWO+nERklwSr90qB61kGkMrn5EdYzk7t9JLvdY3RKnKMtg8o1+9 QandsM+wL7dmGYpceSepQG4AR6PVDgy4GPekjG+vS9dnMbGb4ijBV5lyPyO+g8wdfRSn D5yzd85WdD4rrmDp8DCGxDL4XXDai8mZOPZKO+kd9xszR9j5hpVD413riWEkg5cM6RGI CcaUbn6lGo3Ap+Q3l5NJxBYTqxsfUnBXr3jKGzD39eTztkR8tTbF7yntXzOhX6n101zm 5TwG4w8OaEP1aVgp8YUD0GxLk2eq9QA3Mu6eqyV5ry2dl0pAZo4Q7VSdMbIqcFDEszDE JSQA== X-Gm-Message-State: AA+aEWZf4JTbUGdqBCqAHx/oz9XoH7t7x1X7XSVa7y0OPnbaPJhL9xXd ed01ZP9Cmgl/U65qq4rf+87A+NKGZQPhChQ4s5liyAknzyOjnRR0lblS1Pgi8AKOLjLLxU8XI+Q uN9o415xechdzTnrQ+83iUW43yMdkhb+CUd3ROduRT5CudfQVTTqwZkNv6RWFxS7zMWV27i95jN EPkkIr/pDoQLLcQUN+2ENiIw6IQFK9IFQrjslytke9fH7E6jyuOOMp1PI4aELn3IsMSR9PLFJcN J18hUcf03K5ZczWIg/yf5NAJKkW5/xqb9DbXiggFR8ClipLio7yIvX1gKQ4+oC1AOyjE42zucgH J+T797GOZm7uj77oMbjSQ9hbRrsL2Dx7A86Z4tpZn0e8GYoec+oQ9QR755pcm/NAvxKNs5Minki A X-Received: by 2002:a7b:c095:: with SMTP id r21mr7986267wmh.118.1542648455814; Mon, 19 Nov 2018 09:27:35 -0800 (PST) X-Received: by 2002:a7b:c095:: with SMTP id r21mr7986203wmh.118.1542648454380; Mon, 19 Nov 2018 09:27:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648454; cv=none; d=google.com; s=arc-20160816; b=l4FgGosLN7+SRjuyEKysQLrVkV7IymwJHMdiXV9sYN6soD9Ha7byKdmfavR+mq61lE oijhKvJPniwxjzyhlzVWoxnjc6LwZ1Ojxa5+sbAzEsf0N0a+Q+RbEZGxwppD4DdmZKGc Mz8/bgU15mmJ+l7Yaqe7KpLAAl6TzwKfhgzijvBj099ZP8zXJH/ld+fSGhkA7IUM0mCF 6WePwDr54l2OAjI4ey1vUz7y+lTX0dJhGecHLvC1ZtcMiozN1/s3pTqroAfY75eqhcir ZBBl0hxerutNuRTJnB01Eyq+w8WcfCg8DHw99j3tK2I3TC2Q6iILhL2OpxjqTW0KpEj9 zO5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=DA1vCDk3HizYxPCF2Izvx7BouF2QAzUDMdkLcLPOtIQ=; b=UKAcOQ5KWvMILRgAk8uL7shCqcXlWIMYHOdxhoA9bQHXuNTkD3ep/nVKkaWRGnbw8k Ko8C+MNbPvUVHjftwmNNrtFzNC3VpZNUYlKOUrBd/znl7xes6eiLFc4Rd8DbuZDo/KTa 93tmhuny53RKCnBDMpLvuYD0G/pC6HF2TawtFZA2DIddL67C1a6aIEi5icJyQE91n3YG oaYihAKcosPM5PhMDuORrkKZa1wS4Kp3vnzzAy7ZK8pKemFWQBOSbNjk834891VImBJr RHDWdtJr3qUb8J+PKn31JuQI68742u9BFeYweg6EGGI90Ae4Tr45QLG27/EpCT0fxH+F uVpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=n+ZXwmNk; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id o17-v6sor26546125wrp.29.2018.11.19.09.27.34 for (Google Transport Security); Mon, 19 Nov 2018 09:27:34 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=n+ZXwmNk; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DA1vCDk3HizYxPCF2Izvx7BouF2QAzUDMdkLcLPOtIQ=; b=n+ZXwmNkZvnmI0wRXOtLnZpPHZUD6yeaCVTzx24pG8EqCOmf7LcX/X/m7UkmEkB9uu tGtum8zRxuo5IDMYaI4Jz4jxsEgSLMOfDQWj0SCw6uDaTg7UjRlZ9HFlTg6OSEEht2Yd U7EDJ5jgmss/RuNlg9WELr1qvspCJn4hdgeW05OZSVLDusITQV/FtdlS1g87ROWu5wQa ezmVfPLR0EISYI/fxOB/vTuwiM1+KD+RwP9YEJdLhC7z65GiwPleqS2DfQ9UnqmQOLaO kmplQAQ7plyM3FR8Jh9UABvwT10d4M9loBp1VGbr/KJn4IBjpkzv6eqnDJD/pultcgqs EOoQ== X-Google-Smtp-Source: AJdET5diaIlOgEpENewncaHwB+l3TU6JCOxlpu4qsuGyj6NQrFd8rIOjXThzVWyrqX19PSRgwWp6jQ== X-Received: by 2002:adf:c6c3:: with SMTP id c3-v6mr18415643wrh.243.1542648453490; Mon, 19 Nov 2018 09:27:33 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:32 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 23/24] kasan: update documentation Date: Mon, 19 Nov 2018 18:26:39 +0100 Message-Id: X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch updates KASAN documentation to reflect the addition of the new tag-based mode. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 232 ++++++++++++++++++------------ 1 file changed, 138 insertions(+), 94 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index aabc8738b3d8..8e956e0588fc 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -4,15 +4,25 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides -a fast and comprehensive solution for finding use-after-free and out-of-bounds -bugs. +KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to +find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN +(similar to userspace ASan) and software tag-based KASAN (similar to userspace +HWASan). -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. -Currently KASAN is supported only for the x86_64 and arm64 architectures. +Generic KASAN is supported in both GCC and Clang. With GCC it requires version +4.9.2 or later for basic support and version 5.0 or later for detection of +out-of-bounds accesses for stack and global variables and for inline +instrumentation mode (see the Usage section). With Clang it requires version +7.0.0 or later and it doesn't support detection of out-of-bounds accesses for +global variables yet. + +Tag-based KASAN is only supported in Clang and requires version 7.0.0 or later. + +Currently generic KASAN is supported for the x86_64, arm64, xtensa and s390 +architectures, and tag-based KASAN is supported only for arm64. Usage ----- @@ -21,12 +31,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC -version 5.0 or later. +and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and +CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN). -KASAN works with both SLUB and SLAB memory allocators. +You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. +Outline and inline are compiler instrumentation types. The former produces +smaller binary while the latter is 1.1 - 2 times faster. + +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line @@ -43,85 +55,85 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access generic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8801f44ec37b by task insmod/2760 + + CPU: 1 PID: 2760 Comm: insmod Not tainted 4.19.0-rc3+ #698 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [] dump_stack+0x46/0x58 - [] print_trailer+0xf8/0x160 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] object_err+0x35/0x40 - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kasan_report_error+0x38a/0x3f0 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kasan_unpoison_shadow+0x14/0x40 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] __asan_store1+0x75/0xb0 - [] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kmalloc_oob_right+0x65/0x75 [test_kasan] - [] init_module+0x9/0x47 [test_kasan] - [] do_one_initcall+0x99/0x200 - [] ? __vunmap+0xec/0x160 - [] load_module+0x2cb3/0x3b20 - [] ? m_show+0x240/0x240 - [] SyS_finit_module+0x76/0x80 - [] system_call_fastpath+0x12/0x17 + dump_stack+0x94/0xd8 + print_address_description+0x73/0x280 + kasan_report+0x144/0x187 + __asan_report_store1_noabort+0x17/0x20 + kmalloc_oob_right+0xa8/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + RIP: 0033:0x7f96443109da + RSP: 002b:00007ffcf0b51b08 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055dc3ee521a0 RCX: 00007f96443109da + RDX: 00007f96445cff88 RSI: 0000000000057a50 RDI: 00007f9644992000 + RBP: 000055dc3ee510b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007f964430cd0a R11: 0000000000000202 R12: 00007f96445cff88 + R13: 000055dc3ee51090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2760: + save_stack+0x43/0xd0 + kasan_kmalloc+0xa7/0xd0 + kmem_cache_alloc_trace+0xe1/0x1b0 + kmalloc_oob_right+0x56/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + + Freed by task 815: + save_stack+0x43/0xd0 + __kasan_slab_free+0x135/0x190 + kasan_slab_free+0xe/0x10 + kfree+0x93/0x1a0 + umh_complete+0x6a/0xa0 + call_usermodehelper_exec_async+0x4c3/0x640 + ret_from_fork+0x35/0x40 + + The buggy address belongs to the object at ffff8801f44ec300 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8801f44ec300, ffff8801f44ec380) + The buggy address belongs to the page: + page:ffffea0007d13b00 count:1 mapcount:0 mapping:ffff8801f7001640 index:0x0 + flags: 0x200000000000100(slab) + raw: 0200000000000100 ffffea0007d11dc0 0000001a0000001a ffff8801f7001640 + raw: 0000000000000000 0000000080150015 00000001ffffffff 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc + >ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 + ^ + ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works. @@ -138,18 +150,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For tag-based KASAN this last report section shows the memory tags around the +accessed address (see Implementation details section). + Implementation details ---------------------- +Generic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address:: @@ -162,12 +180,38 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +Software tag-based KASAN +~~~~~~~~~~~~~~~~~~~~~~~~ + +Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to +store a pointer tag in the top byte of kernel pointers. Like generic KASAN it +uses shadow memory to store memory tags associated with each 16-byte memory +cell (therefore it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation tag-based KASAN generates a random tag, tags the +allocated memory with this tag, and embeds this tag into the returned pointer. +Software tag-based KASAN uses compile-time instrumentation to insert checks +before each memory access. These checks make sure that tag of the memory that +is being accessed is equal to tag of the pointer that is used to access this +memory. In case of a tag mismatch tag-based KASAN prints a bug report. + +Software tag-based KASAN also has two instrumentation modes (outline, that +emits callbacks to check memory accesses; and inline, that performs the shadow +memory checks inline). With outline instrumentation mode, a bug report is +simply printed from the function that performs the access check. With inline +instrumentation a brk instruction is emitted by the compiler, and a dedicated +brk handler is used to print bug reports. + +A potential expansion of this mode is a hardware tag-based mode, which would +use hardware memory tagging support instead of compiler instrumentation and +manual shadow memory manipulation. From patchwork Mon Nov 19 17:26:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10689079 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F12214E2 for ; Mon, 19 Nov 2018 17:28:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45D2F2A2DE for ; Mon, 19 Nov 2018 17:28:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3890D2A2E0; Mon, 19 Nov 2018 17:28:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B9B032A2E4 for ; Mon, 19 Nov 2018 17:28:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 217F46B1B9E; Mon, 19 Nov 2018 12:27:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1EF156B1B9F; Mon, 19 Nov 2018 12:27:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DECE6B1BA0; Mon, 19 Nov 2018 12:27:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id A09A66B1B9E for ; Mon, 19 Nov 2018 12:27:37 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id w16so28903888wrk.10 for ; Mon, 19 Nov 2018 09:27:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=34hTykRKcWhaMS1fQrkJbK7/jAHF4emrjHnaRSxm5Ss=; b=ThB3xKRINbgvc/5immtXQvGfqNsZMY88oo3iXxhLt8mXSEviAm07/MM/6tGFf/8AM2 w+CSj46E2DqBFDQZToJa2T99nbebzew0vkTCnErMLmzFaCcZwNQ1oeL+2ghg4Y+BiGwr 9kvx6Y6yQxH9yELp381cLf2Qkq+qHNMFeFyINzMirm8Zct0/Dfwt6K1DYCDrz2z36Wpq G5sOSxg2GO+5nWA5drgiTgKirI7nI3Z1QbVdzvZ0vTZkXxqh7iSjJqiJNvI/v3vOURw6 izdtML8PSPxD/bl8lZQ9+vYZ8EXZ/tHUoMDZQf2VHqkNxefUo56l7TqTntLlnIz0J7LW sw1g== X-Gm-Message-State: AA+aEWbh79ontCsl5TBuFYfBofTVkIB6oOCo3E4EJUKPujoC88W7cdW/ 6cxxmbIlE4Cql0DfhRJOJi1IKR28UG82Sxwu0IxP5g8sm42/aL6lO1/x8Rf9an0dv0w6Rj+wMLl auqYu1VRDfXadwx1nsXm4V06Iro/0m2XpVWvM5QVtlh9lPyoG+JBdwlf026YNAJNNqNpqQSoehY Hmc6/fKQNQTw6219bKg97+lM7O3PY2VF6yDaHW5x3qBasykGoagMjZaP98fn2/iTh7RQ0Z8u4Lt xDyA1hVVWqoRME4q8nU7SNNWvUSGlJwklY+DSaL2i/xVIadTR9FWxY1/fr4x5wxaY7VcO7Tny3I jzfthlKe8n5gsvx0FL65/A+vr54zTWImrqPxwXitrPkcHijsQvz7hpxvPkMNdhlb36WeQ05GTb7 6 X-Received: by 2002:a05:6000:104b:: with SMTP id c11mr6538476wrx.303.1542648457196; Mon, 19 Nov 2018 09:27:37 -0800 (PST) X-Received: by 2002:a05:6000:104b:: with SMTP id c11mr6538433wrx.303.1542648456379; Mon, 19 Nov 2018 09:27:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542648456; cv=none; d=google.com; s=arc-20160816; b=WJ0kwHXQca98h9kk7q6OP5Ppjjp1uUqi7dh/vtw10K78Pddvcc+HMWXjOJXyIcjJrz amhidas5ndE2eFtuLbnPiwP/zWpPnU1Yc+g4mh37/I09JA2e7lQ+WPr+00cw8lYoBBUa D1u6sl24LaiaPiBAhA5b/PhgTFCUh/IsdTsM9ng89/TWWRbUls4LM3jFMx3FwwACwQpz bEXuZXZcbyhtWsS5zisvZjZzSx3vqqULfCin0KkNK26WtsT9kbjx+hV3sVFr4TbkYvxE luvr+JH8MIz9gRASbS6/ifXhHJCkWr4/gCgFCeIMEnu7XEMrNl9dVxkvNOFMY5EulX1S Imzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=34hTykRKcWhaMS1fQrkJbK7/jAHF4emrjHnaRSxm5Ss=; b=wZBGWL5jw+w6DSvubJifUH2PVhLjDXYfODxXcusCpwQz/ukq/cJoE5Zf/gtcfOyZy7 dnmRZXqZTy67vYlXtpQFfRis8cCfYSah6EtW73wmW4x7zQqBG7mw5eF4OTwlimPTm19Q k3HspHgScxhCeoXznusE1NHtOuo4T4eLKtlQSlG71y/PqTq8dqUyk1822j8Hpvz+QsbL UAcxQYc9EhjlS93loaZWUD/nSx7KuXEDfIG5jtvx/D8mxgX98J40DDs9/aV/b+5iUb8e 1GhU9dzWTLJMTqLqwk4hrLtIoYNuNTzUHCOkr9t+8IolhtjkWG+trte1vyio3i1xGCQU zBqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=lyLV3L8w; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id t134-v6sor18327318wmt.3.2018.11.19.09.27.36 for (Google Transport Security); Mon, 19 Nov 2018 09:27:36 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=lyLV3L8w; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=34hTykRKcWhaMS1fQrkJbK7/jAHF4emrjHnaRSxm5Ss=; b=lyLV3L8wTcUcKgrTnPNQTK30cd26lfH8yp9gsif8Ds82qAL//+kmhUmF/f2cU3Cl7R K9yyY/I69P0vbudaiZx3TqZ1AiwAMlP+VRIUKAh+jciqdxI2TTYUWHDylkDlzzOIDQQd 1CMzpWxJf7J3SporiQRVPWZU3f6uKjJujpGJyyMpljb3pQSw93arORHPmS6VsqGwv7tB Rh/ypAwy0JT33yKg6j4zZi2u7s1gWHbUhH2WBr1GaJWk2P0oKGtRMpTh0Cpx21PjEOxb tAdkyIBnC7Haphnrm4W6YedjmLE77ca4FImSOtQIA9yxWeUCGNx8LhsooDFHhVYAmkSR sWAQ== X-Google-Smtp-Source: AJdET5cE1luzhk6bqo2a67kWH0NiMR++tuu8gFzhI8kOTUxou2v2fiPAyFsC2WBOp3TVhTbZH0Y96w== X-Received: by 2002:a1c:3a07:: with SMTP id h7mr5536450wma.41.1542648455754; Mon, 19 Nov 2018 09:27:35 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id l143-v6sm23685190wmb.23.2018.11.19.09.27.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Nov 2018 09:27:34 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v11 24/24] kasan: add SPDX-License-Identifier mark to source files Date: Mon, 19 Nov 2018 18:26:40 +0100 Message-Id: <95c5fe49a551b0aae873ed959656ae6fd3421e14.1542648335.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.1.1215.g8438c0b245-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 1 + mm/kasan/generic.c | 1 + mm/kasan/generic_report.c | 1 + mm/kasan/init.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + mm/kasan/tags.c | 1 + mm/kasan/tags_report.c | 1 + 8 files changed, 8 insertions(+) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index ba8e78eb0c67..6628a9f0c03f 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN code. * diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index b8de6d33c55c..ccb6207276e3 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core generic KASAN code. * diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index a4604cceae59..5e12035888f2 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains generic KASAN specific error reporting code. * diff --git a/mm/kasan/init.c b/mm/kasan/init.c index 2b21d3717d62..34afad56497b 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. * diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index b209dbaefde8..57334ef2d7ef 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. * diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 214d85035f99..ca9418fe9232 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN error reporting code. * diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 1d1b79350e28..0777649e07c4 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core tag-based KASAN code. * diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 573c51d20d09..8eaf5f722271 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains tag-based KASAN specific error reporting code. *