From patchwork Tue Nov 27 16:55:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700877 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62FE31869 for ; Tue, 27 Nov 2018 16:55:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4FF082C43F for ; Tue, 27 Nov 2018 16:55:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4D7E42C4AB; Tue, 27 Nov 2018 16:55:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31FD22C43F for ; Tue, 27 Nov 2018 16:55:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E13AD6B4933; Tue, 27 Nov 2018 11:55:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D9C4C6B4936; Tue, 27 Nov 2018 11:55:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3BBF6B4937; Tue, 27 Nov 2018 11:55:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 580AD6B4933 for ; Tue, 27 Nov 2018 11:55:52 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id q18so18556356wrx.0 for ; Tue, 27 Nov 2018 08:55:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wUF41xJvycEPFX/Id9fnR3A/13bzUOgF7zYgrCLBS7I=; b=rjPcOLZBwGqL24/mFCguetZeyqd9KOoMmsyYEJrP61UjKRTRu3NwnUpHKCK3Fop1lB ZVV9R3noQXUQ/Ak4WpQkPHBsXWMsGOv2RSMKTLwH5qbp+oY6BmGlX8nbpceHjNBZqCgV mPaKM0p/s3wskyR7tY1ZdhwPYNvZv+ajgtC7LT3zxmjquKrXgUf9Znl7hRZH5uc52vXA R+5PMuZ8kPGLqsIaBLU9DCeL2O80K309QsIY40+AYTwBex6RJBhsZT4UU0IAthr4cRUd ebzNn6BrgUmWr/J/7eJw/sHThES/XENAnSu6AXegbbeXu/xAYOOIN9j3eD6hwV7uxdaH hC+w== X-Gm-Message-State: AA+aEWbO7FfbLBObFbrg5192cAEgbuWe6326CMg4qFGNCjW8FzYQ4NUx 4EXNTQhsPOM43osDlhHwBqy41JDelLpJwHoM4PGDltDnd+C3dfzA88tVrXrX5HGA91/rTGiiM6P 8iDhlhgjvyGqnB93kU5AY6vioED284qOFOqBhIfcJ7ywPV1qsvxP7tU6+Bpqe4g1EpJ9q42WN/a Tg6i18aG43xZerivjvUXgWE65lftKF7GL7uSYShiNBS4v5BsQzXxCSWma9HZiAWN2XUAbpHnnjI JkKePkirfGDQ/eXLw3kU+4WR5gR6JmefUAUBdUO1gOowkgsVIrbcRd4jmjJfCGIwGqV8b+kbW5H 3U1eNzSc5xWWdrl43Fi0pOsxKI57XvHQOr5XJ4ikaFN5kqnWFPFS3ucCjDlKhvNwstgvVr0b0+2 s X-Received: by 2002:a05:6000:108d:: with SMTP id y13mr25831712wrw.135.1543337751766; Tue, 27 Nov 2018 08:55:51 -0800 (PST) X-Received: by 2002:a05:6000:108d:: with SMTP id y13mr25831638wrw.135.1543337750447; Tue, 27 Nov 2018 08:55:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337750; cv=none; d=google.com; s=arc-20160816; b=tjVRWl+DKMyvX3bLOtRUwQwZY+c2momRrG4F3OA5JiIH3afxffQJl1xRvOGwbTAL0T xeWGRuUnojmkLO4ymiN5ta973D56Lf4jYrmVLvAicEd3iEl6xyjJa0RaOWfLLAV3owC0 YYHmzhCGYzbJ3D6jnSiedkO2r8/akxFpNhkFl+6YDTDo/VLFek9cDUyMftgKulwY32Cv iagaBYk8ND0lB1i0eZwspqDBYf+KcmZlox82DTTbHYQ4nAdTp/QXuIjwPOxMtcC6nRjz lElXwbhj9ztLM5Y7eDLB1vOWGroXCQW7oSWD6gX3+AsDSkbCfln5vK+w04x60wp7blGS vxeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=wUF41xJvycEPFX/Id9fnR3A/13bzUOgF7zYgrCLBS7I=; b=iKE7LNHlUUIVWWAMpp+QtXnHadWfhDss8yHFAaSrvV7ZVmGV/w+s7QQ666q2BhptCy c6VEXU8XyUwUAOVmO6WCDFeMPVHtYliHr9Iywl71Ubq3ag7h6IHEWWC/yuPu737ePF6/ E4WEhPh9dfNU93xlq0HLckKj1gy/2tHNGBeW/daSZRFM6xtrt2Sofl9l6CylxD0H0zZk PyAeZmqj2gGBrH2EJnYxm0gQtO6CzsnzxAwRXgZ674UDs94MA8RFXEOUk6ZVLR+OwqkG alrJP5zXD1PpveZZCdV3OGHiyuW8AlSphobESy4Ir7fVJJoy2cjiY0Lel2yTaY2oIBUa wI7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hYhi655K; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g184sor3383771wmg.17.2018.11.27.08.55.50 for (Google Transport Security); Tue, 27 Nov 2018 08:55:50 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hYhi655K; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wUF41xJvycEPFX/Id9fnR3A/13bzUOgF7zYgrCLBS7I=; b=hYhi655KVAzODIHwz7iQnmzXthDAYY41LfSREPv1o2q0Ho8zYBdhiDCL71mGI5LxLG x5M0Af2rFpY6LrOqwUDP2+DBK2on3cHmWQoa8IJg4ewKBIoYpt/PFcb575n9XPGkXxZz eswIh/wUPLjfH4Bn7fVRjOF4gdUcYRbeZ7DVnQ3FSoTNVmE+k4h9YpQNPAtlKCwvl5DD ssj7aLnG7dHkuuNjiLkNE7mxedEjcVQ1WF1RyCp6pSz9Gu/rs35NCWrlMCwkWMQzEJ80 x8x9ljMoRSzw7GYCJ4+aubbx8ILe1dx1CKs1FYUjKrl9By9C76HYLXwm09m/N94KX8qn Tjqw== X-Google-Smtp-Source: AJdET5dbsFWeKT8hUfKOeSM4s2yRBEyIsgZMT/S20E4h8nudfAhdQs/tUHk0clGtoHREwqXrjc0Hzw== X-Received: by 2002:a1c:e90d:: with SMTP id q13mr29603004wmc.106.1543337749713; Tue, 27 Nov 2018 08:55:49 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:55:48 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 01/25] kasan, mm: change hooks signatures Date: Tue, 27 Nov 2018 17:55:19 +0100 Message-Id: <10068968c5dbdd1913bd60f89262e3c1a50f3d38.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN changes the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 43 +++++++++++++++++++++++++++++-------------- include/linux/slab.h | 4 ++-- mm/kasan/kasan.c | 30 ++++++++++++++++++------------ mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 7 files changed, 65 insertions(+), 45 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 46aae129917c..52c86a568a4e 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -51,16 +51,16 @@ void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -105,19 +105,34 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) {} -static inline void kasan_init_slab_obj(struct kmem_cache *cache, - const void *object) {} +static inline void *kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + return (void *)object; +} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/include/linux/slab.h b/include/linux/slab.h index 918f374e7156..351ac48dabc4 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -444,7 +444,7 @@ static __always_inline void *kmem_cache_alloc_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc(s, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -455,7 +455,7 @@ kmem_cache_alloc_node_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc_node(s, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } #endif /* CONFIG_TRACING */ diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..55deff17a4d9 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -474,20 +474,22 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, return (void *)object + cache->kasan_info.free_meta_offset; } -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; if (!(cache->flags & SLAB_KASAN)) - return; + return (void *)object; alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +530,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +540,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +553,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +568,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +578,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index 2a5654bb3b3f..26f60a22e5e0 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3681,7 +3681,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3719,7 +3719,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 58c6c1c2a78e..4190c24ef0e9 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7eb8dc136c1c..5f3504e26d4c 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1204,7 +1204,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1482,7 +1482,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index e3629cd7aff1..fdd4a86aa882 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1372,10 +1372,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2768,7 +2768,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2796,7 +2796,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3784,7 +3784,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3801,8 +3801,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3829,7 +3828,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } From patchwork Tue Nov 27 16:55:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700881 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC21414E2 for ; Tue, 27 Nov 2018 16:55:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ABDD52C431 for ; Tue, 27 Nov 2018 16:55:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9A7B2C43F; Tue, 27 Nov 2018 16:55:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 517ED2C469 for ; Tue, 27 Nov 2018 16:55:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62A4C6B4936; Tue, 27 Nov 2018 11:55:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 562296B4938; Tue, 27 Nov 2018 11:55:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B2AE6B4939; Tue, 27 Nov 2018 11:55:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id CFB136B4936 for ; Tue, 27 Nov 2018 11:55:53 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id a9so18010770wrs.6 for ; Tue, 27 Nov 2018 08:55:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=6oli7mrp3g3rho/Q8+rQCYCaCVtzysg9eZI+aaASr7k=; b=rjTSMiRuhwDozoAZ4J8RDA+ZqOPiFlr3XL9aak9XS2I6S3skS+MfHZA7A9M+6QyPBI +v/Jlj5X9nn7knlWcpi88PRkPcA+eQ4UWEogMXxMS2hfocUYrFtAmRMLiEgNpjN91YtE EqmBBJkOzh9SuTKhgsL59g+MADXbULkS2kqthCOYPzdy/9JizNuaOxN6ZQ3SeO9Q+IbO EWjJL0AezLGX+zgHV7BpdkP2al5NiA/4vMKuS32QQ4okywHOyZxIxGNQNPupukJRfZBj bRjc/LkUd1am+kyZ+Q/NMqZvhRn1pO6sfc5V18iwrkSplm9lAf4QmUADeFis0pskkxdU 26Dg== X-Gm-Message-State: AGRZ1gKNCdKsX74WSzIaaZ1LoxsZT8Ies66SsxYsm9G6eAgym4m+Kwzo a9x3fAelfFxxzn+62yL2BvvP64fIGJRFATbua+1z0JytmYKBCngko4lPnhcaSG8WL1W04GtIs7B CI4V5EO0prfWjWHNBOtDUuu8jbNLvw+qu1K3VAo5EJSN7/GcjDGgVshJYIgYjVPuGgnSvf+ERCC LmQuB2zptPpsSvtEzf07EheSDDz/mSKL36VfRM080STJFTE/oMaCyCQCp5t6/V39cnah/SVbvDr 6j2GSRC5yWY5U3V8zao1dlr57otOc2j+SfYaGq3FtciHL3SjH8dO15OnhNcZwHnvTV6jCytv4XK vOpngzWiAiiUyvaWa9j5mCeYghNTQAp8NwWKSc3w1QrAw1XtgWPIljUDYo7IU0SU8lu2rmOM7M9 Z X-Received: by 2002:a1c:8095:: with SMTP id b143mr27153944wmd.63.1543337753359; Tue, 27 Nov 2018 08:55:53 -0800 (PST) X-Received: by 2002:a1c:8095:: with SMTP id b143mr27153879wmd.63.1543337752168; Tue, 27 Nov 2018 08:55:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337752; cv=none; d=google.com; s=arc-20160816; b=q6IIBiPlTXYK3tMYEUqsjffkZHQFK9IRWoxxQcvvADbLhXa51+SllmrZdxtOxaVZFW 578PF2k05YPzs6VcOcvz8NzUYjnCxNemGu3lWD1cE8Nr+7X/5dkPuHE8lwCeebi/fo27 ahrXjWILbZauhbyYXq/YB0GifupCEujdO20JiMWHlUl94qMXGQ5JhkuTK549mKfCz8oe nGJPv0Fb/ZrD1pVY/+2e6B93+CvNgBBSDDOpYD2e1DkmFWykLLpaL9sowELosKaQr4CO Feeznb5F0POUdI5utSurIP5KABRYJQpHVe6sDopwZ9cqoXSyUnIRrJPnaMT7Kd9xCSPi wzSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=6oli7mrp3g3rho/Q8+rQCYCaCVtzysg9eZI+aaASr7k=; b=NTP7/m2kC8L4rSsR7pjGh6RT9MQ/XGbqv4LW6nrEPP2dC+MMhll6Ze34umkvOJiP9x 1uhsgg0m3txIzA2b555TQH0Z92nz7Yg8FMTynU8pLdiwa+iH90R0s7mSaU40uKTf5x9k 1+EYZS3QPBB9HnvGuVMK4/ANsgJ5M0ZXwK5bMW/a/qzJBIB6gMEgpUOUPPlHS71MKqBx UvVX1C3xfr+LmJJsI0lgN1czwLWjHk+CbC0/p54X3EV10uei3rC8LWAw1OaBibho6E/6 n8Ho748M/1sW18zhYS4WsJHBMY9tMs/h9biXEA5d6YmDN7F+7GiZYlhKE5L8KEJ3u6nE Fniw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aKhKOY0O; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r201sor3377199wmb.26.2018.11.27.08.55.52 for (Google Transport Security); Tue, 27 Nov 2018 08:55:52 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aKhKOY0O; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6oli7mrp3g3rho/Q8+rQCYCaCVtzysg9eZI+aaASr7k=; b=aKhKOY0OenSL2Quk581daGUh2I4SrPb/lcGdloHjtGv3j0n+5niFdy9ol4BcCBQ7ks 62cBb14ijs9os54D4xWD8JNgz79PmXwjxifzFhWrEpNSlggyZ5NU2DESqPbG4cUfrzZK +7Wp5K21Hs+NeLhTYqsmNPpDvO2OCfjxPuncqcytdoDMYnVJn5y5jI8t2/rZUVjVp9ls LbG/My2qhXCzylS+TNZHQ5emt4LmdEcjeI1SmnN3HnBF6cHDho2vC4Tnn4KpNxa+PKT8 9UfKSfQcm+YNGouv2+PZMimvKBXCp9DADxnrFIFJy7bVnUX4aWQFUZfh9p133A9kQ4MT c0Cw== X-Google-Smtp-Source: AJdET5fEpG3BRcVasfBfKMtW8R7+Lp6qKg2+6JWWzRf/Y3iyg9IKemH7sTOMfTXBIo1huXU/BZYsIQ== X-Received: by 2002:a7b:c1d6:: with SMTP id a22mr27114136wmj.48.1543337751557; Tue, 27 Nov 2018 08:55:51 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:55:50 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 02/25] kasan, slub: handle pointer tags in early_kmem_cache_node_alloc Date: Tue, 27 Nov 2018 17:55:20 +0100 Message-Id: <3459b9f5d4daf96668d9579da2cf15eb5523284b.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The previous patch updated KASAN hooks signatures and their usage in SLAB and SLUB code, except for the early_kmem_cache_node_alloc function. This patch handles that function separately, as it requires to reorder some of the initialization code to correctly propagate a tagged pointer in case a tag is assigned by kasan_kmalloc. Signed-off-by: Andrey Konovalov --- mm/slub.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index fdd4a86aa882..8561a32910dd 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3364,16 +3364,16 @@ static void early_kmem_cache_node_alloc(int node) n = page->freelist; BUG_ON(!n); - page->freelist = get_freepointer(kmem_cache_node, n); - page->inuse = 1; - page->frozen = 0; - kmem_cache_node->node[node] = n; #ifdef CONFIG_SLUB_DEBUG init_object(kmem_cache_node, n, SLUB_RED_ACTIVE); init_tracking(kmem_cache_node, n); #endif - kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node), + n = kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node), GFP_KERNEL); + page->freelist = get_freepointer(kmem_cache_node, n); + page->inuse = 1; + page->frozen = 0; + kmem_cache_node->node[node] = n; init_kmem_cache_node(n); inc_slabs_node(kmem_cache_node, node, page->objects); From patchwork Tue Nov 27 16:55:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700899 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E6D2114E2 for ; Tue, 27 Nov 2018 16:56:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D47CC2C449 for ; Tue, 27 Nov 2018 16:56:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C7B5B2C490; Tue, 27 Nov 2018 16:56:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B965D2C4B0 for ; Tue, 27 Nov 2018 16:56:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FD646B4938; Tue, 27 Nov 2018 11:55:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 935216B493C; Tue, 27 Nov 2018 11:55:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6EB4A6B493D; Tue, 27 Nov 2018 11:55:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id D884C6B4938 for ; Tue, 27 Nov 2018 11:55:57 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id t1so17389256wmt.5 for ; Tue, 27 Nov 2018 08:55:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XbeEdOHvERsmHNpZqb6zQ1Htz/eYx+qJrhR+fMpnvF0=; b=JlE7wjPcvk2EwDEHgmzLR1BdZtf70sXYH1E9wolqD0/X5p1ziKot+ovackEvzj98sb xMm4nsrQPY6+PgOXqKKQLFPu8RyuMcI9o3l+nJPkjUuG5h61Vun3PEZEqxR9WGukja6F U4510hzHZIOO5WqdKGUsdJMDK9y/ZoAniGS94krjsVOWCANnAXQQnFZLBaWQU+euS6DE V86ykj2Z1NZwevFgEh/54Cmq3vI5TfZ0Qs5GYoPCV1IvCeeO7SPhWPSqelnFHLwIOFR3 Zx40zXccrNP0dSCmCwZVfr+xS5+RpwDNruPk3gw8u++zQL9Hq7Ai3HoXnXpj5Td9wl8m orWw== X-Gm-Message-State: AA+aEWazRX2fb1bDx48KjIhBHVjhd9cfSlNV4z7qs30XPwJ+alckGRG0 3UD9mzWXlf4F1hdjMaBJPPgNUmhmr4qfFI7YexmNUjlUKvQ+ALBQ0dKhxPnDQSBgcI8ftmhJO2n TXMd9vHP60FTF+xDgiIBPBj1Tv5vM8wIwcsviinS9IXzBKG6AA+msgdCMxaQU/axf1awB0w5zr9 AX/Ei6pTDHPiEYEX88jblXIHp5DpYxfI1mAQwuqaYQ6+SGVaEVjqfGAasN38RIlxnY92LIPgzaY IvqDCL7/Hdztci2RAm30vc2N7bf+Ep9NS4l5idhaaxt7O1qoLgWo3u/DkojWhupjS2eAPb9Ry3l A+P8GaaQvHBqY16AtPR8czJdh5wb+C0WNQXFbDZrLVOqBYS5Y3+QNsTC4rSykR4B/sJ+xklXNEu X X-Received: by 2002:a1c:9692:: with SMTP id y140mr12993662wmd.67.1543337757219; Tue, 27 Nov 2018 08:55:57 -0800 (PST) X-Received: by 2002:a1c:9692:: with SMTP id y140mr12993520wmd.67.1543337754744; Tue, 27 Nov 2018 08:55:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337754; cv=none; d=google.com; s=arc-20160816; b=0L/910LMa/muKk9LD1IpeO/cEMLYLffHruHRsufFxevirD/gm62EGFbEuOzNpxNwtP lVXBh6MlDtlzFQT8GPlPfWNQPW2FQwzlMo8zqSng4Q5+jEhMPgCBwtgfHD5BLDBswoFg QO5nmvGUSt2HjQEH5tlU2RNPyd/s2uUT8RAsdHAAq5JmWoAigwZBsppEDHEqCalZYkqt TZKmMucgyCF6CCFo/DbKc2vk4bF6QUGTI0rQ51TggB6mpI2bx4+0O3SfzoqhNsRlJ9Xx upfcdVUTbulvTsNv981XjBbIm8lUPUup8HBfbRtgVjHbfDA8xgPCkU8VFzd+YgdXl8az h83w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=XbeEdOHvERsmHNpZqb6zQ1Htz/eYx+qJrhR+fMpnvF0=; b=hhNIqeD97KKY+Lr9rjbt4qipDwMUJFcziMuquODk5WdfZTA7bd/qEjzRsP6nfS+St0 ugmbiUbDfulgtO6uh28pJqU35hnJTsE1L3I+SdaNpePZ3mQe2kGiXH7Gy+pVUcDjZn54 IMx6h4mjdMDfOoTdrKd3SRuVWKTK/IqOzbfGEa1g5Ikyh4EWw7PxIe9q2vcwYOIjPYrx UAmIhUqjGQYGCchh8PlxnohqFTLUP6yyJVmNzz/k1Nup2vkatn8yKlothKni+MkYzWmI soqEOzJm1UpdkOnhewLl0o2GFGx/sya9N6aKwDB2dOCqmgNJCfA0bmb1T0HKgkzN0vW2 8uQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="n0JzTqr/"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 125sor1754394wmm.15.2018.11.27.08.55.54 for (Google Transport Security); Tue, 27 Nov 2018 08:55:54 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="n0JzTqr/"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XbeEdOHvERsmHNpZqb6zQ1Htz/eYx+qJrhR+fMpnvF0=; b=n0JzTqr//C8A5zQxhqz25Ufo8ZtoYA62Wq2h1zL2KYhlOAiEHpscGNEhZp60fTKQ86 Nfl3cLlpabvQVOMkwUkRu5BMZyfanMsJmhWNL1sm0s4KnbwT7O14zh2B/ylVD4d0yfSx bd8olI/zyHZvgu+5fmokGhYoZlMuInSCVXk+GVV1iwaYFrasBR4yT1x+PCN0HBGhcxYc t44kbQGHpc5hjOdYzan5gKZH2Hkvl8d/ahDWyVDZz5jyvrKi8QjpIqd2rGI6GeW+iIlO gXvqRFaPuTnRfq7nCNNtO9phTp1K8mV2EUOmZwTaj33ILf7HfKv+mqkqPByqRzFDuZh6 pYzw== X-Google-Smtp-Source: AJdET5etnyD3SOhFZz5Vr758nLN0U4kmIImfLFdDBNDfWhITx/4sRUtgChcDuSHXuTYSN2HcKbhUPw== X-Received: by 2002:a1c:1801:: with SMTP id 1mr30184298wmy.153.1543337753506; Tue, 27 Nov 2018 08:55:53 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:55:52 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 03/25] kasan: move common generic and tag-based code to common.c Date: Tue, 27 Nov 2018 17:55:21 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN reuses a significant part of the generic KASAN code, so move the common parts to common.c without any functional changes. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 603 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 570 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 614 insertions(+), 569 deletions(-) create mode 100644 mm/kasan/common.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 3289db38bc87..a6df14bffb6b 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o diff --git a/mm/kasan/common.c b/mm/kasan/common.c new file mode 100644 index 000000000000..5f68c93734ba --- /dev/null +++ b/mm/kasan/common.c @@ -0,0 +1,603 @@ +/* + * This file contains common generic and tag-based KASAN code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +static inline void filter_irq_stacks(struct stack_trace *trace) +{ + int i; + + if (!trace->nr_entries) + return; + for (i = 0; i < trace->nr_entries; i++) + if (in_irqentry_text(trace->entries[i])) { + /* Include the irqentry function into the stack. */ + trace->nr_entries = i + 1; + break; + } +} + +static inline depot_stack_handle_t save_stack(gfp_t flags) +{ + unsigned long entries[KASAN_STACK_DEPTH]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = KASAN_STACK_DEPTH, + .skip = 0 + }; + + save_stack_trace(&trace); + filter_irq_stacks(&trace); + if (trace.nr_entries != 0 && + trace.entries[trace.nr_entries-1] == ULONG_MAX) + trace.nr_entries--; + + return depot_save_stack(&trace, flags); +} + +static inline void set_track(struct kasan_track *track, gfp_t flags) +{ + track->pid = current->pid; + track->stack = save_stack(flags); +} + +void kasan_enable_current(void) +{ + current->kasan_depth++; +} + +void kasan_disable_current(void) +{ + current->kasan_depth--; +} + +void kasan_check_read(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_read); + +void kasan_check_write(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + check_memory_region((unsigned long)addr, len, true, _RET_IP_); + + return __memset(addr, c, len); +} + +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memmove(dest, src, len); +} + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + */ +void kasan_poison_shadow(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void kasan_unpoison_shadow(const void *address, size_t size) +{ + kasan_poison_shadow(address, size, 0); + + if (size & KASAN_SHADOW_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + *shadow = size & KASAN_SHADOW_MASK; + } +} + +static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) +{ + void *base = task_stack_page(task); + size_t size = sp - base; + + kasan_unpoison_shadow(base, size); +} + +/* Unpoison the entire stack for a task. */ +void kasan_unpoison_task_stack(struct task_struct *task) +{ + __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); +} + +/* Unpoison the stack for the current task beyond a watermark sp value. */ +asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) +{ + /* + * Calculate the task stack base address. Avoid using 'current' + * because this function is called by early resume code which hasn't + * yet set up the percpu register (%gs). + */ + void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); + + kasan_unpoison_shadow(base, watermark - base); +} + +/* + * Clear all poison for the region between the current SP and a provided + * watermark value, as is sometimes required prior to hand-crafted asm function + * returns in the middle of functions. + */ +void kasan_unpoison_stack_above_sp_to(const void *watermark) +{ + const void *sp = __builtin_frame_address(0); + size_t size = watermark - sp; + + if (WARN_ON(sp > watermark)) + return; + kasan_unpoison_shadow(sp, size); +} + +void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); +} + +void kasan_free_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_poison_shadow(page_address(page), + PAGE_SIZE << order, + KASAN_FREE_PAGE); +} + +/* + * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. + * For larger allocations larger redzones are used. + */ +static inline unsigned int optimal_redzone(unsigned int object_size) +{ + return + object_size <= 64 - 16 ? 16 : + object_size <= 128 - 32 ? 32 : + object_size <= 512 - 64 ? 64 : + object_size <= 4096 - 128 ? 128 : + object_size <= (1 << 14) - 256 ? 256 : + object_size <= (1 << 15) - 512 ? 512 : + object_size <= (1 << 16) - 1024 ? 1024 : 2048; +} + +void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) +{ + unsigned int orig_size = *size; + int redzone_adjust; + + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } + redzone_adjust = optimal_redzone(cache->object_size) - + (*size - cache->object_size); + + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + + optimal_redzone(cache->object_size))); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } + + *flags |= SLAB_KASAN; +} + +size_t kasan_metadata_size(struct kmem_cache *cache) +{ + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); +} + +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); + return (void *)object + cache->kasan_info.alloc_meta_offset; +} + +struct kasan_free_meta *get_free_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + return (void *)object + cache->kasan_info.free_meta_offset; +} + +void kasan_poison_slab(struct page *page) +{ + kasan_poison_shadow(page_address(page), + PAGE_SIZE << compound_order(page), + KASAN_KMALLOC_REDZONE); +} + +void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_unpoison_shadow(object, cache->object_size); +} + +void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_poison_shadow(object, + round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + KASAN_KMALLOC_REDZONE); +} + +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; +} + +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +{ + return kasan_kmalloc(cache, object, cache->object_size, flags); +} + +static bool __kasan_slab_free(struct kmem_cache *cache, void *object, + unsigned long ip, bool quarantine) +{ + s8 shadow_byte; + unsigned long rounded_up_size; + + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != + object)) { + kasan_report_invalid_free(object, ip); + return true; + } + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + kasan_report_invalid_free(object, ip); + return true; + } + + rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + + if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; +} + +bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +{ + return __kasan_slab_free(cache, object, ip, true); +} + +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, + gfp_t flags) +{ + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(object == NULL)) + return NULL; + + redzone_start = round_up((unsigned long)(object + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = round_up((unsigned long)object + cache->object_size, + KASAN_SHADOW_SCALE_SIZE); + + kasan_unpoison_shadow(object, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); + + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; +} +EXPORT_SYMBOL(kasan_kmalloc); + +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + struct page *page; + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(ptr == NULL)) + return NULL; + + page = virt_to_page(ptr); + redzone_start = round_up((unsigned long)(ptr + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); + + kasan_unpoison_shadow(ptr, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); + + return (void *)ptr; +} + +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +{ + struct page *page; + + if (unlikely(object == ZERO_SIZE_PTR)) + return (void *)object; + + page = virt_to_head_page(object); + + if (unlikely(!PageSlab(page))) + return kasan_kmalloc_large(object, size, flags); + else + return kasan_kmalloc(page->slab_cache, object, size, flags); +} + +void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), + KASAN_FREE_PAGE); + } else { + __kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + +void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (ptr != page_address(virt_to_head_page(ptr))) + kasan_report_invalid_free(ptr, ip); + /* The object will be poisoned by page_alloc. */ +} + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL | __GFP_ZERO, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || + WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 55deff17a4d9..44ec228de0a2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,5 +1,5 @@ /* - * This file contains shadow memory manipulation code. + * This file contains core KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -40,82 +40,6 @@ #include "kasan.h" #include "../slab.h" -void kasan_enable_current(void) -{ - current->kasan_depth++; -} - -void kasan_disable_current(void) -{ - current->kasan_depth--; -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. - */ -static void kasan_poison_shadow(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - memset(shadow_start, value, shadow_end - shadow_start); -} - -void kasan_unpoison_shadow(const void *address, size_t size) -{ - kasan_poison_shadow(address, size, 0); - - if (size & KASAN_SHADOW_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; - } -} - -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_shadow(base, size); -} - -/* Unpoison the entire stack for a task. */ -void kasan_unpoison_task_stack(struct task_struct *task) -{ - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); -} - -/* Unpoison the stack for the current task beyond a watermark sp value. */ -asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) -{ - /* - * Calculate the task stack base address. Avoid using 'current' - * because this function is called by early resume code which hasn't - * yet set up the percpu register (%gs). - */ - void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - - kasan_unpoison_shadow(base, watermark - base); -} - -/* - * Clear all poison for the region between the current SP and a provided - * watermark value, as is sometimes required prior to hand-crafted asm function - * returns in the middle of functions. - */ -void kasan_unpoison_stack_above_sp_to(const void *watermark) -{ - const void *sp = __builtin_frame_address(0); - size_t size = watermark - sp; - - if (WARN_ON(sp > watermark)) - return; - kasan_unpoison_shadow(sp, size); -} - /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -260,121 +184,12 @@ static __always_inline void check_memory_region_inline(unsigned long addr, kasan_report(addr, size, write, ret_ip); } -static void check_memory_region(unsigned long addr, - size_t size, bool write, +void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { check_memory_region_inline(addr, size, write, ret_ip); } -void kasan_check_read(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_read); - -void kasan_check_write(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - check_memory_region((unsigned long)addr, len, true, _RET_IP_); - - return __memset(addr, c, len); -} - -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memmove(dest, src, len); -} - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memcpy(dest, src, len); -} - -void kasan_alloc_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); -} - -void kasan_free_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); -} - -/* - * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. - * For larger allocations larger redzones are used. - */ -static unsigned int optimal_redzone(unsigned int object_size) -{ - return - object_size <= 64 - 16 ? 16 : - object_size <= 128 - 32 ? 32 : - object_size <= 512 - 64 ? 64 : - object_size <= 4096 - 128 ? 128 : - object_size <= (1 << 14) - 256 ? 256 : - object_size <= (1 << 15) - 512 ? 512 : - object_size <= (1 << 16) - 1024 ? 1024 : 2048; -} - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) -{ - unsigned int orig_size = *size; - int redzone_adjust; - - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); - - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); - - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; - } - - *flags |= SLAB_KASAN; -} - void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); @@ -386,277 +201,6 @@ void kasan_cache_shutdown(struct kmem_cache *cache) quarantine_remove_cache(cache); } -size_t kasan_metadata_size(struct kmem_cache *cache) -{ - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); -} - -void kasan_poison_slab(struct page *page) -{ - kasan_poison_shadow(page_address(page), - PAGE_SIZE << compound_order(page), - KASAN_KMALLOC_REDZONE); -} - -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_unpoison_shadow(object, cache->object_size); -} - -void kasan_poison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_poison_shadow(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), - KASAN_KMALLOC_REDZONE); -} - -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline void filter_irq_stacks(struct stack_trace *trace) -{ - int i; - - if (!trace->nr_entries) - return; - for (i = 0; i < trace->nr_entries; i++) - if (in_irqentry_text(trace->entries[i])) { - /* Include the irqentry function into the stack. */ - trace->nr_entries = i + 1; - break; - } -} - -static inline depot_stack_handle_t save_stack(gfp_t flags) -{ - unsigned long entries[KASAN_STACK_DEPTH]; - struct stack_trace trace = { - .nr_entries = 0, - .entries = entries, - .max_entries = KASAN_STACK_DEPTH, - .skip = 0 - }; - - save_stack_trace(&trace); - filter_irq_stacks(&trace); - if (trace.nr_entries != 0 && - trace.entries[trace.nr_entries-1] == ULONG_MAX) - trace.nr_entries--; - - return depot_save_stack(&trace, flags); -} - -static inline void set_track(struct kasan_track *track, gfp_t flags) -{ - track->pid = current->pid; - track->stack = save_stack(flags); -} - -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); - return (void *)object + cache->kasan_info.alloc_meta_offset; -} - -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; -} - -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) -{ - struct kasan_alloc_meta *alloc_info; - - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); - - return (void *)object; -} - -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) -{ - return kasan_kmalloc(cache, object, cache->object_size, flags); -} - -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, - unsigned long ip, bool quarantine) -{ - s8 shadow_byte; - unsigned long rounded_up_size; - - if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != - object)) { - kasan_report_invalid_free(object, ip); - return true; - } - - /* RCU slabs could be legally used after free within the RCU period */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); - return true; - } - - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) - return false; - - set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); - quarantine_put(get_free_info(cache, object), cache); - return true; -} - -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) -{ - return __kasan_slab_free(cache, object, ip, true); -} - -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) -{ - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(object == NULL)) - return NULL; - - redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); - - kasan_unpoison_shadow(object, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); - - if (cache->flags & SLAB_KASAN) - set_track(&get_alloc_info(cache, object)->alloc_track, flags); - - return (void *)object; -} -EXPORT_SYMBOL(kasan_kmalloc); - -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) -{ - struct page *page; - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(ptr == NULL)) - return NULL; - - page = virt_to_page(ptr); - redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); - - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); - - return (void *)ptr; -} - -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) -{ - struct page *page; - - if (unlikely(object == ZERO_SIZE_PTR)) - return ZERO_SIZE_PTR; - - page = virt_to_head_page(object); - - if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); - else - return kasan_kmalloc(page->slab_cache, object, size, flags); -} - -void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), - KASAN_FREE_PAGE); - } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - -void kasan_kfree_large(void *ptr, unsigned long ip) -{ - if (ptr != page_address(virt_to_head_page(ptr))) - kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ -} - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); @@ -797,113 +341,3 @@ DEFINE_ASAN_SET_SHADOW(f2); DEFINE_ASAN_SET_SHADOW(f3); DEFINE_ASAN_SET_SHADOW(f5); DEFINE_ASAN_SET_SHADOW(f8); - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c12dcfde2ebd..659463800f10 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow(const void *address, size_t size, u8 value); + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); From patchwork Tue Nov 27 16:55:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700885 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD6D6109C for ; Tue, 27 Nov 2018 16:56:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9DC7D2C4AB for ; Tue, 27 Nov 2018 16:56:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 912582C495; Tue, 27 Nov 2018 16:56:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17F792C49E for ; Tue, 27 Nov 2018 16:56:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00D0C6B493A; Tue, 27 Nov 2018 11:55:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ED73A6B493C; Tue, 27 Nov 2018 11:55:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDB2C6B493B; Tue, 27 Nov 2018 11:55:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 6FDDD6B4938 for ; Tue, 27 Nov 2018 11:55:57 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id e17so18224235wrw.13 for ; Tue, 27 Nov 2018 08:55:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9dHZ8S+aXQverKkY9zPXDntt+u3jEBNyQXkWLj8HFag=; b=OCIMULe4ecuDnKokDCUbO+9J21lvUbFRL5FubqEe/b0UIu30os5bbJmm5JUJur8CB8 D+J8Hpr8bqMmcbkuzCDt43tMA5/SSFs2RL2LPlkQUFq6ohvP6Qcsiprnw5TVfjMbDiVF q/b6YfecfLrPP/b1GfoOcme5fHP1i8YjEMCmP08KJlOneqjsVTXD3Q14lKdvKuBQ0YBJ e/f4Hbx7WwzBjIf8+D0T1euy6du0VV0KhgP+CxQ3XdXXdvQ3hSxO9oqwSWvWsNG6uUy+ 8My0okTGOnig9zP3m2LtRJzm5vShIkgU7perFyLQRFkTD8N1oq0DElU1ET4/uNda0hVV WXhg== X-Gm-Message-State: AA+aEWY/wn1csW4kFv/T8ma/hf/JLDbxULUlVShtk7Q0I+z7H6I2tnVh FChN9Fx5wPy1yO8uCJJ4spjkxgSQszLF0lObq6deRDSQAN5Bw1x8MqbFlsXMD9n+K5jtWdChp1e zqubU7EtMzZJ6xva7AznSAHZwHfloMWxlECD6ZPgQOvgTgUiWEjPeWhimoMRIOCir72QNNyasdK PnhLWjPi7kWiCjNellfwJtY2NdCAFAXbGasfzdQ/x0iuC8nCilh9fylJPFmcRSA6ILvB6AR4uXp Wv0xpXM4wOHzeL7x2bTrsARJ3lgwuv+JhzMAVoAbTCJAfJG0ZSJ9prE2IIjhhpZ1I0LxNNRoKgF ByfyslZAjHOoL9GDXRgyTvm+pDNny9z3t7cCdATbiTubc09UD3FT77cYwqWxY5dgTKTWa8H+af7 o X-Received: by 2002:a1c:b189:: with SMTP id a131mr25951257wmf.38.1543337756968; Tue, 27 Nov 2018 08:55:56 -0800 (PST) X-Received: by 2002:a1c:b189:: with SMTP id a131mr25951193wmf.38.1543337756037; Tue, 27 Nov 2018 08:55:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337756; cv=none; d=google.com; s=arc-20160816; b=NngDYNIupg8cBwiVoD8BfKHt0h5+qVgIbPi/D9EHe9rWrtnAcyq/jKCLBQtP0BrV9M Qy14A/TI/E8VUkQxjT+GPqW7BYuKaML2rKJsBhXrAWZbTCOhjk20MRz0goLEbh08K9sB UUqEbHpG6uzPClPnvTxO1BjEZ/OOLJjTmZ17hCKQPcUdW33UUKA0Ht4M3WtpVjUpKlVv cOZZUZf0a2uynxOPG1NK/yvo/Jb2f2G0ySrzySmjqhMZL/KiKnkMcJItdoiZxxPJNWdR HH31FfzorZaNKPkGZh6JCGmzgbLLaYmKy9AZA2nI2aThlPc4Vgp/kwKN3JqhMjPKa4Iw OvGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=9dHZ8S+aXQverKkY9zPXDntt+u3jEBNyQXkWLj8HFag=; b=UBhpy+cN83/WC1mWfvg8kIhhg6L8/HAy6KtsrIej8td8OcgPM4ve8fezX+ZWlHPL+F A8FX0XV6/hSXLNJ2PP4bp75auZgEtK3c4sWYzn8Y4YGXiRxRXpx5K+87GL10+Z6RdYgL DvWqE6en7Rgg1K6dAGxvpNPCOuodtXvmFGiE3sWTK5XLgQYT0jU/4T8ODU3O07JPHzzF FVehXzUlGuRW8TM0tcjMSvYOa5hX7CsqSA5Z+qbu/xy3x7saKPkF9ppX34ntNuydI0AB yP8vgW9h3DLM1FSDln2y37zPdvWkRyX33KNAfXnznwiHVdIq0e5sAR44AMliGAr3m5ha m4GA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=h4b6yJLu; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t141sor3446528wme.23.2018.11.27.08.55.55 for (Google Transport Security); Tue, 27 Nov 2018 08:55:55 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=h4b6yJLu; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9dHZ8S+aXQverKkY9zPXDntt+u3jEBNyQXkWLj8HFag=; b=h4b6yJLum0eMnzXMvBzkeSRQ8LHm7k1FjgBW2udsyFDjw+uZtUe6MkbTPWyvgw2RIq B4gc1vfeKlBDo/vrTNe8s7rCUXIB36UrMpV8+SebqCgnHfUgzULzwYjNnZ2+UjGbgj3p +8KAMsOBjeqw3GFRTxskIkWm4zjf3iPC7H6DvYaJEXdPlX1AGzWlbYqr6n6+Pg8GBdKb 8Z7/tmDjexFinDc5Q5DseAbVzZ6G2/DX9qR0qM2XvQEBe1qUiiYK42LoCU5ztSenxHBo WNNToPCtNyYpUb6GVr1vViuXPcoQpSaOQrcdHdJUf66asvaUUzfXyy1Q2A/snH8qQdHp qsGw== X-Google-Smtp-Source: AFSGD/W+Kasr/UmYVWnUYOlOX9N6HnDivsR1UWtpolJAvkPPDE1ylEorpdF9y76ljI7/gq7RsZ2ZaQ== X-Received: by 2002:a1c:c2d5:: with SMTP id s204mr18111130wmf.144.1543337755358; Tue, 27 Nov 2018 08:55:55 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:55:54 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 04/25] kasan: rename source files to reflect the new naming scheme Date: Tue, 27 Nov 2018 17:55:22 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We now have two KASAN modes: generic KASAN and tag-based KASAN. Rename kasan.c to generic.c to reflect that. Also rename kasan_init.c to init.c as it contains initialization code for both KASAN modes. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 8 ++++---- mm/kasan/{kasan.c => generic.c} | 0 mm/kasan/{kasan_init.c => init.c} | 0 3 files changed, 4 insertions(+), 4 deletions(-) rename mm/kasan/{kasan.c => generic.c} (100%) rename mm/kasan/{kasan_init.c => init.c} (100%) diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index a6df14bffb6b..d643530b24aa 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,14 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n -UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_generic.o := n KCOV_INSTRUMENT := n -CFLAGS_REMOVE_kasan.o = -pg +CFLAGS_REMOVE_generic.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o generic.o report.o init.o quarantine.o diff --git a/mm/kasan/kasan.c b/mm/kasan/generic.c similarity index 100% rename from mm/kasan/kasan.c rename to mm/kasan/generic.c diff --git a/mm/kasan/kasan_init.c b/mm/kasan/init.c similarity index 100% rename from mm/kasan/kasan_init.c rename to mm/kasan/init.c From patchwork Tue Nov 27 16:55:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700905 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2A6B18B8 for ; Tue, 27 Nov 2018 16:56:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEB742C493 for ; Tue, 27 Nov 2018 16:56:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ABED42C4B0; Tue, 27 Nov 2018 16:56:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4527F2C493 for ; Tue, 27 Nov 2018 16:56:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E3F336B493C; Tue, 27 Nov 2018 11:56:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DC4EF6B493E; Tue, 27 Nov 2018 11:56:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3DB66B493F; Tue, 27 Nov 2018 11:56:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 4E53F6B493C for ; Tue, 27 Nov 2018 11:56:00 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id v24so6896947wrd.23 for ; Tue, 27 Nov 2018 08:56:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wfaNmKDTONvVhpEcl0AfBuc32R+W952i00DYtoym9bQ=; b=VnVPcXNoqaRL45cgSJedIZt/bLozC9NOthJ70g4KnKHrwOw4kufEWz4psbITayoscE RUQD2nYN5vdwVJmYTRo1ppbjgqOOqa1/Jh+Xt2o26FDSMK7aQGV60qBAXcueLG3VnxJO 2rkGmVoZlh5R9goz9neIaQWxrbJxbJyz7Cf7u+5jpFu3Y4PGifkrowylhey9mV9/pxTI id+EUQqjtttI5HfkCo4Pq5h+5RN4g1ctPnEax1ERrEtCKXYDZgpqezPsVg//oEqdcY1I s/joZF2klGt+Krljp48hz9upDYKEGZAS5aMVpfPwpA6sZDLdZjAJIi50+uBnEIwVSC5I qr6w== X-Gm-Message-State: AA+aEWYNKec65K6utbjvL3c7QiEzSizqzCeP+cc9rKoTBvZu0QZRQ1X1 DO9KuFO9zHSH+XPE9+u/IVhy1mExma+T8SgNBi04IzjGdCMNxrUgNHiV/ifT3iYPlMFq+rHtjPj kKhFfOY3z36R+WFGklFl2/ICmhAOdLG5zUJ6oqwJdGf/XNvDS7MbolEbNQ0QzKyPvvfIqie4+vd f3F60rJ3tnL6VqlirbyEBthymlcW0SvwUR5K312ewHc3rj2cA1jyOvUP3eAJPm0012AFbP5jlel 2ezXclmoOOmew94PDyHMM0VaydqTF7UueqHBNP/nuUvjZfpJ8Ezk7vYsKT2iYhjYtr5KuONGVDq PKBYTrhtBptKgacPxlxls4X8a/g5ADTMk6XgwXZnVOOMs+VYuW8PnG3V0X51O8REeZ6ffn1h+Wv H X-Received: by 2002:a1c:c483:: with SMTP id u125mr27750748wmf.14.1543337759739; Tue, 27 Nov 2018 08:55:59 -0800 (PST) X-Received: by 2002:a1c:c483:: with SMTP id u125mr27750656wmf.14.1543337758161; Tue, 27 Nov 2018 08:55:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337758; cv=none; d=google.com; s=arc-20160816; b=eMNjtUarzCcGvyZnw02SfyxPFWdXrL0lqL7+XBdMYVS61L5feOM4eMMvG8+VNmsT+2 WHQwmkQsWwByNI4+Ewi5oexAK5B7r3EV2fTN+IRWCMedx+4Q7nb8zT+Wnt5g3PAqb7Nf oP+ptAvMHKa++7GplkE4gqyvju9wy1PCnotq9kxDYmvfedT20x2zCYicCqSicmzKEFqH xPj2Gk2CIiyxkq8x75HASfW/grQCY4zAidl1Um6Yclk9cGI438dqojuKB9HVkR4IdmR8 ioOUQoyMHIXXMKR12Kn0u3v2Uuxy1phlwPUxiyA9nDldRc2xSLLyITF3dImJDpMTbu+i TgmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=wfaNmKDTONvVhpEcl0AfBuc32R+W952i00DYtoym9bQ=; b=DlrEwL5XXdHxaVrSK65b8d2HGMx/YxsQZn0HrE1gLfdVavCL0KBF+wUe9jE2IgXVnM Ng6aKmX++HxJ4xlsBUdIVaY5TZRSiZmKH7FgxS5tYBlPzymsxk5jnkx8lEiaKIQQIAtV 8mIOFlNDLMJOhQefpyWIruGJZtRUqL5imBYeJYp+ApronOov3J5zhs+kQACDdGnrpKJw z+3oYGykgjmmMigmlE8l9UJXDMnAGZBSV0jsvHnacrIGWbDfV59EUx1O566imPUhH1Ws 1dnyW3yNCagE3FxavAT2oKFwoOWW2nChj6zZzikuOZ9eRgisZ0DrHl14OERXSjvkH6Rv vbnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="tb17/PKM"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l13sor3134187wre.39.2018.11.27.08.55.58 for (Google Transport Security); Tue, 27 Nov 2018 08:55:58 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="tb17/PKM"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wfaNmKDTONvVhpEcl0AfBuc32R+W952i00DYtoym9bQ=; b=tb17/PKM9cjidZDWsn29+tfOA0gu5vBXqRhGJq1ApgNvpDIWR3VtNDvLhVpRcokW63 W7/2TGdPX2bjyIPdggoBBkQsLis4EV7caZ4dR+uIKq9+bra6VO+wEw/fdR3a1oOuoVV6 sHIZbGARuU01sGlI6AGLed5RDoXoYT5n83np0dSfErV+A2nHwCXGIsIovX61VB3oQSwo AbwSLsEV2vxUOgfr30W5QqyRr0rDXTu355CeIXDscbbPtIIX6FnAUN8uZmdjcTva7Jox 0DYEAwedR9wA52qmKTZsLC64H+ykgxFswVSiM7JFhqsCUVxF+7Jt4xCrmBR2sS0IO+L6 ZFxQ== X-Google-Smtp-Source: AFSGD/XUYfmeclVX0+gEszAXGStTeHVu80o8mG76kh7julwARZ1Fuomb/k1USzu19rr9e9qSnr1JMw== X-Received: by 2002:adf:c108:: with SMTP id r8mr15940507wre.233.1543337757318; Tue, 27 Nov 2018 08:55:57 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:55:56 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 05/25] kasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS Date: Tue, 27 Nov 2018 17:55:23 +0100 Message-Id: <20728567aae93b5eb88a6636c94c1af73db7cdbc.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit splits the current CONFIG_KASAN config option into two: 1. CONFIG_KASAN_GENERIC, that enables the generic KASAN mode (the one that exists now); 2. CONFIG_KASAN_SW_TAGS, that enables the software tag-based KASAN mode. The name CONFIG_KASAN_SW_TAGS is chosen as in the future we will have another hardware tag-based KASAN mode, that will rely on hardware memory tagging support in arm64. With CONFIG_KASAN_SW_TAGS enabled, compiler options are changed to instrument kernel files with -fsantize=kernel-hwaddress (except the ones for which KASAN_SANITIZE := n is set). Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes. This commit also adds empty placeholder (for now) implementation of tag-based KASAN specific hooks inserted by the compiler and adjusts common hooks implementation. While this commit adds the CONFIG_KASAN_SW_TAGS config option, this option is not selectable, as it depends on HAVE_ARCH_KASAN_SW_TAGS, which we will enable once all the infrastracture code has been added. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/compiler-clang.h | 6 +- include/linux/compiler-gcc.h | 6 ++ include/linux/compiler_attributes.h | 13 ---- include/linux/kasan.h | 16 +++-- lib/Kconfig.kasan | 96 +++++++++++++++++++++++------ mm/kasan/Makefile | 6 +- mm/kasan/generic.c | 2 +- mm/kasan/kasan.h | 3 +- mm/kasan/tags.c | 75 ++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 53 +++++++++------- 11 files changed, 216 insertions(+), 62 deletions(-) create mode 100644 mm/kasan/tags.c diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 3e7dafb3ea80..39f668d5066b 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -16,9 +16,13 @@ /* all clang versions usable with the kernel support KASAN ABI version 5 */ #define KASAN_ABI_VERSION 5 +#if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) /* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature(address_sanitizer) #define __SANITIZE_ADDRESS__ +#define __no_sanitize_address \ + __attribute__((no_sanitize("address", "hwaddress"))) +#else +#define __no_sanitize_address #endif /* diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 2010493e1040..5776da43da97 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -143,6 +143,12 @@ #define KASAN_ABI_VERSION 3 #endif +#if __has_attribute(__no_sanitize_address__) +#define __no_sanitize_address __attribute__((no_sanitize_address)) +#else +#define __no_sanitize_address +#endif + #if GCC_VERSION >= 50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1 #endif diff --git a/include/linux/compiler_attributes.h b/include/linux/compiler_attributes.h index f8c400ba1929..7bceb9469197 100644 --- a/include/linux/compiler_attributes.h +++ b/include/linux/compiler_attributes.h @@ -206,19 +206,6 @@ */ #define __noreturn __attribute__((__noreturn__)) -/* - * Optional: only supported since gcc >= 4.8 - * Optional: not supported by icc - * - * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute - * clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis - */ -#if __has_attribute(__no_sanitize_address__) -# define __no_sanitize_address __attribute__((__no_sanitize_address__)) -#else -# define __no_sanitize_address -#endif - /* * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Type-Attributes.html#index-packed-type-attribute * clang: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#index-packed-variable-attribute diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 52c86a568a4e..b66fdf5ea7ab 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); -void kasan_cache_shrink(struct kmem_cache *cache); -void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); @@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} -static inline void kasan_cache_shrink(struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, @@ -155,4 +151,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink(struct kmem_cache *cache); +void kasan_cache_shutdown(struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink(struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ + #endif /* LINUX_KASAN_H */ diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index d0bad1bd9a2b..b5d0cbfce4a1 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -1,35 +1,95 @@ +# This config refers to the generic KASAN mode. config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_SW_TAGS + bool + +config CC_HAS_KASAN_GENERIC + def_bool $(cc-option, -fsanitize=kernel-address) + +config CC_HAS_KASAN_SW_TAGS + def_bool $(cc-option, -fsanitize=kernel-hwaddress) + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs. + See Documentation/dev-tools/kasan.rst for details. + +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: generic KASAN (similar to userspace ASan, + x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and + software tag-based KASAN (a version based on software memory + tagging, arm64 only, similar to userspace HWASan, enabled with + CONFIG_KASAN_SW_TAGS). + Both generic and tag-based KASAN are strictly debugging features. + +config KASAN_GENERIC + bool "Generic mode" + depends on CC_HAS_KASAN_GENERIC depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables generic KASAN mode. + Supported in both GCC and Clang. With GCC it requires version 4.9.2 + or later for basic support and version 5.0 or later for detection of + out-of-bounds accesses for stack and global variables and for inline + instrumentation mode (CONFIG_KASAN_INLINE). With Clang it requires + version 3.7.0 or later and it doesn't support detection of + out-of-bounds accesses for global variables yet. + This mode consumes about 1/8th of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +if HAVE_ARCH_KASAN_SW_TAGS + +config KASAN_SW_TAGS + bool "Software tag-based mode" + depends on CC_HAS_KASAN_SW_TAGS + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enables software tag-based KASAN mode. + This mode requires Top Byte Ignore support by the CPU and therefore + is only supported for arm64. + This mode requires Clang version 7.0.0 or later. + This mode consumes about 1/16th of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + This mode may potentially introduce problems relating to pointer + casting and comparison, as it embeds tags into the top byte of each + pointer. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer + This enables further checks in generic KASAN, for now it only + includes the address-use-after-scope check that can lead to + excessive kernel stack usage, frame size warnings and longer compile time. - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more + See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 choice @@ -53,7 +113,7 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice @@ -67,11 +127,11 @@ config KASAN_S390_4_LEVEL_PAGING 4-level paging instead. config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endif diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index d643530b24aa..68ba1822f003 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_generic.o := n +UBSAN_SANITIZE_tags.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_generic.o = -pg @@ -10,5 +11,8 @@ CFLAGS_REMOVE_generic.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o generic.o report.o init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 44ec228de0a2..b8de6d33c55c 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,5 +1,5 @@ /* - * This file contains core KASAN code. + * This file contains core generic KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 659463800f10..19b950eaccff 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB) +#if defined(CONFIG_KASAN_GENERIC) && \ + (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c new file mode 100644 index 000000000000..04194923c543 --- /dev/null +++ b/mm/kasan/tags.c @@ -0,0 +1,75 @@ +/* + * This file contains core tag-based KASAN code. + * + * Copyright (c) 2018 Google, Inc. + * Author: Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE(size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE(1); +DEFINE_HWASAN_LOAD_STORE(2); +DEFINE_HWASAN_LOAD_STORE(4); +DEFINE_HWASAN_LOAD_STORE(8); +DEFINE_HWASAN_LOAD_STORE(16); + +void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_storeN_noabort); + +void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_tag_memory); diff --git a/mm/slub.c b/mm/slub.c index 8561a32910dd..e739d46600b9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2992,7 +2992,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page, do_slab_free(s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr) { do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr); diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 69552a39951d..25c259df8ffa 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC + ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else @@ -12,36 +13,44 @@ CFLAGS_KASAN_MINIMAL := -fsanitize=kernel-address cc-param = $(call cc-option, -mllvm -$(1), $(call cc-option, --param $(1))) -ifeq ($(call cc-option, $(CFLAGS_KASAN_MINIMAL) -Werror),) - ifneq ($(CONFIG_COMPILE_TEST),y) - $(warning Cannot use CONFIG_KASAN: \ - -fsanitize=kernel-address is not supported by compiler) - endif -else - # -fasan-shadow-offset fails without -fsanitize - CFLAGS_KASAN_SHADOW := $(call cc-option, -fsanitize=kernel-address \ +# -fasan-shadow-offset fails without -fsanitize +CFLAGS_KASAN_SHADOW := $(call cc-option, -fsanitize=kernel-address \ -fasan-shadow-offset=$(KASAN_SHADOW_OFFSET), \ $(call cc-option, -fsanitize=kernel-address \ -mllvm -asan-mapping-offset=$(KASAN_SHADOW_OFFSET))) - ifeq ($(strip $(CFLAGS_KASAN_SHADOW)),) - CFLAGS_KASAN := $(CFLAGS_KASAN_MINIMAL) - else - # Now add all the compiler specific options that are valid standalone - CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ - $(call cc-param,asan-globals=1) \ - $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ - $(call cc-param,asan-stack=1) \ - $(call cc-param,asan-use-after-scope=1) \ - $(call cc-param,asan-instrument-allocas=1) - endif - +ifeq ($(strip $(CFLAGS_KASAN_SHADOW)),) + CFLAGS_KASAN := $(CFLAGS_KASAN_MINIMAL) +else + # Now add all the compiler specific options that are valid standalone + CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ + $(call cc-param,asan-globals=1) \ + $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ + $(call cc-param,asan-stack=1) \ + $(call cc-param,asan-use-after-scope=1) \ + $(call cc-param,asan-instrument-allocas=1) endif ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif # CONFIG_KASAN_GENERIC +ifdef CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif + +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +endif # CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif From patchwork Tue Nov 27 16:55:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700909 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE8A81869 for ; Tue, 27 Nov 2018 16:56:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC9422C4AB for ; Tue, 27 Nov 2018 16:56:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C97EC2C4B7; Tue, 27 Nov 2018 16:56:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59E812C431 for ; Tue, 27 Nov 2018 16:56:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F3C26B493E; Tue, 27 Nov 2018 11:56:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 02E716B4941; Tue, 27 Nov 2018 11:56:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEE066B4942; Tue, 27 Nov 2018 11:56:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 7CD746B493E for ; Tue, 27 Nov 2018 11:56:01 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id y85so18175495wmc.7 for ; Tue, 27 Nov 2018 08:56:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xTUWDQtoCoS0aCkqKcHlSfKscpiq6EmalPX21Pfc5fY=; b=Gf0ViYonmJJVGVBByDxRobVLGEAwCt9g6H/KdT+7UxenZAXR3ayd0y4NIoKH5lsINM Q53RDFII6prG2oc063kX293EN1F0dX4yUGzCf+Pxa7nOYxuo77JglLv3LlJ65JB8ZjK8 KIu/hfDIyY0HCdNUhEhmCOj5J4Sfuqlp2KfS2L+8+aMna6rjuJF33vp9YtF2Lt75cG4W X1cUOirPm9nqexEHrFBjJuPtJDPMUhp3C+WmGxXJV3pLkhqAvKMFWLV//OUtpEvWdmD/ Boq5Sdwmo/h3UZZm4TBINhHWUt2BHzHGKVMdN+WAezxEVNn+WP1M/w7GOeM9PqLOFi8P yzBg== X-Gm-Message-State: AGRZ1gJa11ojb8slwFftlmkSPwvo1Ui1sjSqWdZgWWek2k3Agr8KMvHX UjRkeToUXeVyYXvyspl6vAYQRy+nnqUXNHc0WUIDc63nA7wX0fLKCG0iNVUe1O+rCYoTwLuhWD6 e+gVLeZQy2dD967cw+AwPozF2I94ma98uiAbUkOtViULaod+Z762wNuAww5OU80Z5p/FAGtCV8R HVtJHVpPDrTTXCOth+nOtgd2W8Uvyptp/r/sK+VeZwSNvKbeKEAwZCi4VdCPwbEoZoowX2xD7AX ysQsUntpLvzFODndOVoWlH1qlVjNcFWuhoSeb53mDj0gKh1As8ob/fUWCyPGy2E/CsXIF3dYw1b TNMLQK5zvi0Fe0PwCbl6Q8VgXdx/4aaEkTzUozJNGdZ9HYR6bn+se9a5NKL75H6+46k1j2fjhjX e X-Received: by 2002:a1c:1c86:: with SMTP id c128mr27395340wmc.107.1543337760969; Tue, 27 Nov 2018 08:56:00 -0800 (PST) X-Received: by 2002:a1c:1c86:: with SMTP id c128mr27395291wmc.107.1543337759988; Tue, 27 Nov 2018 08:55:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337759; cv=none; d=google.com; s=arc-20160816; b=llFr+Ovi+2tZo27ChS0QtqIAEn0u4ki+K7f8Rq8VpFg5aW6IMyyj/nVxq2AX4XtTT/ JiUQNnZ7ytxJpc3lqNZ0yKO3A0Y4+1ZmYL6a5D3WH+OkALXZ7a8fqstpPHWJG3cv+Hd8 v3RMCdweVWNk3pgg3MJQFzrL1QTMMWahnoxh8DUPN7MZcc+aQLfBXmOVuHHPTR2x3q7K HycwFE6+P23C1gP0V3XYpM0S1rPYbJOp/K+MPrhkIO6pvhUBCuoqPQRC/MbJ/2M9KFyF OJQDuxImb4iEsn9USk0Ja7sp+vAyOUKwlwaiAJkcDsT/QvoIZlrGnZxD4TYqa16jv3gT 3vhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=xTUWDQtoCoS0aCkqKcHlSfKscpiq6EmalPX21Pfc5fY=; b=unRW26htBnjnGVfsG8eP4zE5LCyI/uaIkUAS1uUR1FzgTqfZEvAdyxi/TLKK0RcMaL pj+RVcEWAN4AO/UO87ZEd/1Z7APDxVfvwFuTTj2IjXRpV3seyh7XOR0FYmacWEFfvQOt tPq87j5Apbs7af2poS2H4QqrAOt2nwCyvGIZFAnvnjG6+zNV0wqhdFZ1yS05p1CO0OZU +Lf4Ga926pklofPnC4xMnZVNQZG0Ut1kynW4BIxHb0KEyWYKUCe6mvpk0M5v8Mh/Ro+E kxpqmPHCQExtpeukH04CayUtf4eXt7Adtdj19YMF+nijDCBSlJurNKKG262afXmR+I2l /dHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=dwUU3Biv; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l127sor3247071wmb.7.2018.11.27.08.55.59 for (Google Transport Security); Tue, 27 Nov 2018 08:55:59 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=dwUU3Biv; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xTUWDQtoCoS0aCkqKcHlSfKscpiq6EmalPX21Pfc5fY=; b=dwUU3BivFrGTU0l5vocuel01QRAE+v2OGhkebJVpy0mfJmO48WW8JjO/aQRDNIxlzp 7Q7mQuPIYO6ENLlB1vL2wNEuEIHZp+ISZDSJKEFWk4t6VwdTO034MsDUqOMVSpnvYHaG EJcyDLbTK9jrS7IEcf09GuU+VgeRwknC0Tlm4O/P5QswfFrC1hiDKmoKuUuLwhOvaaB1 eOqzl7HmKc1Hk+kdVzfJIn3rAmpOXnEkoLkjR1OyF9bg+r+lmbARfTC79ZfPy4UEbmPK TLasPIfGNrBaUGy9nXGM879QdxieJEgd31QTd/Q6LhFdvEscYdJa6SZolCyN4ZLBGe0E +eCQ== X-Google-Smtp-Source: AFSGD/WWRQkAxqiKVcqGdnvTy6zSnKmrDpYhgJDtIRt2URzdNRxagoHNN/L5SvG9mNp5eIbIhcSMqw== X-Received: by 2002:a1c:9692:: with SMTP id y140mr12993788wmd.67.1543337759395; Tue, 27 Nov 2018 08:55:59 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:55:58 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 06/25] kasan, arm64: adjust shadow size for tag-based mode Date: Tue, 27 Nov 2018 17:55:24 +0100 Message-Id: <95fa472a03a8ff268e24b8730ebd108922824e74.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory. This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when the tag-based KASAN mode is enabled. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/Makefile | 11 ++++++++++- arch/arm64/include/asm/memory.h | 7 +++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 6cb9fc7e9382..99e7d08c6083 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -91,10 +91,19 @@ else TEXT_OFFSET := 0x00080000 endif +ifeq ($(CONFIG_KASAN_SW_TAGS), y) +KASAN_SHADOW_SCALE_SHIFT := 4 +else +KASAN_SHADOW_SCALE_SHIFT := 3 +endif + +KBUILD_CFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) +KBUILD_CPPFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) +KBUILD_AFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) + # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b96442960aea..05fbc7ffcd31 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -74,12 +74,11 @@ #define KERNEL_END _end /* - * KASAN requires 1/8th of the kernel virtual address space for the shadow - * region. KASAN can bloat the stack significantly, so double the (minimum) - * stack size when KASAN is in use. + * Generic and tag-based KASAN require 1/8th and 1/16th of the kernel virtual + * address space for the shadow region respectively. They can bloat the stack + * significantly, so double the (minimum) stack size when they are in use. */ #ifdef CONFIG_KASAN -#define KASAN_SHADOW_SCALE_SHIFT 3 #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_THREAD_SHIFT 1 #else From patchwork Tue Nov 27 16:55:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700917 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82389109C for ; Tue, 27 Nov 2018 16:56:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F8872C4B2 for ; Tue, 27 Nov 2018 16:56:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 638232C446; Tue, 27 Nov 2018 16:56:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C92262C4B5 for ; Tue, 27 Nov 2018 16:56:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 195256B4941; Tue, 27 Nov 2018 11:56:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 11BEE6B4944; Tue, 27 Nov 2018 11:56:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEF316B4941; Tue, 27 Nov 2018 11:56:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 670C76B4941 for ; Tue, 27 Nov 2018 11:56:04 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id e17so18224358wrw.13 for ; Tue, 27 Nov 2018 08:56:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+9PASwcFfpBLwSrOwdc0AKJIhUGrXXEqSi5SjKKHdKY=; b=jE91bEWCMzRSnRiqq59p/vcbaRmoNU7BBaX+o9Q6sEl2nuFNRut8863Ck1qeuih+Nv XF0O3oYgKWXqJ3ZIvP9eWWW40RDGdUc70n5UxdHqJayfpeEc9t8xbrNi3qHraQa2LRqn RqsxtGx1owzMd/5nnOQv27LObm6AyU4C1Yi8oTUvNSOMCM0hf0pXXjDWYec4Eb6dtZ7Q 87BgCLZXONRDR5UvdgYxNl4gz/C69qWYIBsjHw724boSpKPmyC7Mbg58cw1aF70VdR3x ompTLntMLb1tFCvFVH+cNLc1eQEl7JzifpTOjkzrZou4kxpTKx42jUdddVk5eHf2Ldpv 2BcQ== X-Gm-Message-State: AA+aEWYwYiNdiYoQwOfJTi4eZ73vjYJdDhr9BW9jHlf8b5m/a1cdX8/M WNWlQmyYxx+Pu9mFHjEF4dpFWht/LvC8/7SJCkkw8/dPSBBP6YVA51ii8RU5+4jPgfepTTn5h4I DiSuQxqDMU1uEeJhIMzevshRgYsSI82YwdrNW9cObN5VPcNIc431sn+BLrVj2SEPpCZQxdUkPFF YtLY1EtLSA22QW5vev/RnO3OIuoO8dMK25qVBDvf2ecuZPsKwOH5WK8fkwkU7bDfIuYeogsDt9E 021VrGlYhbP/qDjV7AeulmP8HniaCLQTUOa3Yzcbklp1YJgRy8xGceHA+Wi8k4LMi/5RGVVvvmF PzHA2TnkAD1OydQgyKMAtwfFMwj7Y0EhJJw7sBGJQxW3yYLREXpigi4Elo+nTOizfs18G1ko27V N X-Received: by 2002:a05:6000:f:: with SMTP id h15mr27657919wrx.248.1543337763876; Tue, 27 Nov 2018 08:56:03 -0800 (PST) X-Received: by 2002:a05:6000:f:: with SMTP id h15mr27657828wrx.248.1543337762174; Tue, 27 Nov 2018 08:56:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337762; cv=none; d=google.com; s=arc-20160816; b=E8/4dPoPLzRDPbyncFcV/jyfS6p0IqnROsj4kiCvm09dv+Grj6+ctGisxkgbiA4+19 DJHFPc+sy4/tVJ5mf/426IQmJZyDGokVB7Ip3vCvjXfzVBDAPlkKICbAwJT7jLDoDVhQ aaCGs0hTQnvqRXfn7CQqPpHMH7fzQFIFpCBclUg3r7ywFqKjicyc/KH5aeEy2lFmTeVL Fy96acw4gUgW9au+LqGzSiJM/0hVbod6iEudH7/M6F/MNpaC9SY9zKhHLls4OJ2RfWy5 L2I4lta6MSCN94+QTb7DqrRJkLgStSRBAg7+uMgPPFzUfT01SeWeTQRum4LWJtDUBxAM Z+HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=+9PASwcFfpBLwSrOwdc0AKJIhUGrXXEqSi5SjKKHdKY=; b=wTg5JZXn6fQdYAfIbrPAEJd5gOccdDymA0iN3K2wV+P5eYfr8bBxlZmtkZRrJQE7/Q ul9JxM15le6eQcvWddL9arfsDnaR5HgaHgkNBhwAWFwJHMC4T5rQJZMSQXa+uSDuzaeu SbNJZ3791kQ1hxLIDsEKDr+QBaWmQMv2aeqMNGfLIi4hP9L5/FqbR5ifLCI/B4wgx8iy fzDZ+SJqjD9PHy5D6WTU/SnkS+fOPFbwnVNJloZKhufaNYxGUyB+j2rwp/KEkjNRwSmm ynR1NWe3Qlq4X28tJFGxb9YkzNtkbtN7hie4M8AtdELqbLFHH3aDsa69OxMgVnEhrLkl nx/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="u/tHG7L/"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i18sor3125296wrw.50.2018.11.27.08.56.02 for (Google Transport Security); Tue, 27 Nov 2018 08:56:02 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="u/tHG7L/"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+9PASwcFfpBLwSrOwdc0AKJIhUGrXXEqSi5SjKKHdKY=; b=u/tHG7L/dvzXO2Z5xqv5L5gom3lFGmLsX7vVGU3qPNV5x4MLKRQwycUOifAbttsa4m ZvfEPPzmxM1/tDWJaUAwQn2EcDG39xu0t2aT03r/UYRdiWnGaMZX8++ZYWlyrA9jt1At ix01aJC3UMH4eLOM4EFhwYq7YJzBbHXOeomWC0QTp57nZbj9udbNjfw7V4nZGXU99r6U kT5QlOOuiFDBSHhjWdndolUwx9T62YImsfthNi2pNijFd4DtQcU0bho3OTqxyi5T6Wfz WdlAyunoD/gA7nXEyHIqPv53u8+3cFoxiFTdfJTtHBs9xyIKWMhsiX+PVm9yCyF4N5cX owuA== X-Google-Smtp-Source: AFSGD/WyX7Oc54FZ4FIdPTTCIIIhOaHS2bfDZhgW5z/SZTk1Hy43v05HyH+kROj1PpvLUIhysHUBoA== X-Received: by 2002:adf:9dd2:: with SMTP id q18mr28381177wre.12.1543337761151; Tue, 27 Nov 2018 08:56:01 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.55.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:00 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 07/25] kasan: rename kasan_zero_page to kasan_early_shadow_page Date: Tue, 27 Nov 2018 17:55:25 +0100 Message-Id: <1aa5a8e562380e0bfb1d58c8ec3130436cff8b47.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With tag based KASAN mode the early shadow value is 0xff and not 0x00, so this patch renames kasan_zero_(page|pte|pmd|pud|p4d) to kasan_early_shadow_(page|pte|pmd|pud|p4d) to avoid confusion. Suggested-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 43 ++++++++++++--------- arch/s390/mm/dump_pagetables.c | 17 +++++---- arch/s390/mm/kasan_init.c | 33 +++++++++------- arch/x86/mm/dump_pagetables.c | 11 +++--- arch/x86/mm/kasan_init_64.c | 55 +++++++++++++------------- arch/xtensa/mm/kasan_init.c | 18 +++++---- include/linux/kasan.h | 12 +++--- mm/kasan/init.c | 70 +++++++++++++++++++--------------- 8 files changed, 145 insertions(+), 114 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 63527e585aac..4ebc19422931 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -47,8 +47,9 @@ static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { if (pmd_none(READ_ONCE(*pmdp))) { - phys_addr_t pte_phys = early ? __pa_symbol(kasan_zero_pte) - : kasan_alloc_zeroed_page(node); + phys_addr_t pte_phys = early ? + __pa_symbol(kasan_early_shadow_pte) + : kasan_alloc_zeroed_page(node); __pmd_populate(pmdp, pte_phys, PMD_TYPE_TABLE); } @@ -60,8 +61,9 @@ static pmd_t *__init kasan_pmd_offset(pud_t *pudp, unsigned long addr, int node, bool early) { if (pud_none(READ_ONCE(*pudp))) { - phys_addr_t pmd_phys = early ? __pa_symbol(kasan_zero_pmd) - : kasan_alloc_zeroed_page(node); + phys_addr_t pmd_phys = early ? + __pa_symbol(kasan_early_shadow_pmd) + : kasan_alloc_zeroed_page(node); __pud_populate(pudp, pmd_phys, PMD_TYPE_TABLE); } @@ -72,8 +74,9 @@ static pud_t *__init kasan_pud_offset(pgd_t *pgdp, unsigned long addr, int node, bool early) { if (pgd_none(READ_ONCE(*pgdp))) { - phys_addr_t pud_phys = early ? __pa_symbol(kasan_zero_pud) - : kasan_alloc_zeroed_page(node); + phys_addr_t pud_phys = early ? + __pa_symbol(kasan_early_shadow_pud) + : kasan_alloc_zeroed_page(node); __pgd_populate(pgdp, pud_phys, PMD_TYPE_TABLE); } @@ -87,8 +90,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, pte_t *ptep = kasan_pte_offset(pmdp, addr, node, early); do { - phys_addr_t page_phys = early ? __pa_symbol(kasan_zero_page) - : kasan_alloc_zeroed_page(node); + phys_addr_t page_phys = early ? + __pa_symbol(kasan_early_shadow_page) + : kasan_alloc_zeroed_page(node); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -205,14 +209,14 @@ void __init kasan_init(void) kasan_map_populate(kimg_shadow_start, kimg_shadow_end, early_pfn_to_nid(virt_to_pfn(lm_alias(_text)))); - kasan_populate_zero_shadow((void *)KASAN_SHADOW_START, - (void *)mod_shadow_start); - kasan_populate_zero_shadow((void *)kimg_shadow_end, - kasan_mem_to_shadow((void *)PAGE_OFFSET)); + kasan_populate_early_shadow((void *)KASAN_SHADOW_START, + (void *)mod_shadow_start); + kasan_populate_early_shadow((void *)kimg_shadow_end, + kasan_mem_to_shadow((void *)PAGE_OFFSET)); if (kimg_shadow_start > mod_shadow_end) - kasan_populate_zero_shadow((void *)mod_shadow_end, - (void *)kimg_shadow_start); + kasan_populate_early_shadow((void *)mod_shadow_end, + (void *)kimg_shadow_start); for_each_memblock(memory, reg) { void *start = (void *)__phys_to_virt(reg->base); @@ -227,14 +231,15 @@ void __init kasan_init(void) } /* - * KAsan may reuse the contents of kasan_zero_pte directly, so we - * should make sure that it maps the zero page read-only. + * KAsan may reuse the contents of kasan_early_shadow_pte directly, + * so we should make sure that it maps the zero page read-only. */ for (i = 0; i < PTRS_PER_PTE; i++) - set_pte(&kasan_zero_pte[i], - pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO)); + set_pte(&kasan_early_shadow_pte[i], + pfn_pte(sym_to_pfn(kasan_early_shadow_page), + PAGE_KERNEL_RO)); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/arch/s390/mm/dump_pagetables.c b/arch/s390/mm/dump_pagetables.c index 363f6470d742..3b93ba0b5d8d 100644 --- a/arch/s390/mm/dump_pagetables.c +++ b/arch/s390/mm/dump_pagetables.c @@ -111,11 +111,12 @@ static void note_page(struct seq_file *m, struct pg_state *st, } #ifdef CONFIG_KASAN -static void note_kasan_zero_page(struct seq_file *m, struct pg_state *st) +static void note_kasan_early_shadow_page(struct seq_file *m, + struct pg_state *st) { unsigned int prot; - prot = pte_val(*kasan_zero_pte) & + prot = pte_val(*kasan_early_shadow_pte) & (_PAGE_PROTECT | _PAGE_INVALID | _PAGE_NOEXEC); note_page(m, st, prot, 4); } @@ -154,8 +155,8 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_zero_pmd)) { - note_kasan_zero_page(m, st); + if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_early_shadow_pmd)) { + note_kasan_early_shadow_page(m, st); return; } #endif @@ -185,8 +186,8 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_zero_pud)) { - note_kasan_zero_page(m, st); + if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_early_shadow_pud)) { + note_kasan_early_shadow_page(m, st); return; } #endif @@ -215,8 +216,8 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_zero_p4d)) { - note_kasan_zero_page(m, st); + if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_early_shadow_p4d)) { + note_kasan_early_shadow_page(m, st); return; } #endif diff --git a/arch/s390/mm/kasan_init.c b/arch/s390/mm/kasan_init.c index acb9645b762b..bac5c27d11fc 100644 --- a/arch/s390/mm/kasan_init.c +++ b/arch/s390/mm/kasan_init.c @@ -107,7 +107,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PGDIR_SIZE) && end - address >= PGDIR_SIZE) { - pgd_populate(&init_mm, pg_dir, kasan_zero_p4d); + pgd_populate(&init_mm, pg_dir, + kasan_early_shadow_p4d); address = (address + PGDIR_SIZE) & PGDIR_MASK; continue; } @@ -120,7 +121,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, P4D_SIZE) && end - address >= P4D_SIZE) { - p4d_populate(&init_mm, p4_dir, kasan_zero_pud); + p4d_populate(&init_mm, p4_dir, + kasan_early_shadow_pud); address = (address + P4D_SIZE) & P4D_MASK; continue; } @@ -133,7 +135,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PUD_SIZE) && end - address >= PUD_SIZE) { - pud_populate(&init_mm, pu_dir, kasan_zero_pmd); + pud_populate(&init_mm, pu_dir, + kasan_early_shadow_pmd); address = (address + PUD_SIZE) & PUD_MASK; continue; } @@ -146,7 +149,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PMD_SIZE) && end - address >= PMD_SIZE) { - pmd_populate(&init_mm, pm_dir, kasan_zero_pte); + pmd_populate(&init_mm, pm_dir, + kasan_early_shadow_pte); address = (address + PMD_SIZE) & PMD_MASK; continue; } @@ -188,7 +192,7 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, pte_val(*pt_dir) = __pa(page) | pgt_prot; break; case POPULATE_ZERO_SHADOW: - page = kasan_zero_page; + page = kasan_early_shadow_page; pte_val(*pt_dir) = __pa(page) | pgt_prot_zero; break; } @@ -256,14 +260,14 @@ void __init kasan_early_init(void) unsigned long vmax; unsigned long pgt_prot = pgprot_val(PAGE_KERNEL_RO); pte_t pte_z; - pmd_t pmd_z = __pmd(__pa(kasan_zero_pte) | _SEGMENT_ENTRY); - pud_t pud_z = __pud(__pa(kasan_zero_pmd) | _REGION3_ENTRY); - p4d_t p4d_z = __p4d(__pa(kasan_zero_pud) | _REGION2_ENTRY); + pmd_t pmd_z = __pmd(__pa(kasan_early_shadow_pte) | _SEGMENT_ENTRY); + pud_t pud_z = __pud(__pa(kasan_early_shadow_pmd) | _REGION3_ENTRY); + p4d_t p4d_z = __p4d(__pa(kasan_early_shadow_pud) | _REGION2_ENTRY); kasan_early_detect_facilities(); if (!has_nx) pgt_prot &= ~_PAGE_NOEXEC; - pte_z = __pte(__pa(kasan_zero_page) | pgt_prot); + pte_z = __pte(__pa(kasan_early_shadow_page) | pgt_prot); memsize = get_mem_detect_end(); if (!memsize) @@ -292,10 +296,13 @@ void __init kasan_early_init(void) } /* init kasan zero shadow */ - crst_table_init((unsigned long *)kasan_zero_p4d, p4d_val(p4d_z)); - crst_table_init((unsigned long *)kasan_zero_pud, pud_val(pud_z)); - crst_table_init((unsigned long *)kasan_zero_pmd, pmd_val(pmd_z)); - memset64((u64 *)kasan_zero_pte, pte_val(pte_z), PTRS_PER_PTE); + crst_table_init((unsigned long *)kasan_early_shadow_p4d, + p4d_val(p4d_z)); + crst_table_init((unsigned long *)kasan_early_shadow_pud, + pud_val(pud_z)); + crst_table_init((unsigned long *)kasan_early_shadow_pmd, + pmd_val(pmd_z)); + memset64((u64 *)kasan_early_shadow_pte, pte_val(pte_z), PTRS_PER_PTE); shadow_alloc_size = memsize >> KASAN_SHADOW_SCALE_SHIFT; pgalloc_low = round_up((unsigned long)_end, _SEGMENT_SIZE); diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index fc37bbd23eb8..c4696ab9a72b 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -380,7 +380,7 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, /* * This is an optimization for KASAN=y case. Since all kasan page tables - * eventually point to the kasan_zero_page we could call note_page() + * eventually point to the kasan_early_shadow_page we could call note_page() * right away without walking through lower level page tables. This saves * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. @@ -388,10 +388,11 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, void *pt) { - if (__pa(pt) == __pa(kasan_zero_pmd) || - (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_zero_p4d)) || - __pa(pt) == __pa(kasan_zero_pud)) { - pgprotval_t prot = pte_flags(kasan_zero_pte[0]); + if (__pa(pt) == __pa(kasan_early_shadow_pmd) || + (pgtable_l5_enabled() && + __pa(pt) == __pa(kasan_early_shadow_p4d)) || + __pa(pt) == __pa(kasan_early_shadow_pud)) { + pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); note_page(m, st, __pgprot(prot), 0, 5); return true; } diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 04a9cf6b034f..462fde83b515 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -211,7 +211,8 @@ static void __init kasan_early_p4d_populate(pgd_t *pgd, unsigned long next; if (pgd_none(*pgd)) { - pgd_entry = __pgd(_KERNPG_TABLE | __pa_nodebug(kasan_zero_p4d)); + pgd_entry = __pgd(_KERNPG_TABLE | + __pa_nodebug(kasan_early_shadow_p4d)); set_pgd(pgd, pgd_entry); } @@ -222,7 +223,8 @@ static void __init kasan_early_p4d_populate(pgd_t *pgd, if (!p4d_none(*p4d)) continue; - p4d_entry = __p4d(_KERNPG_TABLE | __pa_nodebug(kasan_zero_pud)); + p4d_entry = __p4d(_KERNPG_TABLE | + __pa_nodebug(kasan_early_shadow_pud)); set_p4d(p4d, p4d_entry); } while (p4d++, addr = next, addr != end && p4d_none(*p4d)); } @@ -261,10 +263,11 @@ static struct notifier_block kasan_die_notifier = { void __init kasan_early_init(void) { int i; - pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC; - pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE; - pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE; - p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE; + pteval_t pte_val = __pa_nodebug(kasan_early_shadow_page) | + __PAGE_KERNEL | _PAGE_ENC; + pmdval_t pmd_val = __pa_nodebug(kasan_early_shadow_pte) | _KERNPG_TABLE; + pudval_t pud_val = __pa_nodebug(kasan_early_shadow_pmd) | _KERNPG_TABLE; + p4dval_t p4d_val = __pa_nodebug(kasan_early_shadow_pud) | _KERNPG_TABLE; /* Mask out unsupported __PAGE_KERNEL bits: */ pte_val &= __default_kernel_pte_mask; @@ -273,16 +276,16 @@ void __init kasan_early_init(void) p4d_val &= __default_kernel_pte_mask; for (i = 0; i < PTRS_PER_PTE; i++) - kasan_zero_pte[i] = __pte(pte_val); + kasan_early_shadow_pte[i] = __pte(pte_val); for (i = 0; i < PTRS_PER_PMD; i++) - kasan_zero_pmd[i] = __pmd(pmd_val); + kasan_early_shadow_pmd[i] = __pmd(pmd_val); for (i = 0; i < PTRS_PER_PUD; i++) - kasan_zero_pud[i] = __pud(pud_val); + kasan_early_shadow_pud[i] = __pud(pud_val); for (i = 0; pgtable_l5_enabled() && i < PTRS_PER_P4D; i++) - kasan_zero_p4d[i] = __p4d(p4d_val); + kasan_early_shadow_p4d[i] = __p4d(p4d_val); kasan_map_early_shadow(early_top_pgt); kasan_map_early_shadow(init_top_pgt); @@ -326,7 +329,7 @@ void __init kasan_init(void) clear_pgds(KASAN_SHADOW_START & PGDIR_MASK, KASAN_SHADOW_END); - kasan_populate_zero_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK), + kasan_populate_early_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK), kasan_mem_to_shadow((void *)PAGE_OFFSET)); for (i = 0; i < E820_MAX_ENTRIES; i++) { @@ -338,41 +341,41 @@ void __init kasan_init(void) shadow_cpu_entry_begin = (void *)CPU_ENTRY_AREA_BASE; shadow_cpu_entry_begin = kasan_mem_to_shadow(shadow_cpu_entry_begin); - shadow_cpu_entry_begin = (void *)round_down((unsigned long)shadow_cpu_entry_begin, - PAGE_SIZE); + shadow_cpu_entry_begin = (void *)round_down( + (unsigned long)shadow_cpu_entry_begin, PAGE_SIZE); shadow_cpu_entry_end = (void *)(CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE); shadow_cpu_entry_end = kasan_mem_to_shadow(shadow_cpu_entry_end); - shadow_cpu_entry_end = (void *)round_up((unsigned long)shadow_cpu_entry_end, - PAGE_SIZE); + shadow_cpu_entry_end = (void *)round_up( + (unsigned long)shadow_cpu_entry_end, PAGE_SIZE); - kasan_populate_zero_shadow( + kasan_populate_early_shadow( kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM), shadow_cpu_entry_begin); kasan_populate_shadow((unsigned long)shadow_cpu_entry_begin, (unsigned long)shadow_cpu_entry_end, 0); - kasan_populate_zero_shadow(shadow_cpu_entry_end, - kasan_mem_to_shadow((void *)__START_KERNEL_map)); + kasan_populate_early_shadow(shadow_cpu_entry_end, + kasan_mem_to_shadow((void *)__START_KERNEL_map)); kasan_populate_shadow((unsigned long)kasan_mem_to_shadow(_stext), (unsigned long)kasan_mem_to_shadow(_end), early_pfn_to_nid(__pa(_stext))); - kasan_populate_zero_shadow(kasan_mem_to_shadow((void *)MODULES_END), - (void *)KASAN_SHADOW_END); + kasan_populate_early_shadow(kasan_mem_to_shadow((void *)MODULES_END), + (void *)KASAN_SHADOW_END); load_cr3(init_top_pgt); __flush_tlb_all(); /* - * kasan_zero_page has been used as early shadow memory, thus it may - * contain some garbage. Now we can clear and write protect it, since - * after the TLB flush no one should write to it. + * kasan_early_shadow_page has been used as early shadow memory, thus + * it may contain some garbage. Now we can clear and write protect it, + * since after the TLB flush no one should write to it. */ - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); for (i = 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; @@ -380,8 +383,8 @@ void __init kasan_init(void) prot = __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC); pgprot_val(prot) &= __default_kernel_pte_mask; - pte = __pte(__pa(kasan_zero_page) | pgprot_val(prot)); - set_pte(&kasan_zero_pte[i], pte); + pte = __pte(__pa(kasan_early_shadow_page) | pgprot_val(prot)); + set_pte(&kasan_early_shadow_pte[i], pte); } /* Flush TLBs again to be sure that write protection applied. */ __flush_tlb_all(); diff --git a/arch/xtensa/mm/kasan_init.c b/arch/xtensa/mm/kasan_init.c index 6b95ca43aec0..1734cda6bc4a 100644 --- a/arch/xtensa/mm/kasan_init.c +++ b/arch/xtensa/mm/kasan_init.c @@ -24,12 +24,13 @@ void __init kasan_early_init(void) int i; for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_zero_pte + i, - mk_pte(virt_to_page(kasan_zero_page), PAGE_KERNEL)); + set_pte(kasan_early_shadow_pte + i, + mk_pte(virt_to_page(kasan_early_shadow_page), + PAGE_KERNEL)); for (vaddr = 0; vaddr < KASAN_SHADOW_SIZE; vaddr += PMD_SIZE, ++pmd) { BUG_ON(!pmd_none(*pmd)); - set_pmd(pmd, __pmd((unsigned long)kasan_zero_pte)); + set_pmd(pmd, __pmd((unsigned long)kasan_early_shadow_pte)); } early_trap_init(); } @@ -80,13 +81,16 @@ void __init kasan_init(void) populate(kasan_mem_to_shadow((void *)VMALLOC_START), kasan_mem_to_shadow((void *)XCHAL_KSEG_BYPASS_VADDR)); - /* Write protect kasan_zero_page and zero-initialize it again. */ + /* + * Write protect kasan_early_shadow_page and zero-initialize it again. + */ for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_zero_pte + i, - mk_pte(virt_to_page(kasan_zero_page), PAGE_KERNEL_RO)); + set_pte(kasan_early_shadow_pte + i, + mk_pte(virt_to_page(kasan_early_shadow_page), + PAGE_KERNEL_RO)); local_flush_tlb_all(); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); /* At this point kasan is fully initialized. Enable error messages. */ current->kasan_depth = 0; diff --git a/include/linux/kasan.h b/include/linux/kasan.h index b66fdf5ea7ab..ec22d548d0d7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -14,13 +14,13 @@ struct task_struct; #include #include -extern unsigned char kasan_zero_page[PAGE_SIZE]; -extern pte_t kasan_zero_pte[PTRS_PER_PTE]; -extern pmd_t kasan_zero_pmd[PTRS_PER_PMD]; -extern pud_t kasan_zero_pud[PTRS_PER_PUD]; -extern p4d_t kasan_zero_p4d[MAX_PTRS_PER_P4D]; +extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; +extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE]; +extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD]; +extern pud_t kasan_early_shadow_pud[PTRS_PER_PUD]; +extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; -int kasan_populate_zero_shadow(const void *shadow_start, +int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); static inline void *kasan_mem_to_shadow(const void *addr) diff --git a/mm/kasan/init.c b/mm/kasan/init.c index c7550eb65922..2b21d3717d62 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -30,13 +30,13 @@ * - Latter it reused it as zero shadow to cover large ranges of memory * that allowed to access, but not handled by kasan (vmalloc/vmemmap ...). */ -unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; +unsigned char kasan_early_shadow_page[PAGE_SIZE] __page_aligned_bss; #if CONFIG_PGTABLE_LEVELS > 4 -p4d_t kasan_zero_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; +p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; static inline bool kasan_p4d_table(pgd_t pgd) { - return pgd_page(pgd) == virt_to_page(lm_alias(kasan_zero_p4d)); + return pgd_page(pgd) == virt_to_page(lm_alias(kasan_early_shadow_p4d)); } #else static inline bool kasan_p4d_table(pgd_t pgd) @@ -45,10 +45,10 @@ static inline bool kasan_p4d_table(pgd_t pgd) } #endif #if CONFIG_PGTABLE_LEVELS > 3 -pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; +pud_t kasan_early_shadow_pud[PTRS_PER_PUD] __page_aligned_bss; static inline bool kasan_pud_table(p4d_t p4d) { - return p4d_page(p4d) == virt_to_page(lm_alias(kasan_zero_pud)); + return p4d_page(p4d) == virt_to_page(lm_alias(kasan_early_shadow_pud)); } #else static inline bool kasan_pud_table(p4d_t p4d) @@ -57,10 +57,10 @@ static inline bool kasan_pud_table(p4d_t p4d) } #endif #if CONFIG_PGTABLE_LEVELS > 2 -pmd_t kasan_zero_pmd[PTRS_PER_PMD] __page_aligned_bss; +pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD] __page_aligned_bss; static inline bool kasan_pmd_table(pud_t pud) { - return pud_page(pud) == virt_to_page(lm_alias(kasan_zero_pmd)); + return pud_page(pud) == virt_to_page(lm_alias(kasan_early_shadow_pmd)); } #else static inline bool kasan_pmd_table(pud_t pud) @@ -68,16 +68,16 @@ static inline bool kasan_pmd_table(pud_t pud) return 0; } #endif -pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; +pte_t kasan_early_shadow_pte[PTRS_PER_PTE] __page_aligned_bss; static inline bool kasan_pte_table(pmd_t pmd) { - return pmd_page(pmd) == virt_to_page(lm_alias(kasan_zero_pte)); + return pmd_page(pmd) == virt_to_page(lm_alias(kasan_early_shadow_pte)); } -static inline bool kasan_zero_page_entry(pte_t pte) +static inline bool kasan_early_shadow_page_entry(pte_t pte) { - return pte_page(pte) == virt_to_page(lm_alias(kasan_zero_page)); + return pte_page(pte) == virt_to_page(lm_alias(kasan_early_shadow_page)); } static __init void *early_alloc(size_t size, int node) @@ -92,7 +92,8 @@ static void __ref zero_pte_populate(pmd_t *pmd, unsigned long addr, pte_t *pte = pte_offset_kernel(pmd, addr); pte_t zero_pte; - zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_zero_page)), PAGE_KERNEL); + zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_early_shadow_page)), + PAGE_KERNEL); zero_pte = pte_wrprotect(zero_pte); while (addr + PAGE_SIZE <= end) { @@ -112,7 +113,8 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr, next = pmd_addr_end(addr, end); if (IS_ALIGNED(addr, PMD_SIZE) && end - addr >= PMD_SIZE) { - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -145,9 +147,11 @@ static int __ref zero_pud_populate(p4d_t *p4d, unsigned long addr, if (IS_ALIGNED(addr, PUD_SIZE) && end - addr >= PUD_SIZE) { pmd_t *pmd; - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -181,12 +185,14 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, pud_t *pud; pmd_t *pmd; - p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); + p4d_populate(&init_mm, p4d, + lm_alias(kasan_early_shadow_pud)); pud = pud_offset(p4d, addr); - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); pmd_populate_kernel(&init_mm, pmd, - lm_alias(kasan_zero_pte)); + lm_alias(kasan_early_shadow_pte)); continue; } @@ -209,13 +215,13 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, } /** - * kasan_populate_zero_shadow - populate shadow memory region with - * kasan_zero_page + * kasan_populate_early_shadow - populate shadow memory region with + * kasan_early_shadow_page * @shadow_start - start of the memory range to populate * @shadow_end - end of the memory range to populate */ -int __ref kasan_populate_zero_shadow(const void *shadow_start, - const void *shadow_end) +int __ref kasan_populate_early_shadow(const void *shadow_start, + const void *shadow_end) { unsigned long addr = (unsigned long)shadow_start; unsigned long end = (unsigned long)shadow_end; @@ -231,7 +237,7 @@ int __ref kasan_populate_zero_shadow(const void *shadow_start, pmd_t *pmd; /* - * kasan_zero_pud should be populated with pmds + * kasan_early_shadow_pud should be populated with pmds * at this moment. * [pud,pmd]_populate*() below needed only for * 3,2 - level page tables where we don't have @@ -241,21 +247,25 @@ int __ref kasan_populate_zero_shadow(const void *shadow_start, * The ifndef is required to avoid build breakage. * * With 5level-fixup.h, pgd_populate() is not nop and - * we reference kasan_zero_p4d. It's not defined + * we reference kasan_early_shadow_p4d. It's not defined * unless 5-level paging enabled. * * The ifndef can be dropped once all KASAN-enabled * architectures will switch to pgtable-nop4d.h. */ #ifndef __ARCH_HAS_5LEVEL_HACK - pgd_populate(&init_mm, pgd, lm_alias(kasan_zero_p4d)); + pgd_populate(&init_mm, pgd, + lm_alias(kasan_early_shadow_p4d)); #endif p4d = p4d_offset(pgd, addr); - p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); + p4d_populate(&init_mm, p4d, + lm_alias(kasan_early_shadow_pud)); pud = pud_offset(p4d, addr); - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -350,7 +360,7 @@ static void kasan_remove_pte_table(pte_t *pte, unsigned long addr, if (!pte_present(*pte)) continue; - if (WARN_ON(!kasan_zero_page_entry(*pte))) + if (WARN_ON(!kasan_early_shadow_page_entry(*pte))) continue; pte_clear(&init_mm, addr, pte); } @@ -480,7 +490,7 @@ int kasan_add_zero_shadow(void *start, unsigned long size) WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE))) return -EINVAL; - ret = kasan_populate_zero_shadow(shadow_start, shadow_end); + ret = kasan_populate_early_shadow(shadow_start, shadow_end); if (ret) kasan_remove_zero_shadow(shadow_start, size >> KASAN_SHADOW_SCALE_SHIFT); From patchwork Tue Nov 27 16:55:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9150914E2 for ; Tue, 27 Nov 2018 16:56:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EACA2C4BB for ; Tue, 27 Nov 2018 16:56:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C6132C4BD; Tue, 27 Nov 2018 16:56:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 03B4E2C4C7 for ; Tue, 27 Nov 2018 16:56:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E43E66B4944; Tue, 27 Nov 2018 11:56:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D4F0C6B4945; Tue, 27 Nov 2018 11:56:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCC2D6B4946; Tue, 27 Nov 2018 11:56:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id 56BE36B4945 for ; Tue, 27 Nov 2018 11:56:05 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id b186so16862908wmc.8 for ; Tue, 27 Nov 2018 08:56:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fztBcLEiYbKA/LtOsORIwty9Qmu1JSfLSmyXKXMabPw=; b=gkuAcCG78BanHBt7NHcGHFecZFKczR9z0TEcvdRRjMxLtQgBrCjHPB8FhiQEoLIPwd lo+NWTDkEcfjkuWthijQazWLlW0vAwTfN6SYbDAr/cbZZwXnOWYvJydAO8rQICBoF/05 BBI6542JMO0l+472DKS+XzU0tSRlzfANjlQCoHCY6MqHbjD7scGSmqjIIxrCi7MK0JLd mriAmtJ6VlDe6TN7M3yR5xgFK3s74qCexcNXa0+Cau+LJEnBRI4gNFpWffhxvosrK7Yb WNhL4/uFhEmA5dtv0n/A4nmNusg6sMDQjyhVPh2KVOMSh/e8lJ813D9oymt+jIimn2wc vRUw== X-Gm-Message-State: AA+aEWa0cribLkzlyLYssGC06l03DdJn8K+qXsfG/JufO/rawyGnZPkd QKBLqcjuzfSCE906Frm6CUtvbL7rC1nslHXHk1auyL3F9Jy/t2D8oApt2mIQkVhhCBkGC9r51Va wGXxbwcSHJ7VHVFrGBpU61d7azRwwn3K83XBNNugtwQkDauKZh73+qLUcUWXuyFEWAyAc00hs7H CumYc15OkuEHoyIEf9frirku1RwLFwtGp1xOmDougThRzRQDS2Tq5FhwfVbyrBnbNn+kkunDNF0 14jlfQS8tiDzS5VtVIEmsdwLkjkNVQ39nyNLxYX0JlQi83LmdSD3YTZdyaj4ux6bU9RHQYQjYjE lnK7OADPACIPVwmZC/ZzIbojiXx9ZwOFMVVjMfoX0jxXTwM0wCs/VYo0xArl80ylORFJOBjbPNL w X-Received: by 2002:a7b:c255:: with SMTP id b21mr14610128wmj.6.1543337764786; Tue, 27 Nov 2018 08:56:04 -0800 (PST) X-Received: by 2002:a7b:c255:: with SMTP id b21mr14610063wmj.6.1543337763828; Tue, 27 Nov 2018 08:56:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337763; cv=none; d=google.com; s=arc-20160816; b=m7Adfo6CzbbeoLr4hErOIOHW08i+Bx8juKVsKTkyluL2iUFNs9KrNDq6wFwXYMciPo d7FCHOuRCnwkofjfxSkMEwgeDJMBPV538sth/R+DObZLgwfURRProaeQEhQoMk5wBBrX 1bboLs5Mv8lmVZkciBgpL7WXZG4oMWACcgLR9zK5fWM53DkCDfkv5rbeNKKRH7Tj6SVt wv5G9ntkJb091DKHXjAziGpGIZSsYZcdcAoBcaL5oAEwtIFQivdfhODwcCUrFwDiCdfD aOU7AsBZ3r0aB6sFZ2n8ZjHE6fbrpCBBPuR4+jwiUz2OBVjdshdidl3jrYt24KKTyrXl +1WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=fztBcLEiYbKA/LtOsORIwty9Qmu1JSfLSmyXKXMabPw=; b=hsRHgEt/ItlHG01u7tO8lExneSEIsWjQ83f0SE55g5Ad3LW/jo7D9IFBAGZ6WGJ4h4 qdDounKZ3zqT/LHYoFn2qV9qLvirrVAN1lY3N+sec3Nxj/CqWUo4oxa0WIO934GNBCOW c81VMm3ru5nQ6pT3ehm7f92vfL2aZD3zrLaPGa5d5OfiIPB7oTmBmMx8WZ3Kohn1Bfwt N1pdsfrFzwlXfaDPP8Iu6s8hAQVtib2ZS4dWppm2ukA2yItPyDvBsmAFn07oUaZN29Jg CzlbPdZiaZ0wymO6Tv1Ih1C9FQ0Uq8MfI0UaC9UUHzQc9lJmsxN3x1o7tsT/mE0NvbFR lWtA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VN748Muj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u1sor3060213wrr.11.2018.11.27.08.56.03 for (Google Transport Security); Tue, 27 Nov 2018 08:56:03 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VN748Muj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fztBcLEiYbKA/LtOsORIwty9Qmu1JSfLSmyXKXMabPw=; b=VN748MujAw+DWU1wO0qdoor5iHxQuGuy1rSmsVaXIjPu+qp7t1gkSORsGaUi+0f2Z3 aOXiZ5FMoUJXYwqkR1KMlJUvgIzVuATN/AkXP0OnHQ2oR7g4qW0RP0AYlMOcCN9wRapF FjHdh51W3QTBDzipTDU8Het6TyNhlLctUuCuQyxPK0Jh1vMbZhjKZkHuaBUByJ/stl4J YjnUrP1lc/GUlHkFl66SX1lfog0/MdcpT7+9kAdMzPPqRJpx87kcbyQZQqG7yoybRkhP u9It3Zkk6LuI74HWWSL5K7a+7Sj044hjHidCjblgFJ0TaqnsWHAWWzCGCBNfmL6bKRWa 2Tfw== X-Google-Smtp-Source: AFSGD/XGiJkmlg1UCiFrlHGtGTEw55szXT2tYUWA/NiNoHyYu4oBhIyXGW2YjdMJaFRQhK/jKyCjyg== X-Received: by 2002:adf:f703:: with SMTP id r3mr27665915wrp.93.1543337763264; Tue, 27 Nov 2018 08:56:03 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:02 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 08/25] kasan: initialize shadow to 0xff for tag-based mode Date: Tue, 27 Nov 2018 17:55:26 +0100 Message-Id: <9004fd16d56d8772775cf671a8fa66e54ed138dd.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A tag-based KASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with tag-based KASAN we need to initialize shadow memory to 0xff. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 15 +++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 4ebc19422931..7a4a0904cac8 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -43,6 +43,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) return __pa(p); } +static phys_addr_t __init kasan_alloc_raw_page(int node) +{ + void *p = memblock_alloc_try_nid_raw(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, + node); + return __pa(p); +} + static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { @@ -92,7 +101,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, do { phys_addr_t page_phys = early ? __pa_symbol(kasan_early_shadow_page) - : kasan_alloc_zeroed_page(node); + : kasan_alloc_raw_page(node); + if (!early) + memset(__va(page_phys), KASAN_SHADOW_INIT, PAGE_SIZE); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -239,7 +250,7 @@ void __init kasan_init(void) pfn_pte(sym_to_pfn(kasan_early_shadow_page), PAGE_KERNEL_RO)); - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index ec22d548d0d7..c56af24bd3e7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -153,6 +153,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 + void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); @@ -163,4 +165,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_SW_TAGS */ + #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 5f68c93734ba..7134e75447ff 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -473,11 +473,12 @@ int kasan_module_alloc(void *addr, size_t size) ret = __vmalloc_node_range(shadow_size, 1, shadow_start, shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, + GFP_KERNEL, PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, __builtin_return_address(0)); if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); find_vm_area(addr)->flags |= VM_KASAN; kmemleak_ignore(ret); return 0; From patchwork Tue Nov 27 16:55:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88B031869 for ; Tue, 27 Nov 2018 16:56:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 770822C425 for ; Tue, 27 Nov 2018 16:56:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A9632C4D5; Tue, 27 Nov 2018 16:56:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D6A592C4BD for ; Tue, 27 Nov 2018 16:56:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7D1F6B4945; Tue, 27 Nov 2018 11:56:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B0A3B6B4947; Tue, 27 Nov 2018 11:56:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 980EE6B4948; Tue, 27 Nov 2018 11:56:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 3C0DB6B4945 for ; Tue, 27 Nov 2018 11:56:07 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id t62-v6so16769760wmg.6 for ; Tue, 27 Nov 2018 08:56:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=6e1l9Y/ECE5gJLR0cHfFtNGEhOKinaqvIQUjwSbnIxA=; b=ug5qQ4MvRDO/7Tyf2LvwJLtboDj9JQB6ZaBruumJyzfaDA+/CkaMIjTRs/PTaqofhM BispnqPELk+RppT/R6tVqknzn1tKOl7/3XuXzc0YtNekPxvH+ZP3mxDh1IFYeXJAFtms qir+W2Mmu1y6ZKOXgF+FhB8Yx+UTtzrmRiqmCw+HLz6XSaTCREZHOsJxxexqENdpKKuG 5s5JjlnWkNbDkGXgCTPNDnQVpWFWdpjSb6EBOQGPIg9A85McmhlMyrSDeQLyInLFlmo3 dWeydNSwQR3Lf6odO/YfNNxN4rajoBwJJzz5ALE3S4tWZd7V0mOITR3nTtI7QcdHqfyD LzoA== X-Gm-Message-State: AA+aEWY1pbhC5dTi7/ZW9VPcUEaoojGaWs80iqEvgyttJC9enT/9bzVv 2fMVypZFY8uYrG6e3n6IfzVdYAKNc+yRzMWdPcEc1un7HmI7TL8oJzkeGV3cTm+YrsHXzXqvkct z3eRaZtGfN+5ZlqlNL+ouQySYFMo8R1/zMC8YfsJdLNsiTqSsBfcWtBPhjz0RtYykoV88yhKZGj /c1MABvZ5a8hZEi9tHuhS3y6s9AEipr8U7FHeymGM7vZ72ySs6s2Kq92vIsvgHvrvL5pICAsx6U TKXkj2X1Eo4xfmyR7J0ZfdOYx5Gdr9snvH4Xm8EY5xfy53HO7CmhcvaiFdiQYug+/AvXdWFWKEM eyPkClqX2Os8ZtFEIzXFHdswGCyWqYJzUs5XJAJziDy8xmnB2poxF8YorCdDb/KS5NalmI1SBnn U X-Received: by 2002:adf:cc91:: with SMTP id p17mr28949954wrj.118.1543337766803; Tue, 27 Nov 2018 08:56:06 -0800 (PST) X-Received: by 2002:adf:cc91:: with SMTP id p17mr28949901wrj.118.1543337765933; Tue, 27 Nov 2018 08:56:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337765; cv=none; d=google.com; s=arc-20160816; b=gnUq5FRj0sEiCHALqBnUf1d+HOdJnn4RwVn3flTkq0wn+IwfNLYPG0WZq7OfX6Y5pR d6AknbOaxT+pDppUvcv9u+2KmqsrT1v/54U63agYRQMb5TffmcahPH0tsiWI9AdY2cPG So3m/UT+sDLsZBLSuQtf1ATtjT8201RqPUQvceKKqsMdblqKt9LcrUE928SN7yk1sPDD zI8CkjKlRDGdxUrU1rPsdGUle/ZCpbtxTkkphP+X9FNidK0pq3YvFGGOtXtHy9wzEH7a fX2UVaAwM3oDupL6CwgJyuV6EZlwOVHJmL9+hZAU4xxYxSUTZyCv2GH87wA3eNXhHk+W bAAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=6e1l9Y/ECE5gJLR0cHfFtNGEhOKinaqvIQUjwSbnIxA=; b=qQEybTlu2/dTaWDsOHJmd4aYOuotUlo1NzgvvmfgbPQCwvFrF7ZkUskrw1YkDuKyRf RxMBYvRSjnwRm+mFAPO318LNpb60KvHwARXkJ5Xt9YerD6wyf9ZpGi/3h5+amxYwBIAR AtheiDoBWZDqcnLlltEjpK+J9Ouda2QTTE2FiZtCcV+zsFHmJsrjuGlLfueVrOxxvqqg gwNmQIh3BEd6NGkrXdaxYa0i7mU/pYk7iW0Q4vwEGHS7SQHrXRZBGa5AnXrkrA8ixt9W Yl2FjHVph/QVViSBEV/r+ptAKYe0EqUjaXqbod6PETGlY9Bb7PSMUMBroViVSLaaSSGR VONg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=c7RLbeCB; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g18sor3093315wrw.3.2018.11.27.08.56.05 for (Google Transport Security); Tue, 27 Nov 2018 08:56:05 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=c7RLbeCB; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6e1l9Y/ECE5gJLR0cHfFtNGEhOKinaqvIQUjwSbnIxA=; b=c7RLbeCBSVZ4Kt6XmRk41o0eRiADDbVObnmQOr8RuKkIYie1CCJ4fYq8kfj9AfajBv OBvv316qqeqXWjU+4grgAXMDTb6uhS5R6mR7OXy5YlGkW6IH0gS0+aVi0WFb16P7E03z C9Mq+XCW4ZYUNS6UaPDcu9pBuiFLIKyA94mcuUhjZjhUXihex0N5TWIkfeVY82bcQLdr HnqZUKZa05tov8/OoSNckF7DsE/hTRTiN3nnqkdkfHNTjS9r2E1LQsQBFGxVT7sHi56N GABTW4X86jJYnMHRjmDPORCZC1aM5Kpaj34BLZGfRVchAXfgDVsAjf47zSHA1CHobvdE vn/Q== X-Google-Smtp-Source: AFSGD/UuH/6DY8P2w9UpFgpPIClD7KG7B4718Xsjq1f+D3WUkqLpCJuTKqnCRQRqaKolqFu71jv+oA== X-Received: by 2002:adf:9d4c:: with SMTP id o12mr2491101wre.94.1543337765359; Tue, 27 Nov 2018 08:56:05 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:04 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 09/25] arm64: move untagged_addr macro from uaccess.h to memory.h Date: Tue, 27 Nov 2018 17:55:27 +0100 Message-Id: <432ef6686a25b49244f54c4dfd86bc4b20381d8a.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Move the untagged_addr() macro from arch/arm64/include/asm/uaccess.h to arch/arm64/include/asm/memory.h to be later reused by KASAN. Also make the untagged_addr() macro accept all kinds of address types (void *, unsigned long, etc.). This allows not to specify type casts in each place where the macro is used. This is done by using __typeof__. Acked-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 8 ++++++++ arch/arm64/include/asm/uaccess.h | 7 ------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 05fbc7ffcd31..e2c9857157f2 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -211,6 +211,14 @@ static inline unsigned long kaslr_offset(void) */ #define PHYS_PFN_OFFSET (PHYS_OFFSET >> PAGE_SHIFT) +/* + * When dealing with data aborts, watchpoints, or instruction traps we may end + * up with a tagged userland pointer. Clear the tag to get a sane pointer to + * pass on to access_ok(), for instance. + */ +#define untagged_addr(addr) \ + ((__typeof__(addr))sign_extend64((u64)(addr), 55)) + /* * Physical vs virtual RAM address space conversion. These are * private definitions which should NOT be used outside memory.h diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 07c34087bd5e..281a1e47263d 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -96,13 +96,6 @@ static inline unsigned long __range_ok(const void __user *addr, unsigned long si return ret; } -/* - * When dealing with data aborts, watchpoints, or instruction traps we may end - * up with a tagged userland pointer. Clear the tag to get a sane pointer to - * pass on to access_ok(), for instance. - */ -#define untagged_addr(addr) sign_extend64(addr, 55) - #define access_ok(type, addr, size) __range_ok(addr, size) #define user_addr_max get_fs From patchwork Tue Nov 27 16:55:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700939 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38DBF109C for ; Tue, 27 Nov 2018 16:56:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25BF22C49E for ; Tue, 27 Nov 2018 16:56:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 236CD2C4BF; Tue, 27 Nov 2018 16:56:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B18F2C49E for ; Tue, 27 Nov 2018 16:56:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E14866B4947; Tue, 27 Nov 2018 11:56:09 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D9B7A6B4949; Tue, 27 Nov 2018 11:56:09 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEEE96B494A; Tue, 27 Nov 2018 11:56:09 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 54D4E6B4947 for ; Tue, 27 Nov 2018 11:56:09 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id t1so17389453wmt.5 for ; Tue, 27 Nov 2018 08:56:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4oAU6JK5SXFQXeBM1Fe+Zgca3Nk1YiOq990k4sZRE+4=; b=tK9lnAimfiQSby6TGAKIhURbZlbRvfoI7LMFLJr4G6B5YXU3zqd/bM0amOhOd2d4pv ZBeWmGEo4HMwDOvIl6KOaqkEge24oqzD7WR0NbRoCdcab8PW9WpPZy9NJK3IapnXP0ai +tuj3gd8MtCCzSxRxYLoVqQsB4iNthdH+H8lOhDmbsIRPeKDHChDCTPmbhXqljuNKCnY sshHCX/E0nSKS5mFjpBxQXl6j/PrfMGCC8eMA7Q5GKI7xVQrktXiRcCTkgqOQCP+eF+V 7fiUkNAg/Zr/2JNU5OvgAVcR8eP6dPu8mXuKpKFmb9dVIaR8Ev/JdN04+MYd0YPcTLq3 1HZg== X-Gm-Message-State: AGRZ1gJUzbKSMyOMvMIoUviOmppyhddVDXF6qyESKmNzzMpP58BD2jiC 2an0GyXualsWfbKo69941QE48hug2k58K3l6QffqLOUuYpSjxTD3+JL4nOAas29D9xPk4hy5ZJ+ afkqtrCweAGR/V3FKgbR57s58etJ3Z3qIOxQ8dYkK8QOqyypLYej93M8ZiHQkyiT7CGA6lcFDb/ S08cszR66iz8ZO84YOvTwUQZ3JiAvEtixPw+j3YZ9BLVMuXuMaAuZXON2Rq8vF2mjrti0UZEJ6Z 46KVOfamPF3E3HSbsYgjzW97P3VM9tdtF/o7BfNeX8ramnY3+swrL25YX8ArUaFjf+9DA+4AIkR Vnwy7wwNpYO1ZbPKYMioQx9PLKfTE2BFMTn6mIKe/Qh3fQBphC3PF2LC6SxF/MspsysiFfOD7PS t X-Received: by 2002:a1c:7911:: with SMTP id l17mr26645775wme.57.1543337768826; Tue, 27 Nov 2018 08:56:08 -0800 (PST) X-Received: by 2002:a1c:7911:: with SMTP id l17mr26645722wme.57.1543337767685; Tue, 27 Nov 2018 08:56:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337767; cv=none; d=google.com; s=arc-20160816; b=An+6oVWYNpRxWvKgl3qOxBytUkocun7sbTmiXsrIfgrswCz3VtXBf2GotesjJGmlaV 1PvCg5ezySfW4Hnk9h7AKppnR5s1goqgAwvU0Cjn9xFFwcDYKxBFbhBVO8spW4ccKVX2 DFyxC2H1a52T/7cEQ7ZqtbLtcjufgniW9buhCqd0y+kF4J/30BXFTxUv9AUXLNPkueOm p0HQdf0u38CZnaHC/Yv5h7GEnKWRvdx+Gdly1n5hRXylT0CCDHQjARj2CF8l8hpvflu1 xf+p513e2A3D2sN9Ih/K1FP1JuN7yxv5TxF4fbw8I5XrTaqpnzGHdk151dR9ApeRLYKS cu6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=4oAU6JK5SXFQXeBM1Fe+Zgca3Nk1YiOq990k4sZRE+4=; b=u+q6xRS5fhX8XufT9OuRK5UvOgrC31xmFiMCAApRyQUkWehao9x1CTC+WrFmi4rsFh dW/5eCort+hE8sFZ5DTASuUr0mR37xuXQurSjhAa7k0D8VDbrfc6foOUheLXogufsBMv qiXi4BRS3MdQoym+WMX/Y/fv55LVgo1kcD80c6FD70aCo4CUtU3Ams6Oi1tVE2AH+Rnn nvvealdyyTpUZuk44oS/WOwuXg5M0g9Cb7/Id37kha9IxfSqhFqNYrEb0DJQ/gTMpevI /3kSqXhtaU2MgnmcV6DePtI8/YwnuoxX5ipskf6vYKmtgA9RftQaGQeLVWQ1i5U0Ib7Z Pd2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=chHHj6Xi; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id k1sor3064914wrx.19.2018.11.27.08.56.07 for (Google Transport Security); Tue, 27 Nov 2018 08:56:07 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=chHHj6Xi; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4oAU6JK5SXFQXeBM1Fe+Zgca3Nk1YiOq990k4sZRE+4=; b=chHHj6XiLJ1QKKEppmZn3nW3M1nj5ouQtd9MSTUznHKjvgqVF/v4xI9Y1YFsHlbbBq CSzpZBkiHVXiwuP2IvcEvmn2l+ACnXU70oPxn7BBA0EFig7Xp5XTEQ2lbPWBUm1TpnWl ztrMUYtCo/NZxGswmiCsj1OxlNGSTBWUB0zNlh7A2Vt+iiZz/rU8Un83hj8zK6J0e4Q5 9fYt35f8iAvFSDW5y+spN/XSXHhfS8Qn/tKymZQal4z1tzvGfnjhmOOp/TEjf97C2QnL osfR/x0T+JRZhCfPOwTfvnGCqmtng5ADUgrKmUslv7YfxPGHSUFdSoSk0KfyJUNdkUMw tjIA== X-Google-Smtp-Source: AFSGD/UiIJUobpt6jmnooRInVxYH9JCQFfF0BEjmP/f3qfSwiz7KnNbDRbiiK84/8RrZcPMtgzZFsA== X-Received: by 2002:a05:6000:1287:: with SMTP id f7mr29807646wrx.302.1543337767063; Tue, 27 Nov 2018 08:56:07 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:06 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 10/25] kasan: add tag related helper functions Date: Tue, 27 Nov 2018 17:55:28 +0100 Message-Id: <643b46fbcd6433a4be18b3a19ce9f3e727618a8d.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/kasan.h | 8 +++++-- arch/arm64/include/asm/memory.h | 12 +++++++++++ arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 13 ++++++++++++ mm/kasan/kasan.h | 31 +++++++++++++++++++++++++++ mm/kasan/tags.c | 37 +++++++++++++++++++++++++++++++++ 6 files changed, 101 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h index 8758bb008436..b52aacd2c526 100644 --- a/arch/arm64/include/asm/kasan.h +++ b/arch/arm64/include/asm/kasan.h @@ -4,12 +4,16 @@ #ifndef __ASSEMBLY__ -#ifdef CONFIG_KASAN - #include #include #include +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) + +#ifdef CONFIG_KASAN + /* * KASAN_SHADOW_START: beginning of the kernel virtual addresses. * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses, diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index e2c9857157f2..83c1366a1233 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -219,6 +219,18 @@ static inline unsigned long kaslr_offset(void) #define untagged_addr(addr) \ ((__typeof__(addr))sign_extend64((u64)(addr), 55)) +#ifdef CONFIG_KASAN_SW_TAGS +#define __tag_shifted(tag) ((u64)(tag) << 56) +#define __tag_set(addr, tag) (__typeof__(addr))( \ + ((u64)(addr) & ~__tag_shifted(0xff)) | __tag_shifted(tag)) +#define __tag_reset(addr) untagged_addr(addr) +#define __tag_get(addr) (__u8)((u64)(addr) >> 56) +#else +#define __tag_set(addr, tag) (addr) +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif + /* * Physical vs virtual RAM address space conversion. These are * private definitions which should NOT be used outside memory.h diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 7a4a0904cac8..1df536bdabcb 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -253,6 +253,8 @@ void __init kasan_init(void) memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); + kasan_init_tags(); + /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; pr_info("KernelAddressSanitizer initialized\n"); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index c56af24bd3e7..a477ce2abdc9 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -169,6 +169,19 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #define KASAN_SHADOW_INIT 0xFF +void kasan_init_tags(void); + +void *kasan_reset_tag(const void *addr); + +#else /* CONFIG_KASAN_SW_TAGS */ + +static inline void kasan_init_tags(void) { } + +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)addr; +} + #endif /* CONFIG_KASAN_SW_TAGS */ #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 19b950eaccff..b080b8d92812 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -8,6 +8,10 @@ #define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ +#define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ +#define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ + #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ @@ -126,6 +130,33 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif +#ifdef CONFIG_KASAN_SW_TAGS + +u8 random_tag(void); + +#else + +static inline u8 random_tag(void) +{ + return 0; +} + +#endif + +#ifndef arch_kasan_set_tag +#define arch_kasan_set_tag(addr, tag) ((void *)(addr)) +#endif +#ifndef arch_kasan_reset_tag +#define arch_kasan_reset_tag(addr) ((void *)(addr)) +#endif +#ifndef arch_kasan_get_tag +#define arch_kasan_get_tag(addr) 0 +#endif + +#define set_tag(addr, tag) ((void *)arch_kasan_set_tag((addr), (tag))) +#define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) +#define get_tag(addr) arch_kasan_get_tag(addr) + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 04194923c543..1c4e7ce2e6fe 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -38,6 +38,43 @@ #include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void kasan_init_tags(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(prng_state, cpu) = get_random_u32(); +} + +/* + * If a preemption happens between this_cpu_read and this_cpu_write, the only + * side effect is that we'll give a few allocated in different contexts objects + * the same tag. Since tag-based KASAN is meant to be used a probabilistic + * bug-detection debug feature, this doesn't have significant negative impact. + * + * Ideally the tags use strong randomness to prevent any attempts to predict + * them during explicit exploit attempts. But strong randomness is expensive, + * and we did an intentional trade-off to use a PRNG. This non-atomic RMW + * sequence has in fact positive effect, since interrupts that randomly skew + * PRNG at unpredictable points do only good. + */ +u8 random_tag(void) +{ + u32 state = this_cpu_read(prng_state); + + state = 1664525 * state + 1013904223; + this_cpu_write(prng_state, state); + + return (u8)(state % (KASAN_TAG_MAX + 1)); +} + +void *kasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { From patchwork Tue Nov 27 16:55:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 449CD109C for ; Tue, 27 Nov 2018 16:56:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 33E602C431 for ; Tue, 27 Nov 2018 16:56:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 319FB2C4AF; Tue, 27 Nov 2018 16:56:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96C8A2C4A8 for ; Tue, 27 Nov 2018 16:56:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D7966B4949; Tue, 27 Nov 2018 11:56:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 736A96B494B; Tue, 27 Nov 2018 11:56:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 560296B494C; Tue, 27 Nov 2018 11:56:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id F2EAE6B4949 for ; Tue, 27 Nov 2018 11:56:10 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id q18so18556640wrx.0 for ; Tue, 27 Nov 2018 08:56:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=y8jDgA1AyN3NE4eBg/lwMVayjAlQQavKYDtyjUMWkmY=; b=KafB7X51c1qtn3cZALDv5hQyGB3IStzVO5taAhqrBS+8gUzJ0bppT4W1SZE8CXGUbo eS46AA4ST2yYVJrCjLbtgKMjvxd/h0tt+f4GWvvoECluaVRjON5fKrmEPPwFKfgKTDP6 +iiNor7FsdLkEn0u+F7k0+uL4sodctgKphMqhKutyyVa+OcNzSsBOQd/iSO6slWOgj3Y 3YJEFLLxAzAN8OLjKrQiSzpgulvQfHxguXimEMOaIFBExDg+meziCXAKqNUTzhiyAlt5 cw4xp2F85VPe9IjA8hJcZoYW6UNKqyra9aO4dTyddid1jqovLISylAWUvcukDim56i6w ZFvQ== X-Gm-Message-State: AA+aEWbEGNFAxBU3ReATqlTlIs8yG6iiM1BzRI7YCvXNBxGIMAzVDRG0 2x1vJdTtcSUcbdRLPS/S/bpDTs7jJGLBs6cExNHzdHMrFqJQ1/j9u3+uJltnaxlz1BUjFF6tLWN g3wdvV2PUhYT4XKNIp1WYrKbqxOMAnaTy9anufV2743qqBtG4duOB8g7ClUHUhY5FiUjnRNxvF2 mnI9Z7eqhJ1J/f9SdKXVWBqRh2j2aTCHSi7UCQGJeh31U7DBDbYoH1wa5B13d+8XYHiTluRwugw PCQgmgYDAWySQ/N2qzY1AIhTGjlrqkxhsN/ifpn4h54DdlGwVCOAcuxeZ2jiYOk339aGTuCS2oz L/ru9Fm3dJHU/U7Tuj94WBFsyKzyJe/fOyh46cZroCfjCwDsvhQn42ESwuHk37u/UpcXPqCjXYW V X-Received: by 2002:a7b:c397:: with SMTP id s23mr1306508wmj.127.1543337770391; Tue, 27 Nov 2018 08:56:10 -0800 (PST) X-Received: by 2002:a7b:c397:: with SMTP id s23mr1306455wmj.127.1543337769534; Tue, 27 Nov 2018 08:56:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337769; cv=none; d=google.com; s=arc-20160816; b=GwiWMTxo9gKdngNE8YsBC4fBn1GvM1RBSnCV34aCwiQZs3IrvW5OwK1agphPacQi1d OE+LCLU08qia3xLd2VkwG3uXL/NUnmWyzyw1x5y/ahj34BSrGtXvy1hIIM5n4de6Qd5T JT0UjWfNtEDjlq2HRvOOWY2PNP0+0i56mDsbzr2Noc1jlVoPUFy7XAZ/L87dYefIS88m d5aZW6SLYC4XnoEihbsOaAbRS1efLifIVmtSxp+1PYD7Tgmx/JWWDzAidWeoUVda78V4 JCBfe6XcLYejpwpm+Li6xwMHLZ4uLLAf5IrTGehCxVsLrwY6ooFm8di/BFbrjVICSyOF uUYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=y8jDgA1AyN3NE4eBg/lwMVayjAlQQavKYDtyjUMWkmY=; b=QWoaD5NhlRfcjO2fHDicmdKlTsO+dxc9sNr93Ti3L0D+BUpdH9idClYChz0VR92Obu yiCLpLqkcpKHN91etWmWiwm1NmHaMERwydA7OGw88FGalCmZagQGJHQdfAE4Y225ZWXO pbPuQOo4cJnDZ+/5Rt64xEYrGsLywnSdAF3uy/FzjE71O/AOB64ec62hByRCIoVIUkHI k4qJaL7ZFUT1lR9k/E7W/rEN4/8iTSVwxZye98ELjCjP3wOP8edcsMY67S4FEdwF4z0L VF/sNRgFio/q8eMPQG2teWKoebDripDW3L/po/uO4+s/fGwOE6Hc4lGVlnZ76TWUrhpg sohQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Bxo3E4Rb; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y4sor3165331wrl.33.2018.11.27.08.56.09 for (Google Transport Security); Tue, 27 Nov 2018 08:56:09 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Bxo3E4Rb; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y8jDgA1AyN3NE4eBg/lwMVayjAlQQavKYDtyjUMWkmY=; b=Bxo3E4Rb7z32spR+GjTU3/UkKTGzAZUE/6ScwrxnZHct9+8n6DhIPAlWfNPk4yvtAq PkF5qmbAKLUkwJhskz+edVmt8U0gNIS7JXhbzegEAAnxrdyJob+N+sL6TAIVBVcjvmD4 PwOc2fCWgzsAQX/lmgBsIe2l2PS+aXGvAZR8ehnhsFLiJngL0VkrxwD6JHkJ2PO4YGx3 XB9Nzoo5zvl1XhXpRJOFc0QeTA1hfU/SeQYwjx5psTSN9ZZuB3a0BO0gCZ+p/qF4b0oG tG+bFW16AGvNk5yTKTIM2JfWNcFnKaii4O5xUCK1MdbaQ2VOisQTBmWTaf8vD/SjaVeQ oypw== X-Google-Smtp-Source: AFSGD/UriVYn0glnQslEJ+Tb2vBU5XdWPYCPaYmqBE94vU0Fqn7cyDlFXFag3HaZdSEdMQAwYsLMhw== X-Received: by 2002:adf:c711:: with SMTP id k17mr15658629wrg.197.1543337768904; Tue, 27 Nov 2018 08:56:08 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:07 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 11/25] kasan, arm64: untag address in _virt_addr_is_linear Date: Tue, 27 Nov 2018 17:55:29 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP virt_addr_is_linear (which is used by virt_addr_valid) assumes that the top byte of the address is 0xff, which isn't always the case with tag-based KASAN. This patch resets the tag in this macro. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 83c1366a1233..5fe2353f111b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -322,9 +322,10 @@ static inline void *phys_to_virt(phys_addr_t x) #endif #endif -#define _virt_addr_is_linear(kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) -#define virt_addr_valid(kaddr) (_virt_addr_is_linear(kaddr) && \ - _virt_addr_valid(kaddr)) +#define _virt_addr_is_linear(kaddr) \ + (__tag_reset((u64)(kaddr)) >= PAGE_OFFSET) +#define virt_addr_valid(kaddr) \ + (_virt_addr_is_linear(kaddr) && _virt_addr_valid(kaddr)) #include From patchwork Tue Nov 27 16:55:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700945 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2653A14E2 for ; Tue, 27 Nov 2018 16:56:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11C182C4B0 for ; Tue, 27 Nov 2018 16:56:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 055E72C4D3; Tue, 27 Nov 2018 16:56:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D73C2C4C4 for ; Tue, 27 Nov 2018 16:56:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 516976B494B; Tue, 27 Nov 2018 11:56:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 474F16B494D; Tue, 27 Nov 2018 11:56:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EF3C6B494E; Tue, 27 Nov 2018 11:56:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id C790C6B494B for ; Tue, 27 Nov 2018 11:56:12 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id 132so16889996wms.3 for ; Tue, 27 Nov 2018 08:56:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4s2WHY6CF+7V+5ETRMCx8aRRLenfz3ucdYpRRnO2KsM=; b=rBnqLbCwAG7wdE5105kYIHfIrZW2W1PG+eBPTl/WaSd9VUkpiX4OETmD7qu54YlYwa w8LfbboA7xArHu4uahvx9a8aPiZdRbRHVvYYhsVR1aOqjAipLGAT3tztkNyCWbxYVePR MIFV0l0SAn3PYTxiyhMzt0tomctF+i7z2p8oMOCZuaycYJA2M1fVRqq7CQxi1QhDTCJd jSw2zdDadD0te2s0RawMyNYhGH0o7jLLKT+xPvBU3H2PW9c5pyRwM5cPX4wBM7qF2A74 ry3xuCErpVeEN1UfzRM2L/kk6LgssfZL77mLSEMYMGDoMwa0YBXNKVsDQia89j1GMRqs Jw5A== X-Gm-Message-State: AA+aEWZvLqf/wHJAyRDBHzh2Pk5QZf912RWvIj8A6virLDs4qYaBNnUR sjCVMS2ro3mfpGFmLiWuUtc1NCeABieKEgecrcAyHSIjOykzAkk2mhPnXVwf/lA6WdH3Tn3imcY MybrNXXsNEyB/ppSIAGroSwa0TzzZg//ByIVd2qOa0k+509439hvbChn9KW/qIXnhpdzZoCgEoV /NH65UMPVbdwYDSMPJphEmSTGXKl+uk2R8hxGzE4vnnZCa1WbZwr6+1enXsG8resCbQsmEEet5z S/czjD9JsMVo45NaFIG0bkzeGQqIaMhsrDjja1wbe1WaANJVHB56MsR+uQ+ZW8/xVA+ilTGgA1I /zHrwHwiWjjGZSHK3CVtNAtWvRzfg6dRY2pmX/o9NHMLMA/HC/hkMWT7xdFrzL0QvqdAwmFdAEL h X-Received: by 2002:adf:ebd0:: with SMTP id v16mr22661751wrn.109.1543337772325; Tue, 27 Nov 2018 08:56:12 -0800 (PST) X-Received: by 2002:adf:ebd0:: with SMTP id v16mr22661694wrn.109.1543337771369; Tue, 27 Nov 2018 08:56:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337771; cv=none; d=google.com; s=arc-20160816; b=Oc9sYiquvujn1F89JXgo1DxQD/qfo46zBbTvqNL2iBOByxutcqDHA3Xl5HpPRsvyV1 5DxI2NJMqYCVOp2IwybnP1JdnYIWt2tinAbv03zbJVPuyUOes7yJYa+rlINxoTqvcJZY V7GRaXQD0wTApVYQ6JppuwHCp6LZLbiQL6S/H3SvUs8+kcJxheUkpKOXgmeuZx/EPv3o 64sxoR1iqPoc1eNwaVjynyqjGA8byXPDhyoJjJ6xuJJs32cdKoqLYE5Hbz24DlDGg3JE u/k65tEUDE9JUClKg49OWKav5CQclfMSx+lT36cRBVkaXtzpPiW0vSLHdEXTV4Lnmo0x hAHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=4s2WHY6CF+7V+5ETRMCx8aRRLenfz3ucdYpRRnO2KsM=; b=BSw6tx3SMXCONeGBMVZJg1c5Ey3UXfPtRDr5gfRteaUyFV9gr2Xhs5Y/8KRh9so0yM tK0B72Jg07EAcdQ5bdAMI4MvNuD0kJWDOLpaO4+9OpgOJXdRkxlDRu8f2oIAPy1hF2mm ubMobDtywd2cxKgx3+EHuuxKGnaaXLmx377yLZfjZG1qTXgFNqNgNeieSaWbXzwFISKL wiRbqmmVOjPBYA+7Tktv2d4hjzVLibTA8O/Bmi6H8ay8LqxfV58GPN3GMA+Jkg/kJF/f XR0kKbRLOnWtjGooXzTHLNFmEaCGRxhUoCldJKUTgGCBOJZd++MxPFi/GyNBNMFgBz9E cG6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ie7jabn+; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id m65sor3338436wmb.22.2018.11.27.08.56.11 for (Google Transport Security); Tue, 27 Nov 2018 08:56:11 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=ie7jabn+; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4s2WHY6CF+7V+5ETRMCx8aRRLenfz3ucdYpRRnO2KsM=; b=ie7jabn+eKGrSyU6M1ZrcJxSWuT6cq77fF2R9lUfLQIi8Gr2qJk90uP4rQdAwYc3i/ BzkjjICxP9uNZ2wir+A+L1ko8NYWBwSA37ftS+H+Bmn3ZEcr0DAJupJUdUY++MRLSAdl GOVXtJSg0ysWC871IXVicxLiL+ZxjqXjJddmGFKHXYIAtsZ7/5tihtIoxdg6u6JDgmWg FWO4vo+zhbRqj0ebJBVKVL16OatDIs8Xy4oot9zqeDWTgMoQmiHv5WqJrpGree3Dobsg ZoTJ2eZeqDWXjB9kJf7uYraVLaqZSZs9L3pdZCmNeVa6ZGWdIaxsoEQbVKRvWxEIoCoN EziA== X-Google-Smtp-Source: AJdET5conMtQ/SqFrfQKf7PVc7uFAQ/al26Ds5DRdODjlyFRzE6MxacgkfMsqlhDuH9BpluipOVEOw== X-Received: by 2002:a1c:2304:: with SMTP id j4mr27714121wmj.14.1543337770742; Tue, 27 Nov 2018 08:56:10 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:09 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 12/25] kasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU Date: Tue, 27 Nov 2018 17:55:30 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor. The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it. Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/slab.c | 2 +- mm/slub.c | 24 ++++++++++++++---------- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index 26f60a22e5e0..27859fb39889 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2574,7 +2574,7 @@ static void cache_init_objs(struct kmem_cache *cachep, for (i = 0; i < cachep->num; i++) { objp = index_to_obj(cachep, page, i); - kasan_init_slab_obj(cachep, objp); + objp = kasan_init_slab_obj(cachep, objp); /* constructor could break poison info */ if (DEBUG == 0 && cachep->ctor) { diff --git a/mm/slub.c b/mm/slub.c index e739d46600b9..08740c3f3745 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1451,16 +1451,17 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, #endif } -static void setup_object(struct kmem_cache *s, struct page *page, +static void *setup_object(struct kmem_cache *s, struct page *page, void *object) { setup_object_debug(s, page, object); - kasan_init_slab_obj(s, object); + object = kasan_init_slab_obj(s, object); if (unlikely(s->ctor)) { kasan_unpoison_object_data(s, object); s->ctor(object); kasan_poison_object_data(s, object); } + return object; } /* @@ -1568,16 +1569,16 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page) /* First entry is used as the base of the freelist */ cur = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + cur = setup_object(s, page, cur); page->freelist = cur; for (idx = 1; idx < page->objects; idx++) { - setup_object(s, page, cur); next = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + next = setup_object(s, page, next); set_freepointer(s, cur, next); cur = next; } - setup_object(s, page, cur); set_freepointer(s, cur, NULL); return true; @@ -1599,7 +1600,7 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) struct page *page; struct kmem_cache_order_objects oo = s->oo; gfp_t alloc_gfp; - void *start, *p; + void *start, *p, *next; int idx, order; bool shuffle; @@ -1651,13 +1652,16 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) if (!shuffle) { for_each_object_idx(p, idx, s, start, page->objects) { - setup_object(s, page, p); - if (likely(idx < page->objects)) - set_freepointer(s, p, p + s->size); - else + if (likely(idx < page->objects)) { + next = p + s->size; + next = setup_object(s, page, next); + set_freepointer(s, p, next); + } else set_freepointer(s, p, NULL); } - page->freelist = fixup_red_left(s, start); + start = fixup_red_left(s, start); + start = setup_object(s, page, start); + page->freelist = start; } page->inuse = page->objects; From patchwork Tue Nov 27 16:55:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB2D7109C for ; Tue, 27 Nov 2018 16:56:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9AEDE2C431 for ; Tue, 27 Nov 2018 16:56:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8E5072C4D7; Tue, 27 Nov 2018 16:56:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 019FA2C431 for ; Tue, 27 Nov 2018 16:56:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F3D76B494D; Tue, 27 Nov 2018 11:56:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 27C5A6B494F; Tue, 27 Nov 2018 11:56:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1200F6B4950; Tue, 27 Nov 2018 11:56:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 99D2D6B494D for ; Tue, 27 Nov 2018 11:56:14 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id o76so16748003wmg.4 for ; Tue, 27 Nov 2018 08:56:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DWpBc9Aeyl2K6ftokSjq3RmCgQaem7DwUgJr7g7CaTw=; b=PP30jYUUdigvKIJL7767RPkZHNJASb6JZ6QoowSiEJu2Du0tIvd31N8TuSL6FoyUOn N6bLf8pOWRTI9LHAdpqtR+AuXqF8fq6UYuCLT5kZGq5fkjf3RKpj8krBhROnnteUiR5u drlohe/1zDW3XvDvgOeuwU+KXbP/J0lf1Jj2DakFHHP8eF2C7LzU9MpGFhSZ+VQuZF70 3790f7tJvrapXUM2ej5ortdVo7eZePJz/47gqRA/tFBqq1fH0FFwWvKwn90gRSqvoSo/ pR5OiI6DYXK3wzTaJLeVI99OL0wLQziN+j/jpekcxIezXLoItXfB7JmiVHJFcR0rDsN9 nsEQ== X-Gm-Message-State: AGRZ1gIr9hcpmC3fVPMMSmsjCJ/UJbRBHVEWRlpCYtiFjyVSRUf2U2M1 DXw4R4jwc25VxTKn3wsYxxAqABTCwaqPVYhqeaGCTVIv4RV13sVf0Viz5neNGYfrFHuRGNQ8c1q Cscq7SzD8TklWSB4QKYxxcBSOg2KhQbIAu6I90dLbRX9MgGSXQVliQrSHttQnaKhgP5dUBkuAQS Veji0vK/fzCodWObP2qA/2kRwCNQKR0jN30MrC/MGPzAKcd1kogajY4rEqKc7OFl0Y2WWXawg1B WqmMNaLhlyxDSeSmsSWEeNDiBPAbflILjgSxP6SjRtSuEN9Ohi10OSDr/Rl7ArFRdaa+leRptIA /ZRNnUTh5K6bZI5qKVyoM6x8hyjlmQttt6pPiBW10Ylc5VBaouUOhBN6rk3VobwhSnFxfZh0+lQ u X-Received: by 2002:a1c:48d:: with SMTP id 135mr27487460wme.1.1543337774085; Tue, 27 Nov 2018 08:56:14 -0800 (PST) X-Received: by 2002:a1c:48d:: with SMTP id 135mr27487406wme.1.1543337773127; Tue, 27 Nov 2018 08:56:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337773; cv=none; d=google.com; s=arc-20160816; b=Q66mf/RYw19ua8MfItOrks54fgOxsbi0cStAMjbaegINZ84E59qytEV+YC9+EX5jqi fYgxWrKS4mJlfnu8b1NRWIDYqsLvW3GcMc6Fp0zwPaE/QqqoXsoK3V4yXbRZXTWwW7dK ABoWXJaUOGS/s4D0K9/coaJCtnYx3JqBfEwhegadkESSsAaT4Kj2W4p/S7xJfdkGCeJ3 T5Vt8esWG7mQlAm71ergGnvqhZHVUD9VTAAPhKg4m3SH0lpAuSQguxeOuNznjxzSoOa5 EAHBwZjULC8bCeiDZ94+H3KXA+r8mit1znL+fou+KmxEojPnGmdpPqUJFKjhodz0sbvB oH5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=DWpBc9Aeyl2K6ftokSjq3RmCgQaem7DwUgJr7g7CaTw=; b=TYR7ZdkB5itigA7VbVhoCn0MWdcsScUt9v5KVoR8wgKCOCvYMmn8o0QT9Qc5NB+fm7 jq9C+dNGXKMj7thOwiDGgRkLjus8ive3Mu6yBeLTD096LGipVAfGGZKQRZyR7sMYKf6+ MMpFiDDdBuqIjiWdt/FQt6x3TpRhPoABN6dVCURARmriFRWJH20A8ubHWiOCMmdWaOj1 Jd9e7027kv4JJrJcir2vw+ZN+9+QQ84mmADP1VqLLFN5ajgmJd/jJ9u8hpJFZbV1NGPO p8iJ9kQ3m2O0rcLi9Oyh5ThLYTbwS4dY8bUFycSn8bSEIzSp2Cpnf04+QBSDd0cND/8j 1FxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=CU8JL4KP; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r7sor3151229wru.16.2018.11.27.08.56.12 for (Google Transport Security); Tue, 27 Nov 2018 08:56:13 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=CU8JL4KP; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DWpBc9Aeyl2K6ftokSjq3RmCgQaem7DwUgJr7g7CaTw=; b=CU8JL4KPkewTdTLRjln1dlwqsvsiyrscJZOuR7rFVNXaIZbugslXoYSzN6maTJ3REU WnFZV9E5LNn91F/uCnW2q9c7gLjwsvrb6QytrWNhqn3j9K5G0nZhbhKSSSbYf4EPL5mx EfK84ABAj2pc5LpSKnh6gopYytxoe6kGOmp6cDFbAvqmDYk6K9Q9wpPKQGFzusCXUqIZ k/nrPeE+U6O8pixdoT7CMphwn6LrHgyLZxD50OG/n0MhDI9y15pYB+1ASa4z5G8dDM36 J4qlbijrEz7EyL8E8n9G6w1zONJEmEJlxwfqNZYYLFzetj/LJptLT/0D0492p53sRBbD P74w== X-Google-Smtp-Source: AFSGD/Xe9cER+usCYMTC3LCrcB5TdOdSebMRHg8Xr7Z5fxtuscp04b5ztXp1ydkB/YLYsdYJs87WnA== X-Received: by 2002:a5d:4acb:: with SMTP id y11mr27674359wrs.281.1543337772547; Tue, 27 Nov 2018 08:56:12 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:11 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 13/25] kasan, arm64: fix up fault handling logic Date: Tue, 27 Nov 2018 17:55:31 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Right now arm64 fault handling code removes pointer tags from addresses covered by TTBR0 in faults taken from both EL0 and EL1, but doesn't do that for pointers covered by TTBR1. This patch adds two helper functions is_ttbr0_addr() and is_ttbr1_addr(), where the latter one accounts for the fact that TTBR1 pointers might be tagged when tag-based KASAN is in use, and uses these helper functions to perform pointer checks in arch/arm64/mm/fault.c. Suggested-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/mm/fault.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 7d9571f4ae3d..c1d98f0a3086 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -132,6 +133,18 @@ static void mem_abort_decode(unsigned int esr) data_abort_decode(esr); } +static inline bool is_ttbr0_addr(unsigned long addr) +{ + /* entry assembly clears tags for TTBR0 addrs */ + return addr < TASK_SIZE; +} + +static inline bool is_ttbr1_addr(unsigned long addr) +{ + /* TTBR1 addresses may have a tag if KASAN_SW_TAGS is in use */ + return arch_kasan_reset_tag(addr) >= VA_START; +} + /* * Dump out the page tables associated with 'addr' in the currently active mm. */ @@ -141,7 +154,7 @@ void show_pte(unsigned long addr) pgd_t *pgdp; pgd_t pgd; - if (addr < TASK_SIZE) { + if (is_ttbr0_addr(addr)) { /* TTBR0 */ mm = current->active_mm; if (mm == &init_mm) { @@ -149,7 +162,7 @@ void show_pte(unsigned long addr) addr); return; } - } else if (addr >= VA_START) { + } else if (is_ttbr1_addr(addr)) { /* TTBR1 */ mm = &init_mm; } else { @@ -254,7 +267,7 @@ static inline bool is_el1_permission_fault(unsigned long addr, unsigned int esr, if (fsc_type == ESR_ELx_FSC_PERM) return true; - if (addr < TASK_SIZE && system_uses_ttbr0_pan()) + if (is_ttbr0_addr(addr) && system_uses_ttbr0_pan()) return fsc_type == ESR_ELx_FSC_FAULT && (regs->pstate & PSR_PAN_BIT); @@ -319,7 +332,7 @@ static void set_thread_esr(unsigned long address, unsigned int esr) * type", so we ignore this wrinkle and just return the translation * fault.) */ - if (current->thread.fault_address >= TASK_SIZE) { + if (!is_ttbr0_addr(current->thread.fault_address)) { switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_DABT_LOW: /* @@ -455,7 +468,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (addr < TASK_SIZE && is_el1_permission_fault(addr, esr, regs)) { + if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die_kernel_fault("access to user memory with fs=KERNEL_DS", @@ -603,7 +616,7 @@ static int __kprobes do_translation_fault(unsigned long addr, unsigned int esr, struct pt_regs *regs) { - if (addr < TASK_SIZE) + if (is_ttbr0_addr(addr)) return do_page_fault(addr, esr, regs); do_bad_area(addr, esr, regs); @@ -758,7 +771,7 @@ asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, * re-enabled IRQs. If the address is a kernel address, apply * BP hardening prior to enabling IRQs and pre-emption. */ - if (addr > TASK_SIZE) + if (!is_ttbr0_addr(addr)) arm64_apply_bp_hardening(); local_daif_restore(DAIF_PROCCTX); @@ -771,7 +784,7 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr, struct pt_regs *regs) { if (user_mode(regs)) { - if (instruction_pointer(regs) > TASK_SIZE) + if (!is_ttbr0_addr(instruction_pointer(regs))) arm64_apply_bp_hardening(); local_daif_restore(DAIF_PROCCTX); } @@ -825,7 +838,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr, if (interrupts_enabled(regs)) trace_hardirqs_off(); - if (user_mode(regs) && instruction_pointer(regs) > TASK_SIZE) + if (user_mode(regs) && !is_ttbr0_addr(instruction_pointer(regs))) arm64_apply_bp_hardening(); if (!inf->fn(addr, esr, regs)) { From patchwork Tue Nov 27 16:55:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F39871869 for ; Tue, 27 Nov 2018 16:56:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E331B2C3F0 for ; Tue, 27 Nov 2018 16:56:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D70A82C4A4; Tue, 27 Nov 2018 16:56:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 758FE2C48A for ; Tue, 27 Nov 2018 16:56:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCC276B494F; Tue, 27 Nov 2018 11:56:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B798C6B4951; Tue, 27 Nov 2018 11:56:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F5856B4952; Tue, 27 Nov 2018 11:56:16 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id 414376B494F for ; Tue, 27 Nov 2018 11:56:16 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id t199so17858300wmd.3 for ; Tue, 27 Nov 2018 08:56:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5TjQJgkFl+PVCxLsWj9vCgbjvK0haYDKUt+yI6asfBc=; b=gaZo3NOjXZdc1/jVVOp7lB1U/7i3seNCGu4jlJ6LkNDDi8jFLli5uOOhYhRznNsQTd eiXcdYlGD8m4Dwwyr3qOUiVfFY/uPrbO5+BegD5GIvBZDgmmFI0rjYk+uDEOZ3LJLUaE XLrmgghMVVpqsN60YSbgWXhpvjay2cre8zBqAG6aB3AOV+vjsF8VsSS9i2yTHkaUepF0 A4UWP76maOVvwb/XVVktVibNbkAVqcjEFfcZ3G+FCCSZXMizqTtB+1yz9ZHvX8jTwE9Z 9vvI8c+b68bvOw2jhTRloPoY3iq2EngVJd5QJvNd6q5Cy3O5julHlJXQIDa1ZuO0/Ppd 3YFA== X-Gm-Message-State: AA+aEWYulfcd1dVxXyJcoXMdqjO7mq4Z4eGECYSc+zg9DDpt9MdS/S7A qbSsw7TsFHtaDXA1CIjKxZ3tBmnpMhdRUsAXBNAdqVqfEW+dAX2paSjfJH/eDCbb56KVUUsRRSd pLnKoJhfAg7hP2C8GA5jBn7vIbhq/uIEUwjZLEBzk/tDcObNzCmJl66JSqePy5S/prFm8GT+CfI EsUkX70U+SMhFq8+vCx/qxaXtxhFzcCtsKB3CZIfJWWgaXl4oo0HxOkZ0RvO1pG0okW0z+fU1Sk 6nSU0ixY5K0SvKnbqA5WJjGDzgmIGmm+cLYDOc+eravTpHVtsZ8a0E+R71DQFem5p6lHm+OjhTy oAon51/NY9Zvoa1ldogIrJRVBAzbayRxiLvUiaOnTpI3t/yMAnm5dKMCgMD6xnX+egawzaDGlzY 6 X-Received: by 2002:adf:9467:: with SMTP id 94mr27961525wrq.305.1543337775740; Tue, 27 Nov 2018 08:56:15 -0800 (PST) X-Received: by 2002:adf:9467:: with SMTP id 94mr27961473wrq.305.1543337774947; Tue, 27 Nov 2018 08:56:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337774; cv=none; d=google.com; s=arc-20160816; b=M4mHDCRjzrUZa06Msl/hHAqTQH4Ng8iiWQaSNPb6mo/qSp6+NreOT6aDsnOMTi6El5 1wAfashLD4L7Tz6Dk+rMsrA4KTn9t+fDC1T3F+ATS3pkJMAofx+H2yOapjOhyosiNaAp YT4He5KDDn0aNLQCe3d2lNNE4EvfEKvzGwbjl1NrfepSOguV4Ob+/mc99rRyfYO9StZI id9f4Y66DCq/Olkeh9xOWr0GwjB0hPbmLlJShfipK83sjFysrajFiGws3R01Bntv/OXt eEYvQAhgzp4EDgDJVs7Ejwt8gZenRwC7FpeQWZ6TghuIxgOjRWys+inufcA3z2DjIzll cyVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=5TjQJgkFl+PVCxLsWj9vCgbjvK0haYDKUt+yI6asfBc=; b=y5wLNfl/rfFQLClHgnal07ige6E+MtwAebOuKsvL5mi3QkAAw8AKKWHyD0N5i7kiUo i9Uae/JXafJ1+V+DK+gY95cceZIVn49xEDbst4D3ypzOYxFLMeqoB83EoWUwXvcMugzt bWr+AgS2zMbvS5z80vlFg0g9O3oBsQLa8bUQFEw5LGJwU+LNoNAlnim4G9jvzxy8WL+l BgtMJODxD/TjeS/EbrM/5USGrwRZyO6yMUrSUzWugvsHJ3PnySGlaoceS196GZbSx3OA kVdrWXDMRytGGtuS0HILls8hmCdPzBg3dIiqAt60NsPChNYB48ZIdV86RFpQQWcgcaW0 gUgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Vq8xJz4L; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i18sor3125757wrw.50.2018.11.27.08.56.14 for (Google Transport Security); Tue, 27 Nov 2018 08:56:14 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Vq8xJz4L; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5TjQJgkFl+PVCxLsWj9vCgbjvK0haYDKUt+yI6asfBc=; b=Vq8xJz4L04oXAVPDvOzYuzz+wHl1QGBnQgoEk8zjm/CZt1EqPmCV4MXLs83CCDHriG nZQo2iAfvSYq4/G3CcCYjx4+keudcdowz/V+f1xuztB78kVIxdEUZPxUlxAPXdNGfB8r xuY642dJ4/Jr9SFclo8bI/fnE/NGVNXmJm4HZJ33i2h7k3awNqoEHgHv2j3ghoHSetBq N8DthHjipq15+ycjHXS4dovDb+hu7nsTOkP31YrSMZgr2yUOtGuEakHyfOu0gQJ0gQKp +mRR4yL+iYIE+weqA0Qvd988IjrsHCP9yGsIKBnEAOfNH/W7HPJ8EjTjXk2hbR05NzUV UPEQ== X-Google-Smtp-Source: AFSGD/VnXoSEtN3ApgD8MSpZdtG9X13UEEYBU7XpEpIdqGrEHViBzH7UhGky5kkFLvu375ZFYdDdzA== X-Received: by 2002:adf:c711:: with SMTP id k17mr15658939wrg.197.1543337774345; Tue, 27 Nov 2018 08:56:14 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:13 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 14/25] kasan, arm64: enable top byte ignore for the kernel Date: Tue, 27 Nov 2018 17:55:32 +0100 Message-Id: <1ed03d53ee679cba52ba7118d2acbef948d21fcc.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when tag-based KASAN is used. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 1d7d8da2ef9b..d43b870c39b3 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -291,6 +291,7 @@ #define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 2c75b0b903ae..d861f208eeb1 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_SW_TAGS +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /* @@ -445,7 +451,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /* From patchwork Tue Nov 27 16:55:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700963 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87BC714E2 for ; Tue, 27 Nov 2018 16:56:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72FFF2C455 for ; Tue, 27 Nov 2018 16:56:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6637D2C4B2; Tue, 27 Nov 2018 16:56:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E7082C49D for ; Tue, 27 Nov 2018 16:56:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 848FA6B4951; Tue, 27 Nov 2018 11:56:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7D8016B4953; Tue, 27 Nov 2018 11:56:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64CCC6B4954; Tue, 27 Nov 2018 11:56:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 097026B4951 for ; Tue, 27 Nov 2018 11:56:18 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id w12so17568684wru.20 for ; Tue, 27 Nov 2018 08:56:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cdKIYS87jEKsGaVGv0J/kyzmIxkse0BLBOe4/s6kF2g=; b=Y5xiYF5S2BDiduHVBZyz9gu3Uq7MX0UgloZNLwGbd/+tBLhQczobc1E2XXkclOVG3j 7MH108K90Q/IYZGYO1DNPYwes6nStAIg/e9qg2CSo7ekJT9BuM/PZ0WR0aoWniv6sF7u XvuSOumBrhVLzX3xbFqWsQB8l7xKICLUTErJFY6XzSlBsiDncEau0BtA2ajZ/ksEhoAt vwux5gXGxs+jZFg+L3EH4vRNurkhvgf6xIl4jwpisEbm3J4Aafx7Q47fa/R3zwAEzste UBQ7UHRV90LiD2r+q/49qvfTeWFXjY2UFpXmPzdPnJZ/I5loMIKMp97ykwr2U27UTT8f sePw== X-Gm-Message-State: AA+aEWYqGD7HP5YeOsiy6Djw5hAA5g/E79LujTytw6nUwnRP04c6rfPZ qvhNab7m2xniToEo9IXnIm+LhpU1cv51UfvLpmZNSvEv7BSszS2WILMqsJTLmKn2DrRkm/RVkfJ uXcGz1skf/oDToAqII1jhegbrw7wa/Ip1C+F9HN7stUhyq0c3siXArgBuzHLlGV67bzAaTnYnQz 7PuIm+tc/b8dWsn3b/68hGD0a/ivJN1bCN1g2MJE4m37sS9EjZG0fXPAezsEnCIEYhSOj0XWrs9 K6KVzEz/sxy6+xa97hNqtWAO0EJmloObLcE0PDkYxiHjPAOjcXBAr0HExgRb7zrACoS7Zban26a VOWNEmMPIMrkX+hI1ztE/0VF5SWnaCmrfOvo/kY6vwMy17UsExLGcD8Fxm859kupi3VO9UXRp0Z N X-Received: by 2002:a5d:418b:: with SMTP id m11mr26108288wrp.8.1543337777436; Tue, 27 Nov 2018 08:56:17 -0800 (PST) X-Received: by 2002:a5d:418b:: with SMTP id m11mr26108253wrp.8.1543337776720; Tue, 27 Nov 2018 08:56:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337776; cv=none; d=google.com; s=arc-20160816; b=kkDj1Mc5rwsSVdXkJXBGcEJ9/vXzwteUFc0fe5QiSpTjs9qqT4709cLoQ3b7nczK5F 6BMPW2vGnvrShBdptOOa40cj+oaNmL7Lpj9t0EpTHU4ieeNEpb/zyQD2Z373uMwZ0SDw FWc/ecjjdR1AiqG2H3VSD8iLcPc/on7HQTBqXM/9tKTUgH4m5IaWs/0xCLNf0NuAB33q ipMu4vlY6kdSGFXf+N2ct0OkJ4D8GQzT1IFxVuaWHdg+MEZe8NgWsg/ZYJMuuFKFwwLm SAGCYdn9JNLawHCuJ9dJNE2PT3pKe52ZzyF+45mwUyMVbBtIdR18YJFeurp5QpDS/oxt qGNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=cdKIYS87jEKsGaVGv0J/kyzmIxkse0BLBOe4/s6kF2g=; b=h2k0bND2aav7867m7JHtp5fzJ/49Px0agWxuf53JF8h1C24ZGB59P30E7lafb6/hab 4ow05jkl0UND4yp7qesfP8Y4/QbZWuzRnJeDnPAMMQf9Kr0DjohbXuN1iGCSsjdhKPQ4 RKOR0BMFLG13bbAbRjzZt5yyhGRszXsyvqrPLTGvnGcVsXYUr6eyQzmCkgA0l0uxAQYT xjhfZEdPh8grgTpuQBxr7sHFIkLcJnQNYKA5SWek9xXjOxqFy6p7wkeUOPxiGFr24+km 8v6vtg0w/i9gF5MFljXZgZpHCl8UGObsJfSFp2ZeNgV5QKHEbhw45gMpK1GUkGQJCbqG nyGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Ip5ogpaH; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t18-v6sor3058862wrm.30.2018.11.27.08.56.16 for (Google Transport Security); Tue, 27 Nov 2018 08:56:16 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Ip5ogpaH; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cdKIYS87jEKsGaVGv0J/kyzmIxkse0BLBOe4/s6kF2g=; b=Ip5ogpaHKgzTpQygvmjooBMBVAtgRQRl+qgkAzJ7PDkUUpZM+3enXRCKJyUfAgUpE2 KN4RrLzeJzOlxK/NsObxYkCP+bixblO+FW6Kw8UNmAe67lAywwzxntNPQRi2tWP/U1Nv LDacTNsQuT1QzFPoYQfPMDS7CAGKpR6YYaA9G48zPxMzb8HczWqI6KI9Jw5GL5v4+IIF bjmifMvo2LAjKTHeBt10/F+/nr5I2kCBQsXpc0Va3VG3isKuzUhCiZMtkIYhODra3bQw lH/pUxLW+O25D3hQXEqFG11Y2SK/POzwZxDFO/ZXGOESU2l6kT/8Hbc/gMs22CZbK1Gu AlPQ== X-Google-Smtp-Source: AFSGD/Va46vaty03/ntt8AicD2YvJXz8QAHtsdGhxXNOFGp6Fe24s3POR4AZg6sU6IkJKiYDLbbUeg== X-Received: by 2002:adf:fd87:: with SMTP id d7mr27062120wrr.74.1543337776184; Tue, 27 Nov 2018 08:56:16 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:15 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 15/25] kasan, mm: perform untagged pointers comparison in krealloc Date: Tue, 27 Nov 2018 17:55:33 +0100 Message-Id: <5045db8a8e249a1eda3199b952120035eacb3bd4.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. Tag-based KASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 5f3504e26d4c..5aabcbd32d82 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1534,7 +1534,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret = __do_krealloc(p, new_size, flags); - if (ret && p != ret) + if (ret && kasan_reset_tag(p) != kasan_reset_tag(ret)) kfree(p); return ret; From patchwork Tue Nov 27 16:55:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700969 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13D7014E2 for ; Tue, 27 Nov 2018 16:57:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F13DC2C492 for ; Tue, 27 Nov 2018 16:57:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E1F472C4B2; Tue, 27 Nov 2018 16:57:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A014E2C4D5 for ; Tue, 27 Nov 2018 16:57:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D01F36B4953; Tue, 27 Nov 2018 11:56:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C8ABB6B4955; Tue, 27 Nov 2018 11:56:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A6CD66B4956; Tue, 27 Nov 2018 11:56:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 2F9F16B4953 for ; Tue, 27 Nov 2018 11:56:21 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id b186so16863192wmc.8 for ; Tue, 27 Nov 2018 08:56:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=palwGaixv/Wt9FyOgrJvUwZx+w02pRpq6CF1mb0/y3U=; b=XtWDDLQf4/y3yOUS1cd1T83VTeOKOUX5Wxk1J9iyGGm85IKUp7OlWhYst2qNjtE2/F 6nFYLdbmEs4KhLEm04F1D1T24lAHLCN25x4GTqTaC0Y22DM6RkOG358oHyR7q+vCAUXN HGwt9Gvru0RN8LY2OfIda0WBJf2xIJ+nPrCNkFWW4Yy0i78+EYjFvmU2MwbS/ipNV7e+ uxw6LK2ujJDY3PZQ0FCPKzpZAXhViJWGBx9tGQwGNsj7UPGoqdr0XAYk9Ag2XzOUT2BV FpJqBThGMC6cfRnlrh6Cg0w2rcc6jjzzZOoRZh6tISfm6tT6wRNIsMJb92u7iyXDgm5m S31g== X-Gm-Message-State: AA+aEWYuSlJwAVfj1t07t/kO0zSuCXzx/1BM1RoAzZH7zqGLMaO9HR+8 dw0MuIeK4i2HOwfpVCGDpDQP7RwVMttwEqiLa8MWHbpc16QMB8QIvsh/Lv79psYMmi+0S4aQw5E TVQD+ZIRy6ZPZt6NfVDLuhjXgj9FklJh9CIcu8jCRm7WOF4BcNrGzEq1np8bpxSQhyH2NIXzP33 SveHoCQq5pw0LQzMFp3FjD9evc7l6NlOePKHvkSxJ34hUM8CgYs0uekjOyiED86G8JdmVs1jzXe 8diDrbkQoJ12Jj0jHdZpzqjOQv4m7nlFxSS/PPxALYD1BrjwC57AunVu+Bj4xMJfJaEaAmtJ5v5 NidyQMIwLCjTh15xqQ2xsBFDLR87vooymyeByXejU/lENi1dWKmncaCyxXqYjcOpoDMsCB1Jtsm m X-Received: by 2002:adf:fe8f:: with SMTP id l15mr27056501wrr.313.1543337780634; Tue, 27 Nov 2018 08:56:20 -0800 (PST) X-Received: by 2002:adf:fe8f:: with SMTP id l15mr27056416wrr.313.1543337779221; Tue, 27 Nov 2018 08:56:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337779; cv=none; d=google.com; s=arc-20160816; b=avVRQe3rTMJaPbx+WYtdSPk6Ax6+lswZX9anmDLWJZfnWGUeaFWPW4HVdBA69lWMmw S8ROoxlH4P9XGNXcbcsb2ko5KpXCNVwxiUSK/5irjExEQGhi9tGky2loWtPiC06YJcxT ENgFZidPMH1KuZC5HjNwhKqleboCUK3k3HMwtqiGaDkKbVD0eBeSRmp59zX3aX9RVuk1 VzxEXkUFkHGGQK2p++x2htIU43AnPwxyDOmbHcflAvtZDVj97LPsKC8BaECwi6lsDSZE MB0stYt5YPcAWLqA2N83PRtbV9GY6Po2eXVeJS4K3Z5gVH/hZKeYyDSwn8GpgCYqJaWv 9JoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=palwGaixv/Wt9FyOgrJvUwZx+w02pRpq6CF1mb0/y3U=; b=bJOPTMcmAiP+3/BkkgmM33ArOl6LMi8bXLtVVysdPMQeLPT5jF6qrl+rzAH7Wo3pSt zbDStAhvmyk9emcP94fQwHPNwCTx6pHfwxXvg6us/meqmvitetoaUhRYLzM8dyFGz5tX 8ivSZFsqxf3V6BUjyqqTgFTnID3CP0xM6xbgX8Pcry5R8On8z7mTcIzK9YjGG7mA7Dcn e6c8b+sbyl9HDiJOMRoKTKjHKOITXcMkLVHEfyuo5+sHg316CzOYMnC3lFunCKU9B8AE a2qFKVGUX9qCAPFvgv2G+idp9NkUhWE/S7srS7tQf8tC+0jivfRjOBGyyx22q8OWtgsu rpFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RFsixoyY; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i74sor3227954wmd.20.2018.11.27.08.56.19 for (Google Transport Security); Tue, 27 Nov 2018 08:56:19 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RFsixoyY; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=palwGaixv/Wt9FyOgrJvUwZx+w02pRpq6CF1mb0/y3U=; b=RFsixoyYXacko871sKkkAAbvRLeBBPGYReqX2vRo89uLtu5ykrgXpEATfMQ5CnL8I6 nP5/gdncyGAHWrrAA3AJZn5qQ9/umOOsb9e/pnDcpXvynMLeuOdXHFLScV38KPCCw5X5 zzWSvPrMZI2QgJJmbQqwKwciqsGof8E/bu6zfSAtnZXj6GjV1LvdslvhblVJZl1foL/J NycvnptL51Pw3yc688VJfxoAGeI1LGjp2BynGqAQkvxn4pWE61aac2ggFnjwEQQiPBZS KvdDUdQTDYwjkiksIjoD9HYu4uYEZfv78K4CX8GuKDMydmta6y6NTKI6g6hjJmgYA/F5 qo6g== X-Google-Smtp-Source: AJdET5fT38oj6Y9Md8HTTlRdeebfNnC3Ohg4vEjuzu333OnEXL2c0t9seyr/DwzuAjyedtSLAg3Oqg== X-Received: by 2002:a1c:8d46:: with SMTP id p67mr26739407wmd.28.1543337778289; Tue, 27 Nov 2018 08:56:18 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:17 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 16/25] kasan: split out generic_report.c from report.c Date: Tue, 27 Nov 2018 17:55:34 +0100 Message-Id: <9030fe246a786be1348f8b08089f30e52be23ec4.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch moves generic KASAN specific error reporting routines to generic_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by tag-based KASAN. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 4 +- mm/kasan/generic_report.c | 158 +++++++++++++++++++++++++ mm/kasan/kasan.h | 7 ++ mm/kasan/report.c | 234 +++++++++----------------------------- mm/kasan/tags_report.c | 39 +++++++ 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/generic_report.c create mode 100644 mm/kasan/tags_report.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 68ba1822f003..0a14fcff70ed 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -14,5 +14,5 @@ CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += tags.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c new file mode 100644 index 000000000000..5201d1770700 --- /dev/null +++ b/mm/kasan/generic_report.c @@ -0,0 +1,158 @@ +/* + * This file contains generic KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +static const void *find_first_bad_addr(const void *addr, size_t size) +{ + u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); + const void *first_bad_addr = addr; + + while (!shadow_val && first_bad_addr < addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +static const char *get_shadow_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + u8 *shadow_addr; + + info->first_bad_addr = find_first_bad_addr(info->access_addr, + info->access_size); + + shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); + + /* + * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * at the next shadow byte to determine the type of the bad access. + */ + if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + shadow_addr++; + + switch (*shadow_addr) { + case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + /* + * In theory it's still possible to see these shadow values + * due to a data race in the kernel code. + */ + bug_type = "out-of-bounds"; + break; + case KASAN_PAGE_REDZONE: + case KASAN_KMALLOC_REDZONE: + bug_type = "slab-out-of-bounds"; + break; + case KASAN_GLOBAL_REDZONE: + bug_type = "global-out-of-bounds"; + break; + case KASAN_STACK_LEFT: + case KASAN_STACK_MID: + case KASAN_STACK_RIGHT: + case KASAN_STACK_PARTIAL: + bug_type = "stack-out-of-bounds"; + break; + case KASAN_FREE_PAGE: + case KASAN_KMALLOC_FREE: + bug_type = "use-after-free"; + break; + case KASAN_USE_AFTER_SCOPE: + bug_type = "use-after-scope"; + break; + case KASAN_ALLOCA_LEFT: + case KASAN_ALLOCA_RIGHT: + bug_type = "alloca-out-of-bounds"; + break; + } + + return bug_type; +} + +static const char *get_wild_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + + if ((unsigned long)info->access_addr < PAGE_SIZE) + bug_type = "null-ptr-deref"; + else if ((unsigned long)info->access_addr < TASK_SIZE) + bug_type = "user-memory-access"; + else + bug_type = "wild-memory-access"; + + return bug_type; +} + +const char *get_bug_type(struct kasan_access_info *info) +{ + if (addr_has_shadow(info->access_addr)) + return get_shadow_bug_type(info); + return get_wild_bug_type(info); +} + +#define DEFINE_ASAN_REPORT_LOAD(size) \ +void __asan_report_load##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, false, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_load##size##_noabort) + +#define DEFINE_ASAN_REPORT_STORE(size) \ +void __asan_report_store##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, true, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_store##size##_noabort) + +DEFINE_ASAN_REPORT_LOAD(1); +DEFINE_ASAN_REPORT_LOAD(2); +DEFINE_ASAN_REPORT_LOAD(4); +DEFINE_ASAN_REPORT_LOAD(8); +DEFINE_ASAN_REPORT_LOAD(16); +DEFINE_ASAN_REPORT_STORE(1); +DEFINE_ASAN_REPORT_STORE(2); +DEFINE_ASAN_REPORT_STORE(4); +DEFINE_ASAN_REPORT_STORE(8); +DEFINE_ASAN_REPORT_STORE(16); + +void __asan_report_load_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_load_n_noabort); + +void __asan_report_store_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index b080b8d92812..33cc3b0e017e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow(const void *addr) +{ + return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +} + void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +const char *get_bug_type(struct kasan_access_info *info); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 5c169aa688fd..64a74f334c45 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,5 +1,5 @@ /* - * This file contains error reporting code. + * This file contains common generic and tag-based KASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -39,103 +39,34 @@ #define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) #define SHADOW_ROWS_AROUND_ADDR 2 -static const void *find_first_bad_addr(const void *addr, size_t size) -{ - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; - - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; -} +static unsigned long kasan_flags; -static bool addr_has_shadow(struct kasan_access_info *info) -{ - return (info->access_addr >= - kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -} +#define KASAN_BIT_REPORTED 0 +#define KASAN_BIT_MULTI_SHOT 1 -static const char *get_shadow_bug_type(struct kasan_access_info *info) +bool kasan_save_enable_multi_shot(void) { - const char *bug_type = "unknown-crash"; - u8 *shadow_addr; - - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); - - /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look - * at the next shadow byte to determine the type of the bad access. - */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) - shadow_addr++; - - switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: - /* - * In theory it's still possible to see these shadow values - * due to a data race in the kernel code. - */ - bug_type = "out-of-bounds"; - break; - case KASAN_PAGE_REDZONE: - case KASAN_KMALLOC_REDZONE: - bug_type = "slab-out-of-bounds"; - break; - case KASAN_GLOBAL_REDZONE: - bug_type = "global-out-of-bounds"; - break; - case KASAN_STACK_LEFT: - case KASAN_STACK_MID: - case KASAN_STACK_RIGHT: - case KASAN_STACK_PARTIAL: - bug_type = "stack-out-of-bounds"; - break; - case KASAN_FREE_PAGE: - case KASAN_KMALLOC_FREE: - bug_type = "use-after-free"; - break; - case KASAN_USE_AFTER_SCOPE: - bug_type = "use-after-scope"; - break; - case KASAN_ALLOCA_LEFT: - case KASAN_ALLOCA_RIGHT: - bug_type = "alloca-out-of-bounds"; - break; - } - - return bug_type; + return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); -static const char *get_wild_bug_type(struct kasan_access_info *info) +void kasan_restore_multi_shot(bool enabled) { - const char *bug_type = "unknown-crash"; - - if ((unsigned long)info->access_addr < PAGE_SIZE) - bug_type = "null-ptr-deref"; - else if ((unsigned long)info->access_addr < TASK_SIZE) - bug_type = "user-memory-access"; - else - bug_type = "wild-memory-access"; - - return bug_type; + if (!enabled) + clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); -static const char *get_bug_type(struct kasan_access_info *info) +static int __init kasan_set_multi_shot(char *str) { - if (addr_has_shadow(info)) - return get_shadow_bug_type(info); - return get_wild_bug_type(info); + set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); + return 1; } +__setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info) +static void print_error_description(struct kasan_access_info *info, + const char *bug_type) { - const char *bug_type = get_bug_type(info); - pr_err("BUG: KASAN: %s in %pS\n", bug_type, (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", @@ -143,25 +74,9 @@ static void print_error_description(struct kasan_access_info *info) info->access_addr, current->comm, task_pid_nr(current)); } -static inline bool kernel_or_module_addr(const void *addr) -{ - if (addr >= (void *)_stext && addr < (void *)_end) - return true; - if (is_module_address((unsigned long)addr)) - return true; - return false; -} - -static inline bool init_task_stack_addr(const void *addr) -{ - return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); -} - static DEFINE_SPINLOCK(report_lock); -static void kasan_start_report(unsigned long *flags) +static void start_report(unsigned long *flags) { /* * Make sure we don't end up in loop. @@ -171,7 +86,7 @@ static void kasan_start_report(unsigned long *flags) pr_err("==================================================================\n"); } -static void kasan_end_report(unsigned long *flags) +static void end_report(unsigned long *flags) { pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); @@ -249,6 +164,22 @@ static void describe_object(struct kmem_cache *cache, void *object, describe_object_addr(cache, object, addr); } +static inline bool kernel_or_module_addr(const void *addr) +{ + if (addr >= (void *)_stext && addr < (void *)_end) + return true; + if (is_module_address((unsigned long)addr)) + return true; + return false; +} + +static inline bool init_task_stack_addr(const void *addr) +{ + return addr >= (void *)&init_thread_union.stack && + (addr <= (void *)&init_thread_union.stack + + sizeof(init_thread_union.stack)); +} + static void print_address_description(void *addr) { struct page *page = addr_to_page(addr); @@ -326,29 +257,38 @@ static void print_shadow_for_address(const void *addr) } } +static bool report_enabled(void) +{ + if (current->kasan_depth) + return false; + if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + return true; + return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); +} + void kasan_report_invalid_free(void *object, unsigned long ip) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); pr_err("\n"); print_address_description(object); pr_err("\n"); print_shadow_for_address(object); - kasan_end_report(&flags); + end_report(&flags); } static void kasan_report_error(struct kasan_access_info *info) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); - print_error_description(info); + print_error_description(info, get_bug_type(info)); pr_err("\n"); - if (!addr_has_shadow(info)) { + if (!addr_has_shadow(info->access_addr)) { dump_stack(); } else { print_address_description((void *)info->access_addr); @@ -356,41 +296,7 @@ static void kasan_report_error(struct kasan_access_info *info) print_shadow_for_address(info->first_bad_addr); } - kasan_end_report(&flags); -} - -static unsigned long kasan_flags; - -#define KASAN_BIT_REPORTED 0 -#define KASAN_BIT_MULTI_SHOT 1 - -bool kasan_save_enable_multi_shot(void) -{ - return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); - -void kasan_restore_multi_shot(bool enabled) -{ - if (!enabled) - clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); - -static int __init kasan_set_multi_shot(char *str) -{ - set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); - return 1; -} -__setup("kasan_multi_shot", kasan_set_multi_shot); - -static inline bool kasan_report_enabled(void) -{ - if (current->kasan_depth) - return false; - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) - return true; - return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); + end_report(&flags); } void kasan_report(unsigned long addr, size_t size, @@ -398,7 +304,7 @@ void kasan_report(unsigned long addr, size_t size, { struct kasan_access_info info; - if (likely(!kasan_report_enabled())) + if (likely(!report_enabled())) return; disable_trace_on_warning(); @@ -411,41 +317,3 @@ void kasan_report(unsigned long addr, size_t size, kasan_report_error(&info); } - - -#define DEFINE_ASAN_REPORT_LOAD(size) \ -void __asan_report_load##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, false, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_load##size##_noabort) - -#define DEFINE_ASAN_REPORT_STORE(size) \ -void __asan_report_store##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, true, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_store##size##_noabort) - -DEFINE_ASAN_REPORT_LOAD(1); -DEFINE_ASAN_REPORT_LOAD(2); -DEFINE_ASAN_REPORT_LOAD(4); -DEFINE_ASAN_REPORT_LOAD(8); -DEFINE_ASAN_REPORT_LOAD(16); -DEFINE_ASAN_REPORT_STORE(1); -DEFINE_ASAN_REPORT_STORE(2); -DEFINE_ASAN_REPORT_STORE(4); -DEFINE_ASAN_REPORT_STORE(8); -DEFINE_ASAN_REPORT_STORE(16); - -void __asan_report_load_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_load_n_noabort); - -void __asan_report_store_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c new file mode 100644 index 000000000000..8af15e87d3bc --- /dev/null +++ b/mm/kasan/tags_report.c @@ -0,0 +1,39 @@ +/* + * This file contains tag-based KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} From patchwork Tue Nov 27 16:55:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 295FA14E2 for ; Tue, 27 Nov 2018 16:57:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 174032C427 for ; Tue, 27 Nov 2018 16:57:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0AD862C455; Tue, 27 Nov 2018 16:57:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6CD7E2C482 for ; Tue, 27 Nov 2018 16:57:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 190416B4955; Tue, 27 Nov 2018 11:56:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 119446B4959; Tue, 27 Nov 2018 11:56:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAD6D6B4958; Tue, 27 Nov 2018 11:56:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id 884626B4955 for ; Tue, 27 Nov 2018 11:56:22 -0500 (EST) Received: by mail-wm1-f71.google.com with SMTP id y85so18175912wmc.7 for ; Tue, 27 Nov 2018 08:56:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NGY24kEYlKV9za6j4+lLg+Mk37D+Fr2A0V+VMyaOAWQ=; b=ejvdbInhX1fD5GE5Sr+bGhU5KUslKKcmCpsfXrUCuXMEVgms5Flq+USDQ/vgm6L4ER n+z0fAQ6ViBTi1wVToINUd9DEOr90a+JUPi5FoRem2DOKMmclVsTSzt5EUWnTBNvoEaj m7/GcFTQ+es8aHO4U4M6SxiyrzxcTHOhwgv+swV4GhY+EvDQ4Fc+7MKJs3whP9zHMxWL itSfnAB//92QvVe8mgUC8dvbzFqYCW35nmE2rLxq8dYZBgAP3RPcrCFTDboimg9akJ0p 8tiuSUhpDfciZIXTD8KH0H/aOVNFan65D8jdisMg2fb47dj+ZsBLPaV5QY/IMkrKObIm QmRA== X-Gm-Message-State: AGRZ1gIySb9rKDwlP5oTVjhsnDF9yCoy5dLJwrAHqvO6aVJq4pWZ5r+r MGoEWI/iocvTzKI+Y+1iZTEjsYA9J1SYTAU0JjuD7KM/mfTICi8xFSOIf97X7nHllLJS4b5t7AG U7VoFZnQrSaTfkGVMzJoBbzRUPYTE8FsbPCZuYv2MTfeN6ccYj4he4mBqbF9pVIDpxprHq58RVK PnzNX9K7GNyhw0pZBNoFAtkkGwEO39k9gPaD1/WdyWcpGCrlAZmOvn2Cjy7GA0V6A5JB8TlKsZR w7Z4b9mWJ/ciEJbdJGpg2ghEbsiV8P6yobyKpZrZtfntS1dbVmQ/HxBSeDT6XGfusOlEkiU3nPv SISv7G3gJsFvpbTk7j0tyoXmH1drOgz2TSZW+uz8v64TWtmz60FA58Q1wNU7SfCl9Rn2OCgftZ0 G X-Received: by 2002:a1c:9c0a:: with SMTP id f10mr28947846wme.73.1543337782044; Tue, 27 Nov 2018 08:56:22 -0800 (PST) X-Received: by 2002:a1c:9c0a:: with SMTP id f10mr28947778wme.73.1543337780779; Tue, 27 Nov 2018 08:56:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337780; cv=none; d=google.com; s=arc-20160816; b=y2hCCxw3fJq5YFzrIioeZvUWyqNCHFPhF7CwGGmu5zw52KS3C4yrepLM+Te5nH+M16 0LroRGemRlo+4XUte5FShWtDVCdFHLn/wchsDrZMT1JeSxXvwwkmkOyRzTLrd6DpQ8hW ikY/ntbRy57Om3AEuJgvObUMN7AXy6DT4R5fBsXmPcQlv3AmocAuL1/CwvcxkSywjNCc W2bD0l/0Dfgrq3GV8az0zLWtDXl8O9Q+lFtpxOyDPPvvYr/I1AWPwesiZDiIFXBd7idW TjWv0gHv+WheRxIeZfjbF8F3C2B/22Z5skXGxQpTtsDEfKpPC6qbf9nmVuLGkjgDYoke SnKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=NGY24kEYlKV9za6j4+lLg+Mk37D+Fr2A0V+VMyaOAWQ=; b=aCiUEEJUv/z42+6jmgBlQlmnLOuRv6sZpL2YOwzTs2iheKuDwvRlPAtF2wdkTx8HbA 6buPvd+0jfD8WXjtZhdSEt0bqRPnbxCwhemHhqylNWJWDuWZo8iQEUd3/g4w079YBLLD Xh+N/cynb36lmUPBwRXfn2JSXaEJpyXqILD0hB7Gunc5WCZxRmP1c30IM8URI/yfeZQD FdL7CzteAjsC7i7/M/5ET9oIBL9mWsxmB8996j4GYk9wVCqW7rkoXc7V+KfEppyWkuzC OGrGOJexG4iTrqskW5mnQYQHXuPZWY3q6ksudmlc6Ifihm6ortwHUBvQxC+iQgwhYc0+ TuYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Y7JqEyQS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 51sor3115673wra.51.2018.11.27.08.56.20 for (Google Transport Security); Tue, 27 Nov 2018 08:56:20 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Y7JqEyQS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NGY24kEYlKV9za6j4+lLg+Mk37D+Fr2A0V+VMyaOAWQ=; b=Y7JqEyQSNecPNfIakkOPNWEwozlvQ7ETwuh0OMtbtbMYqXQfovnxd3NByCHMQnfWdE ebrdGC7fbXgpWh7tVKpiOQ3SBN0iFuxb8A6HilwShILfNEkMxIO1gsHrq8F3oY0cdw1U mt+Rayis9WOPMzHc3QMBeO3T+clBxfiO2BGANOhY7g24ixnqihd710PDAtHJNxbEoWfB UhTp/V3tfBoEDp3a4IEvOzJDd94a+ALJ8vy4G7KwlLH4epF1VTUxj7iqVGgj4KPBBfbT 5SXPz5hbzYMoZMa3xobi8WbquZxaw5JcCRwjPmT629uqE6pzCrMCBw1ojI/VEgbU71Q1 ydng== X-Google-Smtp-Source: AFSGD/U/BXacGpHDneGOXiqEqOPWRyahWqAl1r+NjMlboEuKVzUM6mURIC25l9XjhMIdhU76elSmew== X-Received: by 2002:adf:c042:: with SMTP id c2mr16699964wrf.158.1543337780158; Tue, 27 Nov 2018 08:56:20 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:19 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 17/25] kasan: add bug reporting routines for tag-based mode Date: Tue, 27 Nov 2018 17:55:35 +0100 Message-Id: <996c09ec2c8f11294c106973f3b1a211417fa74e.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds rountines, that print tag-based KASAN error reports. Those are quite similar to generic KASAN, the difference is: 1. The way tag-based KASAN finds the first bad shadow cell (with a mismatching tag). Tag-based KASAN compares memory tags from the shadow memory to the pointer tag. 2. Tag-based KASAN reports all bugs with the "KASAN: invalid-access" header. Also simplify generic KASAN find_first_bad_addr. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/generic_report.c | 16 ++++------- mm/kasan/kasan.h | 5 ++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ mm/kasan/tags_report.c | 18 +++++++++++++ 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index 5201d1770700..a4604cceae59 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -33,16 +33,13 @@ #include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr(const void *addr, size_t size) +void *find_first_bad_addr(void *addr, size_t size) { - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; + void *p = addr; - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; + while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p))) + p += KASAN_SHADOW_SCALE_SIZE; + return p; } static const char *get_shadow_bug_type(struct kasan_access_info *info) @@ -50,9 +47,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info) const char *bug_type = "unknown-crash"; u8 *shadow_addr; - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 33cc3b0e017e..82a23b23ff93 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -119,6 +119,7 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void kasan_report(unsigned long addr, size_t size, @@ -139,10 +140,14 @@ static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #ifdef CONFIG_KASAN_SW_TAGS +void print_tags(u8 addr_tag, const void *addr); + u8 random_tag(void); #else +static inline void print_tags(u8 addr_tag, const void *addr) { } + static inline u8 random_tag(void) { return 0; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 64a74f334c45..214d85035f99 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) } __setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info, - const char *bug_type) +static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - bug_type, (void *)info->ip); + get_bug_type(info), (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, info->access_addr, current->comm, task_pid_nr(current)); @@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip) start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); + print_tags(get_tag(object), reset_tag(object)); + object = reset_tag(object); pr_err("\n"); print_address_description(object); pr_err("\n"); @@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags); } -static void kasan_report_error(struct kasan_access_info *info) -{ - unsigned long flags; - - start_report(&flags); - - print_error_description(info, get_bug_type(info)); - pr_err("\n"); - - if (!addr_has_shadow(info->access_addr)) { - dump_stack(); - } else { - print_address_description((void *)info->access_addr); - pr_err("\n"); - print_shadow_for_address(info->first_bad_addr); - } - - end_report(&flags); -} - void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { struct kasan_access_info info; + void *tagged_addr; + void *untagged_addr; + unsigned long flags; if (likely(!report_enabled())) return; disable_trace_on_warning(); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr = reset_tag(tagged_addr); + + info.access_addr = tagged_addr; + if (addr_has_shadow(untagged_addr)) + info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; - kasan_report_error(&info); + start_report(&flags); + + print_error_description(&info); + if (addr_has_shadow(untagged_addr)) + print_tags(get_tag(tagged_addr), info.first_bad_addr); + pr_err("\n"); + + if (addr_has_shadow(untagged_addr)) { + print_address_description(untagged_addr); + pr_err("\n"); + print_shadow_for_address(info.first_bad_addr); + } else { + dump_stack(); + } + + end_report(&flags); } diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 8af15e87d3bc..573c51d20d09 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -37,3 +37,21 @@ const char *get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } + +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *p = reset_tag(addr); + void *end = p + size; + + while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) + p += KASAN_SHADOW_SCALE_SIZE; + return p; +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); +} From patchwork Tue Nov 27 16:55:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D660614E2 for ; Tue, 27 Nov 2018 16:57:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C44F02C482 for ; Tue, 27 Nov 2018 16:57:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B74C22C496; Tue, 27 Nov 2018 16:57:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27FF02C455 for ; Tue, 27 Nov 2018 16:57:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7ABAA6B4959; Tue, 27 Nov 2018 11:56:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 70E1C6B495B; Tue, 27 Nov 2018 11:56:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5614C6B495C; Tue, 27 Nov 2018 11:56:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id EDBE26B4959 for ; Tue, 27 Nov 2018 11:56:23 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id o76so16748170wmg.4 for ; Tue, 27 Nov 2018 08:56:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=gUtzPQYfY+TlBxAchiIeeRXD6wCQXCysNds7bqypcX0=; b=n9kfURhnObA44J23nmUzyhjeTT2yjiXVbzZH0wdsEW20CkAiU0FkRWL5GDheTz4Ab8 lpUHej0ZNdjjesDupdOS/mWfweo1FE/k6g16H58rW8uYemIZCsHuDMk+ckGjEgXEtOBN 6Rvf59TF6gnY7A1uj6OEbZ9QDhlv1hUYz07B+R/goXIm25nxLXPSm5sQMiw4XcNK1S64 hDfwqUXa+7wZu5Vu/HyakpENMRtF+HJppKTh1m1NRINRvQBMxQaBbdtanTuUgalstEfc n1CPXnWKMGTsUdGmuArRdzGFCPWB7fEssi09EEs1u3dxfnPJvHOp8R42Zm2JrHyGKcbX DwZQ== X-Gm-Message-State: AA+aEWaE3+Tf+bLvGtpobTpSvd5H1s5Kzhj/45Y3sIcDj9ljx8gbPOTj ysglqYx+zvaqYITemruLBrlH8hEthsyKpVlph+dGd25b+Fj3Plaik16g3j/54pTHLH1h/Dqi2WW EYWp9Ts8o2B4xBTysNCm9DjXqG1p7UtxOVWf2jcefZzuWDFUseCzk9QrEWD9p7iBSAClKrnv237 GZYkVv+tq8sc62MVmD/gzKmy9BKLS2Pjj1sSBhPDgxtpqAsO69ZUpdCnY8/ZQD6Vonh+YZpe2Tp qlqOUURAFD2pyv4u+YBjVHyEACnnAL1i6JJFRgYF8zYoTUhaIdR2nBUN8ELlcEE43Ru1lnRCfYA bldEX6idtCXO4N7jbhzpjoBVlFJ2+qF+o4usSCHcV8CmF3oenTLag0wuTASSDGKBASvOW3FjSdY i X-Received: by 2002:adf:fa05:: with SMTP id m5mr23929651wrr.155.1543337783439; Tue, 27 Nov 2018 08:56:23 -0800 (PST) X-Received: by 2002:adf:fa05:: with SMTP id m5mr23929601wrr.155.1543337782518; Tue, 27 Nov 2018 08:56:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337782; cv=none; d=google.com; s=arc-20160816; b=j4Qn6Uhfse1T+zOW+5pPhMr1eTW9opC4DXgilzakMGxXASCETi2HgZA5d2FnItwasH yHoNxg5YQc1G55PEn+UH6+jn78h17jkfgUcAPCAEAvfcTrwRagpM0KWn8bbD+biCaveb riDoB4ZPcnjjg4p5oNWN/ErlBPtWojeHeUz9cJ7nZThbC/K9CbnXeYp9YtrfLPCl5n0A BLVnuHPpnWguEx1QCeFq9pfV6L39TuJXzVpGP1Jr03/5OLGY4wOQU9qItX3Tfy9N2dq3 clRymc/rnK9i6GZgE4KBwTw6C4radYu/k+tCJrLV5e8LdpOG50GLtzPzp1YCdUKfabmx MvOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=gUtzPQYfY+TlBxAchiIeeRXD6wCQXCysNds7bqypcX0=; b=J5nUb0aVu6sGO9N9E4XgPhOPQsszaGvFomOBglNJFlZMOeLr0yt0VjJhb4o6S3VEWa 0+4bp4tlSIbRV+VIOfDycNH0FwmIjCYe/bBFuKhfSLt/uU7NQxXWF68IQR/F9PkPDIpO NJUE5V0EpHmBW59Ik/VCLOZ6rZnF1/LWEtxwHVG/woV0WDDYOrUHiOHl9c5CyU8jzjMJ qMrkzB1r6hS2a2wDVe+1UQimV3bQ0mGf81pzTLZhlCS4XL/VArgtDC1dAD3h/TTqWyYz UB/E4X2xFeM60JgMiKJpfoDbP1ssC3HGRV4I87tBsfCn+XDAfQtpyLeOxBpYMdfK6KA8 XDRw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=IMJYfJiI; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r201sor3378349wmb.26.2018.11.27.08.56.22 for (Google Transport Security); Tue, 27 Nov 2018 08:56:22 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=IMJYfJiI; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gUtzPQYfY+TlBxAchiIeeRXD6wCQXCysNds7bqypcX0=; b=IMJYfJiI/vB4T61XMcgVTV8zpG/cNUZ9nUDQYlZotMywFSqsb9p6E5fGevB3WlOUPN WYEvLbC1mSpKJzaGwPyy4OY2YppWyioLYLVvVb2CTW36kbdDIZZULSxTmqVX0yCtLGZr h7ait1GfvypAFS10mVxKoj2RlvQLCgzvXNgOOzQ/gI4BzcJfQWOT8XhNOy4hcBvljuF3 m6lSda+HiC/s2RX543RqEk/6NhgRyhKa+80UGh0EpNFOe+b4oZiJW+HH4RvKtKxC/GCq 4CxUQoyDF4r1KcfNAGQtcjiexUSciNdRwhomD+90qhKiRoDRT0rbnTmTSXi3u6aQ4z7T QA2w== X-Google-Smtp-Source: AJdET5cRyGAnPAul2HIkjRAXyOS/E0C99g3NbwpaMk33p87uBm2YavwrII5dayxtmukuFuwV97cF/w== X-Received: by 2002:a1c:b607:: with SMTP id g7mr29334808wmf.97.1543337781882; Tue, 27 Nov 2018 08:56:21 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:21 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 18/25] mm: move obj_to_index to include/linux/slab_def.h Date: Tue, 27 Nov 2018 17:55:36 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP While with SLUB we can actually preassign tags for caches with contructors and store them in pointers in the freelist, SLAB doesn't allow that since the freelist is stored as an array of indexes, so there are no pointers to store the tags. Instead we compute the tag twice, once when a slab is created before calling the constructor and then again each time when an object is allocated with kmalloc. Tag is computed simply by taking the lowest byte of the index that corresponds to the object. However in kasan_kmalloc we only have access to the objects pointer, so we need a way to find out which index this object corresponds to. This patch moves obj_to_index from slab.c to include/linux/slab_def.h to be reused by KASAN. Acked-by: Christoph Lameter Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/slab_def.h | 13 +++++++++++++ mm/slab.c | 13 ------------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h index 3485c58cfd1c..9a5eafb7145b 100644 --- a/include/linux/slab_def.h +++ b/include/linux/slab_def.h @@ -104,4 +104,17 @@ static inline void *nearest_obj(struct kmem_cache *cache, struct page *page, return object; } +/* + * We want to avoid an expensive divide : (offset / cache->size) + * Using the fact that size is a constant for a particular cache, + * we can replace (offset / cache->size) by + * reciprocal_divide(offset, cache->reciprocal_buffer_size) + */ +static inline unsigned int obj_to_index(const struct kmem_cache *cache, + const struct page *page, void *obj) +{ + u32 offset = (obj - page->s_mem); + return reciprocal_divide(offset, cache->reciprocal_buffer_size); +} + #endif /* _LINUX_SLAB_DEF_H */ diff --git a/mm/slab.c b/mm/slab.c index 27859fb39889..d2f827316dfc 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -406,19 +406,6 @@ static inline void *index_to_obj(struct kmem_cache *cache, struct page *page, return page->s_mem + cache->size * idx; } -/* - * We want to avoid an expensive divide : (offset / cache->size) - * Using the fact that size is a constant for a particular cache, - * we can replace (offset / cache->size) by - * reciprocal_divide(offset, cache->reciprocal_buffer_size) - */ -static inline unsigned int obj_to_index(const struct kmem_cache *cache, - const struct page *page, void *obj) -{ - u32 offset = (obj - page->s_mem); - return reciprocal_divide(offset, cache->reciprocal_buffer_size); -} - #define BOOT_CPUCACHE_ENTRIES 1 /* internal cache of cache description objs */ static struct kmem_cache kmem_cache_boot = { From patchwork Tue Nov 27 16:55:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700979 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D603714E2 for ; Tue, 27 Nov 2018 16:57:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2D862C455 for ; Tue, 27 Nov 2018 16:57:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B66D02C45B; Tue, 27 Nov 2018 16:57:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B394E2C455 for ; Tue, 27 Nov 2018 16:57:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07F7E6B495B; Tue, 27 Nov 2018 11:56:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F25D16B495D; Tue, 27 Nov 2018 11:56:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D271E6B495E; Tue, 27 Nov 2018 11:56:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 6126B6B495B for ; Tue, 27 Nov 2018 11:56:26 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id p16so16920671wmc.5 for ; Tue, 27 Nov 2018 08:56:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Qoowd8ryNjw4tIo6irdC9emag24JrZFGOlLB83qwqU4=; b=Ve5uC9oN5gfcUrurOAEkjwdDrH4vpl4xFJNjXfrgt1IDDZIqZ6Ob+9rQJzqehfmPLk si8oMCHCq4sRu6IKBUNuQ1TmwQQm91kBabFezLcV5KoWI1KPl5xtzviaWfb1NM0zOOHz CtJ4+QYPqPeTvUHU9/7vMpB0DWszjF4x/t+AEVDtPeG5iXNWQSO33X8ab8JCBKDmlYxD IN+85HKNtToKXK1RwYQxKEpOFXpubQU4jOJ18U/+HXa8cTV7+EoKebW1J0ilnIDQZJga 3gNBE4n6w4NG3s8u9lXvd6a96GYQdotb0S/nG5bV+WCOkuWB7M+UHDTGxPP+4BZaDFPL mJxQ== X-Gm-Message-State: AA+aEWbgxJarTHatzA/pbI8Wrnjo6jgWY7Rj/YTdPACOHc/hXAsX/IWE 5tqVKf7n0rG+SC4s73mEwiyNdVYJ/09NTtan4Lu3/FS0/FkWL6zOb8DCVkMBOsF5GcIzwcIpa+8 SvkXPI66r+avEihg6ZZnIHEM3n1hObhNtKkDrkel41I7nZSH/aZo21h/fh10IlT+gWZ0aAluWLE VmVT7+wioNn67aNaprOYwd2Yvzw75OIxFkm4QsQqlg0D5y9Gk1wCw/0JkwfG+GpuNBSmIOdjcHs iF9RxcxfUEyjXoMpzh4cy/GVV4AVk5Vi+MGTvcc1zXdJxYmqEKEbi94FFNUTkstXpR6RdKzArrd P8e0yrCqqr3T2Cf3a52zKs13QfgDEMpPti/hrTDFTsUaSONQqjcTgvX65tsE3wcXnX/c83/q2aI E X-Received: by 2002:a5d:694d:: with SMTP id r13mr27352232wrw.323.1543337785866; Tue, 27 Nov 2018 08:56:25 -0800 (PST) X-Received: by 2002:a5d:694d:: with SMTP id r13mr27352177wrw.323.1543337784603; Tue, 27 Nov 2018 08:56:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337784; cv=none; d=google.com; s=arc-20160816; b=KfJmAvfcZknPHNLM8MlZiy59+edbGJD7jMQ4zgQ8JS2UIzs2BUUWSUG/RisyRa/I+p MbOHK1rs5USY6gbut9w9qFO3dA+SE4Tjl9Y1ldINFsf9cQBfA/LrmmmwH76r1jvIG9ol stjO2f8zxfBZbMudMkwW6UgwUkjnSJy7VcBcp9vpICP2Lbuz+gZ6pMdrl6OWnziWn0fB zO8tlRgcKrtsbKNYiArbbZpWaieyUVFrx9E7Gnw54Dodnk5C+rf0jR8U8qzSr2WbD0J5 dHpTFDa5b7iA+4yY+Y9rCA1RxrdkRtnLNnNW8y0OciELJAUWfMfeTBNqeQXIywQnPPl6 i5uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=Qoowd8ryNjw4tIo6irdC9emag24JrZFGOlLB83qwqU4=; b=Y16/bjBgiSFr/aM0Op9XD/WJiLAdeS8sA3aSirpOlOHNRuKIr7hHRyMRuwg04cLotY hc3yR9G99jreim5J40Z1/OZUOGTcWyOanQdVDEfjswfOn2KI2I2bEp6essC5HWT9GisP WqjrBugKmJQd39GeIvxiflRL2V50O7FnfLwepr41NhaP1YNjJQX0uiwVW4QaQmO7Nc1b eW6V+bIM+xACI+Fs4V5UB30zXedzQalIGK+TJEMFnWcwhYJW7AClsc5Ua/7mBJZuhvJq 75qTETzDUQ5LCKLeBWcG8hzVY33tELD8BhGXmaH92D4DiVciz322/u9G6sEGEdu+QX+z KD+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=d4UHp4UK; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h7sor3173057wrv.20.2018.11.27.08.56.24 for (Google Transport Security); Tue, 27 Nov 2018 08:56:24 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=d4UHp4UK; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Qoowd8ryNjw4tIo6irdC9emag24JrZFGOlLB83qwqU4=; b=d4UHp4UKFHQ92lBZFEqvErWapi1CGrlf/eFfRSrnvRFAp1oJ3caR+Y0OXZQjYmTYDb KR+BpgNHAsAnEyUINkYBbUIO0OpowoyIWxk31QWeGwZxN7lDQvT0C2j8d6aW+lUDx0oS T512WX5hDXr3CzH7lYPCB5AoSKaRmynH2UvVew2IqooYX4rWrO6OO62+ElFpnf0olGHN y2FBWc/wARXaDkXhgyq1x82tvGjQvOMv7hCq+O2/RxIXWy+5HcwzLd9vK0vmM35zmLz4 8d6d/qeJMxl0cJK1YtWm+sQPfq7I19l769rFpEn5ZSI0eb6iqsyyTjGszUAMIwK2rX5I rgYg== X-Google-Smtp-Source: AFSGD/VT1GpfD9w/yk+1slasakqekg8TIXk3GEtsTswM2vLn4GzGw7vLGxir3qhe5sIwM4B+VtwV9w== X-Received: by 2002:adf:a743:: with SMTP id e3mr27294693wrd.56.1543337783830; Tue, 27 Nov 2018 08:56:23 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:22 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 19/25] kasan: add hooks implementation for tag-based mode Date: Tue, 27 Nov 2018 17:55:37 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds tag-based KASAN specific hooks implementation and adjusts common generic and tag-based KASAN ones. 1. When a new slab cache is created, tag-based KASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16). 2. On each kmalloc tag-based KASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer. 3. On each kfree tag-based KASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs. The rest of the logic of the hook implementation is very much similar to the one provided by generic KASAN. Tag-based KASAN saves allocation and free stack metadata to the slab object the same way generic KASAN does. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 116 ++++++++++++++++++++++++++++++++++++++-------- mm/kasan/kasan.h | 8 ++++ mm/kasan/tags.c | 48 +++++++++++++++++++ 3 files changed, 153 insertions(+), 19 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 7134e75447ff..27f0cae336c9 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -140,6 +140,13 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_poison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -148,11 +155,24 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) void kasan_unpoison_shadow(const void *address, size_t size) { - kasan_poison_shadow(address, size, 0); + u8 tag = get_tag(address); + + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_unpoison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + + kasan_poison_shadow(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; + + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + *shadow = tag; + else + *shadow = size & KASAN_SHADOW_MASK; } } @@ -200,8 +220,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + if (unlikely(PageHighMem(page))) + return; + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) @@ -218,6 +239,9 @@ void kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + return 0; + return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -232,6 +256,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { unsigned int orig_size = *size; + unsigned int redzone_size; int redzone_adjust; /* Add alloc meta. */ @@ -239,20 +264,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size += sizeof(struct kasan_alloc_meta); /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); if (redzone_adjust > 0) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); + max(*size, cache->object_size + redzone_size)); /* * If the metadata doesn't fit, don't enable KASAN at all. @@ -265,6 +290,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, return; } + cache->align = round_up(cache->align, KASAN_SHADOW_SCALE_SIZE); + *flags |= SLAB_KASAN; } @@ -309,6 +336,32 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object) KASAN_KMALLOC_REDZONE); } +/* + * Since it's desirable to only call object contructors once during slab + * allocation, we preassign tags to all such objects. Also preassign tags for + * SLAB_TYPESAFE_BY_RCU slabs to avoid use-after-free reports. + * For SLAB allocator we can't preassign tags randomly since the freelist is + * stored as an array of indexes instead of a linked list. Assign tags based + * on objects indexes, so that objects that are next to each other get + * different tags. + * After a tag is assigned, the object always gets allocated with the same tag. + * The reason is that we can't change tags for objects with constructors on + * reallocation (even for non-SLAB_TYPESAFE_BY_RCU), because the constructor + * code can save the pointer to the object somewhere (e.g. in the object + * itself). Then if we retag it, the old saved pointer will become invalid. + */ +static u8 assign_tag(struct kmem_cache *cache, const void *object, bool new) +{ + if (!cache->ctor && !(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return new ? KASAN_TAG_KERNEL : random_tag(); + +#ifdef CONFIG_SLAB + return (u8)obj_to_index(cache, virt_to_page(object), (void *)object); +#else + return new ? random_tag() : get_tag(object); +#endif +} + void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; @@ -319,6 +372,9 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + object = set_tag(object, assign_tag(cache, object, true)); + return (void *)object; } @@ -327,15 +383,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) return kasan_kmalloc(cache, object, cache->object_size, flags); } +static inline bool shadow_invalid(u8 tag, s8 shadow_byte) +{ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return shadow_byte < 0 || + shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + else + return tag != (u8)shadow_byte; +} + static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { s8 shadow_byte; + u8 tag; + void *tagged_object; unsigned long rounded_up_size; + tag = get_tag(object); + tagged_object = object; + object = reset_tag(object); + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { - kasan_report_invalid_free(object, ip); + kasan_report_invalid_free(tagged_object, ip); return true; } @@ -344,20 +415,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return false; shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); + if (shadow_invalid(tag, shadow_byte)) { + kasan_report_invalid_free(tagged_object, ip); return true; } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) return false; set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); quarantine_put(get_free_info(cache, object), cache); - return true; + + return IS_ENABLED(CONFIG_KASAN_GENERIC); } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -370,6 +443,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, { unsigned long redzone_start; unsigned long redzone_end; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -382,14 +456,18 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(object, size); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + tag = assign_tag(cache, object, false); + + /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ + kasan_unpoison_shadow(set_tag(object, tag), size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); - return (void *)object; + return set_tag(object, tag); } EXPORT_SYMBOL(kasan_kmalloc); @@ -439,7 +517,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { + if (reset_tag(ptr) != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -452,7 +530,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (ptr != page_address(virt_to_head_page(ptr))) + if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 82a23b23ff93..ea51b2d898ec 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,10 +12,18 @@ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KASAN_TAG_INVALID +#endif + #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ /* diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 1c4e7ce2e6fe..1d1b79350e28 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -78,15 +78,60 @@ void *kasan_reset_tag(const void *addr) void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { + u8 tag; + u8 *shadow_first, *shadow_last, *shadow; + void *untagged_addr; + + if (unlikely(size == 0)) + return; + + tag = get_tag((const void *)addr); + + /* + * Ignore accesses for pointers tagged with 0xff (native kernel + * pointer tag) to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KASAN_TAG_KERNEL (0xFF)). + */ + if (tag == KASAN_TAG_KERNEL) + return; + + untagged_addr = reset_tag((const void *)addr); + if (unlikely(untagged_addr < + kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { + kasan_report(addr, size, write, ret_ip); + return; + } + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + size - 1); + for (shadow = shadow_first; shadow <= shadow_last; shadow++) { + if (*shadow != tag) { + kasan_report(addr, size, write, ret_ip); + return; + } + } } #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -98,15 +143,18 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { + kasan_poison_shadow((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); From patchwork Tue Nov 27 16:55:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700985 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2EAE109C for ; Tue, 27 Nov 2018 16:57:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0EC02C449 for ; Tue, 27 Nov 2018 16:57:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4E4E2C45B; Tue, 27 Nov 2018 16:57:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C9B92C455 for ; Tue, 27 Nov 2018 16:57:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 847DA6B495D; Tue, 27 Nov 2018 11:56:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7FA866B495F; Tue, 27 Nov 2018 11:56:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69D1B6B4960; Tue, 27 Nov 2018 11:56:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 0C30C6B495D for ; Tue, 27 Nov 2018 11:56:28 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id e17so18224767wrw.13 for ; Tue, 27 Nov 2018 08:56:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zstYfFUM8JCpnxw1Tbdi2+5o2Ofx4Ch1nUlOE/Rv2dg=; b=HP5enw9vH6siJSN1gxNe0zr1ycksZsdSuWuh1XIcSJn+UL/lnSYF/kUnTYdLsFT0Lw Gb2j3M7CZc1t1tiJf8BlF19RQ4TVRcMcQ7XeQ36ldb47KtcM6duU/dRU8hE0cOjwxnM9 a+p7I4gyVlwKom0bwxUFayjHN6ddGpfeGDE3aGEn8lfH7zbxeSz0ANtWeldSluqIKa8k cMTdr3SZobxC5Kj4B4lP5oSVc685WbW3lqqGvzxYo/YsgVGZ6X+M6O9Kx6POB1vtNW1v +0DoWzGZwdAfhqf4hIJ4AnM1E2jt+c12G2SG6ObnXodbBviTAMBGPIMTX/d3zA8Mhysj l/Dg== X-Gm-Message-State: AA+aEWbp9bqYjUH/14neddNgm93nip2IIZRmNoY7c7S9JIU1d0aguyop oPXvG+dI5b1qf3V/M6G0IQIAbma5zqk18ISP6Gb9CTIOxDv0NGsycMYcRhzZEJjdj/BQFMXnVyC weGzE81nrvmVtnCQWwAu8i/54Ygsus5+ImkAg0uGy7zpwFy7NFHhz/Lh6uJONEfOaQ5x6gDbl3r FEGVKNMUhs5p6RWJTYXBWnJbm3IstuNjwCvhrdppeqrE9tuygE+jW6EFxEyGzFlKrbMKGCFthNC jJG/p51SFAvWZIWG8DQcgjT0uYcv8bRqPzSXEyzMv9AXVE+naZkXo1ZdWO4j7OEG+xoqP3Z2Goz +TYgfWwiwHOzTuNi0S/r6JXk+vBxChXyBeRgCpoLMOpFDnEGiFMLQFfbaktsFPO5jYO78tAxDh8 3 X-Received: by 2002:adf:ec11:: with SMTP id x17mr28892445wrn.88.1543337787511; Tue, 27 Nov 2018 08:56:27 -0800 (PST) X-Received: by 2002:adf:ec11:: with SMTP id x17mr28892398wrn.88.1543337786478; Tue, 27 Nov 2018 08:56:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337786; cv=none; d=google.com; s=arc-20160816; b=EZcL3mlH71K3Lq9V+MzzDj0xrD4Y/iY2EU8U4sK5KVQvzPDat4+KhzDBLI89MfafWO 6U2C+nVko/wNyqC4+wXQqFKHQ7GlG+jXEb1h/g4YrKw9zQzLqa9u4LpdvsKbomDGxNFi Z+1AfbkKMZRnYBjq76qIleKLop7jKjJ/b9voha//2JqbLS8pt8Bk05jBXdOFiEsa0b54 LNciq93VGU789d/UWxZD0zfz6+tgTObxwaNk4N/fZufC+bfjdJYAH00jEeuPKcmNRZYi 32B0Z2V9BgY8ygbQBrt0oobKZNIoqsUXUlPDZpnNZiEfWOfD9VhDAJPIg9MXW2kjpc8J 9rSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=zstYfFUM8JCpnxw1Tbdi2+5o2Ofx4Ch1nUlOE/Rv2dg=; b=j0I2kufeucN45/imcEql2uyngGgbgNcIaO3L9SQZz13wBwrPlKQjz7qUREln19uq26 tnS0A3gNuXQcn24g3XGmrJoHCWUx1GL0CpxbhV5ij/u0e+fmh4Nsu3odkGH62wT8E23a D7hRNmJR3loc5vQXc9yh4w7thNEgm5FQJw2V2ZLBYLCrw/QZRnJtr6BQOEpH06wm/EVn gObHoH4boHFYICJVBLhNUn6TDNWhr/Ndl0SPW/C8ynbzLrJqxV8O3+Iw3jqJDItSwpfF cE0QWSgDzr+D3+ZlyQfCEsvrqmwQivP+A8bZb/XDWMoYsLLBgnmSOZO5oMdtOagYWBOG s3LA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aR4pRpym; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g18sor3094079wrw.3.2018.11.27.08.56.26 for (Google Transport Security); Tue, 27 Nov 2018 08:56:26 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aR4pRpym; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zstYfFUM8JCpnxw1Tbdi2+5o2Ofx4Ch1nUlOE/Rv2dg=; b=aR4pRpymE+CP9XTk9XMcEkb4x8W0Sjx8bq24P8tbJj9npfknYJBXYh+ERBr61KUf75 Q/qLV5uDTb/q0AezDQJJHyH796cuqqbtPY5/dAwSDXQ35uaHW0DXZs2NHdyHYnBDdIJA NLwl1lcyglLSnayHUj/Nt/+G5ly+BlUblicnl2Ho9dyrLl1lwN+9IpEvq1qxG1RXo3+z 6QMNRROEeuEkhzU+Pbp5vhT9RM5U4Gr2rgfpLUZh+e5nckbcsgQaWeLJgp6ftpu8lgTR DVR/JZKcQkEp6rOpkks2FA/pOkD3p1lnaiG4hZ+9mKcowCDGVf7hyv7pQo1Gx0RXnlLt MiuA== X-Google-Smtp-Source: AFSGD/Xbrc3BeOjO5BB99nM1cCXcMcyl7v/lfxPXx4Msnm6Carr2MM5IwjmALehkzEUGkVbadaIBnw== X-Received: by 2002:a05:6000:1287:: with SMTP id f7mr29808746wrx.302.1543337785909; Tue, 27 Nov 2018 08:56:25 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:24 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 20/25] kasan, arm64: add brk handler for inline instrumentation Date: Tue, 27 Nov 2018 17:55:38 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit adds a tag-based KASAN specific brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 68 +++++++++++++++++++++++++++++++- include/linux/kasan.h | 3 ++ 3 files changed, 71 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/brk-imm.h b/arch/arm64/include/asm/brk-imm.h index ed693c5bcec0..2945fe6cd863 100644 --- a/arch/arm64/include/asm/brk-imm.h +++ b/arch/arm64/include/asm/brk-imm.h @@ -16,10 +16,12 @@ * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps + * 0x9xx: tag-based KASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 #define BUG_BRK_IMM 0x800 +#define KASAN_BRK_IMM 0x900 #endif diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 5f4d9acb32f5..04bdc53716ef 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -284,10 +285,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +void __arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +{ + __arm64_skip_faulting_instruction(regs, size); /* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -959,7 +964,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) } /* If thread survives, skip over the BUG instruction and continue: */ - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); return DBG_HOOK_HANDLED; } @@ -969,6 +974,58 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_ESR_RECOVER 0x20 +#define KASAN_ESR_WRITE 0x10 +#define KASAN_ESR_SIZE_MASK 0x0f +#define KASAN_ESR_SIZE(esr) (1 << ((esr) & KASAN_ESR_SIZE_MASK)) + +static int kasan_handler(struct pt_regs *regs, unsigned int esr) +{ + bool recover = esr & KASAN_ESR_RECOVER; + bool write = esr & KASAN_ESR_WRITE; + size_t size = KASAN_ESR_SIZE(esr); + u64 addr = regs->regs[0]; + u64 pc = regs->pc; + + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + kasan_report(addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; this is controlled by + * current->kasan_depth). All these accesses are detected by the tool, + * even though the reports for them are not printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KASAN", regs, 0); + + /* If thread survives, skip over the brk instruction and continue: */ + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + return DBG_HOOK_HANDLED; +} + +#define KASAN_ESR_VAL (0xf2000000 | KASAN_BRK_IMM) +#define KASAN_ESR_MASK 0xffffff00 + +static struct break_hook kasan_break_hook = { + .esr_val = KASAN_ESR_VAL, + .esr_mask = KASAN_ESR_MASK, + .fn = kasan_handler, +}; +#endif + /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -976,6 +1033,10 @@ static struct break_hook bug_break_hook = { int __init early_brk64(unsigned long addr, unsigned int esr, struct pt_regs *regs) { +#ifdef CONFIG_KASAN_SW_TAGS + if ((esr & KASAN_ESR_MASK) == KASAN_ESR_VAL) + return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; +#endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; } @@ -983,4 +1044,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr, void __init trap_init(void) { register_break_hook(&bug_break_hook); +#ifdef CONFIG_KASAN_SW_TAGS + register_break_hook(&kasan_break_hook); +#endif } diff --git a/include/linux/kasan.h b/include/linux/kasan.h index a477ce2abdc9..8da7b7a4397a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -173,6 +173,9 @@ void kasan_init_tags(void); void *kasan_reset_tag(const void *addr); +void kasan_report(unsigned long addr, size_t size, + bool is_write, unsigned long ip); + #else /* CONFIG_KASAN_SW_TAGS */ static inline void kasan_init_tags(void) { } From patchwork Tue Nov 27 16:55:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700991 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 93020109C for ; Tue, 27 Nov 2018 16:57:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 806F72C3C7 for ; Tue, 27 Nov 2018 16:57:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7433A2C449; Tue, 27 Nov 2018 16:57:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ACFC32C3C7 for ; Tue, 27 Nov 2018 16:57:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F25546B495F; Tue, 27 Nov 2018 11:56:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ED5886B4961; Tue, 27 Nov 2018 11:56:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4EF66B4962; Tue, 27 Nov 2018 11:56:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 6E76A6B495F for ; Tue, 27 Nov 2018 11:56:30 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id d11so18444557wrw.4 for ; Tue, 27 Nov 2018 08:56:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BhY1iQFDZFZ0u+IiVAFsds0hu0VM1uAnvbg+4E10gFc=; b=U1jIlxFv8ubFOQX2JNya1hl9NBq88GuwYKq9NwDk8A0qTmumdd9gzXmZbIHh56/+gG 4n/qM/SC4wwmObJiprsblwlDCl63VmOqht0L941OdEb/87YSmoasr75BdH2GTBCqCvEP MUcZ2Cg+UoUqH2J7D1jjzhD1i77TVQ8E1k5O1h/N/PUJspEAjxXt78UYljtSaxls5LsP 5sCgmt/eEukzdtseXP0HIZ7jBwm+gUtMq6iQNC627Mn7Ugr/oIFZr+4MnFujq0QLfMos yQczWO77QYhOHp3u2q2DH3y5RyE+EkPU553Yi20zUu3+eLd1BIsdWRk4tpnleAn7GXt/ qdjA== X-Gm-Message-State: AGRZ1gL3FKm2L60z/yF5dDD8i2bZ3ka7qBMEEyb9SxdYt35d1J2P3yuz LjQmyzoM046893qWYJTw64QF5Y1CX2uPKcotqYkkICyingljmeQWSslGFYhADsFrtUuKg0UMB3U LCKVvaFN7/IlgV5esuaJBvWr/wXIj6MdBaJj0ajZ4W+Ya08vkIirzg76Vm7rosvY/AJ1Q/EF1oA GR5sohYM4zpSpoJR1fLpwGBn2ETmgz6daDcf/w81iz+qd5t9g846obI4AEtSfoSUjdrXzbn7E5N eCsJ0Q0Q+hhCHxkVgQQ7Xa4OpZHtaKXzN/2jv3Cjvv4nZ26SsA3jI5wVRSMH9wrbQscUzU893ct 5uaFvboIHN4Gd1nZZfuzLAeGfuFLe8byUwRA1urFUr5OLVVAH6zZ+x77A1JxknB/0PR0Oh1f1KW 4 X-Received: by 2002:a1c:9a4c:: with SMTP id c73mr29558257wme.35.1543337789947; Tue, 27 Nov 2018 08:56:29 -0800 (PST) X-Received: by 2002:a1c:9a4c:: with SMTP id c73mr29558183wme.35.1543337788638; Tue, 27 Nov 2018 08:56:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337788; cv=none; d=google.com; s=arc-20160816; b=xDeCZTrr2QQU/q0Q24VEPClyk/gfPly9fBbb56EV0P948XX3iIXcvYSjaqarxOXfvq 0VUR3KUTjC5q00xtNdOT/sn9tMpXwL64lSS9IEz1bMDIA/u6fF1QQrxo2Oltb9tRu3/i Vtsdn7K1L1hUiO2BzFfkWnSzwQupt0E2lEhqJe58h3xjIWyex5kxIZAIYnbI/8pHjKFE Tkm5MurODkxjeJl3icAcMO/Lorjv+RpZ4h+qDwT/zrebu9MiCOEyK92olzwhyBCrRnio uJexVn15K7dbqR6sX3gmoYGC7KnuUBAsnmJetdbq+jvisFJ8h6VY80Co+Xc9lmzzy3jZ +u8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=BhY1iQFDZFZ0u+IiVAFsds0hu0VM1uAnvbg+4E10gFc=; b=TJtGqHF9mM9nejAZMRwyu/cYhmqYWJJJlZeoSDdJXAfPaaVCpYNoKszLGU6l2cKSrf FR5dvVxuzarU/n02Au5Rv2AF5sL3uddNl1ukA3GjEEwo5z1Ca3FanAOhCzVnJYwoLwBk weY1ZPlu4P+crHg4UDto/MkQBoo5KD0aP9uvHvF/BAJBWBKgtJHbGg7Xgc/c9O2QADd2 DQw4kALV1A4RweXlYHzW/tZDkUXhAvFQJPmpgKkpovWg2lX+nFJUjF5xFXKQa+E9OqBb pARt5Gvs0KPkvvUI+sh8AcVOt0RgbYsuWX+y1KwC4K+sZ37wF9ApWm8GGq9TSSg1IaWJ 7NRw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=FKNwBJWj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id x137sor3411110wmf.24.2018.11.27.08.56.28 for (Google Transport Security); Tue, 27 Nov 2018 08:56:28 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=FKNwBJWj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BhY1iQFDZFZ0u+IiVAFsds0hu0VM1uAnvbg+4E10gFc=; b=FKNwBJWjoGF0o0RluRjfVYjY6gb1EHWHFwlx1GtlFYi3gmsAhj3FvzadOKaUtBgCDl iuUhmCEmC14IZjS3CyH+H03H1+3axzjpLP/UAqtYkoD5+/UZ6sXTjP48fF4u7TCjxQWm KPQPVjZrFwzT97FFJ4wa23N6ej1AHkWdr48wJ2p4GlhIMOiFwA4ar/dziT5l5Gi1RpGz 5iGAAuwTJDLLORwagOFqJ58DjS7ZVLHw5GOSze3YCWO8O5KHG77oUYdUKZMEil/nAVuL NAG8VyMDWbHxuvur9nja2fypLzc7l34wcoA7tRug/JwDuFnBkd91iwYdbCEGztgklcGi o3cw== X-Google-Smtp-Source: AFSGD/WJTtcUbR+qy0BJljY84iI1qGuVqSXNIJRrm7uO0xBr+kj+JN0LMXJ5/M7JOAtEAav+sJH2RQ== X-Received: by 2002:a1c:8d8e:: with SMTP id p136mr2541985wmd.74.1543337787930; Tue, 27 Nov 2018 08:56:27 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:27 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 21/25] kasan, mm, arm64: tag non slab memory allocated via pagealloc Date: Tue, 27 Nov 2018 17:55:39 +0100 Message-Id: <6c1004acf28880f6a5cc7d2f974ba08adb2853ea.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently. For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag. This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 8 +++++++- include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 15 +++++++++++++-- mm/page_alloc.c | 1 + mm/slab.c | 2 +- 7 files changed, 72 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 5fe2353f111b..7db28404609b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -314,7 +314,13 @@ static inline void *phys_to_virt(phys_addr_t x) #define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page)) #define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) -#define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#define page_to_virt(page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = __tag_set(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) + #define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START)) #define _virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \ diff --git a/include/linux/mm.h b/include/linux/mm.h index 5411de93a363..b4d01969e700 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -804,6 +804,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH) #define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH) #define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) /* * Define the bit shifts to access each section. For non-existent @@ -814,6 +815,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0)) #define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0)) #define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0)) /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS @@ -836,6 +838,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) static inline enum zone_type page_zonenum(const struct page *page) @@ -1101,6 +1104,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) } #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_SW_TAGS +static inline u8 page_kasan_tag(const struct page *page) +{ + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; +} + +static inline void page_kasan_tag_reset(struct page *page) +{ + page_kasan_tag_set(page, 0xff); +} +#else +static inline u8 page_kasan_tag(const struct page *page) +{ + return 0xff; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) { } +static inline void page_kasan_tag_reset(struct page *page) { } +#endif + static inline struct zone *page_zone(const struct page *page) { return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)]; diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7ec86bf31ce4..1dda31825ec4 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -82,6 +82,16 @@ #define LAST_CPUPID_WIDTH 0 #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH+LAST_CPUPID_WIDTH+KASAN_TAG_WIDTH \ + > BITS_PER_LONG - NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif + /* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit. diff --git a/mm/cma.c b/mm/cma.c index 4cb76121a3ab..c7b39dd3b4f6 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + size_t i; struct page *page = NULL; int ret = -ENOMEM; @@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, trace_cma_alloc(pfn, page, count, align); + /* + * CMA can allocate multiple page blocks, which results in different + * blocks being marked with different tags. Reset the tags to ignore + * those page blocks. + */ + if (page) { + for (i = 0; i < count; i++) + page_kasan_tag_reset(page + i); + } + if (ret && !no_warn) { pr_err("%s: alloc failed, req-size: %zu pages, ret: %d\n", __func__, count, ret); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 27f0cae336c9..195ca385cf7a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -220,8 +220,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { + u8 tag; + unsigned long i; + if (unlikely(PageHighMem(page))) return; + + tag = random_tag(); + for (i = 0; i < (1 << order); i++) + page_kasan_tag_set(page + i, tag); kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } @@ -319,6 +326,10 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, void kasan_poison_slab(struct page *page) { + unsigned long i; + + for (i = 0; i < (1 << compound_order(page)); i++) + page_kasan_tag_reset(page + i); kasan_poison_shadow(page_address(page), PAGE_SIZE << compound_order(page), KASAN_KMALLOC_REDZONE); @@ -517,7 +528,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (reset_tag(ptr) != page_address(page)) { + if (ptr != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -530,7 +541,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) + if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6847177dc4a1..5b2a1b6ef1be 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1183,6 +1183,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, init_page_count(page); page_mapcount_reset(page); page_cpupid_reset_last(page); + page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL diff --git a/mm/slab.c b/mm/slab.c index d2f827316dfc..d747433ecdbb 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2357,7 +2357,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, void *freelist; void *addr = page_address(page); - page->s_mem = addr + colour_off; + page->s_mem = kasan_reset_tag(addr) + colour_off; page->active = 0; if (OBJFREELIST_SLAB(cachep)) From patchwork Tue Nov 27 16:55:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10700995 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6071B1869 for ; Tue, 27 Nov 2018 16:57:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C9C82C425 for ; Tue, 27 Nov 2018 16:57:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FD722C469; Tue, 27 Nov 2018 16:57:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B84F62C3C7 for ; Tue, 27 Nov 2018 16:57:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 22EF06B4961; Tue, 27 Nov 2018 11:56:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 194096B4963; Tue, 27 Nov 2018 11:56:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F28656B4964; Tue, 27 Nov 2018 11:56:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 93A576B4961 for ; Tue, 27 Nov 2018 11:56:31 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id x3so17700259wru.22 for ; Tue, 27 Nov 2018 08:56:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=N8jYgDbVKt3tw47AlDQNwZCbiy1BMntGMdo2UlIpEQI=; b=NIC+96u2IFZVmW0hlHEYR7YidplYSb1uYX0x0vzSt/wJeyWFZPHtodrVJupfZktZOt M61hzbwjJysmj5H2mUGTIucb+P5CBa7IvXQet27dZxWRO+mG6kUL0tuqEB1X5IUqmT/M DBamFNSdccx8U1DdMOBGgsdIRGEZFDssfm1G1ijva05A8aEPGE8y85/YoC4ElYciYurv HKUC2B7DzetFLV5FdiE87J/qLDhFkp2a/jbaTwFtgOliEkpUYOfu9QeWXCVOx/DxeND9 6faryPXbEPTclDOvZQtXQ36LiHL/ERXz5jeFCS/qER1kcgp+aRtV9QhSybPSGjo63Hrc Wykw== X-Gm-Message-State: AA+aEWYjAuiIIT0PSTG83Y/MvkdsAgf+any2dU3uqoxzHeX1EB112q+W g6saWqPYTo3C0QfZhZ6H95i8PsPjF32EJFu3maNZ8ell3QdFSYl0WiT/hEF5pnDHyaAwk4mICkU z4LJB2C9NXOT91yElniV1RJrka4m5Y+ichV2uDj6/6/SOO8VoBye2Zi8vm+ixXl6GqBsBCXbtJN 0aKhskF9m5Pg8WNHZQ1c5DkJi2qR+0graCB53osqZBzI0b+jb2XXXrUyZJu3IwR9+xdf7tbQW6q ZFh9P1b4Nu+rF4YXLCzX6I0arLO5dkxB7/3Wrji00kxtV+tvR1k2gXsIelyy8oVeT6p3XzchILn Uw4RRyOvL5Dy/Mg2IEPVM8G1FdZMzsbu0fU1C8s2S+9+BkoYpsWXMWBDtxPcSWvZCCtx2dY+Ahe J X-Received: by 2002:a5d:68c3:: with SMTP id p3mr29009594wrw.34.1543337791116; Tue, 27 Nov 2018 08:56:31 -0800 (PST) X-Received: by 2002:a5d:68c3:: with SMTP id p3mr29009544wrw.34.1543337790338; Tue, 27 Nov 2018 08:56:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337790; cv=none; d=google.com; s=arc-20160816; b=tsyKUiOqwXISjIHO7SfeIL41f5vTFZAyuERBRTS3EP+S1HqgnHY2L5cujUvmlCN9N7 ZvGu4mQjpsuWw12gtbRt33A1D7xYNjxfj6aavVYCdGf6SRTLL8w+oudcaVrMnQeKh/H3 ha8tY3Wnv4Y87wlUapMZX8DdtXty7kekM1Syv8qqSGXvODHZgF5Ky1uMqONjRvypgQhb BYLRWhWpTioMaJlOzBqm4kn8pkFGLI3+M+jBd0PNEC4ycW0/l1dV2+wQS5wBtnb6YtAW +OFAodSUFZ2v3U8d2lhafvFlU4MG+CiMYSPUrFU3VFUrCWytcgvywt51hWQ7Y/+8ajlS WoBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=N8jYgDbVKt3tw47AlDQNwZCbiy1BMntGMdo2UlIpEQI=; b=Zg17iL0X4DQJjJrxQzJhgEsfsGnaPQUhUwtg3zKSfKTb9V0++i4QR4rlez9Cjcg1XN lC6VknlJPxSvtXiNk0cYK/og6LU1Jvv1ARtZzcPkRMxW0I0pDm/UaPqeEFO4z8jeL76B Mffg6nmq6w7P3kog9JdtWUvbFav9MmjGusSjAx11VY3ydi8tfe0WiN+PKkrrNy8C+9gZ 8FWqodhP0+ilWSHA2NuH9Of8Ahis6LUgXRteyYf1emz0BSDVTQOYPol0m5oJwdlWymRh 1aN2wfRQYs1c7PXjrFXwZn2pUq0DlH3Nf1Pq6KzpyBituFL3kl+wjD11HvlRQ7WArmDT QhQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=FKQNuSOT; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id g127sor3243969wmf.27.2018.11.27.08.56.30 for (Google Transport Security); Tue, 27 Nov 2018 08:56:30 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=FKQNuSOT; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N8jYgDbVKt3tw47AlDQNwZCbiy1BMntGMdo2UlIpEQI=; b=FKQNuSOTJW4cI4UcMN2RMDAj67rDzAAuvIhRvYzrA5MRBzvI7iBS9pNRw+tGfyAPy5 J4kITg0pv/5kwt/0WqqCW5vwwjIxYhMXJQnr0Zuo7IfhOi/DhNph6xMuFi/J0eB/vWls ZoYGMPImV6SSIHtBxVj2ohaIPo1AwQqp6QzZUsCVsEir5T2Qwp0at7yHW2Ez9ekmn4Cf Jkdh5pCyYqW7lPMgC7l5jv1YgEyaPp1txFdlImHREgqqUnNbKKTp6d9F/SoqhrG6K5R0 o/mtuEHYAsIeNgDX86Sqr81v4KgPr7XHTjxZnrGJJXGtnkq1k43BVPH0a7rPx0Ah/qEL BO9w== X-Google-Smtp-Source: AFSGD/V0pKFbmSN/3YfdNV+APeYj2FMajP/ADMEQwkFKF66VbuRLak9h2rPCP9UGSIvjwojxKkEfyg== X-Received: by 2002:a1c:174f:: with SMTP id 76mr1184907wmx.116.1543337789747; Tue, 27 Nov 2018 08:56:29 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:28 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 22/25] kasan: add __must_check annotations to kasan hooks Date: Tue, 27 Nov 2018 17:55:40 +0100 Message-Id: <6d8c6f59c5b5a3dde569f893ecf3b56e58030ba1.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds __must_check annotations to kasan hooks that return a pointer to make sure that a tagged pointer always gets propagated. Suggested-by: Andrey Ryabinin Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 16 ++++++++++------ mm/kasan/common.c | 15 +++++++++------ 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 8da7b7a4397a..b40ea104dd36 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -49,16 +49,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, - gfp_t flags); -void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +void * __must_check kasan_krealloc(const void *object, size_t new_size, + gfp_t flags); -void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 195ca385cf7a..1144e741feb6 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -373,7 +373,8 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, bool new) #endif } -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) { struct kasan_alloc_meta *alloc_info; @@ -389,7 +390,8 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) return (void *)object; } -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, + gfp_t flags) { return kasan_kmalloc(cache, object, cache->object_size, flags); } @@ -449,8 +451,8 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) +void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { unsigned long redzone_start; unsigned long redzone_end; @@ -482,7 +484,8 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, } EXPORT_SYMBOL(kasan_kmalloc); -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -506,7 +509,7 @@ void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) return (void *)ptr; } -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; From patchwork Tue Nov 27 16:55:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10701001 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A70714E2 for ; Tue, 27 Nov 2018 16:57:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47CB32C3C7 for ; Tue, 27 Nov 2018 16:57:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3B9E52C45A; Tue, 27 Nov 2018 16:57:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D25AF2C3C7 for ; Tue, 27 Nov 2018 16:57:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CDAAC6B4963; Tue, 27 Nov 2018 11:56:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C649B6B4965; Tue, 27 Nov 2018 11:56:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B054F6B4966; Tue, 27 Nov 2018 11:56:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 561656B4963 for ; Tue, 27 Nov 2018 11:56:33 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id t199so17858592wmd.3 for ; Tue, 27 Nov 2018 08:56:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=25C8CIYC9SPU9SBI9QwghyxTcLF9XSvMZTbWdHBvFm8=; b=k5xYNz6ZcDa+a3AzvuL30YZfAOhffbto9GCe9S6kIm94VWQDJ+xzo5/OyryP8W+yoF VVo5zVptSUnjjoPmb8Ge+U/cSAO1mArI9/pG6silanvaktvrgwIMAlVptI+nkxCMS/TD KBayq2Bogb6Sr+aDKFwWi10hlj/jn5vwHfrlYElTpTUa8YInGw7H6ORt6uPBX1N6sc4M 59jNI64JcX10EKJHx5GVgBezx8t3hTW5AC7LLdtc2udjHmkjuyxt8+wob1M/KTBth0La RRd0s63ScZIyh+GsGjFQqmrg89VWXib2xL2l5UovL3nCmGtCgDYTnoA1RV9DfrmA3R62 7cYA== X-Gm-Message-State: AA+aEWY6VSglRtPoJ62q6okIpGdHcQk6nUFYoc+HhGo+SPJ5wnF9+8SA Xhb2kpQM52m8BoAH1fWw2HfraD+jnTSLv0SNKorffZzAx7LFx74aYpfJfxmfNX/YkUzFeJq6UQT qSzknBru+XcSvKetsjucRMY0gQnKl3Cb5uIgkXlZuayVEqkRu9hmcbxnlCyq0nbhA4vsAh9vq6r /udz03j8PfJVXJ+Tlt0pKKWHwBn2kZ0To1I1zoVMPxJD/X4tvFtSyK4sQczVhGzLqYVIrKu9Xyy Acx+btIRYK5tYszKYNzMhq/8+PgyyEyJeK0pT0RN/DIB9Yn1y/Gm5sAeNHHpdAEVeQfoD1kTRF1 qMfEQ2qONRo//x53vvKmm2xzqSivk3JVKjEd6ZxAZ9mrXPLGlFQrvG+18f9Ackmo09/QLsAtHxa e X-Received: by 2002:adf:ae1a:: with SMTP id x26mr16759378wrc.0.1543337792892; Tue, 27 Nov 2018 08:56:32 -0800 (PST) X-Received: by 2002:adf:ae1a:: with SMTP id x26mr16759338wrc.0.1543337792106; Tue, 27 Nov 2018 08:56:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337792; cv=none; d=google.com; s=arc-20160816; b=sWcKxrNl6U5loqDGcowmRYkQ1W8piJmJDindCuunQmu8T8A2ko2Mrd6NyBgFFS0kEr zn5/0CsbJ61VUF5UMTrPrrJYXTXIeyHXv5SGUPioSx39qo9hE5I/3yg5vlEcuxrAH8CE kJjKsjIpX5M7GbE+iu97X0XPWTbY7+Vwea8cY39yNRFHGBCM80/lPVtAQAZhTtjJg5ry BTDKsbTy+KmcVoH+RRglAimX19hWK+pq3gh1gLgJ0+t8ar9/A5vohR4Uk8H2QgDt6KsZ GW3wMn2YuLiZOllULvEBGLpmBDpqoIj6hatUC5HPkvhHAJ3u7daP38qCWZ35gDH1aqt4 T2gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=25C8CIYC9SPU9SBI9QwghyxTcLF9XSvMZTbWdHBvFm8=; b=SHZsttRlsLC6GFKXy88mRcsSRdoNQP06DN+0bqp1WdmfVzivUnyazqpqqSQ0c6KBgt 2cqXvdfgoWMucDBBbK1Nylv25bM+JT+fZhmFcZaQPRUVJUpFYHXhwVEq2uFLW4i+gZfx M51wOcCnlxQwwXX6+lJkb4wO5+XJzn+D/BnB023qykR32BYFDHsem5mVKjW0VcjpPDJD LA9QjScccUbiY01O0RhvkbMSzDDPt7qe0qW/AnhbFFUnEGQF6JLJ7jPaw6oO/Fm4V6Cm LcoAWIO0L1kcmplCPY85S86J6JBflixH0eMiWg0h6GXUyJ/jqu6UnXEpiWOIzjzuYfUX QPOQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=uH0SgLiO; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id q193sor3344614wme.12.2018.11.27.08.56.31 for (Google Transport Security); Tue, 27 Nov 2018 08:56:32 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=uH0SgLiO; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=25C8CIYC9SPU9SBI9QwghyxTcLF9XSvMZTbWdHBvFm8=; b=uH0SgLiO2qJbc33GzR5cf2JRTRQSW8zulwrxK/hoUgGgbafXk6UjDGyoGxOOx3yngy 5c00VcRJCyxcL2Ph7XpOtbaV2bk2uik5gfamA5GwT4W7hoj3qiGR8QgE0pXggaU4jVAx 7G3MfcTqmzJ0xcXdCTdjiFWW4nElcJUYi20y32Xq5iru0Uk0YeMnVelD/8Jq9ijBY5UQ HGIO0a8dSjuTcDRktBQUpifl4dngA+P7svdpob0De+rPlPH2xheaba1MpWjV/2U1hiZJ XmWbZaE5COdjciH4j/7xRajIIKeGdB4dVoZNR6MtEORI/8BGGjlnmzCkU4/tckzR+vEG BC+A== X-Google-Smtp-Source: AJdET5dk/kPxqsbkvgpXQKeInuDe0u2bnU8ufhv8gGWZux10fKqn1JFTOMvJKKz7gPe9OYlyoPUDYA== X-Received: by 2002:a1c:ee86:: with SMTP id j6mr30100231wmi.135.1543337791517; Tue, 27 Nov 2018 08:56:31 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:30 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 23/25] kasan, arm64: select HAVE_ARCH_KASAN_SW_TAGS Date: Tue, 27 Nov 2018 17:55:41 +0100 Message-Id: <996c9b3898bb3c5de977d00215ddc4bf8cf154c1.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now, that all the necessary infrastructure code has been introduced, select HAVE_ARCH_KASAN_SW_TAGS for arm64 to enable software tag-based KASAN mode. Signed-off-by: Andrey Konovalov --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 787d7850e064..8b331dcfb48e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -111,6 +111,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_SW_TAGS if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT From patchwork Tue Nov 27 16:55:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10701009 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83E721869 for ; Tue, 27 Nov 2018 16:57:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6FDDF2C425 for ; Tue, 27 Nov 2018 16:57:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 62C3F2C469; Tue, 27 Nov 2018 16:57:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E3072C425 for ; Tue, 27 Nov 2018 16:57:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F3D46B4965; Tue, 27 Nov 2018 11:56:37 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9A7196B4967; Tue, 27 Nov 2018 11:56:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FA166B4968; Tue, 27 Nov 2018 11:56:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 02BBD6B4965 for ; Tue, 27 Nov 2018 11:56:37 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id j6so18248168wrw.1 for ; Tue, 27 Nov 2018 08:56:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5FiJGuqXSWAwAtwMcnIFO+DIijLU0aw+loRT63wynOA=; b=rpH8PvSQTOKznIjg8ALNIga/dcoEYTHjbSckSJjQ3Dna2I10R39yApiqqZZDP2w6UQ vhziXl2M1sFxNr0BW0Du1D3RDMUHUTnvMNYt0WYIECxoh9sqQPD2x71tbeT28ssfqtip FdEvh15Yirh6I14FIn15skFg6Z4N1E+YSusPWgLNAW3fLHfX66id1+W/fFE+nK0LRKat EaE48XIMd8u08v7XYuy75aK8i/KFhQf9R5SSbNAG9UbzL0VPClg+tzLNx+AWd9u5eEXp wcFcYcirXJl2kqGZwKEKCgXLeCmfy0WlR+/PpMvMCRv43xDJaGHn8Er5++OmKSP8nnFr T1IA== X-Gm-Message-State: AA+aEWbocQ49GwD0VUx8FcCSkEt6d07Afhx658XQespLDPiTXMJOxIDE r2VZ7yevEYmR9eMtM+XKHvJi5IcuNSOUrmtLfwuEdGeSr8qan0xu6PHXlFBia8n+gNtFXFgdx9l iFIsOOtW+YtNBu1cKQh8CVufL0B9H+gxHYnl8dgYVTq9yocvQevoyXCXd0fj4T8z7KSKWD9TbXe 0ohV7BoGs+pejw4hVuj9mArs9B1LVPPxc83fRm1FwvGVvc1Wamm84KrGLbEHhi36Hxz3lweuFg0 3uhALmR0P3Pb8tjigdgsHT7dl9+Bet3vfkeOOsKmdDw+7q9N6x6ebdKdQlEq7VeOYdB8mz8Pi0x /IatV6Pm2nLbBO+5jasb+trK+U22MxBOXnbQLbGd2LnMTBWzkC2GSQsngedMAQx5Dh+UcNGc1k8 + X-Received: by 2002:adf:ae1a:: with SMTP id x26mr16759547wrc.0.1543337796428; Tue, 27 Nov 2018 08:56:36 -0800 (PST) X-Received: by 2002:adf:ae1a:: with SMTP id x26mr16759456wrc.0.1543337794623; Tue, 27 Nov 2018 08:56:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337794; cv=none; d=google.com; s=arc-20160816; b=PmzPxfjbECDM3D2dpvVXMlGsCndA4aTVdfyOMTH6GQR8JDYVz24JUbTRRBUiq1oqb7 aRt2ODm8SgFGF1z7CiwRSXKllN8O/5zeSZI7Zu/MjsIXsbFCxBuFNIrIwRb6kYi7Up5i TLytC5desNxGkaZ2EaK51oPgVwmv4g0v/va8+O8cmZvr5qtedquX/vuXs/22rJYzJ8va aL/lvVq7ij90Bfys8JnOEy3ltKQq0lH6NQQFPDh8cV3MeSP5xETnFDycJTxfb6hToGi6 dfF9PbDLHNXWeXcZ6nSP5bVfr+fMTLMgAiqciHKaElhTILWHlY/nqHfmQFvmCQnk/xwJ BUag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=5FiJGuqXSWAwAtwMcnIFO+DIijLU0aw+loRT63wynOA=; b=wyGZsFvq9CAi9s81S4ysDJjyts359rYVhvAeumRpjq2hoqlhADyh5SHeJr3/1dwPmX tcIdQ/Oe0KujbNNLQPv+rlEF7XwysT8a/eTOTMcBRIJ5oKtUV/+07L8J4v/Xys+Xd0e7 FYcQQq+OAtEkQEEJ+xbsIx8mijIfXA89ZautSKcmjbCB2tTghGwCmeMihkoNkbyNx/Kl j5h3nZJnXEnsuy2r73CLopMtbKbqNyBRt1pnXRrVNaPXKoLnAIsppzlPQnD/foxJpoII M0cUEnwt0mQ53PSXXL5TKYlzh0kYwVEZNMK9URFCnMZNoHqx1VmcjY8hoMlkIs4pUDt7 shjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BaAWRNuZ; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r5sor3142438wrw.12.2018.11.27.08.56.34 for (Google Transport Security); Tue, 27 Nov 2018 08:56:34 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BaAWRNuZ; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5FiJGuqXSWAwAtwMcnIFO+DIijLU0aw+loRT63wynOA=; b=BaAWRNuZbw4ewUUmamyAGbqTlU5bCMsCQ/tXzXZlAqEBRBtz77sU9BLyJmUeRsHr7o 0L9IHHKOJe80SbiE7fSB+lnhXTLQmpT1RVLtH32L71odkGXmx04j3ja/p2mGNrz9M7ZQ xxQNtcdWrZL8nTh/Ssy4q0vbUz31MYmoP+PMvSpQE1gOCb385qbizBSP5AQxe93LT1HW y+nLsJLhEc5+8Vl3B4+h+nkPMIbQa5roAlYl3jAO7uzLHH1gIakC42kSGspF1kCe5Wo5 Gz17fMSTBwBd0mZ4WECDJQctxPn6GnqpWRTdH13iXc9FM1n7xYZMZzfvYlDTkX8CHEpG 8RvA== X-Google-Smtp-Source: AFSGD/VJcuTZNeTY1jGaQY/6d5ClXRzlCflcjPSQa7XG8eRFJ74PZJVbV+72tNiLaCIs/wvGgY+GiQ== X-Received: by 2002:a05:6000:1287:: with SMTP id f7mr29809179wrx.302.1543337793791; Tue, 27 Nov 2018 08:56:33 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:32 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 24/25] kasan: update documentation Date: Tue, 27 Nov 2018 17:55:42 +0100 Message-Id: <1ace22e3a154ce363661bda6328f8c5eb05a091c.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch updates KASAN documentation to reflect the addition of the new tag-based mode. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 232 ++++++++++++++++++------------ 1 file changed, 138 insertions(+), 94 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index aabc8738b3d8..8e956e0588fc 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -4,15 +4,25 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides -a fast and comprehensive solution for finding use-after-free and out-of-bounds -bugs. +KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to +find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN +(similar to userspace ASan) and software tag-based KASAN (similar to userspace +HWASan). -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. -Currently KASAN is supported only for the x86_64 and arm64 architectures. +Generic KASAN is supported in both GCC and Clang. With GCC it requires version +4.9.2 or later for basic support and version 5.0 or later for detection of +out-of-bounds accesses for stack and global variables and for inline +instrumentation mode (see the Usage section). With Clang it requires version +7.0.0 or later and it doesn't support detection of out-of-bounds accesses for +global variables yet. + +Tag-based KASAN is only supported in Clang and requires version 7.0.0 or later. + +Currently generic KASAN is supported for the x86_64, arm64, xtensa and s390 +architectures, and tag-based KASAN is supported only for arm64. Usage ----- @@ -21,12 +31,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC -version 5.0 or later. +and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and +CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN). -KASAN works with both SLUB and SLAB memory allocators. +You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. +Outline and inline are compiler instrumentation types. The former produces +smaller binary while the latter is 1.1 - 2 times faster. + +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line @@ -43,85 +55,85 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access generic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8801f44ec37b by task insmod/2760 + + CPU: 1 PID: 2760 Comm: insmod Not tainted 4.19.0-rc3+ #698 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [] dump_stack+0x46/0x58 - [] print_trailer+0xf8/0x160 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] object_err+0x35/0x40 - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kasan_report_error+0x38a/0x3f0 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kasan_unpoison_shadow+0x14/0x40 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] __asan_store1+0x75/0xb0 - [] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kmalloc_oob_right+0x65/0x75 [test_kasan] - [] init_module+0x9/0x47 [test_kasan] - [] do_one_initcall+0x99/0x200 - [] ? __vunmap+0xec/0x160 - [] load_module+0x2cb3/0x3b20 - [] ? m_show+0x240/0x240 - [] SyS_finit_module+0x76/0x80 - [] system_call_fastpath+0x12/0x17 + dump_stack+0x94/0xd8 + print_address_description+0x73/0x280 + kasan_report+0x144/0x187 + __asan_report_store1_noabort+0x17/0x20 + kmalloc_oob_right+0xa8/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + RIP: 0033:0x7f96443109da + RSP: 002b:00007ffcf0b51b08 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055dc3ee521a0 RCX: 00007f96443109da + RDX: 00007f96445cff88 RSI: 0000000000057a50 RDI: 00007f9644992000 + RBP: 000055dc3ee510b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007f964430cd0a R11: 0000000000000202 R12: 00007f96445cff88 + R13: 000055dc3ee51090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2760: + save_stack+0x43/0xd0 + kasan_kmalloc+0xa7/0xd0 + kmem_cache_alloc_trace+0xe1/0x1b0 + kmalloc_oob_right+0x56/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + + Freed by task 815: + save_stack+0x43/0xd0 + __kasan_slab_free+0x135/0x190 + kasan_slab_free+0xe/0x10 + kfree+0x93/0x1a0 + umh_complete+0x6a/0xa0 + call_usermodehelper_exec_async+0x4c3/0x640 + ret_from_fork+0x35/0x40 + + The buggy address belongs to the object at ffff8801f44ec300 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8801f44ec300, ffff8801f44ec380) + The buggy address belongs to the page: + page:ffffea0007d13b00 count:1 mapcount:0 mapping:ffff8801f7001640 index:0x0 + flags: 0x200000000000100(slab) + raw: 0200000000000100 ffffea0007d11dc0 0000001a0000001a ffff8801f7001640 + raw: 0000000000000000 0000000080150015 00000001ffffffff 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc + >ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 + ^ + ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works. @@ -138,18 +150,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For tag-based KASAN this last report section shows the memory tags around the +accessed address (see Implementation details section). + Implementation details ---------------------- +Generic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address:: @@ -162,12 +180,38 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +Software tag-based KASAN +~~~~~~~~~~~~~~~~~~~~~~~~ + +Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to +store a pointer tag in the top byte of kernel pointers. Like generic KASAN it +uses shadow memory to store memory tags associated with each 16-byte memory +cell (therefore it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation tag-based KASAN generates a random tag, tags the +allocated memory with this tag, and embeds this tag into the returned pointer. +Software tag-based KASAN uses compile-time instrumentation to insert checks +before each memory access. These checks make sure that tag of the memory that +is being accessed is equal to tag of the pointer that is used to access this +memory. In case of a tag mismatch tag-based KASAN prints a bug report. + +Software tag-based KASAN also has two instrumentation modes (outline, that +emits callbacks to check memory accesses; and inline, that performs the shadow +memory checks inline). With outline instrumentation mode, a bug report is +simply printed from the function that performs the access check. With inline +instrumentation a brk instruction is emitted by the compiler, and a dedicated +brk handler is used to print bug reports. + +A potential expansion of this mode is a hardware tag-based mode, which would +use hardware memory tagging support instead of compiler instrumentation and +manual shadow memory manipulation. From patchwork Tue Nov 27 16:55:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10701015 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A109214E2 for ; Tue, 27 Nov 2018 16:57:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88F4A2C449 for ; Tue, 27 Nov 2018 16:57:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B7802C45A; Tue, 27 Nov 2018 16:57:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D7E42C449 for ; Tue, 27 Nov 2018 16:57:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38DE56B4967; Tue, 27 Nov 2018 11:56:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 316B36B4968; Tue, 27 Nov 2018 11:56:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 190A26B496A; Tue, 27 Nov 2018 11:56:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id A7F936B4968 for ; Tue, 27 Nov 2018 11:56:37 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id b186so16863486wmc.8 for ; Tue, 27 Nov 2018 08:56:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XqHRGLTjehAPPlhG5MrM8/SiVdROoXpoESjJ74s2A4U=; b=rD1yGTMdbGysFHnOc67gr8CCwgLuMZNli+EaQrVJgnHkkNmVGmq9KbJ+YqUgdDaNtq X+3/GT8etrE23mf3txJe7S0o2Fczx2PhmzS4tVepADAqTJOVYq2fletjw4kJX+4suyz7 TC5DbgX7jTkkUDJo0jiblRSjZfWiQnYxrfx564HyJ0ZgBSWymNhKOoyp83tIxyjD5CR8 bxWAi2Qru9csJkAomiOEhW2CY30OpE2aNTdBZdpn+269oKskonwNo0f5G9Of2WcO6peg 1o6+U6eQyYQAMSn/WeFN/wKILTkSFqC5cdS0PP9nlZI7hyqH7+egdQWz7WkOPykjfXXk nReA== X-Gm-Message-State: AA+aEWapn1yvwtZi1AWHEtZtRcHR7ALltiEZDYIHKQnIFqh2Ve5utBLS OEX30MUbuGQ/7jcTr/FxdGbrN5pL8xJuaqUdGVJvGVOtharzqQLGu1gwXbGs1G8XhIBWdDGoQSQ L4Ag83f3FcdZEaanafcnDp2+x6A6btX9mi2mdjgNVHR2OlUaFzTuLowLN0fCYiNF9U/lHxWA/gD YT8W0EcJ7JZYLYTrNlNlaipQp32vp2qt1GV7nZaeRC54XIsbP3jYxc5eqZcmWTF2PGe6gffxM9+ cWtpwkMwIM5kTSxckh6BuvYGW37481Scz+xpF87JYIOCYOwYQZWxLiilOj87ljuNkaeJVq0lmBe JD7Ig++X6+Y1fFFona1DKKw2v5grdWj3fBUtmKRvz0+fspTc2lZrayq4lgtGZHiy8XvDKu61vqF L X-Received: by 2002:adf:f091:: with SMTP id n17mr28205151wro.292.1543337797237; Tue, 27 Nov 2018 08:56:37 -0800 (PST) X-Received: by 2002:adf:f091:: with SMTP id n17mr28205095wro.292.1543337796340; Tue, 27 Nov 2018 08:56:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543337796; cv=none; d=google.com; s=arc-20160816; b=wSYagakuOsfoq3dwSwSKQBKqe+CJMF3alW+4RQqsxm8FXUeGWadaZEVy4Zr/PYPwEL ovFjYptg15n+rndOldfhIe5OqEcU5XFKjHtEtkp+hqW3kc5IMoBjvQZqqPJcpm2ImA0B 8ZR9g+pAwqRRk6iNAZEkt/MbPGj0S04uJNWYNobhOTX2qlhFjD5hfprzJCM0uCvzaogD SIoxsoNwHifVm7Sxw8yx+FHTu2BUEF+/2BUsiYbhiajNL/J9isk+ThrW73WolcR+9Ip7 4Jl7Y4aiJ2+b3ZOWzeQ9ZqZDHMYVZwqFW5+sHVR/7Hvv/FdV7vmxAw0QMaSTWI26Nhbx hhGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=XqHRGLTjehAPPlhG5MrM8/SiVdROoXpoESjJ74s2A4U=; b=mY8DHzNm3W2s6XW6w5pXs6a3q8tt/Nf+zBsWs1tiW1zM2FlcleXwdl2pw9Vtca5VeD Vb/gPaFzTgH2qifR5CoL/loePAhZN7iOVhx1J/5J3kKJ2xBQWx8+IA+HbUfrd+WgP/mJ 4oIAgMhl3eTnWs+6xz1nJ8Br0NIbfEFazBK1WLFCbb64ZWhpli3nFqqGYTFxIQY3eXNo dTgXIW8ji6GxnJLbsl5ACrYDt4aSTVnOV7ARYYUaMD4RvBGdxXGiuhXYgTdDuepewSRz 9ZwZcd5duRLLtINXa+6OlY1yrZ35TWJtf0VyyGnzgGYp9OTIULAD/DS2xjileoS5JUaE C9bw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=AXmQ7q7R; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id x9-v6sor3431243wmh.17.2018.11.27.08.56.36 for (Google Transport Security); Tue, 27 Nov 2018 08:56:36 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=AXmQ7q7R; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XqHRGLTjehAPPlhG5MrM8/SiVdROoXpoESjJ74s2A4U=; b=AXmQ7q7R1HXMKgMVeS27Rj4kiDof/yHUPibkT4fIvoAy/nTR7z7f1eT6vpHnsqcvJW MLWi1ZV9OiVSrzfsGHMAKHmYej7Z6vktknz/E5w3Ba58xGUPhJ0IwsoCNVU2TRMQHe8b KxQSdeIyh0MgVfcqUfns/A97+FybMDUqlhE0HUEYQgW+BId0WXHFuMwt56GYob+/ow72 z8YftlzfcnQD/leNdS+++KZsR0USvXU9aVG903NwP9hEZrwuWSZ97/yC4jgIUGQ3fO/a hcOQavZjNhwSveJOH46IRpXb6aJukStVP8XYVvIon2ejxulynl4IHoDlgh/h1VpFrPtZ 23PQ== X-Google-Smtp-Source: AJdET5ecLjjQdWCfN0/+r/icJ0YYKjaCZ80FZLUq4u6JkOTfiBS3pz5PsaOfYZOxHXiyBNNuZUKO+w== X-Received: by 2002:a1c:8d46:: with SMTP id p67mr26740300wmd.28.1543337795769; Tue, 27 Nov 2018 08:56:35 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id k73sm6383099wmd.36.2018.11.27.08.56.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Nov 2018 08:56:34 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v12 25/25] kasan: add SPDX-License-Identifier mark to source files Date: Tue, 27 Nov 2018 17:55:43 +0100 Message-Id: <8e26a568b12ea02e11c35b681f3c36aff2fc1d77.1543337629.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc0.387.gc7a69e6b6c-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 1 + mm/kasan/generic.c | 1 + mm/kasan/generic_report.c | 1 + mm/kasan/init.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + mm/kasan/tags.c | 1 + mm/kasan/tags_report.c | 1 + 8 files changed, 8 insertions(+) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 1144e741feb6..03d5d1374ca7 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN code. * diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index b8de6d33c55c..ccb6207276e3 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core generic KASAN code. * diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index a4604cceae59..5e12035888f2 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains generic KASAN specific error reporting code. * diff --git a/mm/kasan/init.c b/mm/kasan/init.c index 2b21d3717d62..34afad56497b 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. * diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index b209dbaefde8..57334ef2d7ef 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. * diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 214d85035f99..ca9418fe9232 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN error reporting code. * diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 1d1b79350e28..0777649e07c4 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core tag-based KASAN code. * diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 573c51d20d09..8eaf5f722271 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains tag-based KASAN specific error reporting code. *