From patchwork Thu Dec 6 12:24:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715733 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1156A109C for ; Thu, 6 Dec 2018 12:24:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F29E32CED0 for ; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E651C2D325; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 946302CED0 for ; Thu, 6 Dec 2018 12:24:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD2FF6B79ED; Thu, 6 Dec 2018 07:24:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B84DE6B79EE; Thu, 6 Dec 2018 07:24:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 989E16B79EF; Thu, 6 Dec 2018 07:24:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 27EFD6B79ED for ; Thu, 6 Dec 2018 07:24:51 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id h11so90076wrs.2 for ; Thu, 06 Dec 2018 04:24:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=u7kkCgM2bR7dpslIwzvVoZC2X1ySw6gfdlixt56XeAA=; b=WgF+aNCi1pS9ukYYld4g747OMsbq9m6H/Dirbz/sLopZ09I0qdTtvgQ/VgJQQ2C5Rf afJGV9CAgXCEiUAZiknerOHmamOT0KxEXrwJ7Ej5DS2zMgpuOJQzBZB0MZDu5VoHaP6Y Bmxd8f0+jbFnuenRe2mgmLla8JsJBpysHz3P/5+Zl33fM8okvqK9FOWWnfyNf0UpGTmj fA+CirzKh7c29MUaRtelzBRl4DHxJue2tEM6vwGEuBgDBGw4+NOKsXcw/9rEXU6LNFHj mTK6nOTmv/I68wgX47ZyofLicPKdzlTwmb1QoV1ggIR6eTsoeRDqcikI0ib3f7m3+4vc 2ogw== X-Gm-Message-State: AA+aEWYiwmmEZWjqC+YVf3OQd8hiHfVO3GnOC6/zODeNPVh+vMjdWb08 8CsyF3kWQRhIe1khbsj51jWGoubOFQJtE/ptqQ7723avIoYg2+kPRTmm+3vQ3wRdxFnjMDtPv/u YnZpwYePMYvs8pPMgSRmSFrzwwZJ1BsKZSKXNEsA51n9wTFH32XzX7DWp+HxKJgQAL4yOCrFkH2 cEE4eOebbERI//zOyK1JdBqRx+zFF2pERt4E+MDx8ZAZwKnT4ZCR8asjb4pwCtbMbQOGM9yg7hx BCeq6xgYYpM6+NSGpbXg9G3uqzhpxm06oWLGXr7NCqklrCkF2W3s2/K5myyTeZBS4qCYrQ9Cupp vuxf8cG8vUHLLLX/2j2GLHg47mHuYjBMlOIl+xsRx1x8SUjeJYn8TsjICYUdftJNHpYGyvXNE1h V X-Received: by 2002:a1c:c008:: with SMTP id q8mr17619738wmf.99.1544099090563; Thu, 06 Dec 2018 04:24:50 -0800 (PST) X-Received: by 2002:a1c:c008:: with SMTP id q8mr17619667wmf.99.1544099089312; Thu, 06 Dec 2018 04:24:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099089; cv=none; d=google.com; s=arc-20160816; b=nPUbvJkfjBsQY61VXiBrdoZ/0VWWV1/7qEDkpP/DoanPsNEMxOvOlzhtlSn0s1Q/LI JaON5v3TmmK2XWjdH7Obcwts+ZuPyFxo/xCDg1etPnZQzT64znSowETjOlGd60IlC/sZ SiPklTEzytQSPFC8Mlr7V3eK2wn1Mjff9+0fajGjYUdgEksHVvqhEIEJwPPT78F/lB+N o+r2ZU/h7lWZEg8Qgr0kvgKZdlc+dQG9VkPpTEigVDnAXDUbeheb3Lw8oA3aE1LQabUT nuKbOfjvt5puUFrqw49n+qA1h3FnDEtiQrYHr8557ZyGMtnZZR7lhc+w5ujRt/cUjsYb UJqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=u7kkCgM2bR7dpslIwzvVoZC2X1ySw6gfdlixt56XeAA=; b=HQZuD8cdjamUaLi/282L0QuObn1qIWpCx54HsoBQlimSjaPPBZ2344qRNS0dSy0ACy f3T0voPX4Ae8DCn3NWAjvVf/0rscOOQHSTYDahy77V81TVJWLn/VbyWUyqfq4Yca5Xyu gIkxta6HJvHRBdSeYNSuUgVjysmpt6rRXR7PNK7P/6l5z5nf/PbnKJKDs2hLmfQIngNa S4emsch/TamhVqL5Jni20Txgh/tP67Aiw7CKnPVAnxmfNNtDRJqSqCZ57YCpBP0sH9wR f+KTmoRGMMsMz8z1PjyhRU9HQXNWITwfZmp9U2s4NcTs8hYSh23VpjUcNC94HBHJKcKH aKIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Enf16wiL; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u1sor191570wrr.11.2018.12.06.04.24.49 for (Google Transport Security); Thu, 06 Dec 2018 04:24:49 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Enf16wiL; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u7kkCgM2bR7dpslIwzvVoZC2X1ySw6gfdlixt56XeAA=; b=Enf16wiLZ5gbJcXeaYiv9T1lvd3QlfqZdXV2wcaMmxRbE8VKhpm7vz5gMTdXUDRxqr KCtqw72+zySmDtyFsn+cC0pk5UjavJ97GSAQYQevUz1cTUEbhiEhnEJ4uktbWx0/Kxig x3KFPWluyH+Ov5kOAYDbwp+fQJ1zrdEN/ZxFJWo9/Jjps6twi0JjR19lI1+gn9wxnjd3 /YnUmARwPIqmdyaYp/EYE7RKjSf0iCP5RYloFX9D6CfyhKp6CfJQYJ4RjyaaQbICrJuK QjYAb2YFRySjHVkaiC9dQqebDjzP+V+aX6TQd5aoK5ZzvW6JGQT+z3KnKLahJ2Pq8ky6 cN0A== X-Google-Smtp-Source: AFSGD/UpRIgk9h+y1gmV55lfHXBpLAxVo71KoMj2eiGthQAxJl9eXORdHev95etTM9WXfeqMgeXWhQ== X-Received: by 2002:adf:b243:: with SMTP id y3mr10682390wra.184.1544099088593; Thu, 06 Dec 2018 04:24:48 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:47 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 01/25] kasan, mm: change hooks signatures Date: Thu, 6 Dec 2018 13:24:19 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN changes the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 43 +++++++++++++++++++++++++++++-------------- include/linux/slab.h | 4 ++-- mm/kasan/kasan.c | 30 ++++++++++++++++++------------ mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 7 files changed, 65 insertions(+), 45 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 46aae129917c..52c86a568a4e 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -51,16 +51,16 @@ void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -105,19 +105,34 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) {} -static inline void kasan_init_slab_obj(struct kmem_cache *cache, - const void *object) {} +static inline void *kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + return (void *)object; +} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/include/linux/slab.h b/include/linux/slab.h index 918f374e7156..351ac48dabc4 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -444,7 +444,7 @@ static __always_inline void *kmem_cache_alloc_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc(s, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -455,7 +455,7 @@ kmem_cache_alloc_node_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc_node(s, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } #endif /* CONFIG_TRACING */ diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..55deff17a4d9 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -474,20 +474,22 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, return (void *)object + cache->kasan_info.free_meta_offset; } -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; if (!(cache->flags & SLAB_KASAN)) - return; + return (void *)object; alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +530,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +540,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +553,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +568,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +578,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index 2a5654bb3b3f..26f60a22e5e0 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3681,7 +3681,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3719,7 +3719,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 58c6c1c2a78e..4190c24ef0e9 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7eb8dc136c1c..5f3504e26d4c 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1204,7 +1204,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1482,7 +1482,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index e3629cd7aff1..fdd4a86aa882 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1372,10 +1372,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2768,7 +2768,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2796,7 +2796,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3784,7 +3784,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3801,8 +3801,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3829,7 +3828,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } From patchwork Thu Dec 6 12:24:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715731 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B14531731 for ; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A08542D17C for ; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93A242D325; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F9612D17C for ; Thu, 6 Dec 2018 12:24:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA8B16B79EF; Thu, 6 Dec 2018 07:24:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CE18D6B79F0; Thu, 6 Dec 2018 07:24:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA9396B79F1; Thu, 6 Dec 2018 07:24:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 5ADF96B79EF for ; Thu, 6 Dec 2018 07:24:52 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id y74so8559364wmc.0 for ; Thu, 06 Dec 2018 04:24:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QE+Q4nZZeBuS5Au8e+/piROKf4cgt6sP42L8SlWJEXM=; b=RtqzrS7grH/GF9dX8SI2rtiekkhDZ+4WRCQi/vvYxPXUqh12i+8xpbY2kzoYNCZA4Y srGXfORSG7B7efSte6HQCRmZr5vdb+Ff0aCl0ZLkHEFS+vfVT/7FaIB3GdzxKPmAGQfe 2VcMSYlYqFTzqzU9unj8Ppqr0augw9LwNE8lsitLC7JhZ3GDhMgPjgAxaTcUD4SsQIhJ Mk3F96ap+xdk7+4c80UDQoNPdT1AjR8+hdvDG/0N/ceofm0bO2jEXlBQ3HpBRa/v8VSh 1IACaoVY3ZhLiJbgUByTOgGST6gg4J1sOTFgpv2eBnvmgXhkPoH/7WpNrwp/ZLdloviG EYAQ== X-Gm-Message-State: AA+aEWZei1/vJesHzim1evJDcpNRCVJT7A8gdcqLqUNXDIgcuRxGH5Tz chk7td7kHJhnm5qzBV2QEcHJzjtgWVX7wGZt6wC2V/Ej5xW9UmyrdRcYPIRcvFi8D1xWXTRW3VX 20zVXM7fcUxgpUN20kdrFIqQ+IE3/SbtWg7ljNiutr9xUm7OFhri4fgNN+LbbA6AfDEZKa6yksK SOglUeonSSdOHJOrR3dckyZBvuhkS5NlplfYFep0VVkvU5CBnHkxZ3+6xYCgLETaw0FXFZtNGdK pSJkbK2rwJ+R0+PgEobzy7atnfyMZdTANvPJQrzKn6HhInCB1B+ACG9TQq0UBms60TwAunnP9Cx nSNtmaSEg5Yxsy5Xk42wh9NldbFiAgVqCLMbwAppNkvbGYiK6wX5uKPyI7ZvLHfXy9TJbijnULM 1 X-Received: by 2002:adf:9521:: with SMTP id 30mr23779125wrs.192.1544099091717; Thu, 06 Dec 2018 04:24:51 -0800 (PST) X-Received: by 2002:adf:9521:: with SMTP id 30mr23779091wrs.192.1544099090904; Thu, 06 Dec 2018 04:24:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099090; cv=none; d=google.com; s=arc-20160816; b=cPgKkg+69Muep8Ft00gRanrIgPSA4BKuNBKq3hsN/H4jfBNDRwpbb8fZ05/jIGtw5U 7YXTDWIx8ABch67eDYFTlSenuiH1BZUqbHzOOVay0sy7wQg3FnvXzW2GK05EWyZwgZdJ bVxBSLuym0l/HK/x8PkluhW3uL6QO0nsJmotr+nGpMkBXeOP3xLw0PV03/yGeY4OCVLV fBssnw5iz0yL6LHn23DbSY2JMW90z2ltDICv2tJXHdHHvfbudl0QwsfW17qbidOl522l ckr66v+YCp513PrqtylE+K0CrEM0I4Uz12NErMCKm39TfNJRrvppis6U9K15VQGCAT9m caog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=QE+Q4nZZeBuS5Au8e+/piROKf4cgt6sP42L8SlWJEXM=; b=sW8+rFohrbXJoh7Mxn6A+O/3GeQdh9C0FlIpO2VSrma1kC1Clzqo0fFHK/8VONTh8c 8SX7jFTEPnJrnzlBZiIPWKgP4TfVU5wyk3SGO9ydGTCajOCe4cIMiOvMQN7seXzQEhQw jLlVo3K5NMnh1qUw6Xh0sSQDG6EmI+a0uKBHDxgorzwdrWgYsaMo/uT6iX0Pe0BSWhQs L020q/b7OPNK04SZ9nQgydPMSPmPGK0PQLz+b+jKQJpQCFhJXOM94fxoBnCMI3uHjYj8 vO7iN47Tg7+PhdwO1ALaJbiREhMcMCAqIo5U6CuQGtJzLbPM6QjGrUMAuJC3XY7nj4A2 /xRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=wVfJoDn2; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y15sor177318wrd.35.2018.12.06.04.24.50 for (Google Transport Security); Thu, 06 Dec 2018 04:24:50 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=wVfJoDn2; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QE+Q4nZZeBuS5Au8e+/piROKf4cgt6sP42L8SlWJEXM=; b=wVfJoDn2JNGToLJ7T1uR10yBYoDyUvZHDPfnc33migagJxNgK+X3SnnMPkKaXVWcF5 v6DU0pyjTpUKWwu0GEPHwrHDIrKhpC24bXdBcz+YJcAfaB/02+ok4ph+SNNgl1+MJDM4 2PaoZbYfH88/GrmDHUvLd2wPovhStnou6VwHZqIxFuXbq1dEWnuoMs68mCqn/3V8dT/m r/FsTY+w6qZiGf1nry8kOTCuRubMXKayBenfsABcza7lF8t6/ApNTC0pVXwtJVEzfn/w 2gGTTyhI6k4TX6dIeS1j4PSmYrg9rFHuvrEsBSU5NzDgCEnyQyIViFt+X8uGtgGXYdIf dcIw== X-Google-Smtp-Source: AFSGD/UAxL1TxZ6RYc2RrvhCYipZjB8cIoKC8RPAEZ0NrTXPhYFAwqyGlH1xcwr5wkyHaLQL8hqwWw== X-Received: by 2002:adf:f091:: with SMTP id n17mr9677058wro.292.1544099090436; Thu, 06 Dec 2018 04:24:50 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:49 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 02/25] kasan, slub: handle pointer tags in early_kmem_cache_node_alloc Date: Thu, 6 Dec 2018 13:24:20 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The previous patch updated KASAN hooks signatures and their usage in SLAB and SLUB code, except for the early_kmem_cache_node_alloc function. This patch handles that function separately, as it requires to reorder some of the initialization code to correctly propagate a tagged pointer in case a tag is assigned by kasan_kmalloc. Signed-off-by: Andrey Konovalov --- mm/slub.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index fdd4a86aa882..8561a32910dd 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3364,16 +3364,16 @@ static void early_kmem_cache_node_alloc(int node) n = page->freelist; BUG_ON(!n); - page->freelist = get_freepointer(kmem_cache_node, n); - page->inuse = 1; - page->frozen = 0; - kmem_cache_node->node[node] = n; #ifdef CONFIG_SLUB_DEBUG init_object(kmem_cache_node, n, SLUB_RED_ACTIVE); init_tracking(kmem_cache_node, n); #endif - kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node), + n = kasan_kmalloc(kmem_cache_node, n, sizeof(struct kmem_cache_node), GFP_KERNEL); + page->freelist = get_freepointer(kmem_cache_node, n); + page->inuse = 1; + page->frozen = 0; + kmem_cache_node->node[node] = n; init_kmem_cache_node(n); inc_slabs_node(kmem_cache_node, node, page->objects); From patchwork Thu Dec 6 12:24:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C90491731 for ; Thu, 6 Dec 2018 12:25:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8822292C8 for ; Thu, 6 Dec 2018 12:25:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AABE02D93C; Thu, 6 Dec 2018 12:25:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77EEF292C8 for ; Thu, 6 Dec 2018 12:25:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3950A6B79F4; Thu, 6 Dec 2018 07:24:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 345036B79F6; Thu, 6 Dec 2018 07:24:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 196B16B79F5; Thu, 6 Dec 2018 07:24:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 8ECBB6B79F4 for ; Thu, 6 Dec 2018 07:24:56 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id l1so89747wrn.3 for ; Thu, 06 Dec 2018 04:24:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=z6elTJ8OUfMY93x3uy3CoVTdYQtrheT4VSO4MmlPCAo=; b=nQd0g6EbCR/S2uyCDEJyKdfJ8DbJdNHSJrbMxecv/1TgxwEijDEAuUMspYKjqjsAqV J+OVdxH+Ik29x6zdp0aYpCH4f/0kUhm4bOdo8zGVKS6NsxPPj7njbTP9HSxrk0TnZUMo jyIWLmwX0N3Qs3my2W3J0E86Fep5AUCc66lMlEPLVT3yJWGYq0zadGMev1dZuGBZiWvI E2MOkKdUPosIJMqsAl/h1WxhIGh+4ZJdn44/KRgOAOHrcv4SdKHdbzYpmIJA86gsad95 tESYEiTrvzk+aayoI6Ot50WNNo9SImats/EqMEAwPUuc8iFp+3oDgmJzpuDlzpLhm49s nyMg== X-Gm-Message-State: AA+aEWagCOME0Ho9sKKhaervpAW3xsh6v8WMtTpMAqo9CyY5taXL5bES 3NFPzJWybqf5omuaeH2232DXxWC9SBdnIgc4WiS1yXJYcsvJyuai9IgpAxLdcfL8Tuu4QbLGyqm mptX/0cLn1RUYtWR0SyyKdwqbvBBJq4o9/VSvOhbORS/LsNpYyyd/DzY3S+dG5azWYlk6zJFyJC zXtOXi+4mKYXr50tzdKuS1PSQqV+GwfuqeOFZHV709+DPWp9jh8ZLhfq2aK1QUQdITAjPBG0G+d nTLeKcoEAbuPkDhL2pzylWWswj8N5q8VK25UR4KK1dkiPlIewscMrcKJ6/tibxKhB2uP++1V4NC ebn0LV7Efz9HVvSKiXJp6gFJpF8OBjxbYkqgOSTKGGfWHyrHYFMh3Gql6Lsvk8WgbWf/nrjs5xk v X-Received: by 2002:a1c:f8f:: with SMTP id 137mr18971025wmp.96.1544099095943; Thu, 06 Dec 2018 04:24:55 -0800 (PST) X-Received: by 2002:a1c:f8f:: with SMTP id 137mr18970859wmp.96.1544099093232; Thu, 06 Dec 2018 04:24:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099093; cv=none; d=google.com; s=arc-20160816; b=LSBdvGmOyyuaBKsBx3YiogVXkLFJxVayXnrzjAnusFxSjya1AvNSkiiOgg3YJ68P8Y 3jvgut5KZgt/6g9rKyUFqD4zsRV59zquOg0QiDq7FAth7cKGZTsZ2Ssp9a/xtKEzkarJ kgBExoRqA4HMrlvqlp+oonCKs8zMoV2Jv7fu8oZ8XEfJlgJPICKdplq7FC9uZCTnhmL8 cOc2d64TzLrBP3xXu7/Av6y7/CdMnQSUXLfMWZ+pdxl/zNSUdKul/6PwoXPYD0bmSjqb 6kZtLL6ye1mx7MOsRjiNbl5/4a7ZTw9vXGYC03qa63TPHdzVx9jfkG/hqdhCdIAvv+zW km5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=z6elTJ8OUfMY93x3uy3CoVTdYQtrheT4VSO4MmlPCAo=; b=tWW2rJttEnR5WaZ51JLJW8c5MGBvkGPwE/IO7km51TnnRkjbRt+jThFPMLTOW/g+Yp k5pUwVqCxxDxlw+bh23YbENVFaYRZQp/5AKydGC3xQ80zhjKcD6z21qIfJ4dEf1qdUU1 TI5d68cojs9Z1swXxH9+DWGQH/6Z8qdkk6pHZQsfDkJCBtsJnyeGiK7jiDqlUIpHudlG JWDafsl71RWjBab35HGmU9mbRPYvGj8rQs99simzOONzoy5EMqEfUpXytikTvZPfEfxs +gPSgWbbtdHt/hu2ezCDl1r4TGZ8CCW9FvCtUUi35oSJr2T6n74mmK5gclsLhhuLcpnn kbLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=KDfVTsls; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n12sor196723wrm.10.2018.12.06.04.24.53 for (Google Transport Security); Thu, 06 Dec 2018 04:24:53 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=KDfVTsls; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z6elTJ8OUfMY93x3uy3CoVTdYQtrheT4VSO4MmlPCAo=; b=KDfVTslsk9bpmSWjPxQHh8mgZXc8IoRoiV8hvJ0wiO82SvhccF9gQS9A6rRws8LQg9 Lja4Y4/nZRADgsJ+clV5Sls8/V8zICOWs+WSYjgMeowAHLpcjN3KggYUI7QF30rCqL0h Mz64e1JVFGMZpSL0j+h76juYRyF5wn7jRA23FFNG3++4zF6db29E5IWLl8hUKFDBLwDV vqKagdwiru9P6YcT35dw7x0VK3Sfl7kDzHn/ul/jT3wmWc/fPFBp5XHaiTABrP9WLGHd Z7kaRbvsNVHAx97jg+fdfyBr0n9wggSrkRwebj/g86A0UR/S9WPnAYcSgquqvr07sqrA hZ+g== X-Google-Smtp-Source: AFSGD/VRQ/sHc+Au3eQZ3e0i2PYrSMxan116Yu1xqL1XIcwwcKLbmH0g1gecn/AdOHuVUFZ5+u6J7w== X-Received: by 2002:adf:e5c4:: with SMTP id a4mr1950283wrn.157.1544099092155; Thu, 06 Dec 2018 04:24:52 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:51 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 03/25] kasan: move common generic and tag-based code to common.c Date: Thu, 6 Dec 2018 13:24:21 +0100 Message-Id: <114064d002356e03bb8cc91f7835e20dc61b51d9.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN reuses a significant part of the generic KASAN code, so move the common parts to common.c without any functional changes. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 603 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 570 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 614 insertions(+), 569 deletions(-) create mode 100644 mm/kasan/common.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 3289db38bc87..a6df14bffb6b 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o diff --git a/mm/kasan/common.c b/mm/kasan/common.c new file mode 100644 index 000000000000..5f68c93734ba --- /dev/null +++ b/mm/kasan/common.c @@ -0,0 +1,603 @@ +/* + * This file contains common generic and tag-based KASAN code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +static inline void filter_irq_stacks(struct stack_trace *trace) +{ + int i; + + if (!trace->nr_entries) + return; + for (i = 0; i < trace->nr_entries; i++) + if (in_irqentry_text(trace->entries[i])) { + /* Include the irqentry function into the stack. */ + trace->nr_entries = i + 1; + break; + } +} + +static inline depot_stack_handle_t save_stack(gfp_t flags) +{ + unsigned long entries[KASAN_STACK_DEPTH]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = KASAN_STACK_DEPTH, + .skip = 0 + }; + + save_stack_trace(&trace); + filter_irq_stacks(&trace); + if (trace.nr_entries != 0 && + trace.entries[trace.nr_entries-1] == ULONG_MAX) + trace.nr_entries--; + + return depot_save_stack(&trace, flags); +} + +static inline void set_track(struct kasan_track *track, gfp_t flags) +{ + track->pid = current->pid; + track->stack = save_stack(flags); +} + +void kasan_enable_current(void) +{ + current->kasan_depth++; +} + +void kasan_disable_current(void) +{ + current->kasan_depth--; +} + +void kasan_check_read(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_read); + +void kasan_check_write(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + check_memory_region((unsigned long)addr, len, true, _RET_IP_); + + return __memset(addr, c, len); +} + +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memmove(dest, src, len); +} + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + */ +void kasan_poison_shadow(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void kasan_unpoison_shadow(const void *address, size_t size) +{ + kasan_poison_shadow(address, size, 0); + + if (size & KASAN_SHADOW_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + *shadow = size & KASAN_SHADOW_MASK; + } +} + +static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) +{ + void *base = task_stack_page(task); + size_t size = sp - base; + + kasan_unpoison_shadow(base, size); +} + +/* Unpoison the entire stack for a task. */ +void kasan_unpoison_task_stack(struct task_struct *task) +{ + __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); +} + +/* Unpoison the stack for the current task beyond a watermark sp value. */ +asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) +{ + /* + * Calculate the task stack base address. Avoid using 'current' + * because this function is called by early resume code which hasn't + * yet set up the percpu register (%gs). + */ + void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); + + kasan_unpoison_shadow(base, watermark - base); +} + +/* + * Clear all poison for the region between the current SP and a provided + * watermark value, as is sometimes required prior to hand-crafted asm function + * returns in the middle of functions. + */ +void kasan_unpoison_stack_above_sp_to(const void *watermark) +{ + const void *sp = __builtin_frame_address(0); + size_t size = watermark - sp; + + if (WARN_ON(sp > watermark)) + return; + kasan_unpoison_shadow(sp, size); +} + +void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); +} + +void kasan_free_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_poison_shadow(page_address(page), + PAGE_SIZE << order, + KASAN_FREE_PAGE); +} + +/* + * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. + * For larger allocations larger redzones are used. + */ +static inline unsigned int optimal_redzone(unsigned int object_size) +{ + return + object_size <= 64 - 16 ? 16 : + object_size <= 128 - 32 ? 32 : + object_size <= 512 - 64 ? 64 : + object_size <= 4096 - 128 ? 128 : + object_size <= (1 << 14) - 256 ? 256 : + object_size <= (1 << 15) - 512 ? 512 : + object_size <= (1 << 16) - 1024 ? 1024 : 2048; +} + +void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) +{ + unsigned int orig_size = *size; + int redzone_adjust; + + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } + redzone_adjust = optimal_redzone(cache->object_size) - + (*size - cache->object_size); + + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + + optimal_redzone(cache->object_size))); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } + + *flags |= SLAB_KASAN; +} + +size_t kasan_metadata_size(struct kmem_cache *cache) +{ + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); +} + +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); + return (void *)object + cache->kasan_info.alloc_meta_offset; +} + +struct kasan_free_meta *get_free_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + return (void *)object + cache->kasan_info.free_meta_offset; +} + +void kasan_poison_slab(struct page *page) +{ + kasan_poison_shadow(page_address(page), + PAGE_SIZE << compound_order(page), + KASAN_KMALLOC_REDZONE); +} + +void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_unpoison_shadow(object, cache->object_size); +} + +void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_poison_shadow(object, + round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + KASAN_KMALLOC_REDZONE); +} + +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; +} + +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +{ + return kasan_kmalloc(cache, object, cache->object_size, flags); +} + +static bool __kasan_slab_free(struct kmem_cache *cache, void *object, + unsigned long ip, bool quarantine) +{ + s8 shadow_byte; + unsigned long rounded_up_size; + + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != + object)) { + kasan_report_invalid_free(object, ip); + return true; + } + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + kasan_report_invalid_free(object, ip); + return true; + } + + rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + + if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; +} + +bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +{ + return __kasan_slab_free(cache, object, ip, true); +} + +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, + gfp_t flags) +{ + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(object == NULL)) + return NULL; + + redzone_start = round_up((unsigned long)(object + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = round_up((unsigned long)object + cache->object_size, + KASAN_SHADOW_SCALE_SIZE); + + kasan_unpoison_shadow(object, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); + + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; +} +EXPORT_SYMBOL(kasan_kmalloc); + +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + struct page *page; + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(ptr == NULL)) + return NULL; + + page = virt_to_page(ptr); + redzone_start = round_up((unsigned long)(ptr + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); + + kasan_unpoison_shadow(ptr, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); + + return (void *)ptr; +} + +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +{ + struct page *page; + + if (unlikely(object == ZERO_SIZE_PTR)) + return (void *)object; + + page = virt_to_head_page(object); + + if (unlikely(!PageSlab(page))) + return kasan_kmalloc_large(object, size, flags); + else + return kasan_kmalloc(page->slab_cache, object, size, flags); +} + +void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), + KASAN_FREE_PAGE); + } else { + __kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + +void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (ptr != page_address(virt_to_head_page(ptr))) + kasan_report_invalid_free(ptr, ip); + /* The object will be poisoned by page_alloc. */ +} + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL | __GFP_ZERO, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || + WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 55deff17a4d9..44ec228de0a2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,5 +1,5 @@ /* - * This file contains shadow memory manipulation code. + * This file contains core KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -40,82 +40,6 @@ #include "kasan.h" #include "../slab.h" -void kasan_enable_current(void) -{ - current->kasan_depth++; -} - -void kasan_disable_current(void) -{ - current->kasan_depth--; -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. - */ -static void kasan_poison_shadow(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - memset(shadow_start, value, shadow_end - shadow_start); -} - -void kasan_unpoison_shadow(const void *address, size_t size) -{ - kasan_poison_shadow(address, size, 0); - - if (size & KASAN_SHADOW_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; - } -} - -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_shadow(base, size); -} - -/* Unpoison the entire stack for a task. */ -void kasan_unpoison_task_stack(struct task_struct *task) -{ - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); -} - -/* Unpoison the stack for the current task beyond a watermark sp value. */ -asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) -{ - /* - * Calculate the task stack base address. Avoid using 'current' - * because this function is called by early resume code which hasn't - * yet set up the percpu register (%gs). - */ - void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - - kasan_unpoison_shadow(base, watermark - base); -} - -/* - * Clear all poison for the region between the current SP and a provided - * watermark value, as is sometimes required prior to hand-crafted asm function - * returns in the middle of functions. - */ -void kasan_unpoison_stack_above_sp_to(const void *watermark) -{ - const void *sp = __builtin_frame_address(0); - size_t size = watermark - sp; - - if (WARN_ON(sp > watermark)) - return; - kasan_unpoison_shadow(sp, size); -} - /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -260,121 +184,12 @@ static __always_inline void check_memory_region_inline(unsigned long addr, kasan_report(addr, size, write, ret_ip); } -static void check_memory_region(unsigned long addr, - size_t size, bool write, +void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { check_memory_region_inline(addr, size, write, ret_ip); } -void kasan_check_read(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_read); - -void kasan_check_write(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - check_memory_region((unsigned long)addr, len, true, _RET_IP_); - - return __memset(addr, c, len); -} - -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memmove(dest, src, len); -} - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memcpy(dest, src, len); -} - -void kasan_alloc_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); -} - -void kasan_free_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); -} - -/* - * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. - * For larger allocations larger redzones are used. - */ -static unsigned int optimal_redzone(unsigned int object_size) -{ - return - object_size <= 64 - 16 ? 16 : - object_size <= 128 - 32 ? 32 : - object_size <= 512 - 64 ? 64 : - object_size <= 4096 - 128 ? 128 : - object_size <= (1 << 14) - 256 ? 256 : - object_size <= (1 << 15) - 512 ? 512 : - object_size <= (1 << 16) - 1024 ? 1024 : 2048; -} - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) -{ - unsigned int orig_size = *size; - int redzone_adjust; - - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); - - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); - - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; - } - - *flags |= SLAB_KASAN; -} - void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); @@ -386,277 +201,6 @@ void kasan_cache_shutdown(struct kmem_cache *cache) quarantine_remove_cache(cache); } -size_t kasan_metadata_size(struct kmem_cache *cache) -{ - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); -} - -void kasan_poison_slab(struct page *page) -{ - kasan_poison_shadow(page_address(page), - PAGE_SIZE << compound_order(page), - KASAN_KMALLOC_REDZONE); -} - -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_unpoison_shadow(object, cache->object_size); -} - -void kasan_poison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_poison_shadow(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), - KASAN_KMALLOC_REDZONE); -} - -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline void filter_irq_stacks(struct stack_trace *trace) -{ - int i; - - if (!trace->nr_entries) - return; - for (i = 0; i < trace->nr_entries; i++) - if (in_irqentry_text(trace->entries[i])) { - /* Include the irqentry function into the stack. */ - trace->nr_entries = i + 1; - break; - } -} - -static inline depot_stack_handle_t save_stack(gfp_t flags) -{ - unsigned long entries[KASAN_STACK_DEPTH]; - struct stack_trace trace = { - .nr_entries = 0, - .entries = entries, - .max_entries = KASAN_STACK_DEPTH, - .skip = 0 - }; - - save_stack_trace(&trace); - filter_irq_stacks(&trace); - if (trace.nr_entries != 0 && - trace.entries[trace.nr_entries-1] == ULONG_MAX) - trace.nr_entries--; - - return depot_save_stack(&trace, flags); -} - -static inline void set_track(struct kasan_track *track, gfp_t flags) -{ - track->pid = current->pid; - track->stack = save_stack(flags); -} - -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); - return (void *)object + cache->kasan_info.alloc_meta_offset; -} - -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; -} - -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) -{ - struct kasan_alloc_meta *alloc_info; - - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); - - return (void *)object; -} - -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) -{ - return kasan_kmalloc(cache, object, cache->object_size, flags); -} - -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, - unsigned long ip, bool quarantine) -{ - s8 shadow_byte; - unsigned long rounded_up_size; - - if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != - object)) { - kasan_report_invalid_free(object, ip); - return true; - } - - /* RCU slabs could be legally used after free within the RCU period */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); - return true; - } - - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) - return false; - - set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); - quarantine_put(get_free_info(cache, object), cache); - return true; -} - -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) -{ - return __kasan_slab_free(cache, object, ip, true); -} - -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) -{ - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(object == NULL)) - return NULL; - - redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); - - kasan_unpoison_shadow(object, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); - - if (cache->flags & SLAB_KASAN) - set_track(&get_alloc_info(cache, object)->alloc_track, flags); - - return (void *)object; -} -EXPORT_SYMBOL(kasan_kmalloc); - -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) -{ - struct page *page; - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(ptr == NULL)) - return NULL; - - page = virt_to_page(ptr); - redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); - - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); - - return (void *)ptr; -} - -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) -{ - struct page *page; - - if (unlikely(object == ZERO_SIZE_PTR)) - return ZERO_SIZE_PTR; - - page = virt_to_head_page(object); - - if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); - else - return kasan_kmalloc(page->slab_cache, object, size, flags); -} - -void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), - KASAN_FREE_PAGE); - } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - -void kasan_kfree_large(void *ptr, unsigned long ip) -{ - if (ptr != page_address(virt_to_head_page(ptr))) - kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ -} - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); @@ -797,113 +341,3 @@ DEFINE_ASAN_SET_SHADOW(f2); DEFINE_ASAN_SET_SHADOW(f3); DEFINE_ASAN_SET_SHADOW(f5); DEFINE_ASAN_SET_SHADOW(f8); - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c12dcfde2ebd..659463800f10 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow(const void *address, size_t size, u8 value); + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); From patchwork Thu Dec 6 12:24:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715741 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E05441731 for ; Thu, 6 Dec 2018 12:25:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF4292BD48 for ; Thu, 6 Dec 2018 12:25:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C12F02BBFB; Thu, 6 Dec 2018 12:25:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 23FCA2CED0 for ; Thu, 6 Dec 2018 12:25:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78B966B79F2; Thu, 6 Dec 2018 07:24:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 719976B79F3; Thu, 6 Dec 2018 07:24:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56B416B79F4; Thu, 6 Dec 2018 07:24:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id E74956B79F2 for ; Thu, 6 Dec 2018 07:24:55 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id d11so88584wrw.4 for ; Thu, 06 Dec 2018 04:24:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Kn5CDDyDqdktmueSgYXSsO0zwDwfFFrMoSLJG2Ivw50=; b=Vt1N1MFWk9B+/PuHVYQ4exQAiWBbbhdTf08uHRKbN0kzp+e8ycWJfXM6VBNPz6jDGc pvOBKbH2UPxaD+EqZBiNQS3VRAIRE5s11ghl/hmnvw2dsD5F0a5fy9kjYh6QigEuhJ1V XMpdUPat0PunvL9h9oo1QAxZnCq47d6HJ9UeUlk6+rtB9TLQyLzG1B6WSir9HE6031Ri adgn8xHUp4SQ+Mios/NmQ78ZRnqpyCQW0S3pygNlGJGBuZdXrbdtymYrC6KFaIwybrgA Eeg398u+ZkwVN97Gvr/vx1U37SpcmFRlTFRpZ1CZVzpMtXU6BlACevD3mzxCMkCUDJo9 Wi3A== X-Gm-Message-State: AA+aEWZsCIN4jeBBg5tIHkF8HtuI5OBN0U1TtWwpohehQk9Vbcl6UPb2 XVVtV+zapK/stJIhfAZhJbeUhcaci1KBDf7POI4vA4Iyk3SdrfGAfHyqxdr3PXhsQMn0tOL6uvU xJRLOxkWGSkervDHDbtTDPH0Eb4itK0Y+JSLjYrw88wxdgljOEKjG2bzQjmXxrK3FEhPs/rdsq8 cK+rNX8ZoJ3ONE/FnTyXSBCquytvICAML7SkVhvBwXBWp7O6iJIE7FVNutPQpDrm8dEZdeXIHg4 98h2N0X6MC/BN+Kl4Ckb7u96Aw64Oo5jT3sBGqeKmZ+hqMkAD10UHANhbbehigeP8E6dWNCWA5a FxYqmPJb3Dwrdfrv5cuWYKfy6/5Lqct4MTxc7WcZN2A7dvG80eb/5/IdOdUCjZHkfnKEbsY0/rZ G X-Received: by 2002:adf:81b6:: with SMTP id 51mr27075929wra.240.1544099095465; Thu, 06 Dec 2018 04:24:55 -0800 (PST) X-Received: by 2002:adf:81b6:: with SMTP id 51mr27075873wra.240.1544099094571; Thu, 06 Dec 2018 04:24:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099094; cv=none; d=google.com; s=arc-20160816; b=mDl6QDaOOq73zcH01eEOM/BvpROf/4sKbCKCDicKZSWE0bayxGlDcIkQKAM6cZYmxj WyQHmHh4BwR4lRN7mLi1Q78VtSpiLe11ARYwP64kTppMrYTDTobs7ZeDtcTheLEKKNFQ bA68AkpeFKIDLqwK7BoChD3M4RwNul7YGhGO2pQLSbnRt9MtkOwlWbSnpUKd93v/hhUE fEVTjOVqS49KEXCaxGqxuyE5sKyK0NFiCQsBPnwTPi7ZW2Nhb/Yqoju6R6Zn1wam2kgK oyKJL0XMnbKKo/1etHKfdYrc3wrX0/3aQ2Vks/cAWusjb3cglQVeGKvoEQyM2HUne/Jh i3tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=Kn5CDDyDqdktmueSgYXSsO0zwDwfFFrMoSLJG2Ivw50=; b=xDAcoK6NoNctudSrLPjkDCo+ExxLLS3FbTmhBbIHhg67GGFyAtoa7KCidJ9q+8F4nK 29mefJ4iBEbjT7pWc+oryGbckDLQpM2ETybtHfVHdtg0nIvatpwq03rf7VREGEiVmrni 9JuhU8xkUsIsUXvS65elzgAZ+rly6gRGvXJGmABHwn//NjYwyb/NvUTN2XSR+Wn+5ERv QAcRqGkjVJzBQv3/BHuG0D9mv5RLFRQYFPoTgmjkFxRDNBHwKKbkgZ46uz+/BWawnk6k DuQlhnFzdaEZH8sB2reNvz/X3no/pTQJSdB2KIS8eWbUrP4NSZ2qs1YJfTzd9y/uWBsF 44GQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Pf39mxwR; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 93sor196985wrb.13.2018.12.06.04.24.54 for (Google Transport Security); Thu, 06 Dec 2018 04:24:54 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Pf39mxwR; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Kn5CDDyDqdktmueSgYXSsO0zwDwfFFrMoSLJG2Ivw50=; b=Pf39mxwRcpa7KrX6UwoJEQfz3CfXq5RBMJ6m5aM5/gDD8ItWtZq5G1xjMawUGn+z+T dS+/sMDdEwBB6CM7bHAttpoCEIgFl/kBeGPn3lwT1LywZDgoWZOOLzWJ2RZ2yetGz9o9 lMtnP1mE+kwTeKyEArzQjzOtctZLFx7BLRHSE0CEcCJLYjXIGKOCJ6xwLrKMHD6bes7C 7uOYfXDimhdwWkbB2RWHOR+L5Z59vziD3aEiJFEdoJazqAi7fgF+yBbpulYxf79uEPLu 6Lu2htdc0fEE4NHxlu9JEzcIxCl/CyyVSkHgAm7pbWxdsZHVrmTWKfD28vT5Pk9MVFVa +D/g== X-Google-Smtp-Source: AFSGD/XqP2P/Wb4WdG1rKPMTei3wp8lKRgpVfvbDuVz/Yb1l+zYY35vzWOnd/pnfJMaPCW4EAswf3w== X-Received: by 2002:a5d:4d11:: with SMTP id z17mr24026901wrt.209.1544099094073; Thu, 06 Dec 2018 04:24:54 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:53 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 04/25] kasan: rename source files to reflect the new naming scheme Date: Thu, 6 Dec 2018 13:24:22 +0100 Message-Id: <88c6fd2a883e459e6242030497230e5fb0d44d44.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We now have two KASAN modes: generic KASAN and tag-based KASAN. Rename kasan.c to generic.c to reflect that. Also rename kasan_init.c to init.c as it contains initialization code for both KASAN modes. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 8 ++++---- mm/kasan/{kasan.c => generic.c} | 0 mm/kasan/{kasan_init.c => init.c} | 0 3 files changed, 4 insertions(+), 4 deletions(-) rename mm/kasan/{kasan.c => generic.c} (100%) rename mm/kasan/{kasan_init.c => init.c} (100%) diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index a6df14bffb6b..d643530b24aa 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,14 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n -UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_generic.o := n KCOV_INSTRUMENT := n -CFLAGS_REMOVE_kasan.o = -pg +CFLAGS_REMOVE_generic.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o generic.o report.o init.o quarantine.o diff --git a/mm/kasan/kasan.c b/mm/kasan/generic.c similarity index 100% rename from mm/kasan/kasan.c rename to mm/kasan/generic.c diff --git a/mm/kasan/kasan_init.c b/mm/kasan/init.c similarity index 100% rename from mm/kasan/kasan_init.c rename to mm/kasan/init.c From patchwork Thu Dec 6 12:24:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715749 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B25F109C for ; Thu, 6 Dec 2018 12:25:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78C8E297D0 for ; Thu, 6 Dec 2018 12:25:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6BDF42D564; Thu, 6 Dec 2018 12:25:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F08852D393 for ; Thu, 6 Dec 2018 12:25:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 47F866B79F5; Thu, 6 Dec 2018 07:24:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3E0876B79F6; Thu, 6 Dec 2018 07:24:59 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 234B86B79F7; Thu, 6 Dec 2018 07:24:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id A4ADD6B79F5 for ; Thu, 6 Dec 2018 07:24:58 -0500 (EST) Received: by mail-wm1-f71.google.com with SMTP id b186so201998wmc.8 for ; Thu, 06 Dec 2018 04:24:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=12vqoHtZfg31pq6q/oP+g6LXOPYAvpj2xawr0AHwgS8=; b=C5TENBA6js+UuBFJ/9rviBLWHY5m3+3AAoX2TjoeX5Elne2Hk2+Bpn6hmq0MqmAMxn s4FqOp8igIuU8sLh2mX3jdIWG7hMMrA6X2QhzeTFEDQqpEZWMuZWU1Q0tLuPWV9F0obY UymDXRN07lYVKXaAXndub5QNYChrvp1Q9jvqmirPhOAQvj9tBxjN+OU1da2wYk6ZgBpq 8TK+gfteYIg9QM5j7OIYA6Yke+x/eJA1g89n1byXzlZiNhmf0jljDDuICUGrbW0ZLH93 5616Jg/KZkHdldraNk7xP91hWulv/QOQn0d9DmpboPG0PObc1yfutIMlNOxHFHEMIGcN ndRA== X-Gm-Message-State: AA+aEWbSpGNRgUAD5dafVhGjRcXOm/RaHR3NuF59KxMyysYfXQNINXZm i6MZaacXI2k0wBGOGeaiq6dK+c1niGBIZNVeDU8v7izu8xeRT41om57MVWoD+hUR8CvL3gElFsk NssiONCRWCPxwvTkiGE1TucjuPyKrkJibomBQTVPy0Sus3a8U/I85Vf0MJOtGEUTWdQ6mKWebAa azdgcGqKJhdoXqjf1qg4YZw16OQ93YyWMzWa6TIxhOqr39T9H1+N5GPn/9wbA57lYu82c+l2R8l N9SLJaZtS2driJd+2VVIvz1W7DioYnJgozoOZ91r+3KECjDZec6QwHcexmK5zlInKygKIkzdR1/ p4OyAt5aT1LeYbAwgLy8M19BvSi55P4yr1XChDjsGlzlRPOMsKCGeDYhMMchtjz3q4woOnMvmQl H X-Received: by 2002:a1c:43:: with SMTP id 64mr20717014wma.72.1544099098155; Thu, 06 Dec 2018 04:24:58 -0800 (PST) X-Received: by 2002:a1c:43:: with SMTP id 64mr20716918wma.72.1544099096624; Thu, 06 Dec 2018 04:24:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099096; cv=none; d=google.com; s=arc-20160816; b=KzeSxRYFdtvGtVzWN3hfwEDtBNzs9VaC2n2toHk0x3Nv0G7b0fpDdgFzEFK2Cy5bTl cLerOGLf5V7feVABo0Ag/JoZSkBCz1qeqRV4fJUxbIGAKxGMEoU0jMPjdwchWn5vaOl4 vOl59/iaAGXIfOnXVGSCkc/PH72Q6kqcehtd9eGsm/Chk1XgU4yglOF0cVjLGk9IgU+z g7DfcxrkFEqJ4WD1+HjnJFGbfzg59gXLCecpYTPBI1Q2APAEn9ijOZXcSTmUb3qQKvI0 yHwWpY6zS22DMF7+TgX72KQANHY28RGD9Zz7ru56Sw5tWOggus4Kx9UpO6iSI3t8LLRO sipA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=12vqoHtZfg31pq6q/oP+g6LXOPYAvpj2xawr0AHwgS8=; b=prKzBGsFL37QGqq/T/EkYLlhEiO8GjJf+aE/5/JDEZ+zkMgJsl5T25bvXVYY2mnp/3 s/SeOOsHrmhsQ7nbbSQH2l10Pd7eSgAnOgyEgSlMxGOCTPhuEuQ6C4SiXbuHkG9LzS2W bEJZXM2HHp3aaCfeQLikNQWz+csVSWEF6sFziAPW97y5tni4qOMwbfeBfSsuzTXiY7ph EV1/Fx+/FljZRW/uoEz8DUB+uV2zSDPTyTUAReMIAsOC7WfPJNJwdTrsEW3kAoebhfrC IBIBAzm6Ru84bQvhaHKjQwwCS+3ZkKhe1ERbT4ul5JHltTl4yMQidtT4oV2lxs3fizYV pTPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=b834rT5j; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id z20sor485553wmc.2.2018.12.06.04.24.56 for (Google Transport Security); Thu, 06 Dec 2018 04:24:56 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=b834rT5j; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=12vqoHtZfg31pq6q/oP+g6LXOPYAvpj2xawr0AHwgS8=; b=b834rT5jJpzu6I1DwI1xlXpHAUKJ6jJKE7AM9uDi8xHpOQpDpTc4xClTGMUjmFEQb9 VPbO6fFnl8aylWUsAjz/BaBEjQ6nmbBVQaeuBWH1hbuDX54qLlF/yE/BXfEYsKaf4DxV TYfYYNmqhub/00e60MrfInzA3lQ6vJS25xsPxXwWmbb/Fb9Mu15/zUsHCxPtUTniWSTL A9j4morPZw6ffFaN5uN/Xp1dHiLh//ObbZ/fxbmWIhGisun8uJ+RJVfIv2x/LyiZABEW 8Wqf/6D/zRVA/BzilGYmI1H+tJQm5fkHDPDY5dT2Oo2OroiAWug4ZE4ZtbZXGqLx8h0s QBfg== X-Google-Smtp-Source: AFSGD/W1/4ViqF1e9tLit7+fR9yan3GJQaVHzZ69MbjHJ3UJUq4CD6zewrsVSRFvYa2mVsjDWpl4yQ== X-Received: by 2002:a1c:e088:: with SMTP id x130mr18266024wmg.23.1544099095865; Thu, 06 Dec 2018 04:24:55 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:54 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 05/25] kasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS Date: Thu, 6 Dec 2018 13:24:23 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit splits the current CONFIG_KASAN config option into two: 1. CONFIG_KASAN_GENERIC, that enables the generic KASAN mode (the one that exists now); 2. CONFIG_KASAN_SW_TAGS, that enables the software tag-based KASAN mode. The name CONFIG_KASAN_SW_TAGS is chosen as in the future we will have another hardware tag-based KASAN mode, that will rely on hardware memory tagging support in arm64. With CONFIG_KASAN_SW_TAGS enabled, compiler options are changed to instrument kernel files with -fsantize=kernel-hwaddress (except the ones for which KASAN_SANITIZE := n is set). Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes. This commit also adds empty placeholder (for now) implementation of tag-based KASAN specific hooks inserted by the compiler and adjusts common hooks implementation. While this commit adds the CONFIG_KASAN_SW_TAGS config option, this option is not selectable, as it depends on HAVE_ARCH_KASAN_SW_TAGS, which we will enable once all the infrastracture code has been added. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/compiler-clang.h | 6 +- include/linux/compiler-gcc.h | 6 ++ include/linux/compiler_attributes.h | 13 ---- include/linux/kasan.h | 16 +++-- lib/Kconfig.kasan | 98 ++++++++++++++++++++++------- mm/kasan/Makefile | 6 +- mm/kasan/generic.c | 2 +- mm/kasan/kasan.h | 3 +- mm/kasan/tags.c | 75 ++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 53 +++++++++------- 11 files changed, 214 insertions(+), 66 deletions(-) create mode 100644 mm/kasan/tags.c diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 3e7dafb3ea80..39f668d5066b 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -16,9 +16,13 @@ /* all clang versions usable with the kernel support KASAN ABI version 5 */ #define KASAN_ABI_VERSION 5 +#if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) /* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature(address_sanitizer) #define __SANITIZE_ADDRESS__ +#define __no_sanitize_address \ + __attribute__((no_sanitize("address", "hwaddress"))) +#else +#define __no_sanitize_address #endif /* diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 2010493e1040..5776da43da97 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -143,6 +143,12 @@ #define KASAN_ABI_VERSION 3 #endif +#if __has_attribute(__no_sanitize_address__) +#define __no_sanitize_address __attribute__((no_sanitize_address)) +#else +#define __no_sanitize_address +#endif + #if GCC_VERSION >= 50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1 #endif diff --git a/include/linux/compiler_attributes.h b/include/linux/compiler_attributes.h index f8c400ba1929..7bceb9469197 100644 --- a/include/linux/compiler_attributes.h +++ b/include/linux/compiler_attributes.h @@ -206,19 +206,6 @@ */ #define __noreturn __attribute__((__noreturn__)) -/* - * Optional: only supported since gcc >= 4.8 - * Optional: not supported by icc - * - * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute - * clang: https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis - */ -#if __has_attribute(__no_sanitize_address__) -# define __no_sanitize_address __attribute__((__no_sanitize_address__)) -#else -# define __no_sanitize_address -#endif - /* * gcc: https://gcc.gnu.org/onlinedocs/gcc/Common-Type-Attributes.html#index-packed-type-attribute * clang: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#index-packed-variable-attribute diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 52c86a568a4e..b66fdf5ea7ab 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); -void kasan_cache_shrink(struct kmem_cache *cache); -void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); @@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} -static inline void kasan_cache_shrink(struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, @@ -155,4 +151,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink(struct kmem_cache *cache); +void kasan_cache_shutdown(struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink(struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ + #endif /* LINUX_KASAN_H */ diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index d0bad1bd9a2b..d8c474b6691e 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -1,36 +1,92 @@ +# This config refers to the generic KASAN mode. config HAVE_ARCH_KASAN bool -if HAVE_ARCH_KASAN +config HAVE_ARCH_KASAN_SW_TAGS + bool + +config CC_HAS_KASAN_GENERIC + def_bool $(cc-option, -fsanitize=kernel-address) + +config CC_HAS_KASAN_SW_TAGS + def_bool $(cc-option, -fsanitize=kernel-hwaddress) config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + depends on (HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC) || \ + (HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS) + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs. + See Documentation/dev-tools/kasan.rst for details. + +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: generic KASAN (similar to userspace ASan, + x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and + software tag-based KASAN (a version based on software memory + tagging, arm64 only, similar to userspace HWASan, enabled with + CONFIG_KASAN_SW_TAGS). + Both generic and tag-based KASAN are strictly debugging features. + +config KASAN_GENERIC + bool "Generic mode" + depends on HAVE_ARCH_KASAN && CC_HAS_KASAN_GENERIC depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables generic KASAN mode. + Supported in both GCC and Clang. With GCC it requires version 4.9.2 + or later for basic support and version 5.0 or later for detection of + out-of-bounds accesses for stack and global variables and for inline + instrumentation mode (CONFIG_KASAN_INLINE). With Clang it requires + version 3.7.0 or later and it doesn't support detection of + out-of-bounds accesses for global variables yet. + This mode consumes about 1/8th of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +config KASAN_SW_TAGS + bool "Software tag-based mode" + depends on HAVE_ARCH_KASAN_SW_TAGS && CC_HAS_KASAN_SW_TAGS + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enables software tag-based KASAN mode. + This mode requires Top Byte Ignore support by the CPU and therefore + is only supported for arm64. + This mode requires Clang version 7.0.0 or later. + This mode consumes about 1/16th of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + This mode may potentially introduce problems relating to pointer + casting and comparison, as it embeds tags into the top byte of each + pointer. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer + This enables further checks in generic KASAN, for now it only + includes the address-use-after-scope check that can lead to + excessive kernel stack usage, frame size warnings and longer compile time. - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more - + See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 choice prompt "Instrumentation type" @@ -53,7 +109,7 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice @@ -67,11 +123,9 @@ config KASAN_S390_4_LEVEL_PAGING 4-level paging instead. config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. - -endif + kernel debugging features like KASAN. diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index d643530b24aa..68ba1822f003 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_generic.o := n +UBSAN_SANITIZE_tags.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_generic.o = -pg @@ -10,5 +11,8 @@ CFLAGS_REMOVE_generic.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o generic.o report.o init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 44ec228de0a2..b8de6d33c55c 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,5 +1,5 @@ /* - * This file contains core KASAN code. + * This file contains core generic KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 659463800f10..19b950eaccff 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB) +#if defined(CONFIG_KASAN_GENERIC) && \ + (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c new file mode 100644 index 000000000000..04194923c543 --- /dev/null +++ b/mm/kasan/tags.c @@ -0,0 +1,75 @@ +/* + * This file contains core tag-based KASAN code. + * + * Copyright (c) 2018 Google, Inc. + * Author: Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE(size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE(1); +DEFINE_HWASAN_LOAD_STORE(2); +DEFINE_HWASAN_LOAD_STORE(4); +DEFINE_HWASAN_LOAD_STORE(8); +DEFINE_HWASAN_LOAD_STORE(16); + +void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_storeN_noabort); + +void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_tag_memory); diff --git a/mm/slub.c b/mm/slub.c index 8561a32910dd..e739d46600b9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2992,7 +2992,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page, do_slab_free(s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr) { do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr); diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 69552a39951d..25c259df8ffa 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC + ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else @@ -12,36 +13,44 @@ CFLAGS_KASAN_MINIMAL := -fsanitize=kernel-address cc-param = $(call cc-option, -mllvm -$(1), $(call cc-option, --param $(1))) -ifeq ($(call cc-option, $(CFLAGS_KASAN_MINIMAL) -Werror),) - ifneq ($(CONFIG_COMPILE_TEST),y) - $(warning Cannot use CONFIG_KASAN: \ - -fsanitize=kernel-address is not supported by compiler) - endif -else - # -fasan-shadow-offset fails without -fsanitize - CFLAGS_KASAN_SHADOW := $(call cc-option, -fsanitize=kernel-address \ +# -fasan-shadow-offset fails without -fsanitize +CFLAGS_KASAN_SHADOW := $(call cc-option, -fsanitize=kernel-address \ -fasan-shadow-offset=$(KASAN_SHADOW_OFFSET), \ $(call cc-option, -fsanitize=kernel-address \ -mllvm -asan-mapping-offset=$(KASAN_SHADOW_OFFSET))) - ifeq ($(strip $(CFLAGS_KASAN_SHADOW)),) - CFLAGS_KASAN := $(CFLAGS_KASAN_MINIMAL) - else - # Now add all the compiler specific options that are valid standalone - CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ - $(call cc-param,asan-globals=1) \ - $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ - $(call cc-param,asan-stack=1) \ - $(call cc-param,asan-use-after-scope=1) \ - $(call cc-param,asan-instrument-allocas=1) - endif - +ifeq ($(strip $(CFLAGS_KASAN_SHADOW)),) + CFLAGS_KASAN := $(CFLAGS_KASAN_MINIMAL) +else + # Now add all the compiler specific options that are valid standalone + CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ + $(call cc-param,asan-globals=1) \ + $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ + $(call cc-param,asan-stack=1) \ + $(call cc-param,asan-use-after-scope=1) \ + $(call cc-param,asan-instrument-allocas=1) endif ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif # CONFIG_KASAN_GENERIC +ifdef CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif + +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +endif # CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif From patchwork Thu Dec 6 12:24:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A3401731 for ; Thu, 6 Dec 2018 12:25:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69A08292C8 for ; Thu, 6 Dec 2018 12:25:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5C5B62D93C; Thu, 6 Dec 2018 12:25:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2742F292C8 for ; Thu, 6 Dec 2018 12:25:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E9016B79F6; Thu, 6 Dec 2018 07:25:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 149776B79F8; Thu, 6 Dec 2018 07:25:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F05746B79F9; Thu, 6 Dec 2018 07:24:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 982656B79F6 for ; Thu, 6 Dec 2018 07:24:59 -0500 (EST) Received: by mail-wm1-f69.google.com with SMTP id f193so205987wme.8 for ; Thu, 06 Dec 2018 04:24:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kdXJRxv4fyHhhgwNYAW9KeDDpz520moerszNmxFBOyU=; b=fBZtfL9lAEmz0El208BeiGn/StLLsMjJtbREZYvvaaZxPv83mckPxcNlKBe+RXHU50 aeQgTpUHd8POZ7Gz2j1vKuGuRnGEI+qGBce0aSwnZRM3xA5IbRx48XFQ742r1FxJc2Gl b2m7jgE/nL8uE3KBiiNJusZAegsQIM07ezC5EMt2UGartCOfZCkUHwkSWrxd7wCD4prl 9MortPpOV52E2rawtARff7Qnoiq7c8rETOJRaua2fCPDI93asItCpQTHlkD5g2lNcDE8 s60B+3rXzRj9eNUIVxZCQpx6uIoi5jHPsEBq3MdyKEwIoQoWF2/Sg7cnXNZp8wt736Gr Dqpg== X-Gm-Message-State: AA+aEWa+oO5KpGYv1s/DEs3zdRkOPdBZB77vlYARY/8K6Js0gfr1qQxT bdHju12gPgiCKG72GWBV5xkhjweKGBHSK0BWsVSYGoPJr9YINmXiN38eygSW68t3Vu9FI0mm6bF Y/sdRr+pRJk4U1hPdnTh32HYZc5lEUgL2lyR/IFbmg3hP0Ztl3uhgJk8Ams/EsaEKMj+HFbxF7g mwnTrWlyTkA/1ngWUU861ZMK2x4zEVhyK3AocgQrBkFiP3bEQThiiGw2Rs5BDSX87iWax2ZTmqe zhUbfgSJJGQavOi5Io8dYINvGgnG9uuystCuNlI1b8e+H+dNHeG4taWINB3hsFQl9iZWG1joJhU v/Ha2zAPko+MmX8C+u03CH+PzXMvT5z0ktlBwLZGyllquig2M8WB3oZDtVXl8cudbbcmGMh57Ng B X-Received: by 2002:adf:cc91:: with SMTP id p17mr25920934wrj.118.1544099099153; Thu, 06 Dec 2018 04:24:59 -0800 (PST) X-Received: by 2002:adf:cc91:: with SMTP id p17mr25920882wrj.118.1544099098217; Thu, 06 Dec 2018 04:24:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099098; cv=none; d=google.com; s=arc-20160816; b=kPFxlPz0NiULztGEGpLCQZerdAJEv+5zLadbGKxDRL/gY08nSm7/95plCIfUeZRY+G gGAwrh4ZHA3tmHTspInmoE+Ki4hx1kENN5vGPU3h1m7X4Mi0u9ChmVtHGIh0LxhUCuhS +Ll7tbi4LRhcfTi9/cYCKszviIr+j8dPPnbzbEOKu4x+Il6JUyQqmib8NCGh8AIdjpLZ ggLuzR+GoRDzn5H/LVhb5rbv+17Dx8PmWabNMjl06qAGqh7cA9fIXZKskWDNeO/b+NNv 9M9Pkpb/+yolgt4upOod32jRZkwrl7OuVxfZDfLSUR0mXE9LYpDAbAerXrDiUFZy43md ZPCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=kdXJRxv4fyHhhgwNYAW9KeDDpz520moerszNmxFBOyU=; b=g2z6vRfuBlSDH4t/QwbwqZ2Ist/rvJvxC6rSviCVSSEl6/NRWoR8C8DnRfCYahQe2D aL9jK77SxD991iuMV/eslryn1AwCahy8zWmp1zSwGqjfhDxZIynVutCSL4P8ofemYGkF RW/xq9DMPVb98Q42u+eDpeR32l/hIqnXy8JutlvbaQYdsuD8fyJvypdld7WWUh1IcdEk ipzKQRff90JzeZaJ07Imo+MoM69kPZPpHmJKfP06waeLspzrdU7LKWMX9RYekHAY9unT PRpR7f2qqk92HwEqpo3ReWA5mCtrqOwpQesZXAG0eKA98I1WSINpt/K3isR5Hg8FYTs9 n65Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=tSllCbYh; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g18sor201934wrw.3.2018.12.06.04.24.58 for (Google Transport Security); Thu, 06 Dec 2018 04:24:58 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=tSllCbYh; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kdXJRxv4fyHhhgwNYAW9KeDDpz520moerszNmxFBOyU=; b=tSllCbYhK8CTwm062EuJ9YxzX5azvNPGwSixQNdttv0wG7HbPvJ4HWqvcMlUyhbXFI qfiFNFadH9dyUfmPDcs07QkW+5tVlzIzfHGL34eDJb+aCwpLDI+2Jq5bSjS1DLLPtZSY QFM+Ke7eo58l5DlGpf90kwlk+F8UGcaPSuYzmSUtZxOfKt7BqigtuJmeH3dzl6XZlk3b 783OT0Gs76sD8FegLNM/9btM6Wz3FXl34UjM6CMRJGkzceNPCf13AnIagkuNdqYjAyt7 RRwXrFNgjBuXIhljtYHX/uaiR16N7FvbWMvVcpbfwmpfzQrZi0JChiYAum3bkh0bt5QX zWyQ== X-Google-Smtp-Source: AFSGD/VDT3EJaRa30kcxnPk5C86z+Ev8IK3EjEUH0BceyiJnKYlqR1isJoV6wqdMtEO6xCDR+R+/9g== X-Received: by 2002:a5d:6a42:: with SMTP id t2mr27012161wrw.50.1544099097573; Thu, 06 Dec 2018 04:24:57 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:56 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 06/25] kasan, arm64: adjust shadow size for tag-based mode Date: Thu, 6 Dec 2018 13:24:24 +0100 Message-Id: <308b6bd49f756bb5e533be93c6f085ba99b30339.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory. This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when the tag-based KASAN mode is enabled. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/Makefile | 11 ++++++++++- arch/arm64/include/asm/memory.h | 7 +++---- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 6cb9fc7e9382..99e7d08c6083 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -91,10 +91,19 @@ else TEXT_OFFSET := 0x00080000 endif +ifeq ($(CONFIG_KASAN_SW_TAGS), y) +KASAN_SHADOW_SCALE_SHIFT := 4 +else +KASAN_SHADOW_SCALE_SHIFT := 3 +endif + +KBUILD_CFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) +KBUILD_CPPFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) +KBUILD_AFLAGS += -DKASAN_SHADOW_SCALE_SHIFT=$(KASAN_SHADOW_SCALE_SHIFT) + # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b96442960aea..05fbc7ffcd31 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -74,12 +74,11 @@ #define KERNEL_END _end /* - * KASAN requires 1/8th of the kernel virtual address space for the shadow - * region. KASAN can bloat the stack significantly, so double the (minimum) - * stack size when KASAN is in use. + * Generic and tag-based KASAN require 1/8th and 1/16th of the kernel virtual + * address space for the shadow region respectively. They can bloat the stack + * significantly, so double the (minimum) stack size when they are in use. */ #ifdef CONFIG_KASAN -#define KASAN_SHADOW_SCALE_SHIFT 3 #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_THREAD_SHIFT 1 #else From patchwork Thu Dec 6 12:24:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DD5F1731 for ; Thu, 6 Dec 2018 12:25:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49BFA292C8 for ; Thu, 6 Dec 2018 12:25:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C4C52D393; Thu, 6 Dec 2018 12:25:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8AB28297D0 for ; Thu, 6 Dec 2018 12:25:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4CB846B79F9; Thu, 6 Dec 2018 07:25:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 47D8E6B7A01; Thu, 6 Dec 2018 07:25:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2380B6B79FE; Thu, 6 Dec 2018 07:25:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 988FF6B79F9 for ; Thu, 6 Dec 2018 07:25:02 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id 1so203874wms.9 for ; Thu, 06 Dec 2018 04:25:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MTc5ovlXUaRXySXmLWHT7OWTT8MlSnX3GRUndZ8+Wx4=; b=IsazFbPu/o1Fb+lJPmtFD+2oObVzuHEtptopT178Ot8Th4kMz9ygellTo7DTRrfRLk jN5F3Fl8ErRLnR/GB9ryTYZXRmouzVoBwFQzPBzDUvfV5u/YxMlaQv5j2jVRsmVz/taW qMrglIE8M+DxmappZPfYRTkGHSugCdL6FLIwGIaqJmD5bwUFPP4XTMge0EdoGp0eIFBR dekOiWfOTDtOLJPDcRyld7S5R1w2rwseyQIm9/df7ALnTCSHNwRyJBQAK4GR/5ftkHMH XwJ1//TVnZjxfByIxWguzf3vjoQEjb9z8AlUdQL1G7Ep8s4U/ujtnGxhVVld9iPLtc+8 pmMw== X-Gm-Message-State: AA+aEWbqbAgoRUEJqZgiFVPD0MSI+ZdoBxDQia5cNKtFap4N51sUAjP1 H4ATLRXjxaotOVCD6hUhfqpeoA9NNOsRHGvPJA5W/IAc4EqFuQolqWSYY62ZAjKvf0HakiGPEYn wTAtgchm82LjlBRaFonLD+b6S/gCxQKVZ5Vj3BwmapDqQNP5aWcDD3ttjc8hkPVwbsKbzOBYLEx nLlWSavQFk/NbY8kOmZnGcL2u+uyArOEVbg2eUPc25xbt5eiVYBTocEBI4/RQvXBJ4uluXwNtl5 9k9R8RkQPI3do22YY9iBdhYax1NUslPlnfTgnEAA9xq3gSm5FeAnB+hhsy8KQS3FLwSXqu+D0Eg eS0SLrP8PEcJE2Gp4tf9pJ2ahGMu2j56k+Ai09EcXopByX9E9jwHBbePo56uB1IHan4Dd2rTErG Y X-Received: by 2002:a1c:8cd0:: with SMTP id o199mr18293439wmd.56.1544099102063; Thu, 06 Dec 2018 04:25:02 -0800 (PST) X-Received: by 2002:a1c:8cd0:: with SMTP id o199mr18293341wmd.56.1544099100199; Thu, 06 Dec 2018 04:25:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099100; cv=none; d=google.com; s=arc-20160816; b=gfpxFVfgwEFlfmX+1+BKt2evdiI/Yi+AkbhPIcJl8zPCG+Vq1PblE+x3gZAqNWb/Lq NzcoFg9U6EAjF+0iGuKQzVSQ5e7KKYETdQgIT1QHtqGbviZlRBPPmZR32a0Rzhc6egIG fjKvuA4hHPxoyXh2je62NkFv46pR45ydcIRTb5ee3t7usu/Xpy8LHbJ6oIcxJVBTkhJz gIo87FYSrqF1yZnNjtlXFxz8275v/59yt9HkC3KJyssA5pipIAikrJp3rypuq7a0LF1Y sZplFiZxExDdHQ2Pr8j/juweqH33Evctk1bqw4jbXIY4u6gyAKoPqPnmZ+AsnbFVvRE2 JQUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=MTc5ovlXUaRXySXmLWHT7OWTT8MlSnX3GRUndZ8+Wx4=; b=JKOTn4Sa42OfKs8WLnbWppkJRCCIfXmAHn14uJyJ4VJBDezoDMOA0LYa/nWLkaPOP/ alqrzm+Nz3fuy5zCtj/K50KebXOPvXQZX6yMjhD2VWQGTDPNrdyTfIizBnbbORcertLR 6JF4hFJFHpHVAmLWCwEUqIsFHp0N2I4vw5hOPEyZZ8hIrX4A3bsrgGMYlPXP3fk7FQ+o EkL4LTD3vp1klTIa8+NI26+eGQi/Vub6MY9T7GtaazW4TNudgPmCbDHcbaukRcz6uN4E r9zv7MZUsQO6IKFrhOnOaJRLMrDSw66sPmC5YLqf7Ok3fqTaT/xXdwyr+K2lMu+PWFWV 1xYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Ym6dJH85; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id f193sor470315wme.9.2018.12.06.04.25.00 for (Google Transport Security); Thu, 06 Dec 2018 04:25:00 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Ym6dJH85; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MTc5ovlXUaRXySXmLWHT7OWTT8MlSnX3GRUndZ8+Wx4=; b=Ym6dJH85gP37m8EGNHXzMD48ua5OKxUh/DQpACsUipqXpw/C4NEl/N5k3AfLv6dKf5 4I0TiSuhYi6Kf1OqflsVaG0lFeNER8guXvwsc/wwXNIc4yV/KmBbaVzEIbRT9yLhwFwx DaFyPGjp5v9sB5RhOwDk3tJz5xqIBYkSDMGkY4ghLJMH3zFHrvzR/esQ9acz8ubSuBPQ CYd4h2akoXaRx51Zo0j4Vq9CvLrIvM8l2kQt0C7kV/4QsCS5FQhyGCEgAnq9pjVVgIiD c7+hmfCCQp8kN8Yw+M/TubgsROfaLmPlLyky9hJ+Ah+seAXxc2AOxd8d8jgmyyZiB67A jBtg== X-Google-Smtp-Source: AFSGD/WErwC/19BPSufojTKncGwYvh+BEpAGM7HmZPV2+jLPltZGbcKOSJ19T8Tu8t+ttfT3o8OPEA== X-Received: by 2002:a1c:18c:: with SMTP id 134mr18916240wmb.88.1544099099289; Thu, 06 Dec 2018 04:24:59 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:58 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 07/25] kasan: rename kasan_zero_page to kasan_early_shadow_page Date: Thu, 6 Dec 2018 13:24:25 +0100 Message-Id: <3fed313280ebf4f88645f5b89ccbc066d320e177.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With tag based KASAN mode the early shadow value is 0xff and not 0x00, so this patch renames kasan_zero_(page|pte|pmd|pud|p4d) to kasan_early_shadow_(page|pte|pmd|pud|p4d) to avoid confusion. Suggested-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 43 ++++++++++++--------- arch/s390/mm/dump_pagetables.c | 17 +++++---- arch/s390/mm/kasan_init.c | 33 +++++++++------- arch/x86/mm/dump_pagetables.c | 11 +++--- arch/x86/mm/kasan_init_64.c | 55 +++++++++++++------------- arch/xtensa/mm/kasan_init.c | 18 +++++---- include/linux/kasan.h | 12 +++--- mm/kasan/init.c | 70 +++++++++++++++++++--------------- 8 files changed, 145 insertions(+), 114 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 63527e585aac..4ebc19422931 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -47,8 +47,9 @@ static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { if (pmd_none(READ_ONCE(*pmdp))) { - phys_addr_t pte_phys = early ? __pa_symbol(kasan_zero_pte) - : kasan_alloc_zeroed_page(node); + phys_addr_t pte_phys = early ? + __pa_symbol(kasan_early_shadow_pte) + : kasan_alloc_zeroed_page(node); __pmd_populate(pmdp, pte_phys, PMD_TYPE_TABLE); } @@ -60,8 +61,9 @@ static pmd_t *__init kasan_pmd_offset(pud_t *pudp, unsigned long addr, int node, bool early) { if (pud_none(READ_ONCE(*pudp))) { - phys_addr_t pmd_phys = early ? __pa_symbol(kasan_zero_pmd) - : kasan_alloc_zeroed_page(node); + phys_addr_t pmd_phys = early ? + __pa_symbol(kasan_early_shadow_pmd) + : kasan_alloc_zeroed_page(node); __pud_populate(pudp, pmd_phys, PMD_TYPE_TABLE); } @@ -72,8 +74,9 @@ static pud_t *__init kasan_pud_offset(pgd_t *pgdp, unsigned long addr, int node, bool early) { if (pgd_none(READ_ONCE(*pgdp))) { - phys_addr_t pud_phys = early ? __pa_symbol(kasan_zero_pud) - : kasan_alloc_zeroed_page(node); + phys_addr_t pud_phys = early ? + __pa_symbol(kasan_early_shadow_pud) + : kasan_alloc_zeroed_page(node); __pgd_populate(pgdp, pud_phys, PMD_TYPE_TABLE); } @@ -87,8 +90,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, pte_t *ptep = kasan_pte_offset(pmdp, addr, node, early); do { - phys_addr_t page_phys = early ? __pa_symbol(kasan_zero_page) - : kasan_alloc_zeroed_page(node); + phys_addr_t page_phys = early ? + __pa_symbol(kasan_early_shadow_page) + : kasan_alloc_zeroed_page(node); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -205,14 +209,14 @@ void __init kasan_init(void) kasan_map_populate(kimg_shadow_start, kimg_shadow_end, early_pfn_to_nid(virt_to_pfn(lm_alias(_text)))); - kasan_populate_zero_shadow((void *)KASAN_SHADOW_START, - (void *)mod_shadow_start); - kasan_populate_zero_shadow((void *)kimg_shadow_end, - kasan_mem_to_shadow((void *)PAGE_OFFSET)); + kasan_populate_early_shadow((void *)KASAN_SHADOW_START, + (void *)mod_shadow_start); + kasan_populate_early_shadow((void *)kimg_shadow_end, + kasan_mem_to_shadow((void *)PAGE_OFFSET)); if (kimg_shadow_start > mod_shadow_end) - kasan_populate_zero_shadow((void *)mod_shadow_end, - (void *)kimg_shadow_start); + kasan_populate_early_shadow((void *)mod_shadow_end, + (void *)kimg_shadow_start); for_each_memblock(memory, reg) { void *start = (void *)__phys_to_virt(reg->base); @@ -227,14 +231,15 @@ void __init kasan_init(void) } /* - * KAsan may reuse the contents of kasan_zero_pte directly, so we - * should make sure that it maps the zero page read-only. + * KAsan may reuse the contents of kasan_early_shadow_pte directly, + * so we should make sure that it maps the zero page read-only. */ for (i = 0; i < PTRS_PER_PTE; i++) - set_pte(&kasan_zero_pte[i], - pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO)); + set_pte(&kasan_early_shadow_pte[i], + pfn_pte(sym_to_pfn(kasan_early_shadow_page), + PAGE_KERNEL_RO)); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/arch/s390/mm/dump_pagetables.c b/arch/s390/mm/dump_pagetables.c index 363f6470d742..3b93ba0b5d8d 100644 --- a/arch/s390/mm/dump_pagetables.c +++ b/arch/s390/mm/dump_pagetables.c @@ -111,11 +111,12 @@ static void note_page(struct seq_file *m, struct pg_state *st, } #ifdef CONFIG_KASAN -static void note_kasan_zero_page(struct seq_file *m, struct pg_state *st) +static void note_kasan_early_shadow_page(struct seq_file *m, + struct pg_state *st) { unsigned int prot; - prot = pte_val(*kasan_zero_pte) & + prot = pte_val(*kasan_early_shadow_pte) & (_PAGE_PROTECT | _PAGE_INVALID | _PAGE_NOEXEC); note_page(m, st, prot, 4); } @@ -154,8 +155,8 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_zero_pmd)) { - note_kasan_zero_page(m, st); + if ((pud_val(*pud) & PAGE_MASK) == __pa(kasan_early_shadow_pmd)) { + note_kasan_early_shadow_page(m, st); return; } #endif @@ -185,8 +186,8 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_zero_pud)) { - note_kasan_zero_page(m, st); + if ((p4d_val(*p4d) & PAGE_MASK) == __pa(kasan_early_shadow_pud)) { + note_kasan_early_shadow_page(m, st); return; } #endif @@ -215,8 +216,8 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, int i; #ifdef CONFIG_KASAN - if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_zero_p4d)) { - note_kasan_zero_page(m, st); + if ((pgd_val(*pgd) & PAGE_MASK) == __pa(kasan_early_shadow_p4d)) { + note_kasan_early_shadow_page(m, st); return; } #endif diff --git a/arch/s390/mm/kasan_init.c b/arch/s390/mm/kasan_init.c index acb9645b762b..bac5c27d11fc 100644 --- a/arch/s390/mm/kasan_init.c +++ b/arch/s390/mm/kasan_init.c @@ -107,7 +107,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PGDIR_SIZE) && end - address >= PGDIR_SIZE) { - pgd_populate(&init_mm, pg_dir, kasan_zero_p4d); + pgd_populate(&init_mm, pg_dir, + kasan_early_shadow_p4d); address = (address + PGDIR_SIZE) & PGDIR_MASK; continue; } @@ -120,7 +121,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, P4D_SIZE) && end - address >= P4D_SIZE) { - p4d_populate(&init_mm, p4_dir, kasan_zero_pud); + p4d_populate(&init_mm, p4_dir, + kasan_early_shadow_pud); address = (address + P4D_SIZE) & P4D_MASK; continue; } @@ -133,7 +135,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PUD_SIZE) && end - address >= PUD_SIZE) { - pud_populate(&init_mm, pu_dir, kasan_zero_pmd); + pud_populate(&init_mm, pu_dir, + kasan_early_shadow_pmd); address = (address + PUD_SIZE) & PUD_MASK; continue; } @@ -146,7 +149,8 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, if (mode == POPULATE_ZERO_SHADOW && IS_ALIGNED(address, PMD_SIZE) && end - address >= PMD_SIZE) { - pmd_populate(&init_mm, pm_dir, kasan_zero_pte); + pmd_populate(&init_mm, pm_dir, + kasan_early_shadow_pte); address = (address + PMD_SIZE) & PMD_MASK; continue; } @@ -188,7 +192,7 @@ static void __init kasan_early_vmemmap_populate(unsigned long address, pte_val(*pt_dir) = __pa(page) | pgt_prot; break; case POPULATE_ZERO_SHADOW: - page = kasan_zero_page; + page = kasan_early_shadow_page; pte_val(*pt_dir) = __pa(page) | pgt_prot_zero; break; } @@ -256,14 +260,14 @@ void __init kasan_early_init(void) unsigned long vmax; unsigned long pgt_prot = pgprot_val(PAGE_KERNEL_RO); pte_t pte_z; - pmd_t pmd_z = __pmd(__pa(kasan_zero_pte) | _SEGMENT_ENTRY); - pud_t pud_z = __pud(__pa(kasan_zero_pmd) | _REGION3_ENTRY); - p4d_t p4d_z = __p4d(__pa(kasan_zero_pud) | _REGION2_ENTRY); + pmd_t pmd_z = __pmd(__pa(kasan_early_shadow_pte) | _SEGMENT_ENTRY); + pud_t pud_z = __pud(__pa(kasan_early_shadow_pmd) | _REGION3_ENTRY); + p4d_t p4d_z = __p4d(__pa(kasan_early_shadow_pud) | _REGION2_ENTRY); kasan_early_detect_facilities(); if (!has_nx) pgt_prot &= ~_PAGE_NOEXEC; - pte_z = __pte(__pa(kasan_zero_page) | pgt_prot); + pte_z = __pte(__pa(kasan_early_shadow_page) | pgt_prot); memsize = get_mem_detect_end(); if (!memsize) @@ -292,10 +296,13 @@ void __init kasan_early_init(void) } /* init kasan zero shadow */ - crst_table_init((unsigned long *)kasan_zero_p4d, p4d_val(p4d_z)); - crst_table_init((unsigned long *)kasan_zero_pud, pud_val(pud_z)); - crst_table_init((unsigned long *)kasan_zero_pmd, pmd_val(pmd_z)); - memset64((u64 *)kasan_zero_pte, pte_val(pte_z), PTRS_PER_PTE); + crst_table_init((unsigned long *)kasan_early_shadow_p4d, + p4d_val(p4d_z)); + crst_table_init((unsigned long *)kasan_early_shadow_pud, + pud_val(pud_z)); + crst_table_init((unsigned long *)kasan_early_shadow_pmd, + pmd_val(pmd_z)); + memset64((u64 *)kasan_early_shadow_pte, pte_val(pte_z), PTRS_PER_PTE); shadow_alloc_size = memsize >> KASAN_SHADOW_SCALE_SHIFT; pgalloc_low = round_up((unsigned long)_end, _SEGMENT_SIZE); diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index fc37bbd23eb8..c4696ab9a72b 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -380,7 +380,7 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, /* * This is an optimization for KASAN=y case. Since all kasan page tables - * eventually point to the kasan_zero_page we could call note_page() + * eventually point to the kasan_early_shadow_page we could call note_page() * right away without walking through lower level page tables. This saves * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. @@ -388,10 +388,11 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, void *pt) { - if (__pa(pt) == __pa(kasan_zero_pmd) || - (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_zero_p4d)) || - __pa(pt) == __pa(kasan_zero_pud)) { - pgprotval_t prot = pte_flags(kasan_zero_pte[0]); + if (__pa(pt) == __pa(kasan_early_shadow_pmd) || + (pgtable_l5_enabled() && + __pa(pt) == __pa(kasan_early_shadow_p4d)) || + __pa(pt) == __pa(kasan_early_shadow_pud)) { + pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); note_page(m, st, __pgprot(prot), 0, 5); return true; } diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 04a9cf6b034f..462fde83b515 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -211,7 +211,8 @@ static void __init kasan_early_p4d_populate(pgd_t *pgd, unsigned long next; if (pgd_none(*pgd)) { - pgd_entry = __pgd(_KERNPG_TABLE | __pa_nodebug(kasan_zero_p4d)); + pgd_entry = __pgd(_KERNPG_TABLE | + __pa_nodebug(kasan_early_shadow_p4d)); set_pgd(pgd, pgd_entry); } @@ -222,7 +223,8 @@ static void __init kasan_early_p4d_populate(pgd_t *pgd, if (!p4d_none(*p4d)) continue; - p4d_entry = __p4d(_KERNPG_TABLE | __pa_nodebug(kasan_zero_pud)); + p4d_entry = __p4d(_KERNPG_TABLE | + __pa_nodebug(kasan_early_shadow_pud)); set_p4d(p4d, p4d_entry); } while (p4d++, addr = next, addr != end && p4d_none(*p4d)); } @@ -261,10 +263,11 @@ static struct notifier_block kasan_die_notifier = { void __init kasan_early_init(void) { int i; - pteval_t pte_val = __pa_nodebug(kasan_zero_page) | __PAGE_KERNEL | _PAGE_ENC; - pmdval_t pmd_val = __pa_nodebug(kasan_zero_pte) | _KERNPG_TABLE; - pudval_t pud_val = __pa_nodebug(kasan_zero_pmd) | _KERNPG_TABLE; - p4dval_t p4d_val = __pa_nodebug(kasan_zero_pud) | _KERNPG_TABLE; + pteval_t pte_val = __pa_nodebug(kasan_early_shadow_page) | + __PAGE_KERNEL | _PAGE_ENC; + pmdval_t pmd_val = __pa_nodebug(kasan_early_shadow_pte) | _KERNPG_TABLE; + pudval_t pud_val = __pa_nodebug(kasan_early_shadow_pmd) | _KERNPG_TABLE; + p4dval_t p4d_val = __pa_nodebug(kasan_early_shadow_pud) | _KERNPG_TABLE; /* Mask out unsupported __PAGE_KERNEL bits: */ pte_val &= __default_kernel_pte_mask; @@ -273,16 +276,16 @@ void __init kasan_early_init(void) p4d_val &= __default_kernel_pte_mask; for (i = 0; i < PTRS_PER_PTE; i++) - kasan_zero_pte[i] = __pte(pte_val); + kasan_early_shadow_pte[i] = __pte(pte_val); for (i = 0; i < PTRS_PER_PMD; i++) - kasan_zero_pmd[i] = __pmd(pmd_val); + kasan_early_shadow_pmd[i] = __pmd(pmd_val); for (i = 0; i < PTRS_PER_PUD; i++) - kasan_zero_pud[i] = __pud(pud_val); + kasan_early_shadow_pud[i] = __pud(pud_val); for (i = 0; pgtable_l5_enabled() && i < PTRS_PER_P4D; i++) - kasan_zero_p4d[i] = __p4d(p4d_val); + kasan_early_shadow_p4d[i] = __p4d(p4d_val); kasan_map_early_shadow(early_top_pgt); kasan_map_early_shadow(init_top_pgt); @@ -326,7 +329,7 @@ void __init kasan_init(void) clear_pgds(KASAN_SHADOW_START & PGDIR_MASK, KASAN_SHADOW_END); - kasan_populate_zero_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK), + kasan_populate_early_shadow((void *)(KASAN_SHADOW_START & PGDIR_MASK), kasan_mem_to_shadow((void *)PAGE_OFFSET)); for (i = 0; i < E820_MAX_ENTRIES; i++) { @@ -338,41 +341,41 @@ void __init kasan_init(void) shadow_cpu_entry_begin = (void *)CPU_ENTRY_AREA_BASE; shadow_cpu_entry_begin = kasan_mem_to_shadow(shadow_cpu_entry_begin); - shadow_cpu_entry_begin = (void *)round_down((unsigned long)shadow_cpu_entry_begin, - PAGE_SIZE); + shadow_cpu_entry_begin = (void *)round_down( + (unsigned long)shadow_cpu_entry_begin, PAGE_SIZE); shadow_cpu_entry_end = (void *)(CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE); shadow_cpu_entry_end = kasan_mem_to_shadow(shadow_cpu_entry_end); - shadow_cpu_entry_end = (void *)round_up((unsigned long)shadow_cpu_entry_end, - PAGE_SIZE); + shadow_cpu_entry_end = (void *)round_up( + (unsigned long)shadow_cpu_entry_end, PAGE_SIZE); - kasan_populate_zero_shadow( + kasan_populate_early_shadow( kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM), shadow_cpu_entry_begin); kasan_populate_shadow((unsigned long)shadow_cpu_entry_begin, (unsigned long)shadow_cpu_entry_end, 0); - kasan_populate_zero_shadow(shadow_cpu_entry_end, - kasan_mem_to_shadow((void *)__START_KERNEL_map)); + kasan_populate_early_shadow(shadow_cpu_entry_end, + kasan_mem_to_shadow((void *)__START_KERNEL_map)); kasan_populate_shadow((unsigned long)kasan_mem_to_shadow(_stext), (unsigned long)kasan_mem_to_shadow(_end), early_pfn_to_nid(__pa(_stext))); - kasan_populate_zero_shadow(kasan_mem_to_shadow((void *)MODULES_END), - (void *)KASAN_SHADOW_END); + kasan_populate_early_shadow(kasan_mem_to_shadow((void *)MODULES_END), + (void *)KASAN_SHADOW_END); load_cr3(init_top_pgt); __flush_tlb_all(); /* - * kasan_zero_page has been used as early shadow memory, thus it may - * contain some garbage. Now we can clear and write protect it, since - * after the TLB flush no one should write to it. + * kasan_early_shadow_page has been used as early shadow memory, thus + * it may contain some garbage. Now we can clear and write protect it, + * since after the TLB flush no one should write to it. */ - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); for (i = 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; @@ -380,8 +383,8 @@ void __init kasan_init(void) prot = __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC); pgprot_val(prot) &= __default_kernel_pte_mask; - pte = __pte(__pa(kasan_zero_page) | pgprot_val(prot)); - set_pte(&kasan_zero_pte[i], pte); + pte = __pte(__pa(kasan_early_shadow_page) | pgprot_val(prot)); + set_pte(&kasan_early_shadow_pte[i], pte); } /* Flush TLBs again to be sure that write protection applied. */ __flush_tlb_all(); diff --git a/arch/xtensa/mm/kasan_init.c b/arch/xtensa/mm/kasan_init.c index 6b95ca43aec0..1734cda6bc4a 100644 --- a/arch/xtensa/mm/kasan_init.c +++ b/arch/xtensa/mm/kasan_init.c @@ -24,12 +24,13 @@ void __init kasan_early_init(void) int i; for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_zero_pte + i, - mk_pte(virt_to_page(kasan_zero_page), PAGE_KERNEL)); + set_pte(kasan_early_shadow_pte + i, + mk_pte(virt_to_page(kasan_early_shadow_page), + PAGE_KERNEL)); for (vaddr = 0; vaddr < KASAN_SHADOW_SIZE; vaddr += PMD_SIZE, ++pmd) { BUG_ON(!pmd_none(*pmd)); - set_pmd(pmd, __pmd((unsigned long)kasan_zero_pte)); + set_pmd(pmd, __pmd((unsigned long)kasan_early_shadow_pte)); } early_trap_init(); } @@ -80,13 +81,16 @@ void __init kasan_init(void) populate(kasan_mem_to_shadow((void *)VMALLOC_START), kasan_mem_to_shadow((void *)XCHAL_KSEG_BYPASS_VADDR)); - /* Write protect kasan_zero_page and zero-initialize it again. */ + /* + * Write protect kasan_early_shadow_page and zero-initialize it again. + */ for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_zero_pte + i, - mk_pte(virt_to_page(kasan_zero_page), PAGE_KERNEL_RO)); + set_pte(kasan_early_shadow_pte + i, + mk_pte(virt_to_page(kasan_early_shadow_page), + PAGE_KERNEL_RO)); local_flush_tlb_all(); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, 0, PAGE_SIZE); /* At this point kasan is fully initialized. Enable error messages. */ current->kasan_depth = 0; diff --git a/include/linux/kasan.h b/include/linux/kasan.h index b66fdf5ea7ab..ec22d548d0d7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -14,13 +14,13 @@ struct task_struct; #include #include -extern unsigned char kasan_zero_page[PAGE_SIZE]; -extern pte_t kasan_zero_pte[PTRS_PER_PTE]; -extern pmd_t kasan_zero_pmd[PTRS_PER_PMD]; -extern pud_t kasan_zero_pud[PTRS_PER_PUD]; -extern p4d_t kasan_zero_p4d[MAX_PTRS_PER_P4D]; +extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; +extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE]; +extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD]; +extern pud_t kasan_early_shadow_pud[PTRS_PER_PUD]; +extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; -int kasan_populate_zero_shadow(const void *shadow_start, +int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); static inline void *kasan_mem_to_shadow(const void *addr) diff --git a/mm/kasan/init.c b/mm/kasan/init.c index c7550eb65922..2b21d3717d62 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -30,13 +30,13 @@ * - Latter it reused it as zero shadow to cover large ranges of memory * that allowed to access, but not handled by kasan (vmalloc/vmemmap ...). */ -unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; +unsigned char kasan_early_shadow_page[PAGE_SIZE] __page_aligned_bss; #if CONFIG_PGTABLE_LEVELS > 4 -p4d_t kasan_zero_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; +p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D] __page_aligned_bss; static inline bool kasan_p4d_table(pgd_t pgd) { - return pgd_page(pgd) == virt_to_page(lm_alias(kasan_zero_p4d)); + return pgd_page(pgd) == virt_to_page(lm_alias(kasan_early_shadow_p4d)); } #else static inline bool kasan_p4d_table(pgd_t pgd) @@ -45,10 +45,10 @@ static inline bool kasan_p4d_table(pgd_t pgd) } #endif #if CONFIG_PGTABLE_LEVELS > 3 -pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; +pud_t kasan_early_shadow_pud[PTRS_PER_PUD] __page_aligned_bss; static inline bool kasan_pud_table(p4d_t p4d) { - return p4d_page(p4d) == virt_to_page(lm_alias(kasan_zero_pud)); + return p4d_page(p4d) == virt_to_page(lm_alias(kasan_early_shadow_pud)); } #else static inline bool kasan_pud_table(p4d_t p4d) @@ -57,10 +57,10 @@ static inline bool kasan_pud_table(p4d_t p4d) } #endif #if CONFIG_PGTABLE_LEVELS > 2 -pmd_t kasan_zero_pmd[PTRS_PER_PMD] __page_aligned_bss; +pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD] __page_aligned_bss; static inline bool kasan_pmd_table(pud_t pud) { - return pud_page(pud) == virt_to_page(lm_alias(kasan_zero_pmd)); + return pud_page(pud) == virt_to_page(lm_alias(kasan_early_shadow_pmd)); } #else static inline bool kasan_pmd_table(pud_t pud) @@ -68,16 +68,16 @@ static inline bool kasan_pmd_table(pud_t pud) return 0; } #endif -pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; +pte_t kasan_early_shadow_pte[PTRS_PER_PTE] __page_aligned_bss; static inline bool kasan_pte_table(pmd_t pmd) { - return pmd_page(pmd) == virt_to_page(lm_alias(kasan_zero_pte)); + return pmd_page(pmd) == virt_to_page(lm_alias(kasan_early_shadow_pte)); } -static inline bool kasan_zero_page_entry(pte_t pte) +static inline bool kasan_early_shadow_page_entry(pte_t pte) { - return pte_page(pte) == virt_to_page(lm_alias(kasan_zero_page)); + return pte_page(pte) == virt_to_page(lm_alias(kasan_early_shadow_page)); } static __init void *early_alloc(size_t size, int node) @@ -92,7 +92,8 @@ static void __ref zero_pte_populate(pmd_t *pmd, unsigned long addr, pte_t *pte = pte_offset_kernel(pmd, addr); pte_t zero_pte; - zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_zero_page)), PAGE_KERNEL); + zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_early_shadow_page)), + PAGE_KERNEL); zero_pte = pte_wrprotect(zero_pte); while (addr + PAGE_SIZE <= end) { @@ -112,7 +113,8 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr, next = pmd_addr_end(addr, end); if (IS_ALIGNED(addr, PMD_SIZE) && end - addr >= PMD_SIZE) { - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -145,9 +147,11 @@ static int __ref zero_pud_populate(p4d_t *p4d, unsigned long addr, if (IS_ALIGNED(addr, PUD_SIZE) && end - addr >= PUD_SIZE) { pmd_t *pmd; - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -181,12 +185,14 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, pud_t *pud; pmd_t *pmd; - p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); + p4d_populate(&init_mm, p4d, + lm_alias(kasan_early_shadow_pud)); pud = pud_offset(p4d, addr); - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); pmd_populate_kernel(&init_mm, pmd, - lm_alias(kasan_zero_pte)); + lm_alias(kasan_early_shadow_pte)); continue; } @@ -209,13 +215,13 @@ static int __ref zero_p4d_populate(pgd_t *pgd, unsigned long addr, } /** - * kasan_populate_zero_shadow - populate shadow memory region with - * kasan_zero_page + * kasan_populate_early_shadow - populate shadow memory region with + * kasan_early_shadow_page * @shadow_start - start of the memory range to populate * @shadow_end - end of the memory range to populate */ -int __ref kasan_populate_zero_shadow(const void *shadow_start, - const void *shadow_end) +int __ref kasan_populate_early_shadow(const void *shadow_start, + const void *shadow_end) { unsigned long addr = (unsigned long)shadow_start; unsigned long end = (unsigned long)shadow_end; @@ -231,7 +237,7 @@ int __ref kasan_populate_zero_shadow(const void *shadow_start, pmd_t *pmd; /* - * kasan_zero_pud should be populated with pmds + * kasan_early_shadow_pud should be populated with pmds * at this moment. * [pud,pmd]_populate*() below needed only for * 3,2 - level page tables where we don't have @@ -241,21 +247,25 @@ int __ref kasan_populate_zero_shadow(const void *shadow_start, * The ifndef is required to avoid build breakage. * * With 5level-fixup.h, pgd_populate() is not nop and - * we reference kasan_zero_p4d. It's not defined + * we reference kasan_early_shadow_p4d. It's not defined * unless 5-level paging enabled. * * The ifndef can be dropped once all KASAN-enabled * architectures will switch to pgtable-nop4d.h. */ #ifndef __ARCH_HAS_5LEVEL_HACK - pgd_populate(&init_mm, pgd, lm_alias(kasan_zero_p4d)); + pgd_populate(&init_mm, pgd, + lm_alias(kasan_early_shadow_p4d)); #endif p4d = p4d_offset(pgd, addr); - p4d_populate(&init_mm, p4d, lm_alias(kasan_zero_pud)); + p4d_populate(&init_mm, p4d, + lm_alias(kasan_early_shadow_pud)); pud = pud_offset(p4d, addr); - pud_populate(&init_mm, pud, lm_alias(kasan_zero_pmd)); + pud_populate(&init_mm, pud, + lm_alias(kasan_early_shadow_pmd)); pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, lm_alias(kasan_zero_pte)); + pmd_populate_kernel(&init_mm, pmd, + lm_alias(kasan_early_shadow_pte)); continue; } @@ -350,7 +360,7 @@ static void kasan_remove_pte_table(pte_t *pte, unsigned long addr, if (!pte_present(*pte)) continue; - if (WARN_ON(!kasan_zero_page_entry(*pte))) + if (WARN_ON(!kasan_early_shadow_page_entry(*pte))) continue; pte_clear(&init_mm, addr, pte); } @@ -480,7 +490,7 @@ int kasan_add_zero_shadow(void *start, unsigned long size) WARN_ON(size % (KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE))) return -EINVAL; - ret = kasan_populate_zero_shadow(shadow_start, shadow_end); + ret = kasan_populate_early_shadow(shadow_start, shadow_end); if (ret) kasan_remove_zero_shadow(shadow_start, size >> KASAN_SHADOW_SCALE_SHIFT); From patchwork Thu Dec 6 12:24:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715763 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 55B52109C for ; Thu, 6 Dec 2018 12:25:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 44996292C8 for ; Thu, 6 Dec 2018 12:25:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 37A0F2BBFB; Thu, 6 Dec 2018 12:25:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B485A292C8 for ; Thu, 6 Dec 2018 12:25:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D5EC6B79FB; Thu, 6 Dec 2018 07:25:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 937356B7A01; Thu, 6 Dec 2018 07:25:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DC306B7A02; Thu, 6 Dec 2018 07:25:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id 033D36B79FB for ; Thu, 6 Dec 2018 07:25:03 -0500 (EST) Received: by mail-wm1-f71.google.com with SMTP id 1so203878wms.9 for ; Thu, 06 Dec 2018 04:25:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=G5Ka/D9ThH8pnE4eoPJ0OLfzNTsBwnEIL720J8sMTz0=; b=FlCneCb3znUnNhnmEN+THzNaeYWN5niChE/XPr3TvSH1LLvX3qWVT6b6pz8Ubwq6tU kLgD/rGhQ/hIdkxax5ynFIxJ1tPq515BZr6OcO6BZ9fQWC2ctVMzlwd6vzBdKHEqKl2X Fhxur20c3jKNi8k3wFi/j25gmDWz//5gnLT+dlHnomeBhPW5e33Y2IEWPKpeWBw5quTD LOkB+eS0XbvUhLx3Rlrwij4SoTMRuO3c36bKbGG/G0mKtJfRttNqBknIeM9VqYaRl20d ES4ldZYKK16ohjjcva14+dg97sjNUj7eJmQ3AwvCLYi52sviJG0FuGF/2yS61PyzSFX0 sn9A== X-Gm-Message-State: AA+aEWbD4B7dbrScRGIcko2WXirFST+7faM2KhCOspMbCu8LEiNEQbXH H9esQtSWvTqCQ4asCLN6kF1iYfheBRt1STuFy96G5Y3+QWSiKOgG8eFkZ9PukDKQax+uZflQAQ8 gzPo40K/OlRo7et7l6PcrVcza+TlTDspcebxEawUE3oudlWXPGKoNUJ1k2nehNWAvZLUGN/cNSV O9afuLflrwqdpegFuZUn+MiEGuyFN1kIcdjkw8wNjkEeSl99JD741ZFkhfSVOU7MTTfPKlXTuzI UwFuXvjiX4ndw8UXcj84YjhVAsqJ4o3666UdCU1KwYAzXxGopp0T3AdMuGajr7kBff/gBBO8Zxt XL7hxUXqjpZVynpHPlgt/K//nCwNDYZkGWxH+zbxnhTqLa4ozWv89+gC1FpffehkJejvGJYGpKZ n X-Received: by 2002:a1c:864f:: with SMTP id i76mr15385683wmd.83.1544099102530; Thu, 06 Dec 2018 04:25:02 -0800 (PST) X-Received: by 2002:a1c:864f:: with SMTP id i76mr15385625wmd.83.1544099101544; Thu, 06 Dec 2018 04:25:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099101; cv=none; d=google.com; s=arc-20160816; b=0OCA33TBdttUe2anb6fkFwCG1tNGZiYf8Kui9CxMnKxTGvSKhAaJh0r7gVGlCriYbj A0KRobzti0IFc1odhkGD1nCnwjqubpexqg0J7nDsyZykFm2Ufl9DdB8FTDSRoDU0vuWR wduIYG6Kisl8K+zK+TTtPSW15/mHzQVVsl0mN8u/ltJiIqcOVdILj9yTeQTKa1exBsu7 bQIE2DvYRfh8HtQXSb4e8DminqgMy5PnShWAr7CoAQDpvu3ohOq3Qxdt8UExAVyNrEmd akcmkKaIPb3y3i3J9izgDTQlSdhJgjWM/mquPX9T7NjFebZBcfXKq4q/Ew5Epypv1F2/ Yq4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=G5Ka/D9ThH8pnE4eoPJ0OLfzNTsBwnEIL720J8sMTz0=; b=WM991UNbDgL2dqg0ETj34S+43tO9yyYomPTys/0at8V6bDBIMGQsGtn8lkEAIMSCux wkJOPgWmOSr/SYhMD0s2PV1r9TO6wHnjpKGrp0WNhLmAQVfvoRAPd5/87WBU3b8GxlkY rO+exgleN+qWb1a2lNR93Ly5Mm5tjEd2jVQMo8afUrbfLyTA2KsO10iIGsqCDBj/CTAC YGTebLfM4WoFgIrh388fbTKX3gpkAbO0CVro57jaN+2wWsdBhqC4Zq6t837hX9iY1hFV OxIEd6WDZk0IZ12hh3ip2tkh72eybBoQHWe7TZjCLAeJMIi/X3ExXw6PKQOyc61NhJZo wQFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=dhrVAvtp; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b125sor446384wmd.19.2018.12.06.04.25.01 for (Google Transport Security); Thu, 06 Dec 2018 04:25:01 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=dhrVAvtp; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G5Ka/D9ThH8pnE4eoPJ0OLfzNTsBwnEIL720J8sMTz0=; b=dhrVAvtpao4EYg7FPVcAkM1wtgiCBBI9ASJhoyX/vv+zvJ5zk95ov0eIQZZadClZMy C9GuvdBwcXqwqHElxuDXEIz889yekq6iNXAe7oMkWsjIBsJ9rt8EwF02GzX729gCYrKI ynUxvY44E8YwCyPeGFjutDX/Ka4dGW+JpBDkXy6pmjwo/Rq5KY/j3JJ5r3kwmG4hQNSM 4stTjLLbbsYQYnRqF3a4u1Q0q/jojd03Tl+wDvTna6jK/WpUnbO2YeYOwxaFWZG0Yj1N LhSyyeYMWO8CsVUWLNmXcxBL3cZLwTNssIUkFccRsslhrLDvpgG3oyko66ErTK7fQUUV ZqYw== X-Google-Smtp-Source: AFSGD/XJ8zcmMWXD8FWzGxaH3vNIgkyiPtwnihpR7W2ZVOr16EQd1yHbYp8lY9MsaH6CEj9t73DTeA== X-Received: by 2002:a1c:8d49:: with SMTP id p70mr1774408wmd.68.1544099101009; Thu, 06 Dec 2018 04:25:01 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:00 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 08/25] kasan: initialize shadow to 0xff for tag-based mode Date: Thu, 6 Dec 2018 13:24:26 +0100 Message-Id: <5cc1b789aad7c99cf4f3ec5b328b147ad53edb40.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A tag-based KASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with tag-based KASAN we need to initialize shadow memory to 0xff. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov Reviewed-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
--- arch/arm64/mm/kasan_init.c | 15 +++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 4ebc19422931..7a4a0904cac8 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -43,6 +43,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) return __pa(p); } +static phys_addr_t __init kasan_alloc_raw_page(int node) +{ + void *p = memblock_alloc_try_nid_raw(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, + node); + return __pa(p); +} + static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { @@ -92,7 +101,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, do { phys_addr_t page_phys = early ? __pa_symbol(kasan_early_shadow_page) - : kasan_alloc_zeroed_page(node); + : kasan_alloc_raw_page(node); + if (!early) + memset(__va(page_phys), KASAN_SHADOW_INIT, PAGE_SIZE); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -239,7 +250,7 @@ void __init kasan_init(void) pfn_pte(sym_to_pfn(kasan_early_shadow_page), PAGE_KERNEL_RO)); - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index ec22d548d0d7..c56af24bd3e7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -153,6 +153,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 + void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); @@ -163,4 +165,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_SW_TAGS */ + #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 5f68c93734ba..7134e75447ff 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -473,11 +473,12 @@ int kasan_module_alloc(void *addr, size_t size) ret = __vmalloc_node_range(shadow_size, 1, shadow_start, shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, + GFP_KERNEL, PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, __builtin_return_address(0)); if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); find_vm_area(addr)->flags |= VM_KASAN; kmemleak_ignore(ret); return 0; From patchwork Thu Dec 6 12:24:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715765 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A36491731 for ; Thu, 6 Dec 2018 12:25:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 92002292C8 for ; Thu, 6 Dec 2018 12:25:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 85A9C2D325; Thu, 6 Dec 2018 12:25:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58AAE297D0 for ; Thu, 6 Dec 2018 12:25:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3220F6B7A01; Thu, 6 Dec 2018 07:25:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 25B026B7A03; Thu, 6 Dec 2018 07:25:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0ACF66B7A04; Thu, 6 Dec 2018 07:25:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id A1C486B7A01 for ; Thu, 6 Dec 2018 07:25:04 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id x3so70113wru.22 for ; Thu, 06 Dec 2018 04:25:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=F/Rs74E1jKAK3ACh/wE42WMWNbPFeAMFJIPcMbt9WD4=; b=kx9LKu5dJ6JW1krfVjY1x9n618TWK/neJCYmLs6SZNqUXco7EZT05lFyUKGJtPNFvs NmkTibRcIi22+tVPeFrqzMHVN0IiODPp/TSDBFHU5SZyKx1QK9jyXoGnwOLe681f1840 CvU5ET14VMpnB51D42qPePmmAnAAMcWF0zfqFUNKG/PGCLjBjdl1yKQ/fwaKSMT6fcMF D4UImKuVEZRAcXr5bN4nmHcD08+Fehry+LIqELNxIe5o28CaLLa01p2HbaKdaB3lUVQX lNKwFSyan0pwT8+vJtyrqEwVQTOJ6SuAMGg9Jqhnws7CzT1dGtF6gJZcTjOiVGkvTFpv fYfQ== X-Gm-Message-State: AA+aEWYR4bX8QlpYsHTKiRvJ6LpslPbIa7Llq5z4QzsoLz1gOSPHiX6A 4jBokXcaVXX9jogVoqflyTKlZz9jSswIi3x2/DP0OvGqJFHEBYEBR5jLmKRIzcCnsyhDKIZHRNP xst/6FXDV2KL1h2TbUAWgKfLgos8ZWnWXvA/+6PhzvN/Pj1Wd3URS3I4Z5se+K69We/CL9ShKoJ j78p18RlEJ/ujgc1ygfRkHyi87/2dJNdCCCPZd0IQZCFF+FHU6zdA5Ta2itvlRxhwtXHhEhKz5K pLLd+gpNRFgOpx+JGJM0UYWjZMsHlvwXtfFsRfNPJr3DFh+gkPAZS/Jrnur/ZgwrYniNSwtc1Pc AAW3LrkOms6vxU/7DHpzWDqC2OxtFmbB+Ola5alXNdwBdoC7xLXyJMk+66Agp6H1AGNfoIrgnC5 Z X-Received: by 2002:a5d:5502:: with SMTP id b2mr25938773wrv.330.1544099104186; Thu, 06 Dec 2018 04:25:04 -0800 (PST) X-Received: by 2002:a5d:5502:: with SMTP id b2mr25938711wrv.330.1544099103289; Thu, 06 Dec 2018 04:25:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099103; cv=none; d=google.com; s=arc-20160816; b=qS+f5euD22Hcfnc3dleZerT8Xi05q+F+aq4+jYhDpksGnI5fOEZIP02BhQklUO7pi0 vlkb/1TPDFpUhFjlJqnyBRppeCQq9UP4DiGnYsapGlbwSyV9DytWvJsb/3DV22gjj6Vm GsaeJQNdLXx/0t0wvcb45hv1zFf2V9hBTRwvE3DHmLuB2xVvJPmg8mpjCpIplV97W2OE sDx6GdZAYsBvtH8sWgyp/hluKwe9yghyJfS7Kt3FUo8LBcRy7X/E0BynRD8ejWrfGlK3 X7EcDIIjAJPpZLRWuzth/fOPryFExPjrmBkmkTR8+p3EIRXI02TahViJ9kL5PH3wKnwE gFow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=F/Rs74E1jKAK3ACh/wE42WMWNbPFeAMFJIPcMbt9WD4=; b=QnZYbU3dSO9RxGoQV0WJHzkzfgRSNU4+wh2DdVr2zj1uZbNGUoymIivS+iOi3PzOXS 98wOxzMjuGxbrhDV+LQ8nBOk1jsrbDvWv6q3Kg+B2FONIZIWKx75bbt5aZptMztIAkp1 tdAWIrwR8CoRBBCaiGBS0oUalW0pW+vXOR7n6a4/l+G4jBaNdk3+7XP2WLh2hHigV4PC hJB27k57WCAIbO2b/YUXY00pq2n/fb6Lqsl8IbvKuY4dSJ8osGfcaMzoAQ34UiTOOCzu 5Nlt1vI6EGFeK4wnAMcqOMva1ZBIA8BD1Id79tUUqIsbYyiSOzOiyigD0WQml3i9Aseb mY5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VAlGzqot; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l13sor178206wre.39.2018.12.06.04.25.03 for (Google Transport Security); Thu, 06 Dec 2018 04:25:03 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VAlGzqot; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=F/Rs74E1jKAK3ACh/wE42WMWNbPFeAMFJIPcMbt9WD4=; b=VAlGzqot4KggH/2lBiRkMM4lITHs5IpRZ4p+F4SbO+qt098Hg0g4wlLA22lQa2+sgu 1ZO/pc5wB2nDiSRi5FFQsNwlSmKERZzC6+C1bm7VRPFwLoKQrSId434VtYvpzCsk7c3i /vH7cIfPKbr+V1Dilr9fKTxodHnjd41ZTfyz597HV372In0cpd9mvAmlNFP+HJxA+AQ+ mzStQ2w6DGPYRDzykQjqYtFlASCb4Bva887fDY6JzKFwMetT4nD71IbFlWztZ4E/AIat lVr7320saaRT4uROOn2botn655EPyWEYbn4KMSnBzLpXEAJMIWiYQJli7yQdCgLhi50x ffvA== X-Google-Smtp-Source: AFSGD/XTYTqzsshlDGi9NHkEBAbQ2NsOtZrsn1Z0Y2zjYz0lfHZbxea+X0lOoDWgzEFhq8Dhw9ARjA== X-Received: by 2002:adf:b783:: with SMTP id s3mr26257157wre.274.1544099102810; Thu, 06 Dec 2018 04:25:02 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:01 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 09/25] arm64: move untagged_addr macro from uaccess.h to memory.h Date: Thu, 6 Dec 2018 13:24:27 +0100 Message-Id: <2e9ef8d2ed594106eca514b268365b5419113f6a.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Move the untagged_addr() macro from arch/arm64/include/asm/uaccess.h to arch/arm64/include/asm/memory.h to be later reused by KASAN. Also make the untagged_addr() macro accept all kinds of address types (void *, unsigned long, etc.). This allows not to specify type casts in each place where the macro is used. This is done by using __typeof__. Acked-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 8 ++++++++ arch/arm64/include/asm/uaccess.h | 7 ------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 05fbc7ffcd31..e2c9857157f2 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -211,6 +211,14 @@ static inline unsigned long kaslr_offset(void) */ #define PHYS_PFN_OFFSET (PHYS_OFFSET >> PAGE_SHIFT) +/* + * When dealing with data aborts, watchpoints, or instruction traps we may end + * up with a tagged userland pointer. Clear the tag to get a sane pointer to + * pass on to access_ok(), for instance. + */ +#define untagged_addr(addr) \ + ((__typeof__(addr))sign_extend64((u64)(addr), 55)) + /* * Physical vs virtual RAM address space conversion. These are * private definitions which should NOT be used outside memory.h diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 07c34087bd5e..281a1e47263d 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -96,13 +96,6 @@ static inline unsigned long __range_ok(const void __user *addr, unsigned long si return ret; } -/* - * When dealing with data aborts, watchpoints, or instruction traps we may end - * up with a tagged userland pointer. Clear the tag to get a sane pointer to - * pass on to access_ok(), for instance. - */ -#define untagged_addr(addr) sign_extend64(addr, 55) - #define access_ok(type, addr, size) __range_ok(addr, size) #define user_addr_max get_fs From patchwork Thu Dec 6 12:24:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715767 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBC851731 for ; Thu, 6 Dec 2018 12:25:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A933F292C8 for ; Thu, 6 Dec 2018 12:25:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9C5A22D325; Thu, 6 Dec 2018 12:25:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5868292C8 for ; Thu, 6 Dec 2018 12:25:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2CF1E6B7A03; Thu, 6 Dec 2018 07:25:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 280696B7A05; Thu, 6 Dec 2018 07:25:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 087346B7A06; Thu, 6 Dec 2018 07:25:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 9C3D66B7A03 for ; Thu, 6 Dec 2018 07:25:06 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id p12so75474wrt.17 for ; Thu, 06 Dec 2018 04:25:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=qsCjCKV+q57Vg4Qo+ex38n8fMkQGTNd9MzBlJ42c62w=; b=GHIk5JGyREk5FGP9k9Smw2afO3zTfISjPrF8+F4Xewvxg+bzYDwyk6pzd/7eeEp51Y Wc4V11ueh1NquQSzREibBqnUUy9qaKdTjS3lCwJD8O1X2DzTSKyLJewJW8Iwnyg0oFE/ m6QYyEwHY9WgdQ8B2pscGZXuDnndP+UU3VqBdnvuZoXe7KsXLc8im0HP07oJrzFr2Quq YbpND9zor20NcUvn61i04nHxKpp9VrDQHDlIQCSAXzwyTOFqH/Ty9nXdr6IeEeIzbQki 8tPcYPD3a7QbTHOVN1vXZunVIh5SH5Lfi1NTHgqSQNygXQcGQcN1FLll3Qpx0XNby1zC F9EA== X-Gm-Message-State: AA+aEWZz7CYSfYn9MKjZPlqHgtkiMFkk+HJ3kh1O58QLakVG033c8TGq psQqjywOlgvTVJWu78BQyCKIOBUj5MKL9YJP86hXV9m6LZKNjf1fEU4xX9KT0d3Qz7ecnqnYHP1 LZY4PeqMY+MrpkHmKu8o/IDdPbd08T9HqUhp+TK3EU9IjVrCoXXHStDTLvWho/1chgG5Ycyvmj4 r9qHW5fJ4HHPvK6GWZJ9mllSRPLzy93oe1MuI+uifLTukAWYQOob6+nJhTYzIrbWJKkVQ4PlR+a uLtytEj78ZzxHU+EI/5hSyw/pieQMmXDc304RYwLORhiIN6bcdwS4jnmP13LftdCAVPZyh6gfXs D5M3OlgWX8aH0I5MKAmTWJq2MMiBYuT4Dmtp4KEKKwKQHwR3UF6fz8I1HegAvNuWUtHkxg4WYbO X X-Received: by 2002:adf:e891:: with SMTP id d17mr27019173wrm.140.1544099106158; Thu, 06 Dec 2018 04:25:06 -0800 (PST) X-Received: by 2002:adf:e891:: with SMTP id d17mr27019108wrm.140.1544099105060; Thu, 06 Dec 2018 04:25:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099105; cv=none; d=google.com; s=arc-20160816; b=dzKHPgnHjUTL+cKWP/Vh6wGb7kG1OYkdAd/HgMILofPHonKmO39npZOZhEzs3/IBu6 oSMfnV8n+dnrnBtWlRvXomO+tYpNPWhUtib/PUt78Ij4+MudkJK8EhSvCfJedC2IMCmI WdaQKznJPpCCqqMK/WWLPt69AMQkOpD8y0i8Eb6scieD259PmONGH9nwqyq+UvxOdpxg Jg9ec6zJPfWWNvJnB+raPIMgzp3l53k753KB3DQvu+tGu7kDz8K9rGw109qtW6LEIA28 BSvc2XYz2/xufGssRuAiE+BxzZTOHSHgTn6gcWF6sjPWd0xAt7DeD1MYIktxg4VOM4+x Fy3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=qsCjCKV+q57Vg4Qo+ex38n8fMkQGTNd9MzBlJ42c62w=; b=K2rb0qBEDjrqVhRGdRW2V0ddI54oaLYUNjMr+i0YSCbkI4Biq1HVdoT+2B+/HvpBYN CtMDuZCc0uJpKGZ82rNn9ftnP7zJieATr/0leHofuaQG2Doj16t0oFFi16RebK+8G06/ bcthEVfXGre24z08AuCT3Vf1GcwP1vL1FGzs1A336bbW7fTEE3SqwRdaT+QG7wC9a2bv EsPg45/rG37QhyotVA+qtyFfU3mEqFlfE7pEdbbQA22eqktGhyK5t0YuajEAcy8qVf8x iQvgZu6PcBflG2gzYvxyK/pUekS8L95uGAxgCrMoM46A7RcLiFRCjGQ9ljlqzAEC1mcv IQ5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=lYPWD4dz; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v18sor172191wrn.45.2018.12.06.04.25.04 for (Google Transport Security); Thu, 06 Dec 2018 04:25:05 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=lYPWD4dz; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qsCjCKV+q57Vg4Qo+ex38n8fMkQGTNd9MzBlJ42c62w=; b=lYPWD4dzXCyGlWApicPbPWrOJcyzUAMrHRa9ML+WypDuXdVkzJ/cNmx3Tj3z5kn0X+ wXsJE/+j+Upc/1yIDjCHMrh4af5GeudR6utpq6u2ci47XkTpdttHYOChrnJgRCp1Jes2 estwO/nvoxmr4u9CJvAs4fwDMRhOR3SASD46MBoLNGjCKdYhlnqD4Qy52vO72QuyB+Fb jiSB4eJh3TYn8dAVPZx5dk/WDuwIDlyfcLXcTABmAqAcpqSsx3o3o9xSpKnO97j6yKsd ROuu+YVoBACV0QzThJXynS8N/eIMhCvtBaD5xvna4tCWj53zdpp907T0luqWbFsjQUj8 NWdA== X-Google-Smtp-Source: AFSGD/XUku20Jyax7BJye38HG84DVpktJbYsI3QLS0+xkAhyhZ+Y+sA1HXilshs7C7kohJO9GetxgA== X-Received: by 2002:adf:9591:: with SMTP id p17mr26240808wrp.224.1544099104494; Thu, 06 Dec 2018 04:25:04 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:03 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 10/25] kasan: add tag related helper functions Date: Thu, 6 Dec 2018 13:24:28 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/kasan.h | 8 +++++-- arch/arm64/include/asm/memory.h | 12 +++++++++++ arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 13 ++++++++++++ mm/kasan/kasan.h | 31 +++++++++++++++++++++++++++ mm/kasan/tags.c | 37 +++++++++++++++++++++++++++++++++ 6 files changed, 101 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h index 8758bb008436..b52aacd2c526 100644 --- a/arch/arm64/include/asm/kasan.h +++ b/arch/arm64/include/asm/kasan.h @@ -4,12 +4,16 @@ #ifndef __ASSEMBLY__ -#ifdef CONFIG_KASAN - #include #include #include +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) + +#ifdef CONFIG_KASAN + /* * KASAN_SHADOW_START: beginning of the kernel virtual addresses. * KASAN_SHADOW_END: KASAN_SHADOW_START + 1/N of kernel virtual addresses, diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index e2c9857157f2..83c1366a1233 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -219,6 +219,18 @@ static inline unsigned long kaslr_offset(void) #define untagged_addr(addr) \ ((__typeof__(addr))sign_extend64((u64)(addr), 55)) +#ifdef CONFIG_KASAN_SW_TAGS +#define __tag_shifted(tag) ((u64)(tag) << 56) +#define __tag_set(addr, tag) (__typeof__(addr))( \ + ((u64)(addr) & ~__tag_shifted(0xff)) | __tag_shifted(tag)) +#define __tag_reset(addr) untagged_addr(addr) +#define __tag_get(addr) (__u8)((u64)(addr) >> 56) +#else +#define __tag_set(addr, tag) (addr) +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif + /* * Physical vs virtual RAM address space conversion. These are * private definitions which should NOT be used outside memory.h diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 7a4a0904cac8..1df536bdabcb 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -253,6 +253,8 @@ void __init kasan_init(void) memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); + kasan_init_tags(); + /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; pr_info("KernelAddressSanitizer initialized\n"); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index c56af24bd3e7..a477ce2abdc9 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -169,6 +169,19 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #define KASAN_SHADOW_INIT 0xFF +void kasan_init_tags(void); + +void *kasan_reset_tag(const void *addr); + +#else /* CONFIG_KASAN_SW_TAGS */ + +static inline void kasan_init_tags(void) { } + +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)addr; +} + #endif /* CONFIG_KASAN_SW_TAGS */ #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 19b950eaccff..b080b8d92812 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -8,6 +8,10 @@ #define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ +#define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ +#define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ + #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ @@ -126,6 +130,33 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif +#ifdef CONFIG_KASAN_SW_TAGS + +u8 random_tag(void); + +#else + +static inline u8 random_tag(void) +{ + return 0; +} + +#endif + +#ifndef arch_kasan_set_tag +#define arch_kasan_set_tag(addr, tag) ((void *)(addr)) +#endif +#ifndef arch_kasan_reset_tag +#define arch_kasan_reset_tag(addr) ((void *)(addr)) +#endif +#ifndef arch_kasan_get_tag +#define arch_kasan_get_tag(addr) 0 +#endif + +#define set_tag(addr, tag) ((void *)arch_kasan_set_tag((addr), (tag))) +#define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) +#define get_tag(addr) arch_kasan_get_tag(addr) + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 04194923c543..1c4e7ce2e6fe 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -38,6 +38,43 @@ #include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void kasan_init_tags(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(prng_state, cpu) = get_random_u32(); +} + +/* + * If a preemption happens between this_cpu_read and this_cpu_write, the only + * side effect is that we'll give a few allocated in different contexts objects + * the same tag. Since tag-based KASAN is meant to be used a probabilistic + * bug-detection debug feature, this doesn't have significant negative impact. + * + * Ideally the tags use strong randomness to prevent any attempts to predict + * them during explicit exploit attempts. But strong randomness is expensive, + * and we did an intentional trade-off to use a PRNG. This non-atomic RMW + * sequence has in fact positive effect, since interrupts that randomly skew + * PRNG at unpredictable points do only good. + */ +u8 random_tag(void) +{ + u32 state = this_cpu_read(prng_state); + + state = 1664525 * state + 1013904223; + this_cpu_write(prng_state, state); + + return (u8)(state % (KASAN_TAG_MAX + 1)); +} + +void *kasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { From patchwork Thu Dec 6 12:24:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715769 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13916109C for ; Thu, 6 Dec 2018 12:25:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 025E3297D0 for ; Thu, 6 Dec 2018 12:25:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EAB882D393; Thu, 6 Dec 2018 12:25:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9307A297D0 for ; Thu, 6 Dec 2018 12:25:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD2446B7A05; Thu, 6 Dec 2018 07:25:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B4DE46B7A07; Thu, 6 Dec 2018 07:25:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98F546B7A08; Thu, 6 Dec 2018 07:25:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 36D8E6B7A05 for ; Thu, 6 Dec 2018 07:25:08 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id w16so81811wrk.10 for ; Thu, 06 Dec 2018 04:25:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=UqMDiL5pFL/+tkei1G/AIx7zwZMP99eD4KugQS7K55g=; b=qX5aVg3bhi3r32FZ/Us4FhO7ZfoxishG2IX7gvq7aZSb2ldsTf5gajn/JB5PgA/x1m XKDA3DtO9vZ0bNCNzFWcJ4qD3ISkt6wEaPWOuUbbBT3iZu9bsG5yQnap/K75DvD/gzBb jU0K9aCpPafFeaYFw3QugIPQ3BONgEE/maN1ki1LFhqQpsB3iLBBTn+wr5hqUIr02iYh Go94DApO0odEtLPLtW9jcc+0vWuQ9GuPA4DHQPi3iGEQ5V7EcMdx+x1B8giDp4vH2D2u A3jKpkgtFOk+C93lW31udifGoS/dqT6ZKlA6Nvf7z8t5PMTTCcgVMLvzZebNVRpyqo3g Sv9Q== X-Gm-Message-State: AA+aEWZdf1teN4abLmMZGPFIR18EotNsm12+ZVidXbkjIXCcTIFP7iA/ Wqr+1eRjvpudIifrjPGH4joV+NKXg86lAl7/d1ZbI9ovpY/Rk4XERH8RsW9ETJnECSh4JBHkclr lTjW0JcLPz8Nr1jiZ8i4B9hSfIbQ0psQm6PojVBp9Ui6ZjKRLMTHZe5jJqMc4QibXb+4V/kJuur yVWfGwLb41x73EwsKlxilmlP13Al9JBibtWu8JETaZtL24TF6KDrmRw3yHVywAM1AaVZvO/JUil oTH60bNJEt2pUAGMqtCs8yrTJEmXaWFJJoxBOLOuhPiv8t+GVfOMc5odCnA8l+tVprimO14EQgz 9oXOb1XpQWf8371DuxKjN/HCz6GzVJBFusnVR58KYkXV/230rapuprHqakH4s5wHCgUwk9/7uu6 9 X-Received: by 2002:a1c:2d42:: with SMTP id t63mr2681344wmt.9.1544099107753; Thu, 06 Dec 2018 04:25:07 -0800 (PST) X-Received: by 2002:a1c:2d42:: with SMTP id t63mr2681284wmt.9.1544099106870; Thu, 06 Dec 2018 04:25:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099106; cv=none; d=google.com; s=arc-20160816; b=PCZ6xOHYKQCBW9h261Yr/EQL81ehFWnnOdCnxpxVCrge9k1XcLVky4fTbpRXkymqSR FyLwQRCTLm8VQOTsovzTRGa4jbcdVzfaMdgMjv/ZFO8diH+BtRO0vspwlYeNszHQasSU uHPI1lOEU2GU4rNKeDxmQzyCiHi4OT9+I7+qjfB2Sv6oQk3DxE7/jVjPCiZHPa+opL1i fRuAj0h3QG9iI9VtdzwFoSG3cpSxE6kWdeyHEc6ds0tSngp0ekoAZARcskq+uOf1p35i 9tg/rmKKyfXpghE+nJnE5l3gxwDyADe/5tchqFvZ7u0QhuGC1jirUSNB+ckmiHhY5I7r 4r9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=UqMDiL5pFL/+tkei1G/AIx7zwZMP99eD4KugQS7K55g=; b=S4L3UjQpWkxXEn5hxVCDNqmovH1duzAWwgvoCt81p3cxhbyn/HdOf5vFMfMOSAzjRO leY3aT/qX/KjW4IA76xLmWZABKny0BycOd4nWNSiXSGqiqoLg+dMyuyvkmmVNL8XzX+j 1fEGORpXIEDmbOtT5hGypvmPSUJMGAQsX7Ayrj2mqJWqW5V1Z5DNCtovasY+UsC9/1rH Sxt11f2OGLxLYNsiM0LZ1GPaWk+4BcsBfwaGqNPkfE6J0AJ41wjrWDhAwJsvHlsRudpB j8Et0BS1tFocu7q/PtFA4LoUKFMIatzfkRlLMF7MX8TIjXO5oymcj0gKv7VdRAdlpo98 32fg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Y2VohwGt; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id s2sor492084wme.1.2018.12.06.04.25.06 for (Google Transport Security); Thu, 06 Dec 2018 04:25:06 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Y2VohwGt; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UqMDiL5pFL/+tkei1G/AIx7zwZMP99eD4KugQS7K55g=; b=Y2VohwGtLIqo6t/3aMQeakdW1sJ7UgqyXRDUNidOjVdD5M1bxlNFiaWPSn1bXLUyyF mCCWKClMkXpRl/CkEYHFhkecgRkp6UBK6sfIgjgkARPZuNzpSqHnINsuMeuueMsTmNvD Mb970K6ZZtXOqMs4Gaek4O18mqjDkafHoZWEvQa4fKAWweR9DbpR2Q7eZR4APBPmIkFH N3qGtFuiC24t7PZgMXta7ZPzCIcCq8Yl7LCg8wCFP5JpIj2RfyVIiKy7tRwzK6WESWKq SjD/5xD4Wzs6YWFY2xHp4H4kAcqBkIOdh05TJR5kuzLQ14sH63IMtTkNdNhpUY1CH8hi OqTg== X-Google-Smtp-Source: AFSGD/UGniPFpytnR5EvX1mvUQCqi8xKkHyAIiafnkUS1yg6vUphxPqdq7r3o6WSEF+mrgSfvm3zrg== X-Received: by 2002:a1c:2007:: with SMTP id g7mr18924469wmg.79.1544099106379; Thu, 06 Dec 2018 04:25:06 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:05 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 11/25] kasan, arm64: untag address in _virt_addr_is_linear Date: Thu, 6 Dec 2018 13:24:29 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP virt_addr_is_linear (which is used by virt_addr_valid) assumes that the top byte of the address is 0xff, which isn't always the case with tag-based KASAN. This patch resets the tag in this macro. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 83c1366a1233..5fe2353f111b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -322,9 +322,10 @@ static inline void *phys_to_virt(phys_addr_t x) #endif #endif -#define _virt_addr_is_linear(kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) -#define virt_addr_valid(kaddr) (_virt_addr_is_linear(kaddr) && \ - _virt_addr_valid(kaddr)) +#define _virt_addr_is_linear(kaddr) \ + (__tag_reset((u64)(kaddr)) >= PAGE_OFFSET) +#define virt_addr_valid(kaddr) \ + (_virt_addr_is_linear(kaddr) && _virt_addr_valid(kaddr)) #include From patchwork Thu Dec 6 12:24:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715773 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF7FF1731 for ; Thu, 6 Dec 2018 12:25:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE123292C8 for ; Thu, 6 Dec 2018 12:25:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C1C542D325; Thu, 6 Dec 2018 12:25:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49F8F292C8 for ; Thu, 6 Dec 2018 12:25:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 718146B7A07; Thu, 6 Dec 2018 07:25:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6C7E26B7A09; Thu, 6 Dec 2018 07:25:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5430F6B7A0A; Thu, 6 Dec 2018 07:25:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id EFED56B7A07 for ; Thu, 6 Dec 2018 07:25:09 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id t77so211922wmt.5 for ; Thu, 06 Dec 2018 04:25:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=sISq/ZW0TVJBTVtL8WDFsSQvFMzwvGrKZuo3jHEW0rs=; b=qcNDRpo1YB9DmJmvy/VfqMOigO6ZD4fWh/1YfCIj8gqph2weS5SU93LL/ZFdQ0v3Ow tM/HEgK4O4E8S5Ql5HXOs+o9tJKZABw1RjEYtiVcETn8xjI8Hu/xj94IRo1cDHKhDJY3 xl1wuhn9Lx4z9RgCwA3vjCZ0YzYqArrj2b3RGvszX1B2raMRMr31akrxhcJBIFsALfPe +YiF2pB8DNkOK8ptXKSVqHCb5HFqhviyWK9dgmhGR7xA7cl+Be8SbudKJXeWLmUDuErO JkmR9sDBz0R9i4tRpEDB59Z4BKMAE3Tk1wmbENGEJK+BdBdzQ0G6ci3UmbgtgcpuC7RT YIOg== X-Gm-Message-State: AA+aEWYmUVfT4ZRffWdj/AM8pEC+e2d6N8sCyIK1vurMIk2p7faEc2t4 VRGDZllI+ErFdkadKnPIzvwyypCC18JAiMShxwiVpRoXwhfIfgWLxrCGin1gzfCg/BWbwXRqGXr er8ChXtbiOTd1COl8q2V9ycPJN+Ep7yJiDVf0bTHLWs3oW9/SMcV4JVxijcTze9cVlM16LhLCVf cwjuPwZMtakw/iGorOXup0CSx1zrATES0TNlOxebfplpCCM+m31pxkUA3zheCTVJqM+qsx2Ejj5 +QqRS4Kodz4MkGYfnmUQdmdhiHNgWZXnyBaet5jZb5vDQrJWd45z7stOZlSVclyOkoP4Xjp5vY/ gVTskGCM60Z1F7NKJSw4KenfZLgpeStc4TJT0mdx07hEavXyeqZVv/gMmIfssAhCJKcgpjZlRoF 7 X-Received: by 2002:a1c:770c:: with SMTP id t12mr19713355wmi.101.1544099109484; Thu, 06 Dec 2018 04:25:09 -0800 (PST) X-Received: by 2002:a1c:770c:: with SMTP id t12mr19713299wmi.101.1544099108533; Thu, 06 Dec 2018 04:25:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099108; cv=none; d=google.com; s=arc-20160816; b=YULJVbWX4d6Duw21d/N0oUwwb/YOTjnkQphoWj8n5V6sVMU6U0XTbfhegkrke7MQjJ YDkvtxa8xIGOWData9qE6fmRuCtR6NNkTD7eajsjJVtucXLYE8H6Y8CQZ4BJ+G+ShwuL uxYUrgGT/btdHlLxfpRPqVSOvRWKcgPJt3ANu1ii3TKm7d1Q2cHm2B6NVFmmdTw0/LxR 2nDyt9MOpdKytSSg4egdjylFLui6ZtREzeHV+6BkYBEpr0qQCvr0ansaIzelzZd3Hyg8 sSJ+qhNjyHg4/7+J8sYoRaYMpFtZjhJDbBePqnQwmFNkct0b028AhLZOABTfGvDa7RDX c6pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=sISq/ZW0TVJBTVtL8WDFsSQvFMzwvGrKZuo3jHEW0rs=; b=ft+xL/5yEuJWRGb0Qi6yHq2mG+j4qkngnFQuWz3xKu7CTEDH/EKEsj64QZUEgRCtYt 7K9xQ2LvAXNoEwjO9xour/UTiaCx62byyfCOtj9BUKYqRJ9ksQWxjmJK04jKrxQY7R1h c7464kaecXGEgwVhkIYCR0P0lJn/l9q68maa1uLeInaGLFNQfE2A1GDWbAYSXjaOIuZd xKNyOsfgTrhmyahapehqmS9GhGDFbITTew/s27dTmhv4EFBGoCvE5GifOEwQvF4yivlG eN9fUvEuL2+IAIlTNnNiOwcUoN9vuHLoZRvz8sCwUdvW8lRRkBFek6lyBVvpsWi8nuo1 VOWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EIaG8FbV; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i13sor447490wmb.11.2018.12.06.04.25.08 for (Google Transport Security); Thu, 06 Dec 2018 04:25:08 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EIaG8FbV; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sISq/ZW0TVJBTVtL8WDFsSQvFMzwvGrKZuo3jHEW0rs=; b=EIaG8FbVvjfEZBwPVnSoxYhmFaPSG+vHC9ub8Q9UvflzLci7qWq1MIvHhHMYHp+qDe aYCb2A46h0HyozZYwd+MH09I2ZtN17SqYJSvjpXS3OWlMZyrAYkp8oRdU0Rvsn1ii2PH hgxdrby1RcW8uBjfYxUDDMAMeg/GAfy+QZp72weuTc9DFFG8WxesnWT3ce5AlEqG1XPu NUxxu1xI3NGg4i+djtH88GpUdqSqviOaUnu2vrK/1hHU/TEC5YZsyp6TnaTdAvOQTaUe zkHqUUKT1p1wYmKjUILEhMirxb9nhAqh+8GNSNz+rwKkyZke8WRAAJA6DaJN86coRzJI SjIA== X-Google-Smtp-Source: AFSGD/Uvku6Y8UP5YFct8xBdwbQSBOubcGBRJqYe7H9p2WfZHwE/gOIPH+PVv5TfLbG/g+zd551lTw== X-Received: by 2002:a1c:e088:: with SMTP id x130mr18266740wmg.23.1544099108033; Thu, 06 Dec 2018 04:25:08 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:07 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 12/25] kasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU Date: Thu, 6 Dec 2018 13:24:30 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor. The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it. Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/slab.c | 2 +- mm/slub.c | 24 ++++++++++++++---------- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index 26f60a22e5e0..27859fb39889 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2574,7 +2574,7 @@ static void cache_init_objs(struct kmem_cache *cachep, for (i = 0; i < cachep->num; i++) { objp = index_to_obj(cachep, page, i); - kasan_init_slab_obj(cachep, objp); + objp = kasan_init_slab_obj(cachep, objp); /* constructor could break poison info */ if (DEBUG == 0 && cachep->ctor) { diff --git a/mm/slub.c b/mm/slub.c index e739d46600b9..08740c3f3745 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1451,16 +1451,17 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, #endif } -static void setup_object(struct kmem_cache *s, struct page *page, +static void *setup_object(struct kmem_cache *s, struct page *page, void *object) { setup_object_debug(s, page, object); - kasan_init_slab_obj(s, object); + object = kasan_init_slab_obj(s, object); if (unlikely(s->ctor)) { kasan_unpoison_object_data(s, object); s->ctor(object); kasan_poison_object_data(s, object); } + return object; } /* @@ -1568,16 +1569,16 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page) /* First entry is used as the base of the freelist */ cur = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + cur = setup_object(s, page, cur); page->freelist = cur; for (idx = 1; idx < page->objects; idx++) { - setup_object(s, page, cur); next = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + next = setup_object(s, page, next); set_freepointer(s, cur, next); cur = next; } - setup_object(s, page, cur); set_freepointer(s, cur, NULL); return true; @@ -1599,7 +1600,7 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) struct page *page; struct kmem_cache_order_objects oo = s->oo; gfp_t alloc_gfp; - void *start, *p; + void *start, *p, *next; int idx, order; bool shuffle; @@ -1651,13 +1652,16 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) if (!shuffle) { for_each_object_idx(p, idx, s, start, page->objects) { - setup_object(s, page, p); - if (likely(idx < page->objects)) - set_freepointer(s, p, p + s->size); - else + if (likely(idx < page->objects)) { + next = p + s->size; + next = setup_object(s, page, next); + set_freepointer(s, p, next); + } else set_freepointer(s, p, NULL); } - page->freelist = fixup_red_left(s, start); + start = fixup_red_left(s, start); + start = setup_object(s, page, start); + page->freelist = start; } page->inuse = page->objects; From patchwork Thu Dec 6 12:24:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715777 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44046109C for ; Thu, 6 Dec 2018 12:25:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31983297D0 for ; Thu, 6 Dec 2018 12:25:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 244542D393; Thu, 6 Dec 2018 12:25:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46439297D0 for ; Thu, 6 Dec 2018 12:25:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE0186B7A09; Thu, 6 Dec 2018 07:25:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E43B26B7A0B; Thu, 6 Dec 2018 07:25:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBB2A6B7A0C; Thu, 6 Dec 2018 07:25:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 7061B6B7A09 for ; Thu, 6 Dec 2018 07:25:11 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id 127so209873wmm.6 for ; Thu, 06 Dec 2018 04:25:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=iKeBMeDd/hvI2FA7NoeqR+ehI2COJOXIft3otGKQ0o8=; b=hd/ZoA6jfJDipLhkNfdoKkIMaEo1vonIur55jHZnJ0IOfTkzNlpzx4RMwVCrrzgqpm MHL2cRHFCBiM/NqioPvu+y5UMK1vjBg568Nx4UQOr+w28OBN7jpsK+u/ACvLzjcI8q7m I29MaGr3CjS6huzUGOj5Ex+5MeF4EH9XG8+Cj96Zst9ANYo2zwtjUNYf2Q60VhuYUcr5 2srQCXSGsBgvX8ZZvC/r6Udd2f19eYCwmBrvYVdmdpEq0xeOINyN1pBWzOad1cxcXEk5 wD0AOOqhCAEfhskbJ19FdEK5cB56uqmlGnGO5sBb5qjNYBhyjrFPwGUT/L93uh4ghBlv hAtw== X-Gm-Message-State: AA+aEWZ94ylXryxWYcfmBsig0sy4ODjUm0peCyEfUFsEj6xvJgjcKvcc cfr+wJUk5ADmoeyDDM2/RmMRI0CjC1KrDWAM8A/ifJXtoJ6R3/97krVadTvA1WERd1D/HGyEDnj 0M04RX7gGqitWJUnAEo+B58LtZulsVefAyZQesvpfE0UcqVDY7r5rny1J4zC5lRsQZoWaEUlzml 02a0Q3guErOAvY1kwb3Axv3veVumvWOgPOQO9pfGNW/NfH+lJhJDsfc7nie6CbiGjtDybp4DDx8 wq2YH9pbTidRAU+7kVGIUfSBlNHJ0Ec7x8hA5f6nzix+/K/p52aoWE7jqt05KoCTJrvpqTHG3NU kiQjr7Kq6p17h5Eao6suAEBwfT4EiBRgSjk219WCKlJu/VJrtgSEyMEE4iG+3e4ha8mc2hiPOF7 B X-Received: by 2002:adf:d4c9:: with SMTP id w9mr25030223wrk.119.1544099110967; Thu, 06 Dec 2018 04:25:10 -0800 (PST) X-Received: by 2002:adf:d4c9:: with SMTP id w9mr25030172wrk.119.1544099110090; Thu, 06 Dec 2018 04:25:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099110; cv=none; d=google.com; s=arc-20160816; b=mMQX7VEJMsxzupySCIZtNFMDd5xmx5IDdnwuKbtRRylJjm6SRhN2eL0ZcqRd0WfWMo rI/pTMeGI6yHaiY7tsiR2uXvQtDpGJzYLaG/oYJTvM84qKsXvdqanyMJs1HzZjW9PIpK 3LuXwRp1T3XR08j6y7D12DBmIlCO0l7NSf4UIcO7lBkEVUsL/xZq482lNWzShU6iCoJk QLRyRov/PugLO1ht1e1EbDzJv8eEmOmqkqI7DmfzM0GSXsxoag7oWTimVWuGC3SiujO9 coT/1vtw7dH54VA9BIC/j3n6EoTtyWPLfrreo3xAUW7DMALhcYWueAUMx3BbEo2x1sYs UxUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=iKeBMeDd/hvI2FA7NoeqR+ehI2COJOXIft3otGKQ0o8=; b=Mw5qbak2hmm9cjhxvMWW/W+4aelpJT4wp+sVbKjEbA/iONW1hk8iUnSdzhNkyLySUT MWKsXDqHMSHgcFiYcYh0Khe2JqleiVVV1hZYLgUiQ6SRdTYDSe6Rxsg+kCB671LNUDSv X5UigT2x0YqdGdRb6qAYgPpmUsLM/oBuJ+N7iel5EPmtOOflAuEgQvePyIoXknad9YX1 jNdyy6UO6wsE5rIkKeCW4voo4Pmm3S7wWK0ygQ0FEPvWNqFOc0HyaS7CCMHRIkYGjspX VNWwIEHN1Lhrmtm+Hxka+zolaX3EKM49nUvwZe1BxrJMzVk102rD9NDN91G8XkB4FYnw W25g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="f1rf2n/s"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t7sor189500wrv.18.2018.12.06.04.25.09 for (Google Transport Security); Thu, 06 Dec 2018 04:25:10 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="f1rf2n/s"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iKeBMeDd/hvI2FA7NoeqR+ehI2COJOXIft3otGKQ0o8=; b=f1rf2n/scEn9iSpaT9i5tOsqH/hxKSAj6AnhXUunXk4GSnehLOZkxC8cS0pFgw4bCr 42W27ioXpFlJswetDzyyU0v692af90Dwqs2quECtDhWBNax5gIvsKM9hLqrPPRE7j0Ll lmKjB6prDWZDTCgiFHkvtWUXldCGgNI4PTdYxfbNiIsm2fa8ndTPa1e2B+IMHiZLgluF UvIdIRqvosMF9vBLQPQf3Fu54KS26Ppo8EJfjWJw1gIDuHptmHhmoyMxHbTYk0g63dy+ hltg6bPlJ/Z27FVdlMne0eMdsDAKMqpXWJn7svGa5OscnMUePNv3cTmSevfnP+UDZuxC SKFQ== X-Google-Smtp-Source: AFSGD/Xy9XUT3vO1le8FTX7MP1mbCfMSAzWTSUArSSG7mR6VEHKyozrE6h4RIddLRyoXver8Ony3nw== X-Received: by 2002:a5d:61c4:: with SMTP id q4mr24022433wrv.308.1544099109600; Thu, 06 Dec 2018 04:25:09 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:08 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 13/25] kasan, arm64: fix up fault handling logic Date: Thu, 6 Dec 2018 13:24:31 +0100 Message-Id: <3f349b0e9e48b5df3298a6b4ae0634332274494a.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Right now arm64 fault handling code removes pointer tags from addresses covered by TTBR0 in faults taken from both EL0 and EL1, but doesn't do that for pointers covered by TTBR1. This patch adds two helper functions is_ttbr0_addr() and is_ttbr1_addr(), where the latter one accounts for the fact that TTBR1 pointers might be tagged when tag-based KASAN is in use, and uses these helper functions to perform pointer checks in arch/arm64/mm/fault.c. Suggested-by: Mark Rutland Signed-off-by: Andrey Konovalov --- arch/arm64/mm/fault.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 7d9571f4ae3d..c1d98f0a3086 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include #include @@ -132,6 +133,18 @@ static void mem_abort_decode(unsigned int esr) data_abort_decode(esr); } +static inline bool is_ttbr0_addr(unsigned long addr) +{ + /* entry assembly clears tags for TTBR0 addrs */ + return addr < TASK_SIZE; +} + +static inline bool is_ttbr1_addr(unsigned long addr) +{ + /* TTBR1 addresses may have a tag if KASAN_SW_TAGS is in use */ + return arch_kasan_reset_tag(addr) >= VA_START; +} + /* * Dump out the page tables associated with 'addr' in the currently active mm. */ @@ -141,7 +154,7 @@ void show_pte(unsigned long addr) pgd_t *pgdp; pgd_t pgd; - if (addr < TASK_SIZE) { + if (is_ttbr0_addr(addr)) { /* TTBR0 */ mm = current->active_mm; if (mm == &init_mm) { @@ -149,7 +162,7 @@ void show_pte(unsigned long addr) addr); return; } - } else if (addr >= VA_START) { + } else if (is_ttbr1_addr(addr)) { /* TTBR1 */ mm = &init_mm; } else { @@ -254,7 +267,7 @@ static inline bool is_el1_permission_fault(unsigned long addr, unsigned int esr, if (fsc_type == ESR_ELx_FSC_PERM) return true; - if (addr < TASK_SIZE && system_uses_ttbr0_pan()) + if (is_ttbr0_addr(addr) && system_uses_ttbr0_pan()) return fsc_type == ESR_ELx_FSC_FAULT && (regs->pstate & PSR_PAN_BIT); @@ -319,7 +332,7 @@ static void set_thread_esr(unsigned long address, unsigned int esr) * type", so we ignore this wrinkle and just return the translation * fault.) */ - if (current->thread.fault_address >= TASK_SIZE) { + if (!is_ttbr0_addr(current->thread.fault_address)) { switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_DABT_LOW: /* @@ -455,7 +468,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (addr < TASK_SIZE && is_el1_permission_fault(addr, esr, regs)) { + if (is_ttbr0_addr(addr) && is_el1_permission_fault(addr, esr, regs)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die_kernel_fault("access to user memory with fs=KERNEL_DS", @@ -603,7 +616,7 @@ static int __kprobes do_translation_fault(unsigned long addr, unsigned int esr, struct pt_regs *regs) { - if (addr < TASK_SIZE) + if (is_ttbr0_addr(addr)) return do_page_fault(addr, esr, regs); do_bad_area(addr, esr, regs); @@ -758,7 +771,7 @@ asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, * re-enabled IRQs. If the address is a kernel address, apply * BP hardening prior to enabling IRQs and pre-emption. */ - if (addr > TASK_SIZE) + if (!is_ttbr0_addr(addr)) arm64_apply_bp_hardening(); local_daif_restore(DAIF_PROCCTX); @@ -771,7 +784,7 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr, struct pt_regs *regs) { if (user_mode(regs)) { - if (instruction_pointer(regs) > TASK_SIZE) + if (!is_ttbr0_addr(instruction_pointer(regs))) arm64_apply_bp_hardening(); local_daif_restore(DAIF_PROCCTX); } @@ -825,7 +838,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr, if (interrupts_enabled(regs)) trace_hardirqs_off(); - if (user_mode(regs) && instruction_pointer(regs) > TASK_SIZE) + if (user_mode(regs) && !is_ttbr0_addr(instruction_pointer(regs))) arm64_apply_bp_hardening(); if (!inf->fn(addr, esr, regs)) { From patchwork Thu Dec 6 12:24:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715779 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C45E51731 for ; Thu, 6 Dec 2018 12:25:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B201D297D0 for ; Thu, 6 Dec 2018 12:25:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4D4B2D393; Thu, 6 Dec 2018 12:25:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BBBC297D0 for ; Thu, 6 Dec 2018 12:25:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 986E76B7A0B; Thu, 6 Dec 2018 07:25:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 939986B7A0D; Thu, 6 Dec 2018 07:25:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73DFF6B7A0E; Thu, 6 Dec 2018 07:25:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 1A0486B7A0B for ; Thu, 6 Dec 2018 07:25:13 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id e14so75965wru.19 for ; Thu, 06 Dec 2018 04:25:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AkvfG/IDbgZMxYwPX5lSFsN5elmH6vlYt7vVIn43a0U=; b=Kf9D9SLNO4oUMY+KNDSOJ3gEH9gp5+RmO85vMQASUKmOiuJxIwyxUHs1ECBDy4F+ko Y7mIV/6jrcPCVYYbC20NdVCtqEvTl922umJVQBj3MJTLUov2b+GgYxv07AH3xYuvntC+ jowqrsK5MgculILzc+RG6CVin+76dOG0xdbx4Tc5qwQS2pc6iOQGwJCBIp42bjS0K505 0rnTiCwwtg6rLLsHR+l7g0QwJ8dXtvzfTHygMC1RLpmuVo6QJ9UbzoBmZnk4YZXMKU/s fsFiUdhsC+2+SoLJv9EsMkv2/wTUMTI7ZxEq9jdsDQA1hmKW2wyufjjV62jYLkqoQ+sz G3Xg== X-Gm-Message-State: AA+aEWaKOKXcsOC/64uD0K5WH6zhRXDW0+PjcUzWEY72t2lgf6AVbYNj jOAiaNhxQU64SlPW899XfbuuJGJnJRSMEY3vHSAxnvi6yQUb4uGR/IT74OtQn6adGhh4PQHkuwu AYTDBsHYbPmWwYHrlhAZqfX/R2rkniKztpcQ2tcAMBqYv8IIYyZExeRPPlJN1XptYcFQW4eYln7 WABjaFHPYkJxGEXDIz55rQpOjaLT+46i3zRrUR2CEB2+zk/zwm1QF8GpJyyFi9QNkHKJRRjHe/n SsQxqiZtt4UrpefqfEdOYnsbzOk+/w1QjRH3zKyzuw0pM7lj2xnR+wDqPtgx4WQHEy4KBuImobT YkZvJMCrf1LWX2cXCbXg7ITJIeBu/6WEH87Lk8cKbAWTMoSYVe3Dwr51/eDqMzRwhVd+bOmCA25 J X-Received: by 2002:a1c:9692:: with SMTP id y140mr20334329wmd.67.1544099112585; Thu, 06 Dec 2018 04:25:12 -0800 (PST) X-Received: by 2002:a1c:9692:: with SMTP id y140mr20334271wmd.67.1544099111742; Thu, 06 Dec 2018 04:25:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099111; cv=none; d=google.com; s=arc-20160816; b=Z+WWam3eR6tditnmUQWksEAHxyK378wn5+ELy89EoD8kOZh2EHYHNgGWAn69lfqqNH kHVGNf67+q4jW+Ow5ntm7ImqpYEdsqTznHclLC88X44m3DJd5uuJcPUAzdBxTSBw0OqW HWggnSO6Na7JEzYVUe1S13MUGl/SYxrOg4aUHCDZV2sz2MQ+W+ZUrYlZF1rL8kZZliia Ll9CXlpm1RDed7aBFGlauAPW8ivh4bnr7KxkvwvZ5xuhkCGqGDkk7A9RzHh2qbo09Glc HMOKxcondv9nKqxoKbBP307YM2URhPoyTcKyruIcPLl6RqX3YG7QHsXdFi33MmrAdMEG OX6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=AkvfG/IDbgZMxYwPX5lSFsN5elmH6vlYt7vVIn43a0U=; b=KA/VhvURYaQH1HikS2CfvdfWescdeGwUhs79BXVe8ZGOg84CKxofnUsmaIs4p82n+A 1LFDwee4jJmyyRCSep4xSegRru60Q7GHmkW7c5Wl7rBave/PUZEA6Rtn/COqi7n9HEZm rOwEQ0Io1C3Uhmab9BmElgAxELPIJUb0BDNwHh0ZgQU+c/gqPzLGLkoJbFI0l0d9FGbA lLJwQoHpgMr2jx/NPyQiPXjSlSmWl541ah3lYfbaIdssDkekifKB44HVXWkcI06Aur0C yw/JLRCdvEubuUq00bUgWous2vx7pYuLD8nWzg8l2m9r6nPbJMxSojVnxngBBmKpANiR 5HoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=a0Vwo7+R; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j10sor190251wrx.15.2018.12.06.04.25.11 for (Google Transport Security); Thu, 06 Dec 2018 04:25:11 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=a0Vwo7+R; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AkvfG/IDbgZMxYwPX5lSFsN5elmH6vlYt7vVIn43a0U=; b=a0Vwo7+R9UWJKWhJ5tdXON+TcRVk8bAd3GrGkAupa9U7c521S0OEfq/71bnrAZ7YI9 qO3e/k1mYl3KHhZ9c+3uOMl2aTRoIRr9SlDaXPW4D8lhbk32Md5fm2bGf9/zOG/6Meus zm9A28pu+SVWQtklpcqHi8Gz496+QwhVd5wIpCU6J9hBR5i/FXMe9o6PAjf92B1EtKkE wUnBhqf9UhEVYjxuBsO+8OlgbhSabAPKLFgHzoSxcvnZIIVzwQ57/4kGb0byLGwVIbuw xb9Pb7kljtzchG8AcB3Q6RRRggT61OjCQ6QMRyg3/B9mgLgiuUws4cV863SNS0HtMxy3 ebpQ== X-Google-Smtp-Source: AFSGD/W4zA6BXuDW4nuwEjrxAwEtRH7jwb51mwQJbpOgSMlyeciRxEfja88tx3NW2n/I+jUstCLcCg== X-Received: by 2002:adf:b243:: with SMTP id y3mr10683692wra.184.1544099111278; Thu, 06 Dec 2018 04:25:11 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:10 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 14/25] kasan, arm64: enable top byte ignore for the kernel Date: Thu, 6 Dec 2018 13:24:32 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when tag-based KASAN is used. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 1d7d8da2ef9b..d43b870c39b3 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -291,6 +291,7 @@ #define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 2c75b0b903ae..d861f208eeb1 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_SW_TAGS +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /* @@ -445,7 +451,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /* From patchwork Thu Dec 6 12:24:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715783 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1FB3C17DB for ; Thu, 6 Dec 2018 12:25:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F7762D325 for ; Thu, 6 Dec 2018 12:25:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 01BCD2D941; Thu, 6 Dec 2018 12:25:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 142142D325 for ; Thu, 6 Dec 2018 12:25:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F3516B7A0D; Thu, 6 Dec 2018 07:25:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4A5166B7A0F; Thu, 6 Dec 2018 07:25:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CDB26B7A10; Thu, 6 Dec 2018 07:25:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id C781A6B7A0D for ; Thu, 6 Dec 2018 07:25:14 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id 49so79583wra.14 for ; Thu, 06 Dec 2018 04:25:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fxqgizsGn0AaIQnRQ33dQkRGCaY8UuAWu0B6VsH5hUQ=; b=Zq4I9YvVc3526uZSqsyMcXMjdzpThvLgQShB/nOu6Up/Qv8ilPv1dAy6hdMY0VJ0dM fMg52qoA40h2SoEpSvUdV3xpjKHDphoIB7jpvLG5FYilaOSAPaMEjMrJ5eP5Gf+jccpF alHr943cWyYo38EykLnBoKt7MJ6EYHKNwOK8K6cNP263zFPG0NPytFDhYhXZcgJLdUED kypZjhaFicnN7Dbaci/+dG0LCU8vUZE72U0G8PCcYkDSV//QktaEQpbwHYpIag0prD/o W8UXGHs2LoMqeO2K7Ip2JdMAJJQi2m/zXrIRck5GsIurjwKmQs4A+U08F9YQBUKKWuKU IBig== X-Gm-Message-State: AA+aEWbCBT6lNC0Ugd39Bi67oHRi2Omgn+IiAYlU/yBQbRoIh/RNf29F 06K0+CMyX0YTcowsEz/sImWWeQXYDJ0Au7REgVBLDX3R+Bd4dncW1Y1/OabcCJwlOMC2Rxrh0pC Y/M5oittTOnkeKAgbyp7tWLoaMJHPdF0ENy39vt6dHxyCAKLzLWTo27ttHPScGsDop5Sz7cGdme H1WRHzdanIPwKcXOTdLjnJS9LVmeX/smtakWaiN9/zQ012wugxD5+aAgyPS9MZNG1R2dBvTNCGv u1c1guDELe/8baRB7MmmdY9fQD7Tnu3ndE81D8w1oaGnzmkx6pF9tA38eizvIWyBvWWg3A103Du 6zUzgmN0GijtzU1K6CWu21sjVGSu7C3IVWkftPz/Xqaw2o9vU3lVdY/x6tCkH0Yg58l3Agob+Ql g X-Received: by 2002:a1c:8089:: with SMTP id b131mr19793576wmd.141.1544099114341; Thu, 06 Dec 2018 04:25:14 -0800 (PST) X-Received: by 2002:a1c:8089:: with SMTP id b131mr19793523wmd.141.1544099113567; Thu, 06 Dec 2018 04:25:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099113; cv=none; d=google.com; s=arc-20160816; b=jsed/c/Vfe/Uxtapj2kOBElbwnyiimNOztxfm1N+VM1/Y9ByRDhK895kXMY6RFhhWQ 8wx0wcLAtChJPZ5SkS5HjSfbcRTL7MjI1O04NGxYgcKps+x+peqERCRwlWcJNd98vVLK dJCN8LO681/czLEqQNuUQ4wSJonFaSF+8OOBJLGCD+889sRhWg6aih8F2nREtg55NlvP dcPcYEWsX+THerBpIkS6oKBO/C3r+5TcZ33B9X+uWjiQUoI+PpTNXQMv/+fInIOTtoeF 3UJU+aNMQmZnCBxHVjrJEDPose4IOsO73f6FtSH+jsTLQUChyK5584Sya6QRvbMEcvNl 4xEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=fxqgizsGn0AaIQnRQ33dQkRGCaY8UuAWu0B6VsH5hUQ=; b=Uk3OeMgDiOMprC7Poz4oLRqjJQtndBas0ltPVySoUTQHEwQDVtJ5jEN6P4X75TDsFj n6KqyTCS0ykwoWzBj21YI6xswtURIdDlbL7FVal6dFaK2X/NofNHd4+C1l10Lz8lrq2E Olb6sJlFl7kEajJVycU8sqh0dy/BbMmW+Ztz3/jcg9wVPW46R/UKZ3PtOwS1qfJczOh1 /dDbeBh1RADO0/Octweci6lsl9av43fYTyz9K8z25+k+8YHHnfbS8EBItjR61nTK1LiG cla60WSdKCRNlqtY5gZ75Fl7vs93Qs2/nxHaKD1VmZveZ59sQoxAPK+BGkK0EKKFWJaZ GGkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Ujk3SHMZ; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j10sor190302wrx.15.2018.12.06.04.25.13 for (Google Transport Security); Thu, 06 Dec 2018 04:25:13 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Ujk3SHMZ; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fxqgizsGn0AaIQnRQ33dQkRGCaY8UuAWu0B6VsH5hUQ=; b=Ujk3SHMZqV7n0tte5Uw8nokkp/oH0FlKFLsFxzElZe9Z0slbPJEAJExlirmXwEUB/j qeFI1DE4ncGLA0+FkTusmS35oYv/U8UXI/jfcSKa3QRtSlDZ5L5lHaiWfjgmPd4nFF/+ vGwiweNbXNO50gJDPLGhzJri8t+wJQe4o/ND+9KUuI2L26oBYQ8hULSaxQMaAnF/iqDX Hpv9wpz9cBx90ytQ87K72yusvxoLQs9ZuSGL0uO4fMjRQ9fR2bvdzrwnlroIA0fQeMGW 0wXe8PTBJ4EYtsNFcnsc8WZUW2aYYzJizcPd8EFhPQh0QD0nyqeteBzhl8XYpy7aMdWS egGA== X-Google-Smtp-Source: AFSGD/X/KL+bObt1e53j+RE7+vhNT1qCIPw5uSMB1tFkwSYT/Sx24y8tmABr3/CaYm5nM5Ne2TIj5Q== X-Received: by 2002:adf:b783:: with SMTP id s3mr26257791wre.274.1544099113048; Thu, 06 Dec 2018 04:25:13 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:12 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 15/25] kasan, mm: perform untagged pointers comparison in krealloc Date: Thu, 6 Dec 2018 13:24:33 +0100 Message-Id: <14f6190d7846186a3506cd66d82446646fe65090.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. Tag-based KASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 5f3504e26d4c..5aabcbd32d82 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1534,7 +1534,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret = __do_krealloc(p, new_size, flags); - if (ret && p != ret) + if (ret && kasan_reset_tag(p) != kasan_reset_tag(ret)) kfree(p); return ret; From patchwork Thu Dec 6 12:24:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715791 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA05E17DB for ; Thu, 6 Dec 2018 12:25:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C67E7292C8 for ; Thu, 6 Dec 2018 12:25:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B931F2D325; Thu, 6 Dec 2018 12:25:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EE55292C8 for ; Thu, 6 Dec 2018 12:25:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B2B96B7A11; Thu, 6 Dec 2018 07:25:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 100D76B7A14; Thu, 6 Dec 2018 07:25:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C46736B7A12; Thu, 6 Dec 2018 07:25:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 406856B7A0F for ; Thu, 6 Dec 2018 07:25:18 -0500 (EST) Received: by mail-wr1-f72.google.com with SMTP id q18so92111wrx.0 for ; Thu, 06 Dec 2018 04:25:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IdZOpp4zUuzRrPM95wHwmBW5z4YbMLmf3M3IVHxt2SI=; b=O1JC3t97Xu8t4vsf8F3OOIa4diGH4T4ZsLRg064b7FXL8vsiL1pgJpVGV7NtzXcptl lMPF7ZvcUv7MDHm6/nJGVjIkMEgeVejTPePG4a8Ti+CaVWR8kFe4bvZ0oYI0SEKGj++5 qt9KcUGZh9gMZVbX6iDi7bfuOFpptZT/KSQICJP8Brl4MgFfVPbcOnCAzuFBefzapwOf xrJ5Od57llMOBqDNCXIDWV7bCvS6O/LjYp/0TiKLh8pTfmyQQefAKgiXtAfdBaGHWxWS LNRmlKnKr9haZs+Ui1QX0T6XPviVOV3oJb6Dkh8Igfap4EZUztdaahJVetsm0dly8Kxs 3mtQ== X-Gm-Message-State: AA+aEWaPcjZx3HYt9JvkwJnS/7PthUN0khMDcJKT8cG6p/By4mOl8KvB vWq3Omx9cxACfRArSAupKtgTxdmZfN+lodxFGlXpF1KxMQZhYB0vWOl8+/i03bYkHnM5Ptix77h e1GERWrVOOO481rLByp8rmMa637hgjbH1tjmsWSNe+3hLn9rjUaB0FE135vBHAJbqQ22L4vaCEQ XPykgelRFIeyhfW7yD44b/CQFXKOG/+OpLRX9w0zAboDcVF2XC3KPz/1YPS0UhSr6222pqRheM4 S9n+HIhkPBWgIFQeIDwWeHov3hs1agkFrWQR8tM8QryO6GnWjvJ6vB6bD1VyCE8wf0gCWTYYgNY nYLQvpm/VCTcbTuTZRY57pk6tRBBhUligK47WPppvy9ceAPwPyLdWqPAz6zk3vLyJndrBorILiZ Z X-Received: by 2002:adf:f605:: with SMTP id t5mr23771777wrp.229.1544099117729; Thu, 06 Dec 2018 04:25:17 -0800 (PST) X-Received: by 2002:adf:f605:: with SMTP id t5mr23771693wrp.229.1544099116249; Thu, 06 Dec 2018 04:25:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099116; cv=none; d=google.com; s=arc-20160816; b=Xazm4mGKCsf3fDOukNdwGxVbBrbQqrDP9JMN2Wo0hH1fivDIMIWjfNQJG1a09l8sBx RUKhvJEF9IdnHHhg68SFcrW936E9BKeGEQ+BD5mkBz6kEWtetXLYjCWV7hyFoy6oPusO qFKvXxU+WXWhVlRZKy7EJe3w/w8Klsig6eBP0tcp1MAz58mNVu0G9W5o8FI8rpKawFRt UHkZ4RioutQhyJYqdb4azZmBkXRCSlnjxV44VBhzKE5iau7GQUzxeOl3uu9XuLod+8Dy DEHxZ2ytH/WyaSUYxNWmzJujnCGFWoav//f8TFRA+V/KxYeU1laDhTZkZOkaK8YLTGNe Tjfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=IdZOpp4zUuzRrPM95wHwmBW5z4YbMLmf3M3IVHxt2SI=; b=h5glk2cIVAAojIh7JWdtXyY8EBJLTbyTYXVGmfigeunGVRQdEBJElpIpDLwZwMCNR7 kNCyefZsHtxf1B1FPdHryKitepB7b1TVIAXVwWOjlILz+gQ+DHTkfKWhNf2n79QKvj/K THbZWqrQieQWYXfg3eucJ3va8mX8rYkv4g4eoxSBAOgo89Tl9YDU8mcORNYuO7e/T5u2 27nTRUsqh8y7HzvgX3KV1ztKY6R9JWU+9xx9rqEb8N8C5MpVLdWi1nnR9TXOdZokUV+S kjrKoVBQ8Ibu6FuWSYNxny4nbQ8IEbYQIoNvxMMS53jnoVa2DXZvRltjk2BUK2GEpTUR lFqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=qpQGKdcX; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 194sor463956wmm.23.2018.12.06.04.25.16 for (Google Transport Security); Thu, 06 Dec 2018 04:25:16 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=qpQGKdcX; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IdZOpp4zUuzRrPM95wHwmBW5z4YbMLmf3M3IVHxt2SI=; b=qpQGKdcXOqnFUYZB/bt5DJCpQNNHJ0obEdk72KBDBL2W013ZoIJ5ZekXJRUBsHl2jl YUmkFLOCy4MeU5heiTu9fkeI6e2Bu6TK8cnijza6byT1TmWnxofK43z6c7meMWDa1/JO 7pBZOslyXAq/Kz4kVw5l4skhDI1Z9eexanfvsLMLnNHz5y8m9tYW6mOml+5z7C5YZbL1 +NS5LN/iMFG4bUnQUa5gV04OPyKHthbpqaEeK57Wi1WhYsqIJWoRL79eJ4yblunMwe0z l1lyJ+lHEU87AQb/D65a5u25AsytNdyb3hnWSH6SI+kH/NumCJKwNPF5BG/r8Roxj4aI xNkQ== X-Google-Smtp-Source: AFSGD/U0CKMPGYtaEAQJLUn3cI+kKqRUiA6og0b1bf3nTAurydbl6edHOyL56yNywAxc8Xayw55hlw== X-Received: by 2002:a1c:7706:: with SMTP id t6mr18225952wmi.57.1544099114689; Thu, 06 Dec 2018 04:25:14 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:13 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 16/25] kasan: split out generic_report.c from report.c Date: Thu, 6 Dec 2018 13:24:34 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch moves generic KASAN specific error reporting routines to generic_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by tag-based KASAN. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 4 +- mm/kasan/generic_report.c | 158 +++++++++++++++++++++++++ mm/kasan/kasan.h | 7 ++ mm/kasan/report.c | 234 +++++++++----------------------------- mm/kasan/tags_report.c | 39 +++++++ 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/generic_report.c create mode 100644 mm/kasan/tags_report.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 68ba1822f003..0a14fcff70ed 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -14,5 +14,5 @@ CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += tags.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c new file mode 100644 index 000000000000..5201d1770700 --- /dev/null +++ b/mm/kasan/generic_report.c @@ -0,0 +1,158 @@ +/* + * This file contains generic KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +static const void *find_first_bad_addr(const void *addr, size_t size) +{ + u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); + const void *first_bad_addr = addr; + + while (!shadow_val && first_bad_addr < addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +static const char *get_shadow_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + u8 *shadow_addr; + + info->first_bad_addr = find_first_bad_addr(info->access_addr, + info->access_size); + + shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); + + /* + * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * at the next shadow byte to determine the type of the bad access. + */ + if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + shadow_addr++; + + switch (*shadow_addr) { + case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + /* + * In theory it's still possible to see these shadow values + * due to a data race in the kernel code. + */ + bug_type = "out-of-bounds"; + break; + case KASAN_PAGE_REDZONE: + case KASAN_KMALLOC_REDZONE: + bug_type = "slab-out-of-bounds"; + break; + case KASAN_GLOBAL_REDZONE: + bug_type = "global-out-of-bounds"; + break; + case KASAN_STACK_LEFT: + case KASAN_STACK_MID: + case KASAN_STACK_RIGHT: + case KASAN_STACK_PARTIAL: + bug_type = "stack-out-of-bounds"; + break; + case KASAN_FREE_PAGE: + case KASAN_KMALLOC_FREE: + bug_type = "use-after-free"; + break; + case KASAN_USE_AFTER_SCOPE: + bug_type = "use-after-scope"; + break; + case KASAN_ALLOCA_LEFT: + case KASAN_ALLOCA_RIGHT: + bug_type = "alloca-out-of-bounds"; + break; + } + + return bug_type; +} + +static const char *get_wild_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + + if ((unsigned long)info->access_addr < PAGE_SIZE) + bug_type = "null-ptr-deref"; + else if ((unsigned long)info->access_addr < TASK_SIZE) + bug_type = "user-memory-access"; + else + bug_type = "wild-memory-access"; + + return bug_type; +} + +const char *get_bug_type(struct kasan_access_info *info) +{ + if (addr_has_shadow(info->access_addr)) + return get_shadow_bug_type(info); + return get_wild_bug_type(info); +} + +#define DEFINE_ASAN_REPORT_LOAD(size) \ +void __asan_report_load##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, false, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_load##size##_noabort) + +#define DEFINE_ASAN_REPORT_STORE(size) \ +void __asan_report_store##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, true, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_store##size##_noabort) + +DEFINE_ASAN_REPORT_LOAD(1); +DEFINE_ASAN_REPORT_LOAD(2); +DEFINE_ASAN_REPORT_LOAD(4); +DEFINE_ASAN_REPORT_LOAD(8); +DEFINE_ASAN_REPORT_LOAD(16); +DEFINE_ASAN_REPORT_STORE(1); +DEFINE_ASAN_REPORT_STORE(2); +DEFINE_ASAN_REPORT_STORE(4); +DEFINE_ASAN_REPORT_STORE(8); +DEFINE_ASAN_REPORT_STORE(16); + +void __asan_report_load_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_load_n_noabort); + +void __asan_report_store_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index b080b8d92812..33cc3b0e017e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow(const void *addr) +{ + return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +} + void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +const char *get_bug_type(struct kasan_access_info *info); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 5c169aa688fd..64a74f334c45 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,5 +1,5 @@ /* - * This file contains error reporting code. + * This file contains common generic and tag-based KASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -39,103 +39,34 @@ #define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) #define SHADOW_ROWS_AROUND_ADDR 2 -static const void *find_first_bad_addr(const void *addr, size_t size) -{ - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; - - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; -} +static unsigned long kasan_flags; -static bool addr_has_shadow(struct kasan_access_info *info) -{ - return (info->access_addr >= - kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -} +#define KASAN_BIT_REPORTED 0 +#define KASAN_BIT_MULTI_SHOT 1 -static const char *get_shadow_bug_type(struct kasan_access_info *info) +bool kasan_save_enable_multi_shot(void) { - const char *bug_type = "unknown-crash"; - u8 *shadow_addr; - - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); - - /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look - * at the next shadow byte to determine the type of the bad access. - */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) - shadow_addr++; - - switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: - /* - * In theory it's still possible to see these shadow values - * due to a data race in the kernel code. - */ - bug_type = "out-of-bounds"; - break; - case KASAN_PAGE_REDZONE: - case KASAN_KMALLOC_REDZONE: - bug_type = "slab-out-of-bounds"; - break; - case KASAN_GLOBAL_REDZONE: - bug_type = "global-out-of-bounds"; - break; - case KASAN_STACK_LEFT: - case KASAN_STACK_MID: - case KASAN_STACK_RIGHT: - case KASAN_STACK_PARTIAL: - bug_type = "stack-out-of-bounds"; - break; - case KASAN_FREE_PAGE: - case KASAN_KMALLOC_FREE: - bug_type = "use-after-free"; - break; - case KASAN_USE_AFTER_SCOPE: - bug_type = "use-after-scope"; - break; - case KASAN_ALLOCA_LEFT: - case KASAN_ALLOCA_RIGHT: - bug_type = "alloca-out-of-bounds"; - break; - } - - return bug_type; + return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); -static const char *get_wild_bug_type(struct kasan_access_info *info) +void kasan_restore_multi_shot(bool enabled) { - const char *bug_type = "unknown-crash"; - - if ((unsigned long)info->access_addr < PAGE_SIZE) - bug_type = "null-ptr-deref"; - else if ((unsigned long)info->access_addr < TASK_SIZE) - bug_type = "user-memory-access"; - else - bug_type = "wild-memory-access"; - - return bug_type; + if (!enabled) + clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); -static const char *get_bug_type(struct kasan_access_info *info) +static int __init kasan_set_multi_shot(char *str) { - if (addr_has_shadow(info)) - return get_shadow_bug_type(info); - return get_wild_bug_type(info); + set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); + return 1; } +__setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info) +static void print_error_description(struct kasan_access_info *info, + const char *bug_type) { - const char *bug_type = get_bug_type(info); - pr_err("BUG: KASAN: %s in %pS\n", bug_type, (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", @@ -143,25 +74,9 @@ static void print_error_description(struct kasan_access_info *info) info->access_addr, current->comm, task_pid_nr(current)); } -static inline bool kernel_or_module_addr(const void *addr) -{ - if (addr >= (void *)_stext && addr < (void *)_end) - return true; - if (is_module_address((unsigned long)addr)) - return true; - return false; -} - -static inline bool init_task_stack_addr(const void *addr) -{ - return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); -} - static DEFINE_SPINLOCK(report_lock); -static void kasan_start_report(unsigned long *flags) +static void start_report(unsigned long *flags) { /* * Make sure we don't end up in loop. @@ -171,7 +86,7 @@ static void kasan_start_report(unsigned long *flags) pr_err("==================================================================\n"); } -static void kasan_end_report(unsigned long *flags) +static void end_report(unsigned long *flags) { pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); @@ -249,6 +164,22 @@ static void describe_object(struct kmem_cache *cache, void *object, describe_object_addr(cache, object, addr); } +static inline bool kernel_or_module_addr(const void *addr) +{ + if (addr >= (void *)_stext && addr < (void *)_end) + return true; + if (is_module_address((unsigned long)addr)) + return true; + return false; +} + +static inline bool init_task_stack_addr(const void *addr) +{ + return addr >= (void *)&init_thread_union.stack && + (addr <= (void *)&init_thread_union.stack + + sizeof(init_thread_union.stack)); +} + static void print_address_description(void *addr) { struct page *page = addr_to_page(addr); @@ -326,29 +257,38 @@ static void print_shadow_for_address(const void *addr) } } +static bool report_enabled(void) +{ + if (current->kasan_depth) + return false; + if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + return true; + return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); +} + void kasan_report_invalid_free(void *object, unsigned long ip) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); pr_err("\n"); print_address_description(object); pr_err("\n"); print_shadow_for_address(object); - kasan_end_report(&flags); + end_report(&flags); } static void kasan_report_error(struct kasan_access_info *info) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); - print_error_description(info); + print_error_description(info, get_bug_type(info)); pr_err("\n"); - if (!addr_has_shadow(info)) { + if (!addr_has_shadow(info->access_addr)) { dump_stack(); } else { print_address_description((void *)info->access_addr); @@ -356,41 +296,7 @@ static void kasan_report_error(struct kasan_access_info *info) print_shadow_for_address(info->first_bad_addr); } - kasan_end_report(&flags); -} - -static unsigned long kasan_flags; - -#define KASAN_BIT_REPORTED 0 -#define KASAN_BIT_MULTI_SHOT 1 - -bool kasan_save_enable_multi_shot(void) -{ - return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); - -void kasan_restore_multi_shot(bool enabled) -{ - if (!enabled) - clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); - -static int __init kasan_set_multi_shot(char *str) -{ - set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); - return 1; -} -__setup("kasan_multi_shot", kasan_set_multi_shot); - -static inline bool kasan_report_enabled(void) -{ - if (current->kasan_depth) - return false; - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) - return true; - return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); + end_report(&flags); } void kasan_report(unsigned long addr, size_t size, @@ -398,7 +304,7 @@ void kasan_report(unsigned long addr, size_t size, { struct kasan_access_info info; - if (likely(!kasan_report_enabled())) + if (likely(!report_enabled())) return; disable_trace_on_warning(); @@ -411,41 +317,3 @@ void kasan_report(unsigned long addr, size_t size, kasan_report_error(&info); } - - -#define DEFINE_ASAN_REPORT_LOAD(size) \ -void __asan_report_load##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, false, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_load##size##_noabort) - -#define DEFINE_ASAN_REPORT_STORE(size) \ -void __asan_report_store##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, true, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_store##size##_noabort) - -DEFINE_ASAN_REPORT_LOAD(1); -DEFINE_ASAN_REPORT_LOAD(2); -DEFINE_ASAN_REPORT_LOAD(4); -DEFINE_ASAN_REPORT_LOAD(8); -DEFINE_ASAN_REPORT_LOAD(16); -DEFINE_ASAN_REPORT_STORE(1); -DEFINE_ASAN_REPORT_STORE(2); -DEFINE_ASAN_REPORT_STORE(4); -DEFINE_ASAN_REPORT_STORE(8); -DEFINE_ASAN_REPORT_STORE(16); - -void __asan_report_load_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_load_n_noabort); - -void __asan_report_store_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c new file mode 100644 index 000000000000..8af15e87d3bc --- /dev/null +++ b/mm/kasan/tags_report.c @@ -0,0 +1,39 @@ +/* + * This file contains tag-based KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} From patchwork Thu Dec 6 12:24:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715787 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 693965C90 for ; Thu, 6 Dec 2018 12:25:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55D3D292C8 for ; Thu, 6 Dec 2018 12:25:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 48281297D0; Thu, 6 Dec 2018 12:25:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32B5E2D93C for ; Thu, 6 Dec 2018 12:25:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E66A86B7A0F; Thu, 6 Dec 2018 07:25:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D7C006B7A11; Thu, 6 Dec 2018 07:25:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B58176B7A13; Thu, 6 Dec 2018 07:25:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id 537556B7A11 for ; Thu, 6 Dec 2018 07:25:18 -0500 (EST) Received: by mail-wm1-f72.google.com with SMTP id p16so205538wmc.5 for ; Thu, 06 Dec 2018 04:25:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8uSAA6A1aRTWO16DLBjPQUae1SegzQbJDGVzmV8tpfs=; b=EyW+syhnkyCWbIoabH1FqYF/VD450dRshdLwxjXOH431YYgVhD+wPB5kEc19sWQyPI 6gfSpm+kCWh9heEuy9Br1IXdD94zLt9v+omiiK1wfdf19EF3W+5lSPF9GCXqZ3czkvdR +yOvH9ZJ15ox7f3cmJVPlA7JdPiTS1AiTEqM9NXBZrqrA3OFUZ47Q1kT99ZWr+xwzFyq 9djtAF1NRXcNBGupWlIr9VdLu1GBb20jWICrD/GFFeVq2GiqArROcjN/F3nfJgL2rUxz 9kXsVGXy9VwsRPkXxSMJLH5k2yWspk/pDsqBN+dy19xb6tkSEdJDg8avRNHBwUJg9pw2 9qSg== X-Gm-Message-State: AA+aEWaTPdFjApmXbp96VeWDqPdVEbpbziIRDEQBkBXp9MQ6VHRaZTvQ 12Y/vJV4hIXg60GniaS1B+b1hLiRCHUpsrmpTyoH6WXywGa4iK7IJOdwXgrx1fGw7UhWlOz80H/ mAQyESq2Bg8lRNohlrqQhhbxOJbvOtRpivBttOGFwEOC4t2f9WjiJ83/YHX0Jr+W9/sRLmU4gFC gGjrWhyB2mk004hvQqxIa/l5oaj5xepvdZ1ATg8JD8j7tl/qg5EZrLEbwOnvhI72CT9dPSPXZfA NDU26tct2N5oOkOyiaaDH1QW6LrmG3c+a/1mGZNqktT3L6TuWpqi+jvuV33pqV/XMuNZqoCUGkd caekO2TCo9HwoiqvPWOB7wemG4WY5xiCnhDrMlgiYnVacVEvvBivR+tjD7tU5/Z2QGYa7Ud1xTu r X-Received: by 2002:adf:80cf:: with SMTP id 73mr25015458wrl.57.1544099117772; Thu, 06 Dec 2018 04:25:17 -0800 (PST) X-Received: by 2002:adf:80cf:: with SMTP id 73mr25015401wrl.57.1544099116865; Thu, 06 Dec 2018 04:25:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099116; cv=none; d=google.com; s=arc-20160816; b=jXJAr0T64IY2LPI2x8d6om+z5UtL6jqHmf8PXMYFLf9SS0naO61WLtpt0QwEpGtsG1 GXg289R65d1U5AFHj2aSCIfOPSQMes2bmHGXln1kK+8fyVly9+w0cdTj9LfEfOLQrqYu uDdRPcvPPOyRl9iJtGfK9ihgNA88iDPVUqe7I8rEcmnSA21FnuCNkQU/l8gaxZNLlQjd 3aYOqubiyBA1RM3jnupWZr85yVbJjOPHvNkr9i6yg6sv5EHGhwXmo9R5lh4uNwgZySif nmqvORBqfXoEwm6mU6c9kDGUUCrwycg5pIEySHbWhSg8WjFNuE68xIv4mDE8Y1tpm+iS lbDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=8uSAA6A1aRTWO16DLBjPQUae1SegzQbJDGVzmV8tpfs=; b=rAp2fG2BHBcPyYIZwKVdLB3TwnLHrUTRqY/eu4HkSHbz0SAqOCC06wV4zEVAFGTeeH Ci1dcpa8yU/RlgXWUE6/uT0s4XTQxuTD/NvGyOp2CzTy2RqNkccZGIA+pt720Jkp8BX6 tNPTpM+VvKMNTRhUPrQ2nLYj7vk/0dEG7+XSHxFd6oekQTgMXsTGH+OEJYY+0xMSpFpo JNxYBYgVZiuEbLUrj2qdBvVgd+Id0wuKhNirPBDFw3kIftQbNT38emOSBkUBBWnyYowh +roHOP5PiO1RBkpSIn8FCYoVyHty6PZZesBhVbIbkQjIw+7CbEm2o1Ce3oSfnJ3tNd8j q82Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=tLSUtkLS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id s15sor463512wmh.13.2018.12.06.04.25.16 for (Google Transport Security); Thu, 06 Dec 2018 04:25:16 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=tLSUtkLS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8uSAA6A1aRTWO16DLBjPQUae1SegzQbJDGVzmV8tpfs=; b=tLSUtkLSM4VsoseHSxkk9UlgsSF9LYHyfSyPHzE/hgw9NdIVRYjEYCt2b2sZUlJG/1 uqDS28//AknGUJ6zk2zY4PKdQ8+4tirjbHwaZwuxUYII8/CJ2BuOvvpkEq/aDfpTbtoA VH3gjYgJrXPk8eyVj8GZp51F8r6L+zIZ6ZpfxVzGvB6LVe2umAITfjSip/j218/bx97S HEvzXQSP2vxF0z4FWOYeSDWfMGAnX7lb9LTbaJAupapKa+mQu+Pwrsg7gUZRweVO212H R6tMZh/eBx1slp1fOaIraS+NDJMvXZ3LH07qR8gRV2FudaCq+whsbKrPssVnGqelpPwL B8eA== X-Google-Smtp-Source: AFSGD/V4zIb54Ou409sQv+ZnuWNPIZMqrE2tacqLuIwacW/LC9rLfpzzJFwYCO48owsIvI7L30/Cmg== X-Received: by 2002:a1c:4681:: with SMTP id t123mr9124048wma.24.1544099116333; Thu, 06 Dec 2018 04:25:16 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:15 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 17/25] kasan: add bug reporting routines for tag-based mode Date: Thu, 6 Dec 2018 13:24:35 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds rountines, that print tag-based KASAN error reports. Those are quite similar to generic KASAN, the difference is: 1. The way tag-based KASAN finds the first bad shadow cell (with a mismatching tag). Tag-based KASAN compares memory tags from the shadow memory to the pointer tag. 2. Tag-based KASAN reports all bugs with the "KASAN: invalid-access" header. Also simplify generic KASAN find_first_bad_addr. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/generic_report.c | 16 ++++------- mm/kasan/kasan.h | 5 ++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ mm/kasan/tags_report.c | 18 +++++++++++++ 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index 5201d1770700..a4604cceae59 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -33,16 +33,13 @@ #include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr(const void *addr, size_t size) +void *find_first_bad_addr(void *addr, size_t size) { - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; + void *p = addr; - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; + while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p))) + p += KASAN_SHADOW_SCALE_SIZE; + return p; } static const char *get_shadow_bug_type(struct kasan_access_info *info) @@ -50,9 +47,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info) const char *bug_type = "unknown-crash"; u8 *shadow_addr; - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 33cc3b0e017e..82a23b23ff93 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -119,6 +119,7 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void kasan_report(unsigned long addr, size_t size, @@ -139,10 +140,14 @@ static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #ifdef CONFIG_KASAN_SW_TAGS +void print_tags(u8 addr_tag, const void *addr); + u8 random_tag(void); #else +static inline void print_tags(u8 addr_tag, const void *addr) { } + static inline u8 random_tag(void) { return 0; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 64a74f334c45..214d85035f99 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) } __setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info, - const char *bug_type) +static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - bug_type, (void *)info->ip); + get_bug_type(info), (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, info->access_addr, current->comm, task_pid_nr(current)); @@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip) start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); + print_tags(get_tag(object), reset_tag(object)); + object = reset_tag(object); pr_err("\n"); print_address_description(object); pr_err("\n"); @@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags); } -static void kasan_report_error(struct kasan_access_info *info) -{ - unsigned long flags; - - start_report(&flags); - - print_error_description(info, get_bug_type(info)); - pr_err("\n"); - - if (!addr_has_shadow(info->access_addr)) { - dump_stack(); - } else { - print_address_description((void *)info->access_addr); - pr_err("\n"); - print_shadow_for_address(info->first_bad_addr); - } - - end_report(&flags); -} - void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { struct kasan_access_info info; + void *tagged_addr; + void *untagged_addr; + unsigned long flags; if (likely(!report_enabled())) return; disable_trace_on_warning(); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr = reset_tag(tagged_addr); + + info.access_addr = tagged_addr; + if (addr_has_shadow(untagged_addr)) + info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; - kasan_report_error(&info); + start_report(&flags); + + print_error_description(&info); + if (addr_has_shadow(untagged_addr)) + print_tags(get_tag(tagged_addr), info.first_bad_addr); + pr_err("\n"); + + if (addr_has_shadow(untagged_addr)) { + print_address_description(untagged_addr); + pr_err("\n"); + print_shadow_for_address(info.first_bad_addr); + } else { + dump_stack(); + } + + end_report(&flags); } diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 8af15e87d3bc..573c51d20d09 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -37,3 +37,21 @@ const char *get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } + +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *p = reset_tag(addr); + void *end = p + size; + + while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) + p += KASAN_SHADOW_SCALE_SIZE; + return p; +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); +} From patchwork Thu Dec 6 12:24:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715793 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 728AD109C for ; Thu, 6 Dec 2018 12:25:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6034D292C8 for ; Thu, 6 Dec 2018 12:25:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 535D32D325; Thu, 6 Dec 2018 12:25:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B91C8292C8 for ; Thu, 6 Dec 2018 12:25:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A7B16B7A13; Thu, 6 Dec 2018 07:25:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 231EB6B7A14; Thu, 6 Dec 2018 07:25:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AD256B7A16; Thu, 6 Dec 2018 07:25:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id 9EF5C6B7A13 for ; Thu, 6 Dec 2018 07:25:19 -0500 (EST) Received: by mail-wm1-f71.google.com with SMTP id g3so216311wmf.1 for ; Thu, 06 Dec 2018 04:25:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FaduiTR0nXd7TQE8kfxwL/3d+/oPKBLmf6eac6TNL3k=; b=NBN/aUYq1QVTCJgItsjMOV5Ec8pvVAoEumb7X0txOYUQBSRRurVKPqlXmJwwC3deqX FGXbblGVh8RlsQt+h6wUQR9QwfrhuoXn7qQ8Q+lc1gwy3cdN2EfBlbkWSBz3OislkRvd gR/N4DdgX3Pqfv3kb6BKu+NDm4yJYMKMugu+h6PasvC0hj0r4u/f/wWOfZxxLUmBD1n1 a3rhVB8mZFMqg8u9ZYnTDPf++7Ucx9iN9TlDkAZebccnOKiG0sA0Rt7m5gYhi1SNu2se deH9a1AVB4JOJgFkrntOwhY1q7YdIL6S/xZqKR8Cqdrfe4ZbDhfqBfjLjSCCCPBfbcbl Tyzg== X-Gm-Message-State: AA+aEWbc6O1F4UExZ4ZazxkVcFxNfO3n3v7jrjN87AH/pD74ZEqngVgT FfA74W7DAhcqde4ziKQrzwfkeYbaxRQCGor+5r8CYnWeMqVTV1o+rbo+EMOJnLZ1hig3OtI8Zuf rmB9QCVtZ41Zrm+vK0OFZCP6U3K0hK7jqeode3xreS24bLHa87na9xkuqST+17eXSOo/ng8ynTk ifv8AEOaA7Z8p+b4ZW5V7scsnFz8KgRbocjREL9n0hxZRV8npATyyCKgxa/+IxE77/Shi6jnhZT flo6pPwi5mLQtknQQYdXTpw4zav3lJJ7qDDlNC4qMBfdLAS18FntP2al25JNO6mjcUkD/0sdaQq 4wgwv938HvFajF25M4PXp29Kot+a4Hm2NcskJvGlYpVubp+s+7zIxKIZVTql5IrYK7qQvHnf7ku f X-Received: by 2002:a1c:868a:: with SMTP id i132mr15396357wmd.49.1544099119149; Thu, 06 Dec 2018 04:25:19 -0800 (PST) X-Received: by 2002:a1c:868a:: with SMTP id i132mr15396298wmd.49.1544099118348; Thu, 06 Dec 2018 04:25:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099118; cv=none; d=google.com; s=arc-20160816; b=P53P+Tpw5f1rX/BhkCw14mYtT8IRrDEzX3uwm8ooZ7z2+EXF8PNiEXWhIjTopbK7ax sYtepJ+zR+CU9LSOD4FIdvZuWfCixhuvbUncg7MHxG9f84tz3yIsT8cLWC98QWdKYYSN 2gFRl7pwgYhDgwBfTqUL3Ee2WaehhZEFTv7xgi5rOHTN3DA8sM41HAJJZSV7HDQChZCL idCEa0ftGqo74pnd0kOVpEGQiJUv9YYh0jK8TULo3NgpSNKdUK+SANPyy7Jq3gZTgqMh ZYsp/h0ZqspmSKf4Awi10aB6yPK+HFEyyi6RV38+frvIZ7kNiAvMp5HG+GInoWdkBKxe ZGXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=FaduiTR0nXd7TQE8kfxwL/3d+/oPKBLmf6eac6TNL3k=; b=Pbl6YvsdGXtg4uUrWsdpHPR+YI8Ieg4MLN0pE/qI7Ec/iqua+HzehsYkH7VzSxZCOe G5+fF29sit+84MGXmcA2gM189j1nqZ+gJFTcW0lTBegR1dA5u0tELDStEqneLlsYGaNY U60xLO5kDOCoHkzCXoVVULxH7M6/nfuwwUTAuKSxnorL+avNaPlX+Fc0dOankf+hX+A8 ZZThWSubhOlwa4wuMLvYUAEoz6ntqD5oE7BprSv02vE7zPM4Xx0s4Sbog+C22PgPsl5j GaizYPcG6dxuSo/YY90hJljhJsD9TPmHjKIYiV3MMIDCC9q1vMrt7Dax8Nq16ql+uNE6 pPdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=amkNNJFo; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t2sor447350wmh.9.2018.12.06.04.25.18 for (Google Transport Security); Thu, 06 Dec 2018 04:25:18 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=amkNNJFo; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FaduiTR0nXd7TQE8kfxwL/3d+/oPKBLmf6eac6TNL3k=; b=amkNNJFo/2s75YdZiL5n/4pvWc/wOQ5QEZTyzJHRBocvyLknCN88PMDvlrNNK9gwNE AmP8SUBWlWQUDUWncQqFRmp6dXktNfMvWYNee0okybhgmzKzdmWhGpx7eT8OTM0tuxCr yHRpBgiG9Q9Lb121RHPzhivLL7zymiGKbF+87vY6gQTCGuObr8dCNOYldKpjaZ/F7tbj QSJJJsuNLdT8o+8BNy/DeiQD99TXxSCjPa/b+qc+Wd537R3TivAl4UuDa7B/uHzpPWyb YIUHha77STDq8QIfw/CO/h7jdBKIpVlPuH3SfA1z5ySRzid61e9FSti/Uf8Fzr9GGYTk FIMg== X-Google-Smtp-Source: AFSGD/U/BIIiVj5yOvWgeJVa0sA+MatzAWdvqK0fNNEjF6B0VUUl4+dlpOmebsfE/BPopXVlB79eDg== X-Received: by 2002:a1c:2457:: with SMTP id k84mr19289771wmk.139.1544099117816; Thu, 06 Dec 2018 04:25:17 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:17 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 18/25] mm: move obj_to_index to include/linux/slab_def.h Date: Thu, 6 Dec 2018 13:24:36 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP While with SLUB we can actually preassign tags for caches with contructors and store them in pointers in the freelist, SLAB doesn't allow that since the freelist is stored as an array of indexes, so there are no pointers to store the tags. Instead we compute the tag twice, once when a slab is created before calling the constructor and then again each time when an object is allocated with kmalloc. Tag is computed simply by taking the lowest byte of the index that corresponds to the object. However in kasan_kmalloc we only have access to the objects pointer, so we need a way to find out which index this object corresponds to. This patch moves obj_to_index from slab.c to include/linux/slab_def.h to be reused by KASAN. Acked-by: Christoph Lameter Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/slab_def.h | 13 +++++++++++++ mm/slab.c | 13 ------------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h index 3485c58cfd1c..9a5eafb7145b 100644 --- a/include/linux/slab_def.h +++ b/include/linux/slab_def.h @@ -104,4 +104,17 @@ static inline void *nearest_obj(struct kmem_cache *cache, struct page *page, return object; } +/* + * We want to avoid an expensive divide : (offset / cache->size) + * Using the fact that size is a constant for a particular cache, + * we can replace (offset / cache->size) by + * reciprocal_divide(offset, cache->reciprocal_buffer_size) + */ +static inline unsigned int obj_to_index(const struct kmem_cache *cache, + const struct page *page, void *obj) +{ + u32 offset = (obj - page->s_mem); + return reciprocal_divide(offset, cache->reciprocal_buffer_size); +} + #endif /* _LINUX_SLAB_DEF_H */ diff --git a/mm/slab.c b/mm/slab.c index 27859fb39889..d2f827316dfc 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -406,19 +406,6 @@ static inline void *index_to_obj(struct kmem_cache *cache, struct page *page, return page->s_mem + cache->size * idx; } -/* - * We want to avoid an expensive divide : (offset / cache->size) - * Using the fact that size is a constant for a particular cache, - * we can replace (offset / cache->size) by - * reciprocal_divide(offset, cache->reciprocal_buffer_size) - */ -static inline unsigned int obj_to_index(const struct kmem_cache *cache, - const struct page *page, void *obj) -{ - u32 offset = (obj - page->s_mem); - return reciprocal_divide(offset, cache->reciprocal_buffer_size); -} - #define BOOT_CPUCACHE_ENTRIES 1 /* internal cache of cache description objs */ static struct kmem_cache kmem_cache_boot = { From patchwork Thu Dec 6 12:24:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715795 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1699109C for ; Thu, 6 Dec 2018 12:25:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE6C7292C8 for ; Thu, 6 Dec 2018 12:25:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D1CEC2D325; Thu, 6 Dec 2018 12:25:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BEE31292C8 for ; Thu, 6 Dec 2018 12:25:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A29746B7A14; Thu, 6 Dec 2018 07:25:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 960026B7A17; Thu, 6 Dec 2018 07:25:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76EAD6B7A18; Thu, 6 Dec 2018 07:25:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id E5E2A6B7A14 for ; Thu, 6 Dec 2018 07:25:21 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id a9so84207wrs.6 for ; Thu, 06 Dec 2018 04:25:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DJ+U9N4tqltdeWL7oxARVkOoswN2PjjxOCfLHWSAnPk=; b=N3qusvQKZ2j4HDY0wxtDsjunlC/+xvg7ZBk0fwGq9weh+G1BRc9NsR0qM3w4J5Z7Ju +6am/efZ2jlCTQ/FACC2p8yoXi7lfqt7zuDlpGxT04Ty69r4igQ0gl2TEFFZ72Fbt9cW VUPybvqR6JmYBWchUev9NtcyWAH6O6IJNxoWRzDQJsPDbeHHCT/fAjBGpT607ZA4MZEf xJdVhpXfVlaGX47jTrtySxjRu93uyqeswbaIrsIjWHlHWZMP9VFbK822JrcKhJ1Qn8FD 8rqjMAOZpd8nLJA9PguZAQYv/bix7UJ8262v2bJ+MUuw3VVMa/a9uaKaj5irqukvjBJT pgGw== X-Gm-Message-State: AA+aEWbBKhTZvOMVA4R0Ptz40MgXfpUMyRg8xoaS3PU5cx3zogtNUcp8 /3GfR0Thlbnz/6LUSf03TRDYjUoF8B21RCeitUSpdG8EP1cr2ube36aKFApdq1sb1jJxFcUAxlN bsgHV0M+2MGMFYJ2C+4cqy8pcY5lAPcH1XkZJoCwwGUm/uuZJOJiKXsXpt00C1tiVW8TIG9SBM3 3NPBcZHAJMk1kbXRhde3PG5WALE40jzGvva2DeEU/m9pWbyYArJCu7R2nZbQiAwQPVHpSkL5r3B n8aeNNIeZ9CTQHqhXDy3AF3845fkBF4U6mYToyDc5+zRh1XLre5FCdrHLuFfOMyjcWX/inh2Z30 ofRO/Fvy5cwL7o+yjs56Y41u5iH48K/gcyCI/yXvRfkeCTdrZYZ+INdZ93ZY5fLXWXDm+AayyaS r X-Received: by 2002:adf:c888:: with SMTP id k8mr27037061wrh.6.1544099121437; Thu, 06 Dec 2018 04:25:21 -0800 (PST) X-Received: by 2002:adf:c888:: with SMTP id k8mr27036981wrh.6.1544099120203; Thu, 06 Dec 2018 04:25:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099120; cv=none; d=google.com; s=arc-20160816; b=SbURDh8k6mQpsGDu7I6azGRxLyf0eYw6JGVC9aibT2j1JI5TD0bwqEIMP/saOITDp/ VpopJ61NfHorFqnfzR7vpp6VOZWlec9onGnlxb8y/phrXGBKOE4LdDTrvlhv9gtD6Rgr sMeUvUyr3JLU+L+XFo7edxaptTSSbVqzjt+3i0Ons3pAw2QsdZR7LVKdava4IFt2Kwlf 2y98RoHMsutlJy6e1zh80B8mAwDeWdpkJ/l8CbfoqoIgXpJMZYlzx6eWOMmlDLdSNu/G YfoZLYHStKhsRXBF6miAZmcsosXTK3p2J+CHnb/GBkUQfaWZBn/bvkKhWb3DdzFAeW3O RcLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=DJ+U9N4tqltdeWL7oxARVkOoswN2PjjxOCfLHWSAnPk=; b=vtJZJXJtvEIY1+pYBU9eFzq2N+DwyBUAbTobDwpGucVbVohtu3gHX6H19ytTcwJ7l1 uHuDX07hfjyNuDgrIJCcoNILj2Qdqpp/uBkbfF+KF2xEZCGG0r8puis7RtgmrWpVAdj9 Qrwuca4kJW30v7yLzp/KE+ZJq4thYG2KxF3OVSHcT44n6LzOlV0OhgXTSGj1rn/PqItd lZ36rT3MCYsCjaC0IxyPEoTj9/j4TrJIFesVXg3Xvs7iSPslr1MDzdVABmF1YO/g2UYW DGq8M6ZvzoPsuR1es+dvfAdgE0F3aVB/0rxOUyVOCOU8JH6vec4n1sU+HHa8uNPItMdD wI7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=m2D25hFr; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 3sor168172wrs.49.2018.12.06.04.25.20 for (Google Transport Security); Thu, 06 Dec 2018 04:25:20 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=m2D25hFr; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DJ+U9N4tqltdeWL7oxARVkOoswN2PjjxOCfLHWSAnPk=; b=m2D25hFrHk+uO36rvOq4/Ol9GKyuxG7XDF4ZkpDX5W8/7eDdj06hB0AKfend8ZmE/8 YYSwXXJGTrsoDf6ahrcVCoYEtAeRV37i6U+j+r7jy9HF4lGWPkQLIGQlf3+VMNgr3ymT wZk9dHFSsKqAja23gea4iejuSS8Snzfn36+3kSALpa6Lr+eqW3we0xkoRqiwqgGMwj5s MOWVmFIyA+qIUc/K2UCiapeFUfq95mp24Aqida26TgydMVJeC7Wsgcv7sTqpAHAxsimT 2VrkBFAbPYHDSl5p5CNUG3XHr+1BAhgeY/gRalHphFA7UOcz/jTHnmsreBsV0BhId/qW auCw== X-Google-Smtp-Source: AFSGD/VlGE+6PF29ssIsFVXzfKxekeQMXwms0kU+kR+9n1ojPpYXGpTFO+O6eR43lym5cBNik7mtnw== X-Received: by 2002:adf:f4c2:: with SMTP id h2mr24250114wrp.21.1544099119535; Thu, 06 Dec 2018 04:25:19 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:18 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 19/25] kasan: add hooks implementation for tag-based mode Date: Thu, 6 Dec 2018 13:24:37 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds tag-based KASAN specific hooks implementation and adjusts common generic and tag-based KASAN ones. 1. When a new slab cache is created, tag-based KASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16). 2. On each kmalloc tag-based KASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer. 3. On each kfree tag-based KASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs. The rest of the logic of the hook implementation is very much similar to the one provided by generic KASAN. Tag-based KASAN saves allocation and free stack metadata to the slab object the same way generic KASAN does. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 116 ++++++++++++++++++++++++++++++++++++++-------- mm/kasan/kasan.h | 8 ++++ mm/kasan/tags.c | 48 +++++++++++++++++++ 3 files changed, 153 insertions(+), 19 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 7134e75447ff..27f0cae336c9 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -140,6 +140,13 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_poison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -148,11 +155,24 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) void kasan_unpoison_shadow(const void *address, size_t size) { - kasan_poison_shadow(address, size, 0); + u8 tag = get_tag(address); + + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_unpoison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + + kasan_poison_shadow(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; + + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + *shadow = tag; + else + *shadow = size & KASAN_SHADOW_MASK; } } @@ -200,8 +220,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + if (unlikely(PageHighMem(page))) + return; + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) @@ -218,6 +239,9 @@ void kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + return 0; + return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -232,6 +256,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { unsigned int orig_size = *size; + unsigned int redzone_size; int redzone_adjust; /* Add alloc meta. */ @@ -239,20 +264,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size += sizeof(struct kasan_alloc_meta); /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); if (redzone_adjust > 0) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); + max(*size, cache->object_size + redzone_size)); /* * If the metadata doesn't fit, don't enable KASAN at all. @@ -265,6 +290,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, return; } + cache->align = round_up(cache->align, KASAN_SHADOW_SCALE_SIZE); + *flags |= SLAB_KASAN; } @@ -309,6 +336,32 @@ void kasan_poison_object_data(struct kmem_cache *cache, void *object) KASAN_KMALLOC_REDZONE); } +/* + * Since it's desirable to only call object contructors once during slab + * allocation, we preassign tags to all such objects. Also preassign tags for + * SLAB_TYPESAFE_BY_RCU slabs to avoid use-after-free reports. + * For SLAB allocator we can't preassign tags randomly since the freelist is + * stored as an array of indexes instead of a linked list. Assign tags based + * on objects indexes, so that objects that are next to each other get + * different tags. + * After a tag is assigned, the object always gets allocated with the same tag. + * The reason is that we can't change tags for objects with constructors on + * reallocation (even for non-SLAB_TYPESAFE_BY_RCU), because the constructor + * code can save the pointer to the object somewhere (e.g. in the object + * itself). Then if we retag it, the old saved pointer will become invalid. + */ +static u8 assign_tag(struct kmem_cache *cache, const void *object, bool new) +{ + if (!cache->ctor && !(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return new ? KASAN_TAG_KERNEL : random_tag(); + +#ifdef CONFIG_SLAB + return (u8)obj_to_index(cache, virt_to_page(object), (void *)object); +#else + return new ? random_tag() : get_tag(object); +#endif +} + void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; @@ -319,6 +372,9 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + object = set_tag(object, assign_tag(cache, object, true)); + return (void *)object; } @@ -327,15 +383,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) return kasan_kmalloc(cache, object, cache->object_size, flags); } +static inline bool shadow_invalid(u8 tag, s8 shadow_byte) +{ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return shadow_byte < 0 || + shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + else + return tag != (u8)shadow_byte; +} + static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { s8 shadow_byte; + u8 tag; + void *tagged_object; unsigned long rounded_up_size; + tag = get_tag(object); + tagged_object = object; + object = reset_tag(object); + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { - kasan_report_invalid_free(object, ip); + kasan_report_invalid_free(tagged_object, ip); return true; } @@ -344,20 +415,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return false; shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); + if (shadow_invalid(tag, shadow_byte)) { + kasan_report_invalid_free(tagged_object, ip); return true; } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) return false; set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); quarantine_put(get_free_info(cache, object), cache); - return true; + + return IS_ENABLED(CONFIG_KASAN_GENERIC); } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -370,6 +443,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, { unsigned long redzone_start; unsigned long redzone_end; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -382,14 +456,18 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(object, size); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + tag = assign_tag(cache, object, false); + + /* Tag is ignored in set_tag without CONFIG_KASAN_SW_TAGS */ + kasan_unpoison_shadow(set_tag(object, tag), size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); - return (void *)object; + return set_tag(object, tag); } EXPORT_SYMBOL(kasan_kmalloc); @@ -439,7 +517,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { + if (reset_tag(ptr) != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -452,7 +530,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (ptr != page_address(virt_to_head_page(ptr))) + if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 82a23b23ff93..ea51b2d898ec 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,10 +12,18 @@ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KASAN_TAG_INVALID +#endif + #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ /* diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 1c4e7ce2e6fe..1d1b79350e28 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -78,15 +78,60 @@ void *kasan_reset_tag(const void *addr) void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { + u8 tag; + u8 *shadow_first, *shadow_last, *shadow; + void *untagged_addr; + + if (unlikely(size == 0)) + return; + + tag = get_tag((const void *)addr); + + /* + * Ignore accesses for pointers tagged with 0xff (native kernel + * pointer tag) to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KASAN_TAG_KERNEL (0xFF)). + */ + if (tag == KASAN_TAG_KERNEL) + return; + + untagged_addr = reset_tag((const void *)addr); + if (unlikely(untagged_addr < + kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { + kasan_report(addr, size, write, ret_ip); + return; + } + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + size - 1); + for (shadow = shadow_first; shadow <= shadow_last; shadow++) { + if (*shadow != tag) { + kasan_report(addr, size, write, ret_ip); + return; + } + } } #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -98,15 +143,18 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { + kasan_poison_shadow((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); From patchwork Thu Dec 6 12:24:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715803 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 975F81731 for ; Thu, 6 Dec 2018 12:25:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 85CBC292C8 for ; Thu, 6 Dec 2018 12:25:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 798332D325; Thu, 6 Dec 2018 12:25:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D23B9292C8 for ; Thu, 6 Dec 2018 12:25:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A4A446B7A17; Thu, 6 Dec 2018 07:25:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9FAE56B7A19; Thu, 6 Dec 2018 07:25:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84D0E6B7A1A; Thu, 6 Dec 2018 07:25:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 30C066B7A17 for ; Thu, 6 Dec 2018 07:25:23 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id d11so89062wrw.4 for ; Thu, 06 Dec 2018 04:25:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8QlaS0qsWG1yDbCjfy8LBUBTA+Rw+V2QPlm0PJyeSSg=; b=e02HMw7nUIAdnTE8AXKcjyNy3JboWHoiufWgkUhtMgf2loFmuvnEyISDJKS5koRzH8 Y3mEORVbuX/0+dRxDmgChllFrhBxSIeUUuFvo6GqLn3amAPMuq/HCB1sWZVCP94UG3Ha tTCAJ/Q8UGork0HVAZAJ4GdZ6e0D5i4kna37J/WY0fH2g2kaklPQmtQMWjKZ7VC18ajs czc2ZXH7T55eKI4SFPj1UoLceqn3T0OtTxU5eJNwLIaGHhUyaQDvqg3Ix/lcNZ2nYgFS NXMUUMvvSmRRlMqryvqkmqO1uc8tU7R8kPAEd87YwCeM5DT+QtUGRy/bx5rQvbTfLyRm WB1g== X-Gm-Message-State: AA+aEWaMGStTnK9hLyOcz5JhX/iQpR0ZivqyjsRvnHpN6Jy5gbiMIZUO UUDsvkMuVjv/MqvnP8WtyHCL7aDw/0xPen7az4Z/gRiOauxu/xfIjOhQ0dfNZMxsz3MMlJidmuf ysQY7vHiL6q2HZ+ms7O/+icg6ndrnbKWgrasxw/QcFLEsUxsl3rI9Mpsc/COg/n8dAFI3CyMbTL o7Pb5Z506DBULOQzIVPNEYPxvFTgDgIk7AnDypC6HjdQrVlmOiOuATSYFOg7IyyQO5QwyrB7ey7 Z5pRIf1FePyKbGR2Tkh5F4wP2oIhdePOk4GAWMjxU9MJcmKKu+INyXAq+bwd6Zi72gEkE4Ep9wA Blv63TOBawZFMymf4uxzbb6u6IAPrz8bM9MpMUDgVHtBTuYKo/1gVViwPwU8+JIvnBGU+CAtD0l s X-Received: by 2002:a1c:8acd:: with SMTP id m196mr10509499wmd.120.1544099122709; Thu, 06 Dec 2018 04:25:22 -0800 (PST) X-Received: by 2002:a1c:8acd:: with SMTP id m196mr10509434wmd.120.1544099121740; Thu, 06 Dec 2018 04:25:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099121; cv=none; d=google.com; s=arc-20160816; b=xvpPwQWJanxHjIxGy4rvbKm0tPudqMZCZs/Lc9lV0dyhIfkTrMa0xiGsZyGH5yDgH1 ft45Ta5O3XS+gUnPxOfHU9GjS2JwnYkXz2fug+mdz0AgQiRBlZe/04CNNV3zbIeFoAlZ xQ7AUfzJCJdUFKiaU5/5W7GFB5NqKTbkXYaCTC6l4GWM0XvDp7PR8OmfifAxH+/XHNAy IrRlsHP8p1PtZDu46K2mFP69MwoBObAQIkdd6+aR6/dZYaGeWkgqlcZgdFg+szBx5I91 BYVQdNP3G+PYSqxaprlbsphtrvqXQiayxonmElpWOc4AXamgOpYtupIoIdlTrfeUTTj8 gs/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=8QlaS0qsWG1yDbCjfy8LBUBTA+Rw+V2QPlm0PJyeSSg=; b=GRbwnU2i/oiltBkfvzFTMF39S9C5Cp73imZuMl37M5lDZbhTKRTRhXmUhAUIguf3nW x84IEIMo4Vp4o9sefjtkbri5WQ+1JPEfr0FsLxs2z+MykRLrIn5ug7srZKalmoiTgovB FiSimBtMLlSq3AYboHH+DRCZ0i/n7tvudPr0e93hCqBL9DEUg67trfdaTqvq9HPI47qy V7HO3dFNjlgYmtGcx6OAKOKUfX4DuOvjaGexW6PKM5pgxOvMTHT0HEZNnmDGARb7G+1K FdN6tCK74ksMIZ3SmHEkNo2s/9AaJ5it9PNPVKsxy7mUS8k2O0Kjy1L322tcDxLpQNmb h9cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=LsjRIxQ2; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id w8sor180158wrl.37.2018.12.06.04.25.21 for (Google Transport Security); Thu, 06 Dec 2018 04:25:21 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=LsjRIxQ2; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8QlaS0qsWG1yDbCjfy8LBUBTA+Rw+V2QPlm0PJyeSSg=; b=LsjRIxQ2Hg3H9BfqO/BSaILu2cdnc7qCbrmOzsOCUGyNFhUc8PnZkY4Egonci94tcT oGHXVZT1R9v4uuwxW3lbEzK0xJNmrXmA0CrGXKmwmvmQgvnJPA8oS6+Mh8Wr515huKnW B1p3JEa9Idn8rpYKkd1KbBxd/uE3XKajY4oEmghl0YtQaZOluoD1ESC50TmFIiB/e8k4 vn+OvVg8Rzgxq2WOQiiif+qeJsYAk+UGskBhy3Bgcq83aIgG5r+YMz+jCc/MQouKb5wU dVg662Xv02eMWz6ce2DWFZEM9V4x0pl3o++UNARFS4dLPxk5oJAsyBCqxWcOZnuTW9y4 xEUg== X-Google-Smtp-Source: AFSGD/VRP3hJMLZE9HUeyiuuoJtI6GiUO+gXdz3duhVebumhgQr+5FnxHw+wLbZ5iNZKH1EKg0nbSA== X-Received: by 2002:a5d:6187:: with SMTP id j7mr25422579wru.300.1544099121225; Thu, 06 Dec 2018 04:25:21 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:20 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 20/25] kasan, arm64: add brk handler for inline instrumentation Date: Thu, 6 Dec 2018 13:24:38 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit adds a tag-based KASAN specific brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/brk-imm.h | 2 ++ arch/arm64/kernel/traps.c | 60 ++++++++++++++++++++++++++++++++ include/linux/kasan.h | 3 ++ 3 files changed, 65 insertions(+) diff --git a/arch/arm64/include/asm/brk-imm.h b/arch/arm64/include/asm/brk-imm.h index ed693c5bcec0..2945fe6cd863 100644 --- a/arch/arm64/include/asm/brk-imm.h +++ b/arch/arm64/include/asm/brk-imm.h @@ -16,10 +16,12 @@ * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps + * 0x9xx: tag-based KASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 #define BUG_BRK_IMM 0x800 +#define KASAN_BRK_IMM 0x900 #endif diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 5f4d9acb32f5..cdc71cf70aad 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -969,6 +970,58 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_ESR_RECOVER 0x20 +#define KASAN_ESR_WRITE 0x10 +#define KASAN_ESR_SIZE_MASK 0x0f +#define KASAN_ESR_SIZE(esr) (1 << ((esr) & KASAN_ESR_SIZE_MASK)) + +static int kasan_handler(struct pt_regs *regs, unsigned int esr) +{ + bool recover = esr & KASAN_ESR_RECOVER; + bool write = esr & KASAN_ESR_WRITE; + size_t size = KASAN_ESR_SIZE(esr); + u64 addr = regs->regs[0]; + u64 pc = regs->pc; + + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + kasan_report(addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; this is controlled by + * current->kasan_depth). All these accesses are detected by the tool, + * even though the reports for them are not printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KASAN", regs, 0); + + /* If thread survives, skip over the brk instruction and continue: */ + arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + return DBG_HOOK_HANDLED; +} + +#define KASAN_ESR_VAL (0xf2000000 | KASAN_BRK_IMM) +#define KASAN_ESR_MASK 0xffffff00 + +static struct break_hook kasan_break_hook = { + .esr_val = KASAN_ESR_VAL, + .esr_mask = KASAN_ESR_MASK, + .fn = kasan_handler, +}; +#endif + /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -976,6 +1029,10 @@ static struct break_hook bug_break_hook = { int __init early_brk64(unsigned long addr, unsigned int esr, struct pt_regs *regs) { +#ifdef CONFIG_KASAN_SW_TAGS + if ((esr & KASAN_ESR_MASK) == KASAN_ESR_VAL) + return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; +#endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; } @@ -983,4 +1040,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr, void __init trap_init(void) { register_break_hook(&bug_break_hook); +#ifdef CONFIG_KASAN_SW_TAGS + register_break_hook(&kasan_break_hook); +#endif } diff --git a/include/linux/kasan.h b/include/linux/kasan.h index a477ce2abdc9..8da7b7a4397a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -173,6 +173,9 @@ void kasan_init_tags(void); void *kasan_reset_tag(const void *addr); +void kasan_report(unsigned long addr, size_t size, + bool is_write, unsigned long ip); + #else /* CONFIG_KASAN_SW_TAGS */ static inline void kasan_init_tags(void) { } From patchwork Thu Dec 6 12:24:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715805 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF1FF1731 for ; Thu, 6 Dec 2018 12:25:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD6E7297D0 for ; Thu, 6 Dec 2018 12:25:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D0D9B2D393; Thu, 6 Dec 2018 12:25:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 166AE297D0 for ; Thu, 6 Dec 2018 12:25:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE85C6B7A19; Thu, 6 Dec 2018 07:25:25 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A6E4E6B7A1B; Thu, 6 Dec 2018 07:25:25 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C2D96B7A1C; Thu, 6 Dec 2018 07:25:25 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 1E36B6B7A19 for ; Thu, 6 Dec 2018 07:25:25 -0500 (EST) Received: by mail-wr1-f71.google.com with SMTP id j6so91591wrw.1 for ; Thu, 06 Dec 2018 04:25:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=heqQTzvwfrIHsAI5JpRjsv0LC8ZlRkJYzlJbdu65MX0=; b=nEdx6WYjzfRdb+jxdS7dajqrRepvVPalfvvs6W+E1FYe5YM1olmAMP9sSc+/vJsaKD KthjAOOkjQ7ic66ViohIjVe/KsfXdtE5e5hyJA23AqjiX3JMli+SXFazKR5qaYkEpAIT 63rSMHfmNrjl2pXyj+jTiyjfvmA4eWvgKNkwDrgo9nZThMHWD7CDKv7jbQrm0MlHvdfh B5I7jjP2vPt1ULaRdIkGCeNwkz67UatsptWqhoaAxzmy5E9NmLuj48U1gCxqNqzfS3Sm gn+It94HGkWff6k51yt2cGahzEMsTXIh8h8Yc4c+Ce70OJr6I4rP5434ajWlfyLzc3Ic Z4JQ== X-Gm-Message-State: AA+aEWYoiJFLlZ0Q3TcPSPXYPKE1Xv9Y/FSB779B4su5ggYJ3wACo0Ly zPp9BhbupmO4jDeA/KkEK32Q5HabVfSrxZyaXinxHqgbEVrJ4C1uQYX5cH1LmiSJj/F1YVnYDbg nRxwiOtIDQZ3KjsBdRqeTRydiwC5Bn4u+fKVVv2kUeM4DEzh0ifZbuzzrOB5UgvnyROo5j4GITV OnJeQTL18795xPragf9EHIG7k64LsRlgTSNo78FFio9pN27TI+QhPs20suei2g7D7gA4DGactPk fYk1A0I/cs0CFGL8EJqKFdTpdkMMQ5/HzR7JhRKpo+GxFKrWd3R5Cl4JQny2z0sB/k8giQnDD2M mL2cixDnGoxiru9LYL3YFMlGfEMDVWUR3BUeVdS5QAMzbFgwkbngI/Ux1z8QXmKSrAnmOjS03Fi D X-Received: by 2002:a1c:bbd6:: with SMTP id l205mr18213211wmf.97.1544099124585; Thu, 06 Dec 2018 04:25:24 -0800 (PST) X-Received: by 2002:a1c:bbd6:: with SMTP id l205mr18213136wmf.97.1544099123322; Thu, 06 Dec 2018 04:25:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099123; cv=none; d=google.com; s=arc-20160816; b=I4HOx38IwDRd848FBkhXjaDrZuGMafEYjG+1R8B+7v1Kl3K2RDSe2Nh9uAxznmd0zE XGR41AjLcWqGTytMj/OOzV36AL7rfybm1WHEUS2c4RBptmcFTbSHKcu5yTMGVpmzYlkN 0uE7tlivUNH8qf68tb4cbU3Sb4AkcWLoLfdH2qkX2adZEwngFrxJOTQLWlhGDPVq/5sb Ml20maEGXsPlKPTAJdHV4+oC8OyQ2U3LAiEagNHKopRsCSghAPFbPRR7OUntNjxyRdZK AkHQaXl9LJQCWIFsBJontA507Eai9wXHMJJNb5NEXuaYjms0e5d0oDn2BZoPWlAk8pbz lrcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=heqQTzvwfrIHsAI5JpRjsv0LC8ZlRkJYzlJbdu65MX0=; b=GGYJUcq+09gIHGirZckQNJCvDrNUTxhLPk0pCS5K+2L36xeGcLCij1cZ/UV6nrc48V Z5I6zkYsg0kdWQqHTKYABjV9tEU2hFI7wI8/vF59MJanpzE6s8tcYTnIeB12p5x9wkb0 t1Z98lygyO25l20oWYnqOCmRxVY5XdvqS7LZSnYoBhDFrNR6ATcqpgor/jdFnxHXa55T AiUQgOETcvw+3M15vZMN3Ggj0u3zKycIFjqXXxjLsaC19vw5dE5FuHRax3KxU9FvPBzg CuVlWkBs9jiTP9rNBm6j/tnhUjz68jRGvVa5XmnXZk+2rU9Y8JHafti7HO27bEXWpnkf o6Lg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=KaUpRP6Z; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j19sor175947wre.41.2018.12.06.04.25.23 for (Google Transport Security); Thu, 06 Dec 2018 04:25:23 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=KaUpRP6Z; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=heqQTzvwfrIHsAI5JpRjsv0LC8ZlRkJYzlJbdu65MX0=; b=KaUpRP6Z9pUaQk+HY1A45U7XZ5prx9o+GpppRKrj4DWhG15C5aL+qZjWEdegDBJ+nF tJaoR0G3pnI84MfNyD7lJq1XVbC+oR56Fc4SqWLnsccWtVb37hQbl8qp5tpJQ0SoL7/j baFDD0vcFVYkJvI8A6neTJ2OnD66nDsh2sz4rqijBfh78S7kGzNauknHYEb9J4Cea4Gm k4XCjleH3OmsHzfrv3HIBlcthOshISPpTe1xaFklziAc623QPnpAj6ugdkO1Ic2OTSlI ots1LbPDPm55KqvLE/KmZQHWEx3xwYApqh34ZgvxasZ5C/j8th9/70jfZ1nTtIKA3nfl dFPQ== X-Google-Smtp-Source: AFSGD/XscvNi8cFnh9sy9YrNAoSCZNKura0NtFA9sFcNnDfyu+tx+uY4MzG8yNMvey4oS4OZxMZayw== X-Received: by 2002:adf:e78f:: with SMTP id n15mr25721190wrm.115.1544099122725; Thu, 06 Dec 2018 04:25:22 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:22 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 21/25] kasan, mm, arm64: tag non slab memory allocated via pagealloc Date: Thu, 6 Dec 2018 13:24:39 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently. For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag. This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 8 +++++++- include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 15 +++++++++++++-- mm/page_alloc.c | 1 + mm/slab.c | 2 +- 7 files changed, 72 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 5fe2353f111b..7db28404609b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -314,7 +314,13 @@ static inline void *phys_to_virt(phys_addr_t x) #define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page)) #define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) -#define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#define page_to_virt(page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = __tag_set(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) + #define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START)) #define _virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \ diff --git a/include/linux/mm.h b/include/linux/mm.h index 5411de93a363..b4d01969e700 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -804,6 +804,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH) #define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH) #define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) /* * Define the bit shifts to access each section. For non-existent @@ -814,6 +815,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0)) #define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0)) #define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0)) /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS @@ -836,6 +838,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) static inline enum zone_type page_zonenum(const struct page *page) @@ -1101,6 +1104,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) } #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_SW_TAGS +static inline u8 page_kasan_tag(const struct page *page) +{ + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; +} + +static inline void page_kasan_tag_reset(struct page *page) +{ + page_kasan_tag_set(page, 0xff); +} +#else +static inline u8 page_kasan_tag(const struct page *page) +{ + return 0xff; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) { } +static inline void page_kasan_tag_reset(struct page *page) { } +#endif + static inline struct zone *page_zone(const struct page *page) { return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)]; diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7ec86bf31ce4..1dda31825ec4 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -82,6 +82,16 @@ #define LAST_CPUPID_WIDTH 0 #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH+LAST_CPUPID_WIDTH+KASAN_TAG_WIDTH \ + > BITS_PER_LONG - NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif + /* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit. diff --git a/mm/cma.c b/mm/cma.c index 4cb76121a3ab..c7b39dd3b4f6 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + size_t i; struct page *page = NULL; int ret = -ENOMEM; @@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, trace_cma_alloc(pfn, page, count, align); + /* + * CMA can allocate multiple page blocks, which results in different + * blocks being marked with different tags. Reset the tags to ignore + * those page blocks. + */ + if (page) { + for (i = 0; i < count; i++) + page_kasan_tag_reset(page + i); + } + if (ret && !no_warn) { pr_err("%s: alloc failed, req-size: %zu pages, ret: %d\n", __func__, count, ret); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 27f0cae336c9..195ca385cf7a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -220,8 +220,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { + u8 tag; + unsigned long i; + if (unlikely(PageHighMem(page))) return; + + tag = random_tag(); + for (i = 0; i < (1 << order); i++) + page_kasan_tag_set(page + i, tag); kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } @@ -319,6 +326,10 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, void kasan_poison_slab(struct page *page) { + unsigned long i; + + for (i = 0; i < (1 << compound_order(page)); i++) + page_kasan_tag_reset(page + i); kasan_poison_shadow(page_address(page), PAGE_SIZE << compound_order(page), KASAN_KMALLOC_REDZONE); @@ -517,7 +528,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (reset_tag(ptr) != page_address(page)) { + if (ptr != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -530,7 +541,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) + if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2ec9cc407216..365dc0930f8c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1183,6 +1183,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, init_page_count(page); page_mapcount_reset(page); page_cpupid_reset_last(page); + page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL diff --git a/mm/slab.c b/mm/slab.c index d2f827316dfc..d747433ecdbb 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2357,7 +2357,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, void *freelist; void *addr = page_address(page); - page->s_mem = addr + colour_off; + page->s_mem = kasan_reset_tag(addr) + colour_off; page->active = 0; if (OBJFREELIST_SLAB(cachep)) From patchwork Thu Dec 6 12:24:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715811 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33AC717DB for ; Thu, 6 Dec 2018 12:26:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24301297D0 for ; Thu, 6 Dec 2018 12:26:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 17BAF2D564; Thu, 6 Dec 2018 12:26:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 901D1297D0 for ; Thu, 6 Dec 2018 12:26:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D7EB6B7A1B; Thu, 6 Dec 2018 07:25:26 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 78C586B7A1D; Thu, 6 Dec 2018 07:25:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 58F7C6B7A1E; Thu, 6 Dec 2018 07:25:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id E730C6B7A1B for ; Thu, 6 Dec 2018 07:25:25 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id t194so204108wmt.7 for ; Thu, 06 Dec 2018 04:25:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AZfZTLqWsknIvYMLnZdolPQtd9B+6yIRrKwPZ48RLc4=; b=DwiD8HrmIub5L4J6AXstZh8GFi8l4hsC/zKtre1wk1txKzLFFLriEuIAnAfwx2tooe ZZ0WHAmSGdQivGq+Is8rJBnYGTFBrLtza3HVGzh+js7WTgO7PEbTsa2ZaooLYdrMgguu MtHc4Xk0J+HGr9d9+QZQFcrGNyUXtvaLvl1WeSW1BuwZvRIrRS/V7zKbNQtuK/asknsk FnK76trYGlxJMKyMZE/aYAOKAlWQqtINMl2NZgh9w3I5tER5O9TfeIs+UBysujvihB6L b0GWFprQ9S/TX7lPEfKGbovo/9Xq3488Aa9SH4zNPanXzVKdD22hz+NzE+Cj64JI6PUr 5b0w== X-Gm-Message-State: AA+aEWbtlQkE0shdYtJWTpJVQkRPmjDN1EJfsdJxMysgR92DFvob201f 3djOpihpyH24qSukxk97KSxBn7o1ytCwflBUmNArLlZF7bD1tY12Fobx9ovd7Ca7J5OrawPD1P3 J5cozO40d1wM6zjy+Cb1rQnK5Gw4czH56Os4NDlVprvMdLqoLC+gobFx+ufsb0EptwVrGCoMvHt 2BUUJKfeRdyd0uJhKV4hoO7m3STppaU2LEMdYNFmh6mPC1QjEPW4yd/LuYpqYzEZiuDrnjRfQ1v hw/TFPG55h3D65iukP7GgFelf+tIDNIR5L0jLLxn4FbisUJ/v1Y9Hvxb4q9/bT/EoxoM31uYSc5 xCUP2M+hGTmYfdwDOy4OIFBZkHfeAFLQrNth2QS91xlfu60lwTALXjQLZVHp3OUA27BeAwgNvrw W X-Received: by 2002:a5d:5443:: with SMTP id w3mr23344493wrv.4.1544099125480; Thu, 06 Dec 2018 04:25:25 -0800 (PST) X-Received: by 2002:a5d:5443:: with SMTP id w3mr23344445wrv.4.1544099124673; Thu, 06 Dec 2018 04:25:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099124; cv=none; d=google.com; s=arc-20160816; b=w2oRrfZGU0vdeYnXpuJmDtUQmQ8pTkV22GY9j0F6/6jx+x5ounhZK6phUNix1e+YXH GeDsw29yNh/w+Wz5n5Q6ww95vqW0x/tS+/egksEYgLO/mPJUeHpMs/MnajRTvOgWhsJo TB9TY9a1WVIuK8lbAy1WemmeFcvSmao8+KdcN2btCacpLQ6PS8gANVwMdVMRz3o1AHFg 4/pkMLjJU6rz+3SV+PD+qPbw9nUhcpv8sT/iPEqwpX3TlfZs4Dbf7jNghn0jpk+xgus5 cqpy8FDG+ytqZDAM5G4i8oME5xeLCNz/D36fWAJkEjlAWKDd7P9+4dbYEP3Jp14fEaH8 Yhxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=AZfZTLqWsknIvYMLnZdolPQtd9B+6yIRrKwPZ48RLc4=; b=jkLOpZqWgV3UhrgwDKAzLtWCTFJlNzsTD0Z53I4vqRM+pJLtpYJLsubfzvn7A6yLrx o9hQUl8wlwn5+MD3VTwEBuxpsTSV4xDjqIsp5ocR3LqlyGB3q4DhnV8xXyuKcVdR+cLy gdWZ6sTElFX/73HfxuWm2EYY0PcDylCXCNWST3GoQoMMPJGiQY1FX6g7B4/ypVa/5Mxe vwfaTnBDlas2Gilsfp5G0q2wjxIawxgeXuaRFK5OQ2hSb8sDtsczXMc2x+rFdB2dp0hX XagaqGIjx5NPX0MMuJGgv8qa/gU4j7M1qTvkpvQVekLM2sqKBE0MPbjQJC7wJpCY6dn6 /j0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="EW/3SMqp"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id n12sor197711wrm.10.2018.12.06.04.25.24 for (Google Transport Security); Thu, 06 Dec 2018 04:25:24 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="EW/3SMqp"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AZfZTLqWsknIvYMLnZdolPQtd9B+6yIRrKwPZ48RLc4=; b=EW/3SMqpcTiMhWNm2orG3EvbNyLZ48vaZIbUCv4K7/oj7HRZ9uPY1RH0eYr1rAgVW0 jyMP2dxynkJIJWAOyFDrXOO+bxX/RQl8Uyo6CKjDuuK1f5Yt+om29IOR4qfuWKYTdBa6 HSbK7N4y6Wq1gjNxQBTGM2y0gddfv8BnrSJP27jRBzimexVneU2mKh133NTPYQ2fqCvG CPhOs/+CryNsf6lHjyX+rz5v7pe0z3I5TS14++RGHvLCAQL4JcZIvyC/7qVGdYVi9+AZ yENfqxMnbM8uAyVl4QWMFZDJgy1UENGmTQUl26yxkghXpslYWoLlLUoaBcjn9yfCweYI TAWw== X-Google-Smtp-Source: AFSGD/W3yTLYMQBOaLzTZ5SXjxjDat4MU8wRZ1yiVk7CrJyHq5HLrO5lLxvOMClpJkkFYNGGBhGn+A== X-Received: by 2002:adf:f8d2:: with SMTP id f18mr20884354wrq.265.1544099124211; Thu, 06 Dec 2018 04:25:24 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:23 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 22/25] kasan: add __must_check annotations to kasan hooks Date: Thu, 6 Dec 2018 13:24:40 +0100 Message-Id: <03b269c5e453945f724bfca3159d4e1333a8fb1c.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds __must_check annotations to kasan hooks that return a pointer to make sure that a tagged pointer always gets propagated. Suggested-by: Andrey Ryabinin Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 16 ++++++++++------ mm/kasan/common.c | 15 +++++++++------ 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 8da7b7a4397a..b40ea104dd36 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -49,16 +49,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, - gfp_t flags); -void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +void * __must_check kasan_krealloc(const void *object, size_t new_size, + gfp_t flags); -void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 195ca385cf7a..1144e741feb6 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -373,7 +373,8 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, bool new) #endif } -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) { struct kasan_alloc_meta *alloc_info; @@ -389,7 +390,8 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) return (void *)object; } -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, + gfp_t flags) { return kasan_kmalloc(cache, object, cache->object_size, flags); } @@ -449,8 +451,8 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) +void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { unsigned long redzone_start; unsigned long redzone_end; @@ -482,7 +484,8 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, } EXPORT_SYMBOL(kasan_kmalloc); -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, + gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -506,7 +509,7 @@ void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) return (void *)ptr; } -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; From patchwork Thu Dec 6 12:24:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B74021731 for ; Thu, 6 Dec 2018 12:26:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A59042D564 for ; Thu, 6 Dec 2018 12:26:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 99330292C8; Thu, 6 Dec 2018 12:26:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC8E5292C8 for ; Thu, 6 Dec 2018 12:26:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28D1C6B7A1D; Thu, 6 Dec 2018 07:25:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 126046B7A1F; Thu, 6 Dec 2018 07:25:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1A8B6B7A20; Thu, 6 Dec 2018 07:25:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 8406C6B7A1D for ; Thu, 6 Dec 2018 07:25:27 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id j10so80606wrt.11 for ; Thu, 06 Dec 2018 04:25:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=c3irOTZqglCe0+q+NuOFKdHcgcEut4vEB08n2fct2jM=; b=cwYJO3Lhuo8vVSbhw/bMuWJdK7ychCiuQOio4FRsRM29ejI21ZpaMXeg/kdYvtS4vm ug/cMK44Subv2xGHTrizyVzWEuKWeRulVqvpNGFGnSvUiarphf3H9y7cA1ab5NIMFcZv nR5UTr8b65qVYuueZRl+GE08a+O9lYfzldNf70Ojpk1tmabcN/KZZkkVyIk/bnsK8g2B A5HwuFR82vNkWOh2RiAnemcxstCdP/KKbRXYk5OWIXvargOGhFbMwiyVMi4VtK0FF8v5 ir4kW0ySjb3pwaXdlxKCVjfCCV284jo4TJfF3CqNwBCi/FuKIhMVQco26tXIiEhCsC1b zmtQ== X-Gm-Message-State: AA+aEWZ3ubMPZYq0p0rNAOnCOHiKqF/rU4z9bJKANsnL+ptsCh/OYUCo Nltp1Z3zJCmRGiAo9NZgU05BGqhCrLPVga7xY9joAaMviouPkym1H3cXA9N0ETIsmKp8VYuQqNI dTKECVClqOwzeaodPpcAd919uxd5brri/fZt0E1sk/Cjb21MXPFZWFNp8M9Y1Bn3+hyBEJFI9KB CMhC99NYS53USVmxycw/NHCMpLhLJtZAMyCKcN+D42DH8paGPugjV1l+DOxg90fmGB5WcPWvEPs S7D5o20RNhrPVw5M5KUJ0/8PZ71Bob05SDxpXYWKg4EwqFtwjWP1zQ/Tph6x6FT/6oSd5Jgb/tn CrPTUeb4lBRI6zpWNYK2F84fJFGHQOIaIxQe0il6q6Iq39wPSgsUg9VVWdgckqt1sck5w5Rsky6 x X-Received: by 2002:adf:f101:: with SMTP id r1mr25630578wro.32.1544099127130; Thu, 06 Dec 2018 04:25:27 -0800 (PST) X-Received: by 2002:adf:f101:: with SMTP id r1mr25630530wro.32.1544099126293; Thu, 06 Dec 2018 04:25:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099126; cv=none; d=google.com; s=arc-20160816; b=JrpQ6EYQ7iyb4mhfxlWN8UtCQKQ0PGjNcSiUrOJ2eYf2F0tjZfLE4Owgbacw4WC2Xu wWTWUmvvOAs0nnAEXUp01qRxvx+Rw6a8LFZDQBB3OOPn9gNaPI8F1p4QOu+0abc0mcMJ oczX4O7hoMiUmAwiEtIF3wNU2RxDdKC6tQIdfFlI0RvuTIRGl00jPPSwrThtle+RK6kV qHmy+GnPfcw4L5HuA0zJuWjHI/uo+dAb2wtnFtwzWrx3nZYjB2CFwqvV/idQmCo9YO54 Pyi9to18pC/72GuVkC3tFqVMZeE7TdEUmg4LUubYTzf8IzZ/L3g1FvABHdonYB+GpSA1 H3Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=c3irOTZqglCe0+q+NuOFKdHcgcEut4vEB08n2fct2jM=; b=U64hdG9EZYjqGHvYs3dJahm7LcfI7TRDBxyIYJcyPB+iqQPHYEDMItNH4eTO1CS2mb k+SQYf4eDq1YjiqNUAuNYFdVoJIDOqEfJZigPdL1/0rCzfuaGBAevq+ncWSVx/K6AaI/ wCT0QVCLh2M9zhGOU/hxcOqdeTWfq2W4i2zYNvlmDK0h6UGF7e6TjaJL7gOcctpJLAu3 bhYD8RhnrqnN8DB4ldbddQv2DDySrk0hpm4d9CuaK6VQ3HqX8MIZjEMjmYaRC47+ti6P uunzwglMfEgNl2jBDBo6P/vSuX2SxXCAPjPlQmA8PJiDahjMOxSFQ6BmI1D+z5BjCFvc jRpw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=neyaAKT0; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a1sor177350wrx.40.2018.12.06.04.25.26 for (Google Transport Security); Thu, 06 Dec 2018 04:25:26 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=neyaAKT0; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c3irOTZqglCe0+q+NuOFKdHcgcEut4vEB08n2fct2jM=; b=neyaAKT0SnRF3m9Va46jiGHE6yqDTX+/qkDaGCzTfyovNE7fgEiBpHCH6E8H6pp+Dg 0V0P6R3gZXIzt6Npzf0dbDRYL8Segztlu9nPAwlHmkpVL6i1WQnVRLnveSf0VX7FUXUK 0cTIsQkBwIH3ZkWUotsAs+TKxDFa5Y9Z2Quc3dcP2YuxWCU42kiKEIGfhwIkNFXUNKSD hCfLOG5PdM3jVSyheXBqSSO9uucECOKJeKjxpGpB7xEix22PdxX5UErNMDlIZlw4oPq9 St5iuKFy2FfMyKSj8DLg2jVhjVWKfuYjzEnkhBGUnYmnyILzY+gq98tl22Emi+J1SvnQ mfsg== X-Google-Smtp-Source: AFSGD/XkE8mejCoGUEy628et/JZL/sxDM+RBbFhKJ78kRjJ2xXA7tnfyWjFXQnlU4GWTOKOYRyZgpA== X-Received: by 2002:adf:f091:: with SMTP id n17mr9679121wro.292.1544099125848; Thu, 06 Dec 2018 04:25:25 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:25 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 23/25] kasan, arm64: select HAVE_ARCH_KASAN_SW_TAGS Date: Thu, 6 Dec 2018 13:24:41 +0100 Message-Id: <25abce9a21d0c1df2d9d72488aced418c3465d7b.1544099024.git.andreyknvl@google.com> X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now, that all the necessary infrastructure code has been introduced, select HAVE_ARCH_KASAN_SW_TAGS for arm64 to enable software tag-based KASAN mode. Signed-off-by: Andrey Konovalov --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index ea2ab0330e3a..66f2c625fd1c 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -111,6 +111,7 @@ config ARM64 select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_SW_TAGS if HAVE_ARCH_KASAN select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT From patchwork Thu Dec 6 12:24:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715821 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1097B109C for ; Thu, 6 Dec 2018 12:26:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0096F2D564 for ; Thu, 6 Dec 2018 12:26:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E89602D942; Thu, 6 Dec 2018 12:26:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 061BF2D941 for ; Thu, 6 Dec 2018 12:26:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 282386B7A21; Thu, 6 Dec 2018 07:25:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1BD536B7A23; Thu, 6 Dec 2018 07:25:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EDE9B6B7A22; Thu, 6 Dec 2018 07:25:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 76EBC6B7A1F for ; Thu, 6 Dec 2018 07:25:30 -0500 (EST) Received: by mail-wm1-f70.google.com with SMTP id b186so202325wmc.8 for ; Thu, 06 Dec 2018 04:25:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lo1jvYJYchvvArlOAxyqi4oqm/VYJkrmrZ6srDrDcxs=; b=NVOSnzqZfgXy/tlpGEb3LXtZ26NbdXIiDTeJFCg4S4Zb3X+hy8aHQX/X0kNNvUtOiB 4zGW5NR9nUcc/pgZqfdZb+NaINUKBioJuRSZ08FoKUCRRLiQdHhLiq+C0A3zMwBF+0Eb WC9FHFmHHfN/qIOMs3kDFAr4LMCN5cEcrzvSP4TzVoT3PoPR1lmmQJssoDNE9t/Oe8sJ BDri2eYJj4zBRlNWmPbU4iiM8FQ2KDE3U3/CYp9RCIU0SY0uFYM6tpBbg6cYdyyRno8l o0KGNx2Ei340n0/ygwKvgVNzPo92oVUxGvLau/VP/XOWLO/vefrbo71f3ICRYq83oebO Nb0w== X-Gm-Message-State: AA+aEWZI0GxbQsuXrbd4Uh+3W7afLkQXi6cgKPuI2jutswUVOg9JXCmw AvTD1aXQuMzYE3799axLwYFMlUOJKFEroGkU1La7coTZ29zqarnSGn49prAn/5P57jdiuA3nGtR S51wiEGTEmp7NVP2dyoH9J7wY1SdEwA+bSKzhSpobo1UblFJcT7M9CYzWBM3tgy8HVtAgbr0U/j A1TwsctPyDPUJRkX8Hea6Jzer2/2hm9cVBCLoQdThMTVibdZpo/ilaoYPP74DtDpil8qqAZtKnH h8cQ7Ah5kOEuY+cWy2dBv1qmUqoK3hWrUs8yLMvAU5fISZM3VcR031z/BoiwpNBc/MM8La0QNc+ Znf9eOBfRmA9qcyL5saoYapi2un/rBqUoaNpgZFiFHN9ef6nijM+gYhtTyvBtYJrja246wOpkEZ X X-Received: by 2002:a1c:f916:: with SMTP id x22mr20261230wmh.87.1544099129945; Thu, 06 Dec 2018 04:25:29 -0800 (PST) X-Received: by 2002:a1c:f916:: with SMTP id x22mr20261125wmh.87.1544099128333; Thu, 06 Dec 2018 04:25:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099128; cv=none; d=google.com; s=arc-20160816; b=RwFmlZO/K18v0tok0K4Q8jhNAaPXuzqooJjVqh66v3orT1yHr4nQ69RldxmkSfj2EK Bf/ilI0n7CExqq8WJkZqQDAKIFIADNKnElGXMTnXvnsUe87KkGa5kUhcTBdSRAbsEpKi P3kclFoa7hRObRjoZQh36UFJ0/nx8p4CYDKIeITPGaEnAv0+zhJ7dPcbuB9clIRxiDR8 8fybuortjPtAC9cJUg8IrswL1qaCS9RsxbjIOTQettKWhyoLD5AQMR6/yK0oQyGImPrh rxuW/vscw7A7PK8exhk4ENE7nktk+6HB0s7Dv5OwL7ZCbLK4CN3n8GCwucyj5LgImBi3 INBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=lo1jvYJYchvvArlOAxyqi4oqm/VYJkrmrZ6srDrDcxs=; b=KaCpVxMfdOIafKnXOcTSwRw7nKilMg4J09k7ydAJdX1Z693nht9mHZZ7nMAzSMHmHT 2q0RKkrsmpvxEmANaFCST3hLGxiRj1xzMPR3MrLFGEHJGTygCRxrcSpTmXOKyYvM73xa AO8tfhSWokLKVPtJSRTR+Eu/iqGEahhT7WtN3UbqRqdtJyh9r4DBgD47mFVX/t0k2ZuI uxT1yKdm3flbBOHBFi02qFRYByVccxwqbwjbdGXezCSkKNEnfy4i7ck7PtHtrhnUc8MM uv9nrCzLqD9+1iycUJbkjYeFzqKwwLxsn7M1C2GmSz5OTMkH6xVjzVUDLctSeAXj3bHP 7duw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=GyOrI3Tx; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c18sor178367wre.34.2018.12.06.04.25.28 for (Google Transport Security); Thu, 06 Dec 2018 04:25:28 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=GyOrI3Tx; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lo1jvYJYchvvArlOAxyqi4oqm/VYJkrmrZ6srDrDcxs=; b=GyOrI3TxX+AXWzhRcEKK1my4tMw7Tq/TLdah4AjC+XyaEOW4ph212uSHTkkmA+BexV y0FcqC5b/kp1PnpyCD2x7WS0dfGo8ig8R0iiblUMmr7CzvsRwSlC6bLNIuGP0s3TFPq+ StsVlNMjQIv05OP7KFySP+oGqr2V6TKi8BnA9sg0pDlXVnUHUuBLa2VsLUkh4uZYyhLH S5dfgC8F0INb2q/YOPJCfiHPQU7v9oS0XTe2/fV+RXLVFd1sMCr0sVEmLa2e3k28+/dO y1I4AUn4NzAHtXAVRrmjZPSHiHzDa4qmotDr83IRWMogu40nnyGa263ffyoZQ0ue2/oF b7cg== X-Google-Smtp-Source: AFSGD/V5oCsauZAPVw54eE9fTASHSaqfs5rpj0WXYV3FMkDCczlOf5TfHzn20IpmU+lPgPk0t5J/7A== X-Received: by 2002:a5d:4d11:: with SMTP id z17mr24028790wrt.209.1544099127520; Thu, 06 Dec 2018 04:25:27 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:26 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 24/25] kasan: update documentation Date: Thu, 6 Dec 2018 13:24:42 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch updates KASAN documentation to reflect the addition of the new tag-based mode. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 232 ++++++++++++++++++------------ 1 file changed, 138 insertions(+), 94 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index aabc8738b3d8..8e956e0588fc 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -4,15 +4,25 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides -a fast and comprehensive solution for finding use-after-free and out-of-bounds -bugs. +KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to +find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN +(similar to userspace ASan) and software tag-based KASAN (similar to userspace +HWASan). -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. -Currently KASAN is supported only for the x86_64 and arm64 architectures. +Generic KASAN is supported in both GCC and Clang. With GCC it requires version +4.9.2 or later for basic support and version 5.0 or later for detection of +out-of-bounds accesses for stack and global variables and for inline +instrumentation mode (see the Usage section). With Clang it requires version +7.0.0 or later and it doesn't support detection of out-of-bounds accesses for +global variables yet. + +Tag-based KASAN is only supported in Clang and requires version 7.0.0 or later. + +Currently generic KASAN is supported for the x86_64, arm64, xtensa and s390 +architectures, and tag-based KASAN is supported only for arm64. Usage ----- @@ -21,12 +31,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC -version 5.0 or later. +and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and +CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN). -KASAN works with both SLUB and SLAB memory allocators. +You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. +Outline and inline are compiler instrumentation types. The former produces +smaller binary while the latter is 1.1 - 2 times faster. + +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line @@ -43,85 +55,85 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access generic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8801f44ec37b by task insmod/2760 + + CPU: 1 PID: 2760 Comm: insmod Not tainted 4.19.0-rc3+ #698 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [] dump_stack+0x46/0x58 - [] print_trailer+0xf8/0x160 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] object_err+0x35/0x40 - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kasan_report_error+0x38a/0x3f0 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kasan_unpoison_shadow+0x14/0x40 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] __asan_store1+0x75/0xb0 - [] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kmalloc_oob_right+0x65/0x75 [test_kasan] - [] init_module+0x9/0x47 [test_kasan] - [] do_one_initcall+0x99/0x200 - [] ? __vunmap+0xec/0x160 - [] load_module+0x2cb3/0x3b20 - [] ? m_show+0x240/0x240 - [] SyS_finit_module+0x76/0x80 - [] system_call_fastpath+0x12/0x17 + dump_stack+0x94/0xd8 + print_address_description+0x73/0x280 + kasan_report+0x144/0x187 + __asan_report_store1_noabort+0x17/0x20 + kmalloc_oob_right+0xa8/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + RIP: 0033:0x7f96443109da + RSP: 002b:00007ffcf0b51b08 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055dc3ee521a0 RCX: 00007f96443109da + RDX: 00007f96445cff88 RSI: 0000000000057a50 RDI: 00007f9644992000 + RBP: 000055dc3ee510b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007f964430cd0a R11: 0000000000000202 R12: 00007f96445cff88 + R13: 000055dc3ee51090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2760: + save_stack+0x43/0xd0 + kasan_kmalloc+0xa7/0xd0 + kmem_cache_alloc_trace+0xe1/0x1b0 + kmalloc_oob_right+0x56/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + + Freed by task 815: + save_stack+0x43/0xd0 + __kasan_slab_free+0x135/0x190 + kasan_slab_free+0xe/0x10 + kfree+0x93/0x1a0 + umh_complete+0x6a/0xa0 + call_usermodehelper_exec_async+0x4c3/0x640 + ret_from_fork+0x35/0x40 + + The buggy address belongs to the object at ffff8801f44ec300 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8801f44ec300, ffff8801f44ec380) + The buggy address belongs to the page: + page:ffffea0007d13b00 count:1 mapcount:0 mapping:ffff8801f7001640 index:0x0 + flags: 0x200000000000100(slab) + raw: 0200000000000100 ffffea0007d11dc0 0000001a0000001a ffff8801f7001640 + raw: 0000000000000000 0000000080150015 00000001ffffffff 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc + >ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 + ^ + ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works. @@ -138,18 +150,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For tag-based KASAN this last report section shows the memory tags around the +accessed address (see Implementation details section). + Implementation details ---------------------- +Generic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address:: @@ -162,12 +180,38 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +Software tag-based KASAN +~~~~~~~~~~~~~~~~~~~~~~~~ + +Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to +store a pointer tag in the top byte of kernel pointers. Like generic KASAN it +uses shadow memory to store memory tags associated with each 16-byte memory +cell (therefore it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation tag-based KASAN generates a random tag, tags the +allocated memory with this tag, and embeds this tag into the returned pointer. +Software tag-based KASAN uses compile-time instrumentation to insert checks +before each memory access. These checks make sure that tag of the memory that +is being accessed is equal to tag of the pointer that is used to access this +memory. In case of a tag mismatch tag-based KASAN prints a bug report. + +Software tag-based KASAN also has two instrumentation modes (outline, that +emits callbacks to check memory accesses; and inline, that performs the shadow +memory checks inline). With outline instrumentation mode, a bug report is +simply printed from the function that performs the access check. With inline +instrumentation a brk instruction is emitted by the compiler, and a dedicated +brk handler is used to print bug reports. + +A potential expansion of this mode is a hardware tag-based mode, which would +use hardware memory tagging support instead of compiler instrumentation and +manual shadow memory manipulation. From patchwork Thu Dec 6 12:24:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B106F17DB for ; Thu, 6 Dec 2018 12:26:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A12022D564 for ; Thu, 6 Dec 2018 12:26:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 947022D942; Thu, 6 Dec 2018 12:26:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 262892D564 for ; Thu, 6 Dec 2018 12:26:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 850CC6B7A1F; Thu, 6 Dec 2018 07:25:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 800546B7A23; Thu, 6 Dec 2018 07:25:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69E926B7A24; Thu, 6 Dec 2018 07:25:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 13A9A6B7A1F for ; Thu, 6 Dec 2018 07:25:31 -0500 (EST) Received: by mail-wr1-f70.google.com with SMTP id w12so73813wru.20 for ; Thu, 06 Dec 2018 04:25:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KjVNxOw/tL33or+YZX4CcI05bhFwdq3tUwMnDPh4OsI=; b=swt7xpGLA5FezYB1hOVRSbmr547fV73NLltgd0w4M/NnzjIU2sbqiDmzsWYLdLl3gW y/NxUFR4GcQifOMEQyCu7nNHSiQjY+ojn+riOKHB+JTmb5nu68AvE/zwxiPfCDdQAN1F esdVG16sX+KhJ+kC8gC53xskIW718XWWjcJm8F15OQCVNE3eri9K2Zk6m3720zBrK7VM 0Mt8Pd7qxM5hNpR1xP2dcVVnuUWqVguUYA64L0nzA/XtZXTkDZW8j2w//EKmNEmKq/P7 fFptIzoEqunfRTmqzeXaAAOQyjiZsm9OLrzeHQCYxz0M0nYl5qDDLx5YXc1/xLYe5NyD uSdw== X-Gm-Message-State: AA+aEWb+Z0a3fI30v0r5S1DETszCfDl35cmdcrI/RrCJ2eXVJYCbuy+D 96us50EH3QkaTN0n02rSX9J1DD7dDN5bIRmTY6y/oXygvIzuj4ojVfeEYM11xCdnBo6Tq86Tqh9 Pr/SP1qkctO30JnkX+tlL31VggrXxmzJoA7JmIm2iu8sVAhD2J2fTpLJDVHLYi9q1mk4C3Z/dGp WvyWiNYU2EvROiNm9TBnDamf1sgCIYEFMuhYQDUHssucm1soR8xuCcYjkCxeIMxFW/AwjkOMWK8 x2ZzqOF/2blgokkkTPVT7PqRxGC5lcbMSWOsEyD6C05lPsxVWDZW98YW7GUFJKxWrT1uiNj5WPa SOrYDjRRuBxoxiRovbOHoc9PnLJfBWX5i/KLYNQex4MJJd6ZRtTm+18ribZP2ffDwCR2jhdUDIi H X-Received: by 2002:adf:cc91:: with SMTP id p17mr25922851wrj.118.1544099130586; Thu, 06 Dec 2018 04:25:30 -0800 (PST) X-Received: by 2002:adf:cc91:: with SMTP id p17mr25922788wrj.118.1544099129570; Thu, 06 Dec 2018 04:25:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099129; cv=none; d=google.com; s=arc-20160816; b=K6dsaFqi/MAYv+CC1PLMTPUGItaV7qODm3l4DMD0UBKvEH/o+ICO1/cJtnZjx1f3RF VXXCv5AXaT0z9c5B4xSazkjK+h/4M6y/TWrspo8SDEysVMzpxmHBbd4RpwTFj7VNS+Ua 8shxj9MFb9ZTTL8dEy0BwMScBJgT0544SQqDK+favOrWjo86e2nnC4QQ2MsxfxdUth2P lCnxTcJF9J22uXJ+XN/gRcyQCW/a6tL/SEzghiNI2VkkNpgNUlBNAVEgkX8OOYqJHJvG jD3j5LvbrvFyOqLdss1nB+48FiVvUWL0m40Upwm6IJeAMhIGHbqRlrme3VsTBscOFbco +0Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=KjVNxOw/tL33or+YZX4CcI05bhFwdq3tUwMnDPh4OsI=; b=E9whLIsYtL8vRHnBkYLJ7ckJ7ZzzIe2zLqdQa7RbbabHMEZZF7CzHtqYv8tB0jd+y1 dOS2laAJWXtMyZqodXFKkkgj7HWd6xPRaVWi+5qOlUVyvJZiv2C8Y4rQpcLX/e4BSa8K y2+Vl4prBDN2qBvERijtiz+IotLod+ez5unZPjCfHEZJzz02Q4N8UT8gB7n7I9Oj6EwO FIvR+NRSOoO1DWcSoedi+4xk6RI+BPvo98tyrM/+SJCvsP419/LYlmJdJMOq0Xqzmv76 UvUXfNJ/3wNCz2huoWGP7ksIx8XbNVumUqAVYZq01Hdn12EZ0mkWb7reNKyX88K5SUZz o6Qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=oo3+H9kl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id g18sor202882wrw.3.2018.12.06.04.25.29 for (Google Transport Security); Thu, 06 Dec 2018 04:25:29 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=oo3+H9kl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KjVNxOw/tL33or+YZX4CcI05bhFwdq3tUwMnDPh4OsI=; b=oo3+H9klV662Z7l/sV9qWe+97OgA+3/vqJSjP7ORklWt6jkvGUcbWvgXiU/UVWdXkO aRLkTeLwT+26S54LZ1tdSLxem3Ftzv1i5So9WjUy3XATaE+uANX/1fhMF+AgEwKXtAwg rAkGmLrzYXP+yKAUEjpYbfuBC8hP5315ZeLkPCdoqz8/K4iUjij89tOCi8V9z1ejLgXv 9vlIuXu8j0RV2wo3dbpaQBpF6J/r83DeeQdLrHkg1Q9J7X1H5aZr7mfCp5owOkx7HZqS vD+vfMjs6CnFWnveWnmJkNm2itflfgBDPdvXFej2Z386iBwGTvlr0kwYTm1qDAB9kXGW pTvg== X-Google-Smtp-Source: AFSGD/WkeEV102m8rxQR2fqB5liDyQXRSpSsg3Lo5uFcLBPqUhzjh0TSww8+pClypG1j70/WQ25UYQ== X-Received: by 2002:adf:e983:: with SMTP id h3mr24228215wrm.232.1544099129139; Thu, 06 Dec 2018 04:25:29 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.25.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:25:28 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 25/25] kasan: add SPDX-License-Identifier mark to source files Date: Thu, 6 Dec 2018 13:24:43 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 1 + mm/kasan/generic.c | 1 + mm/kasan/generic_report.c | 1 + mm/kasan/init.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + mm/kasan/tags.c | 1 + mm/kasan/tags_report.c | 1 + 8 files changed, 8 insertions(+) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 1144e741feb6..03d5d1374ca7 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN code. * diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index b8de6d33c55c..ccb6207276e3 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core generic KASAN code. * diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index a4604cceae59..5e12035888f2 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains generic KASAN specific error reporting code. * diff --git a/mm/kasan/init.c b/mm/kasan/init.c index 2b21d3717d62..34afad56497b 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. * diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index b209dbaefde8..57334ef2d7ef 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. * diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 214d85035f99..ca9418fe9232 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN error reporting code. * diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 1d1b79350e28..0777649e07c4 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core tag-based KASAN code. * diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 573c51d20d09..8eaf5f722271 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains tag-based KASAN specific error reporting code. *