From patchwork Wed Sep 19 18:54:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9F794161F for ; Wed, 19 Sep 2018 18:55:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9323B29178 for ; Wed, 19 Sep 2018 18:55:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86B342AF40; Wed, 19 Sep 2018 18:55:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6559291B2 for ; Wed, 19 Sep 2018 18:55:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E9248E0007; Wed, 19 Sep 2018 14:55:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1716E8E0001; Wed, 19 Sep 2018 14:55:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EDBFF8E0007; Wed, 19 Sep 2018 14:55:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 82E4B8E0001 for ; Wed, 19 Sep 2018 14:55:07 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id t16so179032wrx.2 for ; Wed, 19 Sep 2018 11:55:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WCePxEFakGULsoDfnvXCCo23//uX664Y33hC1JivR2c=; b=Z8IBq77vy/ZZcOKjws3jNql6hCKLxkw7Nlx4ibMapv8hG/bH5tUYC8vGxBcMzKbto3 DsVYPGJFcz4FZk0D3NK5JrNWeIM3CoxH0gIusIJ0Vz5LsKJTeHKdJ0Aj/cNpZAlAXSGH hu0fhOniD2xbHkrmL8WF6EAzNHlXV00qvQX2Hjv0VDeztbT+kENN25dBW4bEZHgD4UZy c5B6xksDxGOSZWtfPTRq/z2i6dlqJno8O2PG2ZxciqEmRNyL+o8ae7zvF28fbR0FZW1v FzFT97YykT4w3uI3wHj1Q3v7nQm5o755CR/cZ1Efg/7LUrFURNmZ5M4oiADKZHySrwGO lLyg== X-Gm-Message-State: APzg51A75L03UOm1DMjCD+uMzg36XJBMIRjpSrFD5rmpYUrWuk7b0KGs wJDJAsXIha1kP9blBCQYi/0JGQ691qy/2so2zpQGvWw6SahIfZ9hFJAS1WbpDRrS09kMRIKTMd6 Am0Nmo0B25wECgLD6c8PU2tUQWsIJoIJCaZZr44Iv3NP9k33NtgmOWsonLsjdXzTB8Q8EJ2u/yi IEE+ouPrWtaE6uwti8H2HcuucYPLC7XrZQqhmqzllY5NhgAyxkvgwhpGnhGXjF82dUmi/HNa4dU dSah2wV/FLxj22vdfyT/4gaW4H7x/UBjFEtDcpsOSVzKbCLMJ6RQS49GGPRk6hkZHdTxpuB5+VF aGIpHicLh4P6Qfey1JCCkVaHBGcsnQ++QCUM8kIFRSb2n3Mu3mP3MIlxFJ863QtZ/uj2Tk6bh0i d X-Received: by 2002:a1c:ee57:: with SMTP id m84-v6mr21093595wmh.9.1537383306982; Wed, 19 Sep 2018 11:55:06 -0700 (PDT) X-Received: by 2002:a1c:ee57:: with SMTP id m84-v6mr21093550wmh.9.1537383305807; Wed, 19 Sep 2018 11:55:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383305; cv=none; d=google.com; s=arc-20160816; b=EtuBBHcwz5vzINLiqEQP2wm0uIsg4MADHgjXZbgISWm1mPe2XfSdzHCGq8yJqVCaqR Qn2PTtmYQ9VFMChx14c8vV5rSWTms3QJD+62T7T7qDMHd8GbxTdkcyfwbA1Cm8WcL414 WVc8dKfZtTvqWlfqeS16XF1Fy+cNskMH2OhAnvuvT2D+vgi7OyEHsqcU07v+JrZ6mA4E F9v0QlhTGx5hEeC3FXtIZcB8T6tCkEqoxy9tXSkntYjXzZl5Yh56wz5lS4BVPylGW0jP a8jXa9LpYlHopi0tgOqqwzFY9kopATzNLqxO86Uwi4xmzUSkbEXjHnXS1tnyaHMZdglE N50A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=WCePxEFakGULsoDfnvXCCo23//uX664Y33hC1JivR2c=; b=YXLUG7iXIsnKE4lPnedQdHu1gjYm7OpQpSof6Z6OlAobKamzP6cuEuf/g3xxLugPZp 8cdmaGPyJ5cavLghDdYhvssmzcQAOm6zZ9UX5SK/RMf+D00vpnz++55DgHPoPD0DjHZl NUj4sZ7rMh789AA79dbj0/eErXU1iCf9neZItoi5+dEan4Dxfs2dxy/LTkGmbBtTLTly pBsPQf7u0/k+0MwMyAE5y6SpBhyZTcjtJuYVRgIoEGMpHh1SHv/2Fd7gKkN5VqUnhkR7 lZc+hZON2lu308cwyHkedztRsqYbxgTQ4KTjoitSPHWWkT8A0+b7fTAD2JHxj2KwZLsW pk3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=GxMotn6R; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u3-v6sor16409027wrw.12.2018.09.19.11.55.05 for (Google Transport Security); Wed, 19 Sep 2018 11:55:05 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=GxMotn6R; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WCePxEFakGULsoDfnvXCCo23//uX664Y33hC1JivR2c=; b=GxMotn6RLRAA6ELuGQaF1bXmTK1TONBXBtQy4HlOP8sw6x2/i+eHyH4maJzD85PUve pvl5XOIUgS45MDRtgaaz4hA77IuXCOq5Abs/xJehl5FxmaGBHHk1UZb1eZocnf2oETNG 9swa8vjok+KKm3Cip2ic6teNvP8/gnw9wDrRlD60/wM6qyRJn/tg13HKp8NpE+EFddWU Lq+U8UKdItpN8GLFBel6/qgumrFloNCi+w5GrwRtE6OjQ8s5Ao2GikCu7ufiZIQiA9x7 giU7vZk/H4zIkfhzoR7LacXUuPGr178I4KlDKJnXZx88aPa6b33JRvdW17o5a8RgB6Ip 0sTQ== X-Google-Smtp-Source: ANB0VdbFql5VkAtTNt6odDURwB4QcHPK3E+I53GAoaQDAsgmTOApo8uVWZaG0QV6Ud1lAMBShWo7Og== X-Received: by 2002:a5d:4a44:: with SMTP id v4-v6mr29967278wrs.278.1537383305212; Wed, 19 Sep 2018 11:55:05 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:04 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 01/20] kasan, mm: change hooks signatures Date: Wed, 19 Sep 2018 20:54:40 +0200 Message-Id: <8b30f2d3e325de843f892e32f076fe9cc726191d.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN changes the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 43 +++++++++++++++++++++++++++++-------------- mm/kasan/kasan.c | 30 ++++++++++++++++++------------ mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 6 files changed, 63 insertions(+), 43 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 46aae129917c..3b019db87892 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -51,16 +51,16 @@ void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -105,19 +105,34 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) {} -static inline void kasan_init_slab_obj(struct kmem_cache *cache, - const void *object) {} +static inline void *kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + return ptr; +} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..55deff17a4d9 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -474,20 +474,22 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, return (void *)object + cache->kasan_info.free_meta_offset; } -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; if (!(cache->flags & SLAB_KASAN)) - return; + return (void *)object; alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +530,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +540,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +553,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +568,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +578,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index aa76a70e087e..6fdca9ec2ea4 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3679,7 +3679,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3715,7 +3715,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 58c6c1c2a78e..4190c24ef0e9 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index fea3376f9816..3abfa0f86118 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1183,7 +1183,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1461,7 +1461,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index 8da34a8af53d..b2172284d421 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1334,10 +1334,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2730,7 +2730,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2758,7 +2758,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3748,7 +3748,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3765,8 +3765,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3793,7 +3792,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } From patchwork Wed Sep 19 18:54:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606221 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D510B161F for ; Wed, 19 Sep 2018 18:55:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C85F32B840 for ; Wed, 19 Sep 2018 18:55:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BBC0D2B5EA; Wed, 19 Sep 2018 18:55:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEB9829178 for ; Wed, 19 Sep 2018 18:55:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 717468E0009; Wed, 19 Sep 2018 14:55:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6A44C8E0001; Wed, 19 Sep 2018 14:55:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 47CD98E0009; Wed, 19 Sep 2018 14:55:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id B9EFC8E0001 for ; Wed, 19 Sep 2018 14:55:10 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id l45-v6so6684616wre.4 for ; Wed, 19 Sep 2018 11:55:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FSVG7dQIIATpfJD6+7MFs3dz3cI5Pw75yb3eyIYepi8=; b=ocZF+rQMjMXauamOHTF6Qnh/fILeI12P+LSRSp6IJlnHnVyHzp/UpGdi/x/EPY6oxQ UW+GnS88vUlHBQjL6JNmButHn4D7teDHsPdQdjz1lAMdYWuovYjTFAZ6ShlwzA38WU2E LLYo6ViiQevwetWdGWBQZJTDBDJTluKCSHH/Cwh/8UPAQh0utgcwyO3+wFJ87zDf8tKM ygGNe0+FKUdsI+alGy+pG/bJQ1ihts3k4TopBMl3EbwOQ3WBIHkYbQEkpDoOIlyfxF6b OUuEH46r970yKKrdjCBXRC3/WpHP0n8FDdpusSDOjV8WIVKAyCfjZi8QCSS5Q5G1IQRU IoXQ== X-Gm-Message-State: APzg51BAd7gXoJu3jXDq1f2rupyqNVUwaZBwGcE3hfOwRdpY+pr5nmYa PLyuiai3MTvulh8L2+kGLpEZP154fl0pRRxV8qIZe1FQUZjBjJtOwEULvun8zi+ffdSnBakiYB1 kKfpU1nxMqMRXVAwuZTV33hL/A4E5+y0nvAX5iaK3vBUxRRKAdAWq6ftLl+EAK3DdhYbR9JlW/S 452cpXpDTaSkuqfW4z/WzXJGKHj4yP6LGUOB/d/OlqPTbnXM/8CAkReiQGwrUEXcK4FxDcWrors FutEYO8lG3OyOeT2M0fJprrmwSgMmk1pHlbW3P5DlkBZMMZ3GonNnQTJ6FRorPRMgSJrwFYAJIQ w1V1I1OVlfEmYH60UmvvclbV27gv7ZuvOb8qgen21Emcty0QFNMCRrD9zq9HDsEcS28ud3Aclka m X-Received: by 2002:adf:ffc7:: with SMTP id x7-v6mr20972593wrs.137.1537383310198; Wed, 19 Sep 2018 11:55:10 -0700 (PDT) X-Received: by 2002:adf:ffc7:: with SMTP id x7-v6mr20972490wrs.137.1537383307898; Wed, 19 Sep 2018 11:55:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383307; cv=none; d=google.com; s=arc-20160816; b=Do8GHkaRsq1adtbKKtPesJC+2MV42XFXKPhCXw6E7XHm1HCH2vRggCHyjmlxihI9un 6QLcPc4xDNvWlcVsUFHR3hhsXLc2QElJoSEfaJa0AWJxRtKezeybsCRQiELoRT/sPZfp e1DN3946BcHJAqULkSZEELpbXsPz9tKECyVLf0H4qxIOwf23O58oRIxFAP97Zd9a6EzD r/wqDq54vcKk5POuOx+L3BY4meaBjZ+9Vue5DMzeAPq52oQVUMVo9mFYD8jmkW5THdoh 6DpDUCNWFYXR0pIDXRjn5Pxjtg2OwrSS87Hk+w5rkkMmVYqUywBHWuT+kFBGNFpkbC6O LimA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=FSVG7dQIIATpfJD6+7MFs3dz3cI5Pw75yb3eyIYepi8=; b=nm0s/x2pFJCSyyE7s6HZcJYJYmFgnjNMtkhSjUlurdflYg31DJUiZUv5qhYD57VJ7u +3yQ/ttmZ0v9FvRTN96B9BW1CKLGEbfX5A17setH/T0IE0Wd0vrJX6JQknWzIqLvmGky R4omN0T/m5z5+In0x3+XwpY7Vjpz6z4cyfSaOZd1VYEZdGr9enVp6TFUo398laT286e4 tp4FtrZ8HqD46e3kgPZboEICcd7hgufvTKnSJ2lB6e+zhllHEAb8yrVA1f31fkwBecaq 0uYOk3CrzVc1z/fPC0jPhTECaWLPBoGr7Y9SGBgof9LCdZkaS26T0WzQteCNvqjbFeBD 4i5w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=lzaBchis; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 75-v6sor10704431wma.3.2018.09.19.11.55.07 for (Google Transport Security); Wed, 19 Sep 2018 11:55:07 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=lzaBchis; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FSVG7dQIIATpfJD6+7MFs3dz3cI5Pw75yb3eyIYepi8=; b=lzaBchis0O2hIfR1qyNUPtV5TDAMJwuTPe1A8vEP1FOPVREGeires9+TZNA1lEjYIu Pa0IxXwhyGqGTiB4gv76MFN+tpxNGQNj4nUsWEd87lUqq3kuHVs4oga5EWRVLX9JodMO 9uEl9+QfiLS2Ms5FEMJhs2buMoE6ooK0VDCIL7Ik1PzVZs3tXUY6rAJaUlJrMpeBlHkW vowx6IvSq95GiMxdruw4k/zYaPMpyYRAIUmWFp73aV+6MT79C4uEW2EYuPfF0F2Jk1Yc 2Cgya50F9H/D2pW72XPQykv9sQ0Gp7CQY8iGq3yY4F2wtGBQ3kzVCfQ8DRzmsItHiDby LK+g== X-Google-Smtp-Source: ANB0VdapylIJf6sEptPSF66CqQKnd5qKjLMwsc0MHjqhODQUOySalz80uRUXVEVt+RaadBy0HLnXMA== X-Received: by 2002:a1c:f03:: with SMTP id 3-v6mr21752610wmp.129.1537383306845; Wed, 19 Sep 2018 11:55:06 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:06 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 02/20] kasan: move common generic and tag-based code to common.c Date: Wed, 19 Sep 2018 20:54:41 +0200 Message-Id: <657b03e54ead4a744339fedb709a25b27d13b2ae.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN reuses a significant part of the generic KASAN code, so move the common parts to common.c without any functional changes. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 603 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 570 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 614 insertions(+), 569 deletions(-) create mode 100644 mm/kasan/common.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 3289db38bc87..a6df14bffb6b 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o diff --git a/mm/kasan/common.c b/mm/kasan/common.c new file mode 100644 index 000000000000..5f68c93734ba --- /dev/null +++ b/mm/kasan/common.c @@ -0,0 +1,603 @@ +/* + * This file contains common generic and tag-based KASAN code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +static inline void filter_irq_stacks(struct stack_trace *trace) +{ + int i; + + if (!trace->nr_entries) + return; + for (i = 0; i < trace->nr_entries; i++) + if (in_irqentry_text(trace->entries[i])) { + /* Include the irqentry function into the stack. */ + trace->nr_entries = i + 1; + break; + } +} + +static inline depot_stack_handle_t save_stack(gfp_t flags) +{ + unsigned long entries[KASAN_STACK_DEPTH]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = KASAN_STACK_DEPTH, + .skip = 0 + }; + + save_stack_trace(&trace); + filter_irq_stacks(&trace); + if (trace.nr_entries != 0 && + trace.entries[trace.nr_entries-1] == ULONG_MAX) + trace.nr_entries--; + + return depot_save_stack(&trace, flags); +} + +static inline void set_track(struct kasan_track *track, gfp_t flags) +{ + track->pid = current->pid; + track->stack = save_stack(flags); +} + +void kasan_enable_current(void) +{ + current->kasan_depth++; +} + +void kasan_disable_current(void) +{ + current->kasan_depth--; +} + +void kasan_check_read(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_read); + +void kasan_check_write(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + check_memory_region((unsigned long)addr, len, true, _RET_IP_); + + return __memset(addr, c, len); +} + +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memmove(dest, src, len); +} + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + */ +void kasan_poison_shadow(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void kasan_unpoison_shadow(const void *address, size_t size) +{ + kasan_poison_shadow(address, size, 0); + + if (size & KASAN_SHADOW_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + *shadow = size & KASAN_SHADOW_MASK; + } +} + +static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) +{ + void *base = task_stack_page(task); + size_t size = sp - base; + + kasan_unpoison_shadow(base, size); +} + +/* Unpoison the entire stack for a task. */ +void kasan_unpoison_task_stack(struct task_struct *task) +{ + __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); +} + +/* Unpoison the stack for the current task beyond a watermark sp value. */ +asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) +{ + /* + * Calculate the task stack base address. Avoid using 'current' + * because this function is called by early resume code which hasn't + * yet set up the percpu register (%gs). + */ + void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); + + kasan_unpoison_shadow(base, watermark - base); +} + +/* + * Clear all poison for the region between the current SP and a provided + * watermark value, as is sometimes required prior to hand-crafted asm function + * returns in the middle of functions. + */ +void kasan_unpoison_stack_above_sp_to(const void *watermark) +{ + const void *sp = __builtin_frame_address(0); + size_t size = watermark - sp; + + if (WARN_ON(sp > watermark)) + return; + kasan_unpoison_shadow(sp, size); +} + +void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); +} + +void kasan_free_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_poison_shadow(page_address(page), + PAGE_SIZE << order, + KASAN_FREE_PAGE); +} + +/* + * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. + * For larger allocations larger redzones are used. + */ +static inline unsigned int optimal_redzone(unsigned int object_size) +{ + return + object_size <= 64 - 16 ? 16 : + object_size <= 128 - 32 ? 32 : + object_size <= 512 - 64 ? 64 : + object_size <= 4096 - 128 ? 128 : + object_size <= (1 << 14) - 256 ? 256 : + object_size <= (1 << 15) - 512 ? 512 : + object_size <= (1 << 16) - 1024 ? 1024 : 2048; +} + +void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) +{ + unsigned int orig_size = *size; + int redzone_adjust; + + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } + redzone_adjust = optimal_redzone(cache->object_size) - + (*size - cache->object_size); + + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + + optimal_redzone(cache->object_size))); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } + + *flags |= SLAB_KASAN; +} + +size_t kasan_metadata_size(struct kmem_cache *cache) +{ + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); +} + +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); + return (void *)object + cache->kasan_info.alloc_meta_offset; +} + +struct kasan_free_meta *get_free_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + return (void *)object + cache->kasan_info.free_meta_offset; +} + +void kasan_poison_slab(struct page *page) +{ + kasan_poison_shadow(page_address(page), + PAGE_SIZE << compound_order(page), + KASAN_KMALLOC_REDZONE); +} + +void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_unpoison_shadow(object, cache->object_size); +} + +void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_poison_shadow(object, + round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + KASAN_KMALLOC_REDZONE); +} + +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; +} + +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +{ + return kasan_kmalloc(cache, object, cache->object_size, flags); +} + +static bool __kasan_slab_free(struct kmem_cache *cache, void *object, + unsigned long ip, bool quarantine) +{ + s8 shadow_byte; + unsigned long rounded_up_size; + + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != + object)) { + kasan_report_invalid_free(object, ip); + return true; + } + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + kasan_report_invalid_free(object, ip); + return true; + } + + rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + + if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; +} + +bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +{ + return __kasan_slab_free(cache, object, ip, true); +} + +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, + gfp_t flags) +{ + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(object == NULL)) + return NULL; + + redzone_start = round_up((unsigned long)(object + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = round_up((unsigned long)object + cache->object_size, + KASAN_SHADOW_SCALE_SIZE); + + kasan_unpoison_shadow(object, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); + + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; +} +EXPORT_SYMBOL(kasan_kmalloc); + +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + struct page *page; + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(ptr == NULL)) + return NULL; + + page = virt_to_page(ptr); + redzone_start = round_up((unsigned long)(ptr + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); + + kasan_unpoison_shadow(ptr, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); + + return (void *)ptr; +} + +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +{ + struct page *page; + + if (unlikely(object == ZERO_SIZE_PTR)) + return (void *)object; + + page = virt_to_head_page(object); + + if (unlikely(!PageSlab(page))) + return kasan_kmalloc_large(object, size, flags); + else + return kasan_kmalloc(page->slab_cache, object, size, flags); +} + +void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), + KASAN_FREE_PAGE); + } else { + __kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + +void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (ptr != page_address(virt_to_head_page(ptr))) + kasan_report_invalid_free(ptr, ip); + /* The object will be poisoned by page_alloc. */ +} + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL | __GFP_ZERO, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || + WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 55deff17a4d9..44ec228de0a2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,5 +1,5 @@ /* - * This file contains shadow memory manipulation code. + * This file contains core KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -40,82 +40,6 @@ #include "kasan.h" #include "../slab.h" -void kasan_enable_current(void) -{ - current->kasan_depth++; -} - -void kasan_disable_current(void) -{ - current->kasan_depth--; -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. - */ -static void kasan_poison_shadow(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - memset(shadow_start, value, shadow_end - shadow_start); -} - -void kasan_unpoison_shadow(const void *address, size_t size) -{ - kasan_poison_shadow(address, size, 0); - - if (size & KASAN_SHADOW_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; - } -} - -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_shadow(base, size); -} - -/* Unpoison the entire stack for a task. */ -void kasan_unpoison_task_stack(struct task_struct *task) -{ - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); -} - -/* Unpoison the stack for the current task beyond a watermark sp value. */ -asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) -{ - /* - * Calculate the task stack base address. Avoid using 'current' - * because this function is called by early resume code which hasn't - * yet set up the percpu register (%gs). - */ - void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - - kasan_unpoison_shadow(base, watermark - base); -} - -/* - * Clear all poison for the region between the current SP and a provided - * watermark value, as is sometimes required prior to hand-crafted asm function - * returns in the middle of functions. - */ -void kasan_unpoison_stack_above_sp_to(const void *watermark) -{ - const void *sp = __builtin_frame_address(0); - size_t size = watermark - sp; - - if (WARN_ON(sp > watermark)) - return; - kasan_unpoison_shadow(sp, size); -} - /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -260,121 +184,12 @@ static __always_inline void check_memory_region_inline(unsigned long addr, kasan_report(addr, size, write, ret_ip); } -static void check_memory_region(unsigned long addr, - size_t size, bool write, +void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { check_memory_region_inline(addr, size, write, ret_ip); } -void kasan_check_read(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_read); - -void kasan_check_write(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - check_memory_region((unsigned long)addr, len, true, _RET_IP_); - - return __memset(addr, c, len); -} - -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memmove(dest, src, len); -} - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memcpy(dest, src, len); -} - -void kasan_alloc_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); -} - -void kasan_free_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); -} - -/* - * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. - * For larger allocations larger redzones are used. - */ -static unsigned int optimal_redzone(unsigned int object_size) -{ - return - object_size <= 64 - 16 ? 16 : - object_size <= 128 - 32 ? 32 : - object_size <= 512 - 64 ? 64 : - object_size <= 4096 - 128 ? 128 : - object_size <= (1 << 14) - 256 ? 256 : - object_size <= (1 << 15) - 512 ? 512 : - object_size <= (1 << 16) - 1024 ? 1024 : 2048; -} - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) -{ - unsigned int orig_size = *size; - int redzone_adjust; - - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); - - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); - - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; - } - - *flags |= SLAB_KASAN; -} - void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); @@ -386,277 +201,6 @@ void kasan_cache_shutdown(struct kmem_cache *cache) quarantine_remove_cache(cache); } -size_t kasan_metadata_size(struct kmem_cache *cache) -{ - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); -} - -void kasan_poison_slab(struct page *page) -{ - kasan_poison_shadow(page_address(page), - PAGE_SIZE << compound_order(page), - KASAN_KMALLOC_REDZONE); -} - -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_unpoison_shadow(object, cache->object_size); -} - -void kasan_poison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_poison_shadow(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), - KASAN_KMALLOC_REDZONE); -} - -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline void filter_irq_stacks(struct stack_trace *trace) -{ - int i; - - if (!trace->nr_entries) - return; - for (i = 0; i < trace->nr_entries; i++) - if (in_irqentry_text(trace->entries[i])) { - /* Include the irqentry function into the stack. */ - trace->nr_entries = i + 1; - break; - } -} - -static inline depot_stack_handle_t save_stack(gfp_t flags) -{ - unsigned long entries[KASAN_STACK_DEPTH]; - struct stack_trace trace = { - .nr_entries = 0, - .entries = entries, - .max_entries = KASAN_STACK_DEPTH, - .skip = 0 - }; - - save_stack_trace(&trace); - filter_irq_stacks(&trace); - if (trace.nr_entries != 0 && - trace.entries[trace.nr_entries-1] == ULONG_MAX) - trace.nr_entries--; - - return depot_save_stack(&trace, flags); -} - -static inline void set_track(struct kasan_track *track, gfp_t flags) -{ - track->pid = current->pid; - track->stack = save_stack(flags); -} - -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); - return (void *)object + cache->kasan_info.alloc_meta_offset; -} - -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; -} - -void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) -{ - struct kasan_alloc_meta *alloc_info; - - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); - - return (void *)object; -} - -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) -{ - return kasan_kmalloc(cache, object, cache->object_size, flags); -} - -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, - unsigned long ip, bool quarantine) -{ - s8 shadow_byte; - unsigned long rounded_up_size; - - if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != - object)) { - kasan_report_invalid_free(object, ip); - return true; - } - - /* RCU slabs could be legally used after free within the RCU period */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); - return true; - } - - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) - return false; - - set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); - quarantine_put(get_free_info(cache, object), cache); - return true; -} - -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) -{ - return __kasan_slab_free(cache, object, ip, true); -} - -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) -{ - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(object == NULL)) - return NULL; - - redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); - - kasan_unpoison_shadow(object, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); - - if (cache->flags & SLAB_KASAN) - set_track(&get_alloc_info(cache, object)->alloc_track, flags); - - return (void *)object; -} -EXPORT_SYMBOL(kasan_kmalloc); - -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) -{ - struct page *page; - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(ptr == NULL)) - return NULL; - - page = virt_to_page(ptr); - redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); - - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); - - return (void *)ptr; -} - -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) -{ - struct page *page; - - if (unlikely(object == ZERO_SIZE_PTR)) - return ZERO_SIZE_PTR; - - page = virt_to_head_page(object); - - if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); - else - return kasan_kmalloc(page->slab_cache, object, size, flags); -} - -void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), - KASAN_FREE_PAGE); - } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - -void kasan_kfree_large(void *ptr, unsigned long ip) -{ - if (ptr != page_address(virt_to_head_page(ptr))) - kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ -} - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); @@ -797,113 +341,3 @@ DEFINE_ASAN_SET_SHADOW(f2); DEFINE_ASAN_SET_SHADOW(f3); DEFINE_ASAN_SET_SHADOW(f5); DEFINE_ASAN_SET_SHADOW(f8); - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c12dcfde2ebd..659463800f10 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow(const void *address, size_t size, u8 value); + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); From patchwork Wed Sep 19 18:54:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 371BD6CB for ; Wed, 19 Sep 2018 18:55:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29FFA29178 for ; Wed, 19 Sep 2018 18:55:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1CAAD2AF40; Wed, 19 Sep 2018 18:55:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DAFE29178 for ; Wed, 19 Sep 2018 18:55:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C95AF8E0008; Wed, 19 Sep 2018 14:55:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C2AB98E0009; Wed, 19 Sep 2018 14:55:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A97D28E0008; Wed, 19 Sep 2018 14:55:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 4A8AF8E0001 for ; Wed, 19 Sep 2018 14:55:10 -0400 (EDT) Received: by mail-wm1-f70.google.com with SMTP id o25-v6so3869039wmh.1 for ; Wed, 19 Sep 2018 11:55:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=TFebpY+4DQvyEj9/6cNobOxFdZpML3D62x2Uz3V3Bjo=; b=GeNNWEzbJntDQsM2/e07A97b4yi+T8/eVYCo5Dfc311jQ7VCY/1Wk6qENNGhatPjH3 3IsSUk5TXNOlYO/uIPsbkHcSCrgy5PdTuaYGW2Pj3Mm5Q1g04f2bSjmqMiwf5iQFYrPX XDeTmLYycVCq5IXOzAyEtnIT3/D3H436zKB1XZMke0SQHQQWmY+NFovBigxUOxPmOYx2 a5AJcojg6ZIKc/5e20RTVBP68UBHjew1SbHAv+l2MUkFOEjfQs0DOD9KwEmnU3PLZLu+ f1eiDOeiVGqFzOQZDZiZkuCoUsa3Uhggd8EgXphB1IF03DIChEeYBRSRltUcvCM9wakX Y6Sg== X-Gm-Message-State: APzg51CAGb2Qyvt/4D03cJgvHqzUOwrRcOIvkKLfKsptLMGAsQwqU7fp 6boxsB9Z7H+Y/EbgjqHoISuvo1rJRZ4o++LDdYf9+L7wCmYYnLigJUsSmKMPmeKKWiwsiAh2fvG J8JmC3J5KhjIZKpf2TulVbM7RmLM7tt4ODQIl8DwDfT4FBzvMIyMIu3nuruD3W1fwTXfNmsRh2u qErgrl871b+9A7zQkrPjXOZBp6LFZEWbnzb5rmeqtHIdBI8OTVd5ZDCC+kbR/OxrHk+2mD00TdT zpNjtGlKpI4cEFw22zum7u8fYeyA2wq2BbXBALOSaQbgpirjgK7xnMam7gevUHggmLyNh65Cclc ZcSioDMPTfH1uDGF8exhYdlBo+MVHtj/ryK92RDFGCtIP7S03j2166I0gU9PeX3j0gg27Tv0dvd Q X-Received: by 2002:a5d:62c2:: with SMTP id o2-v6mr4023922wrv.83.1537383309824; Wed, 19 Sep 2018 11:55:09 -0700 (PDT) X-Received: by 2002:a5d:62c2:: with SMTP id o2-v6mr4023889wrv.83.1537383309301; Wed, 19 Sep 2018 11:55:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383309; cv=none; d=google.com; s=arc-20160816; b=Z26Swxbkq9Nyrg4VCwtiDQVa/4Tg4PANJUnIYnuNoUpcRtdetAZe8buQ6nvsjkfCJ8 c69e7PvEw5Q0ioG4q3rchRYxVo1eB+5OOVunD+8qht51YZ7yqN1acN9KuQHlX8X7yNbe uWLDPXLwAQUZtn13SKaB0RxxKaZjUnvwYvb2h9lXlLjyYOWIswzLLGRnPYdJLm74b1PP runw+lXpP8Kk1McOnswv/XjuPwij1dZZe9GydVm+Ady4PL5u999GnC/hI9BVMYJOWTdn 45upRNHl2m3mWjzBu5XTR+KYucVPVHxf9f8NfoGEFnV/Nu3084Ut3aI4oNDeE5IhLq9Z 3baQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=TFebpY+4DQvyEj9/6cNobOxFdZpML3D62x2Uz3V3Bjo=; b=Up/qRxBNEkEQBfynkztjkcCS1H4+zGN9I4XYIY84Ujzd4G6o8LYtsBudfsEm3OjtUR RKNwjwm0eSNeXXfvUgLzkZi3yo4+4zCr/jN4pDgSVHubq8Q54xRv56KeVbTZFRiCCKxj /rM6lrvKILvBek7int+jHJI76V969y1hURtQH6EqrhR/LNeWUeXxbsgzYzKofSmNVTFG 1/WIlHkzz14qEp4sXkzz968Sm5ElQCVen+cwkaAfqiHdTWqntpCVPxLhdvgfMs9k1rjc rtqR7cBbgnrXbRyxmEhUA3Jb7FlA/O6kWQdJdvQU5cOGDytpMUPvDFkZ3vKvtjoTHPNz VnRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=V7r3yStp; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id x25-v6sor10587174wmc.0.2018.09.19.11.55.09 for (Google Transport Security); Wed, 19 Sep 2018 11:55:09 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=V7r3yStp; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TFebpY+4DQvyEj9/6cNobOxFdZpML3D62x2Uz3V3Bjo=; b=V7r3yStp8H8MLbU05RxHTdCCSUIgot36k2lVklepQ6thP99J4eEl3X/BcbgdKQG1cz ov14PQJBKjiPdC/Kpqf4Y3Q0P37buFLjzhzPU+yNBGBWsxkHBcfkqB8b03K3wNqdybhp fxxpp0jzhrRzTqHV+2WZ08KSFfTbNRn1sDXrw0bU7l2701k9xgSWWuNv3/2JfepTlEAO hKe65XlQMlQdbbKJEqkzx4qdIuCAmo3cxk2HShXqIi2HIeZeZocsqQYGItsSmyTubffR dP/7Hmv8XLY/EY8MIrV6zlserQlbtqkJZ1KjJKzC6/bqj4rHiQrM5VTYpnxvSvXzadEx J8rQ== X-Google-Smtp-Source: ANB0VdZUu+vy1Owz3dpk5x8u2wXrzI0XtzccwYM/JDM58MTR37LaKxyy0LCRQOuaC/+2M+0l7KHF5A== X-Received: by 2002:a1c:f0a:: with SMTP id 10-v6mr21514675wmp.58.1537383308879; Wed, 19 Sep 2018 11:55:08 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:07 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 03/20] kasan: rename source files to reflect the new naming scheme Date: Wed, 19 Sep 2018 20:54:42 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We now have two KASAN modes: generic KASAN and tag-based KASAN. Rename kasan.c to generic.c to reflect that. Also rename kasan_init.c to init.c as it contains initialization code for both KASAN modes. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 8 ++++---- mm/kasan/{kasan.c => generic.c} | 0 mm/kasan/{kasan_init.c => init.c} | 0 3 files changed, 4 insertions(+), 4 deletions(-) rename mm/kasan/{kasan.c => generic.c} (100%) rename mm/kasan/{kasan_init.c => init.c} (100%) diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index a6df14bffb6b..d643530b24aa 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,14 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n -UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_generic.o := n KCOV_INSTRUMENT := n -CFLAGS_REMOVE_kasan.o = -pg +CFLAGS_REMOVE_generic.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o generic.o report.o init.o quarantine.o diff --git a/mm/kasan/kasan.c b/mm/kasan/generic.c similarity index 100% rename from mm/kasan/kasan.c rename to mm/kasan/generic.c diff --git a/mm/kasan/kasan_init.c b/mm/kasan/init.c similarity index 100% rename from mm/kasan/kasan_init.c rename to mm/kasan/init.c From patchwork Wed Sep 19 18:54:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606227 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A65B46CB for ; Wed, 19 Sep 2018 18:55:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9899929178 for ; Wed, 19 Sep 2018 18:55:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B4D12AF40; Wed, 19 Sep 2018 18:55:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49F7D29178 for ; Wed, 19 Sep 2018 18:55:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C1E6D8E0001; Wed, 19 Sep 2018 14:55:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BCEDB8E000A; Wed, 19 Sep 2018 14:55:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A25798E0001; Wed, 19 Sep 2018 14:55:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by kanga.kvack.org (Postfix) with ESMTP id 1E6418E000A for ; Wed, 19 Sep 2018 14:55:13 -0400 (EDT) Received: by mail-wm1-f69.google.com with SMTP id c14-v6so3975315wmb.2 for ; Wed, 19 Sep 2018 11:55:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=z1DpyEItVxsft4UDs0uoNYq6KT+yPmp62IEAxvoeP10=; b=BRChrdd7UXFi+bPVrnZLnXNIqJaah+Tu+l0VoJwNXCl2SAIVzQKlT93IHoPSSw+7BX xndR80eigmfZY2RZn9x57Efk98LS5if1HS8peFuACGLu+68lgzbmlF6mWiJrGjq4nuWW Ny3n83O4kG3tg1eiutfPtsRa3HYnLxPgaCYILlce5nW0qk5zswfUPRYRhx1u2UC02MpH 1sz0iUssIvDy51EYgj0iHpX2MpHdAg5chBaINpYWgruQOQfeiXV+2LfNj6eRbQ+jOed4 jfjQwEcsTujfPswVoofNgst15LHZ84yrddYawMgdRQRS2dWKahG5w+SqbfZcRhhXU6UT a7ow== X-Gm-Message-State: APzg51Buy01sMc7PNclHfGn+wHM6CxJTnyLgeMgnAOp0HebyohQzYQPH HJRrYO4O5Qxc6scpY3gvQL4k4U9LSIwUb9gJDWTsXVv6fjrTnMnWv8xiSgDLhbG3KfClozOYqMI cySjUWmp0KNxYfI2M/NBwBGov2/svN3F5hincd/M7QWeOS8LnwCPRnLPNuVccbw2NclBJVhU78L 8xDKuPpWaEBfJnN8f6U5bBAQR0+E/o5Uo/HhvKTe7iqiZOfHtEYkjSTQZtkmWcCjDtoHrhIFbWX v7qjnK5Xa96rZjvklvz+cR0EdO5A5jMwFcDPKHqGgDnMRItOFTBOMsT7ypa2CF+nRK159UB3E+J mU5FnrRkZmElywQIpIiffM7sL684NkM26LqInN4j7/vgiADndDFQqMjNIkbc+cSq8iu5r2Q/z0B U X-Received: by 2002:a1c:578a:: with SMTP id l132-v6mr20708796wmb.16.1537383312508; Wed, 19 Sep 2018 11:55:12 -0700 (PDT) X-Received: by 2002:a1c:578a:: with SMTP id l132-v6mr20708749wmb.16.1537383311079; Wed, 19 Sep 2018 11:55:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383311; cv=none; d=google.com; s=arc-20160816; b=YDgz7C1e4uy/qV19CwgXL/WUqO51mzWCdyoI9LFh7bs9BkM5kvPTC4Axc0wJBMy5ZK dI8s/Ul104wWY6mqWOhPLgN3dPVhm8+/F71ESrhYw1li5cE0XvRB8p5XU55DwTUVuGar BEN1x+NtkF8CWRXpp7XEjKf8LI+EGfyasHJBJiO+K4z2cHJJO052zAySO4gk1Utcc3II a6MOX1ZNhqFCrOsPUNAY3JK5Jx4ni0afAXcJNqz/PCGD6gVzekE53tET1CV2mvILntj/ zFd0Aa1axuIYCNV9oQ8ytON4bpwt7FLl47/KscITHVkEx5zeGORomR9T5wu4FHyKbWDg ohpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=z1DpyEItVxsft4UDs0uoNYq6KT+yPmp62IEAxvoeP10=; b=Uk3mg6XkcRGDxMgO5uHC1tToLzOXbb5Vfc5pfdrvJCr9KoMp5Gg2bd2o3x1SMAC8fN ZxAhbP8nOS4c7bgTPurnuZY4TSUlI/nNEoJKRzXvwCSS7FuMk6JFFbDkKoeNITJvcrFB 1T66lBYYVR5aJ46l7vhbYkJ4NYF+n5wyDs7BiwBP8NZIuDHhZkfUpjd8FWN6yieBStYs dxESrOu1ubspns7Zp6ZE3Ncm12b60Od4LaLXbdunoUf94ZYeW+y63Vykor0XTEXZ9WCU vsW9iiJUVslpCbAar+lTQY92AHr3/bRGARdENj4ZKks8/XSA0gnwzukfLQeEO3LlESBK z3zg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="q67/fJLv"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y31-v6sor16632205wrd.50.2018.09.19.11.55.11 for (Google Transport Security); Wed, 19 Sep 2018 11:55:11 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="q67/fJLv"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z1DpyEItVxsft4UDs0uoNYq6KT+yPmp62IEAxvoeP10=; b=q67/fJLvxhhQkbBrOp4X3P+0kzwrQ2ubg1Bb+e+UA0pox8eQcanTb2fipIV4iJNR2N V/L8OezRsDLE1yEOvLbx3pFGRDCbm3uTkkNEpcVwwVQW838BQ/QNVHumEwbt+isFWKe0 gZt1AIZZRtLyyP2XVfFgxuZnX0gt/6W3NTz/HzE3Bc0tzsQxa5YwIe8lZmXtf1nTy/Dj 7tYP8Ib7Xwj868q+moTqmCvrgnMDX/6qMgPCE8tnD29dnhzMZ0lnVzUOCYi/ihhcgPwu gfpZu/hrSSLjxGoeOsId1CYUadDuBxSmD59ehnXKBJWK9g1zyn4L2WVMhHKBpPsGe5U7 VBGg== X-Google-Smtp-Source: ACcGV617bqixM7ctYqPGVjF0qqFKJ3+6ZMrrndnZeattBF8UGMI788ETzzR2MMrk/noIGv/OTeaq3g== X-Received: by 2002:adf:e30e:: with SMTP id b14-v6mr359767wrj.158.1537383310437; Wed, 19 Sep 2018 11:55:10 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:09 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 04/20] kasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS Date: Wed, 19 Sep 2018 20:54:43 +0200 Message-Id: <0445e177bfb261d02dae639deefdd1d8ec8aacc2.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit splits the current CONFIG_KASAN config option into two: 1. CONFIG_KASAN_GENERIC, that enables the generic KASAN mode (the one that exists now); 2. CONFIG_KASAN_SW_TAGS, that enables the software tag-based KASAN mode. The name CONFIG_KASAN_SW_TAGS is chosen as in the future we will have another hardware tag-based KASAN mode, that will rely on hardware memory tagging support in arm64. With CONFIG_KASAN_SW_TAGS enabled, compiler options are changed to instrument kernel files with -fsantize=kernel-hwaddress (except the ones for which KASAN_SANITIZE := n is set). Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_SW_TAGS support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes. This commit also adds empty placeholder (for now) implementation of tag-based KASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options. Signed-off-by: Andrey Konovalov --- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 5 +- include/linux/kasan.h | 16 +++++-- lib/Kconfig.kasan | 87 +++++++++++++++++++++++++++------- mm/kasan/Makefile | 6 ++- mm/kasan/generic.c | 2 +- mm/kasan/kasan.h | 3 +- mm/kasan/tags.c | 75 +++++++++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 27 ++++++++++- 10 files changed, 194 insertions(+), 30 deletions(-) create mode 100644 mm/kasan/tags.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1b1a0e95c751..287c32241b68 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -105,6 +105,7 @@ config ARM64 select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_SW_TAGS if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index b1ce500fe8b3..89ebe58259ba 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -17,11 +17,12 @@ #define KASAN_ABI_VERSION 5 /* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature(address_sanitizer) +#if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) #define __SANITIZE_ADDRESS__ #endif -#define __no_sanitize_address __attribute__((no_sanitize("address"))) +#define __no_sanitize_address \ + __attribute__((no_sanitize("address", "hwaddress"))) /* * Not all versions of clang implement the the type-generic versions diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 3b019db87892..f0c96f08f2f1 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); -void kasan_cache_shrink(struct kmem_cache *cache); -void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); @@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} -static inline void kasan_cache_shrink(struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, @@ -155,4 +151,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink(struct kmem_cache *cache); +void kasan_cache_shutdown(struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink(struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ + #endif /* LINUX_KASAN_H */ diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index befb127507c0..181dfa7f4885 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -1,35 +1,86 @@ config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_SW_TAGS + bool + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs. + See Documentation/dev-tools/kasan.rst for details. + +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: generic KASAN (similar to userspace ASan, + x86_64/arm64/xtensa, enabled with CONFIG_KASAN_GENERIC) and + software tag-based KASAN (a version based on software memory + tagging, arm64 only, similar to userspace HWASan, enabled with + CONFIG_KASAN_SW_TAGS). + Both generic and tag-based KASAN are strictly debugging features. + +config KASAN_GENERIC + bool "Generic mode" depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables generic KASAN mode. + Supported in both GCC and Clang. With GCC it requires version 4.9.2 + or later for basic support and version 5.0 or later for detection of + out-of-bounds accesses for stack and global variables and for inline + instrumentation mode (CONFIG_KASAN_INLINE). With Clang it requires + version 3.7.0 or later and it doesn't support detection of + out-of-bounds accesses for global variables yet. + This mode consumes about 1/8th of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +if HAVE_ARCH_KASAN_SW_TAGS + +config KASAN_SW_TAGS + bool "Software tag-based mode" + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enables software tag-based KASAN mode. + This mode requires Top Byte Ignore support by the CPU and therefore + is only supported for arm64. + This mode requires Clang version 7.0.0 or later. + This mode consumes about 1/16th of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + This mode may potentially introduce problems relating to pointer + casting and comparison, as it embeds tags into the top byte of each + pointer. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_SW_TAGS doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer + This enables further checks in generic KASAN, for now it only + includes the address-use-after-scope check that can lead to + excessive kernel stack usage, frame size warnings and longer compile time. - https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more + See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 choice @@ -53,16 +104,16 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endif diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index d643530b24aa..68ba1822f003 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_generic.o := n +UBSAN_SANITIZE_tags.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_generic.o = -pg @@ -10,5 +11,8 @@ CFLAGS_REMOVE_generic.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o generic.o report.o init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 44ec228de0a2..b8de6d33c55c 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,5 +1,5 @@ /* - * This file contains core KASAN code. + * This file contains core generic KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 659463800f10..19b950eaccff 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB) +#if defined(CONFIG_KASAN_GENERIC) && \ + (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c new file mode 100644 index 000000000000..04194923c543 --- /dev/null +++ b/mm/kasan/tags.c @@ -0,0 +1,75 @@ +/* + * This file contains core tag-based KASAN code. + * + * Copyright (c) 2018 Google, Inc. + * Author: Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE(size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE(1); +DEFINE_HWASAN_LOAD_STORE(2); +DEFINE_HWASAN_LOAD_STORE(4); +DEFINE_HWASAN_LOAD_STORE(8); +DEFINE_HWASAN_LOAD_STORE(16); + +void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_storeN_noabort); + +void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_tag_memory); diff --git a/mm/slub.c b/mm/slub.c index b2172284d421..c4d5f4442ff1 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2954,7 +2954,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page, do_slab_free(s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr) { do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr); diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 69552a39951d..5bf3a808a282 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else @@ -42,6 +42,29 @@ ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif + +ifdef CONFIG_KASAN_SW_TAGS + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +ifeq ($(call cc-option, $(CFLAGS_KASAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KASAN_SW_TAGS: \ + -fsanitize=hwaddress is not supported by compiler) + endif +endif + +endif + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif From patchwork Wed Sep 19 18:54:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606231 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4FBD91390 for ; Wed, 19 Sep 2018 18:55:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42B3A29178 for ; Wed, 19 Sep 2018 18:55:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 34F7A2AF40; Wed, 19 Sep 2018 18:55:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8B2A29178 for ; Wed, 19 Sep 2018 18:55:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF6388E000B; Wed, 19 Sep 2018 14:55:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B80B28E000C; Wed, 19 Sep 2018 14:55:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 960118E000B; Wed, 19 Sep 2018 14:55:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id 33EA28E000A for ; Wed, 19 Sep 2018 14:55:14 -0400 (EDT) Received: by mail-wm1-f72.google.com with SMTP id z23-v6so3848524wma.2 for ; Wed, 19 Sep 2018 11:55:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WcXjOX/25U5olMrelAjRtQalbkfy2CfWqZfIuu8HMaY=; b=TLDVdmu9GdEUOPkJvmszTpSvQfqEoKe8QZIax2GipXlbx5DV6jkoToAV/6nPHrXGLq zT6Yaw9QZzKJpUA1VxqXTg80BoibjzeHeeQs0IJIgd3ZXa2SWAB94kggrIB+yxnyBw4/ aTMkSfqkswogrGqQXM9qwiu4Z40BS2ywpuCtW3HahUKvl+yvm+SrQJ/SdzPOSJlVpd05 ia1h/UVBTLZVNklL90oHWUYk9a+MbT0ItNfYyvUypebiaiKlk7YtTuoV7mt1ie0IWNbK Z3NcZA2GJiIEqsQe77sNJZJPBl1pq/chgQN5IpTReReXOyHCDpUPYVNNxpifwuU0A70B 8ujQ== X-Gm-Message-State: APzg51CINn7yS2aqCloEvKYWvgntNidMmrYMiVTGPej/lkyqTK2vd/hy tBQ0pTUdkS/W8fF8X7h7lVIAtuEoDkc6/iM4xKtlREcG0GzaNgPvLe5FSkBBP0fi69JQ1q+c5yu pYKEoL1QpLDkPRtOLe7hyXn0vm5UnQVben+j3HVVr5EFK5lhuBhqLLQLHbXZlkp20iNY/FvZu+h tCaN6Lju62/IJVuVAtjP9ih6VJo9WmleLf3BFDddA8JdNSKhE9/C9jbUi/LkJ0vASehhJ7qYeiI aWVaCK57yf2qB6DFigGBvPXkNb5xIy5EJewK2MPC+0NxwDF2ONt84MGvl1FsNEaGk1XEgYVWDaT spLT0SNmC967PLsIbQ5McaoA1wQBF531oM20QTXBYtx1guOEB9UazI3A3Bw/uRGga8qSvO7HNzL l X-Received: by 2002:adf:f1cc:: with SMTP id z12-v6mr11747282wro.214.1537383313299; Wed, 19 Sep 2018 11:55:13 -0700 (PDT) X-Received: by 2002:adf:f1cc:: with SMTP id z12-v6mr11747242wro.214.1537383312324; Wed, 19 Sep 2018 11:55:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383312; cv=none; d=google.com; s=arc-20160816; b=L1YTDbe5RcFxw71FlPon6w+1bHeUwuT3bZMgpDfntrTGklZhXni/4SOXXPrdSUIvNt E56mBmCnVIpvQsr7aq357xxS3pcS4m2fyVnpGU++lbL7jmdAwVp7msfCm2fmFELR+DWi IA6+608BQCZ72H6WgaCuWxaRuvqAZDENqMFL2cxaSoA1cJP1YV/pLO4ASAn1ILXyQbpI J4nYYQCvqhahIAz1SNfV1fgWmvmTvxzBWq2oZSq0BDLGjlqPLgfWnv19hVZf2CavdZo8 tMDga1jWeyq0+ubhiXVcFQlQxVjnPBRhfifTusrf9MEOLuQSNCF+y3lhh+zrSxIGrLV4 UsEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=WcXjOX/25U5olMrelAjRtQalbkfy2CfWqZfIuu8HMaY=; b=0jy33pdN0P/GPRF5wt8MgO33RK2oldenE+qE/fEMmU49u/Cc97CYOcgeZyvo0mNXqF lyIvCm2bD94dBdK0SUOcSTigrh26cBPF+bYoCmAO0d5PebaowpKrhIOth0/dqkiFZGfk r1WVG8wJbZkTkIex/WWBkAuK/qvUj4ixI88gTUr07iuyY+P+Ipfbn+aRvJMhR4zD65/w Ch2XexSEIoURbJ7IeVkxIXKB70E+JbGveWrbd5kh7MjiFxb5oyYx2Nf2rTRAEjk1oWCZ 63sOT+mPQCXE4+NS5e9X7zUSbK2v23yszOGRmqvCQ/nbceo/vqt8N6LS2TVNZinw4zYd Kkkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Iy7aGVBO; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y12-v6sor16845084wrq.31.2018.09.19.11.55.12 for (Google Transport Security); Wed, 19 Sep 2018 11:55:12 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Iy7aGVBO; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WcXjOX/25U5olMrelAjRtQalbkfy2CfWqZfIuu8HMaY=; b=Iy7aGVBOvNjRzJTMfIM9jvoanw1p4F9qG39Ci75WPpNEx5WlKRhOBs+1ubk8fINpW3 6ur1SEXAWClreRmdxheH3xAYipMoT+LKxG6m2Lw3j91u2prq5DE205aQ/G4GN1OrKXaO zZrekhwgCZBz97AQsO57KjGQc5C4q6S3y92c/u6Q3RCrc/p0+3RMvLfa6b1Y3vmYhzbK PzJ8cYGEJZn6fSHdut0g7KAjSnpHNufUAXwFFpsrp7v9mNW8WH4smg3vQnJhfrfIfwQZ bZJZWOneyEHU4nyh6XB82Xes8h+4Zwz8+bSy5UeRBELaqYWtTxNFY13dp6W+gJ8+oWUL X3Xw== X-Google-Smtp-Source: ACcGV62ncZEsDHCPcaqbfKEUhaeWdWVi+9st0BcKxK2qyZSTgeHH4bT7gZ1IH8gFvskTSZisaMSQrw== X-Received: by 2002:adf:9e92:: with SMTP id a18-v6mr439691wrf.70.1537383311952; Wed, 19 Sep 2018 11:55:11 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:11 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 05/20] kasan, arm64: adjust shadow size for tag-based mode Date: Wed, 19 Sep 2018 20:54:44 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory. This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when the tag-based KASAN mode is enabled. Signed-off-by: Andrey Konovalov --- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/memory.h | 13 +++++++++---- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 106039d25e2f..11f4750d8d41 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -94,7 +94,7 @@ endif # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 +KASAN_SHADOW_SCALE_SHIFT := $(if $(CONFIG_KASAN_SW_TAGS), 4, 3) KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b96442960aea..0f1e024a951f 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -74,12 +74,17 @@ #define KERNEL_END _end /* - * KASAN requires 1/8th of the kernel virtual address space for the shadow - * region. KASAN can bloat the stack significantly, so double the (minimum) - * stack size when KASAN is in use. + * Generic and tag-based KASAN require 1/8th and 1/16th of the kernel virtual + * address space for the shadow region respectively. They can bloat the stack + * significantly, so double the (minimum) stack size when they are in use. */ -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_SCALE_SHIFT 4 +#endif +#ifdef CONFIG_KASAN #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_THREAD_SHIFT 1 #else From patchwork Wed Sep 19 18:54:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606235 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6F6F56CB for ; Wed, 19 Sep 2018 18:55:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6225629178 for ; Wed, 19 Sep 2018 18:55:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 554BF2AF40; Wed, 19 Sep 2018 18:55:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2FB029178 for ; Wed, 19 Sep 2018 18:55:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5F9E8E000C; Wed, 19 Sep 2018 14:55:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A97628E000A; Wed, 19 Sep 2018 14:55:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93D748E000C; Wed, 19 Sep 2018 14:55:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 2E1068E000A for ; Wed, 19 Sep 2018 14:55:15 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id y32-v6so6636570wrd.19 for ; Wed, 19 Sep 2018 11:55:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fbr13c4fj47e45ylSZzn54WLgirbXAILEytxVKinDw0=; b=ZQQ0j+zqrL7NTv0y+P3Ersnx3/jCz7/xxgB99JzfafTVxLhmm5jkWYHx+gLyho0pYq T6VS8K1xDk+xGN2wC1SAYZojgwy1lxqBFlL8Li3anzUVZrsTkZ6HePWEhG+czb1VWoIm 7f39jLtFjFnQ/fVyohZKcUjS27CnqokauUvTHtvDeWcqUa8Yqb6LcoJp50tD9hUCCg03 QD7qO0SjYfSHNKpQgRAWknReUs5MevxNh9epmOy0wHmyzEDsdf2TpE3jtosSK9uUAlJ6 aTb+3EeQELh+7z9FgpynS3SwuLqW8gqXEzS/a5cc0KRLuygBQaBd1ylqRj94rdmD6mb7 C9ag== X-Gm-Message-State: APzg51CPyMKmaJtK5r2dLN6gvlzK4Lpx/u+yEBJpSvYisDWXHEcPPQaU LhmiVUowBkHBNYAbwIzrGIJgJs7Hrm2J01DGp0hJbu0MucoNT7k5/VoE3faF+yGWycmItCyOnYw eVciRxthCI2gUvsX83v1xvIYezZ/hvlh4sMqbfZIgHDsLYv+zAz29rPGBLdP3gH6FaKVMCqUSXJ oSgLhLmVhASYHZSgcPpTWoIUqtmEqMt2nDbX+XhmrgmH65HrYthOjwHJ8UckykaMRGd3NKq/6Jg c0DHu4F+xm2ES5b7pxrkP1isngWMo8ZyL9PYxwc/A78Cv/nPZRRRnXI/CCTIXeC1Nz1EQGs0quD 36CmQlr7SqnERO0ZJlh3MRhuRJnTlYesaNHtv3JjkXq1l4dLnr+pbEPzYrWKrq2MhX5jEltJWVe h X-Received: by 2002:adf:afd3:: with SMTP id y19-v6mr29716863wrd.176.1537383314706; Wed, 19 Sep 2018 11:55:14 -0700 (PDT) X-Received: by 2002:adf:afd3:: with SMTP id y19-v6mr29716820wrd.176.1537383313903; Wed, 19 Sep 2018 11:55:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383313; cv=none; d=google.com; s=arc-20160816; b=XXiH7VyOEbaqxJxycVcCrCMLzFG/N4LSCuaqGEKVifNCNyKtao/F1YUisdtjl66maC YkrrYbNcRjv3mtgivWd58pBzsFvRs+mIQLPW86e58BNQmPQlNUDQDi9HCZl42YtWgWu8 b6WfV6iDdXc9TwXPgh5JVHNH8TCb2CRb8yrrAjd+PPRPRHxBb+g2iSXgy0xc2+oCPH2M HmIMaDx8XtV1Hkigo8MMik+gr7Tlt1laDbKuD3ASSENMp4pXg02ACdy3n+Q7lBsyroNj FkZnyo2xlL+FhqPMRXRj4M/wjguCxVpXCxYnFqfj6WmxI0veglkLNizD1lBIFDpd/sf8 DqgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=fbr13c4fj47e45ylSZzn54WLgirbXAILEytxVKinDw0=; b=fawTvzLTDP+3+jpK51ERaJGqFlNmL1XjxTkYrI1lGl5ONOpIeKycvy0RSYO0rVpPgh tiBDE7eZTDH2+iCu/S+CjEPK+yQAiKcOOKCBot/UvGasKkOPgOu8J7VzT2MF0B8eYRQ4 u01BQIlx173rZH8XmKRj+SMJqoyZktTjbaGIan9hE+8G576+ODVe8AWd/BrUetij8URY Dk6HvMAUDXjICdZmWOeasJGtf30NOJhmwgW+a9lpgoUzltLaCqwGyJC/y5180Iszj8fL k+hiQWQR1PFHrRnQxULQl16sSUTKmBhj4SRfYErL8m+pSoILPjr0a8i4QAHETu7xF1k0 xPxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BJlkoTl7; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id p18-v6sor463501wrr.34.2018.09.19.11.55.13 for (Google Transport Security); Wed, 19 Sep 2018 11:55:13 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BJlkoTl7; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fbr13c4fj47e45ylSZzn54WLgirbXAILEytxVKinDw0=; b=BJlkoTl7DNX07411Z8gMKQZAU8fjdC7LckXv0Baa2hcOKewrzLnPGct4wlnFtiwPSz gnRmNQyCyQs5wm6hbL96VCPVFE37GVc9SXIClOxEVfSLRbFQfj7CfKtiTDwL/+Tlzg81 5wMGVa7eUKqpKQDzqPS6LIFmZMI/xA8kv8oyOce1jcwf5fn1BkAwenslIrjY80wpJysP ApQsqjtBL5oCnnDlTH2gaUBTNj/wZFG92hM1RloiA53KGOo4Z3MSiuzvIDOZwpK5UG7E 18n64L6jYKJpaoLfftbtKnlearmnt54T1LfojC+FU9GbBeJ0Rkxd2j8bY2gZPBaIzqFC ri3A== X-Google-Smtp-Source: ANB0Vda2ndFUzi6RYaDwQ1hahv/tKgAsCAZ9rE1TeNDSNIIlR3lhKDJe9CbUqZ8tKu6Nq3DfP9XRCw== X-Received: by 2002:adf:a10c:: with SMTP id o12-v6mr3930268wro.169.1537383313499; Wed, 19 Sep 2018 11:55:13 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:12 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 06/20] kasan: initialize shadow to 0xff for tag-based mode Date: Wed, 19 Sep 2018 20:54:45 +0200 Message-Id: <8344a2ecd5b7335faa095a2da765f3ddc8d929a9.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A tag-based KASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with tag-based KASAN we need to initialize shadow memory to 0xff. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 16 ++++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 12145874c02b..7a31e8ccbad2 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -44,6 +44,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) return __pa(p); } +static phys_addr_t __init kasan_alloc_raw_page(int node) +{ + void *p = memblock_virt_alloc_try_nid_raw(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, + node); + return __pa(p); +} + static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { @@ -89,7 +98,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, do { phys_addr_t page_phys = early ? __pa_symbol(kasan_zero_page) - : kasan_alloc_zeroed_page(node); + : kasan_alloc_raw_page(node); + if (!early) + memset(__va(page_phys), KASAN_SHADOW_INIT, PAGE_SIZE); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -139,6 +150,7 @@ asmlinkage void __init kasan_early_init(void) KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_START, PGDIR_SIZE)); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, PGDIR_SIZE)); + kasan_pgd_populate(KASAN_SHADOW_START, KASAN_SHADOW_END, NUMA_NO_NODE, true); } @@ -235,7 +247,7 @@ void __init kasan_init(void) set_pte(&kasan_zero_pte[i], pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO)); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_zero_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index f0c96f08f2f1..e7162ca9d66b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -153,6 +153,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 + void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); @@ -163,4 +165,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_SW_TAGS */ + #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 5f68c93734ba..7134e75447ff 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -473,11 +473,12 @@ int kasan_module_alloc(void *addr, size_t size) ret = __vmalloc_node_range(shadow_size, 1, shadow_start, shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, + GFP_KERNEL, PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, __builtin_return_address(0)); if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); find_vm_area(addr)->flags |= VM_KASAN; kmemleak_ignore(ret); return 0; From patchwork Wed Sep 19 18:54:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606241 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3FCC6CB for ; Wed, 19 Sep 2018 18:55:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E636429178 for ; Wed, 19 Sep 2018 18:55:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D9FBC2AF40; Wed, 19 Sep 2018 18:55:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79EA929178 for ; Wed, 19 Sep 2018 18:55:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2ABC58E000D; Wed, 19 Sep 2018 14:55:17 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 259308E000A; Wed, 19 Sep 2018 14:55:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AD3F8E000D; Wed, 19 Sep 2018 14:55:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 9DC448E000A for ; Wed, 19 Sep 2018 14:55:16 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id d10-v6so6527638wrw.6 for ; Wed, 19 Sep 2018 11:55:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=eRNRheN+h8RsbqafzWvAOJxJ8WiMXtz9C+LId9Frw04=; b=AVKjWbuGZS3dDSK02te3GMC5yL/jcWUXNPNQXk2cHud4WzzLVbaQuAqZ6H5Lhsa9Km A9/z2oxP7AY93QV9kybKX5SeaLwZULz+/U/xeJRKwVehKfl4fIw1DG1ArXx4b5ow9SFT 2CMijHCtTHN3FSP3VBrZIzIaMkjL0E0JXmtoUUQ3kWx3+hlJzCF6ADrvEuM3fnkaleBy Q3E3dfFvcrQuBIRKN2cJeJcET3oRfaBRlb6LYHQIr25qDCfuW02LZFMyATXaM69zjCG/ YSPOZcspx9CzvL8rL+caHXs1iWfZS0lp18hAyTbqyWVY2g1HKITE58j2qyu0WPFTFSUc aKnw== X-Gm-Message-State: APzg51AAbijK4Naqv5hOFMqLKHc2W2CG7g8nX7+hARn685Dg8aW+/AFR 9Wbb7QVD/6CcUlaTjFwjy0twPKVWx3Mp0uy2wcijR/yW4/HDD2O8/B9aM8uJ/3kFWBb6kYCBtl3 pWmylk0c3NsVUajRy9KIH2h9HO4qstBTVYf0s6mONw8ij6ydE6C/39FYE7uCg2mDZT/5vcum4tc 6INXbkrzrd+cH7juy+OkzrLzOXtJr09Piwe5Kgkq5sIEVba4GeKU8I2+YTWaVuBUqTsuBgek1ov 0sQXZGqPn5sAhXgxlgpGNj76OY7qR5uYiIExRUR0DtZ2wtOEdrcharWj8mG5Ii4OKs9CwkgQSOm gG7dwmowOFJ5gHG3+dVdS+YcSSsjRU/8GpelJSsqOKpqO+EGtyxhC6zUefuXABRGf9BnGwK1d5Q Z X-Received: by 2002:a1c:4007:: with SMTP id n7-v6mr22270761wma.108.1537383316111; Wed, 19 Sep 2018 11:55:16 -0700 (PDT) X-Received: by 2002:a1c:4007:: with SMTP id n7-v6mr22270737wma.108.1537383315528; Wed, 19 Sep 2018 11:55:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383315; cv=none; d=google.com; s=arc-20160816; b=yB3R5t4p8Q8aq9kS22tW3CWFjpILKYqTuGV1bkIDs92zf3R0qTeEaQfkJEhv42S9Qa Ig7bw7SU/doG2NU0umgO3igNet6nVPRKXLU+6BLB0s/xF7Vrb5+QUcyt2RXQ0EadbLRl GU95cEG+dW6yvMcD2gKUuUMqts7heqd/Igc5ZPgdOL3XImgZhWWUsMoISjvN38H8zBFa FvhPFCGNR8HpBBhbON1X9lPSmCqdcOX3PIN8FXR9kwnZO8+G29AXoMZ7zKUdsN5Vw2ij Jb1iBkVdhsStCY3Y/aave2fJDGtxL04QKHfog7VKG/wqcuvPlVSX3W4+4nL74rESZRMY 1FPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=eRNRheN+h8RsbqafzWvAOJxJ8WiMXtz9C+LId9Frw04=; b=i0GI3UJkQUdN0NkLdYM9fjagC0D8hYXGVxTSfm1CbQOfIp4q0Sn0Bb4g+uW9RKAr7j J9MgyEF/gFVfDHs7GYhsFeEE5djVpPGQlysSAWFobcmtkyGIWCZzpmge7H7BpL2mNBEf oy5IocfLzLMJT1KyDN3ScsYxbdJIjauM/ghdmpLXTvEoxVznIUDU5tygKg/Pj4HmMwIT x6or6QUnpRcOmO8LdziSA1ochUHH1m/SzkP8WWlWMhMGu4HlH8cP594EZqSCILZrbkv4 PGOwkrMcanpvbskujSAfRQOWDr0eb05gXlyA0QMmuHFWeNsx5vuk4kpG0w70o070KCRo wDKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=MrRk9MtI; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b13-v6sor16452353wrp.25.2018.09.19.11.55.15 for (Google Transport Security); Wed, 19 Sep 2018 11:55:15 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=MrRk9MtI; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eRNRheN+h8RsbqafzWvAOJxJ8WiMXtz9C+LId9Frw04=; b=MrRk9MtIebs4o/HyArY1E0aq5mKRJMMVRS6DtcxAg3aS/ENPF8FI0535IMqycP0G7N V5Nm3fBRY3ldIRCbeZBJ/UjBMdsrFRivnu5nenUO3Fm+HcPzQs8D1rQBDdZnxXjQnCcU ERBoyJqSY7yGkrjermBojUfLDvGzo+ot+PNzPb2ZhIcn8BaTPU5qVkqZWWK5lwKvOyV4 RmPgsVvfQuyKpjSKcuupzoQKLG9yZCRItu+v1p8qU/P4whHPuRsUxQIq+5hv6Q7yAw5u Uoey/mkc0JUL3MDRJUncBOzYqEvPDQriD/mpfthdAI5Jg4bv+Lulf7EuIAgZE6eKvwmu KEiQ== X-Google-Smtp-Source: ANB0VdZXUz3Jw7tnOptVVqdK2gjZKMwpM/ovb5nJajY8SApsIVS/Yof7DtDmQu6n1HGZWPsW76imGg== X-Received: by 2002:adf:f749:: with SMTP id z9-v6mr30759088wrp.85.1537383315077; Wed, 19 Sep 2018 11:55:15 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:14 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 07/20] kasan, arm64: untag address in __kimg_to_phys and _virt_addr_is_linear Date: Wed, 19 Sep 2018 20:54:46 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP __kimg_to_phys (which is used by virt_to_phys) and _virt_addr_is_linear (which is used by virt_addr_valid) assume that the top byte of the address is 0xff, which isn't always the case with tag-based KASAN. This patch resets the tag in those macros. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 0f1e024a951f..3226a0218b0b 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -92,6 +92,15 @@ #define KASAN_THREAD_SHIFT 0 #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_SHIFTED(tag) ((unsigned long)(tag) << 56) +#define KASAN_SET_TAG(addr, tag) (((addr) & ~KASAN_TAG_SHIFTED(0xff)) | \ + KASAN_TAG_SHIFTED(tag)) +#define KASAN_RESET_TAG(addr) KASAN_SET_TAG(addr, 0xff) +#else +#define KASAN_RESET_TAG(addr) addr +#endif + #define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT) /* @@ -232,7 +241,7 @@ static inline unsigned long kaslr_offset(void) #define __is_lm_address(addr) (!!((addr) & BIT(VA_BITS - 1))) #define __lm_to_phys(addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET) -#define __kimg_to_phys(addr) ((addr) - kimage_voffset) +#define __kimg_to_phys(addr) (KASAN_RESET_TAG(addr) - kimage_voffset) #define __virt_to_phys_nodebug(x) ({ \ phys_addr_t __x = (phys_addr_t)(x); \ @@ -308,7 +317,8 @@ static inline void *phys_to_virt(phys_addr_t x) #endif #endif -#define _virt_addr_is_linear(kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) +#define _virt_addr_is_linear(kaddr) (KASAN_RESET_TAG((u64)(kaddr)) >= \ + PAGE_OFFSET) #define virt_addr_valid(kaddr) (_virt_addr_is_linear(kaddr) && \ _virt_addr_valid(kaddr)) From patchwork Wed Sep 19 18:54:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606247 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 355166CB for ; Wed, 19 Sep 2018 18:55:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2287729178 for ; Wed, 19 Sep 2018 18:55:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 15CEE2AF40; Wed, 19 Sep 2018 18:55:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72BB229178 for ; Wed, 19 Sep 2018 18:55:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF5928E000E; Wed, 19 Sep 2018 14:55:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BA6028E000A; Wed, 19 Sep 2018 14:55:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A20B38E000E; Wed, 19 Sep 2018 14:55:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 447638E000A for ; Wed, 19 Sep 2018 14:55:18 -0400 (EDT) Received: by mail-wm1-f70.google.com with SMTP id j129-v6so3844331wmj.3 for ; Wed, 19 Sep 2018 11:55:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MFWmZ2ymGw7SFNCT+sDf2yM6FOtMRHS5/lhwAKyi41M=; b=Tb6KBE/gucLaeCjkUSMI8fOtmcYfCbq3Jk6tzBaEeIjTZ6VtVy7TtbtL0xB3grE8KP 8AoJU3g6T3u+XvbEI5kmKncgoQkcQRd/En6hpnpj1AqQvROErDDWEXIoQfTxW4RXg3If ZjK+pCNVySbDMieWGKkJjX/xc0QV91+UQy4QFFqEZXnmG+J/1kvJuAgfHLEuyrtkEkoE iEFRqF7tgNb6DkwkOd9U7Tsjyq9C4Z3ffYVoTzEh6O0LDjd6Lg9JT00ZQhyQRvKWx2Oq RXgkr6LxawbAch0FxVOD83/sAexBBFd+fVCG7UwUXOA+GddDVYkXuobMvcJE1knxxB7g PBDQ== X-Gm-Message-State: APzg51DcoBguXyRtcHG4aamn97OlULVkTpaKKRSIs46Fq9qk1euKCuCI 6h2zULljqpHkOKnWOGnIqPeDzU36KZlkd1KpleSfg/JKl5BGWDDKMyRWxnPn4HbWxSUOV93p1v8 /K1GrKLPlkXMOOoVNc52alNcoGTq4B9Tbqi0QKV6g/u3FWKuF8HsDTPcP/r8GwhpzKDfFMC+ps6 E11FYXBE2NSvqOnax8goQg4UpMCmNb1aRy3YEcepr1JcsQgNiO3ebMFYJCUxr6JEKDnhmq8DTHl uerQP6EejCJr2FfDQX+STf4ZXpv+r5FXeYuD4q759LNJvQ7dSVrOs5pM43PSi7Ii4+moInm2zWi pNn494/fbb/oynzckzNTZs/3vGZ8GlMJkMG0zdmTWRc1jxlRCaoOsat5XFaXkwnTPBKYe2sZ17N s X-Received: by 2002:a1c:3b56:: with SMTP id i83-v6mr20799963wma.66.1537383317773; Wed, 19 Sep 2018 11:55:17 -0700 (PDT) X-Received: by 2002:a1c:3b56:: with SMTP id i83-v6mr20799934wma.66.1537383317040; Wed, 19 Sep 2018 11:55:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383317; cv=none; d=google.com; s=arc-20160816; b=fJKI39M1Mr/vJ9dinCuoEurbQBQIjw4LccPf08OpMHHEuEetdQ2sKEDL+hWsUmrcWX h1X/XLa9glo5OvblfuayfkFWOr+eRvvSPCBtJ8VjY7m0EzQ32niw5fSnxmNfc27tuTOv tkakbmhFuJGopQpDRlphF54mlmvIv/DqfT/KQbQ5V3/hOqndE7u199B2PYbJTSct20hQ 6FKRp02emlKM5tHdQk0jcB+XIMI3tBFY+WO9klgkj4xVapMCuxDAvIHd88Q60kTC9Dhl MLcc/PUJRVttEGTWbz3EDN8Jrfky3cFETTOk1Lcye+KZulFvm4ocrS9qKom/js/ctqZd 9Rcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=MFWmZ2ymGw7SFNCT+sDf2yM6FOtMRHS5/lhwAKyi41M=; b=S8P8Ru9hO1omFNyeszc1FdLiL8bQDjL6cCzhavUd6sFDJNIO3E+nRZzF257Zljw5ki 9p2aU8B8IXohTeKvoHmSXAogav06zOJQSRfM9GYOu/cQk/Kkwfs5TzKEWVzo9TRF5zKM hY48FwXEfohlUzYK79pEvWzxOAYTKcQ5lxuOzXI7adijmrmcihz0I22hnEeqwHlY7yZT vkf4mxWCV3ov8/YGh6xXi0ccje/f1umrFxOEXOojr9D7wOuHh1cMpKNIg53dFOsPD9k6 HQRWgHzpP4X4hYZDToB1lF49Ogq5faRpvi5fAeTyWc7NRotkjZwO7gpGj8zXAIYfySan YQ8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=IqYYK6SU; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id o133-v6sor10635703wmg.8.2018.09.19.11.55.16 for (Google Transport Security); Wed, 19 Sep 2018 11:55:17 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=IqYYK6SU; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MFWmZ2ymGw7SFNCT+sDf2yM6FOtMRHS5/lhwAKyi41M=; b=IqYYK6SU9pnRTgV15qs62JcLXf/m3ATgrjIeRvDfbSFBDxyovt3Dm4Lg786w5AseU5 Mae0w7RS8HUuPeA7f+uXCK1rM0x4/jv0+9M9dFUdiLqu62wpofAN28FjvVPhPK6chlvJ e45Vw6cbLFf1HeO4sgAnOyP5eiI74zXKJGiOuKPsiGQNlGkI7gdTwl1TygXQGf+I+wM1 a96JjD6bXs2c0ZNGqaIWn7CA75MiAsSV1e66KOMRFmquPJIgKpK9LrCcqz8KQjLDp4Uj 4l8Pi9agIB2MUaOjJcbdQbH+95GpX9EV2HsxGT9AKOGujouVX+qFVKm1V5f40rPIlZXs kNaw== X-Google-Smtp-Source: ANB0VdbYUdT1KRAy58Mt7lu/XViex7l4kcxG8+LuGI0hRyyJeq/zNg/WgbiXvKSO+4udIQquLyu5aQ== X-Received: by 2002:a1c:9290:: with SMTP id u138-v6mr21163466wmd.52.1537383316609; Wed, 19 Sep 2018 11:55:16 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:16 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 08/20] kasan: add tag related helper functions Date: Wed, 19 Sep 2018 20:54:47 +0200 Message-Id: <79532ad7be1a40637e6646eb0ada6195974df503.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset (set to 0xff) the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 13 +++++++++ mm/kasan/kasan.h | 55 ++++++++++++++++++++++++++++++++++++++ mm/kasan/tags.c | 37 +++++++++++++++++++++++++ 4 files changed, 107 insertions(+) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 7a31e8ccbad2..ecd3f25cc323 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -250,6 +250,8 @@ void __init kasan_init(void) memset(kasan_zero_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); + kasan_init_tags(); + /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; pr_info("KernelAddressSanitizer initialized\n"); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index e7162ca9d66b..7e5be87a05b3 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -169,6 +169,19 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #define KASAN_SHADOW_INIT 0xFF +void kasan_init_tags(void); + +void *kasan_reset_tag(const void *addr); + +#else /* CONFIG_KASAN_SW_TAGS */ + +static inline void kasan_init_tags(void) { } + +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)addr; +} + #endif /* CONFIG_KASAN_SW_TAGS */ #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 19b950eaccff..63ab9775275b 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -8,6 +8,10 @@ #define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ +#define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ +#define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ + #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ @@ -126,6 +130,57 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_TAG_SHIFT 56 +#define KASAN_TAG_MASK (0xFFUL << KASAN_TAG_SHIFT) + +u8 random_tag(void); + +static inline void *set_tag(const void *addr, u8 tag) +{ + u64 a = (u64)addr; + + a &= ~KASAN_TAG_MASK; + a |= ((u64)tag << KASAN_TAG_SHIFT); + + return (void *)a; +} + +static inline u8 get_tag(const void *addr) +{ + return (u8)((u64)addr >> KASAN_TAG_SHIFT); +} + +static inline void *reset_tag(const void *addr) +{ + return set_tag(addr, KASAN_TAG_KERNEL); +} + +#else /* CONFIG_KASAN_SW_TAGS */ + +static inline u8 random_tag(void) +{ + return 0; +} + +static inline void *set_tag(const void *addr, u8 tag) +{ + return (void *)addr; +} + +static inline u8 get_tag(const void *addr) +{ + return 0; +} + +static inline void *reset_tag(const void *addr) +{ + return (void *)addr; +} + +#endif /* CONFIG_KASAN_SW_TAGS */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 04194923c543..700323946867 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -38,6 +38,43 @@ #include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void kasan_init_tags(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(prng_state, cpu) = get_random_u32(); +} + +/* + * If a preemption happens between this_cpu_read and this_cpu_write, the only + * side effect is that we'll give a few allocated in different contexts objects + * the same tag. Since tag-based KASAN is meant to be used a probabilistic + * bug-detection debug feature, this doesn’t have significant negative impact. + * + * Ideally the tags use strong randomness to prevent any attempts to predict + * them during explicit exploit attempts. But strong randomness is expensive, + * and we did an intentional trade-off to use a PRNG. This non-atomic RMW + * sequence has in fact positive effect, since interrupts that randomly skew + * PRNG at unpredictable points do only good. + */ +u8 random_tag(void) +{ + u32 state = this_cpu_read(prng_state); + + state = 1664525 * state + 1013904223; + this_cpu_write(prng_state, state); + + return (u8)(state % (KASAN_TAG_MAX + 1)); +} + +void *kasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { From patchwork Wed Sep 19 18:54:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606251 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9C2BD6CB for ; Wed, 19 Sep 2018 18:55:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8ED2029178 for ; Wed, 19 Sep 2018 18:55:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 822652AF40; Wed, 19 Sep 2018 18:55:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 101B629178 for ; Wed, 19 Sep 2018 18:55:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BD5A8E000F; Wed, 19 Sep 2018 14:55:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 347AD8E000A; Wed, 19 Sep 2018 14:55:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 173968E000F; Wed, 19 Sep 2018 14:55:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by kanga.kvack.org (Postfix) with ESMTP id ABB4F8E000A for ; Wed, 19 Sep 2018 14:55:19 -0400 (EDT) Received: by mail-wm1-f71.google.com with SMTP id v1-v6so3851550wmh.4 for ; Wed, 19 Sep 2018 11:55:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tRTggFtQ5Xof0bdpE6trnqe/KQMT5YATFM7eSCvBAOE=; b=ehb75zMEmNnuhGVG9P/rZPynAq2FCDeWChY77XKwcCu/4DwGZ9LtLDVBYrJQfGDOdy MfPLrsopBFWoIvYCgjVtuansGb52Pj9ljNDJXJdZ/8nfuP1oB6igMQ/C9xdsRR8qIZyy gkqvtKJaNqQGNulSSVBCfJilKD0Fd+7DiOQVMoF1M0kfQS4Zjne6jydIDbLr+Y7z9Hck NGZRPO9TdBUCN/qolxYss7sF9YRTPGRNYS+L9i99iUOUWlZ9urwmYlBuki5WyvhE4qLb LeN1M7DZ3zAPZp0vutDvBrxuPlvHVNPUFp5jjLVdzhHedSJ377PDPZQCaIm14K/QwEpT bvBw== X-Gm-Message-State: APzg51BZT/tOO1fcHhm4J6HzKkMk/jhKvbBa+wKdDATxdUZzaNQcAAP0 vLddXtGoUTmLuZBEEGE5qytpjDbyBut864qvf8oWh+qWW4u/N8IfYEfKaPfkGoXBlZTTLt2RAk2 6G33KUBG0O30K/Qt61X34X1ZLBfntmUSegC1Ddg75PFHNtGGk4Vb3PcHczmJrilCcvTywoFaB4u UN9kVlrwMa0XjzdJyJDwEjTN9mPZLgAY6aL4fUUuqC5G+RTb50urshIgMx0uY/JkfzQloOoPIEH nV2evoi7nXsVanM8Rh/zIRUvIGKWV3XOrDdyfPrNTcUqkJwEtMKwXaegZ9bAGb0kxurIkLP5/gw t6jMdZFIdGSNtYYnoB1me6mL6L/5jo781Q1mkTJUWuoGWgoiOFPLL0YRQ8xLubrAapjR4zC0JPL i X-Received: by 2002:adf:c454:: with SMTP id a20-v6mr29173277wrg.20.1537383319208; Wed, 19 Sep 2018 11:55:19 -0700 (PDT) X-Received: by 2002:adf:c454:: with SMTP id a20-v6mr29173241wrg.20.1537383318519; Wed, 19 Sep 2018 11:55:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383318; cv=none; d=google.com; s=arc-20160816; b=IMH5rajrG1yTKzWKIr7oW2RgjCHNaXWMaSXAnMr5QlyLCQIRqYpFvzgoJ1DhP1przA x0LfN9reqOWQdmLmemBA0ynV9nOmgkzrSOw5HYaa5Cy1OjlIGlLtkb7/941IUUuYo9ur 6dD23ADMrFb1c3q9ajB5Gb2rb+Z/VW+WG+Y1oKtqQfjscoCXW5o/0heGmh7DH3P1Seqt lJQ9OQhi1vsvUovNqFO9Z3OwtOdPgnnafDrlSmblapuaCz5K3gKBBTuWbPt6F2btRF2Q BV3Ql190PUvBSCi8ceVISy3UfR/jdmRW66Ra36Y6c5PnN90IOlqOxIBh3e+Du3c71+HF EdKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=tRTggFtQ5Xof0bdpE6trnqe/KQMT5YATFM7eSCvBAOE=; b=lbF2GHSfUpxpRfV7jfpK0UFCqRSKJZpVkQWdeOvnd2iKqYein7GdF8aMKqH8qJKhqQ Ctog7q2KY+yn75b3bTlvpem5yy0oihJ2krCTDIVuFkZxGi5jxXEZzw/XMoWTEO2ZopWF SquSTIEBf2Ntu3cWXQGrgMX4l3X2CjN8MWIb/xdZoWbpZknc/BAReGFlH1i/MLAIweQY sRK6KUlkGKmWYH/9JIcWIUKMpTJKJLhrHNT43dnK9Gw/QPtBf75zEsnBlLq8GdH2zd7j WlcDN+YP0Bhb7zxoK4zlvIwyM67nE5T4qmJCl8i0IH1Zlj3yaoNgfGrZpuGYJ/Je+2qT z6Qg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="b/oxOrN6"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id w14-v6sor16161120wrr.11.2018.09.19.11.55.18 for (Google Transport Security); Wed, 19 Sep 2018 11:55:18 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="b/oxOrN6"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tRTggFtQ5Xof0bdpE6trnqe/KQMT5YATFM7eSCvBAOE=; b=b/oxOrN65ba3FkcfZ2rFk/U//pgLsJhOJqSgx6/bVZO7dAe8XpR+r9MKsZ+EhAMzd6 F8j0gHTQORVFzJ/G8j3QHlxTriBz4n6iOR9sEGqx4ykMuyqHm6wD2otk1fbcciHVRcx9 TF/WhauK2uM9LGI7Zf0HPXltws79YU1D9IHHnMbkDEqdyCcGmeKL8LKnhpU3qgBAiOOQ z3yAKeCgdq5hSYtWPyvE2e818NqwPNnSUeTDxgJcDkdSIG/aOh4l1Uxq4P0B1GLXqn6i 8sgmAvE/YejHR7OKDCAHBn1FsK3JC7WYYsCr++MTr6qDjp0Am/TaPUBEz2pxALKJtV46 TstQ== X-Google-Smtp-Source: ANB0VdYyUSbjWAVxz/tZw9sJ0Nn/OHR0lGdwfdGP/aqrt1Zlm7xgeFQykgyrZTq/UUXdEolG+C4GSw== X-Received: by 2002:a5d:63c4:: with SMTP id c4-v6mr5404693wrw.106.1537383318103; Wed, 19 Sep 2018 11:55:18 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:17 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 09/20] kasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU Date: Wed, 19 Sep 2018 20:54:48 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor. The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it. Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed. Signed-off-by: Andrey Konovalov --- mm/slab.c | 2 +- mm/slub.c | 24 ++++++++++++++---------- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index 6fdca9ec2ea4..fe0ddf08aa2c 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2574,7 +2574,7 @@ static void cache_init_objs(struct kmem_cache *cachep, for (i = 0; i < cachep->num; i++) { objp = index_to_obj(cachep, page, i); - kasan_init_slab_obj(cachep, objp); + objp = kasan_init_slab_obj(cachep, objp); /* constructor could break poison info */ if (DEBUG == 0 && cachep->ctor) { diff --git a/mm/slub.c b/mm/slub.c index c4d5f4442ff1..75fc76e42a1e 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1413,16 +1413,17 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, #endif } -static void setup_object(struct kmem_cache *s, struct page *page, +static void *setup_object(struct kmem_cache *s, struct page *page, void *object) { setup_object_debug(s, page, object); - kasan_init_slab_obj(s, object); + object = kasan_init_slab_obj(s, object); if (unlikely(s->ctor)) { kasan_unpoison_object_data(s, object); s->ctor(object); kasan_poison_object_data(s, object); } + return object; } /* @@ -1530,16 +1531,16 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page) /* First entry is used as the base of the freelist */ cur = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + cur = setup_object(s, page, cur); page->freelist = cur; for (idx = 1; idx < page->objects; idx++) { - setup_object(s, page, cur); next = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + next = setup_object(s, page, next); set_freepointer(s, cur, next); cur = next; } - setup_object(s, page, cur); set_freepointer(s, cur, NULL); return true; @@ -1561,7 +1562,7 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) struct page *page; struct kmem_cache_order_objects oo = s->oo; gfp_t alloc_gfp; - void *start, *p; + void *start, *p, *next; int idx, order; bool shuffle; @@ -1613,13 +1614,16 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) if (!shuffle) { for_each_object_idx(p, idx, s, start, page->objects) { - setup_object(s, page, p); - if (likely(idx < page->objects)) - set_freepointer(s, p, p + s->size); - else + if (likely(idx < page->objects)) { + next = p + s->size; + next = setup_object(s, page, next); + set_freepointer(s, p, next); + } else set_freepointer(s, p, NULL); } - page->freelist = fixup_red_left(s, start); + start = fixup_red_left(s, start); + start = setup_object(s, page, start); + page->freelist = start; } page->inuse = page->objects; From patchwork Wed Sep 19 18:54:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606255 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84FB21390 for ; Wed, 19 Sep 2018 18:55:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 762D229178 for ; Wed, 19 Sep 2018 18:55:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 688E22AF40; Wed, 19 Sep 2018 18:55:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEB8429178 for ; Wed, 19 Sep 2018 18:55:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9622A8E0010; Wed, 19 Sep 2018 14:55:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 90E248E000A; Wed, 19 Sep 2018 14:55:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 718438E0010; Wed, 19 Sep 2018 14:55:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 0DDDF8E000A for ; Wed, 19 Sep 2018 14:55:21 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id j22-v6so6521374wre.7 for ; Wed, 19 Sep 2018 11:55:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PIceIrvWyef1xlEqXhKzbMfNSt9Di8ecFmfot9dbvKM=; b=EffTGQQJb+NvqjP0nmCGa9C8RnqF34ezQr6LN1azGkX7OH25DT6NYt5CQedpX6l4RC 9Cx3JvCqgacdL76popw+8bbpxVa7JwTflnfS0gvRDi/WNSQMy98YJNUC9u4lwd14Yqiz z/GFZJ3GRvgLGt/L7nlyU4Pk/TQirNfUD5JoTjdEvCxzGTkI4XoIw0HlTp0fmHqOvNGh 3m6/hwDp1tLhH4MilwEMO/D++12oaQwuekM6lBrVAdz8J+ox79YKZPWxIJqTchY09fsw 1Drvm7AP1AQ0d6KzlKfXrFLX6CFDwlRXL0CcmZWQBLse/P7l2mIfn4D9ATTK0zteqoGP eCxg== X-Gm-Message-State: APzg51B9NAyfFvG82ErhEq4Mcld7t8hFA9wbsr9hd7wRQ2uzcb15XH0B MFs2RQUm8shquQ40uPuZEY3Cin38QxgvHFliaT6Rs3gN4sR6R+nkhxzdlAlq2rlqb9i0Rpxltjv nJH1NxxXwkdt7Ue+m6KShAfGUzyy2OT2yWZ52lpOgAlsIE2XPJqjTEMLt9JFOUnbzNS59Lnf97x aWY0B2hPBMOeJFriX9rBGOVfJ+zZwCEtFbhUhqzYzVKZiRbU9mvCpF0AQUF5oduNk7e0+7TpA/d 8AV9kqbKqMNeKgCaQdeNEL9YPx78fvkbIDqKkIlf8O3lV0UOk7pcK/0qleK43nmnP52oWkBrAsb gdyCEDWd5azVPJtPKfMllSjQPxiCabMQ8XVm/0YC99K8uXHBDF5Gr00BMhHrAj6rc/OlA/dVjpx d X-Received: by 2002:adf:eb87:: with SMTP id t7-v6mr31247357wrn.123.1537383320504; Wed, 19 Sep 2018 11:55:20 -0700 (PDT) X-Received: by 2002:adf:eb87:: with SMTP id t7-v6mr31247329wrn.123.1537383319965; Wed, 19 Sep 2018 11:55:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383319; cv=none; d=google.com; s=arc-20160816; b=iWnx/BZtzLfxwdRVK6ebPBthJ4KkHNa8sgyxj6DiyDXNJ6hxaZb2q1RVVUx7U98lg4 L1FoJkhsL4ckpUZWjQ8XL2SheBiCA5tKtwzM6G4zKwTfIBFKr2heTkoLnzsvLcb79qYT ktem0936iL9rnJEZhbRLjnqQ+KIy5lnNOV3xF08cGWwOZsPDJ4p9eZ0VC9N5olwFjeeo NI92FTjcO37DXotaVZD0wkDuNquCao9ECHcmHMmgLDb1M9Jn0CsxVO0zD+dmaHg0rDe5 /2LJy5J1/xUYqdmVjl1G1u5t9l2594SGcHhE6o58Qm74VeDaUcFwaUKpJau1JfFJ64Tk sjTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=PIceIrvWyef1xlEqXhKzbMfNSt9Di8ecFmfot9dbvKM=; b=F5hAZZtDsv3SG6A/QNUY3Sx88EJQiL4rri10WDATE7Qsn+oL28qe9BBsUFhoOh1ZDx gBLP5JHiHhl/JZDDbnDBziY+PD0jWYP9y1rCpbwjlR/PY73bdRcSCs72GeSlOlryjORZ jMMUzHjDQZo/j6/Qq6kOwlGTYQpYkim7lAalSZOt8zhku9HWtt7MnM8U/Tby3jolvMLn 6yblonK51R347tCiZ+AA+fUa0TxuIUEtRM7Ipz7gluz26yqz2BubKpdksieMi7/lZj8Z ZziLquu8S2/U8Y2vjUzo86wKc2PV4WtGKFelDgPsejzFF57Ub86hPc+sBvsejHD/xh5z Gm3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Vnc101Zm; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y33-v6sor16313175wrd.44.2018.09.19.11.55.19 for (Google Transport Security); Wed, 19 Sep 2018 11:55:19 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Vnc101Zm; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PIceIrvWyef1xlEqXhKzbMfNSt9Di8ecFmfot9dbvKM=; b=Vnc101Zm5F6hDUo3GCgeWj0T1ajKKdHv+fSoJ5cPZmtrZy+76KwfSvlBYifVYbaBm4 aT52eLOZvNN3mOoRXhXDmsrE0FSKlQbc4NUYXZZltyVU+gNWEuXYnfdqFBaon3aBbFbz Rrf7heXVQrKNnZ993RTjCusodK9ahYJmmjxFNSO2m66AWNUoiGrqZKiWA6DXeRqoSmP0 CRNQNV56bk1ptsTjko1IcufN+aJH4xQ+qzmSUXjXQIk/T9e0w4InsMgesDaVxDZtzsPy AtZx87+m6lGcPD+Yo0NCHIOHCqBTdNKWhpQcGE6W24gyD9Pm+JggCEBYU8leqhBj+gue DptA== X-Google-Smtp-Source: ANB0VdYOeki63tDIAVwp8ieNxxHhTyhrB+nzOntZG8kNpeZXd21UUF5kmU3s+AZ5Ilpiw1n4VWV5Bg== X-Received: by 2002:a5d:5248:: with SMTP id p8-v6mr30297536wrv.198.1537383319545; Wed, 19 Sep 2018 11:55:19 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:18 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 10/20] mm: move obj_to_index to include/linux/slab_def.h Date: Wed, 19 Sep 2018 20:54:49 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP While with SLUB we can actually preassign tags for caches with contructors and store them in pointers in the freelist, SLAB doesn't allow that since the freelist is stored as an array of indexes, so there are no pointers to store the tags. Instead we compute the tag twice, once when a slab is created before calling the constructor and then again each time when an object is allocated with kmalloc. Tag is computed simply by taking the lowest byte of the index that corresponds to the object. However in kasan_kmalloc we only have access to the objects pointer, so we need a way to find out which index this object corresponds to. This patch moves obj_to_index from slab.c to include/linux/slab_def.h to be reused by KASAN. Signed-off-by: Andrey Konovalov --- include/linux/slab_def.h | 13 +++++++++++++ mm/slab.c | 13 ------------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/include/linux/slab_def.h b/include/linux/slab_def.h index 3485c58cfd1c..9a5eafb7145b 100644 --- a/include/linux/slab_def.h +++ b/include/linux/slab_def.h @@ -104,4 +104,17 @@ static inline void *nearest_obj(struct kmem_cache *cache, struct page *page, return object; } +/* + * We want to avoid an expensive divide : (offset / cache->size) + * Using the fact that size is a constant for a particular cache, + * we can replace (offset / cache->size) by + * reciprocal_divide(offset, cache->reciprocal_buffer_size) + */ +static inline unsigned int obj_to_index(const struct kmem_cache *cache, + const struct page *page, void *obj) +{ + u32 offset = (obj - page->s_mem); + return reciprocal_divide(offset, cache->reciprocal_buffer_size); +} + #endif /* _LINUX_SLAB_DEF_H */ diff --git a/mm/slab.c b/mm/slab.c index fe0ddf08aa2c..6d8de7630944 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -406,19 +406,6 @@ static inline void *index_to_obj(struct kmem_cache *cache, struct page *page, return page->s_mem + cache->size * idx; } -/* - * We want to avoid an expensive divide : (offset / cache->size) - * Using the fact that size is a constant for a particular cache, - * we can replace (offset / cache->size) by - * reciprocal_divide(offset, cache->reciprocal_buffer_size) - */ -static inline unsigned int obj_to_index(const struct kmem_cache *cache, - const struct page *page, void *obj) -{ - u32 offset = (obj - page->s_mem); - return reciprocal_divide(offset, cache->reciprocal_buffer_size); -} - #define BOOT_CPUCACHE_ENTRIES 1 /* internal cache of cache description objs */ static struct kmem_cache kmem_cache_boot = { From patchwork Wed Sep 19 18:54:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606259 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92E0C6CB for ; Wed, 19 Sep 2018 18:56:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 861732B5EA for ; Wed, 19 Sep 2018 18:56:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 79C372B840; Wed, 19 Sep 2018 18:56:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 18DE82B5EA for ; Wed, 19 Sep 2018 18:56:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D4D28E0011; Wed, 19 Sep 2018 14:55:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 084B28E000A; Wed, 19 Sep 2018 14:55:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8F258E0011; Wed, 19 Sep 2018 14:55:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 89DDF8E000A for ; Wed, 19 Sep 2018 14:55:22 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id y32-v6so6636841wrd.19 for ; Wed, 19 Sep 2018 11:55:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xtJgUAWOXZXX0dAHUptbPqpenfy7NgkmLyignEvLyTk=; b=r7c3awza1YRUks/zfGFEjFMvt2p+qJ2luB2d0ndd9nGmQusWqtBT7Kf6BMw0KsbfSO JIMJqUo1AJG2tQ6a4KvnzlHiKzUlW13qG4Q3mIKcb3Cx2kcxYC9027fXYTx21VTDvWO6 ykxraxKLICjnY4KzN2U+Rg7o9/VhHEWsqp/bGOWjE6CXLNbT6NpU5V9sSpVHtwezqM9e y5L1XXKReT7vhLSsHCoW+HI0efnrCIG3MbEMX47GzGBUd+ZAoW214sMxgKQoGVklgEdt bHv0vwFaXE6mrmeGyoADJHNYjCCB5syuR2GjdreAe8ckoWylDG/6LaEYisqh9ha5G1Fz 7JVA== X-Gm-Message-State: APzg51C8UNOuF1omNjTc5JGt7XQy9d/hl13NbEWYsehhktfXpJCtYDj2 tCT17073srrzvc5+Grj127dqKF43KW6tpQfMT/EevVARTh+o2PsFFx4uX3Hljuu7507D7UE7nmN CGHXVwVstCjWIwBNFNi0qxPBJyYEntHfCdhKWAVvWBIISJ8VWJn0shYyekFtY8TWUNOX0HNX8jJ 6vyugwJ11dLKUopksDVxEXyxlz6xh1pXimzeIbp35x0vou1+91DZwI8/II7Rmr1YCLb8LrKti1J /mjPWujEuLSbxBKN5A3svCoiB9mDvJmWmC+k+9WlWXw1fLMl6qMvbWEuFLJKr5V3wQtC7yNb/Ew TgKBiBFaoKrsmZUkooSplrk2gfJJJAnMa7B9oiMRFV99s6cO5RfdbpEICs5H7hQ80IfQN0Zqnjk U X-Received: by 2002:a1c:9d02:: with SMTP id g2-v6mr21123696wme.122.1537383322097; Wed, 19 Sep 2018 11:55:22 -0700 (PDT) X-Received: by 2002:a1c:9d02:: with SMTP id g2-v6mr21123676wme.122.1537383321611; Wed, 19 Sep 2018 11:55:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383321; cv=none; d=google.com; s=arc-20160816; b=aM8I4E31d5HMBL/d4LBFf+Ou9gLebM4ysIxaDB9gq4+zpUiStK+sB17bUogSNFlzWn X9g5NO9nob2oBN/c0DdztwgkBYsvEqX22XllAlDHMO+XFxt2bP3khLVBvyQlF/aWw5J4 SB23mtlOzIFSbAerOmjq0+iERuw+DZUUPZmutDC31usciWSgceOIz3HD8+DdMwUsAXB/ WkdfjiO86ocfBH0SQ9wdsFuRAlwnLEXC4S2sPanmoekk0Po/MIJDYA5YhWY629+5jCbV dh+9sUMK1qgrkYCDJuGRN/UU87FdYtVdiiGVzz5Ft+ygQ3sZVYxnMRnfGEMOUq8iJ93x PLYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=xtJgUAWOXZXX0dAHUptbPqpenfy7NgkmLyignEvLyTk=; b=lq4lkrRh/czQpipUqvZYaMQDu3ip11AuBR2AabJP85OTxwAiOJT92/FqpTaKVQpFuu 2/j0R4a+p9zF/0tD7Y2FrkKLda8x2SzrTkb1N1qJFPwvuImYMZFrgMOGQGX+fDCbEb39 fGS+OZ5IVfZMPjVdROvwk2ofDnPaYdPVc+m3XIhXQbEZGplvwISFlospxLzK/9d+hEJK q7XjtVqm9gReHy1yrboGwxkKzVLlOQvXzlQP06w2uann8e47+zAib6rpY2stCPx6W8Tw v+6EInX7v6ZwYKJmQAR4g2tKwYch6PcLlRXR+2XTRlop8M28pZzxVptHciaC+SwOMP3H 41Zw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=vPuyihbl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 69-v6sor16557783wra.43.2018.09.19.11.55.21 for (Google Transport Security); Wed, 19 Sep 2018 11:55:21 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=vPuyihbl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xtJgUAWOXZXX0dAHUptbPqpenfy7NgkmLyignEvLyTk=; b=vPuyihblvxcn0sLFem62r23wIGYYYskYXdS/f0UduHxfmXZcYhZa+1inXCT+EVC71b xR9esBpfVNjAw5O1xd94Y0Ta9SiVI5dJ17z93VEQu3KYeWrfq33uQUDvbvmcGV6iZTUb b+9TkA37Wz+GEuxDBV724TGKdy1T52jCVQWAgTPE79jxFsVOoaSYOixroijjzYzATXJ1 jj4LdW4Md3upKR14f85br611JPh1UrMa8LADi2b3KZ1rAmXkL7gTDaLh1qx5/UUIf96h Ykig492trKfPPYXL59tNXdfV/Ll/v7d93Di9WJRRygDSUcHJLfMTsoWN2l6fYWF3kink jUjw== X-Google-Smtp-Source: ANB0VdblUI7MFFp8hCfvF9UePgtJJWb/kUMM90uaKcYDdSH4MK9TlKqZCLxx5yQlpghBFlT3+9tF4A== X-Received: by 2002:adf:9464:: with SMTP id 91-v6mr30486547wrq.231.1537383321264; Wed, 19 Sep 2018 11:55:21 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:20 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 11/20] kasan, arm64: fix up fault handling logic Date: Wed, 19 Sep 2018 20:54:50 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP show_pte in arm64 fault handling relies on the fact that the top byte of a kernel pointer is 0xff, which isn't always the case with tag-based KASAN. This patch resets the top byte in show_pte. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/fault.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 50b30ff30de4..78328c864d01 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -134,6 +135,8 @@ void show_pte(unsigned long addr) pgd_t *pgdp; pgd_t pgd; + addr = (unsigned long)kasan_reset_tag((void *)addr); + if (addr < TASK_SIZE) { /* TTBR0 */ mm = current->active_mm; From patchwork Wed Sep 19 18:54:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606267 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 20E0C1390 for ; Wed, 19 Sep 2018 18:56:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 13CD129178 for ; Wed, 19 Sep 2018 18:56:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 063FF2AF40; Wed, 19 Sep 2018 18:56:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DA9429178 for ; Wed, 19 Sep 2018 18:56:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5C1D8E0012; Wed, 19 Sep 2018 14:55:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AE3EB8E000A; Wed, 19 Sep 2018 14:55:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 936BC8E0012; Wed, 19 Sep 2018 14:55:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id 375848E000A for ; Wed, 19 Sep 2018 14:55:24 -0400 (EDT) Received: by mail-wm1-f72.google.com with SMTP id v24-v6so3979867wmh.5 for ; Wed, 19 Sep 2018 11:55:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8pJXKvzLB+e1tJ1Hv9TztYhic7mAKsr7ltM5Jvr+GUA=; b=ICZw1RObaun2xXLdWx47NlYmrYlk+0fzEETzliIOBCcOexJFvDoOPsCYwRfEY359I1 IbnWXTmYio79d9Kf02JhEFuLYMBDq4Thv/LD08ZV8OO1XjYuFIOWC3nPIiqKHtO85zGV 53ayurd9zndNKYtJxm97HrOAlY0WzwuJPpA1Genn/3jagpqIvvVCPdYlTgH5WL7GVwQ/ 5Mpc3SzFMrFNaDVmSWmWTm8l0jZzNfsDyBnEj+Z8BSsPKl97gS8m4YCl9SlBdY10m7Np BPuXe9izoKvnNDA6AK8GN1Cux+kFy1UZdLZvghgR/aEfk0dLWGC5ioyubjE7ll0yUPF2 a9jg== X-Gm-Message-State: APzg51AEH7ljmMNPsFHE9ePNktZgTJHf5nDJp0puBVm2lP7onMsVUI6e 5Q9kDu/PrBrOX3/Yc8sqenpAn/REhpmz1Pauhx44UPxfYG6pa4wCFjCSxawMdlw4UlCxmEFFR1k 7XLpHGKhJOOLkHsz1byyxlD0mx3NBy1Gh5kIaVLXTM2dUaZaqrer5WaM8JFoi4/FlSwkbiHVHQy zEl2s5EKvSvEX4CH9vsQh4PLIgdHI4AlM4dR/tZMw+zWZ8XpV350RBKWwC9QZYitq6ymqg60WDl cbXhVLAaMAOBodq7kWHH8odF7ZXJBJ68yRrBC8d6iE1gf8yl2+i13oqAG0LUSmLnQMNSePj9T7g 9LtUfpLHP6rwbtNxqgtaXuzAf2rv1svd5cAOG6k79qt5P/29f/def2nrmEDgs1yv3j3EyQOgIQx + X-Received: by 2002:adf:f391:: with SMTP id m17-v6mr30739166wro.279.1537383323742; Wed, 19 Sep 2018 11:55:23 -0700 (PDT) X-Received: by 2002:adf:f391:: with SMTP id m17-v6mr30739143wro.279.1537383323072; Wed, 19 Sep 2018 11:55:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383323; cv=none; d=google.com; s=arc-20160816; b=Q9z5DFXMovqkJCs6simK0bH2l8ZDugM5WRkOR08fV9tCo8yzBgA6szItSIJF5hX9yA /hAgDXDEO46JY7Fq7zBmxHc2Z/ZF+c0NPy4cpsFmUDoLqbruD7wawY+bXdSo15Gdx119 6Ss28EgkEYQaYL3QNrm7O1M4Af2RMpFhw8n5v9lpLHwZpw0jJzHk4BZuTHOGOsggRQDx x/PXGllu1vaZkOi+Q4iEhlU3hydByq1CkKlQwtRmkTpxfwby1H4Qqz1EBr3ljRp4+nrA r+GBO7ZL/e3eGare8zwCBp72TaoHEaJllwmD+4Wmq0nNsoLQYCQEiyAcNMiP91G2N3gf O1rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=8pJXKvzLB+e1tJ1Hv9TztYhic7mAKsr7ltM5Jvr+GUA=; b=ko1fYu3j02NLG/WhYvJAOpUmD4HBSMA7NvI4U67VQ8XSFtJcpL+TpM8Sw2hx55mcAm KX8kpUy465aKqvxo6G3JwvywrRIJtQ7lYQERzf7ElyZd74fNBaFe773+3e5Ln+4I2aSz z+VbQdUvXymOmjmwfxn+HJBu1inGGv44HNt9BBliXHZMxLhoMMmbTvlRSqiH6rR4fd3x J0QxgXM2EpTuODt/LLw3CxwFxv5usiSj8QhTbgpf8rso5znjsHxO2SqbE8tB1RkFbKQD tHfeYYvbPF6SlFVNFPrRuTFWgIZWfFPd/3btPYwjK4AKdmLnTItNmN+hf/juTez+Hg0k v+Dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RSGVy6Ay; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n15-v6sor16126290wrm.3.2018.09.19.11.55.23 for (Google Transport Security); Wed, 19 Sep 2018 11:55:23 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RSGVy6Ay; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8pJXKvzLB+e1tJ1Hv9TztYhic7mAKsr7ltM5Jvr+GUA=; b=RSGVy6Ay3KrzIUqWwv/VIcAW0HO7cVr2nUs+V4EzIDJMC+eescAuwUx6jPg+y7ZRSC CwpxO4q7mN7dtW4XG1FH0/6ZaNnKHzY4eI40IwAiTdZaZAOxQYZWoPxBRqdh3qxp6w08 O+OIPMrqSfiA3DBaWK9GkyWxW6H9iRZ40oYPtlYJs6RzPjkYGChmE0xIjlkBhtYZoatS HU7Ce6ZcPbDxVzM7c+yOONxmzw5KhqXZs5MrExSkGOSUvLtluU3adnmErD6bRNYhiMKp ROSOV7iN4cRnsKYvKAmTG/Wn1PgLR7cpYISOMt7scoyppezouNaPS5jVwbLy3y+ch2BU Zwmw== X-Google-Smtp-Source: ANB0VdZNjLwlO7+Wx7HL+D4ZNaVYjHeUF1fVI5wO9gQ8YjF80pGa7gqec/EbQRlsPW+CRevVCuOBdg== X-Received: by 2002:a5d:4185:: with SMTP id m5-v6mr5167615wrp.8.1537383322708; Wed, 19 Sep 2018 11:55:22 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:22 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 12/20] kasan, arm64: enable top byte ignore for the kernel Date: Wed, 19 Sep 2018 20:54:51 +0200 Message-Id: <5e610a513c44a69b2b78476c20f21960696259c6.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when tag-based KASAN is used. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index fd208eac9f2a..483aceedad76 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -289,6 +289,7 @@ #define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 03646e6a2ef4..b2b44dbdb063 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_SW_TAGS +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /* @@ -440,7 +446,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /* From patchwork Wed Sep 19 18:54:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606273 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B54B7161F for ; Wed, 19 Sep 2018 18:56:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A608629178 for ; Wed, 19 Sep 2018 18:56:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9975D2B5EA; Wed, 19 Sep 2018 18:56:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3AEA42AF40 for ; Wed, 19 Sep 2018 18:56:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 455B08E0013; Wed, 19 Sep 2018 14:55:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3E07B8E000A; Wed, 19 Sep 2018 14:55:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 258FF8E0013; Wed, 19 Sep 2018 14:55:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by kanga.kvack.org (Postfix) with ESMTP id BE0748E000A for ; Wed, 19 Sep 2018 14:55:25 -0400 (EDT) Received: by mail-wm1-f72.google.com with SMTP id x64-v6so5571564wmf.1 for ; Wed, 19 Sep 2018 11:55:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d1iMb39p1prK7niLwUfLvs3mbBI1WfSY9Tra+BtzAxQ=; b=rpxazf2Do27ePSIGGwecx2rGiMsCRLABW3CbfZN+4JmyKHYmYDWceis4gZZ3xoieBI gqRGPnzUYJco6VXTi6I2W8wD4cJ4SL0EnlWNjtDb+3x4UsetIAD8GQLA0TFRqwQXdrHI rgueg8S/9mfrh9ILgyfqIBOX0siq4TP2QeO3Ct3eQ3vBrGTZ1q+OECNhEbF/eephTNzD pQnhg67grx3b3AGVTrH0V15w1uhFZxJpoPNurMz8Q+6CjtGwXwi250wpcoNG6As8kOmy DWRuqjFk/I/74+La1XQLilmFb/yVyT4AptkW6TW4mUbWFJW9iCKKvp8sSqfbw1YZCTJw LV0g== X-Gm-Message-State: APzg51BAChkNTxN9QicXF/J2fGj76CiVx9A5s+c9ixTeNMGqUKWxrgOh r4oWnEIsOgK4JywdT/ODFwiWPOxkZNxsXXvOSVgfV6EhuXZI1CmbK0EyOVQ9tZ2SO5j0a2kV5bP mf307eIawSgdE0X0yEeKDzqucOlPeRaTFiTwRIJuVp3g5cRQ7A5w83CvdSnNjDeWePMDoz/+Lhx O4YhUjCWcH4fqyu5xBD8q8g4IWY+aAUen72tkzHN7j376v8y8Ye3wmvHgAx5Pa8zRNBPu77JC0n kUNO4TSHDakB3bdg9icz0jFU08HZZoBgffHpS2yauoKS+4EswIfnGrk/2NnREZGJZKF3aRUkVwY M+kgYj7oulqStSJqoaVGGpiIMDuFPCRExuuuYy8K4KrboOvNfJeqrEYIGQFOe2y59QksGEyEJFp A X-Received: by 2002:adf:d20a:: with SMTP id g10-v6mr30875413wri.66.1537383325311; Wed, 19 Sep 2018 11:55:25 -0700 (PDT) X-Received: by 2002:adf:d20a:: with SMTP id g10-v6mr30875387wri.66.1537383324821; Wed, 19 Sep 2018 11:55:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383324; cv=none; d=google.com; s=arc-20160816; b=LAcKbkqKMD+QQfISG4bkED8r4K+aBHnLNxBPhu+sGlenP0CmfsVeM7KoqTdd3mLcIU wWtbsVMcRL77+kNgi7xE/EuI4+O5mKeTrUM27jNDtCX7/vtc8K0rOdytobhdlyiL4OS0 JP+iXgBIL7KJ/JHKIDDA+NhjW+83DZI864UWWT8Xc8uev3PcAK5aXAD2MFGq5V8gUGjD VVGOMo0KzXR6B/8UyPhgCcqbL+pdOSp6j/MRZ9n7CcFxFstG6qxWFiXVRfVGT/4dXMag BA3eBmBzamigyop5EkeQJLKGyUr3PjzRa+WV772toKVt8ZskJvveuZx5gE+wncfcFej+ aoRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=d1iMb39p1prK7niLwUfLvs3mbBI1WfSY9Tra+BtzAxQ=; b=wuPWMqM/ieITuRUJAitHNdfkpTINoEgXe8M133ksNWhdz8iqA/CEjGB3CoLQKcT4VO eU3KJESfGB/TGcqlAfiRHBrdSAoCbKbBeiDMKxslUN94B+KhumEzr76Tp9cUm4pE0AL5 VWvIWhptFnoCXQ8qONTnwqWStvl8RfFoukG0M0Jf1LmN2OpvsvN0BUO0dKnFgw2mLpqj R9V8GS8XjchF26YIh5taISCDY67JCDIXsrUc649U1FYd56JKuiaYD57E5FS5wMA5Oiar dblS6DQRbewXOl2aIsNsmJ8l8cX0tl8zc+cQp3VBRaAEte0ZTx6eOejrex2swFOabKLa VuHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=op++z0BS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id w14-v6sor16161308wrr.11.2018.09.19.11.55.24 for (Google Transport Security); Wed, 19 Sep 2018 11:55:24 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=op++z0BS; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=d1iMb39p1prK7niLwUfLvs3mbBI1WfSY9Tra+BtzAxQ=; b=op++z0BSZ/nwOadd89i0fho9uQZiZsdFy7N0GRCwHUqVvYMaWvDvVmYOPZbERPOPFx ad23W/MQBbvV3/GY7+tuqP2Of9HHtQ5OTJHZQdRaqHLIIauiwl5p5TPXuZcuLja8f+lg Gmg1pZr0oyuUyf5jFx3yO97RBema8RQDjlge5Vi/p9u6ZVJyeu/Q/b8sZe3wIn46oBTL 72/Sm5BUPocqr3OOPUjJ579q43JT4k5eYih4kkSQdo03+dbtwIDYkx5ncimrVvHA8k4o PG200V3W+tgEQO46uTpx3RAKoZvVLrCxJ5EK6vEUn1RxqZyP4JQ2BP9MlWsfD2UEcjEy JKhw== X-Google-Smtp-Source: ANB0Vdb5o820qjCsQHYsVlC6o405PmMuMtbuK7P7pWLgWe20tanCx9P4Y5oo2MG9QevgZjPd50wONQ== X-Received: by 2002:a05:6000:10d0:: with SMTP id b16mr30374902wrx.226.1537383324464; Wed, 19 Sep 2018 11:55:24 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:23 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 13/20] kasan, mm: perform untagged pointers comparison in krealloc Date: Wed, 19 Sep 2018 20:54:52 +0200 Message-Id: <286fcda449120b643e4665fc9848e81260a1300c.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. Tag-based KASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not. Signed-off-by: Andrey Konovalov --- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 3abfa0f86118..221c1be3f45f 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1513,7 +1513,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret = __do_krealloc(p, new_size, flags); - if (ret && p != ret) + if (ret && kasan_reset_tag(p) != kasan_reset_tag(ret)) kfree(p); return ret; From patchwork Wed Sep 19 18:54:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606277 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 470796CB for ; Wed, 19 Sep 2018 18:56:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38E5A291B2 for ; Wed, 19 Sep 2018 18:56:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2BCF02B840; Wed, 19 Sep 2018 18:56:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0A37B291B2 for ; Wed, 19 Sep 2018 18:56:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F03D78E0015; Wed, 19 Sep 2018 14:55:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EB5098E000A; Wed, 19 Sep 2018 14:55:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D553E8E0015; Wed, 19 Sep 2018 14:55:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 62EC78E000A for ; Wed, 19 Sep 2018 14:55:28 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id k44-v6so6511256wre.21 for ; Wed, 19 Sep 2018 11:55:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ooKbreW281i8q2O08S3WXdFmOC6TcmBZskYa6xxctq0=; b=JQKbIbfa3M5+e2YTP/AmqdlBHXAGlYiD5ohtgcTDVHK/0Yy6HH73SogVcMwsAFbFv+ 5RaqOQJz4qB80em7352JoPmuz3lzFoK3RYSeExHPkX7FC/Ghia18WhOAnWPHfh8XK9GW RGtAeJ+9eJ/97lSmDzNaCiKlIJQ9wmvaiEAr0u6E0mqwlnxPnwn8nkimtVS5IHf0wbIX 36DiPGyyCGMQ1YAqZGbJxCBpODUz8NUy6rumtBQ8FeEFdF1DI2keQI3ktOb1Yf3qwyv3 S4UYP2peUwuphjEBT4sUiJV4WiPJ138q1C9LfCE5g4833ILjuqBpw2ZVwSg8Nis5NGkj bvhA== X-Gm-Message-State: APzg51DmCeqE89Yma/nBjN0iBB7vAq+ZDEOnHXN9eFJp/KVa9udkSEzw wpw2dZjqaO3pyYZeb3MAjgZHIQ7Rk5Eucr/HU/WmrN11BVgD/Qe3ZcXp5PzG/hIH4Cts0WoKvkE wPLMNAmy3frqZa8d2X1QE2daAl688hBRAY0MF0kkihQCvCkYTV14XuQRvyjJ5DGpqgqkSoAFbo8 DHa0+62srlL2FUvZ9L5CBwbZwcOeilhOgOOISkwIwN+hckqexmI2YmsXTqnGC10AxvwMHIqDZD5 9j+3djn5nl34qt3ASAier58gsUsfZfsJ2FWO3AmFSt4ModLu8O6MZI898rOxdlGOTca/kSLqmVG 4ezOvjafYVwidlTZPWfST02q+zIw+moJcSlXosg20hVvuGPU3fWoxy68M0TB5jC0Xh6xDlolsDQ 5 X-Received: by 2002:a7b:c086:: with SMTP id r6-v6mr19803194wmh.119.1537383327888; Wed, 19 Sep 2018 11:55:27 -0700 (PDT) X-Received: by 2002:a7b:c086:: with SMTP id r6-v6mr19803154wmh.119.1537383326741; Wed, 19 Sep 2018 11:55:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383326; cv=none; d=google.com; s=arc-20160816; b=TX5FqE0+udDizBUipgkHDDBnEgIBgrcmAepRfgY3rfAu3Dqlqd2lDNvXKj93F4LDkj ksEUaEFp9QZ2M2Im8gt86dcOdV43i7pv0AlqPCR20w3PgDoAeDNHVFil55cBiILl/yV1 h2eE82vM2f65Z2tIgQd87VMLQluv8z+gAaThfV2J4Ew5eQCoooYzFOgYWzqRyzbdaURJ oqZ9zZErvLMaOXBd/1h+XqJ3NwZwJRQvMYyYdaecf10lloHJ9gzydwCrp/YVZ61Jo4CU YMCXUZc1JIG/ixXifr6kt98k0k+fqpSMn0+M/1O3R3QbE0GtHJOjLTC8CFv2cDjdPqLU 2WDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=ooKbreW281i8q2O08S3WXdFmOC6TcmBZskYa6xxctq0=; b=DAdF1gBzRcdZ7pIkLIVNwz4w188r0RRFv6Bl2hXoRXvPRBdNr7vC7k9lVZhL+2wyHK ijGYYmhrKBSmV5yivfjxlG2eDZ0hAGVM/UhoYG3+ttcXbtzMwZdSXKrUuAOHpA0x1h5q a7bPiaCvggSZYckQKI9C1XvuR20RJuJgCnWqXLZtQRzCCcPqUosPc7AitYKKl4Hva1fY JOuAlyJPD0rSbUIQXATDJuzbRWIN8g07LYorL3HgTQitPo+qroH4CIhlB/GDqykZu7WR gCMhrn09xQ0hjumuodY/f8Vkbqqzu50z9Yr/0M0Lzhge1f56Jdww8JIdUY5i9ZVomxuq ZFKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="MC/27Wv7"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id z134-v6sor10598814wmc.23.2018.09.19.11.55.26 for (Google Transport Security); Wed, 19 Sep 2018 11:55:26 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="MC/27Wv7"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ooKbreW281i8q2O08S3WXdFmOC6TcmBZskYa6xxctq0=; b=MC/27Wv7sLnwRje//iTz59buN9HYWsXrrVJsVBZaZQdnmGCxFOPJXsSdlR11M0GXmf aE5yEXl1vItnZu+XR6DolN972lXyCAUy1jcTIRtBTP6x2Xu5S+qfklx7DoiPMEddvxvS g9nhvBroMajtW92uQlv5J5v7pCSDW/ZwCngfMzo9h6EYjbprspqWYOHaSnJNxD1dwxgx a0KsdYOhHKXwTthMoG0yp5utbh9f7WQLbai0wZetnfianOrh5+g7tvxRknf/9i/zO3+5 1NZSKyCslp/yTqCtLv2mhJH5hU/+xoRH83yVz7bZmuEr+t4frsS+IJPmqHcYXwApvZgV wFZw== X-Google-Smtp-Source: ANB0Vdbi4uUKmQ++xYJhD928omUjGd8IfG1r6//2RELp8h2YQ/Rzxy9z+u5oKNd0rS0UJhP+U7p55Q== X-Received: by 2002:a1c:a8cf:: with SMTP id r198-v6mr21707575wme.133.1537383326095; Wed, 19 Sep 2018 11:55:26 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:25 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 14/20] kasan: split out generic_report.c from report.c Date: Wed, 19 Sep 2018 20:54:53 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch moves generic KASAN specific error reporting routines to generic_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by tag-based KASAN. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 4 +- mm/kasan/generic_report.c | 158 +++++++++++++++++++++++++ mm/kasan/kasan.h | 7 ++ mm/kasan/report.c | 234 +++++++++----------------------------- mm/kasan/tags_report.c | 39 +++++++ 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/generic_report.c create mode 100644 mm/kasan/tags_report.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 68ba1822f003..0a14fcff70ed 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -14,5 +14,5 @@ CFLAGS_generic.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_tags.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += generic.o quarantine.o -obj-$(CONFIG_KASAN_SW_TAGS) += tags.o +obj-$(CONFIG_KASAN_GENERIC) += generic.o generic_report.o quarantine.o +obj-$(CONFIG_KASAN_SW_TAGS) += tags.o tags_report.o diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c new file mode 100644 index 000000000000..5201d1770700 --- /dev/null +++ b/mm/kasan/generic_report.c @@ -0,0 +1,158 @@ +/* + * This file contains generic KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +static const void *find_first_bad_addr(const void *addr, size_t size) +{ + u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); + const void *first_bad_addr = addr; + + while (!shadow_val && first_bad_addr < addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +static const char *get_shadow_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + u8 *shadow_addr; + + info->first_bad_addr = find_first_bad_addr(info->access_addr, + info->access_size); + + shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); + + /* + * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * at the next shadow byte to determine the type of the bad access. + */ + if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + shadow_addr++; + + switch (*shadow_addr) { + case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + /* + * In theory it's still possible to see these shadow values + * due to a data race in the kernel code. + */ + bug_type = "out-of-bounds"; + break; + case KASAN_PAGE_REDZONE: + case KASAN_KMALLOC_REDZONE: + bug_type = "slab-out-of-bounds"; + break; + case KASAN_GLOBAL_REDZONE: + bug_type = "global-out-of-bounds"; + break; + case KASAN_STACK_LEFT: + case KASAN_STACK_MID: + case KASAN_STACK_RIGHT: + case KASAN_STACK_PARTIAL: + bug_type = "stack-out-of-bounds"; + break; + case KASAN_FREE_PAGE: + case KASAN_KMALLOC_FREE: + bug_type = "use-after-free"; + break; + case KASAN_USE_AFTER_SCOPE: + bug_type = "use-after-scope"; + break; + case KASAN_ALLOCA_LEFT: + case KASAN_ALLOCA_RIGHT: + bug_type = "alloca-out-of-bounds"; + break; + } + + return bug_type; +} + +static const char *get_wild_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + + if ((unsigned long)info->access_addr < PAGE_SIZE) + bug_type = "null-ptr-deref"; + else if ((unsigned long)info->access_addr < TASK_SIZE) + bug_type = "user-memory-access"; + else + bug_type = "wild-memory-access"; + + return bug_type; +} + +const char *get_bug_type(struct kasan_access_info *info) +{ + if (addr_has_shadow(info->access_addr)) + return get_shadow_bug_type(info); + return get_wild_bug_type(info); +} + +#define DEFINE_ASAN_REPORT_LOAD(size) \ +void __asan_report_load##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, false, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_load##size##_noabort) + +#define DEFINE_ASAN_REPORT_STORE(size) \ +void __asan_report_store##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, true, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_store##size##_noabort) + +DEFINE_ASAN_REPORT_LOAD(1); +DEFINE_ASAN_REPORT_LOAD(2); +DEFINE_ASAN_REPORT_LOAD(4); +DEFINE_ASAN_REPORT_LOAD(8); +DEFINE_ASAN_REPORT_LOAD(16); +DEFINE_ASAN_REPORT_STORE(1); +DEFINE_ASAN_REPORT_STORE(2); +DEFINE_ASAN_REPORT_STORE(4); +DEFINE_ASAN_REPORT_STORE(8); +DEFINE_ASAN_REPORT_STORE(16); + +void __asan_report_load_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_load_n_noabort); + +void __asan_report_store_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 63ab9775275b..31efc4ee3ddd 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow(const void *addr) +{ + return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +} + void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +const char *get_bug_type(struct kasan_access_info *info); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 5c169aa688fd..64a74f334c45 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,5 +1,5 @@ /* - * This file contains error reporting code. + * This file contains common generic and tag-based KASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -39,103 +39,34 @@ #define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) #define SHADOW_ROWS_AROUND_ADDR 2 -static const void *find_first_bad_addr(const void *addr, size_t size) -{ - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; - - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; -} +static unsigned long kasan_flags; -static bool addr_has_shadow(struct kasan_access_info *info) -{ - return (info->access_addr >= - kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -} +#define KASAN_BIT_REPORTED 0 +#define KASAN_BIT_MULTI_SHOT 1 -static const char *get_shadow_bug_type(struct kasan_access_info *info) +bool kasan_save_enable_multi_shot(void) { - const char *bug_type = "unknown-crash"; - u8 *shadow_addr; - - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); - - /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look - * at the next shadow byte to determine the type of the bad access. - */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) - shadow_addr++; - - switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: - /* - * In theory it's still possible to see these shadow values - * due to a data race in the kernel code. - */ - bug_type = "out-of-bounds"; - break; - case KASAN_PAGE_REDZONE: - case KASAN_KMALLOC_REDZONE: - bug_type = "slab-out-of-bounds"; - break; - case KASAN_GLOBAL_REDZONE: - bug_type = "global-out-of-bounds"; - break; - case KASAN_STACK_LEFT: - case KASAN_STACK_MID: - case KASAN_STACK_RIGHT: - case KASAN_STACK_PARTIAL: - bug_type = "stack-out-of-bounds"; - break; - case KASAN_FREE_PAGE: - case KASAN_KMALLOC_FREE: - bug_type = "use-after-free"; - break; - case KASAN_USE_AFTER_SCOPE: - bug_type = "use-after-scope"; - break; - case KASAN_ALLOCA_LEFT: - case KASAN_ALLOCA_RIGHT: - bug_type = "alloca-out-of-bounds"; - break; - } - - return bug_type; + return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); -static const char *get_wild_bug_type(struct kasan_access_info *info) +void kasan_restore_multi_shot(bool enabled) { - const char *bug_type = "unknown-crash"; - - if ((unsigned long)info->access_addr < PAGE_SIZE) - bug_type = "null-ptr-deref"; - else if ((unsigned long)info->access_addr < TASK_SIZE) - bug_type = "user-memory-access"; - else - bug_type = "wild-memory-access"; - - return bug_type; + if (!enabled) + clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); -static const char *get_bug_type(struct kasan_access_info *info) +static int __init kasan_set_multi_shot(char *str) { - if (addr_has_shadow(info)) - return get_shadow_bug_type(info); - return get_wild_bug_type(info); + set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); + return 1; } +__setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info) +static void print_error_description(struct kasan_access_info *info, + const char *bug_type) { - const char *bug_type = get_bug_type(info); - pr_err("BUG: KASAN: %s in %pS\n", bug_type, (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", @@ -143,25 +74,9 @@ static void print_error_description(struct kasan_access_info *info) info->access_addr, current->comm, task_pid_nr(current)); } -static inline bool kernel_or_module_addr(const void *addr) -{ - if (addr >= (void *)_stext && addr < (void *)_end) - return true; - if (is_module_address((unsigned long)addr)) - return true; - return false; -} - -static inline bool init_task_stack_addr(const void *addr) -{ - return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); -} - static DEFINE_SPINLOCK(report_lock); -static void kasan_start_report(unsigned long *flags) +static void start_report(unsigned long *flags) { /* * Make sure we don't end up in loop. @@ -171,7 +86,7 @@ static void kasan_start_report(unsigned long *flags) pr_err("==================================================================\n"); } -static void kasan_end_report(unsigned long *flags) +static void end_report(unsigned long *flags) { pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); @@ -249,6 +164,22 @@ static void describe_object(struct kmem_cache *cache, void *object, describe_object_addr(cache, object, addr); } +static inline bool kernel_or_module_addr(const void *addr) +{ + if (addr >= (void *)_stext && addr < (void *)_end) + return true; + if (is_module_address((unsigned long)addr)) + return true; + return false; +} + +static inline bool init_task_stack_addr(const void *addr) +{ + return addr >= (void *)&init_thread_union.stack && + (addr <= (void *)&init_thread_union.stack + + sizeof(init_thread_union.stack)); +} + static void print_address_description(void *addr) { struct page *page = addr_to_page(addr); @@ -326,29 +257,38 @@ static void print_shadow_for_address(const void *addr) } } +static bool report_enabled(void) +{ + if (current->kasan_depth) + return false; + if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + return true; + return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); +} + void kasan_report_invalid_free(void *object, unsigned long ip) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); pr_err("\n"); print_address_description(object); pr_err("\n"); print_shadow_for_address(object); - kasan_end_report(&flags); + end_report(&flags); } static void kasan_report_error(struct kasan_access_info *info) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); - print_error_description(info); + print_error_description(info, get_bug_type(info)); pr_err("\n"); - if (!addr_has_shadow(info)) { + if (!addr_has_shadow(info->access_addr)) { dump_stack(); } else { print_address_description((void *)info->access_addr); @@ -356,41 +296,7 @@ static void kasan_report_error(struct kasan_access_info *info) print_shadow_for_address(info->first_bad_addr); } - kasan_end_report(&flags); -} - -static unsigned long kasan_flags; - -#define KASAN_BIT_REPORTED 0 -#define KASAN_BIT_MULTI_SHOT 1 - -bool kasan_save_enable_multi_shot(void) -{ - return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); - -void kasan_restore_multi_shot(bool enabled) -{ - if (!enabled) - clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); - -static int __init kasan_set_multi_shot(char *str) -{ - set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); - return 1; -} -__setup("kasan_multi_shot", kasan_set_multi_shot); - -static inline bool kasan_report_enabled(void) -{ - if (current->kasan_depth) - return false; - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) - return true; - return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); + end_report(&flags); } void kasan_report(unsigned long addr, size_t size, @@ -398,7 +304,7 @@ void kasan_report(unsigned long addr, size_t size, { struct kasan_access_info info; - if (likely(!kasan_report_enabled())) + if (likely(!report_enabled())) return; disable_trace_on_warning(); @@ -411,41 +317,3 @@ void kasan_report(unsigned long addr, size_t size, kasan_report_error(&info); } - - -#define DEFINE_ASAN_REPORT_LOAD(size) \ -void __asan_report_load##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, false, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_load##size##_noabort) - -#define DEFINE_ASAN_REPORT_STORE(size) \ -void __asan_report_store##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, true, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_store##size##_noabort) - -DEFINE_ASAN_REPORT_LOAD(1); -DEFINE_ASAN_REPORT_LOAD(2); -DEFINE_ASAN_REPORT_LOAD(4); -DEFINE_ASAN_REPORT_LOAD(8); -DEFINE_ASAN_REPORT_LOAD(16); -DEFINE_ASAN_REPORT_STORE(1); -DEFINE_ASAN_REPORT_STORE(2); -DEFINE_ASAN_REPORT_STORE(4); -DEFINE_ASAN_REPORT_STORE(8); -DEFINE_ASAN_REPORT_STORE(16); - -void __asan_report_load_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_load_n_noabort); - -void __asan_report_store_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c new file mode 100644 index 000000000000..8af15e87d3bc --- /dev/null +++ b/mm/kasan/tags_report.c @@ -0,0 +1,39 @@ +/* + * This file contains tag-based KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} From patchwork Wed Sep 19 18:54:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606285 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AED06CB for ; Wed, 19 Sep 2018 18:56:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E17F291B2 for ; Wed, 19 Sep 2018 18:56:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 617462B840; Wed, 19 Sep 2018 18:56:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C89DC291B2 for ; Wed, 19 Sep 2018 18:56:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD7148E0014; Wed, 19 Sep 2018 14:55:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B361B8E000A; Wed, 19 Sep 2018 14:55:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 985C98E0014; Wed, 19 Sep 2018 14:55:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 38D2B8E000A for ; Wed, 19 Sep 2018 14:55:29 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id s18-v6so3238510wrw.22 for ; Wed, 19 Sep 2018 11:55:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Jw2K2wpnoqMaSJ8uFP7eR1+4k/u8wFMFa7zpsX+OS5g=; b=KCvBLG6B54OK7w4NEuQZB4L5bv9HB3MIMb2MpYKEM5tHr10XcypktMOLkWyunrk1on m6P/0NT21z+50J9B+Je4HsBf2PRbFsokiDrQeEDF4d6uLf6vPIPmoPfixrClaP06g7D5 XQmQbwH2k0iCEsPuO7Y5d/7SFc+xJBuuTo90YwEex31Nah6UhPQWiTZQ0W2L2Ki8zHuC REeIEmm2yA3D1zOGVdBJtILDuBD4uSUZvqSxCWeHQeUMjUZad7S+cQFYSBYJsj8Nc5u9 fYDLF3pBfbILAP8lLLBCkS2XMI2o8I3OSPdiyg5SLUATN/ykAr89+d0iCzndANys3wG2 BmAg== X-Gm-Message-State: APzg51CKHemV//54RAGjioRmiCme2ZQSsSv5zgPmiSO7U7F8pL3XRFzY Gj6/J3gfprD9cqAvatXC77981UjsxtsYpbpyur8pLWSQPtDy37T9DG3PsvmprDcSrOqUnBugGH0 lKvSoVfqNAaIKCLG/VA9fi7ERCqNVR0VsfzbiPvItYzHyagSvI8Xun75AnyflzoB5598L62ATLO ViYMstcG4e483LTWbKzsw7pdiIMY12uCD/lOABtaaw1M/kUWFyzfuGzJoAESyPj1tBK1NTmg2nZ swADjMRBmY8t3K0YUwork/M0HFMYUAJUcE6aBmUTFh8l2APIXfdyjB0EJIwi80vXxOxZEcOy9Da +R+e5HSDyQUA1Zbo1ZXihYGDju0ImO+qBDmHFWOWSC6W0opIZeXBvuoxXIjvaPF7aOOAinLMKDC X X-Received: by 2002:a5d:5685:: with SMTP id f5-v6mr30832364wrv.58.1537383328745; Wed, 19 Sep 2018 11:55:28 -0700 (PDT) X-Received: by 2002:a5d:5685:: with SMTP id f5-v6mr30832340wrv.58.1537383328053; Wed, 19 Sep 2018 11:55:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383328; cv=none; d=google.com; s=arc-20160816; b=WnJ9vFhy0eaeborFX2vlarqeYc6+dND4KcKvR60k4fqHhhC248d3jpnejvsPuOkHvt zZGpfkD0W9qjgH4hND5RqXGQqAiEs0zkQJdViwYqYX9M/dFrWjIBEAr9OBL3uxprB67i u7Ibf5aKrESDMm3o5AhAMfQThTZo8aUsrTQFaC70qygXkjx6A/pSCdhwouG5fEB7ZC7n DJQ8YXFL6g18tay6OqwvRk73OL2SIKIjQv7gCt0Fn0vjGZB5ES1Z5D1AFDWjt1nI8JAK 2LxUlhj8kZyA14irIgA3Lc9FZv8VgA5W76mNEKCdac2W/NA1kWFi1tk46i3r8eVKZoXB ahug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=Jw2K2wpnoqMaSJ8uFP7eR1+4k/u8wFMFa7zpsX+OS5g=; b=azDR9sYfCD4JWc6PqkCO0BiO2vrSlfxOa0MKXFd3EC5Ggbw0y8aciH68jKfcEi8scF 4ckmHbjovLCjuPbt3Z4UEWGRxp5toxmQCeOv4c39nTV1LtE0kugKAg4kjNHz7Bn7cRUL 7JcPFH1ASIt53fjB2zPBfQ7bMnKHjvBpPVarBqtj3toSwIWNGuL4FyF8hgLGX4hb/pVh iZ4jb+mcRAe0hc8gkGqkrT9aJ8Vm7/5zPVXVsujo+lhGhqFDUmjmFoUnY8Kxfd7g6Enj wrrvWghq5+TEGPuKLiiQKQzmM9W4DlY57vcRyQumDtbuAM0yFQbJqUM6fIslEd4paPky 9fgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=t7EHud3X; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y31-v6sor16632711wrd.50.2018.09.19.11.55.28 for (Google Transport Security); Wed, 19 Sep 2018 11:55:28 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=t7EHud3X; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Jw2K2wpnoqMaSJ8uFP7eR1+4k/u8wFMFa7zpsX+OS5g=; b=t7EHud3XKEEXHDP922lt6sPMiqiq9nM2wcysZV32bKKA1SE+nz61EfGYmywA1dXGit 9WZxK/jaky0AAHf/Q6UOKXR5Kur6KkoFTcmsu+txVR4FQcO9l/Ee4ZKGyG4SVKgKZ8Pr gYbk8tgvInAzpi+VcB79/1ciBId2DICtjZ+aYyMoPKjrOYnoNBiixC93kom5kXKCYhEe 0JMJ//F5JrWOkDp90btk9Y/UYEEPbE6tKBEF/1CfqofBG+B3HyRtIgsNurICxD8aSZfZ s5GmYCRmAzHs56Vzv0hzONIay3tzghXLGoIPgwqN5VVUl3mfq9UBAe02fcgir1cBUSJD QWCw== X-Google-Smtp-Source: ANB0VdaWHDHKIKf2dppTPO9LjYtesdjVa9cdxTmnTRNARXTenV2GUf/zFzuR7wLZwBB2m8drdztEog== X-Received: by 2002:a5d:4089:: with SMTP id o9-v6mr30434186wrp.133.1537383327612; Wed, 19 Sep 2018 11:55:27 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:27 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 15/20] kasan: add bug reporting routines for tag-based mode Date: Wed, 19 Sep 2018 20:54:54 +0200 Message-Id: <3da501714f11f558ccb2f3e9ca1fa32ebe69b3c1.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds rountines, that print tag-based KASAN error reports. Those are quite similar to generic KASAN, the difference is: 1. The way tag-based KASAN finds the first bad shadow cell (with a mismatching tag). Tag-based KASAN compares memory tags from the shadow memory to the pointer tag. 2. Tag-based KASAN reports all bugs with the "KASAN: invalid-access" header. Also simplify generic KASAN find_first_bad_addr. Signed-off-by: Andrey Konovalov --- mm/kasan/generic_report.c | 16 ++++------- mm/kasan/kasan.h | 5 ++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ mm/kasan/tags_report.c | 18 +++++++++++++ 4 files changed, 59 insertions(+), 37 deletions(-) diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index 5201d1770700..a4604cceae59 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -33,16 +33,13 @@ #include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr(const void *addr, size_t size) +void *find_first_bad_addr(void *addr, size_t size) { - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; + void *p = addr; - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; + while (p < addr + size && !(*(u8 *)kasan_mem_to_shadow(p))) + p += KASAN_SHADOW_SCALE_SIZE; + return p; } static const char *get_shadow_bug_type(struct kasan_access_info *info) @@ -50,9 +47,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info) const char *bug_type = "unknown-crash"; u8 *shadow_addr; - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 31efc4ee3ddd..a2533b890248 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -119,6 +119,7 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void kasan_report(unsigned long addr, size_t size, @@ -139,6 +140,8 @@ static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #ifdef CONFIG_KASAN_SW_TAGS +void print_tags(u8 addr_tag, const void *addr); + #define KASAN_TAG_SHIFT 56 #define KASAN_TAG_MASK (0xFFUL << KASAN_TAG_SHIFT) @@ -166,6 +169,8 @@ static inline void *reset_tag(const void *addr) #else /* CONFIG_KASAN_SW_TAGS */ +static inline void print_tags(u8 addr_tag, const void *addr) { } + static inline u8 random_tag(void) { return 0; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 64a74f334c45..214d85035f99 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) } __setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info, - const char *bug_type) +static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - bug_type, (void *)info->ip); + get_bug_type(info), (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, info->access_addr, current->comm, task_pid_nr(current)); @@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip) start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); + print_tags(get_tag(object), reset_tag(object)); + object = reset_tag(object); pr_err("\n"); print_address_description(object); pr_err("\n"); @@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags); } -static void kasan_report_error(struct kasan_access_info *info) -{ - unsigned long flags; - - start_report(&flags); - - print_error_description(info, get_bug_type(info)); - pr_err("\n"); - - if (!addr_has_shadow(info->access_addr)) { - dump_stack(); - } else { - print_address_description((void *)info->access_addr); - pr_err("\n"); - print_shadow_for_address(info->first_bad_addr); - } - - end_report(&flags); -} - void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { struct kasan_access_info info; + void *tagged_addr; + void *untagged_addr; + unsigned long flags; if (likely(!report_enabled())) return; disable_trace_on_warning(); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr = reset_tag(tagged_addr); + + info.access_addr = tagged_addr; + if (addr_has_shadow(untagged_addr)) + info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; - kasan_report_error(&info); + start_report(&flags); + + print_error_description(&info); + if (addr_has_shadow(untagged_addr)) + print_tags(get_tag(tagged_addr), info.first_bad_addr); + pr_err("\n"); + + if (addr_has_shadow(untagged_addr)) { + print_address_description(untagged_addr); + pr_err("\n"); + print_shadow_for_address(info.first_bad_addr); + } else { + dump_stack(); + } + + end_report(&flags); } diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 8af15e87d3bc..573c51d20d09 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -37,3 +37,21 @@ const char *get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } + +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *p = reset_tag(addr); + void *end = p + size; + + while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) + p += KASAN_SHADOW_SCALE_SIZE; + return p; +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); +} From patchwork Wed Sep 19 18:54:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606287 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 903FC1390 for ; Wed, 19 Sep 2018 18:56:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82200291B2 for ; Wed, 19 Sep 2018 18:56:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7532A2B840; Wed, 19 Sep 2018 18:56:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A15D291B2 for ; Wed, 19 Sep 2018 18:56:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E8478E0016; Wed, 19 Sep 2018 14:55:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1750C8E000A; Wed, 19 Sep 2018 14:55:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E66BC8E0016; Wed, 19 Sep 2018 14:55:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by kanga.kvack.org (Postfix) with ESMTP id 71D278E000A for ; Wed, 19 Sep 2018 14:55:31 -0400 (EDT) Received: by mail-wm1-f70.google.com with SMTP id r14-v6so3974913wmh.0 for ; Wed, 19 Sep 2018 11:55:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=iYNjgVaE7A2cOoLGxcvbQWfG/q900XUOt7RdX7lhCCU=; b=jPLU4xIEw9uFj5vuuHtMQNeAAar+kwMCdQ/QPG2DEgM0aiOfM0QxXh8Ygsv+ytEQvv yawNwwf0+WI/s95leY/YW8TtnDpV3jJ3gJ1xuQHfgznAC34uqdv3RDDWo4qs/MGl+ky0 UJpR/0q5af9ifXl/c78bq+/TljEaLTsR+cTdqDw9THWkD3JQeYN7BKsoDuD6IRRv/l7z z6EmDaCnEgrYy4ed8Okie4ZDCQAC9zuEHza+6o3fPVIqXtwFZDiCa2iSMswmMmklmtKN zGCXGvUsFyt/YI24mrmcVDhlPbXqEAmEQ+BOKvJPE3r0/3ipi8M3XXWf9iDaO2H5+Wyc fNQg== X-Gm-Message-State: APzg51CB6d82uHSj2hsAtGtKeJpHqmR38SWMnYmOoeaVnAd7oB5cSMhk ADm2V7PcsYS+WM8cdEscgFDzmbZWMQNpafOHIGrGLsWTQwimip5XMOsSccpX+GYnxilTfwFZOTL OY4uG0eOrxt73xoXuu23JqkoifYV9OGLnWLnqvJIGNZXA6YyddVpORkZXDcctJA1QSITgbo20fQ KZ1vu/s3tdl9EBlf5Lhzhq9AbymPkshIsfcmROCurNcfv7rmYPKnyGsZ0rpMz+dsXNllzhcuj2z 37ANyge71v22ZTto0QkzS6b/oOR81/AlYFknH1sLzUp0Ql6iur8TS8DN91GzxMFjde9kr5jZ7ez IctDaOfxMHpY7fSkytsJCy8YPLq8dN95j7GDQZOs9aL3pYx+/AM4EHpH0SFS+4hzofEbkEqiBpF A X-Received: by 2002:a5d:63c4:: with SMTP id c4-v6mr5405218wrw.106.1537383330938; Wed, 19 Sep 2018 11:55:30 -0700 (PDT) X-Received: by 2002:a5d:63c4:: with SMTP id c4-v6mr5405175wrw.106.1537383329768; Wed, 19 Sep 2018 11:55:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383329; cv=none; d=google.com; s=arc-20160816; b=Q5LpuzoslL8NS3zBDaWWzFSpkKPQR3PaLcHH9vQT4kYnVoGmMZDwyR0dp5Z7xwxxXN BzfIgVzdHTLRdcZ78jy0GobbJ8QAf7ZhbpINoV1T2G3RpeEghmyjNJsIfLXO22rx5m8S YJe3bl9LqVwiUZ1z/+MfEO/T9fy045vkbRvWgiosMiLp9MlFN4l7KQ+hY6WQD+tGS3Ux 3rOCfLHQkEAeZcwKw+t7ZhlYagRO2Ee9f5OINPLF7gEsL+v+BXYFjJ2VuUYOCC+uFX4d NSsFA4Cr6AUKYkYux07bYu/YjTzEj+yZhWmI5yA8jqKYoUNCK2/B/Szow4SsBCPlIZX/ /68A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=iYNjgVaE7A2cOoLGxcvbQWfG/q900XUOt7RdX7lhCCU=; b=X9QAxhfTjjPqHRFkiKrSIBCRG28Un565VuW869ljwNnXh1dPFoqIuR6M9eGJ4WvPP1 MUZdYoI9lbwV7WnF7tu1kVL4pxPzmysAIa/ec67R39IJPK0UaUatW5W49EJJKcoECOO7 tBigoRyI3aHjm5tmGtxFfUJ00oxEa+VOqeVTXLzLE6lRCpdLMTQFIpjSdeYp0q+kHvlB JfAZ27H4JXisRl1wW2llB2HZMMc2LxASbn+WIF6tCXkXTL+ItgDSgYV61+Jt3DJ8G0Y3 QDHOyxqSBh/MouVqQW1MzU/Trovw5VF16zyMx7kxL855GVeOYwKkw3KE1pnYaDH7RLyP vvtw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=T01NHfFl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i8-v6sor6708872wrs.28.2018.09.19.11.55.29 for (Google Transport Security); Wed, 19 Sep 2018 11:55:29 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=T01NHfFl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iYNjgVaE7A2cOoLGxcvbQWfG/q900XUOt7RdX7lhCCU=; b=T01NHfFlVkQZHHDgnxHieD4LDft4LK+JzNa0vrAX1ojeO2AYK1kRyI67/V4+YMyt3b /+OBQJ6d8ax74Bnu0czJ8xVJRMrx+Gz4u1AlDGlYGxUUTfFTBoxF8GQc70NV+7wZcSix s4tnlXkqQKvEMCiCSZFuaDzIJTQfQsn5yS79mFwUvjOVimKUKIp+o3Z+xthXwN09Emww KXArHznN2EbbNcWQzGi4ayzkKBJiE/TtCqRccn1Wb6K+OJ/5Epz+uwMvOzKB3eHl1SXc nGpIUV9TXN3UMcC5oui3WivR9V1YpTATlbKoovdXtpu07LnujTZkyg88dbCol6OzRjLm Gtkg== X-Google-Smtp-Source: ANB0VdbL37ih7gC3E61AjUqZEYA5SsJw2v5gqQdb6Pt7RN7phYrmGsbE4QzbCzQPUwN4LFBqwg7d5w== X-Received: by 2002:a5d:54cb:: with SMTP id x11-v6mr31452787wrv.150.1537383329271; Wed, 19 Sep 2018 11:55:29 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:28 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 16/20] kasan: add hooks implementation for tag-based mode Date: Wed, 19 Sep 2018 20:54:55 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds tag-based KASAN specific hooks implementation and adjusts common generic and tag-based KASAN ones. 1. When a new slab cache is created, tag-based KASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16). 2. On each kmalloc tag-based KASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer. 3. On each kfree tag-based KASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs. The rest of the logic of the hook implementation is very much similar to the one provided by generic KASAN. Tag-based KASAN saves allocation and free stack metadata to the slab object the same way generic KASAN does. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 118 ++++++++++++++++++++++++++++++++++++++-------- mm/kasan/kasan.h | 8 ++++ mm/kasan/tags.c | 48 +++++++++++++++++++ 3 files changed, 155 insertions(+), 19 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 7134e75447ff..d368095feb6c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -140,6 +140,13 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_poison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -148,11 +155,24 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) void kasan_unpoison_shadow(const void *address, size_t size) { - kasan_poison_shadow(address, size, 0); + u8 tag = get_tag(address); + + /* + * Perform shadow offset calculation based on untagged address, as + * some of the callers (e.g. kasan_unpoison_object_data) pass tagged + * addresses to this function. + */ + address = reset_tag(address); + + kasan_poison_shadow(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; + + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + *shadow = tag; + else + *shadow = size & KASAN_SHADOW_MASK; } } @@ -200,8 +220,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + if (unlikely(PageHighMem(page))) + return; + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) @@ -218,6 +239,9 @@ void kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + return 0; + return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -232,6 +256,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { unsigned int orig_size = *size; + unsigned int redzone_size; int redzone_adjust; /* Add alloc meta. */ @@ -239,20 +264,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size += sizeof(struct kasan_alloc_meta); /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); if (redzone_adjust > 0) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); + max(*size, cache->object_size + redzone_size)); /* * If the metadata doesn't fit, don't enable KASAN at all. @@ -265,6 +290,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, return; } + cache->align = round_up(cache->align, KASAN_SHADOW_SCALE_SIZE); + *flags |= SLAB_KASAN; } @@ -319,6 +346,28 @@ void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + /* + * Since it's desirable to only call object contructors ones during + * slab allocation, we preassign tags to all such objects. + * Also preassign tags for SLAB_TYPESAFE_BY_RCU slabs to avoid + * use-after-free reports. + * For SLAB allocator we can't preassign tags randomly since the + * freelist is stored as an array of indexes instead of a linked + * list. Assign tags based on objects indexes, so that objects that + * are next to each other get different tags. + */ + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) && + (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU)) { +#ifdef CONFIG_SLAB + struct page *page = virt_to_page(object); + u8 tag = (u8)obj_to_index(cache, page, (void *)object); +#else + u8 tag = random_tag(); +#endif + + object = set_tag(object, tag); + } + return (void *)object; } @@ -327,15 +376,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) return kasan_kmalloc(cache, object, cache->object_size, flags); } +static inline bool shadow_invalid(u8 tag, s8 shadow_byte) +{ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return shadow_byte < 0 || + shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + else + return tag != (u8)shadow_byte; +} + static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { s8 shadow_byte; + u8 tag; + void *tagged_object; unsigned long rounded_up_size; + tag = get_tag(object); + tagged_object = object; + object = reset_tag(object); + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { - kasan_report_invalid_free(object, ip); + kasan_report_invalid_free(tagged_object, ip); return true; } @@ -344,20 +408,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return false; shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); + if (shadow_invalid(tag, shadow_byte)) { + kasan_report_invalid_free(tagged_object, ip); return true; } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) return false; set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); quarantine_put(get_free_info(cache, object), cache); - return true; + + return IS_ENABLED(CONFIG_KASAN_GENERIC); } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -370,6 +436,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, { unsigned long redzone_start; unsigned long redzone_end; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -382,14 +449,27 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(object, size); + /* See the comment in kasan_init_slab_obj regarding preassigned tags */ + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) && + (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU)) { +#ifdef CONFIG_SLAB + struct page *page = virt_to_page(object); + + tag = (u8)obj_to_index(cache, page, (void *)object); +#else + tag = get_tag(object); +#endif + } else + tag = random_tag(); + + kasan_unpoison_shadow(set_tag(object, tag), size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); - return (void *)object; + return set_tag(object, tag); } EXPORT_SYMBOL(kasan_kmalloc); @@ -439,7 +519,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { + if (reset_tag(ptr) != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -452,7 +532,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (ptr != page_address(virt_to_head_page(ptr))) + if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a2533b890248..a3db6b8efe7a 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,10 +12,18 @@ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KASAN_TAG_INVALID +#endif + #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ /* diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index 700323946867..a3cca11e4fed 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -78,15 +78,60 @@ void *kasan_reset_tag(const void *addr) void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { + u8 tag; + u8 *shadow_first, *shadow_last, *shadow; + void *untagged_addr; + + if (unlikely(size == 0)) + return; + + tag = get_tag((const void *)addr); + + /* + * Ignore accesses for pointers tagged with 0xff (native kernel + * pointer tag) to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KASAN_TAG_KERNEL (0xFF)). + */ + if (tag == KASAN_TAG_KERNEL) + return; + + untagged_addr = reset_tag((const void *)addr); + if (unlikely(untagged_addr < + kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { + kasan_report(addr, size, write, ret_ip); + return; + } + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + size - 1); + for (shadow = shadow_first; shadow <= shadow_last; shadow++) { + if (*shadow != tag) { + kasan_report(addr, size, write, ret_ip); + return; + } + } } #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -98,15 +143,18 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { + kasan_poison_shadow((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); From patchwork Wed Sep 19 18:54:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606293 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E9C61390 for ; Wed, 19 Sep 2018 18:56:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1153C291B2 for ; Wed, 19 Sep 2018 18:56:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0441D2B840; Wed, 19 Sep 2018 18:56:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 717F1291B2 for ; Wed, 19 Sep 2018 18:56:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DE578E0017; Wed, 19 Sep 2018 14:55:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EDD528E000A; Wed, 19 Sep 2018 14:55:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7C3B8E0017; Wed, 19 Sep 2018 14:55:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 74F568E000A for ; Wed, 19 Sep 2018 14:55:32 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id g36-v6so6657561wrd.9 for ; Wed, 19 Sep 2018 11:55:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=64LRcqLvQSyCxvtv4IJ7rTIUII9l3vNlb4g5rc8vkL0=; b=q5WTXSuKDyM2JyrHmZ/CiwcF6b4aQapAbs1i3THezUhkmW0UPULrxdT7XCUX+awyoz o5Etv8/dyhwZYB/sAv513x9jtwWOAgweq4KE/9crcriEc6G7gjifkENDBOmzmyIajWA6 EqZ3nLx7Ums0ILPG5yRQa//o7FjjkkdiIIVlYyeNehLWKS4dUXxdDu9ndX62jijrjtBc nmdwZ4ofOqpucIJAVw7nyQW8re/hCiy52MMD9s2vMDAwREj/JeyMjeSwrEOf1wzUWBQ5 hBUcORsqJqoDPGVXGg/j9cR7mkK3Pt6P1t5awiOsACDuxe+FIpswSOsNRTWtSVzsI+dV DnOw== X-Gm-Message-State: APzg51Aig+TdwuD1M3hRy+EpkxGiTdkqp970RCz+3XlPixwFVNGhT786 T+oI9M/6MN91N76JUvi3qtXoaCt13zZS8rsORZ7J45Q6UF0NoWtExEKkpIjoIO0fnE+KdaVp+0c Dh1mCAJ+Bl+BP6L97h2afMYWkzedMP8elFrb2rJo3kg2p9u8yvVwNB3UVBG1oc1AZe27RHAOMTv 4SCx8ecsBmQ58/FYiFnIKwp9hG6gqduJuE9AoJhV5j0kkxPuOGPD4/bTaAFpziNT8LOC/haNXnb KFPvhBQfsBgKLVZ4Q6qTSrXQkqZ8G8at3ay+l6mNk7IA8x4XsusYqeNc5+0REWjbR2SACi2gP1u RSRL6w1o34CmUIbNzsrsfXvSDGx8NqnX9dtiRsbeSjSLZKfPSrX9ZSEDBumeGn/R36cU9plD7ZY 4 X-Received: by 2002:adf:a644:: with SMTP id k62-v6mr29707146wrc.78.1537383331982; Wed, 19 Sep 2018 11:55:31 -0700 (PDT) X-Received: by 2002:adf:a644:: with SMTP id k62-v6mr29707125wrc.78.1537383331302; Wed, 19 Sep 2018 11:55:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383331; cv=none; d=google.com; s=arc-20160816; b=BAu70hasqOhBHmXJ7X2BSqrdYPz6EuWh7osgyDhIvs3v6PBLOc5KpygXEqRGpckAvk rkBc31wGCLJ5V6sfv/CNL+SuIYM6nyaynSLsxHzq/+iy+7hC8q73OKSB111IiwQhZlh+ gx2HxFsoHmvzF4+NC3kgITGV7b599IGoQfCDchz3b0Ya9IaWZX9QW6pMjs+at8TSqxLM giwyxi2FlV0DcgUuKkHaOsnTGBu3854kVxSLd7rTTpMVmW7ALtdpUwSgVutrWD778QhS vHGhTc7Xuswj+mLpC4InQDFU9HPZVBFrUBGwM6hWKw28eNXwz3PRY95E4rIigCD8WNO3 PfEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=64LRcqLvQSyCxvtv4IJ7rTIUII9l3vNlb4g5rc8vkL0=; b=Qlu/jURv+iqaSTsNG1UclIad9+lDQPOEsdDffZ59DfOxh+9gulUCmoSX14CU5Zy4y3 kSnWh+rDjZ0KxB4fZ7boFWZ1Kzd8/OfNDKFylmuQlSasf/w76ChMvVx9P+zxRVeBmoLk /ncgBtsCigYChknSH9P1jn3lNV5+7oAjlnQRkiq0wni7QentR8l+IXJmctgKxWh1mQVT yOs31nIHWuJf4kZ9yBSWA9FsY//QH6mDEx5OqxI/KOdk6yIx9UatEt9Ih2EjU7JBv/pd Q095xSMmJp5arP4R1Ae3RORpkgLsgQCFebq1uwnELX+6/zGQ7HROrX/aM6jiXPSz4Mjl dFNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hG9jAuyf; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 4-v6sor10525766wmg.11.2018.09.19.11.55.31 for (Google Transport Security); Wed, 19 Sep 2018 11:55:31 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=hG9jAuyf; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=64LRcqLvQSyCxvtv4IJ7rTIUII9l3vNlb4g5rc8vkL0=; b=hG9jAuyfvJ6nmCLNRdOdKKDU/2noEnfGgoX0leHX6iQZldPCbscRBlWSJE3B2Wzi/O ZaUfp8B2+Qadav3L2H4Up8N03Nw8tAJvYLlfKkEGF4skQXLzxEDrkUT532pP54vzG8qr 7bPUEbYZ0YEL6Tfu2toFS7lfFjhECtPiVupq6PPpWzbVLRvM/K3WwxZf9T3qWMlDBkqI H5NHnPTs/+zAqNY1gbfZw1oPqXfqv43KjJv4HXNZkzPfg1Hubqs6Fx9fHnpb83bxh0jb OhWoUoVm1uiSP67dp6nhom0qRHgvZVJUljvE3sVKrultK0LfGxBxm9t2lV7yJ/4Lgkf3 A90w== X-Google-Smtp-Source: ANB0VdYGftNhd7nwokX4PuREsLaMzahfl8OPDSKHDJ8LczN3UlzH89oiQnT+xB4L6rm0/DrVRyz9yw== X-Received: by 2002:a1c:208c:: with SMTP id g134-v6mr7666265wmg.144.1537383330821; Wed, 19 Sep 2018 11:55:30 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:30 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 17/20] kasan, arm64: add brk handler for inline instrumentation Date: Wed, 19 Sep 2018 20:54:56 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit adds a tag-based KASAN specific brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 68 +++++++++++++++++++++++++++++++- include/linux/kasan.h | 3 ++ 3 files changed, 71 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/brk-imm.h b/arch/arm64/include/asm/brk-imm.h index ed693c5bcec0..2945fe6cd863 100644 --- a/arch/arm64/include/asm/brk-imm.h +++ b/arch/arm64/include/asm/brk-imm.h @@ -16,10 +16,12 @@ * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps + * 0x9xx: tag-based KASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 #define BUG_BRK_IMM 0x800 +#define KASAN_BRK_IMM 0x900 #endif diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 039e9ff379cc..ca0c00f5b6dd 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -269,10 +270,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +void __arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +{ + __arm64_skip_faulting_instruction(regs, size); /* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -775,7 +780,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) } /* If thread survives, skip over the BUG instruction and continue: */ - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); return DBG_HOOK_HANDLED; } @@ -785,6 +790,58 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_SW_TAGS + +#define KASAN_ESR_RECOVER 0x20 +#define KASAN_ESR_WRITE 0x10 +#define KASAN_ESR_SIZE_MASK 0x0f +#define KASAN_ESR_SIZE(esr) (1 << ((esr) & KASAN_ESR_SIZE_MASK)) + +static int kasan_handler(struct pt_regs *regs, unsigned int esr) +{ + bool recover = esr & KASAN_ESR_RECOVER; + bool write = esr & KASAN_ESR_WRITE; + size_t size = KASAN_ESR_SIZE(esr); + u64 addr = regs->regs[0]; + u64 pc = regs->pc; + + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + kasan_report(addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; this is controlled by + * current->kasan_depth). All these accesses are detected by the tool, + * even though the reports for them are not printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KASAN", regs, 0); + + /* If thread survives, skip over the brk instruction and continue: */ + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + return DBG_HOOK_HANDLED; +} + +#define KASAN_ESR_VAL (0xf2000000 | KASAN_BRK_IMM) +#define KASAN_ESR_MASK 0xffffff00 + +static struct break_hook kasan_break_hook = { + .esr_val = KASAN_ESR_VAL, + .esr_mask = KASAN_ESR_MASK, + .fn = kasan_handler, +}; +#endif + /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -792,6 +849,10 @@ static struct break_hook bug_break_hook = { int __init early_brk64(unsigned long addr, unsigned int esr, struct pt_regs *regs) { +#ifdef CONFIG_KASAN_SW_TAGS + if ((esr & KASAN_ESR_MASK) == KASAN_ESR_VAL) + return kasan_handler(regs, esr) != DBG_HOOK_HANDLED; +#endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; } @@ -799,4 +860,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr, void __init trap_init(void) { register_break_hook(&bug_break_hook); +#ifdef CONFIG_KASAN_SW_TAGS + register_break_hook(&kasan_break_hook); +#endif } diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 7e5be87a05b3..59bd67de63d9 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -173,6 +173,9 @@ void kasan_init_tags(void); void *kasan_reset_tag(const void *addr); +void kasan_report(unsigned long addr, size_t size, + bool is_write, unsigned long ip); + #else /* CONFIG_KASAN_SW_TAGS */ static inline void kasan_init_tags(void) { } From patchwork Wed Sep 19 18:54:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 826C86CB for ; Wed, 19 Sep 2018 18:56:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74049291B2 for ; Wed, 19 Sep 2018 18:56:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6767F2B840; Wed, 19 Sep 2018 18:56:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A475A291B2 for ; Wed, 19 Sep 2018 18:56:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E98808E0018; Wed, 19 Sep 2018 14:55:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E18578E000A; Wed, 19 Sep 2018 14:55:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6B8B8E0018; Wed, 19 Sep 2018 14:55:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 634BB8E000A for ; Wed, 19 Sep 2018 14:55:34 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id j6-v6so6602237wrr.15 for ; Wed, 19 Sep 2018 11:55:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=p0Idd63qdfzI4cv80W1qOI9bgu9uN7abns2u9n7hxJk=; b=Oo6aklJ0G67LGY1d+7/YGqe++NMyJ7EzbH1DfUbAnX0qJNlDZ0pyYbDEITeOltq1/3 tbfi0T0ysPI4erE80sIm9U4h6mQ3+wed/OISFOUhfEMeor4q7k1xRLza6GWDOQh+Lggy aE84/HNgH7SnITEmRX0HPjsSisIXjPhmCeAt/T8LOQgJ1ZdqZSd0K7XtgFncArRQIX14 Va2cEcqaYemb+hrJ76gvV1F83RluAjTyr63VCUz9Vw3mncBQqMl4gexN5uQusWZ8Nc1U 8/B33PK+nFqMH25Lfk81MVLVju1hLzEKhrwkUjKYCG9JU47Lq4DQ2iPNGsTB0xf6pFFx zEmg== X-Gm-Message-State: APzg51CQxvdxQsxvBgkmR3q+FEtovVwJsPLv0SzXvqOOcKlfpVgR9sfy /CkGTEbJ5pbf/63ybWwKYk1oLpcRVgZXeIdz2nSGswrvr9VDULDCVONktiLSJYfuz9/RwNdVm1i xIA2rzRfjaamG3t0Wl/5wIFs7ne29dYK13/mDomJdTnE6A+PQ+IJeCaURySmEfDtRQAFZwTw/2Z xOAUY1qk5r5JZbCmOQFSxr1+kFVk+8v2j1TL90Zi99JtfIMkaqF6vewBcyauPx7ltb+DI+w1lFZ nfBLrhtnpkXD4ESS1NUawVc87BUWYeF6CtqvbacDQjgixf4wda3bnvqL+sAjXJihPqNQ28uT/Xd W0AcE3ZR9jRXCJ9wJKbBH3UA7VRB0RMXz4umOfAJPGhPg+iJZ5ABJwDsWUMu5og0spKLTNiXVut 7 X-Received: by 2002:adf:81c3:: with SMTP id 61-v6mr30449432wra.120.1537383333895; Wed, 19 Sep 2018 11:55:33 -0700 (PDT) X-Received: by 2002:adf:81c3:: with SMTP id 61-v6mr30449398wra.120.1537383332856; Wed, 19 Sep 2018 11:55:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383332; cv=none; d=google.com; s=arc-20160816; b=AXV9t87Flpm9lldlz8dAqdAh6e9ghL+qhmZhLrcb7WDhB+NAJdvssAaIXdbAjVpLmI ho1aSfi1D0lsPCGdTP1ZxESb4kJb0x3LEhvNkyLCGAu4LR/51q9OavGUbtmgS42rD4fP m26bJmWMuwItGKr91TjAMOXqhMwFYE83X5Osi8nplhVcwiR4ypGVOqW8brs6PGbXfofQ tTe8X5qThnQ8ZYkT65myEktRj2qW2odZlqlbkfIZ8p8vhlkcE7ZxgFuj2wQCmKpLJ4bt NHapkdGPa9HC6hCo4xr38APedD+nxWbA9OSgcsOyTsWRHH0QXwmlvIdWmWqZ8/SK0E7y jmlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=p0Idd63qdfzI4cv80W1qOI9bgu9uN7abns2u9n7hxJk=; b=DIk8zaDVJ4zE5bn/rajLEw/FxAX2ERs7myLGc7AZ+yZ3pVZsdiA9GzBEGfOeBNBMv7 ZfZFaNeTm9TJ0zVC6ysCwajP7sJLt+YUZFyTwWclZsck9/mb4Cama9JiyJ1sjnbE4dez 2pjGQzTQUBKK2Y+CKam19l80iCYlbCkg5347z6y+j70pAQVpSaoj7wsrK1fkRiMNCie7 8J8kVmbN7ksyuIKkd6b/FfXSInKrsH7r1N1Cm0LAkR/IIV3caVXj9jJq9eHh6P/UT8bq CcYYqDbPrYaxlz7d06rkTMF4orhnrQ77Qo+ODpuOe/A1GjodXhn0K27B7Z/W7Ulok/Mk 2pLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aJwKAEWt; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id q1-v6sor242673wrw.17.2018.09.19.11.55.32 for (Google Transport Security); Wed, 19 Sep 2018 11:55:32 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aJwKAEWt; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p0Idd63qdfzI4cv80W1qOI9bgu9uN7abns2u9n7hxJk=; b=aJwKAEWt2qoZCPE9vcDSyKCeRH5drma8asZpluHHJE5EIsC/79XSa75YryQs/zWD6B d7FHgNP/lW5wzxIu2bBFXHXJjWnNu5FVCEEptCDOC9AhwEY4xqjS039yqF2Y4XjeznL4 ExzmzqQCKnl6CzdRItYT9OBko2ZPamxNAIK6r88bo4hIRUM0P6oiRqKgj6aXirr4OcAc EoKtixcL3ugxOoAqT2lQxj51CPMVCxqdcOeiWsxEYG89SLO5v9HSUFqQVkOv/RXhKgqA h91latSNq0lh6bk7nLALGZmWwEVTW1S0EATnG11XB0Td0Boj6V5yyC17Q4nz/hIyMbvv vD0g== X-Google-Smtp-Source: ANB0Vdb7oa0Uf6wBDtRH7EBoKtrGei8Fwx+CIIPGt3c5p8oXRNNIn/F8ZbYXUxMS2cAo4RKw9VDUHA== X-Received: by 2002:adf:dc46:: with SMTP id m6-v6mr30846752wrj.84.1537383332361; Wed, 19 Sep 2018 11:55:32 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:31 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 18/20] kasan, mm, arm64: tag non slab memory allocated via pagealloc Date: Wed, 19 Sep 2018 20:54:57 +0200 Message-Id: <0c437ef4a8ffdacc5f93a5a22a0a498504f62ea5.1537383101.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently. For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag. This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 9 ++++++++- include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 15 +++++++++++++-- mm/page_alloc.c | 1 + mm/slab.c | 2 +- 7 files changed, 73 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 3226a0218b0b..b7108161732e 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -98,6 +98,7 @@ KASAN_TAG_SHIFTED(tag)) #define KASAN_RESET_TAG(addr) KASAN_SET_TAG(addr, 0xff) #else +#define KASAN_SET_TAG(addr, tag) addr #define KASAN_RESET_TAG(addr) addr #endif @@ -309,7 +310,13 @@ static inline void *phys_to_virt(phys_addr_t x) #define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page)) #define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) -#define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#define page_to_virt(page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = KASAN_SET_TAG(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) + #define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START)) #define _virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \ diff --git a/include/linux/mm.h b/include/linux/mm.h index a61ebe8ad4ca..731e85e2cae3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -804,6 +804,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH) #define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH) #define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) /* * Define the bit shifts to access each section. For non-existent @@ -814,6 +815,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0)) #define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0)) #define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0)) /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS @@ -836,6 +838,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) static inline enum zone_type page_zonenum(const struct page *page) @@ -1081,6 +1084,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) } #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_SW_TAGS +static inline u8 page_kasan_tag(const struct page *page) +{ + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; +} + +static inline void page_kasan_tag_reset(struct page *page) +{ + page_kasan_tag_set(page, 0xff); +} +#else +static inline u8 page_kasan_tag(const struct page *page) +{ + return 0xff; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) { } +static inline void page_kasan_tag_reset(struct page *page) { } +#endif + static inline struct zone *page_zone(const struct page *page) { return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)]; diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7ec86bf31ce4..1dda31825ec4 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -82,6 +82,16 @@ #define LAST_CPUPID_WIDTH 0 #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH+LAST_CPUPID_WIDTH+KASAN_TAG_WIDTH \ + > BITS_PER_LONG - NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif + /* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit. diff --git a/mm/cma.c b/mm/cma.c index 4cb76121a3ab..c7b39dd3b4f6 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + size_t i; struct page *page = NULL; int ret = -ENOMEM; @@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, trace_cma_alloc(pfn, page, count, align); + /* + * CMA can allocate multiple page blocks, which results in different + * blocks being marked with different tags. Reset the tags to ignore + * those page blocks. + */ + if (page) { + for (i = 0; i < count; i++) + page_kasan_tag_reset(page + i); + } + if (ret && !no_warn) { pr_err("%s: alloc failed, req-size: %zu pages, ret: %d\n", __func__, count, ret); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index d368095feb6c..25be74d3738f 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -220,8 +220,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { + u8 tag; + unsigned long i; + if (unlikely(PageHighMem(page))) return; + + tag = random_tag(); + for (i = 0; i < (1 << order); i++) + page_kasan_tag_set(page + i, tag); kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } @@ -319,6 +326,10 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, void kasan_poison_slab(struct page *page) { + unsigned long i; + + for (i = 0; i < (1 << compound_order(page)); i++) + page_kasan_tag_reset(page + i); kasan_poison_shadow(page_address(page), PAGE_SIZE << compound_order(page), KASAN_KMALLOC_REDZONE); @@ -519,7 +530,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (reset_tag(ptr) != page_address(page)) { + if (ptr != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -532,7 +543,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) + if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 89d2a2ab3fe6..36971fd6cc6c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1176,6 +1176,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, init_page_count(page); page_mapcount_reset(page); page_cpupid_reset_last(page); + page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL diff --git a/mm/slab.c b/mm/slab.c index 6d8de7630944..9403dd9b269c 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -2357,7 +2357,7 @@ static void *alloc_slabmgmt(struct kmem_cache *cachep, void *freelist; void *addr = page_address(page); - page->s_mem = addr + colour_off; + page->s_mem = kasan_reset_tag(addr) + colour_off; page->active = 0; if (OBJFREELIST_SLAB(cachep)) From patchwork Wed Sep 19 18:54:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E14381390 for ; Wed, 19 Sep 2018 18:56:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D22E62B5EA for ; Wed, 19 Sep 2018 18:56:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C50D22C862; Wed, 19 Sep 2018 18:56:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AFD962B5EA for ; Wed, 19 Sep 2018 18:56:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0FD5B8E000A; Wed, 19 Sep 2018 14:55:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 083068E0019; Wed, 19 Sep 2018 14:55:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E68468E000A; Wed, 19 Sep 2018 14:55:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 6928C8E0019 for ; Wed, 19 Sep 2018 14:55:36 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id i11-v6so6555760wrr.10 for ; Wed, 19 Sep 2018 11:55:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ueKsyLzEOYvLTBs2T6zom6NHkl/QngEo7OSib1MBtVc=; b=g1Z+Hcy4z/mcFLXla+IW+nMu97khD39KedVgQZMr4LsJW9hmUslGNiwkyOxIUPGepa fac9UP55lTei3f0Y35aswXsPuadenTjfNuJUeaRBByNGiPSsuk89su8BvIGZ551Diw+Z /zAeVyPaQpMZHw5YfftLXJwR93Ksm6iuCYNc2pT4wTT6ZpYIaizbUawTO0DP9BCZIlzr pNFrDUdPa3qBqAtpD7VwKP/BP/8SWUdRq5nlR2y1zYdHM5Y/fRjvWENucLlLmfmsJX9Z 3NAd6MIoux9xXVPrf2+Fn+NYuunZvK2Syojets5+2JakZ0BMBMC1rX4WCsJj5pgTtTWA MKSw== X-Gm-Message-State: APzg51Bn2DvzlXvn9vDxsYx6sozdfTWM3MWF2GukGYIHeY82ZjfImLF2 Zs20hlo27RvtmGzJIHLKX+ubLjMmLq993gvnFTSaiYUyDB9aIg2CsNkGgKf4+FdRbKPzVKC2kJD BnL4QKy/gjSjQiO+b9UqxrDqStKnSKTTWX43ESHVcIcfacZuj466vcjllPpSyLSIKc6dPkOvsLl MRm2tOpkA4xs0ztP6Kh6+PmLUvc8wqBL5AmCWY3miys/13AhGCEbmT2S6M36rScRjwa4a1agb6A o8fVka/eozvG82uowfx2BeoWhB+7MO4reNyDPAIkmC+yFzWvXsKxW6yoGQDEcjtYTtHq+bd4Jnx DwSILKH5TdjT5CkNPLgN0rKnoKk6Upjq/YVekW+gw0SEXv2uxTyFMZVGbiz0KE0fQ5e/5MtVIFk q X-Received: by 2002:adf:c684:: with SMTP id j4-v6mr29724493wrg.243.1537383335898; Wed, 19 Sep 2018 11:55:35 -0700 (PDT) X-Received: by 2002:adf:c684:: with SMTP id j4-v6mr29724426wrg.243.1537383334576; Wed, 19 Sep 2018 11:55:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383334; cv=none; d=google.com; s=arc-20160816; b=ezg1NNqiz6nnk+eamS3pXTj4kYM1WH8RRcn4QZS67Jw8q2AHhxz6fnndI8b8BahUk/ RDH/STJaMeeY6QGcDBJPAPmDWikH/M+M6cKXPkNU34nqHbvEC+6YqW3WNtWubMKbCoI6 /+hPTt4HCQ5nm7zUSB0pJZK4gQKVtLjV80xUreNkpdpRpZQc8Xo8sz5G9P0+TFeeiKlt OhFJ5LrRkgyxmRK4hhHYwg3RvRTzt+Z6deqGsfnBdipitzedajW6h1+h1tiGqlaN+uBp PphbOJJCJ4L5LP3y1AzJU8gi1o/rMzm6uCrxUAl6qc4FLeVy6KVZsuTnjsDMXzN4gj3j oavQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=ueKsyLzEOYvLTBs2T6zom6NHkl/QngEo7OSib1MBtVc=; b=DUw08MnJPUwLeBs7nBUwG0FkIHTeBigqdfXvbNe3nbinyjBQovFpkFqcOJzy0sUxI7 joLmXcbYvEU5Morq6Zfo1U9ySWI2zqKKYIf6z2UIbr8iMrT194B8sF4K2HHBN82tYlKr 39xQKxl/gBPfzSvLaAOSeGaPyVY1GDkiT1HtFw3Ka9F8mDioLVGOa0/vg04DYnkzwcuo aOAPBNrydA+773ukBqBmFfNJWLw+XC/Fb5QM94wABvIx8IM7ZelEYv3WAKE/IVjzG+QS AJVay5fJFLDs6BAofNOvZuGLqWwgw46E5br4GRQBGWprQtfwMXwlW9hhWDf8WW3iLVlI TkSA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=i8qXITZD; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h18-v6sor2553282wro.45.2018.09.19.11.55.34 for (Google Transport Security); Wed, 19 Sep 2018 11:55:34 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=i8qXITZD; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ueKsyLzEOYvLTBs2T6zom6NHkl/QngEo7OSib1MBtVc=; b=i8qXITZDtn8Gf/6zjTDp/eZuvy/n1/2rlFWXSJl+wwwHrsh5SnbfcblCzFcpOPSvSE yO6ubEHeQO9nmVwR9lUnxWpB2ursuEmLJjl/M3OOWDxSRc3zygXUYnxGXNm9lrtaR6tg 75wV8hRF3HbV/W3+qngHyHYODbKCMCKKioC/jfIJRWiR10HOszuqL61EzjF9H8vPzBwv fHd3wOwF4hVPotTteDAsr5RbYjEI0q7c7tTHREQFZ/4VRqEkkdX2XKacUkvAtkZlh66R stlTvGIota9xy8PU09q/30clkJuIIvCBJ11YriT1JWNKnbC5PlTue3ywXKEPv2cU08R8 3mlg== X-Google-Smtp-Source: ANB0Vdb/dnCA8Vd2zzfgv5wuimw5oEGrKhBGBIWL1O2XwjB1jefeReEdZI49Rmb9vounHcFMj+ehNw== X-Received: by 2002:adf:8445:: with SMTP id 63-v6mr30544882wrf.41.1537383333904; Wed, 19 Sep 2018 11:55:33 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:33 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 19/20] kasan: update documentation Date: Wed, 19 Sep 2018 20:54:58 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch updates KASAN documentation to reflect the addition of the new tag-based mode. Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 232 ++++++++++++++++++------------ 1 file changed, 138 insertions(+), 94 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index aabc8738b3d8..a407e18afd32 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -4,15 +4,25 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides -a fast and comprehensive solution for finding use-after-free and out-of-bounds -bugs. +KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to +find out-of-bound and use-after-free bugs. KASAN has two modes: generic KASAN +(similar to userspace ASan) and software tag-based KASAN (similar to userspace +HWASan). -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. -Currently KASAN is supported only for the x86_64 and arm64 architectures. +Generic KASAN is supported in both GCC and Clang. With GCC it requires version +4.9.2 or later for basic support and version 5.0 or later for detection of +out-of-bounds accesses for stack and global variables and for inline +instrumentation mode (see the Usage section). With Clang it requires version +3.7.0 or later and it doesn't support detection of out-of-bounds accesses for +global variables yet. + +Tag-based KASAN is only supported in Clang and requires version 7.0.0 or later. + +Currently generic KASAN is supported for the x86_64, arm64 and xtensa +architectures, and tag-based KASAN is supported only for arm64. Usage ----- @@ -21,12 +31,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC -version 5.0 or later. +and choose between CONFIG_KASAN_GENERIC (to enable generic KASAN) and +CONFIG_KASAN_SW_TAGS (to enable software tag-based KASAN). -KASAN works with both SLUB and SLAB memory allocators. +You also need to choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. +Outline and inline are compiler instrumentation types. The former produces +smaller binary while the latter is 1.1 - 2 times faster. + +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line @@ -43,85 +55,85 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access generic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8801f44ec37b by task insmod/2760 + + CPU: 1 PID: 2760 Comm: insmod Not tainted 4.19.0-rc3+ #698 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [] dump_stack+0x46/0x58 - [] print_trailer+0xf8/0x160 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] object_err+0x35/0x40 - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kasan_report_error+0x38a/0x3f0 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kasan_unpoison_shadow+0x14/0x40 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] __asan_store1+0x75/0xb0 - [] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kmalloc_oob_right+0x65/0x75 [test_kasan] - [] init_module+0x9/0x47 [test_kasan] - [] do_one_initcall+0x99/0x200 - [] ? __vunmap+0xec/0x160 - [] load_module+0x2cb3/0x3b20 - [] ? m_show+0x240/0x240 - [] SyS_finit_module+0x76/0x80 - [] system_call_fastpath+0x12/0x17 + dump_stack+0x94/0xd8 + print_address_description+0x73/0x280 + kasan_report+0x144/0x187 + __asan_report_store1_noabort+0x17/0x20 + kmalloc_oob_right+0xa8/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + RIP: 0033:0x7f96443109da + RSP: 002b:00007ffcf0b51b08 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055dc3ee521a0 RCX: 00007f96443109da + RDX: 00007f96445cff88 RSI: 0000000000057a50 RDI: 00007f9644992000 + RBP: 000055dc3ee510b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007f964430cd0a R11: 0000000000000202 R12: 00007f96445cff88 + R13: 000055dc3ee51090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2760: + save_stack+0x43/0xd0 + kasan_kmalloc+0xa7/0xd0 + kmem_cache_alloc_trace+0xe1/0x1b0 + kmalloc_oob_right+0x56/0xbc [test_kasan] + kmalloc_tests_init+0x16/0x700 [test_kasan] + do_one_initcall+0xa5/0x3ae + do_init_module+0x1b6/0x547 + load_module+0x75df/0x8070 + __do_sys_init_module+0x1c6/0x200 + __x64_sys_init_module+0x6e/0xb0 + do_syscall_64+0x9f/0x2c0 + entry_SYSCALL_64_after_hwframe+0x44/0xa9 + + Freed by task 815: + save_stack+0x43/0xd0 + __kasan_slab_free+0x135/0x190 + kasan_slab_free+0xe/0x10 + kfree+0x93/0x1a0 + umh_complete+0x6a/0xa0 + call_usermodehelper_exec_async+0x4c3/0x640 + ret_from_fork+0x35/0x40 + + The buggy address belongs to the object at ffff8801f44ec300 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8801f44ec300, ffff8801f44ec380) + The buggy address belongs to the page: + page:ffffea0007d13b00 count:1 mapcount:0 mapping:ffff8801f7001640 index:0x0 + flags: 0x200000000000100(slab) + raw: 0200000000000100 ffffea0007d11dc0 0000001a0000001a ffff8801f7001640 + raw: 0000000000000000 0000000080150015 00000001ffffffff 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8801f44ec200: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec280: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc + >ffff8801f44ec300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 + ^ + ffff8801f44ec380: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb + ffff8801f44ec400: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works. @@ -138,18 +150,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For tag-based KASAN this last report section shows the memory tags around the +accessed address (see Implementation details section). + Implementation details ---------------------- +Generic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address:: @@ -162,12 +180,38 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +Software tag-based KASAN +~~~~~~~~~~~~~~~~~~~~~~~~ + +Tag-based KASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to +store a pointer tag in the top byte of kernel pointers. Like generic KASAN it +uses shadow memory to store memory tags associated with each 16-byte memory +cell (therefore it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation tag-based KASAN generates a random tag, tags the +allocated memory with this tag, and embeds this tag into the returned pointer. +Software tag-based KASAN uses compile-time instrumentation to insert checks +before each memory access. These checks make sure that tag of the memory that +is being accessed is equal to tag of the pointer that is used to access this +memory. In case of a tag mismatch tag-based KASAN prints a bug report. + +Software tag-based KASAN also has two instrumentation modes (outline, that +emits callbacks to check memory accesses; and inline, that performs the shadow +memory checks inline). With outline instrumentation mode, a bug report is +simply printed from the function that performs the access check. With inline +instrumentation a brk instruction is emitted by the compiler, and a dedicated +brk handler is used to print bug reports. + +A potential expansion of this mode is a hardware tag-based mode, which would +use hardware memory tagging support instead of compiler instrumentation and +manual shadow memory manipulation. From patchwork Wed Sep 19 18:54:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10606311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35CD56CB for ; Wed, 19 Sep 2018 18:57:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2886C28358 for ; Wed, 19 Sep 2018 18:57:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19E07274A3; Wed, 19 Sep 2018 18:57:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 996C5274A3 for ; Wed, 19 Sep 2018 18:57:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D32ED8E001A; Wed, 19 Sep 2018 14:55:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C91AF8E0019; Wed, 19 Sep 2018 14:55:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE2C68E001A; Wed, 19 Sep 2018 14:55:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 4DC308E0019 for ; Wed, 19 Sep 2018 14:55:37 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id 51-v6so6492159wra.18 for ; Wed, 19 Sep 2018 11:55:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1D2lwL1JaZ9jajF6hdt8cBeNs/CCBgtmxQ7KM/D8RFc=; b=DLz4gdWNj/aiS0sjX233c8gcB4nd+wb/SKHcDcZttyLT8iep1VonTipqgmzRD0FAtY k0fXgnQcQvzX8XSLcErYEurMF3XjYmcCUze704r9TBz1ZFtZ/x0r/PkEpVWbCzqvTLWN ImWpjN/GZXw10FrygWy11C9wDCmCNP8OJXUyyvgsY8XooZHsilnvAjBiSQqUsOXcxp4i E5HLO2B4g6Vt0jSzL97iDOym7IWlBJbsFSEPuveQNj20asr01dkh6siwbbn1uSNZSOHV TAJKTRjgVf/bWm86WAdODjmjimStucilo0/8f7YskBFjh5NshKlvZQvRLp3HCJOQL9Xe Mnhw== X-Gm-Message-State: APzg51CLmQukJhXcMJZibScIO6MXC2gHztKcTaEdmIkuH3v5x23l3RQ7 5l2lqzfSGZQNmwH4D0Uw3GrhzZG8MFg2oDwAlMGd8O8/dN6es+dwCV9Jk0N7w5maVb14oOyLuzi jRORzCiFGu42IL7QLapR8B23ZW9lD8w0q6wdP4nsgIUyes4t4p6LTq98gYsaG9vsCvS6DbXggtF KZE6quSHMMkhHht1eb8pHUbX022BPhDiV5imPc6lCmPBFQkRjiSElmTpLgipUDCccITaoZ+nGt7 ucvLdDNTS1EmMmEKej9jab8JPExpfALJffnPAOZpL2wh0zNR38KiWKxPSRgmjbGk75Va+5F2I8a NCRP9u+x0TToBuL7uaIlQoFEI5jtoOxqAJRK99biVoqwcMhEpzNSUTN5SVStrqOv3h/lJQKacIb 9 X-Received: by 2002:a1c:1748:: with SMTP id 69-v6mr20814067wmx.75.1537383336838; Wed, 19 Sep 2018 11:55:36 -0700 (PDT) X-Received: by 2002:a1c:1748:: with SMTP id 69-v6mr20814036wmx.75.1537383336036; Wed, 19 Sep 2018 11:55:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537383336; cv=none; d=google.com; s=arc-20160816; b=Q+vfmHpVL0/VczCTle2JX3ybWroH1+MDhjv+7bEnQUM9tcW9egEljg+On0t6SVAWMo 05mcJErz0v/qrLngMYH+SpDE+q8YLFYiNHTYJ3DwXHj+MYiWHe/MC2Q/7rd7lwivOOCb ollfAccBFPotrvcGiTJBlLb6hshTpl9osCNoakiPeW1VyvgRozmnCxJ4gguXhQJ/XVcS to/8ZmDISGiRd+9ufOpRChEjdLYIFkxYnSezz8oIDA+8uXHijRNm4xapStOe/X1cjQDT FjHmOmPnTgobTSiMVJF9TuRXHaDRa/0FT2HulfkOcxB7kFpqAjXLx1mv6OSxSiRmEoji Me1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=1D2lwL1JaZ9jajF6hdt8cBeNs/CCBgtmxQ7KM/D8RFc=; b=FcoT1DpBox9ZZAKfnTVlZ3mNIsAo4yb8ZJQ3aU6a9hHQZYO6Y9jR1zNhX74p8bNUwO 8dgYk7XSHrY0gq6rwG/FcvvXtuXtCwj7uWcAcoi4jl6PmsITKC/8dptBMSgzprysXyPG tFDNyOmc1thSr7TcAkqVeXGLHzduSo9Fsj26eJVyv8/RTyfNZUBcYACsu4Kc6tmDny9b VBSAG//+AJD92UAJoWVonV6k92d49k6f/huHUQheq34ne5F+YeyX5ZFas3nV9fwgIMkv Nq3j4ML5hnu5B0+rxx50Q23xwpUL95TQk+k+sRmyBH4tHyhXY1c4ag49JXNCugNV+1bC qw5w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=PxgUzK0v; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id d6-v6sor10617549wmb.10.2018.09.19.11.55.35 for (Google Transport Security); Wed, 19 Sep 2018 11:55:36 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=PxgUzK0v; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1D2lwL1JaZ9jajF6hdt8cBeNs/CCBgtmxQ7KM/D8RFc=; b=PxgUzK0vwDBTyXOPX/cG4NMJdxB4fKJyTnmf8FYPU7Tl7a3W3/HEqI9dXvbmEyWpXP PMy/6ufcrDslWWqrDpyxBbArZ24kZrdbF36ZQCcAF9KKqFiIoj+Xq8lguy/MhBKLdXVF LX5t1Or53C4aX3hsKSOGer+P1D9wvnTYidtqpN4MkA159JXYlv/M/qM+LI5RlaY6IZi+ EQ8Hzjdc/4SK7JzcWQKYUNPiUGuClZk0bRzHrlBJZwT5bKRIqwNikQ7MBLrL/QYSu/YA p7/o6paGlSDwSslz5TTOCe6YiXAIOfZLKsaiynXEfr/6v0HlmT7enSaUlTczy5k87NWH K0Mw== X-Google-Smtp-Source: ANB0VdZAwJs1ECP4eUwXCvbP/BcECN97gCYmPsH+kOoBGhi7wMdxkP2oUW7mAkEM8arF+3YVGck0kQ== X-Received: by 2002:a1c:9290:: with SMTP id u138-v6mr21164170wmd.52.1537383335654; Wed, 19 Sep 2018 11:55:35 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id b10-v6sm8510065wmc.28.2018.09.19.11.55.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Sep 2018 11:55:34 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v8 20/20] kasan: add SPDX-License-Identifier mark to source files Date: Wed, 19 Sep 2018 20:54:59 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.397.gdd90340f6a-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 1 + mm/kasan/generic.c | 1 + mm/kasan/generic_report.c | 1 + mm/kasan/init.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + mm/kasan/tags.c | 1 + mm/kasan/tags_report.c | 1 + 8 files changed, 8 insertions(+) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 25be74d3738f..de76a2bbb375 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN code. * diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index b8de6d33c55c..ccb6207276e3 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core generic KASAN code. * diff --git a/mm/kasan/generic_report.c b/mm/kasan/generic_report.c index a4604cceae59..5e12035888f2 100644 --- a/mm/kasan/generic_report.c +++ b/mm/kasan/generic_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains generic KASAN specific error reporting code. * diff --git a/mm/kasan/init.c b/mm/kasan/init.c index 7a2a2f13f86f..b3c068ab2a85 100644 --- a/mm/kasan/init.c +++ b/mm/kasan/init.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. * diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 3a8ddf8baf7d..0e4dc1a22615 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. * diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 214d85035f99..ca9418fe9232 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common generic and tag-based KASAN error reporting code. * diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index a3cca11e4fed..7b7c21d40851 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core tag-based KASAN code. * diff --git a/mm/kasan/tags_report.c b/mm/kasan/tags_report.c index 573c51d20d09..8eaf5f722271 100644 --- a/mm/kasan/tags_report.c +++ b/mm/kasan/tags_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains tag-based KASAN specific error reporting code. *