From patchwork Wed Aug 29 11:35:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579921 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4E9D17DE for ; Wed, 29 Aug 2018 11:35:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A62A42AB44 for ; Wed, 29 Aug 2018 11:35:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 99F932ABEF; Wed, 29 Aug 2018 11:35:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5ED582AB44 for ; Wed, 29 Aug 2018 11:35:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D88E96B4B77; Wed, 29 Aug 2018 07:35:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D0A956B4B78; Wed, 29 Aug 2018 07:35:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAAE86B4B79; Wed, 29 Aug 2018 07:35:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id 5D88B6B4B77 for ; Wed, 29 Aug 2018 07:35:31 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id v24-v6so2744616wmh.5 for ; Wed, 29 Aug 2018 04:35:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DWx/ckKLAvPjEmwCNB4J7Qa43qgDShgt8CqkXyDh3Gs=; b=SlUYJyWvIWr4GGVfYh+2ilmsTXPR3xbui9ETo2jgdo189xG362V4hjeD4jiyb0rzc1 gUXO6Z67lxyB7z8x/Zv53iw1vTKmCtPjngUdGHYI9dUrc4o+yI+ddnwSdPf87+Yb+oO+ QKhId1gDoTbcLIvFjxQo6r43qG5Artm/bZKqdFN3uvCzXWWjMdXE+CQD0QGFP9Ipr4PZ YZp6zuFP0Lin61Ps+WUrkYsIbMLymuLaBiFTCZ5+0deb8bVHKsj+YQjeMdD+UjhFTt+v 31t7CCRrIDfjaJPtyFjx2+kB09033cAmvfXMRXGTXKfg0MsI0YEX2tWAigs4Q5gjVMo3 Brng== X-Gm-Message-State: APzg51C7FpYCOTxhb/SJu+Tposfuzmf0HDsMI1D4PsidMNzZWdRFrjg/ HI0JIYPYJfNMD6YGqdrijMOAOEfMEgSSMtq4TIY2FshNrZxKALaFRQi8vJQ81JHdUT4vbOL5+gy 3a94XyN4lsuSll/iJMYuSnIsXmzGjTBTo+U956h1V7iXOa7Vsd8DJXtODxQ8nq2vGoO1+3m7JAW fCiDbK8LLXKguv64qu4lkmQrXEHBbGlqn+oGj9uLlxzX/Rnr35jv0f2eiY2QN94Grdam8U/Owlj xM1ihmW1j5ssXAKXapSw/HXZrWzg3qDfGbOJcSlGdQign+hHsTmyZjO4/NBk3GYEG4wNRJb93Li omSxhxdyL0wnNQ0GDCuwsIVE73VLT4ViUJrQt6i31LOlXKN9Rx1odP5LmdfjZ2kWHUcYYVh7b2J K X-Received: by 2002:adf:fd90:: with SMTP id d16-v6mr4138941wrr.276.1535542530876; Wed, 29 Aug 2018 04:35:30 -0700 (PDT) X-Received: by 2002:adf:fd90:: with SMTP id d16-v6mr4138896wrr.276.1535542529758; Wed, 29 Aug 2018 04:35:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542529; cv=none; d=google.com; s=arc-20160816; b=ReL+sA3FSE4IG4KZkOa765bZWwXN97KcsT96pxHP7zU8b3aBsmPNxpCh3XgIgvppZr BfjnY8/ETTvpKIougSCsFkjyh1M0RLQEk9zaV+zJknxzFwpZ8lJkRvX3BjqtpwKpgVzY KstoNB3kkJfo5gDwPKgutb2MM1gS1qJiDxudTvn08sdwK/XNVOoQu+No7xxsbVKi54Si rHsEn+DFSKkkdwpBB1QmVPnqQ46BykfEMgnzmN4/FKUh/qLQBv6vIwdOTxeaZ/RHteLb hsSQKqoZsIAPU7slTaowdKCg3E2XjbWow+wyVturiCvO+dMuIWh6HG2hddfKWC/JWv0o qi1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=DWx/ckKLAvPjEmwCNB4J7Qa43qgDShgt8CqkXyDh3Gs=; b=ac9JlWVHIrKvDlLW8SYtC7uMEc6WIVc9ArO71AO09mVoLYONmToNzF3pMOPz07+Vzh BPrNTACVBDmWiGbGS94GM/hGzHdWX/ZTfpWbvEegLJWPC9ic6pvrhBibQ7PZoxO3peQJ NMbA7H4vCS3Z/2f/IBlZpuYwaCtVSHTus4KummgeBfcZvyA5rt1bstuwHGLNzcHHhZFM xGgEWDfgZ109BVH5Da2tkIZxIOuuE53yl7lDt8tH4LQcUdLLxcbtRmlFfLdJH8rDqg8c 9JaQlRfwXo0CKGdzB2FJfYogq+L9F2nPzpf5A1ZEHfM1IGTIVuRb2n7hY2AnavsuUON1 qU0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VdZleA5f; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n15-v6sor2602847wrm.3.2018.08.29.04.35.29 for (Google Transport Security); Wed, 29 Aug 2018 04:35:29 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=VdZleA5f; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DWx/ckKLAvPjEmwCNB4J7Qa43qgDShgt8CqkXyDh3Gs=; b=VdZleA5f/QrDLXj5hmQCvrACHeAXStfuQeanz0uwaG3vFDeYRMA+iRi27lu0RxtQUq rPYkRA6eoJkgs0ij2+YHikniGjB7O5olzYChG0RfEz2TeHakj7DT1Z1pA83aTcI/Osr3 5aK9fd6NwfQfSHywESx8ZJvcElInulKcBiYoQ8flf3T2lPDN3ke21Loiqz7ffxraF7fs S6j/yFzT/c2BWUfdm8ckr7dNOxp4T+eUEUqji2vtUxmVLPnD+ZgkoTM/LMDt+y0lUfon knksVT69NrAi+QicsGA6GHYG2wljXUVmyqf7E4CEFLRRjkJyfJL7YKkkecZYF8tmVN54 zdlg== X-Google-Smtp-Source: ANB0VdZQ0I3CpxM5TfpRPAosiH7gEdfJBNi4DoeUulGkVHToPjriAGd9aZOhVh8LoXEkQW2fY3Q7pA== X-Received: by 2002:adf:e486:: with SMTP id i6-v6mr4059801wrm.145.1535542528963; Wed, 29 Aug 2018 04:35:28 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:28 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 01/18] khwasan, mm: change kasan hooks signatures Date: Wed, 29 Aug 2018 13:35:05 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN will change the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 34 +++++++++++++++++++++++----------- mm/kasan/kasan.c | 24 ++++++++++++++---------- mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 6 files changed, 53 insertions(+), 38 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 46aae129917c..54d577ad2181 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -53,14 +53,14 @@ void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -108,16 +108,28 @@ static inline void kasan_poison_object_data(struct kmem_cache *cache, static inline void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) {} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..f696c7c143c2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -485,9 +485,9 @@ void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) __memset(alloc_info, 0, sizeof(*alloc_info)); } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +528,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +538,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +551,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +566,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +576,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index aa76a70e087e..6fdca9ec2ea4 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3679,7 +3679,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3715,7 +3715,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 58c6c1c2a78e..4190c24ef0e9 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index fea3376f9816..3abfa0f86118 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1183,7 +1183,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1461,7 +1461,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index ce2b9e5cea77..30b9bf777bab 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1335,10 +1335,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2731,7 +2731,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2759,7 +2759,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3749,7 +3749,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3766,8 +3766,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3794,7 +3793,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } From patchwork Wed Aug 29 11:35:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579931 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E2F01709 for ; Wed, 29 Aug 2018 11:35:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C7112AA9C for ; Wed, 29 Aug 2018 11:35:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7E71F2AB91; Wed, 29 Aug 2018 11:35:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 95B872AA9C for ; Wed, 29 Aug 2018 11:35:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 188F86B4B7B; Wed, 29 Aug 2018 07:35:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 03A8A6B4B7E; Wed, 29 Aug 2018 07:35:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA9276B4B7C; Wed, 29 Aug 2018 07:35:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 53A7F6B4B7A for ; Wed, 29 Aug 2018 07:35:35 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id u12-v6so3327996wrc.1 for ; Wed, 29 Aug 2018 04:35:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3whpCSoEraN6lbdQ6MXCL1Fj+d5eA3/FsJPvazGbB60=; b=oIHAIO5SOMnoHEtfEr5xF+Sx7ITQWLmiTBFSQ5gdg8tqh4gYTZGQqbe/CgBeRNZL67 HFXQhrXUHgtQQ2GPyEkeRhM5hTNu7JNT1EF4857pIDUpcws/Cw/XpOUSjx+jjE5xdc44 qW56Jk5hiXn/0fy5esrVlNnYBMfeYn11WyZvm3yHj+P+z4PTvNt/ZCOjgi9JK3Mh39nE 36HPkvuCtK4mKjBhlb5AObsdgcy3fb37V1MgaYEhCLMHusmIDIjWgXJY/Ikp+3QDj9HA OiLYsVKJNP49S3xShqJE5V934c9f9EX4wg0x/CLO1AlQxq/2fbdCYzJ9S3lPV7vhDBy0 M/Pg== X-Gm-Message-State: APzg51CC13R2aHIKHdxavYbW0zULacoJMypj3lFXOim8wknSBWFmPw7a m9yfyy1LKGLeaK12uApYf3zFp84/ticU0VlFcnza09aULVLChaA9gC9gJfaq9vJN/HvqR5WR7sF TkZhhTMoYWmcQzRKSu6cy+wUV//xsrvppWNdwGCJWi1wyoNMVGx6zOrQYn3lSYffSRZpGpErI8y 6O2Sx8BC9Js/fxzHRaviIByk2NIJ3M5fFEBlXUNlMdTaUUDop6euIftJ3rhxcl8NyLw8e5LQIqA gjz3xIMKc98w9l2EEr8RvYldQJnchiETgI+kIkGxFIZ69rcTWMMS4h5HiiNxhjycYdT8C2qMyPK oJvmei/lKJvI24daw2z3D54K5abhXh7eb3SIp2tSHK6e93TL6pdeOf5OFJSelnG5mzKlAyqyxQf G X-Received: by 2002:a1c:f516:: with SMTP id t22-v6mr3925135wmh.103.1535542534639; Wed, 29 Aug 2018 04:35:34 -0700 (PDT) X-Received: by 2002:a1c:f516:: with SMTP id t22-v6mr3925035wmh.103.1535542532216; Wed, 29 Aug 2018 04:35:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542532; cv=none; d=google.com; s=arc-20160816; b=iASNQx2r0oKu+Gi5/EMwdR+75bGzNUhTkk6SBRI+ltuxR/gQaLsUr5JNdji1D3PW9y IueO7oxGFWYO11srwcfwgIke7HBNmCo9Kl82IhY6q5bMOcitmqcnFADUyB8c3By0rkes UiVxPNElC8ZaGstZC6vII+E5BqO1DdMNFyAxtbPtjiTbb3MmoEpvac3bqBcy/LPS+LDP rqoxyASzDXQG2fAsMV0wLWkZ/u4fmZebOCfjiZlxysMtwS3LktYtyaTdU+Na64TSbMOp A1p3q3eFmpvNFdew23/e1OWss6Izpx90GyvwUCbH0pjFWR5fF7c//3dMOGd4K+ZXe2Z7 apYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=3whpCSoEraN6lbdQ6MXCL1Fj+d5eA3/FsJPvazGbB60=; b=rcXbYog1Vbz9q+bvxTDuUvniwTOleGmhD9UDMYzKgnzB8EuTEyWPtPQqqSlVBvskT0 BnIy7bq1HGMXqmgw7p9kJlVlNckeVjkETASQn5MRuGQMlSBDr56l+QRL6xUk0KoXRtuT P1R30QVn3ZhucNjGaPTr1NZQ3OYVbjGpceuSIW5XdbXx/VLaLc9+swnkVVOM6j9FBBgh X0QbZNIONOaLSC/69Vbi92Td1gV9+xKCbA107H3pKz9/yrgxOhuMmHU4zMHmXtxbmF1a BgfMyCTackZ8NSMt3/MM9aQaaoSOpzFpuF1dn1875O86inOCtKXbpPBqSkXihqPiy92c qKBw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=dUiXdqnj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t23-v6sor1036904wmh.29.2018.08.29.04.35.32 for (Google Transport Security); Wed, 29 Aug 2018 04:35:32 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=dUiXdqnj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3whpCSoEraN6lbdQ6MXCL1Fj+d5eA3/FsJPvazGbB60=; b=dUiXdqnjaOhZTeMxGC9Zfd3R5TdpNDA/LGCPEi4mg1V1trdPxjEAsm1EsI/VauH5NJ 1YYGxjy3F4QVZF+E/OosTPc9tB5jUushSiO/dhUluyoWxln478Y/WgTjGUCKTwbG+HTd MR8j2KlHUI9/axIMAedjqY/ZL3nLAdBdRpNVEKqrpftVQ2Y7GGfzStMODYb+1pt1EIOw LRwXOVvklzGSSSqVroJiGuBLoLk9V9pv467TAkJDd0DOt2SZQX9BKmrcPgalDJ2OYHHw Vn1UsWahwRaVWh8BT0fJyJiuhzXOSwY4nBFMrRjXrvfFnyd1vfNCoK4eZNAH2fgt70Ac 589A== X-Google-Smtp-Source: ANB0VdbmhfXyi5KAJje45kiWruU8TY3Sd4n7wN7WMTdGirHknjQ1LVf6zTcLMLq9ZfKwWpst1iBALA== X-Received: by 2002:a1c:adcc:: with SMTP id w195-v6mr4157393wme.41.1535542530806; Wed, 29 Aug 2018 04:35:30 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:29 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 02/18] khwasan: move common kasan and khwasan code to common.c Date: Wed, 29 Aug 2018 13:35:06 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN will reuse a significant part of KASAN code, so move the common parts to common.c without any functional changes. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 604 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 568 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 615 insertions(+), 567 deletions(-) create mode 100644 mm/kasan/common.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 3289db38bc87..a6df14bffb6b 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o diff --git a/mm/kasan/common.c b/mm/kasan/common.c new file mode 100644 index 000000000000..154010ba6c1f --- /dev/null +++ b/mm/kasan/common.c @@ -0,0 +1,604 @@ +/* + * This file contains common KASAN and KHWASAN code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +static inline void filter_irq_stacks(struct stack_trace *trace) +{ + int i; + + if (!trace->nr_entries) + return; + for (i = 0; i < trace->nr_entries; i++) + if (in_irqentry_text(trace->entries[i])) { + /* Include the irqentry function into the stack. */ + trace->nr_entries = i + 1; + break; + } +} + +static inline depot_stack_handle_t save_stack(gfp_t flags) +{ + unsigned long entries[KASAN_STACK_DEPTH]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = KASAN_STACK_DEPTH, + .skip = 0 + }; + + save_stack_trace(&trace); + filter_irq_stacks(&trace); + if (trace.nr_entries != 0 && + trace.entries[trace.nr_entries-1] == ULONG_MAX) + trace.nr_entries--; + + return depot_save_stack(&trace, flags); +} + +static inline void set_track(struct kasan_track *track, gfp_t flags) +{ + track->pid = current->pid; + track->stack = save_stack(flags); +} + +void kasan_enable_current(void) +{ + current->kasan_depth++; +} + +void kasan_disable_current(void) +{ + current->kasan_depth--; +} + +void kasan_check_read(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_read); + +void kasan_check_write(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + check_memory_region((unsigned long)addr, len, true, _RET_IP_); + + return __memset(addr, c, len); +} + +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memmove(dest, src, len); +} + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + */ +void kasan_poison_shadow(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void kasan_unpoison_shadow(const void *address, size_t size) +{ + kasan_poison_shadow(address, size, 0); + + if (size & KASAN_SHADOW_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + *shadow = size & KASAN_SHADOW_MASK; + } +} + +static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) +{ + void *base = task_stack_page(task); + size_t size = sp - base; + + kasan_unpoison_shadow(base, size); +} + +/* Unpoison the entire stack for a task. */ +void kasan_unpoison_task_stack(struct task_struct *task) +{ + __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); +} + +/* Unpoison the stack for the current task beyond a watermark sp value. */ +asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) +{ + /* + * Calculate the task stack base address. Avoid using 'current' + * because this function is called by early resume code which hasn't + * yet set up the percpu register (%gs). + */ + void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); + + kasan_unpoison_shadow(base, watermark - base); +} + +/* + * Clear all poison for the region between the current SP and a provided + * watermark value, as is sometimes required prior to hand-crafted asm function + * returns in the middle of functions. + */ +void kasan_unpoison_stack_above_sp_to(const void *watermark) +{ + const void *sp = __builtin_frame_address(0); + size_t size = watermark - sp; + + if (WARN_ON(sp > watermark)) + return; + kasan_unpoison_shadow(sp, size); +} + +void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); +} + +void kasan_free_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_poison_shadow(page_address(page), + PAGE_SIZE << order, + KASAN_FREE_PAGE); +} + +/* + * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. + * For larger allocations larger redzones are used. + */ +static inline unsigned int optimal_redzone(unsigned int object_size) +{ + if (IS_ENABLED(CONFIG_KASAN_HW)) + return 0; + + return + object_size <= 64 - 16 ? 16 : + object_size <= 128 - 32 ? 32 : + object_size <= 512 - 64 ? 64 : + object_size <= 4096 - 128 ? 128 : + object_size <= (1 << 14) - 256 ? 256 : + object_size <= (1 << 15) - 512 ? 512 : + object_size <= (1 << 16) - 1024 ? 1024 : 2048; +} + +void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) +{ + unsigned int orig_size = *size; + int redzone_adjust; + + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } + redzone_adjust = optimal_redzone(cache->object_size) - + (*size - cache->object_size); + + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + + optimal_redzone(cache->object_size))); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } + + *flags |= SLAB_KASAN; +} + +size_t kasan_metadata_size(struct kmem_cache *cache) +{ + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); +} + +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); + return (void *)object + cache->kasan_info.alloc_meta_offset; +} + +struct kasan_free_meta *get_free_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + return (void *)object + cache->kasan_info.free_meta_offset; +} + +void kasan_poison_slab(struct page *page) +{ + kasan_poison_shadow(page_address(page), + PAGE_SIZE << compound_order(page), + KASAN_KMALLOC_REDZONE); +} + +void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_unpoison_shadow(object, cache->object_size); +} + +void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_poison_shadow(object, + round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + KASAN_KMALLOC_REDZONE); +} + +void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); +} + +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +{ + return kasan_kmalloc(cache, object, cache->object_size, flags); +} + +static bool __kasan_slab_free(struct kmem_cache *cache, void *object, + unsigned long ip, bool quarantine) +{ + s8 shadow_byte; + unsigned long rounded_up_size; + + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != + object)) { + kasan_report_invalid_free(object, ip); + return true; + } + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + kasan_report_invalid_free(object, ip); + return true; + } + + rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + + if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; +} + +bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +{ + return __kasan_slab_free(cache, object, ip, true); +} + +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, + gfp_t flags) +{ + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(object == NULL)) + return NULL; + + redzone_start = round_up((unsigned long)(object + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = round_up((unsigned long)object + cache->object_size, + KASAN_SHADOW_SCALE_SIZE); + + kasan_unpoison_shadow(object, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); + + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; +} +EXPORT_SYMBOL(kasan_kmalloc); + +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + struct page *page; + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(ptr == NULL)) + return NULL; + + page = virt_to_page(ptr); + redzone_start = round_up((unsigned long)(ptr + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); + + kasan_unpoison_shadow(ptr, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); + + return (void *)ptr; +} + +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +{ + struct page *page; + + if (unlikely(object == ZERO_SIZE_PTR)) + return (void *)object; + + page = virt_to_head_page(object); + + if (unlikely(!PageSlab(page))) + return kasan_kmalloc_large(object, size, flags); + else + return kasan_kmalloc(page->slab_cache, object, size, flags); +} + +void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), + KASAN_FREE_PAGE); + } else { + __kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + +void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (ptr != page_address(virt_to_head_page(ptr))) + kasan_report_invalid_free(ptr, ip); + /* The object will be poisoned by page_alloc. */ +} + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL | __GFP_ZERO, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || + WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index f696c7c143c2..44ec228de0a2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,5 +1,5 @@ /* - * This file contains shadow memory manipulation code. + * This file contains core KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -40,82 +40,6 @@ #include "kasan.h" #include "../slab.h" -void kasan_enable_current(void) -{ - current->kasan_depth++; -} - -void kasan_disable_current(void) -{ - current->kasan_depth--; -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. - */ -static void kasan_poison_shadow(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - memset(shadow_start, value, shadow_end - shadow_start); -} - -void kasan_unpoison_shadow(const void *address, size_t size) -{ - kasan_poison_shadow(address, size, 0); - - if (size & KASAN_SHADOW_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; - } -} - -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_shadow(base, size); -} - -/* Unpoison the entire stack for a task. */ -void kasan_unpoison_task_stack(struct task_struct *task) -{ - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); -} - -/* Unpoison the stack for the current task beyond a watermark sp value. */ -asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) -{ - /* - * Calculate the task stack base address. Avoid using 'current' - * because this function is called by early resume code which hasn't - * yet set up the percpu register (%gs). - */ - void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - - kasan_unpoison_shadow(base, watermark - base); -} - -/* - * Clear all poison for the region between the current SP and a provided - * watermark value, as is sometimes required prior to hand-crafted asm function - * returns in the middle of functions. - */ -void kasan_unpoison_stack_above_sp_to(const void *watermark) -{ - const void *sp = __builtin_frame_address(0); - size_t size = watermark - sp; - - if (WARN_ON(sp > watermark)) - return; - kasan_unpoison_shadow(sp, size); -} - /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -260,121 +184,12 @@ static __always_inline void check_memory_region_inline(unsigned long addr, kasan_report(addr, size, write, ret_ip); } -static void check_memory_region(unsigned long addr, - size_t size, bool write, +void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { check_memory_region_inline(addr, size, write, ret_ip); } -void kasan_check_read(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_read); - -void kasan_check_write(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - check_memory_region((unsigned long)addr, len, true, _RET_IP_); - - return __memset(addr, c, len); -} - -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memmove(dest, src, len); -} - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memcpy(dest, src, len); -} - -void kasan_alloc_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); -} - -void kasan_free_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); -} - -/* - * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. - * For larger allocations larger redzones are used. - */ -static unsigned int optimal_redzone(unsigned int object_size) -{ - return - object_size <= 64 - 16 ? 16 : - object_size <= 128 - 32 ? 32 : - object_size <= 512 - 64 ? 64 : - object_size <= 4096 - 128 ? 128 : - object_size <= (1 << 14) - 256 ? 256 : - object_size <= (1 << 15) - 512 ? 512 : - object_size <= (1 << 16) - 1024 ? 1024 : 2048; -} - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) -{ - unsigned int orig_size = *size; - int redzone_adjust; - - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); - - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); - - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; - } - - *flags |= SLAB_KASAN; -} - void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); @@ -386,275 +201,6 @@ void kasan_cache_shutdown(struct kmem_cache *cache) quarantine_remove_cache(cache); } -size_t kasan_metadata_size(struct kmem_cache *cache) -{ - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); -} - -void kasan_poison_slab(struct page *page) -{ - kasan_poison_shadow(page_address(page), - PAGE_SIZE << compound_order(page), - KASAN_KMALLOC_REDZONE); -} - -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_unpoison_shadow(object, cache->object_size); -} - -void kasan_poison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_poison_shadow(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), - KASAN_KMALLOC_REDZONE); -} - -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline void filter_irq_stacks(struct stack_trace *trace) -{ - int i; - - if (!trace->nr_entries) - return; - for (i = 0; i < trace->nr_entries; i++) - if (in_irqentry_text(trace->entries[i])) { - /* Include the irqentry function into the stack. */ - trace->nr_entries = i + 1; - break; - } -} - -static inline depot_stack_handle_t save_stack(gfp_t flags) -{ - unsigned long entries[KASAN_STACK_DEPTH]; - struct stack_trace trace = { - .nr_entries = 0, - .entries = entries, - .max_entries = KASAN_STACK_DEPTH, - .skip = 0 - }; - - save_stack_trace(&trace); - filter_irq_stacks(&trace); - if (trace.nr_entries != 0 && - trace.entries[trace.nr_entries-1] == ULONG_MAX) - trace.nr_entries--; - - return depot_save_stack(&trace, flags); -} - -static inline void set_track(struct kasan_track *track, gfp_t flags) -{ - track->pid = current->pid; - track->stack = save_stack(flags); -} - -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); - return (void *)object + cache->kasan_info.alloc_meta_offset; -} - -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; -} - -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) -{ - struct kasan_alloc_meta *alloc_info; - - if (!(cache->flags & SLAB_KASAN)) - return; - - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); -} - -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) -{ - return kasan_kmalloc(cache, object, cache->object_size, flags); -} - -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, - unsigned long ip, bool quarantine) -{ - s8 shadow_byte; - unsigned long rounded_up_size; - - if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != - object)) { - kasan_report_invalid_free(object, ip); - return true; - } - - /* RCU slabs could be legally used after free within the RCU period */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); - return true; - } - - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) - return false; - - set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); - quarantine_put(get_free_info(cache, object), cache); - return true; -} - -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) -{ - return __kasan_slab_free(cache, object, ip, true); -} - -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) -{ - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(object == NULL)) - return NULL; - - redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); - - kasan_unpoison_shadow(object, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); - - if (cache->flags & SLAB_KASAN) - set_track(&get_alloc_info(cache, object)->alloc_track, flags); - - return (void *)object; -} -EXPORT_SYMBOL(kasan_kmalloc); - -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) -{ - struct page *page; - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(ptr == NULL)) - return NULL; - - page = virt_to_page(ptr); - redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); - - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); - - return (void *)ptr; -} - -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) -{ - struct page *page; - - if (unlikely(object == ZERO_SIZE_PTR)) - return ZERO_SIZE_PTR; - - page = virt_to_head_page(object); - - if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); - else - return kasan_kmalloc(page->slab_cache, object, size, flags); -} - -void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), - KASAN_FREE_PAGE); - } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - -void kasan_kfree_large(void *ptr, unsigned long ip) -{ - if (ptr != page_address(virt_to_head_page(ptr))) - kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ -} - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); @@ -795,113 +341,3 @@ DEFINE_ASAN_SET_SHADOW(f2); DEFINE_ASAN_SET_SHADOW(f3); DEFINE_ASAN_SET_SHADOW(f5); DEFINE_ASAN_SET_SHADOW(f8); - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c12dcfde2ebd..659463800f10 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow(const void *address, size_t size, u8 value); + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); From patchwork Wed Aug 29 11:35:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 020191709 for ; Wed, 29 Aug 2018 11:35:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E587C2AB91 for ; Wed, 29 Aug 2018 11:35:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D92AE2ABDA; Wed, 29 Aug 2018 11:35:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C4EA72AA9C for ; Wed, 29 Aug 2018 11:35:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4573F6B4B7A; Wed, 29 Aug 2018 07:35:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2F9F86B4B7D; Wed, 29 Aug 2018 07:35:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 176AF6B4B79; Wed, 29 Aug 2018 07:35:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id 6EFA76B4B79 for ; Wed, 29 Aug 2018 07:35:35 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id b186-v6so2735893wmh.8 for ; Wed, 29 Aug 2018 04:35:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mPC1QTpjYa89N9Ou1rQHacuwcgon7c20SdDkLma9mEs=; b=GXitJjrs/yMaXOPNX3jMkcdVfkSPHxSgzXR5GlM0G5SZ3MEvwstiUSWiNLfQJfUDWl inH/5Lh1QDZzBENXoI3bhlrLwQ+SdbSqs5QQKr9HTAUkso4QPyfi6SIhn6ZHXotFvcLZ cW0ssA6T7WaVNKxSeRMorxrE9dUgVDvacddHMF73DZJYQg+HybLfiqQDJzXFrDRwte5Z rd4IWj54xX0DuiKCEtlM39fjEdL7n9UU0gWxCcdV7IwqsvQcwYO7D8yoxpaLso5V6nJZ OYiZIFTDfqBaWtgfLj+NzSV2k3843zBLVFma0rb6F4oUkQTMepaN+nLRhQBFL9tEofJ9 Vr8w== X-Gm-Message-State: APzg51At2JaMAVKNn6KQjiz5fF9Uo+xtmNYq/M7Bj71pT6u8H2b9Hcts WM90DzAxPjf7PGvWyH6XjJNcPLqUTGclWM34fnOrWvyKqO9zllh1BzeJ4hConcmUbFi8m+guO5I pwuwht9QEWUekI2ZHKZnI+u/daR+kYheR9Iazggw8MUmWhEC1LwuqYHfFy4nsR+tHdc5hrjegOf 0ak0pBDN47LZ6Pq7R0xvpMKtTWntOArdkijR7dE3DfMmc8in/TQxppWXFZ12TjKNgOW6mHPEzx5 3UWOsvm8Hu4+Pb09As1nAnhrFpy/tj48Ry4pohtzYhxLM3U4RA/5V04pRWg3x4fYMYIuvByLNDM QutvlGV7jp8QvhQhsP4N3yK4KQKwbtigKIaudkT6EeeDFE+jOXHZQMGuqC8FYW6xR98Tp7WNALP O X-Received: by 2002:a1c:7fc6:: with SMTP id a189-v6mr4102064wmd.42.1535542534854; Wed, 29 Aug 2018 04:35:34 -0700 (PDT) X-Received: by 2002:a1c:7fc6:: with SMTP id a189-v6mr4101997wmd.42.1535542533398; Wed, 29 Aug 2018 04:35:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542533; cv=none; d=google.com; s=arc-20160816; b=l2PQGFnI8zCBkmt6Gjypt/ayMrcdfB0VVL3tU8iBDbBKYUBs7rx3pVfJgVOtGLfL4k 26kClmHOPpiUOEbGr9ZQC0m8+5pRa5Yhk2npzyywtk44eGi9zmnfII/SbQHzteFcCnR6 RgVhjSmjR5ZlyNs5o2ozO5AhtNMNxtscxJA8jfLK4tpUYgGD8GnFWjSujRUiiBls2EEA XrDf2TU1vHeFG5sg5fYwEpqj0UtzpCL13pjCOfVjzsm4W9o4tUXFu0aUBrMaZanKGtFn yihptIczWelgHxeKTRgHAvPyMQua633o55UeSaILvFV82DBFRfrxcL3hB+rPYluhqkZj 0cqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=mPC1QTpjYa89N9Ou1rQHacuwcgon7c20SdDkLma9mEs=; b=kbvM+1jjgDAm2fF3r0JoFgDAXGKC5U9r0cP/VrNVxCo7da3eM4Ig7H9w6DYeC7SFNQ UK5n/GB59JiG+DCD6kZ0l/Jcid7v9rJ7Zo3nQyCx9RI5al0dakTHCVhRm6LyEYhb7mc4 iZ7K0aAFWskJbR7DJ4a0Ovx9MFwxkfTwf0D0zgX0fa7+LX3ypoIrz9L+43+dmlmZtBhL 7wEV9as6+yCzPLzbwLFz8TwRHemyvUhLMmWfKf2K88k2i6hVV2YH5/m9X+khG7WDTc6P rS7c6gBgd33JurifjsJ8tOqpcflOXnJMhkM8rIC4xVqfbzafoWHrvtwl142M7pNXaRc0 pJNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XLNRmErD; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l128-v6sor1153272wmb.53.2018.08.29.04.35.33 for (Google Transport Security); Wed, 29 Aug 2018 04:35:33 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XLNRmErD; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mPC1QTpjYa89N9Ou1rQHacuwcgon7c20SdDkLma9mEs=; b=XLNRmErDpLBhaR+2xjmkMJHtERUQnkLkO5a+oCsPQ1I7lXn8wIGx3FGDTlLddI6eYx 701O0+Pt/ak5lk0FQnK9L4oTduuUj8JuAcAaSuz8ecVONegRizmMRGR8xb5D74fgRbOL O6PsYBdlD1i7zVZUnttLhYJHdLYe5YeKHctqv9uV61PwRnfzPgwdtA/V/AMlAszGW9XZ wEj3pLpCppMawv6PS+UhcZ5khv98EnW+3230yvs0eKjitEbYzwIQ3jXpK6Gmih6VQZ9h yNDbjp82r8HlP4Xm5/zaBmEKFwdL9vsgRV+sZP4FQdGCm4+oOKb7bqvlWL7exjLFHpQg Wo2w== X-Google-Smtp-Source: ANB0Vdba/9/+Z4oI1RcG6EqOmTDFZaDf2YDEDCpFqcujQZw+oxIp6JwGAneRxgMpTlHOu3D4L+IqMg== X-Received: by 2002:a1c:2dc8:: with SMTP id t191-v6mr4162121wmt.94.1535542532431; Wed, 29 Aug 2018 04:35:32 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:31 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 03/18] khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW Date: Wed, 29 Aug 2018 13:35:07 +0200 Message-Id: <868c9168481ff5103034ac1e37b830d28ed5f4ee.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit splits the current CONFIG_KASAN config option into two: 1. CONFIG_KASAN_GENERIC, that enables the generic software-only KASAN version (the one that exists now); 2. CONFIG_KASAN_HW, that enables KHWASAN. With CONFIG_KASAN_HW enabled, compiler options are changed to instrument kernel files wiht -fsantize=hwaddress (except the ones for which KASAN_SANITIZE := n is set). Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes. This commit also adds empty placeholder (for now) implementation of KHWASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options. Signed-off-by: Andrey Konovalov --- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 3 +- include/linux/compiler-gcc.h | 4 ++ include/linux/compiler.h | 3 +- include/linux/kasan.h | 16 +++++-- lib/Kconfig.kasan | 77 ++++++++++++++++++++++++++-------- mm/kasan/Makefile | 6 ++- mm/kasan/kasan.h | 3 +- mm/kasan/khwasan.c | 75 +++++++++++++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 27 +++++++++++- 11 files changed, 188 insertions(+), 29 deletions(-) create mode 100644 mm/kasan/khwasan.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 29e75b47becd..991564148f54 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -105,6 +105,7 @@ config ARM64 select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_HW if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index b1ce500fe8b3..2c258a9d4c67 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -17,11 +17,12 @@ #define KASAN_ABI_VERSION 5 /* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature(address_sanitizer) +#if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) #define __SANITIZE_ADDRESS__ #endif #define __no_sanitize_address __attribute__((no_sanitize("address"))) +#define __no_sanitize_hwaddress __attribute__((no_sanitize("hwaddress"))) /* * Not all versions of clang implement the the type-generic versions diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 763bbad1e258..a186b55c8c4c 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -227,6 +227,10 @@ #define __no_sanitize_address #endif +#if !defined(__no_sanitize_hwaddress) +#define __no_sanitize_hwaddress /* gcc doesn't support KHWASAN */ +#endif + /* * Turn individual warnings and errors on and off locally, depending * on version. diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 681d866efb1e..3f2ba192d57d 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -195,7 +195,8 @@ void __read_once_size(const volatile void *p, void *res, int size) * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 * '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused +# define __no_kasan_or_inline __no_sanitize_address __no_sanitize_hwaddress \ + __maybe_unused #else # define __no_kasan_or_inline __always_inline #endif diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 54d577ad2181..beb56a26fe9b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); -void kasan_cache_shrink(struct kmem_cache *cache); -void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); @@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} -static inline void kasan_cache_shrink(struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, @@ -152,4 +148,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink(struct kmem_cache *cache); +void kasan_cache_shutdown(struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink(struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ + #endif /* LINUX_KASAN_H */ diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index befb127507c0..5a22629f30e7 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -1,34 +1,75 @@ config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_HW + bool + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs. + +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: KASAN (a classic version, similar to userspace + ASan, enabled with CONFIG_KASAN_GENERIC) and KHWASAN (a version + based on pointer tagging, only for arm64, similar to userspace + HWASan, enabled with CONFIG_KASAN_HW). + +config KASAN_GENERIC + bool "KASAN: the generic mode" depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables the generic mode of KASAN. + This is strictly a debugging feature and it requires a GCC version + of 4.9.2 or later. Detection of out-of-bounds accesses to stack or + global variables requires GCC 5.0 or later. + This mode consumes about 1/8 of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +if HAVE_ARCH_KASAN_HW + +config KASAN_HW + bool "KHWASAN: the hardware assisted mode" + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enabled KHWASAN (KASAN mode based on pointer tagging). + This mode requires Top Byte Ignore support by the CPU and therefore + only supported for arm64. + This feature requires clang revision 330044 or later. + This mode consumes about 1/16 of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_HW doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer - compile time. + This enables further checks in KASAN, for now it only includes the + address-use-after-scope check that can lead to excessive kernel + stack usage, frame size warnings and longer compile time. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more @@ -53,16 +94,16 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endif diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index a6df14bffb6b..14955add96d3 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_khwasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg @@ -10,5 +11,8 @@ CFLAGS_REMOVE_kasan.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.o diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 659463800f10..19b950eaccff 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB) +#if defined(CONFIG_KASAN_GENERIC) && \ + (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c new file mode 100644 index 000000000000..e2c3a7f7fd1f --- /dev/null +++ b/mm/kasan/khwasan.c @@ -0,0 +1,75 @@ +/* + * This file contains core KHWASAN code. + * + * Copyright (c) 2018 Google, Inc. + * Author: Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE(size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE(1); +DEFINE_HWASAN_LOAD_STORE(2); +DEFINE_HWASAN_LOAD_STORE(4); +DEFINE_HWASAN_LOAD_STORE(8); +DEFINE_HWASAN_LOAD_STORE(16); + +void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_storeN_noabort); + +void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_tag_memory); diff --git a/mm/slub.c b/mm/slub.c index 30b9bf777bab..4206e1b616e7 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2955,7 +2955,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page, do_slab_free(s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr) { do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr); diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 69552a39951d..49c6e056c697 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else @@ -42,6 +42,29 @@ ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif + +ifdef CONFIG_KASAN_HW + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +ifeq ($(call cc-option, $(CFLAGS_KASAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KASAN_HW: \ + -fsanitize=hwaddress is not supported by compiler) + endif +endif + +endif + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif From patchwork Wed Aug 29 11:35:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579939 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99CB41709 for ; Wed, 29 Aug 2018 11:35:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 861282AA9C for ; Wed, 29 Aug 2018 11:35:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 756242AB91; Wed, 29 Aug 2018 11:35:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEB6E2AA9C for ; Wed, 29 Aug 2018 11:35:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 830626B4B79; Wed, 29 Aug 2018 07:35:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 79A936B4B7C; Wed, 29 Aug 2018 07:35:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E7176B4B7C; Wed, 29 Aug 2018 07:35:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id F04036B4B7A for ; Wed, 29 Aug 2018 07:35:35 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id b17-v6so3346092wrq.0 for ; Wed, 29 Aug 2018 04:35:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Oho6JdHma26yH9i0K5sb+l4Yb9Bpt1LjR+gOl5dkBUo=; b=lcyw7ekqod2UF5OWBHoe8iTORAy3X3kDe1xII0R0YGWKUe3jJnap6yOh+omjRPpGid Em8egqkx9V/MNwPoHzMsijF/KyC5H6+Gq7sRYzSiJg0AGhfPGKq9MI8LxuAeJpllHhen eauZWVIGSoU/5uFLSLvaZDaiasSazRLwu4tfI+gJQIS6G//kQ9VkZ22tCeKUD52mnwXQ 4mX6NqBJMPpT5Y7VebJNBA0QqgDFItB5r3/81jFLqLeWVqSKZIOXMP4K4V2vD6GLJq7u ZhuqcUBN0wh6s8hD5rFB13F7XNPZzfjbM3/fni9IOcmMhYdZy5E17T5BK30xuwQUuNVO mCwA== X-Gm-Message-State: APzg51BfkP7nRv9l441vUibMUIkulScL0B1DydzzcBBsXuwWHSIDakpM lbcCCCAJ9VdVGjpKH5fi+FSzYNCB8wJc96YS8c+96IfmrbCO0rsGksgvAbMa1WYcPAIRsbu4Wdw 0+uzyejrXD/O3W0I4s2CfAMJNQVRu44zWecSWLGL+9tlafpBeXcLNkkee2FolpRSE0sVB3T0IdH /WF3o3BwQDd/3iOuF8D2VUdbIXknHtN84kP5PUU9o+oTBbyDcllz9AHXvdijlF47G/p2wd7DwYU mfPR3x0oAt+ISO+Ig1PK3W6ClzKBEtiDUitdvTGLUypH6f1dAmn+OIbGo/XXctTf58z9S43L+ie sZyBLBYla5KT3ExPWRcajTw1ClEUoauRxzZRwGpEuyj3DpI7yaKftAz3KfnqT+9prShKZ3btGed 1 X-Received: by 2002:a1c:c64f:: with SMTP id w76-v6mr4127995wmf.3.1535542535463; Wed, 29 Aug 2018 04:35:35 -0700 (PDT) X-Received: by 2002:a1c:c64f:: with SMTP id w76-v6mr4127967wmf.3.1535542534680; Wed, 29 Aug 2018 04:35:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542534; cv=none; d=google.com; s=arc-20160816; b=SJnGPeit5GHDqT7pJpuqcYI+jNNFOAJ4uvomYO81cXW8lkP49kLt5HTNi3qPTEMxb4 yibRt1reT+75YuO+rLVndEDwmxK+mk2R2HCS4v+Vq/mGTPlfcNebt9Ou+LkghwJRj8ui tT2KiB2xWS4O0K4/r32QEl1ioZN0i08VrHUeXipw/fkhjxghz0j3BmalQTnWKH1PxJTk 22Cu1K653c3MfeZ2GSUlLxz/ohUOAAZMOIri5wPYGp4cK8XfmgO6wlLAlGNOmjfNIedf F6qjUF1efzFK/c7aDb8hlweNeAaksv+bysDXGndpnU2IeaGVagsaUkGwBRoNlKLhGv/Q IieQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=Oho6JdHma26yH9i0K5sb+l4Yb9Bpt1LjR+gOl5dkBUo=; b=OYOVIczOpUKj8h2Vb7XJecn9gG0jzmPXJr4Vgfcvma8sPeJH9MCoiRjAQ7parxSlOx pbohXcmq5FRNuJXEGDpjpWiOcAP/jrGWxRiTc2fL0uNGkRWn0Hzb4yV1a5LvnMYzb2q+ uhcNEsq4ihf7iHzGiNYLCA2hyK52BlHi/XPtlGoXyZlWRC2e0UBukpFTmpNQtz0bvwpR 3a6esF6Wr0gOan+RHRaeR9ru3je4xvCatlrhZR0ToOceYUGXrhQl9/QF3X2b1gKx/br4 3jJ1DOn7LG1Rnd+UXpVmDdHTUDYhFzwpeqOr987rehuXHlTY9CYsVidmK3G99P3EusrS RnJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=SaDmWZ9f; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id z135-v6sor1138196wmc.2.2018.08.29.04.35.34 for (Google Transport Security); Wed, 29 Aug 2018 04:35:34 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=SaDmWZ9f; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Oho6JdHma26yH9i0K5sb+l4Yb9Bpt1LjR+gOl5dkBUo=; b=SaDmWZ9fK2PewT8oJonwMmFfr8WhrgkwKC1k18mnNmJmAsCXOB3epwtFKQCa9bJOAi mEj8+WOJp2nZjwSiGScw6Xf5tBPmT1MpzndcX+Y+Ct0cNtljF9Ux6NW+JhEZSOSdgrxU yvYeNx0zRXThLLrsda7sUxBeiipmmCaRpWw3EfQPRM0nzAAWxcYpVEaZsynW76N+kdWG uuyyKkgeP3GLgqci7gdynMTo3MsxqG1n2vZYlR2lP32Vi0+iyNDuLjgbLH4od9gAkPMz p3j/Y1rMM2RUu+jDYRBgy/F6h8F3cZXPVHFrtsUqaArAhctqLIcRmQaMiHZ5DTmF+235 K9Dg== X-Google-Smtp-Source: ANB0Vda/VxaGGcnpqyl9kXA7HPIGNrg+oVCYhSMZPHQ7NBy/4oLuVzESIGyzYz5syIS6IPDHguMu6g== X-Received: by 2002:a1c:7fc6:: with SMTP id a189-v6mr4102021wmd.42.1535542534055; Wed, 29 Aug 2018 04:35:34 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:33 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 04/18] khwasan, arm64: adjust shadow size for CONFIG_KASAN_HW Date: Wed, 29 Aug 2018 13:35:08 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KWHASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory. This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when KHWASAN is enabled. Signed-off-by: Andrey Konovalov --- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/memory.h | 13 +++++++++---- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 106039d25e2f..17047b8ab984 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -94,7 +94,7 @@ endif # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 +KASAN_SHADOW_SCALE_SHIFT := $(if $(CONFIG_KASAN_HW), 4, 3) KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b96442960aea..f5e262ee76c1 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -74,12 +74,17 @@ #define KERNEL_END _end /* - * KASAN requires 1/8th of the kernel virtual address space for the shadow - * region. KASAN can bloat the stack significantly, so double the (minimum) - * stack size when KASAN is in use. + * KASAN and KHWASAN require 1/8th and 1/16th of the kernel virtual address + * space for the shadow region respectively. They can bloat the stack + * significantly, so double the (minimum) stack size when they are in use. */ -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_SHADOW_SCALE_SHIFT 4 +#endif +#ifdef CONFIG_KASAN #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_THREAD_SHIFT 1 #else From patchwork Wed Aug 29 11:35:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEA581709 for ; Wed, 29 Aug 2018 11:35:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83E962AA9C for ; Wed, 29 Aug 2018 11:35:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 764572ABDA; Wed, 29 Aug 2018 11:35:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCB912AA9C for ; Wed, 29 Aug 2018 11:35:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 656666B4B7C; Wed, 29 Aug 2018 07:35:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 607CB6B4B7E; Wed, 29 Aug 2018 07:35:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 484C66B4B7F; Wed, 29 Aug 2018 07:35:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id CD9A36B4B7C for ; Wed, 29 Aug 2018 07:35:37 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id r14-v6so2743205wmh.0 for ; Wed, 29 Aug 2018 04:35:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=y36bXvOzaMElAwP/XOX/nFNmvhGdVKzgjaE28JUMW5o=; b=WUMythkGChn6WLS0ib0xFv3WtyM/x/zob2qmRKNVX0o3e6cGJBdwQ65r9XhIenhIle /bnCzfZ6AvnU3GmVwsnJ4t8oCbYtHfG4qh+tUIcx+TrBjUfPvZy02+q6RRDV+gcy0Kjd v3B4QEGdo40pKCXPcxJJLjhOvpRSvJT/Jhjjuhjb4pAbRgFrUN9XbbBh8ZR5KzOCjfVt KN/nsp+ChD6PWjss6YO93MRrSsYSSRNk5DTnz++u4LMpkK+a6VqLDgSu/HVHd9x/wzi/ XH4yoiGIgRJMzms4bKMbP+sGlW6iPG6cfKYGss7OT60fU2YxeuuPJdYv9YsvfbFizFm2 NunQ== X-Gm-Message-State: APzg51BJOnY9dbFZ4pQi0S55sfrQjwWSzxdKpApi3dSwPhkt9aq9pXM1 a8um0Uht5K0jiBSOyxIQLSsBRmoRQIELREIRClqL9sQiX1pIsPb+7fJ9Lv9h8xHRdQ06Bj24Kr9 IMEBvsZlPNVRzGFte5tIKMlqPKqub17gafJpwOPev0xGxINVAW8LahJ+z++sH9/pcgUcbNHAbyv iBBQR8yvBkRVEWdeKd9LO1VVR7aBvb45LyZInDDWM87kSv0jo9u6/zX9vdX8zmgqi30VxiNdw8l NvTTssVq3tOCPHvIpzkv8FlTfkJzeRa0Lzg/0koNL8+X+oyl3tjk9mszSKloJdTWenmrB5cIxVP 0BgoVU2Vmao2HPsoUI8Ti2wRXzrf2XWDToOEC/zZu7NDEYI9ykD4667wAwjIo0Yc2nB6c2g2Vyj D X-Received: by 2002:adf:ed88:: with SMTP id c8-v6mr3983550wro.264.1535542537313; Wed, 29 Aug 2018 04:35:37 -0700 (PDT) X-Received: by 2002:adf:ed88:: with SMTP id c8-v6mr3983513wro.264.1535542536512; Wed, 29 Aug 2018 04:35:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542536; cv=none; d=google.com; s=arc-20160816; b=vgvLNPBiG5dk6Qe3lTbZVRTdVgSHAqqVb0xx+m4zvKepUxEOONayliHK77wWz+ituB JBUmYRcRVUg543EB5FP3ckTdXaGRRPxKRJDP+LOuoZBCC2dRtF/IdEsjc7hYCUw5JnOK gFkAXUKXypbzqCrG6l9uYTd3/0DWLeBcEU4zp6CuOuUfRkXmvJHU6OweLMsRPNubWvrY EogPgF0s4h76Yv1NDvPTJbys0zFhrb0YIn8rlT7P8whro2nLRdocE+JJn7GFTXxdx7vB LQ1/lr5JyFU3NCJipK8mnjoyJJF41TVGFRb2YWzZ1AiyLRb46sHTOQK78Hnf69k+lvQ4 k0pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=y36bXvOzaMElAwP/XOX/nFNmvhGdVKzgjaE28JUMW5o=; b=SJLIgMPO40RqFnS7OiqdDbcJ+PsLPDLyafWYf+f+CvYkZHTPJYpWwZ46H2xrOmagVz pOU1z8xp324jfpGkWP78bbRQDuU/wLRTeUo2fJ2LAdL4njP3nc7E9DNFxRNvLAhzmjhB u4f4rIfjftHloy7pwfj37D6V/JneAb3HyPANQ2YsGhrvHkA7UQlMFVe6zzYYhOmN1gPY ExKZ0UnI6Oky1GMbHvDzw3bjZxf60+SnjYV9rDBrKYuq9W+22PYxfZJtmgYpt1bRhN1s UUHyJlJaR0YuE0IMYlIDewznkqrz7o3cIZ0Z06Z+Thbhxn49OWnto9NhxTA4of6c9ZGd batQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=J9lyKo7V; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v185-v6sor1102425wma.67.2018.08.29.04.35.36 for (Google Transport Security); Wed, 29 Aug 2018 04:35:36 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=J9lyKo7V; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y36bXvOzaMElAwP/XOX/nFNmvhGdVKzgjaE28JUMW5o=; b=J9lyKo7VWJ5jsaGwy1wNWax/WJQTvOe9FhDYud9wWj93OpsSu7DwWZsSZXOKoBVILO KPAyje7e5AQ7hfO8pvsIRzbA08oZTVo2DiKUtQXVxd+G8NDg5TtVpy2SWtuseVEcnGvG evNNyx49Q6IjYcQunMadL9ZRoGuPEWDHW8HnvBgxXuM3qx9w/KVTctwf/CAlAHO35Lyb MLUraaA2zKF6IR/MRT7GjoYhU+q8I0cNEAas1+bqk+CoYXEosJnObP4rz2Tj9gxsSIRh D4DFSGIABffoGHKHQPDQsFIniBny68STd7ZdYFa1kbbnspEUWtSqw/eLoqBMT6Ax9vJy JxAg== X-Google-Smtp-Source: ANB0Vdat/kRDBTM8zngqNeGsn+KvYphqXvjyZ5bXDZLZHa+jSCgaLJ+GwsYRrGfyHH7Np21KuGVweA== X-Received: by 2002:a1c:1bca:: with SMTP id b193-v6mr3968967wmb.6.1535542535574; Wed, 29 Aug 2018 04:35:35 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:34 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 05/18] khwasan: initialize shadow to 0xff Date: Wed, 29 Aug 2018 13:35:09 +0200 Message-Id: <4dcf8e38bf6ef331e351f0ba4a76c12666f9fc08.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A KHWASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with KHWASAN we need to initialize shadow memory to 0xff. This commit does that. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 16 ++++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 12145874c02b..7a31e8ccbad2 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -44,6 +44,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) return __pa(p); } +static phys_addr_t __init kasan_alloc_raw_page(int node) +{ + void *p = memblock_virt_alloc_try_nid_raw(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, + node); + return __pa(p); +} + static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { @@ -89,7 +98,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, do { phys_addr_t page_phys = early ? __pa_symbol(kasan_zero_page) - : kasan_alloc_zeroed_page(node); + : kasan_alloc_raw_page(node); + if (!early) + memset(__va(page_phys), KASAN_SHADOW_INIT, PAGE_SIZE); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -139,6 +150,7 @@ asmlinkage void __init kasan_early_init(void) KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_START, PGDIR_SIZE)); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, PGDIR_SIZE)); + kasan_pgd_populate(KASAN_SHADOW_START, KASAN_SHADOW_END, NUMA_NO_NODE, true); } @@ -235,7 +247,7 @@ void __init kasan_init(void) set_pte(&kasan_zero_pte[i], pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO)); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_zero_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index beb56a26fe9b..1c31bb089154 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -150,6 +150,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 + void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); @@ -160,4 +162,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_HW + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_HW */ + #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 154010ba6c1f..bed8e13c6e1d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -474,11 +474,12 @@ int kasan_module_alloc(void *addr, size_t size) ret = __vmalloc_node_range(shadow_size, 1, shadow_start, shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, + GFP_KERNEL, PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, __builtin_return_address(0)); if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); find_vm_area(addr)->flags |= VM_KASAN; kmemleak_ignore(ret); return 0; From patchwork Wed Aug 29 11:35:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1B92E1805 for ; Wed, 29 Aug 2018 11:35:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D2A62AA9C for ; Wed, 29 Aug 2018 11:35:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00E4F2ABDA; Wed, 29 Aug 2018 11:35:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 952122AB91 for ; Wed, 29 Aug 2018 11:35:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 803966B4B7E; Wed, 29 Aug 2018 07:35:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7DCC36B4B7F; Wed, 29 Aug 2018 07:35:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 592366B4B80; Wed, 29 Aug 2018 07:35:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id EF83D6B4B7E for ; Wed, 29 Aug 2018 07:35:38 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id z23-v6so2816865wma.2 for ; Wed, 29 Aug 2018 04:35:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ScyVG8/8R+wHPT40gRo57Kpvm4BCgl84gWeZYScmJuw=; b=JAoKGCcWwvUk6RgKm/AEAu4M+ce41CBBImCo5qmTZCJFYoTN4W7vAt3v83WPZ6arTF 3btTC/50M/x4oikuTFRqLrKWBj/CSrT3Z+HkB/fSHapwFjvCbdX0DG+uZ0Q5qLHEhpxI yJFNBSr2hjcLdUtiDh8WUBkh/qHIlQM2ouzDr3ZGjnlXajlc+my+pgbMrELMdmciVnVt 3w4maMLrdq5bUIFcVZUeqH/+N6MH7/4u0Xbyr2LgniwD0e81qq8Wk/VxlbkM+YJTzk39 XXwOEFfofhRrJHK+YlXtv1sY7l7SLx0DQi7qsbQsfggqkvlBS8gN5xaD3K+eWseVwbHJ 8KZw== X-Gm-Message-State: APzg51AZh7hSIubdr370j6a6U4wftMpV1Kb/8c3sVqUK0R9172mQLWj4 A6njJiBsEMF92KHlw04i0H8usUSRoL/UPEHop/8x4iTjN0J8DEAuBt1hXpYECWY9wVnOuDmNW5J jk7MvRr39TT7hZmhddyFpN85DOPODh47fAUwfa5RM3/RQteRvL7G7ogxG1jkVFrG/hXEme8hU3U DsbEbOHZd/VHuwMYRpC3JKY3PbZVd/vIX8Qih7T4p9CKU+CKoe6zGRl7HLhOIkKKcsSfAk1P4I1 4ZRfqxEYBxG8Bi7tFQfBKsEHcWjFRYpY9j9U85zdfsuTHSq9u3ytmAo6d8rqH0Pmm8o12xda3uq EqdCL9Hc49xi3VLbHMntaXMUNI55xNg87FButYpIcQm8UNOG4NLOK3rnx1CTMtMvpc4AybMK4Qr b X-Received: by 2002:adf:c454:: with SMTP id a20-v6mr4009512wrg.20.1535542538514; Wed, 29 Aug 2018 04:35:38 -0700 (PDT) X-Received: by 2002:adf:c454:: with SMTP id a20-v6mr4009485wrg.20.1535542537862; Wed, 29 Aug 2018 04:35:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542537; cv=none; d=google.com; s=arc-20160816; b=yxOmP9yLRYarQvlnUs/4BVo4uvpGDq44Nu1n2IxEU2USC9Kka/YkTJhx8t20NurRIe LPHXbdZPxgTckH8cUUl6uuUMV7tZ6SDkSY0MYQtJSp7U8WOA5dZTA+yiWj7QN5DPm1xz S7lH/M5n6sd5CeR2b4paX/skG3MKPBqmljsldTiYQdgTVir5wOONuDjNqSfD/15GHf7t HhWtCbKYYA2g3jVpRMuEXypeE1Yn5eYM+WTvoZICIOwShxpr8/jk3uNLeAd9FT+aeC8Y KWhkk+1i/9BZ8E2ZrXfMzLWirxNMlUI+F4HrVfJgHEkfM/y8YXQ/uccf6eWY5A/kcxPB gzaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=ScyVG8/8R+wHPT40gRo57Kpvm4BCgl84gWeZYScmJuw=; b=CGP4dGHoNFpFzurtHjXrJpjlZBK64GQkDE+EVNGIomf44UJObjeTKS5leMQeybGNtN 7GaARM+fkpCCHqRPJaonC6ZE0GnfUogSN6ejEXaPdku5zegMLlCc5ARjS2TH1USEWLgo CT6Ihb8MOdB4bcI1U7z0noa50Fmt1isDcM2GtqCHYFzXj3IzsWh1SGfLzz9+KTD7a7Wc FeuvU1UHDQpURdfp9LXXOiT5T3M0qXMBq9e2Pu15gonOIysDQrk43rVuF77xGnBULwRS o3qhe2Ofdh/jbPrMzznzP2YfzSgu8OWiFJJHf7hPLbbuADKUj/XXMohEQ3cXMyJoagI8 RDjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=wTs5ODdO; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id s202-v6sor250704wmd.91.2018.08.29.04.35.37 for (Google Transport Security); Wed, 29 Aug 2018 04:35:37 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=wTs5ODdO; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ScyVG8/8R+wHPT40gRo57Kpvm4BCgl84gWeZYScmJuw=; b=wTs5ODdO4PrsRaq9SSLTEZ7b10s0WzDZC6bvpJQCQ8IlNuv8Tm13ZusPLCWI9u5B6h dnuPVb6nNahHsbJlKRNbyTLXNJ/IrGoLuhN0DBBkhShgRKNswPDo36xTMxq9+X8I7ByY Z81NYIk6kveEH6yEiQK5WwgIHPjxiGp1gTsfDHqRN4/FjAOY4vYjr/1t6prtIAlo4x8y C9VP5vf6Ywf4tNQ72tAz0ab4zeeMok9lxjqUAxNz9ZlYoT8Dwgd0SsM6S5SRkdvlI+fi kBtB/ZVfI13XLo2seU5UAo9VpCDwoSqBWnOlSmXFjGovS8hYAI20T+2ppjFYW1flvuWs kqxQ== X-Google-Smtp-Source: ANB0VdZKr6B7zINhz15rEPoFBPLoEFhgTfNWmBJ1/WdmOFNn442wB60NAMI8yonkTKESWF427Y1E1A== X-Received: by 2002:a1c:1182:: with SMTP id 124-v6mr4036911wmr.75.1535542537152; Wed, 29 Aug 2018 04:35:37 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:36 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 06/18] khwasan, arm64: untag virt address in __kimg_to_phys and _virt_addr_is_linear Date: Wed, 29 Aug 2018 13:35:10 +0200 Message-Id: <19d757c2cafc277f0143a8ac34e179061f3487f5.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP __kimg_to_phys (which is used by virt_to_phys) and _virt_addr_is_linear (which is used by virt_addr_valid) assume that the top byte of the address is 0xff, which isn't always the case with KHWASAN enabled. The solution is to reset the tag in those macros. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index f5e262ee76c1..f5e2953b7009 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -92,6 +92,13 @@ #define KASAN_THREAD_SHIFT 0 #endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_SHIFTED(tag) ((unsigned long)(tag) << 56) +#define KASAN_SET_TAG(addr, tag) (((addr) & ~KASAN_TAG_SHIFTED(0xff)) | \ + KASAN_TAG_SHIFTED(tag)) +#define KASAN_RESET_TAG(addr) KASAN_SET_TAG(addr, 0xff) +#endif + #define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT) /* @@ -232,7 +239,12 @@ static inline unsigned long kaslr_offset(void) #define __is_lm_address(addr) (!!((addr) & BIT(VA_BITS - 1))) #define __lm_to_phys(addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET) + +#ifdef CONFIG_KASAN_HW +#define __kimg_to_phys(addr) (KASAN_RESET_TAG(addr) - kimage_voffset) +#else #define __kimg_to_phys(addr) ((addr) - kimage_voffset) +#endif #define __virt_to_phys_nodebug(x) ({ \ phys_addr_t __x = (phys_addr_t)(x); \ @@ -308,7 +320,13 @@ static inline void *phys_to_virt(phys_addr_t x) #endif #endif +#ifdef CONFIG_KASAN_HW +#define _virt_addr_is_linear(kaddr) (KASAN_RESET_TAG((u64)(kaddr)) >= \ + PAGE_OFFSET) +#else #define _virt_addr_is_linear(kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) +#endif + #define virt_addr_valid(kaddr) (_virt_addr_is_linear(kaddr) && \ _virt_addr_valid(kaddr)) From patchwork Wed Aug 29 11:35:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579955 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8846F1709 for ; Wed, 29 Aug 2018 11:35:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75EB42AA9C for ; Wed, 29 Aug 2018 11:35:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6997D2AB91; Wed, 29 Aug 2018 11:35:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8B612AB44 for ; Wed, 29 Aug 2018 11:35:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 87ADC6B4B80; Wed, 29 Aug 2018 07:35:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 803C56B4B81; Wed, 29 Aug 2018 07:35:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65C646B4B82; Wed, 29 Aug 2018 07:35:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id F03366B4B80 for ; Wed, 29 Aug 2018 07:35:40 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id s205-v6so2725457wmf.7 for ; Wed, 29 Aug 2018 04:35:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kb0anrSFrq4OO5HLjnBi1Wtd8/F9idg7tFDv1gOHdc4=; b=D14tbdUbAgbdUGn5qBtdD2FlbUd/cUuQZfdeUzXunQOctHr4bUmgNNa9yZ2DaUMWWW Oaqca5NbWEadjDw5/SyjYK+1cfvTiMNTzYk5qTFLHTkLBwsa/fSu/A3dWy9GTf3Qw3gn l221PWLXvmbB2181tXCwx3WYU1cB3VhtkVDkoD+TJEMcO8n90XovnMAIvdw9BgiS9YF3 H0rs7cfTLCQFotYwSXD2b7cdAKpgmgJa1jXAf9UNKWGZrl/J3rfgOa0RjX4J9jZPhDxp SDF9cIDp3h4UYyR9V6XCMkVIHgOcAwTdb3n49MSyNSTsuMmAEEcPDfTIVCKhmxfB9FbG plUw== X-Gm-Message-State: APzg51DoIw0TzcOegPjGsytD7jTaP+i3j+0lEBE2yDIRzIIzoUxE8Q7c 2MkNDaGWoG8RtOzvhFiYVECntPXT4SVwO4GnBha2DxuhQKIndYSo9Lz10p2H3LbD1atLO1GnVzW POr2sIg9g+LsapmiUIxnGThqz98AbmjZU1ZvHLcnfBBBhbtF7HZLcq0KRv3ZkM5b+xHpuWxwbHE 73ZMhCBpVvtUs0OqvL329QUT/qc292CpkoOect+GtR7gsm0jwedxCTMrfGhFbvqmhx/txJSfZhY UeZ3APEGEC6RHGSwy1Am/cPmLNJqfxOBjX8vQu/gWF3hxXq2PqRlRpbp+K1Hx/BDDWxeuaHgOUM St69JRkRvBkaErLxA3NI9nEfG78v/ye+Eyfw5cLxc+v7ety1R2XNZt4dOaTaXOdYjPVtQdl2ib/ C X-Received: by 2002:a5d:528f:: with SMTP id c15-v6mr4110279wrv.102.1535542540495; Wed, 29 Aug 2018 04:35:40 -0700 (PDT) X-Received: by 2002:a5d:528f:: with SMTP id c15-v6mr4110230wrv.102.1535542539457; Wed, 29 Aug 2018 04:35:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542539; cv=none; d=google.com; s=arc-20160816; b=HRZbd/ky6Qa8bsVwTPpdOhv+frNW3aHADWhtOKCGiR/tk5+s6mUxDR9rseAoxYU2Vd LXSZmrq9o/HwEEKq24Hu8Lyt1UeWaFlIBuhgHr983H3p1L4uJTSRNx7ozGcQxqNiTv7b zDuc45JQ6RScv9FXBsj4V+XUULf1pW6i9v3iJx2oVTcmXxHFYyqA+mT9nPUEg7bSc47B 69rdiPZit3PHZEE02LADtYbX242ElUAWpijv2aXKa70R2bUrRfrFzhRlqzUiHY629la6 ZGOp6kmYa/O2RNNY1Rqjks7YHU3b5rU8JpXW5812o00qlrf7JxzPdG7e0KpuzgMK/HTG YWmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=kb0anrSFrq4OO5HLjnBi1Wtd8/F9idg7tFDv1gOHdc4=; b=no8cktB8oeq6W6yiRwfm1qbqL42pNd8V7TeTKTyvdJCKbbvDBiDaEJ/I+ER2lptYJ1 T/TU2RhzJsMPB3DMXZgAx9JkW9pVhK0eDUQ3JPQQRO+VB8jY1D/0Pcogkscy5H7O9bL2 RGf1EJ1bPykxGXMsB1UXpNdO3HWTKOfioWhbkx9Anr3m0QgSAMTF7DXbMLsXY0uQ4rU4 yNaj7xMC3J98I6Bv9PAEhdRfIaGRKL+ip9Oz4KNaK+B+r0gIXotY2gzICNNdN8OecGJH di+IhRRdw+PDNFnMw0zK5g/EgZD1zkLN7XesaLJMuO1KwzgnItL15nCgtFAvnDlJ9+67 7BBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=V77xIUts; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 201-v6sor972297wmf.4.2018.08.29.04.35.39 for (Google Transport Security); Wed, 29 Aug 2018 04:35:39 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=V77xIUts; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kb0anrSFrq4OO5HLjnBi1Wtd8/F9idg7tFDv1gOHdc4=; b=V77xIUts4G5R5xgxpZRlzf+b6PxUbpyYldPcR1vDedUxJCnCO1Cd7FjmZsRhUO5GK6 DqSPZhRHZ24fg1dYh8PnaOSNZMErt53IHxij5lTJDzXliZRTTZg6ONDGkTsFT29mRK57 1XMNNSGwspzUN55Y05uoa5Yd2K8bcaHFFqkFZFufVMI0yVygdcneSYIykpOHpW3hVuSo t6CB96Bg8wUZTsnJtcciwXjlaeD6HHL+bQ91OTSIWQNBdi2yclcFf3iSbu1yGChiIovQ J1HVI08VWA93LS3Z1IPZxaC2zKyBdBFK7g53WLalomO8XnP9SCMeu4wEPYIdpwdYBXUd 1EzQ== X-Google-Smtp-Source: ANB0VdZAmbo5vWf2UbC1iC4mfzsIcGelknxtQTTEElaiO8Z6gaQQGbaRPFvyEa08sXABod5h77vz/Q== X-Received: by 2002:a1c:adcc:: with SMTP id w195-v6mr4157719wme.41.1535542538782; Wed, 29 Aug 2018 04:35:38 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:38 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 07/18] khwasan: add tag related helper functions Date: Wed, 29 Aug 2018 13:35:11 +0200 Message-Id: <6cd298a90d02068969713f2fd440eae21227467b.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset (set to 0xff) the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 29 +++++++++++++++++ mm/kasan/kasan.h | 55 ++++++++++++++++++++++++++++++++ mm/kasan/khwasan.c | 65 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 151 insertions(+) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 7a31e8ccbad2..e7f37c0b7e14 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -250,6 +250,8 @@ void __init kasan_init(void) memset(kasan_zero_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); + khwasan_init(); + /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; pr_info("KernelAddressSanitizer initialized\n"); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 1c31bb089154..1f852244e739 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -166,6 +166,35 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #define KASAN_SHADOW_INIT 0xFF +void khwasan_init(void); + +void *khwasan_reset_tag(const void *addr); + +void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr); +void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, + const void *addr); + +#else /* CONFIG_KASAN_HW */ + +static inline void khwasan_init(void) { } + +static inline void *khwasan_reset_tag(const void *addr) +{ + return (void *)addr; +} + +static inline void *khwasan_preset_slub_tag(struct kmem_cache *cache, + const void *addr) +{ + return (void *)addr; +} + +static inline void *khwasan_preset_slab_tag(struct kmem_cache *cache, + unsigned int idx, const void *addr) +{ + return (void *)addr; +} + #endif /* CONFIG_KASAN_HW */ #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 19b950eaccff..a7cc27d96608 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -8,6 +8,10 @@ #define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KHWASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ +#define KHWASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ +#define KHWASAN_TAG_MAX 0xFD /* maximum value for random tags */ + #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ @@ -126,6 +130,57 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_TAG_SHIFT 56 +#define KHWASAN_TAG_MASK (0xFFUL << KHWASAN_TAG_SHIFT) + +u8 random_tag(void); + +static inline void *set_tag(const void *addr, u8 tag) +{ + u64 a = (u64)addr; + + a &= ~KHWASAN_TAG_MASK; + a |= ((u64)tag << KHWASAN_TAG_SHIFT); + + return (void *)a; +} + +static inline u8 get_tag(const void *addr) +{ + return (u8)((u64)addr >> KHWASAN_TAG_SHIFT); +} + +static inline void *reset_tag(const void *addr) +{ + return set_tag(addr, KHWASAN_TAG_KERNEL); +} + +#else /* CONFIG_KASAN_HW */ + +static inline u8 random_tag(void) +{ + return 0; +} + +static inline void *set_tag(const void *addr, u8 tag) +{ + return (void *)addr; +} + +static inline u8 get_tag(const void *addr) +{ + return 0; +} + +static inline void *reset_tag(const void *addr) +{ + return (void *)addr; +} + +#endif /* CONFIG_KASAN_HW */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c index e2c3a7f7fd1f..9d91bf3c8246 100644 --- a/mm/kasan/khwasan.c +++ b/mm/kasan/khwasan.c @@ -38,6 +38,71 @@ #include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void khwasan_init(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(prng_state, cpu) = get_random_u32(); +} + +/* + * If a preemption happens between this_cpu_read and this_cpu_write, the only + * side effect is that we'll give a few allocated in different contexts objects + * the same tag. Since KHWASAN is meant to be used a probabilistic bug-detection + * debug feature, this doesn’t have significant negative impact. + * + * Ideally the tags use strong randomness to prevent any attempts to predict + * them during explicit exploit attempts. But strong randomness is expensive, + * and we did an intentional trade-off to use a PRNG. This non-atomic RMW + * sequence has in fact positive effect, since interrupts that randomly skew + * PRNG at unpredictable points do only good. + */ +u8 random_tag(void) +{ + u32 state = this_cpu_read(prng_state); + + state = 1664525 * state + 1013904223; + this_cpu_write(prng_state, state); + + return (u8)(state % (KHWASAN_TAG_MAX + 1)); +} + +void *khwasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + +void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr) +{ + /* + * Since it's desirable to only call object contructors ones during + * slab allocation, we preassign tags to all such objects. + * Also preassign tags for SLAB_TYPESAFE_BY_RCU slabs to avoid + * use-after-free reports. + */ + if (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU) + return set_tag(addr, random_tag()); + return (void *)addr; +} + +void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, + const void *addr) +{ + /* + * See comment in khwasan_preset_slub_tag. + * For SLAB allocator we can't preassign tags randomly since the + * freelist is stored as an array of indexes instead of a linked + * list. Assign tags based on objects indexes, so that objects that + * are next to each other get different tags. + */ + if (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU) + return set_tag(addr, (u8)idx); + return (void *)addr; +} + void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { From patchwork Wed Aug 29 11:35:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579965 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7BD5217DE for ; Wed, 29 Aug 2018 11:35:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 693B92AA9C for ; Wed, 29 Aug 2018 11:35:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5C5EC2AB44; Wed, 29 Aug 2018 11:35:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DAE9B2ABDA for ; Wed, 29 Aug 2018 11:35:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 67ECA6B4B82; Wed, 29 Aug 2018 07:35:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 62F526B4B83; Wed, 29 Aug 2018 07:35:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DCBE6B4B85; Wed, 29 Aug 2018 07:35:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id CB1FF6B4B82 for ; Wed, 29 Aug 2018 07:35:42 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id b134-v6so2796070wmd.6 for ; Wed, 29 Aug 2018 04:35:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tx0f81qpatBTU8Zf6Dt4IAw08NEBAMzFHV/7KuZJzc4=; b=VMEg2gu7CWtofr1LGHXKcmLt0umRGafZeMsJohlzV0hdE9T4vd6wUzQQ7NN2uARGrs 8XmJyDXRCa1UBi+updhNBheSD1reQlzXIPaaWW+2amabnMKB8CeUUYQE0CWN6zhNYflt ByLhfxC2O+gso6YQ6oAikCU+e4/d07hj/Ttks4rp13zuUIEdLpkbn9s6oFt/ekq5Awuf P8ApRyi1QM14op4jua201iUN6BtPzkrpnCunuf6DuPvjVRTnCdx3L9Zv6TAIlS7SmngV CRwB3glUFKBTkqejH+Yh89d8NGh8JVBEjgqzRPlNYtXyPRdG/k7lUjmPdtwKoyc8YVQz zhAg== X-Gm-Message-State: APzg51B14qUtsj6RFLul+skm2d7QlzbISA8HYTsT675Nm0ugTqkpq500 6H5Ba1Y+OUcsNljpSaWCpihzfMZUlbuzps2k3z4EBpOqE0BHrdu9d6yfSsw6s7vDlUAr1Kbz7zO k3IEYSMtlMXdqS1Zjxs1sA9/4obxuqUtLaeKUWL6N9Eimib9DhOD32QK+M0Sh4GtshYfF66kiYe 1MJcaUC4Esph2GLBw5y3SvBiaPh/U/G+VftNUvDJkxyEBMGdGhZflDXdrGLNMPBsZ8MjQWgVxC3 hBfJKLWDfNQLroqotuDcBX5V+qhPVBysCM2PpvbqIQG+CHXutxn7QJZwHOevHR4eXWNL3Zg4PtZ EBlToEMIT9gOlLkXBRcjs2wAj0I4BNMU8Nfvq3nDXBMpmmGnoAkMIWGi0SmhXY1BUytpB0nQdMH I X-Received: by 2002:a1c:2108:: with SMTP id h8-v6mr71713wmh.108.1535542542290; Wed, 29 Aug 2018 04:35:42 -0700 (PDT) X-Received: by 2002:a1c:2108:: with SMTP id h8-v6mr71688wmh.108.1535542541565; Wed, 29 Aug 2018 04:35:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542541; cv=none; d=google.com; s=arc-20160816; b=GkMzYd2r6KKoeqL/HBUHV0qzPvzKGGK901Q8F4JP37d4KwIMjjB8vDmuNzVF7RQgGH 1F1XBpoedjvKdqm5tea+jOh85DbfscGvwrJkeD7w0PywkaMuar7Yck+dfo9Aa1kWFLMH jZJMkj0RkdQligy6aNjvnd9RyZMAu9Qx5n6sdtzdpnq5XqcOg0kkPTXTCHmeZaUCEzxe YcMbEFsIrecYzfVc3eizEhtzvVpFhme+TdRWd8iET6vvV4YzNcnsH6N7SmZZubXn7m2Z rcXxr2TowQNCVHSEk8MkCGcPNkSMq0/4iofxpzpEUh8O6K2VVvgI4fTAnoPqzMpxPK3y 8wyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=tx0f81qpatBTU8Zf6Dt4IAw08NEBAMzFHV/7KuZJzc4=; b=UUend2KQeZECX8zRwm4Kup8zjlYnZOdH4ZUFzBFnxe2hKPoJy34oKalKY4wvf5yuNH JvxIa1ZlEM38zgdaox1FGE2bx5DgNYkwzZfLKTQahOc7EWUBvyYBeEO0raKe0SNcsQYM g4nr4AezJ8/ftv21uYvDMXUwBAfOhu0dDvTeaoZVHznmx3sGTVcrZrOsJHV9JkLpIATw yX6ZCz1w/UK6/93301rp1+G22voks6bxkgZG/GBRtryqizj8LRyyrU7m0hOWAREq6rCm /XwHYUTHZ/yaMoDPi2kTXfIEXfmRPcTRuZGW72ZQ2Ic5yckzHKADn6YZUaf0sPgbV6j1 lU3Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=tGIcisOz; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 15-v6sor1122665wmm.59.2018.08.29.04.35.41 for (Google Transport Security); Wed, 29 Aug 2018 04:35:41 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=tGIcisOz; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tx0f81qpatBTU8Zf6Dt4IAw08NEBAMzFHV/7KuZJzc4=; b=tGIcisOz2STm0aJshASbTJeJ69RT8JMoAH3f4aCxVwgBItaA52XRJ9qdZYIaZsI2u5 84WxLKUwRjMLHaDEbGCci5rvpBoXQf4II3EtOgQRIvriYehFbI+64zM0dxij1O8dZcgA 0d4nuejpivMGGQO1gx9cPHhEWv7CdpbHsbyN8BudnoODPUQmrsBQrUnuYsKojZIqQrCt Y9gel9LN3l73cl3wtQZRwiyWtT1nUqrnnz+hgzlv/Tl+9TGC4ePGJhT2Lb0s6waiatxg 2jcKeIiW7gFzRHE/xwny+3A+SxEd+mYTqitxyw5Ii/nxehKFNx+YeMhys9uxy7VlTNq1 bVyQ== X-Google-Smtp-Source: ANB0VdY0nxFKwxx75PWHyKBWLlnvXmuWojLgvitD/kFGr51SPbBFXRZ2fLWPnFpHFc/eIOY2zo7EzA== X-Received: by 2002:a1c:2dc8:: with SMTP id t191-v6mr4162464wmt.94.1535542540551; Wed, 29 Aug 2018 04:35:40 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:39 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 08/18] khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU Date: Wed, 29 Aug 2018 13:35:12 +0200 Message-Id: <95b5beb7ec13b7e998efe84c9a7a5c1fa49a9fe3.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor. The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it. Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed. Signed-off-by: Andrey Konovalov Reviewed-by: Christoph Lameter --- mm/slab.c | 6 +++++- mm/slub.c | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/slab.c b/mm/slab.c index 6fdca9ec2ea4..3b4227059f2e 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -403,7 +403,11 @@ static inline struct kmem_cache *virt_to_cache(const void *obj) static inline void *index_to_obj(struct kmem_cache *cache, struct page *page, unsigned int idx) { - return page->s_mem + cache->size * idx; + void *obj; + + obj = page->s_mem + cache->size * idx; + obj = khwasan_preset_slab_tag(cache, idx, obj); + return obj; } /* diff --git a/mm/slub.c b/mm/slub.c index 4206e1b616e7..086d6558a6b6 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1531,12 +1531,14 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page) /* First entry is used as the base of the freelist */ cur = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + cur = khwasan_preset_slub_tag(s, cur); page->freelist = cur; for (idx = 1; idx < page->objects; idx++) { setup_object(s, page, cur); next = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + next = khwasan_preset_slub_tag(s, next); set_freepointer(s, cur, next); cur = next; } @@ -1613,8 +1615,10 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) shuffle = shuffle_freelist(s, page); if (!shuffle) { + start = khwasan_preset_slub_tag(s, start); for_each_object_idx(p, idx, s, start, page->objects) { setup_object(s, page, p); + p = khwasan_preset_slub_tag(s, p); if (likely(idx < page->objects)) set_freepointer(s, p, p + s->size); else From patchwork Wed Aug 29 11:35:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C4E31709 for ; Wed, 29 Aug 2018 11:36:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C6A92AA9C for ; Wed, 29 Aug 2018 11:36:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5F48C2AB91; Wed, 29 Aug 2018 11:36:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB77C2AA9C for ; Wed, 29 Aug 2018 11:35:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F5726B4B83; Wed, 29 Aug 2018 07:35:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4CE176B4B84; Wed, 29 Aug 2018 07:35:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3989D6B4B85; Wed, 29 Aug 2018 07:35:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id CC84D6B4B83 for ; Wed, 29 Aug 2018 07:35:43 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id o43-v6so3276004wrf.10 for ; Wed, 29 Aug 2018 04:35:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=/C7rrwe4N6BQx9M4EE8DtUTnDGQn+bG0YjkQ0aHndSs=; b=SsXo6CMLeUlsJthi1dPkDLmJtVGP7HusIZNONlTvsiyF72MSR4+Axc+5tFD8ng9lb7 R90ESoRvQthql2KjZF5ML0ZupPlNyxYtBfNsvETP5swQ4AYiIm1nHsyRWXf7e9qZocL5 S5Lis3HxRWZd6/U+Ea6wEBagXjkhFnf4k9XY6PZlpfxcUHTZUKEUVvQ8YS0CP3FMXZtM 6Y+dD5/EvUzwmaxWHoxmzwbl+OXv+p7KIf9its6pQ4NL+h82qqWVYyL+wNIfXBYjqNRG DLrL6IyJxtFFskasPxngOJUJjYbhhyn+GFBqkJoS+xuQq0vdi6Uj+ekX2MU8EqBTGN4S XfaA== X-Gm-Message-State: APzg51DDzEfhjFzooitx8+g01QFsd4itWz6Vqj3q011uZmkrAoMxhvsn 1P2SUX0SoCgzEtuXk/LvYqu2pI5TOA38pFsz4Be11jcNU1lntAKTABlxfLDyu1asJwHq+tAlrFX ZFNG2D8Ir3GVMnJmLxuLNJ/k0WcMs5ybRrhuEljvkGFgC69k9TP9Ihx5tgOxFMIZp+1fMTSNix9 7q4VoYs964HdbBFQrjeSWCDG6F7FzfoMHpD/YB0mj/odIIBVcIFmcGoJbpOfb4cXnH0iv0NbCMr adRbEk+sAcfB2TDkF7vYAGYU0izcrRA7xu2MVSQGnJmDmfI/YjCXYZW4LigNC/SKoI/0IZI8HuE SeV+ZLy6uc0r4+1aUaj7llh4osr46sU935fwW36fXZP+RGHAVmNz927EvxoB57xsXFlKyeDX6Vg x X-Received: by 2002:adf:f84b:: with SMTP id d11-v6mr3952449wrq.172.1535542543379; Wed, 29 Aug 2018 04:35:43 -0700 (PDT) X-Received: by 2002:adf:f84b:: with SMTP id d11-v6mr3952423wrq.172.1535542542694; Wed, 29 Aug 2018 04:35:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542542; cv=none; d=google.com; s=arc-20160816; b=qKJjx/RNATb3HDub/vy0Vk2zuJ//Jd5Tt31VB2QmznU4Kk+jGF61LfioVwCpvkEZGi HzGQUBfP8qS6F7hRlgTFdY6xDyvZY/+JcIwf6nH7YArnytN0143CAvITA/wVUDhuksT7 dDO10fIGq+YDmBE+YcaBG/rgjrUgyun4In4+Zya+jFKkbPpMFyPEpKk/7zGqVDaB51DL o3tyaaNHbYndI7dtXA7TOf0FiYAW/29Yqwi7lUGRSO056J6Y0pXWB9GHS6HjxPae9vkT htRNlrDbaiRKQP+Je18sDQ2AG8ShtGFIAszRGar4OCNGZxmbmelTMFityK4ll7Vqv2wY P9vQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=/C7rrwe4N6BQx9M4EE8DtUTnDGQn+bG0YjkQ0aHndSs=; b=sCOefOx+xWgWZBiqoPnBlBLRkOe1eCrpD6eGDPqd7eJkQdjNzNaurxWF1z7DPNqA/2 yjEF7IPZAWHZB0y6PNiMyCMb+hU/OoQNaa/q7BxGNcqmPDaskx8donJQlSvZvEaAguoT trQtA5jId6Nj22t1wrCtuRT9LBwHEegvEV2l3lR49C964IRHVNf59XbtdBBZHrkQ+duG IingkgH6XhXGLzVwcqpVA3C60vy4QahbUrZY3j3iZLZB/Oa0MSl8TPbwVU2S7itvyEJb uklfg2+kgx8T9v7TFbBAlU28YjkXeRJVM4RfKuTi+kDOSoBGDwpZUtiEMJtrbxfWSrpO DPEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="dU/Fx2La"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id x5-v6sor1641082wrd.41.2018.08.29.04.35.42 for (Google Transport Security); Wed, 29 Aug 2018 04:35:42 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="dU/Fx2La"; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/C7rrwe4N6BQx9M4EE8DtUTnDGQn+bG0YjkQ0aHndSs=; b=dU/Fx2LaujUbvtHauwsP0t1aA/m2D+06oXMHwZ3eXwP1Nzw3gC8PebFtKC1EnSsH69 c0BZGLLNtAdh454C93JmrKX65W2EMHdPt55gfbxnmI6ZBwf48jJ0+K0uyUO+N5x56L61 H05vabh86vY1pFtsPWG7/c0vwk/TX9FzE/hL4njIAk0P4vbuWYNdMbg1msffocHP+j4S 02zcVCF+LR2Gffm0s0xI7A0hw2Fm44sDbs+XriYfO7scgMEmeWN4DNgjhPtc4Br6jBit X9y0Oa4JtjU3dat2ZfvzOzi9tZrU9ASdv/eeSGhQIx4BwERGx2cCYod7MNlV1snr6UJ0 XzsA== X-Google-Smtp-Source: ANB0Vdb2BlaS8zRz1NV8bmHOiV24XFb36lvecxL3punEHE/BVCiUxdBbrADrOo26tG+iGem0MA87hQ== X-Received: by 2002:adf:f608:: with SMTP id t8-v6mr4068969wrp.186.1535542542109; Wed, 29 Aug 2018 04:35:42 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:41 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 09/18] khwasan, arm64: fix up fault handling logic Date: Wed, 29 Aug 2018 13:35:13 +0200 Message-Id: <4da4495dc20525db1654d3dd9d578b7965d98507.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP show_pte in arm64 fault handling relies on the fact that the top byte of a kernel pointer is 0xff, which isn't always the case with KHWASAN enabled. Reset the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/fault.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 50b30ff30de4..35feee78a9bd 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -134,6 +135,8 @@ void show_pte(unsigned long addr) pgd_t *pgdp; pgd_t pgd; + addr = (unsigned long)khwasan_reset_tag((void *)addr); + if (addr < TASK_SIZE) { /* TTBR0 */ mm = current->active_mm; From patchwork Wed Aug 29 11:35:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579973 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9C371709 for ; Wed, 29 Aug 2018 11:36:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9A772AA9C for ; Wed, 29 Aug 2018 11:36:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9C62B2AB91; Wed, 29 Aug 2018 11:36:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E92A2AA9C for ; Wed, 29 Aug 2018 11:36:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E4256B4B85; Wed, 29 Aug 2018 07:35:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 03C666B4B86; Wed, 29 Aug 2018 07:35:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5AC26B4B87; Wed, 29 Aug 2018 07:35:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 75A876B4B85 for ; Wed, 29 Aug 2018 07:35:45 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id u12-v6so3328241wrc.1 for ; Wed, 29 Aug 2018 04:35:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=OrkXoXQH/F+TSi0uzJOTsE67Q+iABNFRWIJoC0xEk4I=; b=cs2Pj/0oIZNvtQWkHb279uhkmHG9PuJtxEQn5prn5mv6MIUr+aobrMoiI8JLwPoVH+ Snwkrq/RRCceEK9sjyt0tMvNzZhyd2lLZb3+G5Rw0os9SWCqOOAR9NNhTRKg/+Y83wJb IljAWRhV1wY1hzzOyoDBqxMXFhLnji3igpPras+WRVMrtvE97etljcsxQnB9BjNR+KRA 9IigfG+tEEILGVXTTjmjAiB0hzQwKFSlNZGY8KUHYnttI+j6NKsz+icdJ36jLqKRjg/A H4mSlAZJkQn/D8hz+JQndxxmRTk2yJq/bi7/GNvJMOGukYiGspJcBc4QdNaCzqEhE2p1 gC/g== X-Gm-Message-State: APzg51B5eVKuWKagqhiPQTf6zDUAPNj2/u9FIr30w7aJBMbVdvN5KOQv sk1N8+eiMITm8CJkNbWedrCmVtEhh9ZLmDVBJcMzny+yJeltf5ihnHj9SY0mU9027wo6UHnGEPI HRrOjRKk9mGlPfAWlPmxcDJYI4nU/hyTBgmCIBbhLVdRy/ixfvVj2yjVs2cqa0SsI7uCycsXUWg KMupzyfIvslV53zJChivnD22ByakGbKZM3f0L8O2fAH/mtRPvJxQc5/uY4JCdxlBoGzrCm1AO/d XHSscVn0k58T+tawNrypTW8xsxohNMi5pTHSo674/twXdXcLnUPp8uTCUAslbVjqT3FA6DwyaOl V2SWg/5FzxWb9RFujiPu0PnPmdLBNNb05GTi58ji5rTGVvPhQHUffpuFGK4NPphjdgUtSFFYz1L c X-Received: by 2002:adf:e3c4:: with SMTP id k4-v6mr3867650wrm.94.1535542544872; Wed, 29 Aug 2018 04:35:44 -0700 (PDT) X-Received: by 2002:adf:e3c4:: with SMTP id k4-v6mr3867634wrm.94.1535542544199; Wed, 29 Aug 2018 04:35:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542544; cv=none; d=google.com; s=arc-20160816; b=ASIZGdEptLFpiZhsfGk0fCRa7i32C0b+gjyS1riN7/zgWzG6epHGI0YIquXKqW9b2i mYoMJmc9z5RWuUV5DEVyXVGb5Fvhw65npDRlPcEYoTPCHPhwA4/ZF6nd2dgO8Y1gg9k6 sx9x/h8lTx9pER+Rru5cLyu/KSDRE62Kj9k/+p1nGWq+tsxZWuoiwvuFR/tJUJgLvUlY p4Ol93xzgtZ/hZREQlWPFOg8hlt0V2OnaY2hYkiDAhriWyRz8sAW9p5Rf3tDRAkgkO0H 4stjtZ3iIN/haqZClXXh9DtpQ+/83hd3PDNTVDG4rhzRNkMSA3nHemEtnEJAyB2Yw4UG gk7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=OrkXoXQH/F+TSi0uzJOTsE67Q+iABNFRWIJoC0xEk4I=; b=mNidQXfQ4TYmnDZjNHirboA+3ZIFhDUNAiRme3RwQM3hBY2o+bBZ5DaYyQESpzlK3U WzNGI7xjlR4K93ZxPpSumNMQ2tr7Q1xGI5ivKakZK/eeb+2CYRH0ojbEgndctO+L31r8 2umO9DLGAMdiDs+ijxENbBlfMXCKpa0bAUIp+KkvaCxA3MhmA+V5POKVrJoOMm8nhKNL 2uburdpqmDGifWHBP88XaOTHANgxfWeBPhPgAMhMmVd1xDeKmkrxxtM60mjDDLaT/mxO hsiFuih6eX5U7jH7b6BrCOZ7pGFmZbcTn/kguPH5eIVdXT4iAWPqtcEvMbUFpTIAKBsl UhEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=WTz2QY2H; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v16-v6sor2624946wrp.22.2018.08.29.04.35.44 for (Google Transport Security); Wed, 29 Aug 2018 04:35:44 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=WTz2QY2H; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OrkXoXQH/F+TSi0uzJOTsE67Q+iABNFRWIJoC0xEk4I=; b=WTz2QY2HKqYf+swr0X7Ae2YyC/6YPEZ7qJCGjn+gto47vutHzDmySwPeJpJN11E0ZE 5/N+Lf5FI2i9lD6qnaIKa8vC/iEgp6tV3xlwZxOCsUOpRsPK5x8OxbGPeHqoaLRnz3tl 9y2MSALdGNvdhBDLQvs0+f/FvlAZbS/GTFpeuBXd0Di7KJovjN8/Xes0AHrVObh18/kk l3VioLZaGqerA6Z1z05WFImABwS9LljXfzhrpinvyCSxMUXJUyb6Mgz0x0wZN2PV/gRR k8/pJKOTpSsY6SdKFgIhYeExLmV+9l/hayV09vG0Ai1tN5FuaJADVtrnzjbraHGluFQU kEgg== X-Google-Smtp-Source: ANB0Vdb/wq3vWEnItljvIkYES5qFH6+aaA3t/puEBlFbr7sKQYhGOPxWZqcCP6CtYnWV4cowVJ+NpA== X-Received: by 2002:adf:a512:: with SMTP id i18-v6mr4050713wrb.220.1535542543576; Wed, 29 Aug 2018 04:35:43 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:43 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 10/18] khwasan, arm64: enable top byte ignore for the kernel Date: Wed, 29 Aug 2018 13:35:14 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when KHWASAN is used. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index fd208eac9f2a..483aceedad76 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -289,6 +289,7 @@ #define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 03646e6a2ef4..c5175e098d02 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_HW +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /* @@ -440,7 +446,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /* From patchwork Wed Aug 29 11:35:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579975 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE7611709 for ; Wed, 29 Aug 2018 11:36:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE5122AA9C for ; Wed, 29 Aug 2018 11:36:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C0DF02AB91; Wed, 29 Aug 2018 11:36:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 748992AA9C for ; Wed, 29 Aug 2018 11:36:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98F396B4B86; Wed, 29 Aug 2018 07:35:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 93AB16B4B88; Wed, 29 Aug 2018 07:35:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78E6A6B4B89; Wed, 29 Aug 2018 07:35:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 189F86B4B86 for ; Wed, 29 Aug 2018 07:35:47 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id w6-v6so3238692wrc.22 for ; Wed, 29 Aug 2018 04:35:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WeSMiuMtzDEN1FdWmqPLXOd7CyZ+jfvYb8JwPl6GEEQ=; b=Z92EUPZO9/mwEhCUqAFOSFq/kUNKkivRF7y/8PCWGXTc75ONcco0tg8gzlMaxAxj78 KuwZAPUAMuEUqAFYiAOPKVOMYxQ3u99k0ecxWuYUKNAXIE3zE0GsKVkw6jDe3bthabEf RmtAUh4sltVfSYaKX9wTwr+aez7L3mdq1u2UPvq4RqRhJJszxaiDopZiZDIRLkEhWo8F /jTD7QKUq0WfVqTr4ZTscDJPXI85Xlgfct+sk5ZHbScX5BoDAL9YVgQZIrOZkjjlgsIF h6I1n4qs11gFeKclQVB0m7SVjfTgl7pX8iUq7vD/Pm+aacU+bNuYH5bQFIdADYj2jC51 I+iQ== X-Gm-Message-State: APzg51BfajUKi9sani8hcTkIDq+MHqy82zrh7FHUBG5C7MWszq+RaWfK yIiiX4T0+fPQUjpGppxh9fYmMS+p1Fq3ZwIe+VST6urNvMnr34JzY74DJtFxOG2esJyMHDFvRkB LX0Ebh5ptp9bW9wZU2oRvm0tAByjLnIevQQfnjbvrIan/rjLWNExDJ9q0aXFboEZt4vPfeHAOZ8 8UZxLV08Jwd/mGB/lIMlWWsJ5q2lKA09MsiQvbQ8RfDXzz3WMQcE+NR97nNGJFJjUpAeZ6KngiV 4tlkCTpNwHWMSomasMxh+VjzSJxVHb+GGNSWl4XLyiHqg9q8Iz2+GqrJBisSAu7LBuCc+zDiGJf W7HKq6ww5YVd/xOn834DYq5RrxP9FdJ3PKhVhR/LaTw0fWRLyd1h1COymOmCYzHEs7to8PCKnjJ Q X-Received: by 2002:a1c:a614:: with SMTP id p20-v6mr4021648wme.68.1535542546552; Wed, 29 Aug 2018 04:35:46 -0700 (PDT) X-Received: by 2002:a1c:a614:: with SMTP id p20-v6mr4021611wme.68.1535542545826; Wed, 29 Aug 2018 04:35:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542545; cv=none; d=google.com; s=arc-20160816; b=W195/w9CVhMSJVj4QnzHanDpnHDpxevt1Gz7KpkmxNlMH2yXE63n9egUVbk657SoGF njjjBwvdlloVHTJgxmhn+2xqEklcPVX9+0ElzXraaYmOKb5ROCa6eew0YYeHIy8fQzRG Qu977kqH7D+SFUnVPBzSRCah22sSL7ncOt0SWQNM6raBiexgS35XAJ+FDUzjiGgCYo+K Sx9tP1HG3a+Vu7ABuZI/I0LAr8tfeNiOhnvRYJ0YYN/5NZLFZxOEv864I2H0ibI4Llc1 bWzPozu8WTcd46GC/okw9TKOpwut+fFxvmu5KLtyM1zBWVex228x8Lx4Rfs5eA+p99OO 5CZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=WeSMiuMtzDEN1FdWmqPLXOd7CyZ+jfvYb8JwPl6GEEQ=; b=JqARmE2M4ExcWjW7V5hratedNTp5KAwEkIwOUp1T66+ANeZZb+mWaAvT3KM2WAmKtF TsFoju+/ey0ff64TK0o8ZfaPLTi54JXVb1pWhqJz4xhaYjRP1/3pEkP2UjJrOO1yi65K quha3Ph2A+SAvmxvSqSE3KlDmu+4TGuccHaVtAzj6jvCX4SkxB4Big+5aEq661AAMyL8 09owdLcsJmgAZJRSWUF0T0ZvVYEsVLVf/dgFAkeCmVYD7kJbiUK5msoP8h94SIFdbfuW JvEfnSVGutlUiZSkiFD1BEYyhN2FX1xPInUmNYvv3eMIZASYaGXjIS5Cj7enP54FrZcd 0KNA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=DB2fTUZP; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id w196-v6sor1087317wme.41.2018.08.29.04.35.45 for (Google Transport Security); Wed, 29 Aug 2018 04:35:45 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=DB2fTUZP; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WeSMiuMtzDEN1FdWmqPLXOd7CyZ+jfvYb8JwPl6GEEQ=; b=DB2fTUZPwYG/vC54X0ylcYITM5VLrqYbpoKAgttKvxIgjM/CMpHyEfZop0XiAxCjAR SqefjUTWrUaMFbrzlRaCkfPP2ENFxVz4o+3GyKyWZfACMcIkY8BvkKjpZ5rLUNzQv+vC UaiuKzQpQaLSdmSwRMrFir9YX1tzVhMmHfnFRDWuVX9c7EM0JdeHcgrdRy8butdZgjsO O5jsznGZWbNwzg1Hwvpy4cG2lxNJ8pAsWYQXASFciem61/ZlgRxIf1J82I/AV/Gi5n/V efOWPmlesPVvERDd7ijRqLcvXF7OOC/l5jMPEgZPmfTjC7dW6V65A5vbC7e7im+oXG2d LbdQ== X-Google-Smtp-Source: ANB0Vdbc1gSJCzQt/c4lE4zyKdI4J/AL4sF1sTjGC2+4JWrxUIp67ZMN/Z0VxXD8BaXnDRSB0wO+UQ== X-Received: by 2002:a1c:200c:: with SMTP id g12-v6mr4128408wmg.102.1535542545044; Wed, 29 Aug 2018 04:35:45 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:44 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 11/18] khwasan, mm: perform untagged pointers comparison in krealloc Date: Wed, 29 Aug 2018 13:35:15 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. KHWASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not. Signed-off-by: Andrey Konovalov Reviewed-by: Christoph Lameter --- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 3abfa0f86118..0d588dfebd7d 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1513,7 +1513,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret = __do_krealloc(p, new_size, flags); - if (ret && p != ret) + if (ret && khwasan_reset_tag(p) != khwasan_reset_tag(ret)) kfree(p); return ret; From patchwork Wed Aug 29 11:35:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7ED5717DE for ; Wed, 29 Aug 2018 11:36:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6EDC72AA9C for ; Wed, 29 Aug 2018 11:36:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 619722AB91; Wed, 29 Aug 2018 11:36:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43AC92AA9C for ; Wed, 29 Aug 2018 11:36:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 520436B4B88; Wed, 29 Aug 2018 07:35:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4A9266B4B89; Wed, 29 Aug 2018 07:35:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AF736B4B8A; Wed, 29 Aug 2018 07:35:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id A72E96B4B88 for ; Wed, 29 Aug 2018 07:35:49 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id a37-v6so3337465wrc.5 for ; Wed, 29 Aug 2018 04:35:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QE7zu/J6JpHtjxjbj5eyHg3o6vc4NUhXiKZMTLxkjT8=; b=ivayShL9UHLnyroAkv7SafOJYks+iWDGv1+CbINiomtpSMbT/ZuiljiZvSF04tvN4g kYf3cWp48ekYvkIkD9RNGNXFZU8hWTBeYprxgluPEFbHjzi8nyoabuCO3vY8oX4DFvFT heo/b14MbsnIK294Zdrz7d9G9p9CgFQMwu7MXgsQG/n8jBC9THMUPBECUzR+AwBk7LUt FB6b4VWU/Z1Y/dTF1hPeZ3o8jxT5vC9/yvX8iR2NDsacATxUtr0P48Up6m5i2qkAox3T 4KqFClV6LxZlXEXpatpzFKECVMkz7vQphE9xQMZP3/P4hp9pfr/YgjxbRwqxG6QjHkSj QmYw== X-Gm-Message-State: APzg51BfB2vl9LoiSeKxtsTU1Bd68lwBZcEqWlS00ubSOZQn4urbOse6 0MF1Xigyt2XuxbnEzDahmmMovtz+OiSvTQu5eVFNs2PC7zHqQVclixgrcgSIdHJt8qamOsDBOfW P8xVsgSPjEN7NaydxH8WYxAkAn+gf8OcTRLKsy4j2MggFQTWorznxbPElanIedjdYVoujr3RwcO OmrGixvGdJTXri4sbiBa2CR6XV/E7/u/fvs/FZcsEarN+fB1MesIgYR/ibtTdQcKlScvtAiSMnF 1HDg9pBLoyNCBly4PkQ1WCuMf2uyYK2JafjjI3NEH/W8YTRyyZPG2Kx1szyBfXCalyenZnLH80a K9ySEa5qxmgFeoVmv0i+8A1+hcDLLJ8V5n3fyW+hL+fBxSmBo+czlkXmhwoevtyxvAYnzJYk+1b g X-Received: by 2002:a1c:780c:: with SMTP id t12-v6mr3985366wmc.133.1535542549162; Wed, 29 Aug 2018 04:35:49 -0700 (PDT) X-Received: by 2002:a1c:780c:: with SMTP id t12-v6mr3985310wmc.133.1535542547753; Wed, 29 Aug 2018 04:35:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542547; cv=none; d=google.com; s=arc-20160816; b=r+1UlvtzQEkZRqaG+6kKqhM/hO+74sz2WW/z5H6go0hErBaPJLTRNCXdN3bqB5/Ruk Er6K5Jwh9yeRa5KcIzZsgR3WYLLhmoVHpfHvYwqoul1BS4Y2EgJXl43MNoBr8o+QLM6P AH3YScVDjQu2sLNjQG7Ypbw0mvWR4xJtgesSI0aO0hrbg/fbs7dbxYUwSGixqYCJtPIa P18Ysv6FijYQ5AJNbQaZ3X9V0r1s2tP8Rmnk89oDo/rsj7KP96ucqpOrdfF3oe6Mk1T2 boEWFcC5lm+l6JsdaZBCafXveT+J+aUFZVS+XnZfHER3vz862obrfvibokEH8skwQ4bu MVqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=QE7zu/J6JpHtjxjbj5eyHg3o6vc4NUhXiKZMTLxkjT8=; b=ErNh5/7mJp4ArsH6OJvyEmWqt4Gw2ZR9gxL0tZOOcGB3eqhrxOG7mWcKRwnU5piXOg JqAAJwD0GniunIToHcdF7zLU8o6sHEkYfNc5YWIGbErMfot/UPszWlGoW/QxiFQBTrvP xKMlpL7waWpi6FztVwVXgFzTFiX4qmcGmT3s6dEV7fpwadBh76eR8hS9m7jojzt87qAC /Dgj5NXQ448M/LPrH392/4NDsrQSZcCgd6GaWyTFpUea2Tq2xZnMRqUUyu/Q6+0xNpSt dt/Irv8nfXGxf0ETuFL0ADJjLICWIu8tMGdnECezHjrbEe5Gx69UGmlhTXkaHJrvLoZn sTrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=poTOiZ7O; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b77-v6sor999197wma.72.2018.08.29.04.35.47 for (Google Transport Security); Wed, 29 Aug 2018 04:35:47 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=poTOiZ7O; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QE7zu/J6JpHtjxjbj5eyHg3o6vc4NUhXiKZMTLxkjT8=; b=poTOiZ7OEJDvRXqnW6M11CljhGrcvxa6MB8NNDTuden3Y9qpIq+g3QXXKrK3tpMXIU Xx5qX4G3tZy6VjbhFNGGa3l9wwSYgxFUBnke/Xw4MVD5Xp/TfGNMsGh9kG82HVgZPtCJ F3XioOwutDCy3fGRanaHVRYYSHTu+N2woqrSzUO26GhsuSi9w4DRLccTijKVQIBIoFT8 Xweewr2hAoWwQdXAolEPidw9AYzvJc7EglK4uN8jONYu3Flf3RP7se0dvA6s/9NCIwWn 52Y+pVpH1/KUIZuUwTpbPOk7Ca2uM8jJ2SeUP6H7gqVddJnCSqwJxfw40snOOXpL2AcF BMzg== X-Google-Smtp-Source: ANB0VdY7lfZmikPKZfbmgvOqeer3iRdA2rG9AgDe8U1fHtiwKkC/+fBVEuUZi1JjrGIyrrNWDpcHRA== X-Received: by 2002:a1c:f03:: with SMTP id 3-v6mr3949734wmp.129.1535542546650; Wed, 29 Aug 2018 04:35:46 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:45 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 12/18] khwasan: split out kasan_report.c from report.c Date: Wed, 29 Aug 2018 13:35:16 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch moves KASAN specific error reporting routines to kasan_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by KHWASAN. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 4 +- mm/kasan/kasan.h | 7 ++ mm/kasan/kasan_report.c | 158 +++++++++++++++++++++++++ mm/kasan/khwasan_report.c | 39 +++++++ mm/kasan/report.c | 234 +++++++++----------------------------- 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/kasan_report.c create mode 100644 mm/kasan/khwasan_report.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 14955add96d3..7ef536390365 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -14,5 +14,5 @@ CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o -obj-$(CONFIG_KASAN_HW) += khwasan.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o kasan_report.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.o khwasan_report.o diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a7cc27d96608..82672473740c 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow(const void *addr) +{ + return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +} + void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +const char *get_bug_type(struct kasan_access_info *info); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/kasan_report.c b/mm/kasan/kasan_report.c new file mode 100644 index 000000000000..2d8decbecbd5 --- /dev/null +++ b/mm/kasan/kasan_report.c @@ -0,0 +1,158 @@ +/* + * This file contains KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +static const void *find_first_bad_addr(const void *addr, size_t size) +{ + u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); + const void *first_bad_addr = addr; + + while (!shadow_val && first_bad_addr < addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +static const char *get_shadow_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + u8 *shadow_addr; + + info->first_bad_addr = find_first_bad_addr(info->access_addr, + info->access_size); + + shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); + + /* + * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * at the next shadow byte to determine the type of the bad access. + */ + if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + shadow_addr++; + + switch (*shadow_addr) { + case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + /* + * In theory it's still possible to see these shadow values + * due to a data race in the kernel code. + */ + bug_type = "out-of-bounds"; + break; + case KASAN_PAGE_REDZONE: + case KASAN_KMALLOC_REDZONE: + bug_type = "slab-out-of-bounds"; + break; + case KASAN_GLOBAL_REDZONE: + bug_type = "global-out-of-bounds"; + break; + case KASAN_STACK_LEFT: + case KASAN_STACK_MID: + case KASAN_STACK_RIGHT: + case KASAN_STACK_PARTIAL: + bug_type = "stack-out-of-bounds"; + break; + case KASAN_FREE_PAGE: + case KASAN_KMALLOC_FREE: + bug_type = "use-after-free"; + break; + case KASAN_USE_AFTER_SCOPE: + bug_type = "use-after-scope"; + break; + case KASAN_ALLOCA_LEFT: + case KASAN_ALLOCA_RIGHT: + bug_type = "alloca-out-of-bounds"; + break; + } + + return bug_type; +} + +static const char *get_wild_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + + if ((unsigned long)info->access_addr < PAGE_SIZE) + bug_type = "null-ptr-deref"; + else if ((unsigned long)info->access_addr < TASK_SIZE) + bug_type = "user-memory-access"; + else + bug_type = "wild-memory-access"; + + return bug_type; +} + +const char *get_bug_type(struct kasan_access_info *info) +{ + if (addr_has_shadow(info->access_addr)) + return get_shadow_bug_type(info); + return get_wild_bug_type(info); +} + +#define DEFINE_ASAN_REPORT_LOAD(size) \ +void __asan_report_load##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, false, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_load##size##_noabort) + +#define DEFINE_ASAN_REPORT_STORE(size) \ +void __asan_report_store##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, true, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_store##size##_noabort) + +DEFINE_ASAN_REPORT_LOAD(1); +DEFINE_ASAN_REPORT_LOAD(2); +DEFINE_ASAN_REPORT_LOAD(4); +DEFINE_ASAN_REPORT_LOAD(8); +DEFINE_ASAN_REPORT_LOAD(16); +DEFINE_ASAN_REPORT_STORE(1); +DEFINE_ASAN_REPORT_STORE(2); +DEFINE_ASAN_REPORT_STORE(4); +DEFINE_ASAN_REPORT_STORE(8); +DEFINE_ASAN_REPORT_STORE(16); + +void __asan_report_load_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_load_n_noabort); + +void __asan_report_store_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/khwasan_report.c b/mm/kasan/khwasan_report.c new file mode 100644 index 000000000000..2edbc3c76be5 --- /dev/null +++ b/mm/kasan/khwasan_report.c @@ -0,0 +1,39 @@ +/* + * This file contains KHWASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 5c169aa688fd..155247a6f8a8 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,5 +1,5 @@ /* - * This file contains error reporting code. + * This file contains common KASAN and KHWASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -39,103 +39,34 @@ #define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) #define SHADOW_ROWS_AROUND_ADDR 2 -static const void *find_first_bad_addr(const void *addr, size_t size) -{ - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; - - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; -} +static unsigned long kasan_flags; -static bool addr_has_shadow(struct kasan_access_info *info) -{ - return (info->access_addr >= - kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -} +#define KASAN_BIT_REPORTED 0 +#define KASAN_BIT_MULTI_SHOT 1 -static const char *get_shadow_bug_type(struct kasan_access_info *info) +bool kasan_save_enable_multi_shot(void) { - const char *bug_type = "unknown-crash"; - u8 *shadow_addr; - - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); - - /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look - * at the next shadow byte to determine the type of the bad access. - */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) - shadow_addr++; - - switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: - /* - * In theory it's still possible to see these shadow values - * due to a data race in the kernel code. - */ - bug_type = "out-of-bounds"; - break; - case KASAN_PAGE_REDZONE: - case KASAN_KMALLOC_REDZONE: - bug_type = "slab-out-of-bounds"; - break; - case KASAN_GLOBAL_REDZONE: - bug_type = "global-out-of-bounds"; - break; - case KASAN_STACK_LEFT: - case KASAN_STACK_MID: - case KASAN_STACK_RIGHT: - case KASAN_STACK_PARTIAL: - bug_type = "stack-out-of-bounds"; - break; - case KASAN_FREE_PAGE: - case KASAN_KMALLOC_FREE: - bug_type = "use-after-free"; - break; - case KASAN_USE_AFTER_SCOPE: - bug_type = "use-after-scope"; - break; - case KASAN_ALLOCA_LEFT: - case KASAN_ALLOCA_RIGHT: - bug_type = "alloca-out-of-bounds"; - break; - } - - return bug_type; + return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); -static const char *get_wild_bug_type(struct kasan_access_info *info) +void kasan_restore_multi_shot(bool enabled) { - const char *bug_type = "unknown-crash"; - - if ((unsigned long)info->access_addr < PAGE_SIZE) - bug_type = "null-ptr-deref"; - else if ((unsigned long)info->access_addr < TASK_SIZE) - bug_type = "user-memory-access"; - else - bug_type = "wild-memory-access"; - - return bug_type; + if (!enabled) + clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); -static const char *get_bug_type(struct kasan_access_info *info) +static int __init kasan_set_multi_shot(char *str) { - if (addr_has_shadow(info)) - return get_shadow_bug_type(info); - return get_wild_bug_type(info); + set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); + return 1; } +__setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info) +static void print_error_description(struct kasan_access_info *info, + const char *bug_type) { - const char *bug_type = get_bug_type(info); - pr_err("BUG: KASAN: %s in %pS\n", bug_type, (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", @@ -143,25 +74,9 @@ static void print_error_description(struct kasan_access_info *info) info->access_addr, current->comm, task_pid_nr(current)); } -static inline bool kernel_or_module_addr(const void *addr) -{ - if (addr >= (void *)_stext && addr < (void *)_end) - return true; - if (is_module_address((unsigned long)addr)) - return true; - return false; -} - -static inline bool init_task_stack_addr(const void *addr) -{ - return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); -} - static DEFINE_SPINLOCK(report_lock); -static void kasan_start_report(unsigned long *flags) +static void start_report(unsigned long *flags) { /* * Make sure we don't end up in loop. @@ -171,7 +86,7 @@ static void kasan_start_report(unsigned long *flags) pr_err("==================================================================\n"); } -static void kasan_end_report(unsigned long *flags) +static void end_report(unsigned long *flags) { pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); @@ -249,6 +164,22 @@ static void describe_object(struct kmem_cache *cache, void *object, describe_object_addr(cache, object, addr); } +static inline bool kernel_or_module_addr(const void *addr) +{ + if (addr >= (void *)_stext && addr < (void *)_end) + return true; + if (is_module_address((unsigned long)addr)) + return true; + return false; +} + +static inline bool init_task_stack_addr(const void *addr) +{ + return addr >= (void *)&init_thread_union.stack && + (addr <= (void *)&init_thread_union.stack + + sizeof(init_thread_union.stack)); +} + static void print_address_description(void *addr) { struct page *page = addr_to_page(addr); @@ -326,29 +257,38 @@ static void print_shadow_for_address(const void *addr) } } +static bool report_enabled(void) +{ + if (current->kasan_depth) + return false; + if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + return true; + return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); +} + void kasan_report_invalid_free(void *object, unsigned long ip) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); pr_err("\n"); print_address_description(object); pr_err("\n"); print_shadow_for_address(object); - kasan_end_report(&flags); + end_report(&flags); } static void kasan_report_error(struct kasan_access_info *info) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); - print_error_description(info); + print_error_description(info, get_bug_type(info)); pr_err("\n"); - if (!addr_has_shadow(info)) { + if (!addr_has_shadow(info->access_addr)) { dump_stack(); } else { print_address_description((void *)info->access_addr); @@ -356,41 +296,7 @@ static void kasan_report_error(struct kasan_access_info *info) print_shadow_for_address(info->first_bad_addr); } - kasan_end_report(&flags); -} - -static unsigned long kasan_flags; - -#define KASAN_BIT_REPORTED 0 -#define KASAN_BIT_MULTI_SHOT 1 - -bool kasan_save_enable_multi_shot(void) -{ - return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); - -void kasan_restore_multi_shot(bool enabled) -{ - if (!enabled) - clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); - -static int __init kasan_set_multi_shot(char *str) -{ - set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); - return 1; -} -__setup("kasan_multi_shot", kasan_set_multi_shot); - -static inline bool kasan_report_enabled(void) -{ - if (current->kasan_depth) - return false; - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) - return true; - return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); + end_report(&flags); } void kasan_report(unsigned long addr, size_t size, @@ -398,7 +304,7 @@ void kasan_report(unsigned long addr, size_t size, { struct kasan_access_info info; - if (likely(!kasan_report_enabled())) + if (likely(!report_enabled())) return; disable_trace_on_warning(); @@ -411,41 +317,3 @@ void kasan_report(unsigned long addr, size_t size, kasan_report_error(&info); } - - -#define DEFINE_ASAN_REPORT_LOAD(size) \ -void __asan_report_load##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, false, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_load##size##_noabort) - -#define DEFINE_ASAN_REPORT_STORE(size) \ -void __asan_report_store##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, true, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_store##size##_noabort) - -DEFINE_ASAN_REPORT_LOAD(1); -DEFINE_ASAN_REPORT_LOAD(2); -DEFINE_ASAN_REPORT_LOAD(4); -DEFINE_ASAN_REPORT_LOAD(8); -DEFINE_ASAN_REPORT_LOAD(16); -DEFINE_ASAN_REPORT_STORE(1); -DEFINE_ASAN_REPORT_STORE(2); -DEFINE_ASAN_REPORT_STORE(4); -DEFINE_ASAN_REPORT_STORE(8); -DEFINE_ASAN_REPORT_STORE(16); - -void __asan_report_load_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_load_n_noabort); - -void __asan_report_store_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_store_n_noabort); From patchwork Wed Aug 29 11:35:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579983 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E84BB17DE for ; Wed, 29 Aug 2018 11:36:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D813E2AA9C for ; Wed, 29 Aug 2018 11:36:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA7FA2AB91; Wed, 29 Aug 2018 11:36:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3CBB22AA9C for ; Wed, 29 Aug 2018 11:36:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 591CD6B4B89; Wed, 29 Aug 2018 07:35:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 51BFB6B4B8A; Wed, 29 Aug 2018 07:35:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BBF46B4B8B; Wed, 29 Aug 2018 07:35:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id D359A6B4B8A for ; Wed, 29 Aug 2018 07:35:50 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id u12-v6so3328368wrc.1 for ; Wed, 29 Aug 2018 04:35:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RptGi1N09kqi5nALd3Lw1kS3KWHrO9/wmh1vLWiCPsQ=; b=UuxC4gh+M0KrgQDkweSSNC3/ieEUFUcWTRKhwpZC+nSDNIa8Drc8FzNUSVLYAKAUIC NVmT4fJQE9++YdBbuWhMpXM4oT5OpXK4qAUmCpeXQZunCphANOsYsKEe0lwK/1CL/uxJ F4xfMzeL3bkKR49qV097lR/LvRzan8zGxbLO6j+t3iFV9/pVHwRTxRuy4il09FmnMmtP xUyaCkIIzZVLGWskBKQilzgtMNGBG2FoLJXBoZlxvnBrSI+CwCS6W6voUvsI8sbd0EBV 1LF+xJ0l8ojna9Wq7aJZztskccmtKBLg8IJtL1E/CZ6NkDC59wHYtN7Q2U+C5AiW9TZK dQgA== X-Gm-Message-State: APzg51DXW1W6xRh7+4G9+rbPtKSu3XqkWL1kqA9Mp7LzX2j/KfWk96xw JH7Z+0Rn/TqJ5z5eipkhrqjy7tEh+aMy9818vqndskkxQcPilVWdKB/PYp7L5/SAa7s69gnkWMe ebhEpFtsx0pRDoIMdIs+Xq035So6E5KtP/TTZRr5f6RrxQ3x0+XrmkZQmqifyuYts5liaoWaXrj wI76bIQJfBiZs23dX3zPfGBO9YZNJ2J/vstRQ1rMNAQv2pq0aYLi9LkDuimxBLF5bGPAjjiyOQx 4xzcOlVPjt549Omn4LQ0R7KcUDLbAuWymPEpFgsAJ5UBfLnk3aZkoF15mRL8y0aafE9veL1288j j4xl8p/RSoobgHLLszsgkz2ll2SgQxLJjLGhk6srgqNRd46fAVN6DtdgNh6Vzeh9s0I+JKK64YZ S X-Received: by 2002:a1c:1d0:: with SMTP id 199-v6mr4140535wmb.2.1535542550364; Wed, 29 Aug 2018 04:35:50 -0700 (PDT) X-Received: by 2002:a1c:1d0:: with SMTP id 199-v6mr4140494wmb.2.1535542549445; Wed, 29 Aug 2018 04:35:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542549; cv=none; d=google.com; s=arc-20160816; b=OzTg04Fz+i41leCFNIP8e86J6WGg/DTEGuhDZPoleKUdkEHcanjPaUDSBSHpLSsVYA BhuazgxYPo73BJnpH/MkGiilrW2/8mWbKSOa+vr4/XXhnrsWR9OgqWVWXCpfR/zep9NX vaGJUVzul2nlSA80HGr/RrKndwo1QALIY2Q7Q6pHsGC0fAAxe5UyWTobKBm0Src05myF 15so6OrX3W0BGHc4Ci0IXspdwlzM+2Vz8YtfEfWWpe9yy8X0nAC6xjEjcDVSWuGoe/RO l/VM6FBrXMPDLVU6fCRgYEq8QldHnkrXkagtrQQOorjj/AGtfA9f9Z4g0OsoSR0VPPC5 8bng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=RptGi1N09kqi5nALd3Lw1kS3KWHrO9/wmh1vLWiCPsQ=; b=fmAppIseiVXtEdn9BWpcLjRoJxxw/7tt/OH6K2YQA5FvdiwWJE2/wE7F672kf5K605 2XG2vZCQzBYw/oCBFBuPNudTJOEQhIaYPZR6bGIOu9u2uDtX3mRmhIaAtgW3zsBykFQ0 +zFjicPkdqbBZWVHIyajU+5sTTUwsRIPOp2MxbSO4eqHcMcWqmoGcZIW/2Nqq3xFTsH4 9ZtdfftxA5mVjL3yG1auBzmuLUmkWsWFZfT6Jf338VEngJgUnP4Eig9yV9Ox1RuaAhWa omiGRU8ASxaph0xqySsIrDMPmNozHeolEBxldkmLH2KRfjf8HPUuK73MN2Jzs7KbHAad BJLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sprGRxk6; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id t20-v6sor1077853wmt.83.2018.08.29.04.35.49 for (Google Transport Security); Wed, 29 Aug 2018 04:35:49 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sprGRxk6; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RptGi1N09kqi5nALd3Lw1kS3KWHrO9/wmh1vLWiCPsQ=; b=sprGRxk69Bgq2fEeJZYij7WuPS5dd42cNx8LOGTtKuHyxeGw9qNWBhl7CaazsAv2L8 0Z5a+YfEPdFTzLPIEMD0qo3Onb3R87L1f883KaLzvddIJcQv8umtpkJKssdcauC6ACw/ L2Q2LRhzqv+ozMgzDO81VYgZMrR0AQeQbFew/ywJAN8mDBTqbD+n6xgBHehW5wYdR8AX 3TJ2uGALKi2Ff9a9/USYAZSCJWtp8SCmomSn8P5XwrObk8hFwmvXLbEkrjX3O0GIfLyL Q989jfOQUiPJoa45r149PVVlaqKZtoDfLMB5LI0YyIsYbrKIphQZgUdF1iq5rf0z9kKm ZgIw== X-Google-Smtp-Source: ANB0VdapbVuh8UM8tmw1fbsxfHLxbzYLXGvQJN/t9MYIn48sIEE2gDAiYdQOxGPzZ6wTAybzKarjqw== X-Received: by 2002:a1c:8406:: with SMTP id g6-v6mr3789441wmd.18.1535542548680; Wed, 29 Aug 2018 04:35:48 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:47 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 13/18] khwasan: add bug reporting routines Date: Wed, 29 Aug 2018 13:35:17 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds rountines, that print KHWASAN error reports. Those are quite similar to KASAN, the difference is: 1. The way KHWASAN finds the first bad shadow cell (with a mismatching tag). KHWASAN compares memory tags from the shadow memory to the pointer tag. 2. KHWASAN reports all bugs with the "KASAN: invalid-access" header. This is done, so various external tools that already parse the kernel logs looking for KASAN reports wouldn't need to be changed. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 3 +++ mm/kasan/kasan.h | 7 +++++ mm/kasan/kasan_report.c | 7 ++--- mm/kasan/khwasan_report.c | 21 +++++++++++++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ 5 files changed, 64 insertions(+), 31 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 1f852244e739..4424359a9dfa 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -174,6 +174,9 @@ void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr); void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, const void *addr); +void kasan_report(unsigned long addr, size_t size, + bool write, unsigned long ip); + #else /* CONFIG_KASAN_HW */ static inline void khwasan_init(void) { } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 82672473740c..d60859d26be7 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -119,8 +119,15 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); +#ifdef CONFIG_KASAN_HW +void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { } +#endif + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/kasan_report.c b/mm/kasan/kasan_report.c index 2d8decbecbd5..fdf2d77e3125 100644 --- a/mm/kasan/kasan_report.c +++ b/mm/kasan/kasan_report.c @@ -33,10 +33,10 @@ #include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr(const void *addr, size_t size) +void *find_first_bad_addr(void *addr, size_t size) { u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; + void *first_bad_addr = addr; while (!shadow_val && first_bad_addr < addr + size) { first_bad_addr += KASAN_SHADOW_SCALE_SIZE; @@ -50,9 +50,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info) const char *bug_type = "unknown-crash"; u8 *shadow_addr; - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* diff --git a/mm/kasan/khwasan_report.c b/mm/kasan/khwasan_report.c index 2edbc3c76be5..51238b404b08 100644 --- a/mm/kasan/khwasan_report.c +++ b/mm/kasan/khwasan_report.c @@ -37,3 +37,24 @@ const char *get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } + +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *untagged_addr = reset_tag(addr); + u8 *shadow = (u8 *)kasan_mem_to_shadow(untagged_addr); + void *first_bad_addr = untagged_addr; + + while (*shadow == tag && first_bad_addr < untagged_addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow = (u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); +} diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 155247a6f8a8..e031c78f2e52 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) } __setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info, - const char *bug_type) +static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - bug_type, (void *)info->ip); + get_bug_type(info), (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, info->access_addr, current->comm, task_pid_nr(current)); @@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip) start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); + print_tags(get_tag(object), reset_tag(object)); + object = reset_tag(object); pr_err("\n"); print_address_description(object); pr_err("\n"); @@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags); } -static void kasan_report_error(struct kasan_access_info *info) -{ - unsigned long flags; - - start_report(&flags); - - print_error_description(info, get_bug_type(info)); - pr_err("\n"); - - if (!addr_has_shadow(info->access_addr)) { - dump_stack(); - } else { - print_address_description((void *)info->access_addr); - pr_err("\n"); - print_shadow_for_address(info->first_bad_addr); - } - - end_report(&flags); -} - void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { struct kasan_access_info info; + void *tagged_addr; + void *untagged_addr; + unsigned long flags; if (likely(!report_enabled())) return; disable_trace_on_warning(); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr = reset_tag(tagged_addr); + + info.access_addr = tagged_addr; + if (addr_has_shadow(untagged_addr)) + info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; - kasan_report_error(&info); + start_report(&flags); + + print_error_description(&info); + if (addr_has_shadow(untagged_addr)) + print_tags(get_tag(tagged_addr), info.first_bad_addr); + pr_err("\n"); + + if (addr_has_shadow(untagged_addr)) { + print_address_description(untagged_addr); + pr_err("\n"); + print_shadow_for_address(info.first_bad_addr); + } else { + dump_stack(); + } + + end_report(&flags); } From patchwork Wed Aug 29 11:35:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579989 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A04F917DE for ; Wed, 29 Aug 2018 11:36:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F4E92AA9C for ; Wed, 29 Aug 2018 11:36:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 824F22AB44; Wed, 29 Aug 2018 11:36:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A82202ABEF for ; Wed, 29 Aug 2018 11:36:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6584C6B4B8B; Wed, 29 Aug 2018 07:35:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 606926B4B8C; Wed, 29 Aug 2018 07:35:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40B9A6B4B8D; Wed, 29 Aug 2018 07:35:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f71.google.com (mail-wm0-f71.google.com [74.125.82.71]) by kanga.kvack.org (Postfix) with ESMTP id CC4226B4B8B for ; Wed, 29 Aug 2018 07:35:52 -0400 (EDT) Received: by mail-wm0-f71.google.com with SMTP id z11-v6so2744463wma.4 for ; Wed, 29 Aug 2018 04:35:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8D+0d2svCGP44PBWvbZeNTmiiLzDyhC5ASiSUcDXIBM=; b=it8N1mcJvL1vqKomN8947V+9tBFFDPge4jUZU3oPVz5+jQeQwzjBwsvI1B6+7KKn9U GWmjqjRyEmleJTFZtdv6BNCjgvV6BMM5Qsbu2uoPm3SPHS0D0HC4N488U2o+WH4qNcgw pH1EO694N8u86IrVINjHjxhMqilfKvm8jHxcPI3rFtpOoqRnrUnJEXLA1BfKE/NXdMGt YTHBAxO5IlgfZtyTFoDq3plM6w/ptJiXwrXEGxXc4ppi4TPwLNBY/HUQVR26dNs46Gsp NiVw4AYNCelaE/qV2s9ql/iv6/UjNwD4h9l6/1x7IZnFLAVhe1xHsMwd/eZah9JONwSd p1wA== X-Gm-Message-State: APzg51Dt9F/eMR9sC4+20y/HI0e4bKkWcp0ISs37jFYRi8wzUJjpYvd9 jBKCNyIMTdtqxBzel17h/cFCkTCF8KXFB2KK0dK3Ph31xCM1Bt5jGidrczxwGP2JECXN/H2FFyB G9oWOW2MBuLXFrOHhhyxUAVKCXm4F3pHL5cDOT0hiCr0Rm17pIToXYSzjn8I2xCjTrvAxWxS6ad Gaci8jqeIrag3Yso5TF4hZtkqoyaduiQByoS6YYd2fGWBWzCXz5tdF476wtA3ySafllZEjTkkAO pIijxlHQbVR6nEU2XYasKrG8KC9rbYeb42llp/1SiAsWdYvEzBeJX26jwyvuHBpML3D1o3JShlf OFi4KOOPOda/PpFdQK+tKlZKM/msqGtcmW8xvqJ3MDzak9l3dTh/RyVx7Sh8zCAGVQHZEAtjAhy 3 X-Received: by 2002:adf:f7c4:: with SMTP id a4-v6mr4242039wrq.86.1535542552320; Wed, 29 Aug 2018 04:35:52 -0700 (PDT) X-Received: by 2002:adf:f7c4:: with SMTP id a4-v6mr4241987wrq.86.1535542551216; Wed, 29 Aug 2018 04:35:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542551; cv=none; d=google.com; s=arc-20160816; b=y5EDJtHB+ZPlQNQvmuuhtzjPNBoZAFnn0H3UBlX6oscRf1yPfsHDk54TEEEyXW+mCZ C8qh5rZkX8oepoik0ZLKFJ5qGR7P7d/tzC2LbwGi2K0gFA8JtPQE9/dctU8Bry2Pj2n/ Nayon3lsPdHTEyHTPRHSzZobI4ulSZX1hJYnwdK7DDAIFKYLbHWYLMB4b11VqW2J8UGV plxjKn7IQsSIbENxl1tJ0GzdY7de/SQzLu382pO8hsUMKxRenbt1w+VHK0GBGkTf1KJ+ PTehDtYFCTnfXL/yE6FcFJ2IlUzu7IcDG8u92clXEKchZh5C2dGezPZiUvhbYm/7tpGi CfBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=8D+0d2svCGP44PBWvbZeNTmiiLzDyhC5ASiSUcDXIBM=; b=vJqHLgm2Z29z49xScKE/QG4okgGKEh4s3IkTdhauf5TNnO1OJNWeUXe5228O+ONHAl 4yQWDOoJcj2jzERRCeXKYaQVj3Bj857PSISfIZnHajOFwyyhNRE4L+m73cbgst54LpIB lH3YFwTIc2avG6Yvz9tAAmHno3fPzaYhAoQvWcVxWehfakAwGRvuXTJkp5bN5IxwTMBU q8Fjx6Jej6T3xyvrCpxrKhxehnfQx4DOEC8cib8tzGh6pAcrATPMfkerQ9brvIYt6+8A hFtDrb0FBIPseByaOT8tQJkMYAFfb4Rks4IbUwIRkPcn5p8Q7dPWNLLrz7wit/iJCb3t lZsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=QtQcEXUy; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y6-v6sor2589785wrh.27.2018.08.29.04.35.51 for (Google Transport Security); Wed, 29 Aug 2018 04:35:51 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=QtQcEXUy; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8D+0d2svCGP44PBWvbZeNTmiiLzDyhC5ASiSUcDXIBM=; b=QtQcEXUyerMEAIN4bI7SNvX7nNqbWN5LK9jTYoIJDkqY5Q5c3fSKZ4xkqnhK9MFDH/ tZ9g2GxDgWDATPmlwURmxQ+hw4VaOJgNOonZ1O8zuntjsIEf3nCWGzh9QuY2BJ5p5pvX GVycuBJ+Pfb4J6uHCHIMg76EGZBuXijowmPpcK6jCIDpIYfkMmar5+e2RKM5Os25zbSI Y3T+Z4Lf2qcJeMs/jN4+8DrpcWCHKWAAqy6eQZtN+4Mcr58qr3NnAQjZv3PNHhuEqN7W nNVwXzCOjz9LD0TmQYWo3UIaz7r3V30d0D9HX56VyZs/FFrq1VBjZWlPvPfue7B5pou/ Zj+g== X-Google-Smtp-Source: ANB0VdYSvLtsgBG/WuBH81PfDXVs9z6Iikx300/AlzFlnV6Vf6FKqUSQ1UyHSHnNUp9vHvBbouUo4g== X-Received: by 2002:adf:fe06:: with SMTP id n6-v6mr4196360wrr.171.1535542550487; Wed, 29 Aug 2018 04:35:50 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:49 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 14/18] khwasan: add hooks implementation Date: Wed, 29 Aug 2018 13:35:18 +0200 Message-Id: <4267d0903e0fdf9c261b91cf8a2bf0f71047a43c.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds KHWASAN specific hooks implementation and adjusts common KASAN and KHWASAN ones. 1. When a new slab cache is created, KHWASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16). 2. On each kmalloc KHWASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer. 3. On each kfree KHWASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs. The rest of the logic of the hook implementation is very much similar to the one provided by KASAN. KHWASAN saves allocation and free stack metadata to the slab object the same was KASAN does this. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 82 +++++++++++++++++++++++++++++++++++----------- mm/kasan/kasan.h | 8 +++++ mm/kasan/khwasan.c | 40 ++++++++++++++++++++++ 3 files changed, 111 insertions(+), 19 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index bed8e13c6e1d..938229b26f3a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -140,6 +140,9 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; + /* Perform shadow offset calculation based on untagged address */ + address = reset_tag(address); + shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -148,11 +151,20 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) void kasan_unpoison_shadow(const void *address, size_t size) { - kasan_poison_shadow(address, size, 0); + u8 tag = get_tag(address); + + /* Perform shadow offset calculation based on untagged address */ + address = reset_tag(address); + + kasan_poison_shadow(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; + + if (IS_ENABLED(CONFIG_KASAN_HW)) + *shadow = tag; + else + *shadow = size & KASAN_SHADOW_MASK; } } @@ -200,8 +212,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + if (unlikely(PageHighMem(page))) + return; + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) @@ -235,6 +248,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { unsigned int orig_size = *size; + unsigned int redzone_size = 0; int redzone_adjust; /* Add alloc meta. */ @@ -242,20 +256,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size += sizeof(struct kasan_alloc_meta); /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); if (redzone_adjust > 0) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); + max(*size, cache->object_size + redzone_size)); /* * If the metadata doesn't fit, don't enable KASAN at all. @@ -268,6 +282,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, return; } + cache->align = round_up(cache->align, KASAN_SHADOW_SCALE_SIZE); + *flags |= SLAB_KASAN; } @@ -328,15 +344,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) return kasan_kmalloc(cache, object, cache->object_size, flags); } +static inline bool shadow_invalid(u8 tag, s8 shadow_byte) +{ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return shadow_byte < 0 || + shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + else + return tag != (u8)shadow_byte; +} + static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { s8 shadow_byte; + u8 tag; + void *tagged_object; unsigned long rounded_up_size; + tag = get_tag(object); + tagged_object = object; + object = reset_tag(object); + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { - kasan_report_invalid_free(object, ip); + kasan_report_invalid_free(tagged_object, ip); return true; } @@ -345,20 +376,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return false; shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); + if (shadow_invalid(tag, shadow_byte)) { + kasan_report_invalid_free(tagged_object, ip); return true; } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) return false; set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); quarantine_put(get_free_info(cache, object), cache); - return true; + + return IS_ENABLED(CONFIG_KASAN_GENERIC); } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -371,6 +404,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, { unsigned long redzone_start; unsigned long redzone_end; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -383,14 +417,24 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(object, size); + /* + * Objects with contructors and objects from SLAB_TYPESAFE_BY_RCU slabs + * have tags preassigned and are already tagged. + */ + if (IS_ENABLED(CONFIG_KASAN_HW) && + (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU)) + tag = get_tag(object); + else + tag = random_tag(); + + kasan_unpoison_shadow(set_tag(object, tag), size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); - return (void *)object; + return set_tag(object, tag); } EXPORT_SYMBOL(kasan_kmalloc); @@ -440,7 +484,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { + if (reset_tag(ptr) != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -453,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (ptr != page_address(virt_to_head_page(ptr))) + if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d60859d26be7..6f4f2ebf5f57 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,10 +12,18 @@ #define KHWASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KHWASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KHWASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KHWASAN_TAG_INVALID +#endif + #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ /* diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c index 9d91bf3c8246..6b1309278e39 100644 --- a/mm/kasan/khwasan.c +++ b/mm/kasan/khwasan.c @@ -106,15 +106,52 @@ void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { + u8 tag; + u8 *shadow_first, *shadow_last, *shadow; + void *untagged_addr; + + tag = get_tag((const void *)addr); + + /* Ignore accesses for pointers tagged with 0xff (native kernel + * pointer tag) to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KHWASAN_TAG_KERNEL (0xFF). + */ + if (tag == KHWASAN_TAG_KERNEL) + return; + + untagged_addr = reset_tag((const void *)addr); + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + size - 1); + + for (shadow = shadow_first; shadow <= shadow_last; shadow++) { + if (*shadow != tag) { + kasan_report(addr, size, write, ret_ip); + return; + } + } } #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -126,15 +163,18 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { + kasan_poison_shadow((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); From patchwork Wed Aug 29 11:35:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579993 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 962C61709 for ; Wed, 29 Aug 2018 11:36:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 86C4A2ABDF for ; Wed, 29 Aug 2018 11:36:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7A4E92AB91; Wed, 29 Aug 2018 11:36:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED9442AB44 for ; Wed, 29 Aug 2018 11:36:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC6756B4B8C; Wed, 29 Aug 2018 07:35:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A4D346B4B8E; Wed, 29 Aug 2018 07:35:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C98E6B4B8F; Wed, 29 Aug 2018 07:35:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70]) by kanga.kvack.org (Postfix) with ESMTP id 2CC816B4B8C for ; Wed, 29 Aug 2018 07:35:54 -0400 (EDT) Received: by mail-wm0-f70.google.com with SMTP id s18-v6so2814404wmh.0 for ; Wed, 29 Aug 2018 04:35:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=CCBGVbMzHdvCWnsB/Yhk2lUOwxfWB4ZppCmhRRUM9JA=; b=aY5XsWURByk4f1k3abFNsAgVuAJLu2+/jI6jE1aQr863w6THaqyegbGM6W18y8fEUP sx1yqGHgDBvBim240u8dZfaUdxT6rvf5ej8STaONS+lwhbVEIDCm74EFGjGd+9H19vi3 trZ7ybzRR2639V5sf37OkouPbAO1pUM+3F+CJy5IDNNLSzVDI3QQm5IEegkVrjrLIc8U Xs2IsZpOlwKABfBWdI9cP0CZplcTwZSYRntUd9oKsIG2WEEViofABneaXgEbSv/u383i NG4HenMyRi/Obnr+wyN0ZglUyw03U8AFZ+zTdq1QsCumvUGEsFPzcjxlFbTvtn4zmbZg BVlw== X-Gm-Message-State: APzg51DevYAmhMAEFyuw2QeWAX6Vxcy+VjCiUK+ftoKU/riE+8F4tHvQ Ih+jtyvlfZNF/QQ9vN3t/q9H++eghkbcUEWkceUaT+Rx8K09d2xOexNwyJVi7o8KGMwmjN1XnBM v17qaylLhWyxPR8n253j73TiTpK3iITL29SGSJvkJnFsAsMrnfgabweOp04V1WhPI1S7qv9rZSz 8Qf0lZbrNZ4FbOIK6saCEG68mEHlutFDVOAnpRhYm2Ngx6PfgIWSDN42lCH5ub1L3LjgeJdedUH /lzmszlmVfxaeZpvcZ8oB9VLsJkA87qWxb/Hn5u7U8LLPlO+BLt+XR351c6KcmLifc09kmXFsFN X7f3h5cYMFXHeIbO7qEOvqsQgpXy64JNSuM/Q9gP1RyI4v4qwu+WvMZs8dTRMdfk9csBfGn+uGw j X-Received: by 2002:a5d:4e0a:: with SMTP id p10-v6mr4055231wrt.48.1535542553663; Wed, 29 Aug 2018 04:35:53 -0700 (PDT) X-Received: by 2002:a5d:4e0a:: with SMTP id p10-v6mr4055195wrt.48.1535542552776; Wed, 29 Aug 2018 04:35:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542552; cv=none; d=google.com; s=arc-20160816; b=tXHny8XZBIAALRwDOHJo0StUGGDRN76BqN8jFiBVMmdduIv265gMvFIFztkU0XGEgs nbi9dC778V6+9pr6fMFEURtCEgkYflzlUmRW/MpVex5uY6vGkJn0BP02iSU5KrLt6Yc+ ie2/9MuhA0jjyHKMnhh8PbGUSUqNZToyQSUoFG/YK+l3SJwrc6BxFBg+fFJ1KyKsvR8/ ugIXdNYUaAWTbEf9avNQ3isbCOz+agjFA4FJJ+UqKkP7g8uZTH4e5pYV4Q/pqcMTsLqZ pAZl8vdXBPc/SAImaixRZk8FE+uYFN0zyo2WLTVx12Uz48AljfcdTHMLG1Ui7DCqAo2j j0sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=CCBGVbMzHdvCWnsB/Yhk2lUOwxfWB4ZppCmhRRUM9JA=; b=wmQTmTfuZEEUecQ03sSVgmZRYZq6RwzIYkws9CIxUNB/EkQ7wPS8fi6+00e4eaGzKR dPl5ldxxp20W6j76u/dHKT1jiOCDOrMz34ko7huwYChUJP9Y1p9MFyGUYyVnGppnNBO3 hDKkgxcDoAb6coO6iq3N+AcYx+8/g/iDnkpEQBzEY3O1Jc7PL/SD9JhsGUwK/KTNGUig TOVzqZuOWX3SKF+b5+64GhS6jvwl8Y93x3IlmaQgHyzKOvpA45qaEmaY1pJ1yXZD9voh +cM68fzygH+n2flS7DS5e7aWbVEq6eLTUtAxmwMV54vdxUWTv2rHaCURH7Lsaje8bg29 fooQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=P+4HLBVN; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id o10-v6sor2605047wrf.33.2018.08.29.04.35.52 for (Google Transport Security); Wed, 29 Aug 2018 04:35:52 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=P+4HLBVN; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CCBGVbMzHdvCWnsB/Yhk2lUOwxfWB4ZppCmhRRUM9JA=; b=P+4HLBVNOxPKN9sWnmTedOcFE/v3k7mqmCD4VSAgFSMl7of3OTjZlSlH3AnkitKzKX JAU41HXyYVWcUHQAIswpIi1IcmwpgAEdWg2AJBHPz3piJeWXckqIhM5+FxSDEMK4gYbP Jqhj6ue6k2WMJZrncYGU96Fx+RXGfhyWuPXz0Oe8l7GHkmQnmvh5ZRx9KTsdzO65tUQg bqgezWXIEtPPcAp9SKFZNI8q8MTba6UZXLfbPIDXEr23IYXon4oTWUDX/O78Yt1INXVl fKdJjPeFsXUUxxufErLlE5iMPDNfHZtEA3hpjNEMu12w1uep9wsis+IuSqkZ7PZamP9M J8PA== X-Google-Smtp-Source: ANB0VdY7wAqGQmJ0i2uIa33LwCvnvSHs2CfWQfVuo6/YpmsR4i13uiWmzLzd40PO+KO2QnZtwWBklw== X-Received: by 2002:a5d:4648:: with SMTP id j8-v6mr3974471wrs.52.1535542552181; Wed, 29 Aug 2018 04:35:52 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:51 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation Date: Wed, 29 Aug 2018 13:35:19 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit add a KHWASAN brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 69 +++++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/brk-imm.h b/arch/arm64/include/asm/brk-imm.h index ed693c5bcec0..e4a7013321dc 100644 --- a/arch/arm64/include/asm/brk-imm.h +++ b/arch/arm64/include/asm/brk-imm.h @@ -16,10 +16,12 @@ * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps + * 0x9xx: KHWASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 #define BUG_BRK_IMM 0x800 +#define KHWASAN_BRK_IMM 0x900 #endif diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 039e9ff379cc..fd70347d1ce7 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -269,10 +270,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +void __arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +{ + __arm64_skip_faulting_instruction(regs, size); /* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -775,7 +780,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) } /* If thread survives, skip over the BUG instruction and continue: */ - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); return DBG_HOOK_HANDLED; } @@ -785,6 +790,59 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_ESR_RECOVER 0x20 +#define KHWASAN_ESR_WRITE 0x10 +#define KHWASAN_ESR_SIZE_MASK 0x0f +#define KHWASAN_ESR_SIZE(esr) (1 << ((esr) & KHWASAN_ESR_SIZE_MASK)) + +static int khwasan_handler(struct pt_regs *regs, unsigned int esr) +{ + bool recover = esr & KHWASAN_ESR_RECOVER; + bool write = esr & KHWASAN_ESR_WRITE; + size_t size = KHWASAN_ESR_SIZE(esr); + u64 addr = regs->regs[0]; + u64 pc = regs->pc; + + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + kasan_report(addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KHWASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; same is true for KASAN; + * this is controlled by current->kasan_depth). All these accesses are + * detected by the tool, even though the reports for them are not + * printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KHWASAN", regs, 0); + + /* If thread survives, skip over the brk instruction and continue: */ + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + return DBG_HOOK_HANDLED; +} + +#define KHWASAN_ESR_VAL (0xf2000000 | KHWASAN_BRK_IMM) +#define KHWASAN_ESR_MASK 0xffffff00 + +static struct break_hook khwasan_break_hook = { + .esr_val = KHWASAN_ESR_VAL, + .esr_mask = KHWASAN_ESR_MASK, + .fn = khwasan_handler, +}; +#endif + /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -792,6 +850,10 @@ static struct break_hook bug_break_hook = { int __init early_brk64(unsigned long addr, unsigned int esr, struct pt_regs *regs) { +#ifdef CONFIG_KASAN_HW + if ((esr & KHWASAN_ESR_MASK) == KHWASAN_ESR_VAL) + return khwasan_handler(regs, esr) != DBG_HOOK_HANDLED; +#endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; } @@ -799,4 +861,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr, void __init trap_init(void) { register_break_hook(&bug_break_hook); +#ifdef CONFIG_KASAN_HW + register_break_hook(&khwasan_break_hook); +#endif } From patchwork Wed Aug 29 11:35:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579995 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52CF21709 for ; Wed, 29 Aug 2018 11:36:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 422D22AA9C for ; Wed, 29 Aug 2018 11:36:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3583F2AB91; Wed, 29 Aug 2018 11:36:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 774712AA9C for ; Wed, 29 Aug 2018 11:36:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EECF66B4B8F; Wed, 29 Aug 2018 07:35:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E1E3C6B4B90; Wed, 29 Aug 2018 07:35:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C95F26B4B91; Wed, 29 Aug 2018 07:35:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 64E936B4B8F for ; Wed, 29 Aug 2018 07:35:56 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id l45-v6so3334421wre.4 for ; Wed, 29 Aug 2018 04:35:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hOQLXIfZsQqSVQUxeIhnGNQSnIjX11Dj2Kz0Wy/GlMc=; b=ILwJDAY+J8HdVxjgLOni28cW87rcMUyan7Aw8pApn7aoX4jis65YkWs4FZPcudq0hA aAe2ws3D5BTXNqMGktvoBeS5+ls8Oh4Hb+LTAt/gxRwkgFpi4HQInLKcIknS1OhNN5lD U4cEjtPNjvEddiL/F/GeWO/mmTIc79PFsOs1tGrsx4YZ4XG8SwLVMJNUpIRcxgcGNLH0 hGF718tZAFVBZeZwg3sgcBi9tuEAn0JIDk5+j0mJHQn6kEnfxEeRH2yD4ojUDbSiETPO qJTJq9J8pO0/vZlxzQ910KXNAJzduDDa7XqJ+HGW8uU2RJ8UaMxzMz2oJGp4eyYK/FGQ /73g== X-Gm-Message-State: APzg51Bs3jcNRfcDz+gXlBm42KYnDUesOca1xGNFxHBslk1k3YyxtG58 QRJDU3U9DdPu8OXCqruC5SO2bfLrTQR/R+AWFA4zIcCjByQ/6kGlWhxXxn+gU0psFjfUNTiwKV8 rURMhDNGg2mMQ5M4b2NbXY7tfEfkrEJ1JGYuqzMt5H2CAlnkP5JcaJRwH7x0EWn4I8Rjvz99RHn SRbDgQcZz7uV0sOfjQTX32gSVnz/pumzzIP+3Cbr4aN1NEae42zxhZ/J39wbUg6vKyzqwV2tGsi iVa1UBGjv5DUv5xYumFNLXVqGqXtznX4JRQ1d9HOGbIkeFUQGrcxLSYEqmX6BfGxsRXOhFzce5t LwGOtPzA2D9LoruTal4szvwt/WkE3hOcDqEM4dyAazORuDDj8BMWGgnFRbBy9Xgqqb5zDftKmgp X X-Received: by 2002:adf:f608:: with SMTP id t8-v6mr4069560wrp.186.1535542555921; Wed, 29 Aug 2018 04:35:55 -0700 (PDT) X-Received: by 2002:adf:f608:: with SMTP id t8-v6mr4069508wrp.186.1535542554878; Wed, 29 Aug 2018 04:35:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542554; cv=none; d=google.com; s=arc-20160816; b=HECBCVLhPcWKZio7++iROQeX33prTMwxtTxMrqgaMROa3dTseeF9jUdD6zqPikvlAj R3SN4Orznsjcczt+Ek1/aWkaabVmvZTEyr7XwFRMGqCQ/2UmC67LhExuaF8SBKBt9fZa /jSLnhMmmzA4ujTKG7QkIpLd/AWcXI23R1zxil4+m7vdqDOYqgC4VuRDrlachyppqKGn I6VaHunCjBgXUM117eQZIrNpZ5CWx/fFlZhd8c40py0VkbHHiMegdIvP9N8nHcm3nMff LLpeMy0Y9MQpUTnSs56QrUbWm3LnBbUflmBmbQDOmBkQkc/2XxHohlfSWhy1AdS3BPpu ptew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=hOQLXIfZsQqSVQUxeIhnGNQSnIjX11Dj2Kz0Wy/GlMc=; b=FkGaIOpb/kBKxSTajU6idmsSuA5MHKsD7H7LObYGaHBlS6E610Y4AXl5Fs6rjT/0Ph /jqTznTbWSII2jWa+wIxIs2DYN8yef0rml8AhIUC+pK6BmGSO53R401ICyQVvTuQoe0W lCp1C0UsJVBwbkzJTSam2b09q9+JCp/UBLOm1i1DcdLst4kA9/zcyN/2KkE/+CE/ikxT RTttkRM+V3sKwPOkMsdyj3tpmLvfa5Nimfm5SOD8uHKpl8e/YQdRwo9Dsdn+vziWqHLd Hg8ZiH81l70osdi+9C3bWhOD9x40rLUeY9KyRe0SEZ3FvnsdqbZ5sHWcyq2xvodvVgSv AzhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=TaZcvmI0; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id w3-v6sor1100064wmf.50.2018.08.29.04.35.54 for (Google Transport Security); Wed, 29 Aug 2018 04:35:54 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=TaZcvmI0; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hOQLXIfZsQqSVQUxeIhnGNQSnIjX11Dj2Kz0Wy/GlMc=; b=TaZcvmI0t1XFHEh6yOlxuZw05ltSwTYH4y2haKbNdn63CoUCzRt6VcwIqTDxNGL66A GxKmJ6FhmLfN0zt6eC4DnrrCu0SjZFE2t9xoGzQFDm6maNps6Z7dZ0baKBn7LUpK1G1H O01pK5A/HKL4rtlqDa0fkBjop51hE4findPDL1M/OdmsupWe4crxxtc+EDm2JIw6WWoI WKUF67a7gkfI/+QYir8h6LyNt5LtsIV4cKYnUPkTe1dOx+tlYYTsL7cxds1xuolc++Kp /HbkOdy965OxZlviklRmvThGhn+lu8ELFpL9/y1dehcGyHjOzHRpBB2b3xM+I3HZuWbp v0BA== X-Google-Smtp-Source: ANB0VdZtQ5/7G9QQ7MUik7zJ4qXfPY1KcSVwoBTA9GmRMM+L4+4n4J7u33BYwOaHbNL4MdJFu7wVeA== X-Received: by 2002:a1c:2283:: with SMTP id i125-v6mr4103692wmi.28.1535542554046; Wed, 29 Aug 2018 04:35:54 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:53 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 16/18] khwasan, mm, arm64: tag non slab memory allocated via pagealloc Date: Wed, 29 Aug 2018 13:35:20 +0200 Message-Id: X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KWHASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently. For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag. This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 10 ++++++++++ include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 17 +++++++++++++++-- mm/page_alloc.c | 1 + 6 files changed, 76 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index f5e2953b7009..ea7f928aba31 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -312,7 +312,17 @@ static inline void *phys_to_virt(phys_addr_t x) #define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page)) #define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) +#ifndef CONFIG_KASAN_HW #define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#else +#define page_to_virt(page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = KASAN_SET_TAG(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) +#endif + #define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START)) #define _virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \ diff --git a/include/linux/mm.h b/include/linux/mm.h index a61ebe8ad4ca..a1e7c590d925 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -804,6 +804,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH) #define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH) #define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) /* * Define the bit shifts to access each section. For non-existent @@ -814,6 +815,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0)) #define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0)) #define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0)) /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS @@ -836,6 +838,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) static inline enum zone_type page_zonenum(const struct page *page) @@ -1081,6 +1084,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) } #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_HW +static inline u8 page_kasan_tag(const struct page *page) +{ + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; +} + +static inline void page_kasan_tag_reset(struct page *page) +{ + page_kasan_tag_set(page, 0xff); +} +#else +static inline u8 page_kasan_tag(const struct page *page) +{ + return 0xff; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) { } +static inline void page_kasan_tag_reset(struct page *page) { } +#endif + static inline struct zone *page_zone(const struct page *page) { return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)]; diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7ec86bf31ce4..8dbad17664c2 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -82,6 +82,16 @@ #define LAST_CPUPID_WIDTH 0 #endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH+LAST_CPUPID_WIDTH+KASAN_TAG_WIDTH \ + > BITS_PER_LONG - NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif + /* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit. diff --git a/mm/cma.c b/mm/cma.c index 4cb76121a3ab..c7b39dd3b4f6 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + size_t i; struct page *page = NULL; int ret = -ENOMEM; @@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, trace_cma_alloc(pfn, page, count, align); + /* + * CMA can allocate multiple page blocks, which results in different + * blocks being marked with different tags. Reset the tags to ignore + * those page blocks. + */ + if (page) { + for (i = 0; i < count; i++) + page_kasan_tag_reset(page + i); + } + if (ret && !no_warn) { pr_err("%s: alloc failed, req-size: %zu pages, ret: %d\n", __func__, count, ret); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 938229b26f3a..e5648f4218eb 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -212,8 +212,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { + u8 tag; + unsigned long i; + if (unlikely(PageHighMem(page))) return; + + tag = random_tag(); + for (i = 0; i < (1 << order); i++) + page_kasan_tag_set(page + i, tag); kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } @@ -311,6 +318,12 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, void kasan_poison_slab(struct page *page) { + unsigned long i; + + if (IS_ENABLED(CONFIG_SLAB)) + page->s_mem = reset_tag(page->s_mem); + for (i = 0; i < (1 << compound_order(page)); i++) + page_kasan_tag_reset(page + i); kasan_poison_shadow(page_address(page), PAGE_SIZE << compound_order(page), KASAN_KMALLOC_REDZONE); @@ -484,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (reset_tag(ptr) != page_address(page)) { + if (ptr != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -497,7 +510,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) + if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e75865d58ba7..eb5627f89853 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1177,6 +1177,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, init_page_count(page); page_mapcount_reset(page); page_cpupid_reset_last(page); + page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL From patchwork Wed Aug 29 11:35:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10579997 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44F0C17DE for ; Wed, 29 Aug 2018 11:36:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 33C282AA9C for ; Wed, 29 Aug 2018 11:36:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 275742AB91; Wed, 29 Aug 2018 11:36:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B8202AA9C for ; Wed, 29 Aug 2018 11:36:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4ABC36B4B92; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 409D56B4B95; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E8DA6B4B94; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 993116B4B91 for ; Wed, 29 Aug 2018 07:35:58 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id j22-v6so3258768wre.7 for ; Wed, 29 Aug 2018 04:35:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vJC5xQ8pJGrlY0SOUXZh+qYJjhRZ5GDqSExb58jFjnI=; b=McwEiV0bdvZiSkFt2SBXQ3Kqj0YplC3mJ3DrB4a7khJpkEUMPk4wMVoSy/d+RIyQCI h9/pG68BkxUaXVEsO39wBnZ2HPyp9Xu5eEmBhBQUykE7Yqx0K0lbEXSy2bhf+QxoiIfp w88mKriHZhiUl9Z5Zvuj2DthzdIhH5IZ25SO9zpm5NyNDMjBTf0Q/yH6vyrribOVonjs 2yyouavOyT86siRCHO1DQ9jI6rU/TCXjtPv/lR+PYzthu0UAE3keyRhE3ssBYYJ7x+BD hEi3r02kZaNqdClZQr7JWT7Y6lAjZcG9TEfkz6SjQItpbImWIKAx12Gmbce/7BX9ynbU 6tAQ== X-Gm-Message-State: APzg51CwnQu9fPeA6wmHW0vXSEuGlf7s6KYtR46f7AR27jr37MePxT6+ c98TZOfkFSLcbkzIprmiv6GRCc37G/sEOBu2ajttbrKXBL6o/szt5LZhebKezRvbQUQIeSJFqLl aGT2B8mkj5Q0iH2ussMbnFZVbcA9UfLH1dvQQFv2YVZCQSYxd1sbIgWAVS4IRVIUMVfafymirkJ V1GL7PDVlMF0i5+NoDIYdCbOxbH3qKM512yu+7zX4kmBoB+kCi1zTcbnyLOZAzBBrU6oSZH+91e 4QhPK/5qfi+JqseTnrPjCWsY3ohEAyzquVgq51SShkRfnbwQ2fMkXf2uUCumpSQrBTV5evYiygo xZ/lMZuLyLudDo09ZHB3OeLP2ytB28Jzl5h8tAlETIMUUptK99oMcMtUvVycIvVP8lXy/sI/rT2 M X-Received: by 2002:a1c:b213:: with SMTP id b19-v6mr3794811wmf.141.1535542558086; Wed, 29 Aug 2018 04:35:58 -0700 (PDT) X-Received: by 2002:a1c:b213:: with SMTP id b19-v6mr3794745wmf.141.1535542556534; Wed, 29 Aug 2018 04:35:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542556; cv=none; d=google.com; s=arc-20160816; b=tIzKk2CwKMgDW7SwL8sOOU1QWtM8iFA72kM8lDmI7CKqrldoqJBghb+Yv6dRr4NXMo UZTbIdGRUA+9fxqTpKnyLhJB4ce5ZM12agHf7UkxfWic+mgj6oubQdflP8SX02+GaOhW ywHf96/vbmwo8YHpl50NnNh5FwakSakfFGcHxQI17jnktZEEs8gloE81kiFqd5d+koHK J7UaAP/tS3xgy31ML7QR1OIY52pSDk+ELFh1VfXampqNQoQth59IL7OrY7FWfIoXH+sv oDcHO3a/PlSM/ME1GvQ6EoesacGOQoppvZg3gFTorRPGFcoHja5PCoik12CkSg7Wgn9F 3DJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=vJC5xQ8pJGrlY0SOUXZh+qYJjhRZ5GDqSExb58jFjnI=; b=i/Mic/e9xMZpCM+TF+KUrHFvRigDkFkTyEHEbvBRVMH+TMAYHeca7+n/oaxgZa8EE2 hxoOD4kJstfuGgR3A4OmF86j6DV6Lbjl3TB8B3dM9uHZAqNlHFIwzPigdEJRUGGByTX7 GrAdUqo7QQNhMzMSEO7r3YCebdq5XU0lBy/LypME9l9i/Se2svUujpQbzh1V3D7ALwNi HnfVhA7HTS8EvkfSbExT6YoI1S5X7KVWp1zaiV7X0E7H9vZa4Qgonaovq422ETJSBZhB s1Dn4l3OJJzWxlA9TsUDYA05IbnU1QhMqCyRNx294e3tmYNHZyUxKevo8RyVtJgxCsip Y/4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=qJAcLTFV; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j21-v6sor2604490wre.30.2018.08.29.04.35.56 for (Google Transport Security); Wed, 29 Aug 2018 04:35:56 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=qJAcLTFV; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vJC5xQ8pJGrlY0SOUXZh+qYJjhRZ5GDqSExb58jFjnI=; b=qJAcLTFVf4jjzOuUrHuSgg4X1b5aKZN+flrCId6IaSxzXLS7EW/DsiX7OzI//yDEfG EeE4wD8ykUMBNgNCoy+gq5c/86CciPaNGqdcVgIKz1sWH6eR7ocm7OIl6d1cMo888fi7 sH5VPpKeouC8Z8e6r5vriWbsWNrUxsRYsStc7pesHdbGCUT/TnPtto7cc89zvXO/Ak6Z 8h0/SWW4S2M1nC5fz0rx8bgDiVd+yJ/jTTgRsMBQMGaxFVAArJv8rLhGCjr/Dkj3tOWR VZ4mKlbbWEsz+0m+uiXq6bck3G5DilUYrFUnR0nUWK3XHA5Gt7+2aK9MfHPOoaN1GYm0 uneA== X-Google-Smtp-Source: ANB0VdYcqFyVg3uxEGMf1v0dVmH3BtpZPumwH0uAt2yr5lrUB8kJJY4hdWzm9/jq2Sndqz1dcZjC0Q== X-Received: by 2002:adf:8024:: with SMTP id 33-v6mr4063636wrk.16.1535542555697; Wed, 29 Aug 2018 04:35:55 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:54 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 17/18] khwasan: update kasan documentation Date: Wed, 29 Aug 2018 13:35:21 +0200 Message-Id: <1a3b3030b6ee01931b397583b69f3af94e2a2308.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch updates KASAN documentation to reflect the addition of KHWASAN. Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 213 +++++++++++++++++------------- 1 file changed, 123 insertions(+), 90 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index aabc8738b3d8..842d95af74d3 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -8,11 +8,19 @@ KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides a fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN has two modes: classic KASAN (a classic version, similar to user space +ASan) and KHWASAN (a version based on memory tagging, similar to user space +HWASan). -Currently KASAN is supported only for the x86_64 and arm64 architectures. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. +For classic KASAN you need GCC version 4.9.2 or later. GCC 5.0 or later is +required for detection of out-of-bounds accesses on stack and global variables. +KHWASAN in turns is only supported in clang and requires revision 330044 or +later. + +Currently classic KASAN is supported for the x86_64, arm64 and xtensa +architectures, and KHWASAN is supported only for arm64. Usage ----- @@ -21,12 +29,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC +and choose between CONFIG_KASAN_GENERIC (to enable classic KASAN) and +CONFIG_KASAN_HW (to enabled KHWASAN). You also need to choose choose between +CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and inline are compiler +instrumentation types. The former produces smaller binary while the latter is +1.1 - 2 times faster. For classic KASAN inline instrumentation requires GCC version 5.0 or later. -KASAN works with both SLUB and SLAB memory allocators. +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line @@ -43,85 +53,80 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access classic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8800696f3d3b by task insmod/2734 + + CPU: 0 PID: 2734 Comm: insmod Not tainted 4.15.0+ #98 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [] dump_stack+0x46/0x58 - [] print_trailer+0xf8/0x160 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] object_err+0x35/0x40 - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kasan_report_error+0x38a/0x3f0 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kasan_unpoison_shadow+0x14/0x40 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] __asan_store1+0x75/0xb0 - [] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kmalloc_oob_right+0x65/0x75 [test_kasan] - [] init_module+0x9/0x47 [test_kasan] - [] do_one_initcall+0x99/0x200 - [] ? __vunmap+0xec/0x160 - [] load_module+0x2cb3/0x3b20 - [] ? m_show+0x240/0x240 - [] SyS_finit_module+0x76/0x80 - [] system_call_fastpath+0x12/0x17 + __dump_stack lib/dump_stack.c:17 + dump_stack+0x83/0xbc lib/dump_stack.c:53 + print_address_description+0x73/0x280 mm/kasan/report.c:254 + kasan_report_error mm/kasan/report.c:352 + kasan_report+0x10e/0x220 mm/kasan/report.c:410 + __asan_report_store1_noabort+0x17/0x20 mm/kasan/report.c:505 + kmalloc_oob_right+0xa8/0xbc [test_kasan] lib/test_kasan.c:42 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + RIP: 0033:0x7fdd79df99da + RSP: 002b:00007fff2229bdf8 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055c408121190 RCX: 00007fdd79df99da + RDX: 00007fdd7a0b8f88 RSI: 0000000000055670 RDI: 00007fdd7a47e000 + RBP: 000055c4081200b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007fdd79df5d0a R11: 0000000000000202 R12: 00007fdd7a0b8f88 + R13: 000055c408120090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2734: + save_stack+0x43/0xd0 mm/kasan/common.c:176 + set_track+0x20/0x30 mm/kasan/common.c:188 + kasan_kmalloc+0x9a/0xc0 mm/kasan/kasan.c:372 + kmem_cache_alloc_trace+0xcd/0x1a0 mm/slub.c:2761 + kmalloc ./include/linux/slab.h:512 + kmalloc_oob_right+0x56/0xbc [test_kasan] lib/test_kasan.c:36 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + + The buggy address belongs to the object at ffff8800696f3cc0 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8800696f3cc0, ffff8800696f3d40) + The buggy address belongs to the page: + page:ffffea0001a5bcc0 count:1 mapcount:0 mapping: (null) index:0x0 + flags: 0x100000000000100(slab) + raw: 0100000000000100 0000000000000000 0000000000000000 0000000180150015 + raw: ffffea0001a8ce40 0000000300000003 ffff88006d001640 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8800696f3c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc + ffff8800696f3c80: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00 + >ffff8800696f3d00: 00 00 00 00 00 00 00 03 fc fc fc fc fc fc fc fc + ^ + ffff8800696f3d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc fc + ffff8800696f3e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works. @@ -138,18 +143,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For KHWASAN this last report section shows the memory tags around the accessed +address (see Implementation details section). + Implementation details ---------------------- +Classic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Classic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address:: @@ -162,12 +173,34 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +KHWASAN +~~~~~~~ + +KHWASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to store +a pointer tag in the top byte of kernel pointers. KHWASAN also uses shadow +memory to store memory tags associated with each 16-byte memory cell (therefore +it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation KHWASAN generates a random tag, tags allocated memory +with this tag, and embeds this tag into the returned pointer. KHWASAN uses +compile-time instrumentation to insert checks before each memory access. These +checks make sure that tag of the memory that is being accessed is equal to tag +of the pointer that is used to access this memory. In case of a tag mismatch +KHWASAN prints a bug report. + +KHWASAN also has two instrumentation modes (outline, that emits callbacks to +check memory accesses; and inline, that performs the shadow memory checks +inline). With outline instrumentation mode, a bug report is simply printed +from the function that performs the access check. With inline instrumentation +a brk instruction is emitted by the compiler, and a dedicated brk handler is +used to print KHWASAN reports. From patchwork Wed Aug 29 11:35:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10580001 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19F281709 for ; Wed, 29 Aug 2018 11:36:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 098982AA9C for ; Wed, 29 Aug 2018 11:36:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F0AFB2AB91; Wed, 29 Aug 2018 11:36:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88A4F2AB44 for ; Wed, 29 Aug 2018 11:36:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B631E6B4B91; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B13F16B4B93; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 940E26B4B94; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 245466B4B93 for ; Wed, 29 Aug 2018 07:35:59 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id z17-v6so3253972wrr.16 for ; Wed, 29 Aug 2018 04:35:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fxCqcDEOJAYWOW+coTAwwOen8+y8z0lpUdFzp+YpL8s=; b=swwGOic/SK0SpUKkhQmeTAsF/1FsHDfCMKgZAmV1jdKQfsIMBvvv58QvAlaDEgGAyy JIUv8ybPO8tn5vFVzA+7U3GsSz8xzNCLnPfP+02gD6xNYetrM37cVH6SM8BkBwa//+fQ v0nU8d7YO+pZBJXeGPZjipGy7aLFo4GCOHVaetllPvV6+hUGMvfQd9/C4GucKXwVLHJ3 XnXTNGmiogf2ODYCvOZ1VJuQDBP2g6J5VVGNpKj12UWTROGVugsmA2Vxe1ogIKher+3w 3yYLsesxhwA86CZcNtK4zt/DWQwGg4655He1Q9Ei268sfizjDusJgBUxeExJ8L7VL6Me 6Q2w== X-Gm-Message-State: APzg51BC70VXIjvC5LPrm+xgKTuXfpRrXq9jWa0l6NjobweWprEDZJTM relBMwB43hIrrVb3PalN+FbTwoZitUwcYp+vUTnNmd2ETMs5+2Bxm9muu1o7vP50TM6ZwDc+ta1 blKl/zYT68HvepQwlnBhqgBWBrPltTdJcD903hnJTrzjMLBrHUIA12RSCNXIR2x/DKIKNOsnzd+ WoSZHRvwzod3dOwt+DOy6P9gNFK9FB3hH+ccHW8JCB9YSUnNN0Brv2S729Ux/ctur6cInTYphG2 GliobYhOcibwRHvzejtwkqaVlCNqd8av6WlP7lTxkElgRcFq0lbPGFd4rWf9/0w9ZJR7Dj3XrC2 AV/mc1//SuuXfdDgHGihZU+hbYKElmKw5yRdDphRcqUo6ayh7b+mM9tiNCE036ZyXxscas9Z7JS D X-Received: by 2002:a1c:9706:: with SMTP id z6-v6mr3845627wmd.0.1535542558672; Wed, 29 Aug 2018 04:35:58 -0700 (PDT) X-Received: by 2002:a1c:9706:: with SMTP id z6-v6mr3845599wmd.0.1535542557881; Wed, 29 Aug 2018 04:35:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535542557; cv=none; d=google.com; s=arc-20160816; b=ewSSNup5P4clYaJriQxop+dV5HWGi4c3GDFpKIYV34ZivP5zx+aBoKx0qU1CMGIGPN dqSoopGzCh2BxyKy2JVnsHqucbalzX/j6ooUHfRxZOhWQ8t59azu7a3qp1bsAVPN4eE8 W3aDDS8HWZ93kNXI0ycc7rx8G5uAC93WBV2l7zYpn/InDSMRtlxYd/dKm3mtkOCk514i Y/7BQuDY4NHTSOtMCaYoCI8FyBWoU3fpAtkusiZWsy3zo6rSkZ2hrt0pfiKAT+WUXRVc nQkn2LIXxcAdlYP15ifeaQA5inxbV6pXp8+Py7qJ2caCAgDYT50iTlk9M+CaRjUX8n7V 0nBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=fxCqcDEOJAYWOW+coTAwwOen8+y8z0lpUdFzp+YpL8s=; b=bs0VxLmASFKMCZeKJ+KCKEeuktGWMiVxFcdFS+3U7XwdOhtQgp3ppkvx6kvcXikow+ ZR2fbEp/K1jOHGfkh/anZXpAb7KPRWYf6pHNJK3/GiT1exDX0gg/TEy+sJZDndbzMbkm GAdUuUGbEzDHBRTMs4zJ0fLJgujccANvDplcUe8fNU7dY/ejl+Vx9q0ptIwX/6W8Pwqb aD1WO4qfL6MLJQB5ZGrsgrCdtMYDirw+2NkMCP9f0ohi0/nNV2a4kXpsC/sztVsLK3Nv lV2iQptQGOCrftJ4WSzyDUcaWHxnqfiflX/ydsX9XMH/QOs0nYRzMDP/O/f9z6gaf20h WNiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=wBz8nJBl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 4-v6sor393144wmg.11.2018.08.29.04.35.57 for (Google Transport Security); Wed, 29 Aug 2018 04:35:57 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=wBz8nJBl; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fxCqcDEOJAYWOW+coTAwwOen8+y8z0lpUdFzp+YpL8s=; b=wBz8nJBlP7WJmUDyY6ZQg9OZ98m9I4kVQ0/iNjFksCFvpkX9VUkTR8H42Wa5fTvl7W yHKyYHjRj6QY6Cys95+oEbJISGFIyfG08FglJWgqPn8xar8ex2ofCLvHuofME6f1wx8x iGMFzwIYhWk1J7RUmbOdfjVAbJFRbQGaVFSuk+Qf5uyligglapJ/oaLAWuXGWSJkgFCO IrgyU2i3uZTnSpAL2hzNTPYUDextfTsSRKUR6pOy4K4Z9M7FXvNc95d60ESNaaQPDVtZ xOpMrzhsUGD4WUj3sAMho5xj3oHzxsKmwkryhgCGWzvpDdm5sjIWua0RAbShswPP5lwv jdvg== X-Google-Smtp-Source: ANB0VdZY5Q5ghsB6/FAuaqsSNTcnbdJLNhGWHJJm7IxUICy0dkLDbKXcJ9+2ZB2n1DPb4B3LEWku/g== X-Received: by 2002:a1c:7fc6:: with SMTP id a189-v6mr4102995wmd.42.1535542557270; Wed, 29 Aug 2018 04:35:57 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id s10-v6sm7800454wmd.22.2018.08.29.04.35.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Aug 2018 04:35:56 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v6 18/18] kasan: add SPDX-License-Identifier mark to source files Date: Wed, 29 Aug 2018 13:35:22 +0200 Message-Id: <878e719ff82dd0e43d9ede66856f2f76d072c417.1535462971.git.andreyknvl@google.com> X-Mailer: git-send-email 2.19.0.rc0.228.g281dcd1b4d0-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 1 + mm/kasan/kasan.c | 1 + mm/kasan/kasan_init.c | 1 + mm/kasan/kasan_report.c | 1 + mm/kasan/khwasan.c | 1 + mm/kasan/khwasan_report.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + 8 files changed, 8 insertions(+) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index e5648f4218eb..f2576d93e74c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common KASAN and KHWASAN code. * diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 44ec228de0a2..128a865c9e05 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core KASAN code. * diff --git a/mm/kasan/kasan_init.c b/mm/kasan/kasan_init.c index 7a2a2f13f86f..b3c068ab2a85 100644 --- a/mm/kasan/kasan_init.c +++ b/mm/kasan/kasan_init.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. * diff --git a/mm/kasan/kasan_report.c b/mm/kasan/kasan_report.c index fdf2d77e3125..48da73f4ef7c 100644 --- a/mm/kasan/kasan_report.c +++ b/mm/kasan/kasan_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains KASAN specific error reporting code. * diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c index 6b1309278e39..934f80b2d22e 100644 --- a/mm/kasan/khwasan.c +++ b/mm/kasan/khwasan.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core KHWASAN code. * diff --git a/mm/kasan/khwasan_report.c b/mm/kasan/khwasan_report.c index 51238b404b08..4e193546d94e 100644 --- a/mm/kasan/khwasan_report.c +++ b/mm/kasan/khwasan_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains KHWASAN specific error reporting code. * diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 3a8ddf8baf7d..0e4dc1a22615 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. * diff --git a/mm/kasan/report.c b/mm/kasan/report.c index e031c78f2e52..633b4b245798 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common KASAN and KHWASAN error reporting code. *