From patchwork Thu Aug 9 19:20:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9004A1390 for ; Thu, 9 Aug 2018 19:21:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EC3C2B92A for ; Thu, 9 Aug 2018 19:21:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C38A2B93D; Thu, 9 Aug 2018 19:21:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD5732B92A for ; Thu, 9 Aug 2018 19:21:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 569876B0003; Thu, 9 Aug 2018 15:21:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 518E36B0007; Thu, 9 Aug 2018 15:21:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 391CE6B0008; Thu, 9 Aug 2018 15:21:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id CE9156B0003 for ; Thu, 9 Aug 2018 15:21:18 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id r1-v6so5094552wrp.11 for ; Thu, 09 Aug 2018 12:21:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+9lusn/uftl3FAZ6ZyFaXTT1KH6bEEBFmRc1hQR6FkE=; b=TlZNYgG79y5EdP9rz+I/+QyAZ/bkxUhdkSBrwoyx/8lTovJzADgOTS+p4RivWjDfPa LQ6FRAnT4DqplBy/auwXANEmz1Xh2QarbLPMPEE0OceC9DSW5oA8k27M5Sh7+s9558cR Xv5kOw0ORbvD/GgzNuNIT72UH9OCKYfxwPp8oVwU3O1LawXB5UHbNALKPAd5GUnjAGAk JeWR1DvhMjPmSCkeuQRQuDYXDcnrIY56v1O2wdUVGc/CJofA/j/1uCKoimxTFAn8gA+T aZcFnyyJytudPGe9a0YGCzvtraATNESGR37b0QoJ2MUsDWqx9sSghLV7EEQsEoK1oymp hhMw== X-Gm-Message-State: AOUpUlECCtWKamsn8jrluva2W6gaXidBiZi0B55Wgco6ulDiuvJkcvNz 1JPimG27aj5bjQmLbJxH0zmtUmlrKvKvlyRzldGgBaQ0JEx1A0vVMZL1KSeObEkkwjTZ5AGoXuD t68jdpkGM13zhWCl9pfEP/hG/O3On/cMAGP0VMbqAJw0WWnFqXy2cbiFkenCat/7LZVCzCckSjD azplXuPqH2EB/2pAKZBtaV+vDwyhol6t1yLhxrW6mlvK5+EARaqgiX+49T/kEVr7Ya/cDspKUms E0U7tfb9vdPNRwRW1BP46ksWNsaPDS5rJFCHJTUGKMTi344n746FWm+UdC1Xb5TL6ETJlwm1pfU 52PP8HN33d/ZdrfZlIlNLJn7PE/X4Qf4wtA5PNPfXU5KCYB3MqGDX/smJQ22Oxxp06HaopP8Dk6 G X-Received: by 2002:a1c:b6d6:: with SMTP id g205-v6mr2628420wmf.17.1533842478222; Thu, 09 Aug 2018 12:21:18 -0700 (PDT) X-Received: by 2002:a1c:b6d6:: with SMTP id g205-v6mr2628392wmf.17.1533842477066; Thu, 09 Aug 2018 12:21:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842477; cv=none; d=google.com; s=arc-20160816; b=A9pZrISEBO11tbAJMu1e96l9+T4jtRYxewTAQN8YtJz+ZYGNMbjqOElvLRVo5+DsV9 Tb8x8fTPZ8FogSA9KGZiSmOk6mcYJo6RO+Okt78UcXiymwLqAcBhXpYCpaO7wNuwngFP k48pLBVbIlXTX7HMOMsaM9xf2pWN5Mqdd7oAazT4JPSjK2CuiAGrFBidHicgQN7eY22d 27crWMug9mdQfr4BLC4Cq0xv4Uy6HbxgKhQ/wWKrmWWCQStP4UQiIix+xK8JiLBivCmT AD6+VlTpXdrRcDNaPPiuA34CNra41/7unXaO8xM1sb9e4mqrcTnO0EplyCQkZjyw5HV0 zM6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=+9lusn/uftl3FAZ6ZyFaXTT1KH6bEEBFmRc1hQR6FkE=; b=XlgX7Vd9n2U1K04EQWAijE9LkVBL775INsFhVkdMwkryJj+FWw15nh9lV1eAY+L3cV +c1NOsW4VIKpzwuwiCMCc0889qYP7Sv1bd2OeqctUZ57N+SRLmFaD/qe2gvPmY4eQAjN TKX9iSZztDQRTEB5GvUK62LOtbrn+THoYnGpa56hCYMooZt9Jm0NiSH6s3MS3lWjICDO 3PkqIWsNLoZ7eOjYVxfCg57RnzQcyR0vANIw4dMBKRS/4TKB7qDD0VDRqyeQSpgVliCN IrLM1xmTe1jsVSynXqDGkaVzw1dCMx90nvTsuH3spb2WD6G09lO6dYC1bc3xkVYG02OP L4Bg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BPNZ1wes; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 74-v6sor2171040wml.30.2018.08.09.12.21.16 for (Google Transport Security); Thu, 09 Aug 2018 12:21:17 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BPNZ1wes; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+9lusn/uftl3FAZ6ZyFaXTT1KH6bEEBFmRc1hQR6FkE=; b=BPNZ1wesv1SQWrM7ZXd26Nb/a887DVg1Q4uAMrz6ZZ43xJBa/U2mLTMaeWt2YS49ox JXq+eNlQ/TDGSxYskHve/rM3PunMOAKRq+VNJfWAXkzUedMPtlNxKeqoAFAPBXbBfSiI idPC4AkwFfKGAJVUAijhbOTFzj+1iIW/tuY2XkMaD/0t16JjGR7dSyOH3SZtWFufb/o8 KVIVpZHntMFtz/hsFcwMW1K7t2cwbkQfndYDF66Ciu39L0eXvcm8XFlduxOQVIhm0UXi HsR8BH9qAd7gc8mDOXLak+hX9S9UDDzo2PUBiKaRGjNNyFBo9BpZawjLGDhcG875eXeB e7JQ== X-Google-Smtp-Source: AA+uWPzfScz5wbXrQM7e19ntyOUB7vmfS0pu1nKtGl9yJmPUZzRTRbtO5WyN97uQVZ9z1fLx27zNog== X-Received: by 2002:a1c:6354:: with SMTP id x81-v6mr2227952wmb.23.1533842476252; Thu, 09 Aug 2018 12:21:16 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:15 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 01/18] khwasan, mm: change kasan hooks signatures Date: Thu, 9 Aug 2018 21:20:53 +0200 Message-Id: <327ede8363658620073d173c615a26ba9faf38d0.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN will change the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 34 +++++++++++++++++++++++----------- mm/kasan/kasan.c | 24 ++++++++++++++---------- mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 6 files changed, 53 insertions(+), 38 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index de784fd11d12..cbdc54543803 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -53,14 +53,14 @@ void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -105,16 +105,28 @@ static inline void kasan_poison_object_data(struct kmem_cache *cache, static inline void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) {} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..f696c7c143c2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -485,9 +485,9 @@ void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) __memset(alloc_info, 0, sizeof(*alloc_info)); } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +528,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +538,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +551,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +566,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +576,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index aa76a70e087e..6fdca9ec2ea4 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3679,7 +3679,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3715,7 +3715,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 68bdf498da3b..15ef6a0d9c16 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index 2296caf87bfb..a99a1feb52c4 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1183,7 +1183,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1461,7 +1461,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index 51258eff4178..382bc0fea498 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1336,10 +1336,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2732,7 +2732,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2760,7 +2760,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3750,7 +3750,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3767,8 +3767,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3795,7 +3794,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } From patchwork Thu Aug 9 19:20:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561709 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 70B2E1057 for ; Thu, 9 Aug 2018 19:21:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CC982B913 for ; Thu, 9 Aug 2018 19:21:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57E692B922; Thu, 9 Aug 2018 19:21:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E24B2B93B for ; Thu, 9 Aug 2018 19:21:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D92A6B0008; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7676D6B000A; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F67A6B000E; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id A77D66B000A for ; Thu, 9 Aug 2018 15:21:22 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id d10-v6so5193540wrw.6 for ; Thu, 09 Aug 2018 12:21:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Rmmovj74tZp39oBjbYNlRr0eMJcHft929BQo0kMga78=; b=VFCmjEU2CjvhomunF2gDbsHK659YO2ainWr53i0hfgDQDNVajiYokuzlo5Tr/CS+cJ SsR4Bxyg+Pk8JbNqSJV0DF6s9kl+LK1E+3se2FdxOg2FUx0wBjH+vWh99IGgUpOY9Eui F4Cix35am54VBZaC5HfLI6aJbnM/2y/vYjNjtuStCPcFchYL2ljrMTHapfA3Qeh2pblH 1s6dMG/9alXSGaBTSEyC5cuUvHxCj69xS5e2Yqjq+5XFWG2ykfD3J93RscL/iRQIIf5f ByrjLcZEgNHqdxdp2QPPiyqe8UDLzeo2qqftDR4tx743rUYpt8oHhNsl/AOP+lk+vLDF /auA== X-Gm-Message-State: AOUpUlEbBieGXlNRwF+AlppcrA46Kc5WUSYcptXGXUwNBTFX6PB682G1 Y8dssjQhMrlMQuoF7Z6x6rrt5/w+7YepZJfKUL1HG8lwlqSOCk3YpfdOlbSqDksrF6oBTAUy1um E6EMs7+CrqmcfvH4Pfh/+6i4hrX9KHVFOTqvlEuGGgyimAjxZOukDxHs1W2U/fc5nMzJcJ0V56V kR54mpdD8SINQuU2Nyd63GSqHDA19IJ0b5dPbB9grepqxC/Vyc1MEzG1oydBn4FQzDIHfwabYBo KeiZUMm6g4JG/JlHGqdAx4nrUyaiGfv7MLAwRe94y+A+earRlxRE/ocXTXRI9xX8TiKycnGfF5L 2nk/0xqoeaAKtfQRY6ZVF1VOsXSbOJN6IGVjw59L6wh4N0gGEYxTgYSOM6tJ0OJV0QL1Yldag9q F X-Received: by 2002:adf:81c3:: with SMTP id 61-v6mr2296256wra.120.1533842482123; Thu, 09 Aug 2018 12:21:22 -0700 (PDT) X-Received: by 2002:adf:81c3:: with SMTP id 61-v6mr2296180wra.120.1533842479632; Thu, 09 Aug 2018 12:21:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842479; cv=none; d=google.com; s=arc-20160816; b=ulxLhrv1099EWBxIbu1Ky4JgLYX0IRvQyCA0qxOwab34mBwVLmrhNoQPdh7CrN4awd +8DzXXsSOlh4OmgDWDeOiUtP1ljFhBKGTqE8J1IvPRX4435pxDdQhv6IjZq8Luj1GzSk k0TkwONXW4qoLPKrdoFZM0ROEgGaF1QCDa36C2YBWntp5lLzQ/cnyVUml3MEn6ehHyzK A+IYkVnCaWk8pe9oHJtGB/nXAxt9S9ejJjdGsktRovZ4y2oa9dZdqyYVNZ/4MLenav4v ksOxl+SurY0EDQI2ue9er90zM473gKDQwo8I9JVlk/7LxR1y9kKVd5hv8KXzbhl4YYtE 6JXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=Rmmovj74tZp39oBjbYNlRr0eMJcHft929BQo0kMga78=; b=OuA0XWcnZecpqIPsr3k7PyzjO5/jP7M2TbolbEjjluRImai3o1mYlzs/JV4vOARm74 xbbnc2nqKedrnUBgbkcFPaAljFNfMOtOzulba/FYeHL3XaxCPl5PSuE1/z+Y5c3PhGeg vaPu8zXqD24uAN+GDfgFwPVUL6sdW9gqtrB0Pp75M/jBbLCC+fNaNeSJxfj5/9PJYJ8v q9WoAaKeYJqppLpycQd2r3sA/SgH08Oyv9DcBb8S0hYC20foLveZZGsozE1hC2qeUSxy NPU6CctwTwdBf4OT+UgKrwWDHPqL58bdWnFyCxG3y67aXzY7DcR4m/ofhbKl27wPMsnA CTIw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EocrGrb+; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a21-v6sor1914021wmg.77.2018.08.09.12.21.19 for (Google Transport Security); Thu, 09 Aug 2018 12:21:19 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EocrGrb+; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rmmovj74tZp39oBjbYNlRr0eMJcHft929BQo0kMga78=; b=EocrGrb+VO62hSJ1A2/mA2nZJAx8W5xDPuPrJAdtHEorfymIbnyi5fpAYmoB/01sMf nfXkv48MhSP4Jfu+IoDndqJt/n7BK8KLyhPOp9A8S3HUiox+7sjSq/VJQOQCaowu0+tU u1bGH0mzlUj3MFnWnmsayrm1BnD/oWC+bAq23cpC1+tlSxFIHTQNQVL0UQlBtK1ODA7n MLdDiXVeGBkj47f1ka1WyuZAyFWZGdXsTUS35IfQ636VrXYonZ+i1gP2a0ERDOs1QaGy GiXYGVc0Ef9O/ABXiJn549LN4rE6nHULDUfJjDrba2jQbhlQb2IzzS1w4gt4vmB4Ge09 WjzQ== X-Google-Smtp-Source: AA+uWPwmE4C5TBeUfquXLHtlVl0CmZMXZEyDjX4Accu6rtRgoy3WJR1/VMjwMAsm+SYRmBv5/EPlqA== X-Received: by 2002:a1c:f001:: with SMTP id a1-v6mr117267wmb.160.1533842478142; Thu, 09 Aug 2018 12:21:18 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:17 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 02/18] khwasan: move common kasan and khwasan code to common.c Date: Thu, 9 Aug 2018 21:20:54 +0200 Message-Id: <1551ebaee0f47dd41d28bfe68e726a440d1f4d34.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN will reuse a significant part of KASAN code, so move the common parts to common.c without any functional changes. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 604 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 568 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 615 insertions(+), 567 deletions(-) create mode 100644 mm/kasan/common.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 3289db38bc87..a6df14bffb6b 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o diff --git a/mm/kasan/common.c b/mm/kasan/common.c new file mode 100644 index 000000000000..154010ba6c1f --- /dev/null +++ b/mm/kasan/common.c @@ -0,0 +1,604 @@ +/* + * This file contains common KASAN and KHWASAN code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +static inline int in_irqentry_text(unsigned long ptr) +{ + return (ptr >= (unsigned long)&__irqentry_text_start && + ptr < (unsigned long)&__irqentry_text_end) || + (ptr >= (unsigned long)&__softirqentry_text_start && + ptr < (unsigned long)&__softirqentry_text_end); +} + +static inline void filter_irq_stacks(struct stack_trace *trace) +{ + int i; + + if (!trace->nr_entries) + return; + for (i = 0; i < trace->nr_entries; i++) + if (in_irqentry_text(trace->entries[i])) { + /* Include the irqentry function into the stack. */ + trace->nr_entries = i + 1; + break; + } +} + +static inline depot_stack_handle_t save_stack(gfp_t flags) +{ + unsigned long entries[KASAN_STACK_DEPTH]; + struct stack_trace trace = { + .nr_entries = 0, + .entries = entries, + .max_entries = KASAN_STACK_DEPTH, + .skip = 0 + }; + + save_stack_trace(&trace); + filter_irq_stacks(&trace); + if (trace.nr_entries != 0 && + trace.entries[trace.nr_entries-1] == ULONG_MAX) + trace.nr_entries--; + + return depot_save_stack(&trace, flags); +} + +static inline void set_track(struct kasan_track *track, gfp_t flags) +{ + track->pid = current->pid; + track->stack = save_stack(flags); +} + +void kasan_enable_current(void) +{ + current->kasan_depth++; +} + +void kasan_disable_current(void) +{ + current->kasan_depth--; +} + +void kasan_check_read(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, false, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_read); + +void kasan_check_write(const volatile void *p, unsigned int size) +{ + check_memory_region((unsigned long)p, size, true, _RET_IP_); +} +EXPORT_SYMBOL(kasan_check_write); + +#undef memset +void *memset(void *addr, int c, size_t len) +{ + check_memory_region((unsigned long)addr, len, true, _RET_IP_); + + return __memset(addr, c, len); +} + +#undef memmove +void *memmove(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memmove(dest, src, len); +} + +#undef memcpy +void *memcpy(void *dest, const void *src, size_t len) +{ + check_memory_region((unsigned long)src, len, false, _RET_IP_); + check_memory_region((unsigned long)dest, len, true, _RET_IP_); + + return __memcpy(dest, src, len); +} + +/* + * Poisons the shadow memory for 'size' bytes starting from 'addr'. + * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. + */ +void kasan_poison_shadow(const void *address, size_t size, u8 value) +{ + void *shadow_start, *shadow_end; + + shadow_start = kasan_mem_to_shadow(address); + shadow_end = kasan_mem_to_shadow(address + size); + + __memset(shadow_start, value, shadow_end - shadow_start); +} + +void kasan_unpoison_shadow(const void *address, size_t size) +{ + kasan_poison_shadow(address, size, 0); + + if (size & KASAN_SHADOW_MASK) { + u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); + *shadow = size & KASAN_SHADOW_MASK; + } +} + +static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) +{ + void *base = task_stack_page(task); + size_t size = sp - base; + + kasan_unpoison_shadow(base, size); +} + +/* Unpoison the entire stack for a task. */ +void kasan_unpoison_task_stack(struct task_struct *task) +{ + __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); +} + +/* Unpoison the stack for the current task beyond a watermark sp value. */ +asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) +{ + /* + * Calculate the task stack base address. Avoid using 'current' + * because this function is called by early resume code which hasn't + * yet set up the percpu register (%gs). + */ + void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); + + kasan_unpoison_shadow(base, watermark - base); +} + +/* + * Clear all poison for the region between the current SP and a provided + * watermark value, as is sometimes required prior to hand-crafted asm function + * returns in the middle of functions. + */ +void kasan_unpoison_stack_above_sp_to(const void *watermark) +{ + const void *sp = __builtin_frame_address(0); + size_t size = watermark - sp; + + if (WARN_ON(sp > watermark)) + return; + kasan_unpoison_shadow(sp, size); +} + +void kasan_alloc_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); +} + +void kasan_free_pages(struct page *page, unsigned int order) +{ + if (likely(!PageHighMem(page))) + kasan_poison_shadow(page_address(page), + PAGE_SIZE << order, + KASAN_FREE_PAGE); +} + +/* + * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. + * For larger allocations larger redzones are used. + */ +static inline unsigned int optimal_redzone(unsigned int object_size) +{ + if (IS_ENABLED(CONFIG_KASAN_HW)) + return 0; + + return + object_size <= 64 - 16 ? 16 : + object_size <= 128 - 32 ? 32 : + object_size <= 512 - 64 ? 64 : + object_size <= 4096 - 128 ? 128 : + object_size <= (1 << 14) - 256 ? 256 : + object_size <= (1 << 15) - 512 ? 512 : + object_size <= (1 << 16) - 1024 ? 1024 : 2048; +} + +void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) +{ + unsigned int orig_size = *size; + int redzone_adjust; + + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } + redzone_adjust = optimal_redzone(cache->object_size) - + (*size - cache->object_size); + + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + + optimal_redzone(cache->object_size))); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } + + *flags |= SLAB_KASAN; +} + +size_t kasan_metadata_size(struct kmem_cache *cache) +{ + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); +} + +struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); + return (void *)object + cache->kasan_info.alloc_meta_offset; +} + +struct kasan_free_meta *get_free_info(struct kmem_cache *cache, + const void *object) +{ + BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + return (void *)object + cache->kasan_info.free_meta_offset; +} + +void kasan_poison_slab(struct page *page) +{ + kasan_poison_shadow(page_address(page), + PAGE_SIZE << compound_order(page), + KASAN_KMALLOC_REDZONE); +} + +void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_unpoison_shadow(object, cache->object_size); +} + +void kasan_poison_object_data(struct kmem_cache *cache, void *object) +{ + kasan_poison_shadow(object, + round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), + KASAN_KMALLOC_REDZONE); +} + +void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +{ + struct kasan_alloc_meta *alloc_info; + + if (!(cache->flags & SLAB_KASAN)) + return; + + alloc_info = get_alloc_info(cache, object); + __memset(alloc_info, 0, sizeof(*alloc_info)); +} + +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +{ + return kasan_kmalloc(cache, object, cache->object_size, flags); +} + +static bool __kasan_slab_free(struct kmem_cache *cache, void *object, + unsigned long ip, bool quarantine) +{ + s8 shadow_byte; + unsigned long rounded_up_size; + + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != + object)) { + kasan_report_invalid_free(object, ip); + return true; + } + + /* RCU slabs could be legally used after free within the RCU period */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + + shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); + if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { + kasan_report_invalid_free(object, ip); + return true; + } + + rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); + + if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); + quarantine_put(get_free_info(cache, object), cache); + return true; +} + +bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +{ + return __kasan_slab_free(cache, object, ip, true); +} + +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, + gfp_t flags) +{ + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(object == NULL)) + return NULL; + + redzone_start = round_up((unsigned long)(object + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = round_up((unsigned long)object + cache->object_size, + KASAN_SHADOW_SCALE_SIZE); + + kasan_unpoison_shadow(object, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); + + if (cache->flags & SLAB_KASAN) + set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; +} +EXPORT_SYMBOL(kasan_kmalloc); + +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + struct page *page; + unsigned long redzone_start; + unsigned long redzone_end; + + if (gfpflags_allow_blocking(flags)) + quarantine_reduce(); + + if (unlikely(ptr == NULL)) + return NULL; + + page = virt_to_page(ptr); + redzone_start = round_up((unsigned long)(ptr + size), + KASAN_SHADOW_SCALE_SIZE); + redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); + + kasan_unpoison_shadow(ptr, size); + kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, + KASAN_PAGE_REDZONE); + + return (void *)ptr; +} + +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) +{ + struct page *page; + + if (unlikely(object == ZERO_SIZE_PTR)) + return (void *)object; + + page = virt_to_head_page(object); + + if (unlikely(!PageSlab(page))) + return kasan_kmalloc_large(object, size, flags); + else + return kasan_kmalloc(page->slab_cache, object, size, flags); +} + +void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), + KASAN_FREE_PAGE); + } else { + __kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + +void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (ptr != page_address(virt_to_head_page(ptr))) + kasan_report_invalid_free(ptr, ip); + /* The object will be poisoned by page_alloc. */ +} + +int kasan_module_alloc(void *addr, size_t size) +{ + void *ret; + size_t scaled_size; + size_t shadow_size; + unsigned long shadow_start; + + shadow_start = (unsigned long)kasan_mem_to_shadow(addr); + scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; + shadow_size = round_up(scaled_size, PAGE_SIZE); + + if (WARN_ON(!PAGE_ALIGNED(shadow_start))) + return -EINVAL; + + ret = __vmalloc_node_range(shadow_size, 1, shadow_start, + shadow_start + shadow_size, + GFP_KERNEL | __GFP_ZERO, + PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, + __builtin_return_address(0)); + + if (ret) { + find_vm_area(addr)->flags |= VM_KASAN; + kmemleak_ignore(ret); + return 0; + } + + return -ENOMEM; +} + +void kasan_free_shadow(const struct vm_struct *vm) +{ + if (vm->flags & VM_KASAN) + vfree(kasan_mem_to_shadow(vm->addr)); +} + +#ifdef CONFIG_MEMORY_HOTPLUG +static bool shadow_mapped(unsigned long addr) +{ + pgd_t *pgd = pgd_offset_k(addr); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + if (pgd_none(*pgd)) + return false; + p4d = p4d_offset(pgd, addr); + if (p4d_none(*p4d)) + return false; + pud = pud_offset(p4d, addr); + if (pud_none(*pud)) + return false; + + /* + * We can't use pud_large() or pud_huge(), the first one is + * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse + * pud_bad(), if pud is bad then it's bad because it's huge. + */ + if (pud_bad(*pud)) + return true; + pmd = pmd_offset(pud, addr); + if (pmd_none(*pmd)) + return false; + + if (pmd_bad(*pmd)) + return true; + pte = pte_offset_kernel(pmd, addr); + return !pte_none(*pte); +} + +static int __meminit kasan_mem_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct memory_notify *mem_data = data; + unsigned long nr_shadow_pages, start_kaddr, shadow_start; + unsigned long shadow_end, shadow_size; + + nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; + start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); + shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); + shadow_size = nr_shadow_pages << PAGE_SHIFT; + shadow_end = shadow_start + shadow_size; + + if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || + WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) + return NOTIFY_BAD; + + switch (action) { + case MEM_GOING_ONLINE: { + void *ret; + + /* + * If shadow is mapped already than it must have been mapped + * during the boot. This could happen if we onlining previously + * offlined memory. + */ + if (shadow_mapped(shadow_start)) + return NOTIFY_OK; + + ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, + shadow_end, GFP_KERNEL, + PAGE_KERNEL, VM_NO_GUARD, + pfn_to_nid(mem_data->start_pfn), + __builtin_return_address(0)); + if (!ret) + return NOTIFY_BAD; + + kmemleak_ignore(ret); + return NOTIFY_OK; + } + case MEM_CANCEL_ONLINE: + case MEM_OFFLINE: { + struct vm_struct *vm; + + /* + * shadow_start was either mapped during boot by kasan_init() + * or during memory online by __vmalloc_node_range(). + * In the latter case we can use vfree() to free shadow. + * Non-NULL result of the find_vm_area() will tell us if + * that was the second case. + * + * Currently it's not possible to free shadow mapped + * during boot by kasan_init(). It's because the code + * to do that hasn't been written yet. So we'll just + * leak the memory. + */ + vm = find_vm_area((void *)shadow_start); + if (vm) + vfree((void *)shadow_start); + } + } + + return NOTIFY_OK; +} + +static int __init kasan_memhotplug_init(void) +{ + hotplug_memory_notifier(kasan_mem_notifier, 0); + + return 0; +} + +core_initcall(kasan_memhotplug_init); +#endif diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index f696c7c143c2..44ec228de0a2 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,5 +1,5 @@ /* - * This file contains shadow memory manipulation code. + * This file contains core KASAN code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -40,82 +40,6 @@ #include "kasan.h" #include "../slab.h" -void kasan_enable_current(void) -{ - current->kasan_depth++; -} - -void kasan_disable_current(void) -{ - current->kasan_depth--; -} - -/* - * Poisons the shadow memory for 'size' bytes starting from 'addr'. - * Memory addresses should be aligned to KASAN_SHADOW_SCALE_SIZE. - */ -static void kasan_poison_shadow(const void *address, size_t size, u8 value) -{ - void *shadow_start, *shadow_end; - - shadow_start = kasan_mem_to_shadow(address); - shadow_end = kasan_mem_to_shadow(address + size); - - memset(shadow_start, value, shadow_end - shadow_start); -} - -void kasan_unpoison_shadow(const void *address, size_t size) -{ - kasan_poison_shadow(address, size, 0); - - if (size & KASAN_SHADOW_MASK) { - u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; - } -} - -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - kasan_unpoison_shadow(base, size); -} - -/* Unpoison the entire stack for a task. */ -void kasan_unpoison_task_stack(struct task_struct *task) -{ - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); -} - -/* Unpoison the stack for the current task beyond a watermark sp value. */ -asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) -{ - /* - * Calculate the task stack base address. Avoid using 'current' - * because this function is called by early resume code which hasn't - * yet set up the percpu register (%gs). - */ - void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - - kasan_unpoison_shadow(base, watermark - base); -} - -/* - * Clear all poison for the region between the current SP and a provided - * watermark value, as is sometimes required prior to hand-crafted asm function - * returns in the middle of functions. - */ -void kasan_unpoison_stack_above_sp_to(const void *watermark) -{ - const void *sp = __builtin_frame_address(0); - size_t size = watermark - sp; - - if (WARN_ON(sp > watermark)) - return; - kasan_unpoison_shadow(sp, size); -} - /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -260,121 +184,12 @@ static __always_inline void check_memory_region_inline(unsigned long addr, kasan_report(addr, size, write, ret_ip); } -static void check_memory_region(unsigned long addr, - size_t size, bool write, +void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { check_memory_region_inline(addr, size, write, ret_ip); } -void kasan_check_read(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, false, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_read); - -void kasan_check_write(const volatile void *p, unsigned int size) -{ - check_memory_region((unsigned long)p, size, true, _RET_IP_); -} -EXPORT_SYMBOL(kasan_check_write); - -#undef memset -void *memset(void *addr, int c, size_t len) -{ - check_memory_region((unsigned long)addr, len, true, _RET_IP_); - - return __memset(addr, c, len); -} - -#undef memmove -void *memmove(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memmove(dest, src, len); -} - -#undef memcpy -void *memcpy(void *dest, const void *src, size_t len) -{ - check_memory_region((unsigned long)src, len, false, _RET_IP_); - check_memory_region((unsigned long)dest, len, true, _RET_IP_); - - return __memcpy(dest, src, len); -} - -void kasan_alloc_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); -} - -void kasan_free_pages(struct page *page, unsigned int order) -{ - if (likely(!PageHighMem(page))) - kasan_poison_shadow(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); -} - -/* - * Adaptive redzone policy taken from the userspace AddressSanitizer runtime. - * For larger allocations larger redzones are used. - */ -static unsigned int optimal_redzone(unsigned int object_size) -{ - return - object_size <= 64 - 16 ? 16 : - object_size <= 128 - 32 ? 32 : - object_size <= 512 - 64 ? 64 : - object_size <= 4096 - 128 ? 128 : - object_size <= (1 << 14) - 256 ? 256 : - object_size <= (1 << 15) - 512 ? 512 : - object_size <= (1 << 16) - 1024 ? 1024 : 2048; -} - -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) -{ - unsigned int orig_size = *size; - int redzone_adjust; - - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); - - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); - - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; - } - - *flags |= SLAB_KASAN; -} - void kasan_cache_shrink(struct kmem_cache *cache) { quarantine_remove_cache(cache); @@ -386,275 +201,6 @@ void kasan_cache_shutdown(struct kmem_cache *cache) quarantine_remove_cache(cache); } -size_t kasan_metadata_size(struct kmem_cache *cache) -{ - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); -} - -void kasan_poison_slab(struct page *page) -{ - kasan_poison_shadow(page_address(page), - PAGE_SIZE << compound_order(page), - KASAN_KMALLOC_REDZONE); -} - -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_unpoison_shadow(object, cache->object_size); -} - -void kasan_poison_object_data(struct kmem_cache *cache, void *object) -{ - kasan_poison_shadow(object, - round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE), - KASAN_KMALLOC_REDZONE); -} - -static inline int in_irqentry_text(unsigned long ptr) -{ - return (ptr >= (unsigned long)&__irqentry_text_start && - ptr < (unsigned long)&__irqentry_text_end) || - (ptr >= (unsigned long)&__softirqentry_text_start && - ptr < (unsigned long)&__softirqentry_text_end); -} - -static inline void filter_irq_stacks(struct stack_trace *trace) -{ - int i; - - if (!trace->nr_entries) - return; - for (i = 0; i < trace->nr_entries; i++) - if (in_irqentry_text(trace->entries[i])) { - /* Include the irqentry function into the stack. */ - trace->nr_entries = i + 1; - break; - } -} - -static inline depot_stack_handle_t save_stack(gfp_t flags) -{ - unsigned long entries[KASAN_STACK_DEPTH]; - struct stack_trace trace = { - .nr_entries = 0, - .entries = entries, - .max_entries = KASAN_STACK_DEPTH, - .skip = 0 - }; - - save_stack_trace(&trace); - filter_irq_stacks(&trace); - if (trace.nr_entries != 0 && - trace.entries[trace.nr_entries-1] == ULONG_MAX) - trace.nr_entries--; - - return depot_save_stack(&trace, flags); -} - -static inline void set_track(struct kasan_track *track, gfp_t flags) -{ - track->pid = current->pid; - track->stack = save_stack(flags); -} - -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_alloc_meta) > 32); - return (void *)object + cache->kasan_info.alloc_meta_offset; -} - -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) -{ - BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)object + cache->kasan_info.free_meta_offset; -} - -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) -{ - struct kasan_alloc_meta *alloc_info; - - if (!(cache->flags & SLAB_KASAN)) - return; - - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); -} - -void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) -{ - return kasan_kmalloc(cache, object, cache->object_size, flags); -} - -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, - unsigned long ip, bool quarantine) -{ - s8 shadow_byte; - unsigned long rounded_up_size; - - if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != - object)) { - kasan_report_invalid_free(object, ip); - return true; - } - - /* RCU slabs could be legally used after free within the RCU period */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - - shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); - return true; - } - - rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) - return false; - - set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); - quarantine_put(get_free_info(cache, object), cache); - return true; -} - -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) -{ - return __kasan_slab_free(cache, object, ip, true); -} - -void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, - gfp_t flags) -{ - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(object == NULL)) - return NULL; - - redzone_start = round_up((unsigned long)(object + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = round_up((unsigned long)object + cache->object_size, - KASAN_SHADOW_SCALE_SIZE); - - kasan_unpoison_shadow(object, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); - - if (cache->flags & SLAB_KASAN) - set_track(&get_alloc_info(cache, object)->alloc_track, flags); - - return (void *)object; -} -EXPORT_SYMBOL(kasan_kmalloc); - -void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) -{ - struct page *page; - unsigned long redzone_start; - unsigned long redzone_end; - - if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); - - if (unlikely(ptr == NULL)) - return NULL; - - page = virt_to_page(ptr); - redzone_start = round_up((unsigned long)(ptr + size), - KASAN_SHADOW_SCALE_SIZE); - redzone_end = (unsigned long)ptr + (PAGE_SIZE << compound_order(page)); - - kasan_unpoison_shadow(ptr, size); - kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, - KASAN_PAGE_REDZONE); - - return (void *)ptr; -} - -void *kasan_krealloc(const void *object, size_t size, gfp_t flags) -{ - struct page *page; - - if (unlikely(object == ZERO_SIZE_PTR)) - return ZERO_SIZE_PTR; - - page = virt_to_head_page(object); - - if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); - else - return kasan_kmalloc(page->slab_cache, object, size, flags); -} - -void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - kasan_poison_shadow(ptr, PAGE_SIZE << compound_order(page), - KASAN_FREE_PAGE); - } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - -void kasan_kfree_large(void *ptr, unsigned long ip) -{ - if (ptr != page_address(virt_to_head_page(ptr))) - kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ -} - -int kasan_module_alloc(void *addr, size_t size) -{ - void *ret; - size_t scaled_size; - size_t shadow_size; - unsigned long shadow_start; - - shadow_start = (unsigned long)kasan_mem_to_shadow(addr); - scaled_size = (size + KASAN_SHADOW_MASK) >> KASAN_SHADOW_SCALE_SHIFT; - shadow_size = round_up(scaled_size, PAGE_SIZE); - - if (WARN_ON(!PAGE_ALIGNED(shadow_start))) - return -EINVAL; - - ret = __vmalloc_node_range(shadow_size, 1, shadow_start, - shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, - PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, - __builtin_return_address(0)); - - if (ret) { - find_vm_area(addr)->flags |= VM_KASAN; - kmemleak_ignore(ret); - return 0; - } - - return -ENOMEM; -} - -void kasan_free_shadow(const struct vm_struct *vm) -{ - if (vm->flags & VM_KASAN) - vfree(kasan_mem_to_shadow(vm->addr)); -} - static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_SHADOW_SCALE_SIZE); @@ -795,113 +341,3 @@ DEFINE_ASAN_SET_SHADOW(f2); DEFINE_ASAN_SET_SHADOW(f3); DEFINE_ASAN_SET_SHADOW(f5); DEFINE_ASAN_SET_SHADOW(f8); - -#ifdef CONFIG_MEMORY_HOTPLUG -static bool shadow_mapped(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - if (pgd_none(*pgd)) - return false; - p4d = p4d_offset(pgd, addr); - if (p4d_none(*p4d)) - return false; - pud = pud_offset(p4d, addr); - if (pud_none(*pud)) - return false; - - /* - * We can't use pud_large() or pud_huge(), the first one is - * arch-specific, the last one depends on HUGETLB_PAGE. So let's abuse - * pud_bad(), if pud is bad then it's bad because it's huge. - */ - if (pud_bad(*pud)) - return true; - pmd = pmd_offset(pud, addr); - if (pmd_none(*pmd)) - return false; - - if (pmd_bad(*pmd)) - return true; - pte = pte_offset_kernel(pmd, addr); - return !pte_none(*pte); -} - -static int __meminit kasan_mem_notifier(struct notifier_block *nb, - unsigned long action, void *data) -{ - struct memory_notify *mem_data = data; - unsigned long nr_shadow_pages, start_kaddr, shadow_start; - unsigned long shadow_end, shadow_size; - - nr_shadow_pages = mem_data->nr_pages >> KASAN_SHADOW_SCALE_SHIFT; - start_kaddr = (unsigned long)pfn_to_kaddr(mem_data->start_pfn); - shadow_start = (unsigned long)kasan_mem_to_shadow((void *)start_kaddr); - shadow_size = nr_shadow_pages << PAGE_SHIFT; - shadow_end = shadow_start + shadow_size; - - if (WARN_ON(mem_data->nr_pages % KASAN_SHADOW_SCALE_SIZE) || - WARN_ON(start_kaddr % (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIFT))) - return NOTIFY_BAD; - - switch (action) { - case MEM_GOING_ONLINE: { - void *ret; - - /* - * If shadow is mapped already than it must have been mapped - * during the boot. This could happen if we onlining previously - * offlined memory. - */ - if (shadow_mapped(shadow_start)) - return NOTIFY_OK; - - ret = __vmalloc_node_range(shadow_size, PAGE_SIZE, shadow_start, - shadow_end, GFP_KERNEL, - PAGE_KERNEL, VM_NO_GUARD, - pfn_to_nid(mem_data->start_pfn), - __builtin_return_address(0)); - if (!ret) - return NOTIFY_BAD; - - kmemleak_ignore(ret); - return NOTIFY_OK; - } - case MEM_CANCEL_ONLINE: - case MEM_OFFLINE: { - struct vm_struct *vm; - - /* - * shadow_start was either mapped during boot by kasan_init() - * or during memory online by __vmalloc_node_range(). - * In the latter case we can use vfree() to free shadow. - * Non-NULL result of the find_vm_area() will tell us if - * that was the second case. - * - * Currently it's not possible to free shadow mapped - * during boot by kasan_init(). It's because the code - * to do that hasn't been written yet. So we'll just - * leak the memory. - */ - vm = find_vm_area((void *)shadow_start); - if (vm) - vfree((void *)shadow_start); - } - } - - return NOTIFY_OK; -} - -static int __init kasan_memhotplug_init(void) -{ - hotplug_memory_notifier(kasan_mem_notifier, 0); - - return 0; -} - -core_initcall(kasan_memhotplug_init); -#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index c12dcfde2ebd..659463800f10 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow(const void *address, size_t size, u8 value); + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); From patchwork Thu Aug 9 19:20:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561711 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 478371390 for ; Thu, 9 Aug 2018 19:21:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 330B92B93D for ; Thu, 9 Aug 2018 19:21:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2AED92B93C; Thu, 9 Aug 2018 19:21:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04E852B93D for ; Thu, 9 Aug 2018 19:21:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9EB606B000A; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 96E946B0266; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B75E6B000D; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id D681B6B0008 for ; Thu, 9 Aug 2018 15:21:22 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id y18-v6so677010wma.9 for ; Thu, 09 Aug 2018 12:21:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=o3yFZF1myIfhLYJ25RsQ37vNtyAe3Y7F14TNVHYVZ0E=; b=lEXdSQnDTM6zKRNSSrViHMohtWPog9Zuh7Jvhv87rvYWlejeoPjOh7vs94dLNE1X/Q i6OFuSLhRvgkbH69ae05pTfWLU/d7OTGrUo6uLeJx7zBtM5VRMgHdHEb84ZLEIJuKPGX F3+pzQPA9tF2Hzlfbe5FEkZLJCUnbC6Pkja13N68yT7JF/Eb3V5WvQgE0YCZ7xTLtMYa ApA0X8kB89Arcqmt57+k0DbCINEoIUPS58lCPB8o1FYRv8Ei5rCT8dYMuvHOM25U4Dkq wvZ0e+Bw6GjEpZGL/AUGZhajF4Kmztxo/x6+OD9Up3UZOwpEOsWVftKmGeObH77To6gq 2ZjQ== X-Gm-Message-State: AOUpUlG/egQmu8IBcoZBA7wSX/wbeX1A7T2ND7mCoj99/6i6PVipE2Q3 J8NBVvvAuifmmlyELtUsmZ6Irr3IjaLfc+b+DhOi7OWQym2XF8E2pgMAejlG9jiLJJu7ZXZuQhn vp5pV++utl5uxH+jY+a6M4O0UxgAav8njokKNvlBSLRX2Lpf+5JSD7H96RLGonFI45te6hCZ43P lJ6phiPCvamA0qAcnFALLkC9XkLv3IxhoSA8cJvAWvCtGk1R1yIvaUoQIVAzlJmqvsLmsp37Kax 7UgbZgmdTLf6VoPG+poOcXPD5UO1TRAdG+KywC3GQ1fVo5uL5l4VPx3Ws7BPrmQnCMylo5/BxE1 zHb243Xr2E40GilBsIJaO29Uukv+f3j4ZCAQAfFWENEH/CbkAdDDy0Dv1+7MXBZsIV3UhLlCwft B X-Received: by 2002:a1c:99c2:: with SMTP id b185-v6mr2250474wme.15.1533842482183; Thu, 09 Aug 2018 12:21:22 -0700 (PDT) X-Received: by 2002:a1c:99c2:: with SMTP id b185-v6mr2250439wme.15.1533842480638; Thu, 09 Aug 2018 12:21:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842480; cv=none; d=google.com; s=arc-20160816; b=Tf2zn+vys0z6mMahshZjXDUrsiKKCNtiB2iHRDU140+mSxAF+WZL0ka+Bvx6qlOcqE UGRvhDWkjg6Isaq7XCQ3oluVYfLPyEnJaUjTolQuOyj0+wJKueEB6Okv8sybrBjhE1BJ AzSsn8sAzVBV+CvGqjTRpTAyYRUrrFGWf+rTb/4aFJ41wTWPJNtVSPqgsAekXYjjrusL jk9JQDilkW21BR7RLn2s74NSzlq0Lud1rHWdkKrAGix4vnVPG613/5hUctimF728DUI1 wXEtFd5hA0pp6t+lNUx+hMwYXVmSIcEtj9Yh6/RzanKlkAOz1AMme+i1ZTPrOL/FECHQ oh2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=o3yFZF1myIfhLYJ25RsQ37vNtyAe3Y7F14TNVHYVZ0E=; b=lvFp83tcwXVZy0bKSEotQo6Oc21l9xi98sWLLnP4Ct60rJJ966Y4iXmTPBH2XFhYPn kRlpDMufdqPYyweVVGGNezM/zKUy+kRtjwe8a8oClcWSYpsa4ix3UxQewINiksgus4d0 tsqr5oyYSgrD9xr+4su3zWNiD1scKAFFzf7+0HuftbNXFEF0aTMcrBf5zp5xpp9iZlkr ppBH2DtGyOmEVIIycpJaWNVoNANqHkjqZWpIHDv+tJuT72wJs8Nzol8mCoAFfXN2hEQ6 LXpMJPH4wZNVJXjNzypVJH8IOD33ZFBOPHDPeW/DWo3FlPwNIq/HDIoSh3lMfvZXtt+V baWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RO0AwDkm; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n15-v6sor3167971wrh.43.2018.08.09.12.21.20 for (Google Transport Security); Thu, 09 Aug 2018 12:21:20 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=RO0AwDkm; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o3yFZF1myIfhLYJ25RsQ37vNtyAe3Y7F14TNVHYVZ0E=; b=RO0AwDkmKRvmWMBn+KW/dieUE85mqtdhWGTk3Y6+3lqmidqht8+EFDf0pbML/ewCKN +a15HA6biBvbnXj64KHmz5F1jGySqTW9uz/XOyaVoEugK8gmbjoJBTfPnKcK+67c3mWR BfttTsA5HQPWTRi+x0wCB6Rlf42sUdpibKLkltJotpqV2G1f3axSmO5DISOt1FmepW7W jbpQASTxgVsvHm/6ii6E41iBUxL6+tMF7van0hV0fW6QFZXK8xUj+/K6E/qT2pCtsFBa Ju4v7Z8z2803/kHSlDRpMFGv8vE/JjZtEh23/2F1v1lyLuvMSPBAFdp0alKAHyJysT2H dFbg== X-Google-Smtp-Source: AA+uWPzmFGemHq3d1tXlp2KgMG5dRF87Kf4dgwJ/M39Excsdr5Z+OAYtLhfWkX9wfaK607ZTcRZAaw== X-Received: by 2002:adf:e78d:: with SMTP id n13-v6mr2285472wrm.136.1533842479764; Thu, 09 Aug 2018 12:21:19 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:19 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 03/18] khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW Date: Thu, 9 Aug 2018 21:20:55 +0200 Message-Id: <7a96e2e1eab929304305126ebb4083aadfdd0d59.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit splits the current CONFIG_KASAN config option into two: 1. CONFIG_KASAN_GENERIC, that enables the generic software-only KASAN version (the one that exists now); 2. CONFIG_KASAN_HW, that enables KHWASAN. With CONFIG_KASAN_HW enabled, compiler options are changed to instrument kernel files wiht -fsantize=hwaddress (except the ones for which KASAN_SANITIZE := n is set). Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes. This commit also adds empty placeholder (for now) implementation of KHWASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options. Signed-off-by: Andrey Konovalov --- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 5 ++- include/linux/compiler-gcc.h | 4 ++ include/linux/compiler.h | 3 +- include/linux/kasan.h | 16 +++++-- lib/Kconfig.kasan | 77 ++++++++++++++++++++++++++-------- mm/kasan/Makefile | 6 ++- mm/kasan/kasan.h | 3 +- mm/kasan/khwasan.c | 75 +++++++++++++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 27 +++++++++++- 11 files changed, 190 insertions(+), 29 deletions(-) create mode 100644 mm/kasan/khwasan.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 42c090cf0292..43d9de526d3a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -92,6 +92,7 @@ config ARM64 select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_HW if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 7087446c24c8..6ac1761ff102 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -21,13 +21,16 @@ #define KASAN_ABI_VERSION 5 /* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature(address_sanitizer) +#if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) #define __SANITIZE_ADDRESS__ #endif #undef __no_sanitize_address #define __no_sanitize_address __attribute__((no_sanitize("address"))) +#undef __no_sanitize_hwaddress +#define __no_sanitize_hwaddress __attribute__((no_sanitize("hwaddress"))) + /* Clang doesn't have a way to turn it off per-function, yet. */ #ifdef __noretpoline #undef __noretpoline diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 573f5a7d42d4..b486f936a21c 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -353,6 +353,10 @@ #define __no_sanitize_address #endif +#if !defined(__no_sanitize_hwaddress) +#define __no_sanitize_hwaddress /* gcc doesn't support KHWASAN */ +#endif + /* * A trick to suppress uninitialized variable warning without generating any * code diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 42506e4d1f53..6439fdd46b4e 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -195,7 +195,8 @@ void __read_once_size(const volatile void *p, void *res, int size) * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 * '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused +# define __no_kasan_or_inline __no_sanitize_address __no_sanitize_hwaddress \ + __maybe_unused #else # define __no_kasan_or_inline __always_inline #endif diff --git a/include/linux/kasan.h b/include/linux/kasan.h index cbdc54543803..6608aa9b35ac 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order); void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags); -void kasan_cache_shrink(struct kmem_cache *cache); -void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); @@ -94,8 +92,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {} static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} -static inline void kasan_cache_shrink(struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, @@ -141,4 +137,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink(struct kmem_cache *cache); +void kasan_cache_shutdown(struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink(struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ + #endif /* LINUX_KASAN_H */ diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index befb127507c0..5a22629f30e7 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -1,34 +1,75 @@ config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_HW + bool + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs. + +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: KASAN (a classic version, similar to userspace + ASan, enabled with CONFIG_KASAN_GENERIC) and KHWASAN (a version + based on pointer tagging, only for arm64, similar to userspace + HWASan, enabled with CONFIG_KASAN_HW). + +config KASAN_GENERIC + bool "KASAN: the generic mode" depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables the generic mode of KASAN. + This is strictly a debugging feature and it requires a GCC version + of 4.9.2 or later. Detection of out-of-bounds accesses to stack or + global variables requires GCC 5.0 or later. + This mode consumes about 1/8 of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +if HAVE_ARCH_KASAN_HW + +config KASAN_HW + bool "KHWASAN: the hardware assisted mode" + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enabled KHWASAN (KASAN mode based on pointer tagging). + This mode requires Top Byte Ignore support by the CPU and therefore + only supported for arm64. + This feature requires clang revision 330044 or later. + This mode consumes about 1/16 of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_HW doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer - compile time. + This enables further checks in KASAN, for now it only includes the + address-use-after-scope check that can lead to excessive kernel + stack usage, frame size warnings and longer compile time. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more @@ -53,16 +94,16 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endif diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index a6df14bffb6b..14955add96d3 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_khwasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg @@ -10,5 +11,8 @@ CFLAGS_REMOVE_kasan.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.o diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 659463800f10..19b950eaccff 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); -#if defined(CONFIG_SLAB) || defined(CONFIG_SLUB) +#if defined(CONFIG_KASAN_GENERIC) && \ + (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c new file mode 100644 index 000000000000..e2c3a7f7fd1f --- /dev/null +++ b/mm/kasan/khwasan.c @@ -0,0 +1,75 @@ +/* + * This file contains core KHWASAN code. + * + * Copyright (c) 2018 Google, Inc. + * Author: Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE(size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE(1); +DEFINE_HWASAN_LOAD_STORE(2); +DEFINE_HWASAN_LOAD_STORE(4); +DEFINE_HWASAN_LOAD_STORE(8); +DEFINE_HWASAN_LOAD_STORE(16); + +void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_storeN_noabort); + +void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) +{ +} +EXPORT_SYMBOL(__hwasan_tag_memory); diff --git a/mm/slub.c b/mm/slub.c index 382bc0fea498..8fa21afcd3fb 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2956,7 +2956,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page, do_slab_free(s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free(struct kmem_cache *cache, void *x, unsigned long addr) { do_slab_free(cache, virt_to_head_page(x), x, NULL, 1, addr); diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 69552a39951d..49c6e056c697 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else @@ -42,6 +42,29 @@ ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif + +ifdef CONFIG_KASAN_HW + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +ifeq ($(call cc-option, $(CFLAGS_KASAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KASAN_HW: \ + -fsanitize=hwaddress is not supported by compiler) + endif +endif + +endif + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif From patchwork Thu Aug 9 19:20:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561715 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5BE51057 for ; Thu, 9 Aug 2018 19:21:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B62502B91F for ; Thu, 9 Aug 2018 19:21:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B390A2B949; Thu, 9 Aug 2018 19:21:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D5FD2B945 for ; Thu, 9 Aug 2018 19:21:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 212826B000D; Thu, 9 Aug 2018 15:21:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1BECA6B000E; Thu, 9 Aug 2018 15:21:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE1EA6B0010; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id 8622E6B000E for ; Thu, 9 Aug 2018 15:21:23 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id z11-v6so646931wma.4 for ; Thu, 09 Aug 2018 12:21:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=f8+p7PdvKfl3q/+F/IGkt+C18+eHZ9wVnzYkW+BehT8=; b=Hquq9FRJ8Du9WVm6egC6j2ghSUg++u6Ed3gryAI2iJzjJu4KhLq1W2BPXxF7gw6cNI E2EC6a9+TthdlJJXnP7IpMEBd5gQY+A+0t/3HQOVYRVTqpnUCjwnKZOJID4F+WpoSYqD XjUwmDPrklT1/eJ2syBYaNxRNEIXyIUOhSVmOhf6XR1tUBbxBBHqBPYrezYwji1h5zdi KJ+gLazRdFN6UUsLIXjb8CRqnOqy/+SHkFgalIFTPNNKMmB+OOxz/QyI8Tc8fLHkgpH5 K0VpPwv7yjQqEH3D6ADPz0VbGu4ZBn0Mc2Ami6ocZJLYJ7c8O6Z6fGAo7S/3AOzl7U1H DZUA== X-Gm-Message-State: AOUpUlE7OhlpFncMla3pnPwRvMbMsiziLolvWO8qAX2Foz324jBQmMD6 tTWEQsZEwreAWPv5WBVRnl99RrjumHJmKinfuHM60OTwymGJGJwgxjQYgcUB1nkjkmoamWtWGuK MjzDxVtfb8WucNJAiOQUBl6wItUBbddje7RX9GzZgqJaCfLKIyK2rHizipILVkq4CyRmimmIhN5 xmjoJfZeck0GAc0b8PuNgDD0ocyxycWITUBU8bAm97XBPqmvIN1oHtKWAgm+z0KIxd0HBL8VShK CFrg0Yc1YTO98yjXlS/5FECzva274Eugk0Byw60RlYaSOiqeop7hcf2x1B7GflY4HX72EA9gIH4 dJy2u2qSWFA5WxJGwHqsIw8Es2WTORcfadkr45ooCJfcKUwgw+5hqHVzF4UFEsGqHI2UnD4migi e X-Received: by 2002:adf:ef03:: with SMTP id e3-v6mr2130085wro.182.1533842483060; Thu, 09 Aug 2018 12:21:23 -0700 (PDT) X-Received: by 2002:adf:ef03:: with SMTP id e3-v6mr2130059wro.182.1533842482226; Thu, 09 Aug 2018 12:21:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842482; cv=none; d=google.com; s=arc-20160816; b=G39ianIxt9qDAJW/5OLk4HTxxC0t+237N6EvaPwRy2w3l2lMbTeu1LTPe5fQwjs/i6 M9yH4mPJkmf4kgnsZEXfKhlQ0r/LOF4tYA38luOK+Z8obCElR1TknviYdvwFCPUAsRSg 2LV2q4HNohneYB/8mOrrWStxK8S9hD8zvGCJ0Z9y2gVcK+FwS93kVUrNd3aB/kx2bMpE +791riJ7lP96hEJaCJmCARZKd/QhdQsfGl5x7+SrYfJnaHiUd3YXfaor18FM5Zj0MibM XGrCbClRa1mou9kGtjJc64mF4AVAdv9r86ehnk+ozaJ7JlWWTdFHj1Uh0jqT/Wt/dcAC cYiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=f8+p7PdvKfl3q/+F/IGkt+C18+eHZ9wVnzYkW+BehT8=; b=B8EMUH+Rleqrv6ApsBbhQ5odXJ7JD+hBH7WlAEXYaf2yuKrjW+nSgD/nNWyqYswwcm ag4TgzZyYAZqL3qXHPh5MPdyXRNV9KA1fyEtwjX5gD09RX4bdLvfBBXCRERgYwHeBdXS Rk/9PPXFn+YpMlUfzv+MiUq3YibIbnC08K0SrxyI+ETdB6NQrntBpLrfXV8qmADC0GYu zVZTGb+kBWkg28Lmt4shW7Q9aZIcXM/QfVdZGxzNRiVFa6u0aWSiYOLZuf8gP0ZHjr1G CqAM2TUKIgXjFScXPXhLgJf60goKslgqCfm8aiKFK53pdkOmOHk/eQ5Du5qd5PAQp0kF +20A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=D73aW0s5; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y188-v6sor1961781wmy.72.2018.08.09.12.21.22 for (Google Transport Security); Thu, 09 Aug 2018 12:21:22 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=D73aW0s5; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f8+p7PdvKfl3q/+F/IGkt+C18+eHZ9wVnzYkW+BehT8=; b=D73aW0s551OGsYp9bn6Vo8ec1xt2vcZEVojeeg/hcq+xlUvedvfiPvD/l23Qztvnw+ wSLucQGfP6ySKLhpmhcRTW8CbF74oP4WdINOeTcHbe8nwxxrmgezabuzCNMGmUKm6fUK SRj9TMDDJRTKOFBl1AmQZS70YMKa8CzKaO5su8SVksnZW2NJ35YRrsXLjntvy6CpThYw jhc++KV7XImPnbR+UvilReSMks8aI8LJhZ/Q5JCfe3j8kwpu5Qt/uNXsGox6/MqM9h8v iW/pdTxfHLAju6h9IZcYcNKAUs7ZpyPeqQ/LQ2kxHatkzBl2m2u48Bp8zbJoniIuPUlQ Hegw== X-Google-Smtp-Source: AA+uWPw3BJp/Y8iB0ddeTjQn74oZlVoIzUW/ZVOeQxFtaBKLU75DdXP/0uzrW82eVwx7QU/jEbWVJw== X-Received: by 2002:a1c:1252:: with SMTP id 79-v6mr2356794wms.70.1533842481486; Thu, 09 Aug 2018 12:21:21 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:20 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 04/18] khwasan, arm64: adjust shadow size for CONFIG_KASAN_HW Date: Thu, 9 Aug 2018 21:20:56 +0200 Message-Id: X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KWHASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory. This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when KHWASAN is enabled. Signed-off-by: Andrey Konovalov --- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/memory.h | 13 +++++++++---- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index e7101b19d590..7c92dcbaf1a6 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -93,7 +93,7 @@ endif # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 +KASAN_SHADOW_SCALE_SHIFT := $(if $(CONFIG_KASAN_HW), 4, 3) KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \ diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 49d99214f43c..6d084431b7f7 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -74,12 +74,17 @@ #define KERNEL_END _end /* - * KASAN requires 1/8th of the kernel virtual address space for the shadow - * region. KASAN can bloat the stack significantly, so double the (minimum) - * stack size when KASAN is in use. + * KASAN and KHWASAN require 1/8th and 1/16th of the kernel virtual address + * space for the shadow region respectively. They can bloat the stack + * significantly, so double the (minimum) stack size when they are in use. */ -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_SHADOW_SCALE_SHIFT 4 +#endif +#ifdef CONFIG_KASAN #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) #define KASAN_THREAD_SHIFT 1 #else From patchwork Thu Aug 9 19:20:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561719 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF75D1057 for ; Thu, 9 Aug 2018 19:21:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B02292B93B for ; Thu, 9 Aug 2018 19:21:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADC372B94C; Thu, 9 Aug 2018 19:21:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 332E22B947 for ; Thu, 9 Aug 2018 19:21:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93A316B000E; Thu, 9 Aug 2018 15:21:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8C82A6B0010; Thu, 9 Aug 2018 15:21:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71B046B0266; Thu, 9 Aug 2018 15:21:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 111816B0010 for ; Thu, 9 Aug 2018 15:21:25 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id p12-v6so5145213wro.7 for ; Thu, 09 Aug 2018 12:21:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3ryRmDkrio5+djlWtgIL1XomBIjT3hIM7Bw2C8CpFhE=; b=p4ODivf5MhTHFFgCAf7RDmPXOHeXR9NrDJe9yWR5Qof4kq5z20eON0l6XM1EJckS5N oPv0aXB1MMmRSyoJrVfIpNQBMn9E4VKEFphdJRpsCuPe63D0QYog+aSXNAH09Pi8k0jS iHFjrMOKekP8UtDIHcBiOgnMNJwjOk7D18BKSWzrPHdJ+gElh8i01/8EbMLDsIQUynyM gz3310VyuzIwwxeMFSpVbNsH0VQCLJIoHmPPjxVzwh1TOtKF19vEUwi0wHu3uhg+lPlu SQvTy2/BXdvFLlrS6eyfRMuu/QNvXhtfkDCpgVdE0P6gNczdsLv/XzuLYMT7ocql3FhZ s6Mw== X-Gm-Message-State: AOUpUlHKiKeuSUioXcCYbKMrLPo9td5TFzTB2yy8oS0lTRUpADL0NdUD T/SG/Kg5kMuzgJSHkIihajXjH0JSxJnIQILH/UGQuW2af6OCcu0TmSMaezkNoH13GeUO/BoISBG VnTZ8+uQt4VS6i/Vze9MvW0YYLTzlIVxg3J33/Ak9KiAY0J7rpGDchgPl6AwudCCDbw3BHx1mir p2r7NhAywP3+QQZi5jeu8NUMQTiMR6gxkdEWsOoUlvPYThCEGIddDh0fOeDXbgo2IGCZMBp0Tgr bwlPjaY8uBSbr1R79L/bBo0HVQEVXP2oDd5MabEU1UzhdO80vhcivibOcCTkgwIcMpSrEWftsgx QGjlL0kIX6g71VUUcMHrf4YJaFpSohruhSCXDuCKmRpe0Mf6D81ncSy77JKQto5rAkcvtf8/yIN c X-Received: by 2002:a1c:6354:: with SMTP id x81-v6mr2228177wmb.23.1533842484514; Thu, 09 Aug 2018 12:21:24 -0700 (PDT) X-Received: by 2002:a1c:6354:: with SMTP id x81-v6mr2228147wmb.23.1533842483623; Thu, 09 Aug 2018 12:21:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842483; cv=none; d=google.com; s=arc-20160816; b=AM1BkDFvDaw/LZDlQPHGgmh0D+RVf2VVtYLLFMZptR/fVY4hONtgZONdTnM5ffl+Jr HvOC7r+LKNv5vzpOz5LTG8vpilzJMuX1Y4JwMEaqBncyqhWIP5JcNmsidKSb1sgh9T/W tTUM6K+gTIR+xcBx8b7r/wQp60PJA8TFbo7BMOgK5w01YGU1Hy4DiTXK8u7ZEmWiq9Dm G4ODU2pS+RPcc+ZOjHByCvz1WGpiZO2sZ/dLQ/su4PGpuLZQcnWzw+7Q++1hvyycCRdo vSvEzCkKYOSrNujBgIE+TCvGWqCykRwP1VpPdLZBAyyq4a1BqWLG2Q2NUebVkm+MEhII UIrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=3ryRmDkrio5+djlWtgIL1XomBIjT3hIM7Bw2C8CpFhE=; b=sjxlyJ71Kddk90QbTgn6Us7UeanHZtZaHOXn+Y2wHb/u7AJvWdtYOdayhLbTId2bJd 3OKnr3RN//krVe4oFBqz4JJ1Y1pEG5/621aWs5eUT+y4oRoTqZNh7cFCSuRR/8Un16dD mEm0QdCuwxovV4pI9WHL48M8ydUvRbSKhzatPeOPrYK/ZgDmIBII9zQuaEAJA5+YNkwO xqkGldIc4DbqzoHeiVYA72NLXsXsbf1iJYXeO5Ea2usId5HPATOj4OG5sXpy1Lydgrsp r+/saiSmh0R5JzDhQEs0XFJP6CX42oWm9xWcsNk/v3j4hXR6hdIXOw80AW6tdEq6nUlQ p+NQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=UxXLnYOX; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r5-v6sor3090510wrj.70.2018.08.09.12.21.23 for (Google Transport Security); Thu, 09 Aug 2018 12:21:23 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=UxXLnYOX; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3ryRmDkrio5+djlWtgIL1XomBIjT3hIM7Bw2C8CpFhE=; b=UxXLnYOXsTtPvMPnq0yBl3j/JI36gd146aOw61VO9TimkxtEC3H4w3CuuBpoGBJ2l4 k4jOBKwSdsBV9M4j/NOLhvMEAlNie09HLGRVEKpJee05ti/9BCejjxR26Q7XepQEjobK qXb7j6ac9uA4mbSKJnK+IXmhFR/62Pcf2uJeV/VV3tYQFFtX66wDkOcE2sUeew9Zf006 X/0Zacy2XHPdihFNBhxxBod8Lxg2QNffvXLxfa8Ph9PvM0eD5lpdBTf6yi0Fp09Egu9l EO+oZ1Xx6Eilm9bAut7o6ds7bvYo8G7x9lAYAwB4NsAVVTzepg/txaIB2zYKntJzZ1fO 1aZw== X-Google-Smtp-Source: AA+uWPxabhR3PXtafftYVtlNjfKNSe8QwxMJ+0l4uKIG4jea94OGFd0TNxvfTX1RzBmHru8meFV/6Q== X-Received: by 2002:adf:a789:: with SMTP id j9-v6mr2120650wrc.277.1533842482963; Thu, 09 Aug 2018 12:21:22 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:22 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 05/18] khwasan: initialize shadow to 0xff Date: Thu, 9 Aug 2018 21:20:57 +0200 Message-Id: X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A KHWASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with KHWASAN we need to initialize shadow memory to 0xff. This commit does that. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 16 ++++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 12145874c02b..7a31e8ccbad2 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -44,6 +44,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) return __pa(p); } +static phys_addr_t __init kasan_alloc_raw_page(int node) +{ + void *p = memblock_virt_alloc_try_nid_raw(PAGE_SIZE, PAGE_SIZE, + __pa(MAX_DMA_ADDRESS), + MEMBLOCK_ALLOC_ACCESSIBLE, + node); + return __pa(p); +} + static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) { @@ -89,7 +98,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, do { phys_addr_t page_phys = early ? __pa_symbol(kasan_zero_page) - : kasan_alloc_zeroed_page(node); + : kasan_alloc_raw_page(node); + if (!early) + memset(__va(page_phys), KASAN_SHADOW_INIT, PAGE_SIZE); next = addr + PAGE_SIZE; set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); } while (ptep++, addr = next, addr != end && pte_none(READ_ONCE(*ptep))); @@ -139,6 +150,7 @@ asmlinkage void __init kasan_early_init(void) KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_START, PGDIR_SIZE)); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, PGDIR_SIZE)); + kasan_pgd_populate(KASAN_SHADOW_START, KASAN_SHADOW_END, NUMA_NO_NODE, true); } @@ -235,7 +247,7 @@ void __init kasan_init(void) set_pte(&kasan_zero_pte[i], pfn_pte(sym_to_pfn(kasan_zero_page), PAGE_KERNEL_RO)); - memset(kasan_zero_page, 0, PAGE_SIZE); + memset(kasan_zero_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); /* At this point kasan is fully initialized. Enable error messages */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 6608aa9b35ac..336385baf926 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -139,6 +139,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 + void kasan_cache_shrink(struct kmem_cache *cache); void kasan_cache_shutdown(struct kmem_cache *cache); @@ -149,4 +151,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_HW + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_HW */ + #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 154010ba6c1f..bed8e13c6e1d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -474,11 +474,12 @@ int kasan_module_alloc(void *addr, size_t size) ret = __vmalloc_node_range(shadow_size, 1, shadow_start, shadow_start + shadow_size, - GFP_KERNEL | __GFP_ZERO, + GFP_KERNEL, PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE, __builtin_return_address(0)); if (ret) { + __memset(ret, KASAN_SHADOW_INIT, shadow_size); find_vm_area(addr)->flags |= VM_KASAN; kmemleak_ignore(ret); return 0; From patchwork Thu Aug 9 19:20:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561721 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B470D1057 for ; Thu, 9 Aug 2018 19:21:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A48612B948 for ; Thu, 9 Aug 2018 19:21:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A205C2B94A; Thu, 9 Aug 2018 19:21:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2642A2B949 for ; Thu, 9 Aug 2018 19:21:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E91B96B0010; Thu, 9 Aug 2018 15:21:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E42BD6B0269; Thu, 9 Aug 2018 15:21:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C95DD6B026A; Thu, 9 Aug 2018 15:21:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 6C1816B0010 for ; Thu, 9 Aug 2018 15:21:26 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id s14-v6so5277029wra.0 for ; Thu, 09 Aug 2018 12:21:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lFiGSfRezZmnQnY92qgEO5G22hCK6sbxZ9zHPdYvrq4=; b=R946srbxgBf3zlCePfkUYEAc47yWRKq/lu5YPbPNU1ajMgfpfI+Tl23cQ+J9nSADmG FVBW3Cw8/hpNc6rzlEOfqyDGj7sjodxGzcw9zSCgpP3YUwAZ9Cf2TvWpKaB7IhXZIgso eUscrT4yZeDNZDilX3Qey95Azcl5oobrd8A4IGAXpFFdEhSk2ZeCha/BTv7Ce6iYX6g8 UUW7ZggxXhCJHoX5FPTeb8eWvyPz9f40/6IeFCj4hZRG2+l4wGqdMvaWoxa0sAlONRUG v+Ipi/QlXCAqi2n4rY8TQDw2DIe2qivQhphebx8Utwobw50RZkTLRe2Ha5eQ6YhRFZfM IOpw== X-Gm-Message-State: AOUpUlHQrfjPRYxfEW4nmlvfKP7pJqYqURBGAi+J4rDXe2UoSiT3Igxe bkWHmMGUlYceRQGAMRpb995DB0Y8VKyp+2MWqp8yd2yXjPF0O/XFV97CpfmTDb3cTJEu+TE52Pz szMD1K3aCcCLbILs+cIwjYUJeNFLckPc5IpKPgUyp+CsJyy5nnF9EXGbFTk366yOT3KADwnb/xJ dPQosnMyh8sCc8sz1MpH6GzzcVEjsSzoDts1CCgwWronhHwWZD33iSkaz3dURM/amS+/9lT2S+C pPQkXSabCdROJZyegJZA5nZYZ7umjhMwdVVA7XAVtCzt+K4vaQBC8CE0hRT4FV8tlavzl1DPbFi RVnvP0O4Ibkd1K3wNPz4tfWXtGN4fsEOB4aGxG4C1nKJdLUejxHpPeXqONRiUS3lkdeRe+aZhWI X X-Received: by 2002:adf:9282:: with SMTP id 2-v6mr2123754wrn.69.1533842485965; Thu, 09 Aug 2018 12:21:25 -0700 (PDT) X-Received: by 2002:adf:9282:: with SMTP id 2-v6mr2123725wrn.69.1533842485227; Thu, 09 Aug 2018 12:21:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842485; cv=none; d=google.com; s=arc-20160816; b=a6rw68jEx7ztnp8i6PG8xDA0Zx77NmskydZ1pzH55RvuTQ79GmpMFQL9yF1Lrc8lAW uL+xZzLhJTppgWccYrOVJoZDXSqJv5/hhgTGPuwqr/re2RZtEZ8QxSnBAb22Ax3cZ7dR T3JHVAZjixeFSom+p+7Rrf7SPywRxksPgILdAov4dyucICaOn82UCctFi76glRl48n28 DzLvVrGrf4xZ0ngoH+Gh0JlJqVHaJgTxNItA+xh791IMpTs0p7JT06MQYJE7Ehg6xMHM E99DmsICLO+DAZpvpr8T5bviBEtBdQRxYL+YBxXHzCH0IzkUeTYNNiW8N8WhtwtZPWRO xcfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=lFiGSfRezZmnQnY92qgEO5G22hCK6sbxZ9zHPdYvrq4=; b=YqJu7Q8zTKxKIkgt6ghNOXhJi/qZYnkOLnIEkJWEwnSc1Ve00ja8I23XSGcHCrw8Hg QSM+FqMKd50KiVB4JF7f1CB6mOu1cARVH7deNEMDeiLicW9mFGJLRhe4s6KGlOdAri49 FBWdIf1ASemTFD2AhGP+BgG3XRarhfToTFgJ5Ui8DP9WlFYSRsDpEiEna+60yvzx4WDf inAxUH6W8ZuABBR/ALcqr71xkwajhTEql8+wirTZWS1kTxQLgojumtQSksLlnwzAHcVf aFvVHCCsTYAHNlhYMdwInMDc0BFD7GqzOzxP4ZG94+HCVvCrfYBE2z9UGsl7juDfCH6Q yeKw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Nos1UDYo; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id j9-v6sor3078651wrq.66.2018.08.09.12.21.25 for (Google Transport Security); Thu, 09 Aug 2018 12:21:25 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Nos1UDYo; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lFiGSfRezZmnQnY92qgEO5G22hCK6sbxZ9zHPdYvrq4=; b=Nos1UDYofE5iwALIoGSkS0Q+oJZ6uWFtPCiNTKNWA9uq8ORDDXJvnDw5AyUJEdsjel dQOGsHXIkwOloSfAXwZjqYrk8IgXuz81I1YuA0MsaDYiQffMvtVseiOIFglGjFxRkHYa Yw/RfC4EQ6qPnIFsWJ8rOrCIV/S0JcmOve0hgF8hmGdu8H9iQMHV3wTsSvAggJuHXhQZ uVbjF50RnjwGVm5u+W7uX2UN1mNoHUpAT4UXxtWh+aMPeILzRP20CNrrvsA9ZLJmj1y1 PmTVJ5ZQjTK/8qyt99la6l0kSmKDHJ2qgqYwtC5HSfCAD9ksGSESs+Nq25oG6F8sMMFp CgQg== X-Google-Smtp-Source: AA+uWPzI7yJOFtEcYKJKQsLAaH/2GfV75jLtgqXee01D7GLmz1Df6TG6DdqoLCgZWUJM/bkd4L7VLQ== X-Received: by 2002:a5d:4007:: with SMTP id n7-v6mr2347871wrp.258.1533842484618; Thu, 09 Aug 2018 12:21:24 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:23 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 06/18] khwasan, arm64: untag virt address in __kimg_to_phys and _virt_addr_is_linear Date: Thu, 9 Aug 2018 21:20:58 +0200 Message-Id: <3e8180bbbc82be79412be86316c5b7ea6470b7dc.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP __kimg_to_phys (which is used by virt_to_phys) and _virt_addr_is_linear (which is used by virt_addr_valid) assume that the top byte of the address is 0xff, which isn't always the case with KHWASAN enabled. The solution is to reset the tag in those macros. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 6d084431b7f7..e9e054dfb1fc 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -92,6 +92,13 @@ #define KASAN_THREAD_SHIFT 0 #endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_SHIFTED(tag) ((unsigned long)(tag) << 56) +#define KASAN_SET_TAG(addr, tag) (((addr) & ~KASAN_TAG_SHIFTED(0xff)) | \ + KASAN_TAG_SHIFTED(tag)) +#define KASAN_RESET_TAG(addr) KASAN_SET_TAG(addr, 0xff) +#endif + #define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT) /* @@ -225,7 +232,12 @@ static inline unsigned long kaslr_offset(void) #define __is_lm_address(addr) (!!((addr) & BIT(VA_BITS - 1))) #define __lm_to_phys(addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET) + +#ifdef CONFIG_KASAN_HW +#define __kimg_to_phys(addr) (KASAN_RESET_TAG(addr) - kimage_voffset) +#else #define __kimg_to_phys(addr) ((addr) - kimage_voffset) +#endif #define __virt_to_phys_nodebug(x) ({ \ phys_addr_t __x = (phys_addr_t)(x); \ @@ -301,7 +313,13 @@ static inline void *phys_to_virt(phys_addr_t x) #endif #endif +#ifdef CONFIG_KASAN_HW +#define _virt_addr_is_linear(kaddr) (KASAN_RESET_TAG((u64)(kaddr)) >= \ + PAGE_OFFSET) +#else #define _virt_addr_is_linear(kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) +#endif + #define virt_addr_valid(kaddr) (_virt_addr_is_linear(kaddr) && \ _virt_addr_valid(kaddr)) From patchwork Thu Aug 9 19:20:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561731 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7577174A for ; Thu, 9 Aug 2018 19:21:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 948D92B942 for ; Thu, 9 Aug 2018 19:21:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 91ED62B94A; Thu, 9 Aug 2018 19:21:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D8A192B942 for ; Thu, 9 Aug 2018 19:21:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71B756B0269; Thu, 9 Aug 2018 15:21:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 67D846B026A; Thu, 9 Aug 2018 15:21:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51BE36B026B; Thu, 9 Aug 2018 15:21:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id E25086B0269 for ; Thu, 9 Aug 2018 15:21:28 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id r13-v6so623795wmc.8 for ; Thu, 09 Aug 2018 12:21:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WHpyWH2OO79/ehEid9c5Gvksh3odR6UxWphsCaQZh0Y=; b=iUvQkARZbVKTiYUqlMVgOLXA9ROvkI7xSmM4+boVN9Q78M4KSbbsNpRxBQjnxS+C14 DSEp/zR6IH1rHl5BHSXK2TuzJEicug+fhFuNpCBWBDpAahDdtKfy/Wd/I8rkJK3OpdvD behzD8SLZGKHGmrYhP8L1WX6znXRyMElNdDhte8uPwAq1cagdyv01O1VkAgY0FfbNysn c0Qws/oI+Sp44Vqda6RasoZNXvm5cyIjVFGvKsUlUeK0tQAasj/3I4sDmyh9rrCpNDL6 pHrGHy2RCWPQdpKzfO4vDsAhrcXT8LSnLZUvCVKhuI7BjouFYAsYyiGBszdCZMsd50qj 937A== X-Gm-Message-State: AOUpUlE8WUuzGjS5joP2CM7NvNxOlkA/GBOjeMDu1x1ACDL+rboG6JqL LndODfZimdrFqotpFvlwMnM6daIGy8wVuz40EqYtnGcgN1FVomOostUfOPgKuA4aCI3PV/Tz4GW pV4f92Ws0XgN0EinbLu0eXgg+gJy1zR1NqAk0+Hmc1K6jOS7vb0isVQM2AzlXY/wmhXChXhwn6V 9AAAHSm0ZQVMYuBYmnOU85CAHt7bkjQ6JVfj2ib42VsMT0F1z9HHF7JdBoy0+2esXiMa/dIb5VJ lucFRjU0oz/VeoToRRQ/BuRinqntMAX/L0OKypTbyPwIbjD8IY7OLo9Pl2G4/KSzhi5rESJPcBb 4CsgY23MHitCw/Ez5+TOPOtMaV9QwPNLZddGsKocPX8zS8fBmXh1JU3mPnUfSa6QpOMfViMrOjJ 9 X-Received: by 2002:a1c:4d09:: with SMTP id o9-v6mr2634772wmh.111.1533842488334; Thu, 09 Aug 2018 12:21:28 -0700 (PDT) X-Received: by 2002:a1c:4d09:: with SMTP id o9-v6mr2634743wmh.111.1533842487154; Thu, 09 Aug 2018 12:21:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842487; cv=none; d=google.com; s=arc-20160816; b=kJqMAXMELXxt9HXQwX6/eOTacP0pWESvk08nW0jDYtS2WEUpnTmVLIcJ2j23arMpGo QsQcI5qASAK+74AbBnK99OIAmw5AJKFl5B+CQbSHndoqbgrkAlhus+oABBLOLaV0fEX/ m7xCNo7ZGYn3Z5V/l94pajXfph/GmsIJ3RiQYlTtni3s15yeyJVqJWdmpU95D6UAjm6g +t1UX6YSDt/vrAhkwRUJAade6mIsx1v9iIYFrlyqd5iHqb4LFw2wqWo4xCLogZFPjMRA z7nqjBYdsH22eOgLVwC78Af2skNHSDXbVuL5MYAkUu8WPvVKMGgzsbqAxhwLbX+23YLY MWWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=WHpyWH2OO79/ehEid9c5Gvksh3odR6UxWphsCaQZh0Y=; b=YXs0gVNtTaKgamGM2BeiqPoUQyAHUsQQjrQj1RRJSoltGbA8miFvgeIvRnjr/ttGtZ ozheZGbTYJpBJyncQozJWW/4U/YB7kbvVcxnR8kZMr0JdGOcfXJbAvSQEprCgwMWB4yg 4jNBCQSyy8Z39WsfMW1Z15n7RRJ0srpUIuw0Omp/NyVKqcyu3sSyEX7NO790DM1UUXxz TQBwI5Tm7uOqbHDfyC/GZj6bPSatosIGhNdrhwprF8sWlhBG4HM3R78aBwkCEbtRvHDh 6wY4pO+3UVnwbkCFTKtvB/+PHa+1ioAcICxCxRfdA/APDBroKTdnIxxijQDE3HVM7gdn dS0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EnnkaLIv; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id m13-v6sor3075618wrh.48.2018.08.09.12.21.27 for (Google Transport Security); Thu, 09 Aug 2018 12:21:27 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EnnkaLIv; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WHpyWH2OO79/ehEid9c5Gvksh3odR6UxWphsCaQZh0Y=; b=EnnkaLIvpvOXPWY4qbbfL97NRSRCEPQrQGRGJk1CiKYBlVUyEgVrJCj81PpbZrFBCW ZafZllqFghylqRXyVT9TAe8aVRUIO+Zf865PmZkBNJfltq3JIbqe/hMgoXIS1u0cx2Za M24l3hNUWZfkNAxZZAfL7DMFbHKCHxe+JLaliL67CyqTTXt8+z+dFVInZ1C9aybdHAgC IdT2fUXRMZZawWK5sNhCAMxmiRhjhppFgU9Pnisjhr3tYCqguc6iMmwJhdidUaEau+kH /9Y0jQTWIjNjkIJI6rPumClXXtDJlY5u+uQhcB4x2kYxfNTSFfNpxGlYR0bd2catV3cX SAEw== X-Google-Smtp-Source: AA+uWPw5veD4jWu6nOuUR+3+zQNTsYjc+2qcmTqTMQnoEecnEvG8RzjHh6QiIZ3buwj7eXplOPFijg== X-Received: by 2002:a5d:63c1:: with SMTP id c1-v6mr2331635wrw.106.1533842486470; Thu, 09 Aug 2018 12:21:26 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:25 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 07/18] khwasan: add tag related helper functions Date: Thu, 9 Aug 2018 21:20:59 +0200 Message-Id: <4a1aae15394409b0b37ca3b7bc2aef322f1f05a1.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset (set to 0xff) the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 29 +++++++++++++++++ mm/kasan/kasan.h | 55 ++++++++++++++++++++++++++++++++ mm/kasan/khwasan.c | 65 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 151 insertions(+) diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index 7a31e8ccbad2..e7f37c0b7e14 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -250,6 +250,8 @@ void __init kasan_init(void) memset(kasan_zero_page, KASAN_SHADOW_INIT, PAGE_SIZE); cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); + khwasan_init(); + /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth = 0; pr_info("KernelAddressSanitizer initialized\n"); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 336385baf926..174f762da968 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -155,6 +155,35 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {} #define KASAN_SHADOW_INIT 0xFF +void khwasan_init(void); + +void *khwasan_reset_tag(const void *addr); + +void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr); +void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, + const void *addr); + +#else /* CONFIG_KASAN_HW */ + +static inline void khwasan_init(void) { } + +static inline void *khwasan_reset_tag(const void *addr) +{ + return (void *)addr; +} + +static inline void *khwasan_preset_slub_tag(struct kmem_cache *cache, + const void *addr) +{ + return (void *)addr; +} + +static inline void *khwasan_preset_slab_tag(struct kmem_cache *cache, + unsigned int idx, const void *addr) +{ + return (void *)addr; +} + #endif /* CONFIG_KASAN_HW */ #endif /* LINUX_KASAN_H */ diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 19b950eaccff..a7cc27d96608 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -8,6 +8,10 @@ #define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KHWASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ +#define KHWASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ +#define KHWASAN_TAG_MAX 0xFD /* maximum value for random tags */ + #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ @@ -126,6 +130,57 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_TAG_SHIFT 56 +#define KHWASAN_TAG_MASK (0xFFUL << KHWASAN_TAG_SHIFT) + +u8 random_tag(void); + +static inline void *set_tag(const void *addr, u8 tag) +{ + u64 a = (u64)addr; + + a &= ~KHWASAN_TAG_MASK; + a |= ((u64)tag << KHWASAN_TAG_SHIFT); + + return (void *)a; +} + +static inline u8 get_tag(const void *addr) +{ + return (u8)((u64)addr >> KHWASAN_TAG_SHIFT); +} + +static inline void *reset_tag(const void *addr) +{ + return set_tag(addr, KHWASAN_TAG_KERNEL); +} + +#else /* CONFIG_KASAN_HW */ + +static inline u8 random_tag(void) +{ + return 0; +} + +static inline void *set_tag(const void *addr, u8 tag) +{ + return (void *)addr; +} + +static inline u8 get_tag(const void *addr) +{ + return 0; +} + +static inline void *reset_tag(const void *addr) +{ + return (void *)addr; +} + +#endif /* CONFIG_KASAN_HW */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c index e2c3a7f7fd1f..9d91bf3c8246 100644 --- a/mm/kasan/khwasan.c +++ b/mm/kasan/khwasan.c @@ -38,6 +38,71 @@ #include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void khwasan_init(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(prng_state, cpu) = get_random_u32(); +} + +/* + * If a preemption happens between this_cpu_read and this_cpu_write, the only + * side effect is that we'll give a few allocated in different contexts objects + * the same tag. Since KHWASAN is meant to be used a probabilistic bug-detection + * debug feature, this doesn’t have significant negative impact. + * + * Ideally the tags use strong randomness to prevent any attempts to predict + * them during explicit exploit attempts. But strong randomness is expensive, + * and we did an intentional trade-off to use a PRNG. This non-atomic RMW + * sequence has in fact positive effect, since interrupts that randomly skew + * PRNG at unpredictable points do only good. + */ +u8 random_tag(void) +{ + u32 state = this_cpu_read(prng_state); + + state = 1664525 * state + 1013904223; + this_cpu_write(prng_state, state); + + return (u8)(state % (KHWASAN_TAG_MAX + 1)); +} + +void *khwasan_reset_tag(const void *addr) +{ + return reset_tag(addr); +} + +void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr) +{ + /* + * Since it's desirable to only call object contructors ones during + * slab allocation, we preassign tags to all such objects. + * Also preassign tags for SLAB_TYPESAFE_BY_RCU slabs to avoid + * use-after-free reports. + */ + if (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU) + return set_tag(addr, random_tag()); + return (void *)addr; +} + +void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, + const void *addr) +{ + /* + * See comment in khwasan_preset_slub_tag. + * For SLAB allocator we can't preassign tags randomly since the + * freelist is stored as an array of indexes instead of a linked + * list. Assign tags based on objects indexes, so that objects that + * are next to each other get different tags. + */ + if (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU) + return set_tag(addr, (u8)idx); + return (void *)addr; +} + void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { From patchwork Thu Aug 9 19:21:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561735 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 688821057 for ; Thu, 9 Aug 2018 19:21:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 594032B907 for ; Thu, 9 Aug 2018 19:21:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 571A22B942; Thu, 9 Aug 2018 19:21:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E636D2B907 for ; Thu, 9 Aug 2018 19:21:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD4C16B026A; Thu, 9 Aug 2018 15:21:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B35A96B026C; Thu, 9 Aug 2018 15:21:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98D656B026D; Thu, 9 Aug 2018 15:21:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f70.google.com (mail-wm0-f70.google.com [74.125.82.70]) by kanga.kvack.org (Postfix) with ESMTP id 39BB16B026A for ; Thu, 9 Aug 2018 15:21:30 -0400 (EDT) Received: by mail-wm0-f70.google.com with SMTP id v24-v6so639473wmh.5 for ; Thu, 09 Aug 2018 12:21:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=VF0nRIiNwsRgtxZxCam3wcoWu6FgQBt9JWN6ZbYEIxE=; b=bvZYRsPKqLPrkHd5Ft4dbgo3C4q+UuPmT2Uoxr0eYplTtsFdUMy++irj9SRRYebysS psQ9DyLqYs2Q81bb6THg9B5HBw6XPoat/Ac2fzWVcVMHOHtBbfGBqBysxTgqRDH/eB/C tVEeEspRfqol+tVrVErATvFt2Jrx0BGcdkSZx0CZxgx/GncC3e+nJetoPkh7LkVOkkhE CxJRcIWAwWSNWI/Ji35TQ1+ewOosR91jmkFnkdMOd42nKNT5b8oq0/Qsy2jkbH5TCA9r lKZCvvtApYfaHm/XlQVKRJ0GX4M0voPNvp7Gp4YsXs9z+1lD/qyYUrj8pILC7wDDoDqA 7wLA== X-Gm-Message-State: AOUpUlFeIdK92+Tb05zYV+Xo2Up0aSAwTdXlwln8IlZ433m83oze9z6G SkSb+VYej7GJMEtt914w45hCAWNlFVbvIV+eisoXvRkX+G+zQEw/PWxzbMZm5FG+52MwFbpY/9g DjyRAuE1qVwVqNRGMLCijQnBH2cLoM4UpQrTCJepEJvwPP3CA9zraZfS3InxWwksAXoXhY6FAfW cg9fFFt1iiAWE5W3h4JJ9ZldJwZUzalj5Fc/GqpR19n02v/oNTyRbw5p0rjK2krFTlloI1LUp/x GynMkeDkV2Tzw0J8R0Tm2h/4d5VYN1PGAoAcWmy/G9okI0o9vBzyE3r2sta/Xz4z3qxVxUx4NDX 3NTyUiY0oL2mfuuFGO2Vty8LhwHi+02PC6DKIngQ2NLl3DlZT8Zqpk0n3IJ1CGiDoa69xG9nYIP H X-Received: by 2002:a1c:f516:: with SMTP id t22-v6mr2318344wmh.103.1533842489630; Thu, 09 Aug 2018 12:21:29 -0700 (PDT) X-Received: by 2002:a1c:f516:: with SMTP id t22-v6mr2318324wmh.103.1533842488742; Thu, 09 Aug 2018 12:21:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842488; cv=none; d=google.com; s=arc-20160816; b=Z+z9Jp6K8ouKkrXUX3lykwRjctGC9JE6WtIObM4NsK7Xn8jlQlWoV/oHXGyNcMLS8u g5TpyyBL+itIasb7MQJAyriXp8VrEhI5I5Sxg/NGsH4pFXhuA1cVySNlEo2AhOm+xicC V/teX1jtyFJLC73OBirrpsBzmSvj9CWbOw3i6f3G34iANscENhr40p9UEUes4q3no/7r 1qrVAxuxn/yqgPoO7p1MHKRTFQ8kuLnDA5KkA/dBTilb7t48Wts4fAJJgzwgJffiQJOt cx0AxA9bUPHoZv8y0k5hTGicNDlk2X6x/kxslI6qAGDjAi6QmDmvHOXpCuWS3r6XS4tW Py9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=VF0nRIiNwsRgtxZxCam3wcoWu6FgQBt9JWN6ZbYEIxE=; b=wkltFPCr60tLWPJiUPyNp08gvxok2pxyxq/4z1jhqMMvV34QDbumIWbc5BREj/ICMZ Yls1kM5nA5dosxbtDvnyGzVSq2JX9zT/dnIA0Iid3OVNh3tyjzHZ98hC7pf2zvVs0XhE WZ483KigGmWkl9IsiivystzmhA7kMtzrVUfQYJ/FpxAIQg9S0/5FpatdEuqNbXWM97W4 /A/xRgtBNqhOdCmP8Z2/aCoacsiUHAd/AQfFNOxZL7RA1WyC7tkw6axBP4UQBqhuuvg3 cQNbqHgUF+XHQ0eb8ud0CDUVS68ulvpbska4ZasoCUitORmV0FMP6+HuWDBb71r3hCP3 DNqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=QpCmIiPM; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h6-v6sor3109586wre.67.2018.08.09.12.21.28 for (Google Transport Security); Thu, 09 Aug 2018 12:21:28 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=QpCmIiPM; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VF0nRIiNwsRgtxZxCam3wcoWu6FgQBt9JWN6ZbYEIxE=; b=QpCmIiPMHe6+oM4RZShxaxZZNbTQRBzBBSRaY11nA6pHh6Cplzw0n7Bv2ZKvLSc+Z4 4UHKKdRN140+D0usZ8bvkHnWnG8txA0JlSKLpWAQfuza/22n8fTJiK0/0pYWU3wYztfI imwCswDdjh8G72+vHnwhhEr+krHLWPfXvoJeBVl8horIV9DgQ3aUKbkBKZWci/UtBDYA PL0EQx54VrD05gyV+JikLNQvat9nDkHWRVe9t+VM5PtRt2mCIrCx1YRfqVdZ9scLkicl DbwqIS5YQJecOXU2pSRkfwTgR/fmR0bIo8aT7DX//C7AsNUp8s3K7DafoAwaXJJicyGk TwKA== X-Google-Smtp-Source: AA+uWPxhYsId7VaUHrC78gT2n+1+K1ulTL3bbqXgHVcHJ0gFRtLR022OJpGJB7i55vRAGW0nyD8Atg== X-Received: by 2002:a5d:6401:: with SMTP id z1-v6mr2271039wru.64.1533842488063; Thu, 09 Aug 2018 12:21:28 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:27 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 08/18] khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU Date: Thu, 9 Aug 2018 21:21:00 +0200 Message-Id: <625d42d5cb7f20bb54ce7af2c4b87910b1474c74.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor. The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it. Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed. Signed-off-by: Andrey Konovalov --- mm/slab.c | 6 +++++- mm/slub.c | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/slab.c b/mm/slab.c index 6fdca9ec2ea4..3b4227059f2e 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -403,7 +403,11 @@ static inline struct kmem_cache *virt_to_cache(const void *obj) static inline void *index_to_obj(struct kmem_cache *cache, struct page *page, unsigned int idx) { - return page->s_mem + cache->size * idx; + void *obj; + + obj = page->s_mem + cache->size * idx; + obj = khwasan_preset_slab_tag(cache, idx, obj); + return obj; } /* diff --git a/mm/slub.c b/mm/slub.c index 8fa21afcd3fb..a891bc49dc38 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1532,12 +1532,14 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page) /* First entry is used as the base of the freelist */ cur = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + cur = khwasan_preset_slub_tag(s, cur); page->freelist = cur; for (idx = 1; idx < page->objects; idx++) { setup_object(s, page, cur); next = next_freelist_entry(s, page, &pos, start, page_limit, freelist_count); + next = khwasan_preset_slub_tag(s, next); set_freepointer(s, cur, next); cur = next; } @@ -1614,8 +1616,10 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) shuffle = shuffle_freelist(s, page); if (!shuffle) { + start = khwasan_preset_slub_tag(s, start); for_each_object_idx(p, idx, s, start, page->objects) { setup_object(s, page, p); + p = khwasan_preset_slub_tag(s, p); if (likely(idx < page->objects)) set_freepointer(s, p, p + s->size); else From patchwork Thu Aug 9 19:21:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D46B174A for ; Thu, 9 Aug 2018 19:21:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C5512B855 for ; Thu, 9 Aug 2018 19:21:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A3CD2B92F; Thu, 9 Aug 2018 19:21:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB5362B93C for ; Thu, 9 Aug 2018 19:21:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4ABBF6B026C; Thu, 9 Aug 2018 15:21:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 40BFB6B026D; Thu, 9 Aug 2018 15:21:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20E0B6B026E; Thu, 9 Aug 2018 15:21:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f71.google.com (mail-wm0-f71.google.com [74.125.82.71]) by kanga.kvack.org (Postfix) with ESMTP id B6F9F6B026C for ; Thu, 9 Aug 2018 15:21:31 -0400 (EDT) Received: by mail-wm0-f71.google.com with SMTP id s18-v6so707144wmh.0 for ; Thu, 09 Aug 2018 12:21:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8+04WRzIPiPgey4OPz4HFnYO/J88O5syronaT3xg5j0=; b=d09DjYF+6dpKYIUYmO9ZnGDyTNsMvQY67w1v12FxEaK0eCJca7JoqJP2Qne+am3v5F 3xwXR3S7PJms+BqYLUKlS/yyApEqpprbAFG1Q2SajvB+U0tV0Bn/ucsW5mY4T35XYuwG lgyVWoz+R1SFc9tEzouHq2lwmWDA/gZxaI9Nr9n8fHFEmz282VFH97G3DXHkA3laTu8W pPmJOuHJ5yxEqpISErjcFgdVJhVJv2o7fz0MhpyJNYs3hpeeLB/WrCV8qDsAID4j7v3w MEJVy54gHv80/jym+6YH1hgr5gSx9C5Vgt9ipi5R9957IbgEXJOdMeECCWG1ESkBYLbu 7xZQ== X-Gm-Message-State: AOUpUlFGvzX3M4XL44a7+evEIoxEJTgYct1JwLqYXCwyfzzETtbabcX5 SeZjbDkB7ydzv9UnK2sk+FNR5IN8TMpdRS+D2EuYRJzgQf6KfRnID3nRoFpsaRKwpB6I0kVm26M Wuj0lyZhmvH3hcz72vvAxUXSOAD/vYkvxw/EeSN2XenMuijOGWeEhc6HxgP8+qFrIZ34IpKuz+z 95KLJZIy4Ocu6TgQrDUnz8bOJJBIwIxKJ1HuTKsrPd5XQTPzSofu3tnyN9h+DFMAKe4Gz5+Dssl PI9ug7XC0JJ9f8dNm+wa1RxQ22XDgUEMwtBo0I+ea40Lx7uNHqWhorhXmQh2nnjWLZizHjmRPvz GnEFjprgtBE2KzV3y2m5wHXGje7YW3+JfTr3Q7K6HYDtdTMQSzaRhT9Rlr45ySI+zzp9mRlkBRP I X-Received: by 2002:a1c:96c8:: with SMTP id y191-v6mr2337743wmd.37.1533842491256; Thu, 09 Aug 2018 12:21:31 -0700 (PDT) X-Received: by 2002:a1c:96c8:: with SMTP id y191-v6mr2337718wmd.37.1533842490480; Thu, 09 Aug 2018 12:21:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842490; cv=none; d=google.com; s=arc-20160816; b=zjztEKhZZbYYa+TtZAAH9/Yga6DU3RB6sJ2Puwfj6K6+XiR4PkHDfq/NKwi2ibJ2Oq lqeLjBcKiARC7+BM3nuK8cEr80+5GZM/Znyo2KYmRtHNKlV1MSY44RMgWFRZ3yhT0T9t TNt5yslfEWHGaHtdZJbGLL2CJ+8zMHKHRCLl7r3sU/U2bl0yzFHvo4sX81hNmqqjWm6F rF72CBoiclXdH3y7dJfIIdSU75TvX4jRAo10sO6nBtpPIhLCyJcE21J6fgyNoN0Cx2ts XXnjha2mlPQKVNKhoadLvuIU3TiibA6iaw+eB/9TM37Pmkd1MZqYM88rXhreQNnrBj6k qtfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=8+04WRzIPiPgey4OPz4HFnYO/J88O5syronaT3xg5j0=; b=aDJh3AFEgYoVT5PSAm10YeKITC72tZfP+Dk4/m4+ogoOKOcRf27ZmejuhlIWR1lsaj kfkMZymxbrH1K1eQ+1PZ0R4E3f8pk9LLEKi5LnjSRm3pL3JkdGT0ES4QPLRcSl+FHEnp lLSn0oiFg9ry4HkoemaZsIWcOW15v+GZeBeVcZ06zcVMjHY8k30Y6vchCaBVCHQ4n3Yb FJnk8qho92Ay7Ypf55xH3LBCN2xcVv6QSjCQZIzcHc5Jm5KzlWO9W/85mze9saDPxFW2 QvIljEJbNQ6Qf/wx2Hat71m9HEygEP5qQlAybX451lEGLe60W7OZwfTW4AD3MECaijBM cl7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=nxQ591Vh; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y1-v6sor3090273wrl.6.2018.08.09.12.21.30 for (Google Transport Security); Thu, 09 Aug 2018 12:21:30 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=nxQ591Vh; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8+04WRzIPiPgey4OPz4HFnYO/J88O5syronaT3xg5j0=; b=nxQ591Vhu03jnMxTYftvDNh225xeMHTn98JxuP6moeqXBbN9Wz89hlVBaEjcdTix/Q JRwhpBR80Gj1SHIRxBzEIJpPHE0ATKaojKZcNppqoD3eSLfSwJt62hQcbI+yB1lR5U0g 2EctZIagziNIyXWykgBIYJMtOuzWS4CSipKmJoUvEfFlMHcj2uAwIxJ7RR5ZA4IPAUhu 5w9rpydzp/Auhu1I81jLZccIFqUC+UGbqcwsEghtsA0vN32f/VyWRYK9wFnoNr4dHP6k kjHB0RKkKxr6fPi5vhpNLvGvoiUJVpk4pPHrJNama0hpw7dPu9m1uD3R+oeqfeucRXyC GqpQ== X-Google-Smtp-Source: AA+uWPwd82lZWG8HymUYNafSfNQe3iX/xSKDivVKRCbcK1qe34tzRwCCiSDAxySAScuUxsFNJYa/KA== X-Received: by 2002:adf:a789:: with SMTP id j9-v6mr2120826wrc.277.1533842489885; Thu, 09 Aug 2018 12:21:29 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:29 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 09/18] khwasan, arm64: fix up fault handling logic Date: Thu, 9 Aug 2018 21:21:01 +0200 Message-Id: X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP show_pte in arm64 fault handling relies on the fact that the top byte of a kernel pointer is 0xff, which isn't always the case with KHWASAN enabled. Reset the top byte. Signed-off-by: Andrey Konovalov --- arch/arm64/mm/fault.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index b8eecc7b9531..b7b152783d54 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -134,6 +135,8 @@ void show_pte(unsigned long addr) pgd_t *pgdp; pgd_t pgd; + addr = (unsigned long)khwasan_reset_tag((void *)addr); + if (addr < TASK_SIZE) { /* TTBR0 */ mm = current->active_mm; From patchwork Thu Aug 9 19:21:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561745 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 435691390 for ; Thu, 9 Aug 2018 19:21:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 318DF2B90A for ; Thu, 9 Aug 2018 19:21:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 236F42B922; Thu, 9 Aug 2018 19:21:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF3F62B924 for ; Thu, 9 Aug 2018 19:21:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 395FB6B026D; Thu, 9 Aug 2018 15:21:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 346206B026F; Thu, 9 Aug 2018 15:21:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0FD556B0270; Thu, 9 Aug 2018 15:21:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 9989B6B026D for ; Thu, 9 Aug 2018 15:21:33 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id t10-v6so5132137wrs.17 for ; Thu, 09 Aug 2018 12:21:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=oD+7K3NJxiAR9JTdyTtA3ICMxYFdVVudUaNngrniyVw=; b=jwqNGFH8F+7DemetjLgef/pNhBdTDm0BATP15JWOE8mahPlVIbrLciA1CH7VHWJhdv /irdwpDAOqn1a9lN+2XNBMmCNdbDhmZ+exSMwX14+dJup8Fu3F4kI+GaJY/JalUr2Bv9 4UKLqJ/K4SlQ2qm9MMVhC0plbM8xkP4qpig+38EReY0i5OxmofvxrC7Kv9mx1T5NexrC qKDlUSjeaxksimcFnIdwqFc7TAsBnsoUQ3Px7XZVMKDch3DbF8g22UNtufTdP7pwCEi9 GPG8pBWG3na1n6kA1vOkluFaj+QA1SZY7Eg8yNFWc1Ig+n1WJSYbjr21EUXprRDehDCZ 2WNQ== X-Gm-Message-State: AOUpUlFN+kT+x9UTFBdmgh9q76zt3U6PoIJoh2Y/rvbDIjFrJNJbZRPc gB5vJZZ90PsB/NSn2jmiGWc8qL8PWRDmxXt53PYqw70OKDp/JAb7+DcpsN2LqonahE4G4QpVadV YoZNtgPyiE3cpzQRwqHls4ZIMoYqNu9dkq5xkyZQ/+r+gmgxOl+94qGPqBsMRIdtRDtlmJaMbJt yxmJ4vl5zJxZ+ztluXHOSbpsuwDcMAiQs1dzRn6cL+O57qSyhWEJxLFcJCdTrMlfOmia/3+CP12 l8RgEafVeuQjbVv+hvWdivXpzGzetYvuZj6OxeAC7VVpH80N309jOucXEbrr7U6s2vVv/gvaxGk uTyUITXHsJPqI9Q7fEFcdWZ6x4GuSvrXQ+5XTDAPFsDKJ6tlLQsfO2AF30jII32qR9CtyMLrSpv a X-Received: by 2002:a1c:8291:: with SMTP id e139-v6mr2578981wmd.39.1533842493020; Thu, 09 Aug 2018 12:21:33 -0700 (PDT) X-Received: by 2002:a1c:8291:: with SMTP id e139-v6mr2578952wmd.39.1533842492195; Thu, 09 Aug 2018 12:21:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842492; cv=none; d=google.com; s=arc-20160816; b=0uAom0YeURTNEoFy0DEQ3VP1Q8luN/FlKckOdniwwwpjdGhAW5DGPY7AjR1us+0yvt TOeM3YM4mBVhBFoZVhzG98bCit8cgj0jurSQIl8CX583yJcq/uSEc7yFTgtUA9S+rYoL s0MYtMHsuSKQXKCsGVhCuC7g8D4FPNOODpd13UHFv0Bmzcpg1PNAcAzpTKLwc72aARbM uixR1mc+8OEu0/VR4TbCKG6FXkmprCS4XvYKypaPQc6HGz37iW4Y7snKnkqq5c7iAsqL 6/IJB/EsKibYY5d+NxkG42R93W0mubZS/CvifULtMoB2ouDvnjvGX+/uLIa+4WSOvUmu I+tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=oD+7K3NJxiAR9JTdyTtA3ICMxYFdVVudUaNngrniyVw=; b=rorwemg68bnF3YKazgBwnyFRT87ZLD/7+f00/6vszIbeRV/++37jN0jJoqfXkmGXM8 gmo6I5O770qXQGZjMvP30LpjI7GSxKYKWhM5LoAMnSVovf/40O2zRlSTadb9qNK/NgXS B0x9KDO5A2eQh1dxer4FRi8lkQNKwGrM6r+pg3XJ36j4f5oQNWJEi5xgIWFkQSakYaT7 QZ3CoROVs9Sn3xOLw5mpmC91tT66GNuuHy5XEuwz2Z2yFHB4fwuo35sWIvSTxtTwqtwt TOokQVnLhHXBwWVwiineZeju3tYr6HFPR4lmEYvb30R17od6oEx4jpMeSyK8pR91S/AZ xT2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=bZPgstXb; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r5-v6sor3090604wrj.70.2018.08.09.12.21.32 for (Google Transport Security); Thu, 09 Aug 2018 12:21:32 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=bZPgstXb; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oD+7K3NJxiAR9JTdyTtA3ICMxYFdVVudUaNngrniyVw=; b=bZPgstXbwZxXDVi2nvh6WzmRfMpGDSA/Fne0ZVeSm8mxfScPujTYtuwX15ylbHkvoc c/CUjAYYRDyjPKkQu0phNVRCGIclqn91uWdAj/0u2yNyNdXYKfI/TpYgcpDV/qnl3qMp Fnq56xNwUj9aAPfL9FR8I49xffr+6JEYJopN+OUgqrUgB6boViYqjt5RxXcC3HRTkn3Y 4ag8Fc1odQbVpT4/brI/VO2Umgdm/QW1JLQIt+ds4fXeZncAAUtjLUXIILemlq80hu8z mtSHWNbAsCGa3lV0+WRa/ADf2/FGivoGULA+O36urx/7C7AN3u3MJsVGmhddajSYooka FZRg== X-Google-Smtp-Source: AA+uWPyIKsGrs3/6MSHl7R25hDOnzR5k5iYFLEI9aiR4odNOMfVMIRu3E50s0D493xzNbwmsjkTjJw== X-Received: by 2002:adf:fac1:: with SMTP id a1-v6mr2167983wrs.74.1533842491567; Thu, 09 Aug 2018 12:21:31 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:30 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 10/18] khwasan, arm64: enable top byte ignore for the kernel Date: Thu, 9 Aug 2018 21:21:02 +0200 Message-Id: <5f0609d502b690ef28a142fd4b96fae6ff4ee722.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when KHWASAN is used. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index fd208eac9f2a..483aceedad76 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -289,6 +289,7 @@ #define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 03646e6a2ef4..c5175e098d02 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_HW +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /* @@ -440,7 +446,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /* From patchwork Thu Aug 9 19:21:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561747 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E4781390 for ; Thu, 9 Aug 2018 19:21:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1F5232B92A for ; Thu, 9 Aug 2018 19:21:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D1102B948; Thu, 9 Aug 2018 19:21:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B51E52B94C for ; Thu, 9 Aug 2018 19:21:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9616B6B026F; Thu, 9 Aug 2018 15:21:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 915AF6B0270; Thu, 9 Aug 2018 15:21:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D91F6B0271; Thu, 9 Aug 2018 15:21:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by kanga.kvack.org (Postfix) with ESMTP id 1A9476B026F for ; Thu, 9 Aug 2018 15:21:35 -0400 (EDT) Received: by mail-wr1-f72.google.com with SMTP id f13-v6so5253094wru.5 for ; Thu, 09 Aug 2018 12:21:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GuTtM2nMrJ4Dauv6kbJ/TNvmQSeA8vc6xmRa98AW0qs=; b=Wq33vInc1S43THA83/6CvNQpXnmeRV75FFu1OkR/xtmOX16+fuB+EZxQyKtqZFCGXr WZcpVJu1QDmJyFIQfQOrxAixY0o5mSQpgGegf5XG0iXV31uHGWl6zYqEDQ7H+dZ2+4L+ 53G9usqw+1lcL3qF0i+7yZaWtHNPmrkNJ5MyZh6vWl9THlF+6UhPDMr0jPIqwBRAnmNH EgtULUCVZerRkV3h3kFCzLscJx9FYzbDaB9s0yGpoFflOcVIsFGj98EK5uGQ2/10sEnW IM2tUbEDu8Xl3RYTireWNpbIF2RO3VKpCyTbeMZrMPMY9+1qeTLPTu8FVsvfjauY0uJS h9Fg== X-Gm-Message-State: AOUpUlEKCO5efcw8QcLLeuLErFrJGbSPEEgPLnMVqENCwQOeAWqbtwUG DHq8q08P4WebzYOaWE4F0zReaTeB2s8hT/q2VBmJc2IDh0gK8SijpZyp4xTC/7Xhb/H1dIrO7pv wCji7BDH3+zRuRVqFk3E2g6b0c1Zb6nmw39aIJENEDcFJovOMo9ztWmLK7VUA3xww6K3K1vwqIN igqHnGRfmSTDftgc4HDTIGOoNlWFQeJvYn2uAb+mBSQBItm14OJtP13aYnfRyd66fxYYHRtnoZL wAmYOEOR4oNQY3Slb6b5u+hsYW29KLxfuK0dx7TCMT00u9GeuPrPIyyWxePunZ07TV65CtJp7vX 7WakC39OjREzx/konZVgfw2RGwLd13oXVI9+k03/TLh/1fsJyPI7w+xcpwgbm7s/dOlXbP1VO6B b X-Received: by 2002:a1c:6c14:: with SMTP id h20-v6mr1339742wmc.138.1533842494596; Thu, 09 Aug 2018 12:21:34 -0700 (PDT) X-Received: by 2002:a1c:6c14:: with SMTP id h20-v6mr1339716wmc.138.1533842493869; Thu, 09 Aug 2018 12:21:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842493; cv=none; d=google.com; s=arc-20160816; b=T3EU/VUVa/cYNQ+qkpif1bkRgrbp661+WTp5iljEOBlfSs6NRZenLiptlub9mrr63y 7i45Lz0AZ2RbB5oEaB8PT3qsyeaH/mtuKP9pIwrIpa8ApfLreYYeEmaG9JHXiJWOVMuz 7SYhu2pyg3imgoSE8zXb93yXasshpsPcuX4bLc9NwE/mqEjuTLPe3sNZOpFKz8EjdnnS IYTEO6+JOG/sEJfGgYcV5seh+hPud04L2K4XiSYkUZG/XKXrerTmP5rGCwo/zmj8ARwz 2kEsPPaNZdvIVht/rCog7+LZV5TD831BbpAgAIs1J8U0tm0uep7Wv1Sw3r1l+BxqcgfP jNRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=GuTtM2nMrJ4Dauv6kbJ/TNvmQSeA8vc6xmRa98AW0qs=; b=0/Q/i98N5pSW1v3E+70EmQj/xfl7Xb1k3/PAJl5fOJjPFyxt8E1ktyMCI5ZiMaV9Mh u1XE9Sxx7+bxoGQ99SupK8K/n+6y50Mdo0g8Agd3nGliTco2FblZKoZVFsPBzeL9JYCe hhY4ssps/hMAlRsr5Wru3M6zPQvFT5zgILOW4Q59AozIfukNvypl7G5dQ6yW2jcTXc8U 1gHSVRBloZx/8FXgo7JHiWS9jtf84NetoOG0UjoLHB+9KoGp8yL2WAkMsoumFy8aw+cz thtA7P63D6X8i1LogLsbb+4xFMLAR6rQNvkXhawpJeHEiipaGwv3UuWVLOy6ec/SVhgg hLvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=S86C41Sx; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id e17-v6sor3091807wri.46.2018.08.09.12.21.33 for (Google Transport Security); Thu, 09 Aug 2018 12:21:33 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=S86C41Sx; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GuTtM2nMrJ4Dauv6kbJ/TNvmQSeA8vc6xmRa98AW0qs=; b=S86C41SxOkkA8LBqFkpGV082ZyZNGC5QXHiWw3T1sDHhydoGXq5vi07VfyMKChS7Sv Iz5d45gyPejwEm//MqYruIpmrVBNH48A3qms4bWpdTCqCleD9znCNyEiox0DrAuzXVNg RmyODypNQp4LL9JXBxYdGfunCTrPLSe2taxfSQavGRSj1pwjmn8u6GTECTagwyUTHx9X s1hfaOhM086GaejmBTAL5KQbzL4Z7liCLMyQkqZQeXkNkfu4Cn38T7tpr8Av/auJ2rj4 4k/Bv3rIHlS8YHxntefOQYqOaUepl2oJX4KlBAZWMyTy+fUp8E9uyScz3vDUpmWoEMCI Is9w== X-Google-Smtp-Source: AA+uWPxhKq2oJvxj9GKtDLnX44F7dLpq6UJjkmFDNe3axY/KVJQUX9aqWFA6v9GYuLkDRiiBufCDGg== X-Received: by 2002:a5d:4007:: with SMTP id n7-v6mr2348121wrp.258.1533842493247; Thu, 09 Aug 2018 12:21:33 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:32 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 11/18] khwasan, mm: perform untagged pointers comparison in krealloc Date: Thu, 9 Aug 2018 21:21:03 +0200 Message-Id: <3fbf2a591009f1b9743e82a51ec09019eb054c10.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. KHWASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not. Signed-off-by: Andrey Konovalov --- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index a99a1feb52c4..e17ed1573d08 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1513,7 +1513,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret = __do_krealloc(p, new_size, flags); - if (ret && p != ret) + if (ret && khwasan_reset_tag(p) != khwasan_reset_tag(ret)) kfree(p); return ret; From patchwork Thu Aug 9 19:21:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 458F61390 for ; Thu, 9 Aug 2018 19:21:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 349332B93E for ; Thu, 9 Aug 2018 19:21:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 31E772B92D; Thu, 9 Aug 2018 19:21:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17AB52B957 for ; Thu, 9 Aug 2018 19:21:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5DBE6B0271; Thu, 9 Aug 2018 15:21:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B111A6B0272; Thu, 9 Aug 2018 15:21:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 965F36B0274; Thu, 9 Aug 2018 15:21:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 1597D6B0271 for ; Thu, 9 Aug 2018 15:21:38 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id p12-v6so5145587wro.7 for ; Thu, 09 Aug 2018 12:21:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bA+nPqF85AuY6wMry6n5c+MB0AtAeSdeQJM3sGttEtE=; b=WRyjh9c1z+EFkrTmb2/dldn6Z4xaBOwmd1dlAcTIhX39ZcePgWmEUIS9STAEmv2RiL NRJosG7lTwLhxfP7kj882fyfENB4zZHiDYluPyBYscXZ7kF0wDpeMI9+Bf9DI059JaJt 6MbwfSIgB5B/2M4GUEfLTD0vFLnepxSxfTYKCbxacjda3w8NQTeeU5Vr/d0FltFqAYso w+P1ptg4DRo2SgU/cjJf/Qm6oitF4elXS9IUjD5bNGVRSjJ1bZlo+iuA9V+b4a5+5iNl meQbIWPnaEjntO1RXw1q2YqbMKBVdO8JcTIgU+rOWFgHETncJTA4OpDWFN7zMEvwL5Nv Guaw== X-Gm-Message-State: AOUpUlEGhplYWiWOt3lTVAa+20aD9YCdou93dh3at50sxur/dNiNxW8b vkPRhnhzWSlL8lS4nzhmcCzDgn+gpdUaSYnqhE3Qwl68R7ooQeOI1UBe0yCISxXGK28VNE4IZPg D9RuZjdwXvqgNZIgWdl2vT+PY5PxQEkloN5x4z3GLQyKiwHTx3q3Q88W9g11LHHbsfiBE2tjl8M HnZZmiUCBqUQdl9ODCLkkRbxtN7bZbfXx2V+tNLEesPZKBF5+Pe5JQ/K0T9UIgx+GpvqoYP3lC+ 1WuMZsdVXEv1TYv2rtKuw/p/uA7+ytzf7hXdHQULnorqSv5Im7l36WPsVyYLVgNSrdMPfowfkf8 nB/lrkPJXy0cCzpbJOJH/YmNxXNKiUOkWv/WJc+3vjyaEjpgpZyoAndFnPllIJrqgEjx1ZSPVJy K X-Received: by 2002:a1c:e146:: with SMTP id y67-v6mr2342116wmg.108.1533842497553; Thu, 09 Aug 2018 12:21:37 -0700 (PDT) X-Received: by 2002:a1c:e146:: with SMTP id y67-v6mr2342077wmg.108.1533842495975; Thu, 09 Aug 2018 12:21:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842495; cv=none; d=google.com; s=arc-20160816; b=xiWN+hmbHGSULiJrH5DfRerONIVlxTM9jQekwl+GgmT254KfvySJvSDmejCbpMd+aV Q9A1uMsr5r2Sy0AJ5YDSsXR2OFvESRzjfIyPhML1K2zp7BmGRBl8W+Z814OxUOsCafnF Rzwj/7FQel2qxJgUJDhd8FmEfX7IOrkv16N82zSJ0UwsEqDKPBVHothP1YQX3NW7ANRg 3Wrl0NswHzW9cVi89ileT/GmPrmTUMRufXtwQnj5dhzJz+dqu6HS69odeZCVDFK96Wsw 3aZlwy+eK2YndpUiIs4h5IMb3WowS9yu5GMpeAKzb2aMyVzBIw7CTadGAw1kpusDffPW /JNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=bA+nPqF85AuY6wMry6n5c+MB0AtAeSdeQJM3sGttEtE=; b=ll1pjhd9AmfiqLoaXb2LT50tcCBg7Du3oEbbje6WAaWR4e4WHk3ahtMdoSqNzCRhwi tMlPaR7Oa4Ik1qr8cU+1IK7GiFNRPQGdnUYw6qtHHEHPBGs9C3Wi7aUua2sbUZ8OE+EA eDFsOt6zM6LdzAAL/MbGYPUH7UvzSRhvBjCspma3tW4XPIa+KOrxhBql1HcXTa3wTyPV fqrleQW2SRUW04NPcCmsJhhcXMi8VJF8czzDvhstvTul9o/nMZmR67d+ZDtcmQl/d5AI DFovVeYt130Hp8mR8aDjIXETv9ckh+iUZc+hGRQrrSIEqNzOBvtW9/bMsnqpJpNrQPS5 VYog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XfCydhJg; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id i196-v6sor1949944wmd.55.2018.08.09.12.21.35 for (Google Transport Security); Thu, 09 Aug 2018 12:21:35 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XfCydhJg; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bA+nPqF85AuY6wMry6n5c+MB0AtAeSdeQJM3sGttEtE=; b=XfCydhJgBE7txO6zCAeEqno2/hLpcqOtXtNQSEAdK/JOrO3k4beBs09ARw9049FemG dL07twOuXO5DSdXbRVhJ3il2g16rmQIiRpLSBRoeY8N2n66dU0dy02x4JQ7MOUHbzUNY kj/beEyti+XUuiTAKdwJ5IFcTorjGSbpjXn9VmTBGxOPDpQxCdRctAxA5C8ed8D//xbV XW8e+dFM/re155outYLvcfaPsQWXlg0sF4wadfqpXlORqx/QCrMRIuRpzCU/creA8O53 ZL4sP2X7k5LJ5v0cq91W7obyhKTI1ptnbuY5l9cTsJoZPwZ843EPHEcWR9pM8mzidOPu 9VFg== X-Google-Smtp-Source: AA+uWPx/3BQyYqiO5bRq/kDKD8Sn3WDE4S9/OmDDt3afIipSxBsWzzWznbSjeWpcLGcSqpt4WUJi3A== X-Received: by 2002:a1c:d946:: with SMTP id q67-v6mr2382046wmg.156.1533842494772; Thu, 09 Aug 2018 12:21:34 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:34 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 12/18] khwasan: split out kasan_report.c from report.c Date: Thu, 9 Aug 2018 21:21:04 +0200 Message-Id: X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch moves KASAN specific error reporting routines to kasan_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by KHWASAN. Signed-off-by: Andrey Konovalov --- mm/kasan/Makefile | 4 +- mm/kasan/kasan.h | 7 ++ mm/kasan/kasan_report.c | 158 +++++++++++++++++++++++++ mm/kasan/khwasan_report.c | 39 +++++++ mm/kasan/report.c | 234 +++++++++----------------------------- 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/kasan_report.c create mode 100644 mm/kasan/khwasan_report.c diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index 14955add96d3..7ef536390365 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -14,5 +14,5 @@ CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o -obj-$(CONFIG_KASAN_HW) += khwasan.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o kasan_report.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.o khwasan_report.o diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a7cc27d96608..82672473740c 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow(const void *addr) +{ + return (addr >= kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); +} + void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +const char *get_bug_type(struct kasan_access_info *info); + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/kasan_report.c b/mm/kasan/kasan_report.c new file mode 100644 index 000000000000..2d8decbecbd5 --- /dev/null +++ b/mm/kasan/kasan_report.c @@ -0,0 +1,158 @@ +/* + * This file contains KASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +static const void *find_first_bad_addr(const void *addr, size_t size) +{ + u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); + const void *first_bad_addr = addr; + + while (!shadow_val && first_bad_addr < addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +static const char *get_shadow_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + u8 *shadow_addr; + + info->first_bad_addr = find_first_bad_addr(info->access_addr, + info->access_size); + + shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); + + /* + * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look + * at the next shadow byte to determine the type of the bad access. + */ + if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) + shadow_addr++; + + switch (*shadow_addr) { + case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: + /* + * In theory it's still possible to see these shadow values + * due to a data race in the kernel code. + */ + bug_type = "out-of-bounds"; + break; + case KASAN_PAGE_REDZONE: + case KASAN_KMALLOC_REDZONE: + bug_type = "slab-out-of-bounds"; + break; + case KASAN_GLOBAL_REDZONE: + bug_type = "global-out-of-bounds"; + break; + case KASAN_STACK_LEFT: + case KASAN_STACK_MID: + case KASAN_STACK_RIGHT: + case KASAN_STACK_PARTIAL: + bug_type = "stack-out-of-bounds"; + break; + case KASAN_FREE_PAGE: + case KASAN_KMALLOC_FREE: + bug_type = "use-after-free"; + break; + case KASAN_USE_AFTER_SCOPE: + bug_type = "use-after-scope"; + break; + case KASAN_ALLOCA_LEFT: + case KASAN_ALLOCA_RIGHT: + bug_type = "alloca-out-of-bounds"; + break; + } + + return bug_type; +} + +static const char *get_wild_bug_type(struct kasan_access_info *info) +{ + const char *bug_type = "unknown-crash"; + + if ((unsigned long)info->access_addr < PAGE_SIZE) + bug_type = "null-ptr-deref"; + else if ((unsigned long)info->access_addr < TASK_SIZE) + bug_type = "user-memory-access"; + else + bug_type = "wild-memory-access"; + + return bug_type; +} + +const char *get_bug_type(struct kasan_access_info *info) +{ + if (addr_has_shadow(info->access_addr)) + return get_shadow_bug_type(info); + return get_wild_bug_type(info); +} + +#define DEFINE_ASAN_REPORT_LOAD(size) \ +void __asan_report_load##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, false, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_load##size##_noabort) + +#define DEFINE_ASAN_REPORT_STORE(size) \ +void __asan_report_store##size##_noabort(unsigned long addr) \ +{ \ + kasan_report(addr, size, true, _RET_IP_); \ +} \ +EXPORT_SYMBOL(__asan_report_store##size##_noabort) + +DEFINE_ASAN_REPORT_LOAD(1); +DEFINE_ASAN_REPORT_LOAD(2); +DEFINE_ASAN_REPORT_LOAD(4); +DEFINE_ASAN_REPORT_LOAD(8); +DEFINE_ASAN_REPORT_LOAD(16); +DEFINE_ASAN_REPORT_STORE(1); +DEFINE_ASAN_REPORT_STORE(2); +DEFINE_ASAN_REPORT_STORE(4); +DEFINE_ASAN_REPORT_STORE(8); +DEFINE_ASAN_REPORT_STORE(16); + +void __asan_report_load_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, false, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_load_n_noabort); + +void __asan_report_store_n_noabort(unsigned long addr, size_t size) +{ + kasan_report(addr, size, true, _RET_IP_); +} +EXPORT_SYMBOL(__asan_report_store_n_noabort); diff --git a/mm/kasan/khwasan_report.c b/mm/kasan/khwasan_report.c new file mode 100644 index 000000000000..2edbc3c76be5 --- /dev/null +++ b/mm/kasan/khwasan_report.c @@ -0,0 +1,39 @@ +/* + * This file contains KHWASAN specific error reporting code. + * + * Copyright (c) 2014 Samsung Electronics Co., Ltd. + * Author: Andrey Ryabinin + * + * Some code borrowed from https://github.com/xairy/kasan-prototype by + * Andrey Konovalov + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type(struct kasan_access_info *info) +{ + return "invalid-access"; +} diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 5c169aa688fd..155247a6f8a8 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,5 +1,5 @@ /* - * This file contains error reporting code. + * This file contains common KASAN and KHWASAN error reporting code. * * Copyright (c) 2014 Samsung Electronics Co., Ltd. * Author: Andrey Ryabinin @@ -39,103 +39,34 @@ #define SHADOW_BYTES_PER_ROW (SHADOW_BLOCKS_PER_ROW * SHADOW_BYTES_PER_BLOCK) #define SHADOW_ROWS_AROUND_ADDR 2 -static const void *find_first_bad_addr(const void *addr, size_t size) -{ - u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; - - while (!shadow_val && first_bad_addr < addr + size) { - first_bad_addr += KASAN_SHADOW_SCALE_SIZE; - shadow_val = *(u8 *)kasan_mem_to_shadow(first_bad_addr); - } - return first_bad_addr; -} +static unsigned long kasan_flags; -static bool addr_has_shadow(struct kasan_access_info *info) -{ - return (info->access_addr >= - kasan_shadow_to_mem((void *)KASAN_SHADOW_START)); -} +#define KASAN_BIT_REPORTED 0 +#define KASAN_BIT_MULTI_SHOT 1 -static const char *get_shadow_bug_type(struct kasan_access_info *info) +bool kasan_save_enable_multi_shot(void) { - const char *bug_type = "unknown-crash"; - u8 *shadow_addr; - - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); - - /* - * If shadow byte value is in [0, KASAN_SHADOW_SCALE_SIZE) we can look - * at the next shadow byte to determine the type of the bad access. - */ - if (*shadow_addr > 0 && *shadow_addr <= KASAN_SHADOW_SCALE_SIZE - 1) - shadow_addr++; - - switch (*shadow_addr) { - case 0 ... KASAN_SHADOW_SCALE_SIZE - 1: - /* - * In theory it's still possible to see these shadow values - * due to a data race in the kernel code. - */ - bug_type = "out-of-bounds"; - break; - case KASAN_PAGE_REDZONE: - case KASAN_KMALLOC_REDZONE: - bug_type = "slab-out-of-bounds"; - break; - case KASAN_GLOBAL_REDZONE: - bug_type = "global-out-of-bounds"; - break; - case KASAN_STACK_LEFT: - case KASAN_STACK_MID: - case KASAN_STACK_RIGHT: - case KASAN_STACK_PARTIAL: - bug_type = "stack-out-of-bounds"; - break; - case KASAN_FREE_PAGE: - case KASAN_KMALLOC_FREE: - bug_type = "use-after-free"; - break; - case KASAN_USE_AFTER_SCOPE: - bug_type = "use-after-scope"; - break; - case KASAN_ALLOCA_LEFT: - case KASAN_ALLOCA_RIGHT: - bug_type = "alloca-out-of-bounds"; - break; - } - - return bug_type; + return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); -static const char *get_wild_bug_type(struct kasan_access_info *info) +void kasan_restore_multi_shot(bool enabled) { - const char *bug_type = "unknown-crash"; - - if ((unsigned long)info->access_addr < PAGE_SIZE) - bug_type = "null-ptr-deref"; - else if ((unsigned long)info->access_addr < TASK_SIZE) - bug_type = "user-memory-access"; - else - bug_type = "wild-memory-access"; - - return bug_type; + if (!enabled) + clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); } +EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); -static const char *get_bug_type(struct kasan_access_info *info) +static int __init kasan_set_multi_shot(char *str) { - if (addr_has_shadow(info)) - return get_shadow_bug_type(info); - return get_wild_bug_type(info); + set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); + return 1; } +__setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info) +static void print_error_description(struct kasan_access_info *info, + const char *bug_type) { - const char *bug_type = get_bug_type(info); - pr_err("BUG: KASAN: %s in %pS\n", bug_type, (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", @@ -143,25 +74,9 @@ static void print_error_description(struct kasan_access_info *info) info->access_addr, current->comm, task_pid_nr(current)); } -static inline bool kernel_or_module_addr(const void *addr) -{ - if (addr >= (void *)_stext && addr < (void *)_end) - return true; - if (is_module_address((unsigned long)addr)) - return true; - return false; -} - -static inline bool init_task_stack_addr(const void *addr) -{ - return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); -} - static DEFINE_SPINLOCK(report_lock); -static void kasan_start_report(unsigned long *flags) +static void start_report(unsigned long *flags) { /* * Make sure we don't end up in loop. @@ -171,7 +86,7 @@ static void kasan_start_report(unsigned long *flags) pr_err("==================================================================\n"); } -static void kasan_end_report(unsigned long *flags) +static void end_report(unsigned long *flags) { pr_err("==================================================================\n"); add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE); @@ -249,6 +164,22 @@ static void describe_object(struct kmem_cache *cache, void *object, describe_object_addr(cache, object, addr); } +static inline bool kernel_or_module_addr(const void *addr) +{ + if (addr >= (void *)_stext && addr < (void *)_end) + return true; + if (is_module_address((unsigned long)addr)) + return true; + return false; +} + +static inline bool init_task_stack_addr(const void *addr) +{ + return addr >= (void *)&init_thread_union.stack && + (addr <= (void *)&init_thread_union.stack + + sizeof(init_thread_union.stack)); +} + static void print_address_description(void *addr) { struct page *page = addr_to_page(addr); @@ -326,29 +257,38 @@ static void print_shadow_for_address(const void *addr) } } +static bool report_enabled(void) +{ + if (current->kasan_depth) + return false; + if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + return true; + return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); +} + void kasan_report_invalid_free(void *object, unsigned long ip) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); pr_err("\n"); print_address_description(object); pr_err("\n"); print_shadow_for_address(object); - kasan_end_report(&flags); + end_report(&flags); } static void kasan_report_error(struct kasan_access_info *info) { unsigned long flags; - kasan_start_report(&flags); + start_report(&flags); - print_error_description(info); + print_error_description(info, get_bug_type(info)); pr_err("\n"); - if (!addr_has_shadow(info)) { + if (!addr_has_shadow(info->access_addr)) { dump_stack(); } else { print_address_description((void *)info->access_addr); @@ -356,41 +296,7 @@ static void kasan_report_error(struct kasan_access_info *info) print_shadow_for_address(info->first_bad_addr); } - kasan_end_report(&flags); -} - -static unsigned long kasan_flags; - -#define KASAN_BIT_REPORTED 0 -#define KASAN_BIT_MULTI_SHOT 1 - -bool kasan_save_enable_multi_shot(void) -{ - return test_and_set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_save_enable_multi_shot); - -void kasan_restore_multi_shot(bool enabled) -{ - if (!enabled) - clear_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); -} -EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); - -static int __init kasan_set_multi_shot(char *str) -{ - set_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); - return 1; -} -__setup("kasan_multi_shot", kasan_set_multi_shot); - -static inline bool kasan_report_enabled(void) -{ - if (current->kasan_depth) - return false; - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) - return true; - return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); + end_report(&flags); } void kasan_report(unsigned long addr, size_t size, @@ -398,7 +304,7 @@ void kasan_report(unsigned long addr, size_t size, { struct kasan_access_info info; - if (likely(!kasan_report_enabled())) + if (likely(!report_enabled())) return; disable_trace_on_warning(); @@ -411,41 +317,3 @@ void kasan_report(unsigned long addr, size_t size, kasan_report_error(&info); } - - -#define DEFINE_ASAN_REPORT_LOAD(size) \ -void __asan_report_load##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, false, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_load##size##_noabort) - -#define DEFINE_ASAN_REPORT_STORE(size) \ -void __asan_report_store##size##_noabort(unsigned long addr) \ -{ \ - kasan_report(addr, size, true, _RET_IP_); \ -} \ -EXPORT_SYMBOL(__asan_report_store##size##_noabort) - -DEFINE_ASAN_REPORT_LOAD(1); -DEFINE_ASAN_REPORT_LOAD(2); -DEFINE_ASAN_REPORT_LOAD(4); -DEFINE_ASAN_REPORT_LOAD(8); -DEFINE_ASAN_REPORT_LOAD(16); -DEFINE_ASAN_REPORT_STORE(1); -DEFINE_ASAN_REPORT_STORE(2); -DEFINE_ASAN_REPORT_STORE(4); -DEFINE_ASAN_REPORT_STORE(8); -DEFINE_ASAN_REPORT_STORE(16); - -void __asan_report_load_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, false, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_load_n_noabort); - -void __asan_report_store_n_noabort(unsigned long addr, size_t size) -{ - kasan_report(addr, size, true, _RET_IP_); -} -EXPORT_SYMBOL(__asan_report_store_n_noabort); From patchwork Thu Aug 9 19:21:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8FD28174A for ; Thu, 9 Aug 2018 19:22:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D9F22B954 for ; Thu, 9 Aug 2018 19:22:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7AF7D2B938; Thu, 9 Aug 2018 19:22:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A3A6C2B953 for ; Thu, 9 Aug 2018 19:22:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19DA56B0272; Thu, 9 Aug 2018 15:21:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 14B996B0273; Thu, 9 Aug 2018 15:21:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0B706B0274; Thu, 9 Aug 2018 15:21:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by kanga.kvack.org (Postfix) with ESMTP id 886C46B0273 for ; Thu, 9 Aug 2018 15:21:38 -0400 (EDT) Received: by mail-wr1-f71.google.com with SMTP id s14-v6so5277383wra.0 for ; Thu, 09 Aug 2018 12:21:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=9RY22I4ZI+5m17mEdEVvhKO3w9/cLmXBeP5nQv6jvIQ=; b=sPRnaz6EMLHlRQP9cTr3oXz59llOMFmCySm1wKNM19OJlXF2hgHkcJtgCE+hlRzmHA Od2X4GFbVV1umPAXwOy28hEllMifXjDBB9ECy7p5aiWTlVjRrjSterEk/YrowCi+P7WA eerliz0dimjSenUBOEQGrNs1lU0bfyfBavJ/SqHrwC16USV2kqQziYKKvEkNvnZv4wnJ dMICU3D4Gj6pG+NAdpyVlQtnBVwUT8KOsPz7/ZjO0nL4wB2iwnE2TKQ5QdXT/hjg+hm6 eJRSgWPZwrep2JplZVa2nqR0Kb54f3108dclXEIkb1WzAQDvgOgsmXLlkAaHple4ItSf b//w== X-Gm-Message-State: AOUpUlF0OqNFDBF87NCGhCr+dgw/u8QpfzMyy0gXgqMghadnRMfhykxG kkBB9kaya8RxR95/t3YLnCHO7LTU5encdLZJ6A/A6ZWSSIEyA6mvIZUHNbb+87vydVonH4IfqTe lP5pSAhl15YkRRKchihXL2cPRTDMH+vRJz22U6zfNL4CPJx/7Xe582qcODg2QyvzuaMlssXSLU4 J89F+6dkTd7x3OS3QhTNAQ/8Nibqzdqz2P/MTnYt/oH4+80Qh/wmxipSsJ0Ik9SytRFmW4IZrsU rRgveOuasE7Y8oPuCdcZXXIzPHpDwpbVlcEz+gFZPtcsExBXffKMlHn6/OH94zSG8oSVLC1Q4zs Xh7COc4TtjcLLzAmVs60UYY+4q8+Y9xzd35vkwTLYLwDCRiTP9Zr1MFVHFWmQU+nJlP8P86mLRJ X X-Received: by 2002:a5d:4b50:: with SMTP id w16-v6mr2260599wrs.87.1533842498044; Thu, 09 Aug 2018 12:21:38 -0700 (PDT) X-Received: by 2002:a5d:4b50:: with SMTP id w16-v6mr2260568wrs.87.1533842497038; Thu, 09 Aug 2018 12:21:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842497; cv=none; d=google.com; s=arc-20160816; b=WWTvnGFwsWVquGx5935QuEsxoCE5BDFIbT/I4U2jZhrPaonRk0oPTdl3MT8VZ6up2I /qV5R0QoBvQHC4uwEFgp5gtERWDznX6FxRhpi8rnxTNgkXVxFAbFSRyXI7VkxFJ2ftIj NlwsiBJpSLl9uHSBddd+oT3lBakXzpZrITm69AunvcW5BU6ZxQ4aWtudJmr/riJvnpJI AOghdwiYf0SEAPvwhu9yRmnJEaEABJqDUTg23MOanDNS3iK4TJgNtDy+AvMalD/vEVo9 ntjl+JzHLuOnTNCHrQPfeGRXqUXCaRawB2L8rWnMGx+4hepUB1bgNUJ+p3Pt/gwDCCWx b3dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=9RY22I4ZI+5m17mEdEVvhKO3w9/cLmXBeP5nQv6jvIQ=; b=cjkp+/woiGSd+iEHJ18XgJFRbWL95PavwKkUhj704hKZY9+S/wplGNn0XIRdW655fC ldAB05tIipkJDHGEMQw+I9GEu3xLEXjVTGCham9AqBBD9/L1xG5Cta1yAy3fBOGG8dd4 JI2OzJa1BMHbWwLGQTOvGZ0wkDH7xtgoV8Ad12GKFXhWRfcdbwp3dRlZKaLkCAxrZaLU 2IbdJudIJHDaTQ4JQnTwflthN88dsrS2Th3Qr9F6XA/XrJPaf6QDc/ISaTEAvkPei4tw Dr9h0LBWxZttKgo7+BEwyTyc7oNf6jlwmrMpl1GUgaZ+YvUlQjYOGAmdPAL9fuzsPWRD /7MQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sTRmDSAN; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id l1-v6sor3080910wro.55.2018.08.09.12.21.36 for (Google Transport Security); Thu, 09 Aug 2018 12:21:37 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=sTRmDSAN; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9RY22I4ZI+5m17mEdEVvhKO3w9/cLmXBeP5nQv6jvIQ=; b=sTRmDSANNDOTqoYP5c7hwuojmjcx8cT29L6v0RJ6DaoseKM/ooHlJfgDUzg4YbCa3+ Dd9BIRUVqv3y6cbl/Okn9eCKa1/Xt5mQ9rh8mC+4Yun0nDEqoU0uCDe9CtufDKYmYlxk Vr3PHFQ37LRTDhQ+G/FhZt4noLgqIMBUiJLsBl9dLn3xsN+9X7a+6pWRi0i8hpbZ80Gz y0l3BsA0h7QKo6Bt7e8bNnj/5jYdfw5tjeRSaa8QfEAbSJqb8NeDgmgF3xwDO0yR1xca XvfeI3RKR7DKeKWZGuE5QA1ASW7QsOJqT8VzpClC/GoMpd5GKlv53naINvXaBloXvnIP s+0g== X-Google-Smtp-Source: AA+uWPxV30SECaRzqCzmq0wopNycNarF38qQC6VWwQusCSj3keOWN70SGd7W3MeHHH7dlw/Shy9ejA== X-Received: by 2002:adf:c109:: with SMTP id r9-v6mr2381830wre.233.1533842496334; Thu, 09 Aug 2018 12:21:36 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:35 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 13/18] khwasan: add bug reporting routines Date: Thu, 9 Aug 2018 21:21:05 +0200 Message-Id: <8a96159040955e465f937e81de9266e7b55846e4.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds rountines, that print KHWASAN error reports. Those are quite similar to KASAN, the difference is: 1. The way KHWASAN finds the first bad shadow cell (with a mismatching tag). KHWASAN compares memory tags from the shadow memory to the pointer tag. 2. KHWASAN reports all bugs with the "KASAN: invalid-access" header. This is done, so various external tools that already parse the kernel logs looking for KASAN reports wouldn't need to be changed. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 3 +++ mm/kasan/kasan.h | 7 +++++ mm/kasan/kasan_report.c | 7 ++--- mm/kasan/khwasan_report.c | 21 +++++++++++++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ 5 files changed, 64 insertions(+), 31 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 174f762da968..b3d9b27dbb4e 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -163,6 +163,9 @@ void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr); void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, const void *addr); +void kasan_report(unsigned long addr, size_t size, + bool write, unsigned long ip); + #else /* CONFIG_KASAN_HW */ static inline void khwasan_init(void) { } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 82672473740c..d60859d26be7 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -119,8 +119,15 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value); void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip); +void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); +#ifdef CONFIG_KASAN_HW +void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { } +#endif + void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); void kasan_report_invalid_free(void *object, unsigned long ip); diff --git a/mm/kasan/kasan_report.c b/mm/kasan/kasan_report.c index 2d8decbecbd5..fdf2d77e3125 100644 --- a/mm/kasan/kasan_report.c +++ b/mm/kasan/kasan_report.c @@ -33,10 +33,10 @@ #include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr(const void *addr, size_t size) +void *find_first_bad_addr(void *addr, size_t size) { u8 shadow_val = *(u8 *)kasan_mem_to_shadow(addr); - const void *first_bad_addr = addr; + void *first_bad_addr = addr; while (!shadow_val && first_bad_addr < addr + size) { first_bad_addr += KASAN_SHADOW_SCALE_SIZE; @@ -50,9 +50,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info) const char *bug_type = "unknown-crash"; u8 *shadow_addr; - info->first_bad_addr = find_first_bad_addr(info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow(info->first_bad_addr); /* diff --git a/mm/kasan/khwasan_report.c b/mm/kasan/khwasan_report.c index 2edbc3c76be5..51238b404b08 100644 --- a/mm/kasan/khwasan_report.c +++ b/mm/kasan/khwasan_report.c @@ -37,3 +37,24 @@ const char *get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } + +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *untagged_addr = reset_tag(addr); + u8 *shadow = (u8 *)kasan_mem_to_shadow(untagged_addr); + void *first_bad_addr = untagged_addr; + + while (*shadow == tag && first_bad_addr < untagged_addr + size) { + first_bad_addr += KASAN_SHADOW_SCALE_SIZE; + shadow = (u8 *)kasan_mem_to_shadow(first_bad_addr); + } + return first_bad_addr; +} + +void print_tags(u8 addr_tag, const void *addr) +{ + u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); + + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); +} diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 155247a6f8a8..e031c78f2e52 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) } __setup("kasan_multi_shot", kasan_set_multi_shot); -static void print_error_description(struct kasan_access_info *info, - const char *bug_type) +static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - bug_type, (void *)info->ip); + get_bug_type(info), (void *)info->ip); pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, info->access_addr, current->comm, task_pid_nr(current)); @@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip) start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); + print_tags(get_tag(object), reset_tag(object)); + object = reset_tag(object); pr_err("\n"); print_address_description(object); pr_err("\n"); @@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip) end_report(&flags); } -static void kasan_report_error(struct kasan_access_info *info) -{ - unsigned long flags; - - start_report(&flags); - - print_error_description(info, get_bug_type(info)); - pr_err("\n"); - - if (!addr_has_shadow(info->access_addr)) { - dump_stack(); - } else { - print_address_description((void *)info->access_addr); - pr_err("\n"); - print_shadow_for_address(info->first_bad_addr); - } - - end_report(&flags); -} - void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip) { struct kasan_access_info info; + void *tagged_addr; + void *untagged_addr; + unsigned long flags; if (likely(!report_enabled())) return; disable_trace_on_warning(); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr = reset_tag(tagged_addr); + + info.access_addr = tagged_addr; + if (addr_has_shadow(untagged_addr)) + info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; - kasan_report_error(&info); + start_report(&flags); + + print_error_description(&info); + if (addr_has_shadow(untagged_addr)) + print_tags(get_tag(tagged_addr), info.first_bad_addr); + pr_err("\n"); + + if (addr_has_shadow(untagged_addr)) { + print_address_description(untagged_addr); + pr_err("\n"); + print_shadow_for_address(info.first_bad_addr); + } else { + dump_stack(); + } + + end_report(&flags); } From patchwork Thu Aug 9 19:21:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561763 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 452511390 for ; Thu, 9 Aug 2018 19:22:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 343312B93C for ; Thu, 9 Aug 2018 19:22:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 31B552B959; Thu, 9 Aug 2018 19:22:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 42DD02B93C for ; Thu, 9 Aug 2018 19:22:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 149BB6B0273; Thu, 9 Aug 2018 15:21:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0FD8D6B027C; Thu, 9 Aug 2018 15:21:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6BE06B027D; Thu, 9 Aug 2018 15:21:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 664CC6B0273 for ; Thu, 9 Aug 2018 15:21:40 -0400 (EDT) Received: by mail-wr1-f69.google.com with SMTP id r3-v6so5138553wrj.21 for ; Thu, 09 Aug 2018 12:21:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uJ6q8PjPYcBwwEzFrC4nHyiwSQq7tYGhjlgVJlHyoQY=; b=GK9SP3s51wuyNhuSY0ese1gUSzITRNcWsoROW/bZVxBLKvpH51Ewwi4qcl/jZJO5sv XjphAE8lFE+OSS+5U68MHBoZAgF7i9Yd/UnaRJ4zYrFYSHdiEfPAfpg+tJUcZS341Upa tuawLlmPh+pJi+HWfr0yegii+5StlzajsPUpx58pabVZpE0KYmuz2BDFwEqVqsHqL2J7 ZJ+adT+FEawu3Rw1m89srJ52BRhdORp9NkrCWODdTF+aLQ/hSONkVALhV8c1mgc0SeYU vuEH6ZyBctYHWUJj5cSQSnxtD2WRwI0YDGxPDDWRfWE5t5uNFHUw61x/YOX7x5I4vAat oP6A== X-Gm-Message-State: AOUpUlFwnEizNpfJ8y2Z+OQu2rA0VgLFeSe9z2fASfpZ09Tpl+RoVpua L7N9HqL9YzuWacWL+OcjMVdBsSLZt+8C1f6or7/b2m/sqXnq5BsWT8+bRQ0cZ2TvV+ck0P7CBio OKc6eM2EEhpaxgL6C4aQTNYYT+4YWb9ERtqzbMCTbfJ3pRMR0u0rsTr02Q4k+oPNSTrpqdEbcf0 44TnUiD2Y5GxMx5w9+hjzhV445uh1dZt67U3OVQTKuSQQ6zTTupi7k6rm76HGaHSLNyiqOHsMj5 DIfhEcYmuqWsGrGOASLHv/b6yxloG7lycvD1duQa0FiFZVdIt5Zrpcfdpd0HWav9u11GPnrM7Jp 3ZDRKJEGNxtEEbDY8oDvxU8bibKSzvhWydpQrLUA9PHIzDgdekQqJ9H/ZttemyaaNA2WOWK4yJG W X-Received: by 2002:a5d:550a:: with SMTP id b10-v6mr2330543wrv.228.1533842499886; Thu, 09 Aug 2018 12:21:39 -0700 (PDT) X-Received: by 2002:a5d:550a:: with SMTP id b10-v6mr2330501wrv.228.1533842498721; Thu, 09 Aug 2018 12:21:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842498; cv=none; d=google.com; s=arc-20160816; b=UQY9ubAk8Xi0wEUuX3sT4Eho60hnCkAaYTzXB8yOSCko2haRoe+O+8frpCEpETWFC7 iUGa1bsWgKWQVhh7MGuiLWjViFqU48fUd4EW3WAgfxE55Ie6QjXMqe4CA2PMBM9fbZa9 +UYkpDVs7Nl5T/Wt7q/MprDYsVKCSLK9avvk2HuL/DD77EdFyt3H+wMw2iX5va0jzfrK qWM4xbKU8J5d8QT/gMNu6lIHz7oBkMV+v03PL69JR3RDWsyrwX5Dt6m1KWkuy9jbrk5q xJ1mQTnXcICa5L93cI8uuVwVquz3jVQd1bgBaEanvloILNRNrEym2KUSxiqCbms5Wc+5 5/DQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=uJ6q8PjPYcBwwEzFrC4nHyiwSQq7tYGhjlgVJlHyoQY=; b=AbAEcUwMNZFllYjFFfwjCgnqR7SM5s7xia2YmEdGZOz3kywFnEscNZKShWFyKu/KHg RxAqGtM8nfSBsPb8Es7xIF+JhJrgqgpO2s2iTxQ2bLPrEYIe9FWE8L9t1YpCIXjFoxyD 3G7TyEiddxlFW1sqFRUHnjVGS8FPLGvKAxnKPlquBtawHYrUAqG7TXG+CeVWDDIMUuFW MD/vy2dg3RnlsQ5BR1wvAV/eRpPH6fJblHseiLpc/TBW4vP7dGFTzW0q3Hnpq/JNGa0m g6haEtfjNSCFD64rlz/ACHnbT8EQ69N8jHmAPB8BCsPA08/eqyP40Msl/P+8jrs0UQQt RC1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=qu6lSut+; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id w13-v6sor3183618wrl.31.2018.08.09.12.21.38 for (Google Transport Security); Thu, 09 Aug 2018 12:21:38 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=qu6lSut+; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uJ6q8PjPYcBwwEzFrC4nHyiwSQq7tYGhjlgVJlHyoQY=; b=qu6lSut+Tppg+jb4FypHvldSjPebTikzUuxLhQxjJWN+Rzv5jjaLu+4sLWEumdtMuo HYYljfxmbqPexcU05ecpIHnxaKv1B+VmTXv1URG97dq4ac5aFWD0Ak3hZZ5T2S3k8oGK R5QjT+nNQpVZzj2a9pgnletyBN0Vp6jZrfpS6zR1/ZoNq9dptRMR0f7H7Gh9z8iDXTem vjmV9QVZ1OpZcafwI9f2N5sgTLginRccYjSno5Emsu5/XOzwkZ2FvLYXtXgb4yQW7XUl qNqSyYoykgM3sEZsRVCpfvpbYGNH5R7MO6nNfA/XVKLFOdXzwIkVZ1raXIzeH+x+KyIn ezLw== X-Google-Smtp-Source: AA+uWPzhM4NLx/ciSlWc45Gb41TTLEfQblnVo/0/nPYFyHekUHUJCDN42hqwXxXnom/Kul8tfyXEvA== X-Received: by 2002:adf:fd12:: with SMTP id e18-v6mr2142157wrr.280.1533842497944; Thu, 09 Aug 2018 12:21:37 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:37 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 14/18] khwasan: add hooks implementation Date: Thu, 9 Aug 2018 21:21:06 +0200 Message-Id: <80bdd0848efe56424cd70258d8f7c1b271ad2e22.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This commit adds KHWASAN specific hooks implementation and adjusts common KASAN and KHWASAN ones. 1. When a new slab cache is created, KHWASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16). 2. On each kmalloc KHWASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer. 3. On each kfree KHWASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs. The rest of the logic of the hook implementation is very much similar to the one provided by KASAN. KHWASAN saves allocation and free stack metadata to the slab object the same was KASAN does this. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 82 +++++++++++++++++++++++++++++++++++----------- mm/kasan/kasan.h | 8 +++++ mm/kasan/khwasan.c | 40 ++++++++++++++++++++++ 3 files changed, 111 insertions(+), 19 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index bed8e13c6e1d..938229b26f3a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -140,6 +140,9 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; + /* Perform shadow offset calculation based on untagged address */ + address = reset_tag(address); + shadow_start = kasan_mem_to_shadow(address); shadow_end = kasan_mem_to_shadow(address + size); @@ -148,11 +151,20 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) void kasan_unpoison_shadow(const void *address, size_t size) { - kasan_poison_shadow(address, size, 0); + u8 tag = get_tag(address); + + /* Perform shadow offset calculation based on untagged address */ + address = reset_tag(address); + + kasan_poison_shadow(address, size, tag); if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); - *shadow = size & KASAN_SHADOW_MASK; + + if (IS_ENABLED(CONFIG_KASAN_HW)) + *shadow = tag; + else + *shadow = size & KASAN_SHADOW_MASK; } } @@ -200,8 +212,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { - if (likely(!PageHighMem(page))) - kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); + if (unlikely(PageHighMem(page))) + return; + kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } void kasan_free_pages(struct page *page, unsigned int order) @@ -235,6 +248,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { unsigned int orig_size = *size; + unsigned int redzone_size = 0; int redzone_adjust; /* Add alloc meta. */ @@ -242,20 +256,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size += sizeof(struct kasan_alloc_meta); /* Add free meta. */ - if (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); } - redzone_adjust = optimal_redzone(cache->object_size) - - (*size - cache->object_size); + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); if (redzone_adjust > 0) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + - optimal_redzone(cache->object_size))); + max(*size, cache->object_size + redzone_size)); /* * If the metadata doesn't fit, don't enable KASAN at all. @@ -268,6 +282,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, return; } + cache->align = round_up(cache->align, KASAN_SHADOW_SCALE_SIZE); + *flags |= SLAB_KASAN; } @@ -328,15 +344,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) return kasan_kmalloc(cache, object, cache->object_size, flags); } +static inline bool shadow_invalid(u8 tag, s8 shadow_byte) +{ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return shadow_byte < 0 || + shadow_byte >= KASAN_SHADOW_SCALE_SIZE; + else + return tag != (u8)shadow_byte; +} + static bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { s8 shadow_byte; + u8 tag; + void *tagged_object; unsigned long rounded_up_size; + tag = get_tag(object); + tagged_object = object; + object = reset_tag(object); + if (unlikely(nearest_obj(cache, virt_to_head_page(object), object) != object)) { - kasan_report_invalid_free(object, ip); + kasan_report_invalid_free(tagged_object, ip); return true; } @@ -345,20 +376,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return false; shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(object)); - if (shadow_byte < 0 || shadow_byte >= KASAN_SHADOW_SCALE_SIZE) { - kasan_report_invalid_free(object, ip); + if (shadow_invalid(tag, shadow_byte)) { + kasan_report_invalid_free(tagged_object, ip); return true; } rounded_up_size = round_up(cache->object_size, KASAN_SHADOW_SCALE_SIZE); kasan_poison_shadow(object, rounded_up_size, KASAN_KMALLOC_FREE); - if (!quarantine || unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) return false; set_track(&get_alloc_info(cache, object)->free_track, GFP_NOWAIT); quarantine_put(get_free_info(cache, object), cache); - return true; + + return IS_ENABLED(CONFIG_KASAN_GENERIC); } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -371,6 +404,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, { unsigned long redzone_start; unsigned long redzone_end; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -383,14 +417,24 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_SHADOW_SCALE_SIZE); - kasan_unpoison_shadow(object, size); + /* + * Objects with contructors and objects from SLAB_TYPESAFE_BY_RCU slabs + * have tags preassigned and are already tagged. + */ + if (IS_ENABLED(CONFIG_KASAN_HW) && + (cache->ctor || cache->flags & SLAB_TYPESAFE_BY_RCU)) + tag = get_tag(object); + else + tag = random_tag(); + + kasan_unpoison_shadow(set_tag(object, tag), size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); - return (void *)object; + return set_tag(object, tag); } EXPORT_SYMBOL(kasan_kmalloc); @@ -440,7 +484,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { + if (reset_tag(ptr) != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -453,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (ptr != page_address(virt_to_head_page(ptr))) + if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d60859d26be7..6f4f2ebf5f57 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,10 +12,18 @@ #define KHWASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KHWASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KHWASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KHWASAN_TAG_INVALID +#endif + #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ /* diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c index 9d91bf3c8246..6b1309278e39 100644 --- a/mm/kasan/khwasan.c +++ b/mm/kasan/khwasan.c @@ -106,15 +106,52 @@ void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, void check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { + u8 tag; + u8 *shadow_first, *shadow_last, *shadow; + void *untagged_addr; + + tag = get_tag((const void *)addr); + + /* Ignore accesses for pointers tagged with 0xff (native kernel + * pointer tag) to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KHWASAN_TAG_KERNEL (0xFF). + */ + if (tag == KHWASAN_TAG_KERNEL) + return; + + untagged_addr = reset_tag((const void *)addr); + shadow_first = kasan_mem_to_shadow(untagged_addr); + shadow_last = kasan_mem_to_shadow(untagged_addr + size - 1); + + for (shadow = shadow_first; shadow <= shadow_last; shadow++) { + if (*shadow != tag) { + kasan_report(addr, size, write, ret_ip); + return; + } + } } #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -126,15 +163,18 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { + check_memory_region(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { + kasan_poison_shadow((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); From patchwork Thu Aug 9 19:21:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561767 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B35721057 for ; Thu, 9 Aug 2018 19:22:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A3B8D2B957 for ; Thu, 9 Aug 2018 19:22:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A152D2B95B; Thu, 9 Aug 2018 19:22:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 219CC2B94E for ; Thu, 9 Aug 2018 19:22:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93A916B027C; Thu, 9 Aug 2018 15:21:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 875CF6B027E; Thu, 9 Aug 2018 15:21:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6EF786B027F; Thu, 9 Aug 2018 15:21:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by kanga.kvack.org (Postfix) with ESMTP id 0EAB96B027C for ; Thu, 9 Aug 2018 15:21:42 -0400 (EDT) Received: by mail-wr1-f70.google.com with SMTP id z13-v6so5279316wrq.3 for ; Thu, 09 Aug 2018 12:21:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=twF6sNhTtfvVSGfxuaySSxIXOdHaX4FE6UrAMHW8nGM=; b=lid+AUtffea33WHsLW8fJjtjYsnnoijYzL7cAWfcFdO0x7lWnrq8PQPBJjzcaCnx2m d7Ar7/UAIWmCvsG60cIGiWeIJcCSuOMuaKQbfdOjoVjMpgKB1ktfpfYGTIVeinkKf3/F WujidoPM33dW4+FoHzdJf6XdK/w0nfu+cEpwWQ84uY8GOVIz05p0jF2+pfcIeXrUWm5h EiSsLb8ByJlTjGIMbgVcJstW413+pKVXU8lHFFYX/7/1l/LlJ5dllugeIsSqdB2pEGoc nBi11dd61EWucWgrZwvGnyDPUScTGRY959YlhA+ukw18t7iAwQ3CbmmYk42yogWZquK6 +vJw== X-Gm-Message-State: AOUpUlH29UMHMCulIjZ7wUuGuHE091aJqb73akV1x76ZJuZdQOwdc8uL xm90mGfbwPUiGN7U3akF5pIeghHujyoRTlEswKFk/x1q3Qh5ukvbvzt0KZPs6rv14CaoWeVBTdf sOzIUXHa2IpzkGqg2e9bJoPwsX3ZCqOND1HiS5YQEQUEDL2hjFKUonrlVpVWf8UqVNgV4KTsCjq p+S+EhtZO97s/Ax//8SHqiGsJljRorsRQX41DdTr3yfF9R3wbR1qggKnmbc2cWbvD7ebPbaEni2 vqvfHD9Rr5IHvlzaJqaPvxhJxM3o3mpU4Ir2p7gVOUL1IiXWgRW0plod8KO7AzaTzds0bot6XKe U+WDwLaOt8WXusHvagWa1YT8llXswdaeMZe7okH/aKO2dA01qBluA41LMblomSZw5OpwseoqHLr D X-Received: by 2002:a1c:9e89:: with SMTP id h131-v6mr2318438wme.13.1533842501525; Thu, 09 Aug 2018 12:21:41 -0700 (PDT) X-Received: by 2002:a1c:9e89:: with SMTP id h131-v6mr2318400wme.13.1533842500423; Thu, 09 Aug 2018 12:21:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842500; cv=none; d=google.com; s=arc-20160816; b=crD3dAHO2lq5SKsqhRPsJjtr6zOVCY+E4M3xzDuIu4bQV1Zzhw+UWZbH30Xh1CySjF shz/v0vLkAVu/S7SBPBO3VYZh2FP8sacRDhhNbpdhhyIbmEYfoWcgq96Ql81otg1E7Od GxmcSSQzGRrh4hjaKmeoBNLiFug3XfjgOq3jDfsltCRYXDufbiDhZVpjmSf1f5j0tgpY AoXL4VqCACuZFKa9K6ZCGz6HoEgLkdkPqbEHo/5+TaU4Y5kbyOvwONiG7vqrPr+l+Hn9 ITLGytJiYZT5UQcMOmK0D5ePVLx40yYttv1sJINtdWjTf7BdJpBEFHNnUG0bwSDWDjW8 DENA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=twF6sNhTtfvVSGfxuaySSxIXOdHaX4FE6UrAMHW8nGM=; b=zSKzRWxgh/bQaPvu7LhIu8z/jz9Mntc2JZSxK9NwArd2Jh1m++gTb7DzV8YiHiXKdC v2FaTZpyx1RBScjLgNOQOwSS6KNXvkNsKtMqp2Vn2Le6sCo39cyGdkHB+fr27/ZBXsyR LdEN23HN8WUteA3Bcyie16ci8K2L9OBsQRFT6PRT/oWoR3koi71RERkESt7uqF7VjSOi jv8wnc0MitFubTM8VJPeGoJodMfXWvZUxldaCxvKKsKHo/6qbYY+SyFqQnhzsldH7t9c Nf21zDPWA6/iRYTTWb0iik6xj/2fv2CHWpJrU5HBL6YAEaYj2/mvfAStzh5lnJEsUEY8 uOGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EC41x2Jj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id g9-v6sor3089678wrw.42.2018.08.09.12.21.40 for (Google Transport Security); Thu, 09 Aug 2018 12:21:40 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=EC41x2Jj; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=twF6sNhTtfvVSGfxuaySSxIXOdHaX4FE6UrAMHW8nGM=; b=EC41x2JjiJzik3Wnv94TlySKVymUooeutcZL8WXdifFiDQVmHKUbYlQDL6Ej2ctx6E oHRBUG14qjk0cxdUsBg+Ir7AnWicRNPrnDNrGfRfEoIgnuwlB5Yy3G+cDOc8HmETBTBV ZcB8Ms4y6tbKEv/RNXoLMJvpHZZmxsFMt8eedataLqmQMLur8uh7rV/6JIAXOs9KH1hM 5mLbypAprgTDjq5AREvz9zOoNJ1c2sy08Hn5HQE37W6BeWTeFsdfeXLuB57wKly0f1Ms HnnCp6n4UA/+5EMcKVUAmPEWCKj04JJ8/a0FTvYGFFoQfXzF9dwQ36ZdGzLUosq0tGg3 mHUw== X-Google-Smtp-Source: AA+uWPyRQtYXfr8FUdpGU+RB5iX5B4yuhSfTZOkAZZey5WmW6ezBoCWEkP49Bu5n5WocQNV2IKBARA== X-Received: by 2002:adf:d207:: with SMTP id g7-v6mr2297275wri.91.1533842499723; Thu, 09 Aug 2018 12:21:39 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:38 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 15/18] khwasan, arm64: add brk handler for inline instrumentation Date: Thu, 9 Aug 2018 21:21:07 +0200 Message-Id: <8f925694d0c885413d38e4fc79a39726d4281a3d.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KHWASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit add a KHWASAN brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 69 +++++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/brk-imm.h b/arch/arm64/include/asm/brk-imm.h index ed693c5bcec0..e4a7013321dc 100644 --- a/arch/arm64/include/asm/brk-imm.h +++ b/arch/arm64/include/asm/brk-imm.h @@ -16,10 +16,12 @@ * 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps + * 0x9xx: KHWASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100 #define KGDB_DYN_DBG_BRK_IMM 0x400 #define KGDB_COMPILED_DBG_BRK_IMM 0x401 #define BUG_BRK_IMM 0x800 +#define KHWASAN_BRK_IMM 0x900 #endif diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index d399d459397b..95152a4fd202 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -269,10 +270,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +void __arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction(struct pt_regs *regs, unsigned long size) +{ + __arm64_skip_faulting_instruction(regs, size); /* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -791,7 +796,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) } /* If thread survives, skip over the BUG instruction and continue: */ - arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); return DBG_HOOK_HANDLED; } @@ -801,6 +806,59 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_ESR_RECOVER 0x20 +#define KHWASAN_ESR_WRITE 0x10 +#define KHWASAN_ESR_SIZE_MASK 0x0f +#define KHWASAN_ESR_SIZE(esr) (1 << ((esr) & KHWASAN_ESR_SIZE_MASK)) + +static int khwasan_handler(struct pt_regs *regs, unsigned int esr) +{ + bool recover = esr & KHWASAN_ESR_RECOVER; + bool write = esr & KHWASAN_ESR_WRITE; + size_t size = KHWASAN_ESR_SIZE(esr); + u64 addr = regs->regs[0]; + u64 pc = regs->pc; + + if (user_mode(regs)) + return DBG_HOOK_ERROR; + + kasan_report(addr, size, write, pc); + + /* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KHWASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; same is true for KASAN; + * this is controlled by current->kasan_depth). All these accesses are + * detected by the tool, even though the reports for them are not + * printed. + * + * This is something that might be fixed at some point in the future. + */ + if (!recover) + die("Oops - KHWASAN", regs, 0); + + /* If thread survives, skip over the brk instruction and continue: */ + __arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); + return DBG_HOOK_HANDLED; +} + +#define KHWASAN_ESR_VAL (0xf2000000 | KHWASAN_BRK_IMM) +#define KHWASAN_ESR_MASK 0xffffff00 + +static struct break_hook khwasan_break_hook = { + .esr_val = KHWASAN_ESR_VAL, + .esr_mask = KHWASAN_ESR_MASK, + .fn = khwasan_handler, +}; +#endif + /* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -808,6 +866,10 @@ static struct break_hook bug_break_hook = { int __init early_brk64(unsigned long addr, unsigned int esr, struct pt_regs *regs) { +#ifdef CONFIG_KASAN_HW + if ((esr & KHWASAN_ESR_MASK) == KHWASAN_ESR_VAL) + return khwasan_handler(regs, esr) != DBG_HOOK_HANDLED; +#endif return bug_handler(regs, esr) != DBG_HOOK_HANDLED; } @@ -815,4 +877,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr, void __init trap_init(void) { register_break_hook(&bug_break_hook); +#ifdef CONFIG_KASAN_HW + register_break_hook(&khwasan_break_hook); +#endif } From patchwork Thu Aug 9 19:21:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561771 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C1B01390 for ; Thu, 9 Aug 2018 19:22:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B3252B94F for ; Thu, 9 Aug 2018 19:22:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 290112B951; Thu, 9 Aug 2018 19:22:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7DD92B94F for ; Thu, 9 Aug 2018 19:22:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62FA76B027E; Thu, 9 Aug 2018 15:21:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 593C06B0280; Thu, 9 Aug 2018 15:21:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20C796B0281; Thu, 9 Aug 2018 15:21:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f72.google.com (mail-wm0-f72.google.com [74.125.82.72]) by kanga.kvack.org (Postfix) with ESMTP id B79886B027E for ; Thu, 9 Aug 2018 15:21:43 -0400 (EDT) Received: by mail-wm0-f72.google.com with SMTP id g25-v6so634432wmh.6 for ; Thu, 09 Aug 2018 12:21:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=yUj4eRjlKSKpRoNwumQrRbs76avC08e5DqBPdbUwBV4=; b=GrzJo4JQrAJPOGRhMCLQ7g6LAxW7OGmQloDBCsUwi0HS/2kxsqg+7ID4obmY1KgtC/ YFqY+kDXDanH+nzSIaKaBSISjOsoq7ph94T5ufyQQj4KNSuDRfz0N42HYnyuBuauFy+P fxyyK4FVsQcAuF1DMBkBPvWjE6OqPEm0RqKcG0ozBey/PGCHToy48uc/So9XdY+FIscI TkGXGBeYyJjhKnPSk2ThD4ONukK7wMZj0o07QHKk6vd/kFhJwt2kf9xlKpAJcKkHfwtT q70aDzHYMswBgMatY4kBdobH9hvJoAU8lEybedvuzQlqDsrU8tQqhWD1yerG9nmgjw1L 3j6g== X-Gm-Message-State: AOUpUlGQw3/g+6cQT0sILY582sHQflN8A2NSdSS37Q/FsSW3p6DuKx0b VFjIW3LKNSnAww3fXjU/JNm4vXVWGSqLqfIDZQZO7AFRHqkC2yqCdj3melqQ1gn+4YuuHgoeLUl idQ/d3niB5azX/JlJCC0qIz/M5vVZhWztU7RUVgnWRB4xM/5p6DGLzpkba5WGFMQ7W4Qa90sWPJ e+U+aex6f4stQT2lLT66z8oyRmTkv2e62MZyoVlDfRvYfUI8/n4s2tE7Enaq6EDobUYfX7hz5mC udNUznz5tzIdQtrXGB/fGSr1GWfP+GZwMyFdAloW/iXaY/vnLZkpABNeVVijE3RclhyASLzAirR Nt0hfuQPA+JuirOhIFmVPGeXvaGcHfnnWfkJmrZ4m5qsz4TlEVbru0KJcoATcAtzGg0u8B3wd6R c X-Received: by 2002:a1c:888e:: with SMTP id k136-v6mr2494191wmd.6.1533842503224; Thu, 09 Aug 2018 12:21:43 -0700 (PDT) X-Received: by 2002:a1c:888e:: with SMTP id k136-v6mr2494162wmd.6.1533842502128; Thu, 09 Aug 2018 12:21:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842502; cv=none; d=google.com; s=arc-20160816; b=UGr+HEIOXVR9IzhYNjlCr41RmYomTM2vQnH+PhsuTIlbiWLteoYp+z50Kh1QUt1Jam 7HIY7Mk/M6Y2gj7yM0+RIdAJXlan+pxqoYFkUGySG0Q2bzKeUwtakntxNIp9mtIhymA1 YZnK7VQp6EghS3V2+elYjfBA4JJiryBGtFVwne8mGkRgn+MotqQVL/LQTk/i9I7JJXS9 AR/Ems7DpDncOprf6xbwYztT3vGOtcDZCEDepwo+BM/tKcqv29AT4No5HIzSQOdiOiWH z5ZCT2y7bO96XgjsK3F5swoSxzEiQys4wquDBaEOvAPSlEAeS6l6qSVEruOX/diXgMHa ySHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=yUj4eRjlKSKpRoNwumQrRbs76avC08e5DqBPdbUwBV4=; b=jmjwZ1IwYe46KFoNFmlly9f//LJlXJ638QsFL4MbPALlkwn/dvjZhP7UiK3xLDzReD lg/UpqzXq7T5xxGpqsyqycelecB4RMEKSoxYbAs0Z3kJVGeeg3GGZDUypSNnGF6mhlDq JQmogvMwJ7LUIrXvtbE4FliCcT5k76XLKHuvAuz3cVxiTPpwa7pEIk4nFrzdkv4mbxpV xq8Q4IdOXeFrgG7eeOuutotYva//FKo649MPOK0hb6GsNdh1sf21p8NebN9BvM9eRGzm rqwIrymbhUbWasTxOAYw9sdYJ1rtmGwJIBwACDQbc9LeFXzii5rkvroXJ5h/6AesUjX7 Y8OA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=IVDcrbpu; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id f193-v6sor2128308wme.75.2018.08.09.12.21.41 for (Google Transport Security); Thu, 09 Aug 2018 12:21:42 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=IVDcrbpu; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yUj4eRjlKSKpRoNwumQrRbs76avC08e5DqBPdbUwBV4=; b=IVDcrbpuPTAlzOG+R76c5T0ml0rQXLOlcOc7dvoXAfLITfcnldny25bYSOukbCzgDq xg/cwZ1YQpqw42MWs9jzjSVydZNARxY4fzNopkev9C4DHp+EskRQqqfCVn3HNmeAqX+l Xr0uIaoex/QpmU+dd5VCIFxZg+mDW07Q3AXP5QLSBMZyiaMB6HXmdMY+NDxMaA1O8iDl W2Hnxq/MRZMNmr3NUVy2Kz2dJxy3cpGy8NghttG1C1rtMCAmKh4ma152HjG1U1kfV/yG PAo3/jps//v5MutE/sbKBjbqXgkAFGBqXUNz7WrDPu1jDdFFMVVtKmI8y4ecGem0PBHZ prng== X-Google-Smtp-Source: AA+uWPx/8/SuP0Ax4R4yOG1Sv+9SnSJm8yIgduGT4XZSjmNkEcW0mn8MZwOUDffvSQPxwSZk7Nvmdw== X-Received: by 2002:a1c:e189:: with SMTP id y131-v6mr2390637wmg.44.1533842501368; Thu, 09 Aug 2018 12:21:41 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:40 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 16/18] khwasan, mm, arm64: tag non slab memory allocated via pagealloc Date: Thu, 9 Aug 2018 21:21:08 +0200 Message-Id: <1e7aea30d46f0dd32fc9880d1f70691f8ed5b3c6.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KWHASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently. For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag. This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated. Signed-off-by: Andrey Konovalov --- arch/arm64/include/asm/memory.h | 10 ++++++++++ include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 17 +++++++++++++++-- mm/page_alloc.c | 1 + 6 files changed, 76 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index e9e054dfb1fc..3352a65b8312 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -305,7 +305,17 @@ static inline void *phys_to_virt(phys_addr_t x) #define __virt_to_pgoff(kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page)) #define __page_to_voff(kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) +#ifndef CONFIG_KASAN_HW #define page_to_virt(page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#else +#define page_to_virt(page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = KASAN_SET_TAG(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) +#endif + #define virt_to_page(vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START)) #define _virt_addr_valid(kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \ diff --git a/include/linux/mm.h b/include/linux/mm.h index 68a5121694ef..1c6c04466374 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -803,6 +803,7 @@ int finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH) #define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH) #define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH) /* * Define the bit shifts to access each section. For non-existent @@ -813,6 +814,7 @@ int finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0)) #define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0)) #define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0)) /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS @@ -835,6 +837,7 @@ int finish_mkwrite_fault(struct vm_fault *vmf); #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) static inline enum zone_type page_zonenum(const struct page *page) @@ -1089,6 +1092,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) } #endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_HW +static inline u8 page_kasan_tag(const struct page *page) +{ + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; +} + +static inline void page_kasan_tag_reset(struct page *page) +{ + page_kasan_tag_set(page, 0xff); +} +#else +static inline u8 page_kasan_tag(const struct page *page) +{ + return 0xff; +} + +static inline void page_kasan_tag_set(struct page *page, u8 tag) { } +static inline void page_kasan_tag_reset(struct page *page) { } +#endif + static inline struct zone *page_zone(const struct page *page) { return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)]; diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-layout.h index 7ec86bf31ce4..8dbad17664c2 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -82,6 +82,16 @@ #define LAST_CPUPID_WIDTH 0 #endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH+LAST_CPUPID_WIDTH+KASAN_TAG_WIDTH \ + > BITS_PER_LONG - NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif + /* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit. diff --git a/mm/cma.c b/mm/cma.c index 5809bbe360d7..fdad7ad0d9c4 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, unsigned long pfn = -1; unsigned long start = 0; unsigned long bitmap_maxno, bitmap_no, bitmap_count; + size_t i; struct page *page = NULL; int ret = -ENOMEM; @@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align, trace_cma_alloc(pfn, page, count, align); + /* + * CMA can allocate multiple page blocks, which results in different + * blocks being marked with different tags. Reset the tags to ignore + * those page blocks. + */ + if (page) { + for (i = 0; i < count; i++) + page_kasan_tag_reset(page + i); + } + if (ret && !(gfp_mask & __GFP_NOWARN)) { pr_err("%s: alloc failed, req-size: %zu pages, ret: %d\n", __func__, count, ret); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 938229b26f3a..e5648f4218eb 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -212,8 +212,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark) void kasan_alloc_pages(struct page *page, unsigned int order) { + u8 tag; + unsigned long i; + if (unlikely(PageHighMem(page))) return; + + tag = random_tag(); + for (i = 0; i < (1 << order); i++) + page_kasan_tag_set(page + i, tag); kasan_unpoison_shadow(page_address(page), PAGE_SIZE << order); } @@ -311,6 +318,12 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, void kasan_poison_slab(struct page *page) { + unsigned long i; + + if (IS_ENABLED(CONFIG_SLAB)) + page->s_mem = reset_tag(page->s_mem); + for (i = 0; i < (1 << compound_order(page)); i++) + page_kasan_tag_reset(page + i); kasan_poison_shadow(page_address(page), PAGE_SIZE << compound_order(page), KASAN_KMALLOC_REDZONE); @@ -484,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page = virt_to_head_page(ptr); if (unlikely(!PageSlab(page))) { - if (reset_tag(ptr) != page_address(page)) { + if (ptr != page_address(page)) { kasan_report_invalid_free(ptr, ip); return; } @@ -497,7 +510,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) void kasan_kfree_large(void *ptr, unsigned long ip) { - if (reset_tag(ptr) != page_address(virt_to_head_page(ptr))) + if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by page_alloc. */ } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a790ef4be74e..c0490aaf7765 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1176,6 +1176,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, init_page_count(page); page_mapcount_reset(page); page_cpupid_reset_last(page); + page_kasan_tag_reset(page); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL From patchwork Thu Aug 9 19:21:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561775 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C379D1057 for ; Thu, 9 Aug 2018 19:22:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0AC22B913 for ; Thu, 9 Aug 2018 19:22:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AE1EF2B946; Thu, 9 Aug 2018 19:22:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9EDEB2B95D for ; Thu, 9 Aug 2018 19:22:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6D876B0280; Thu, 9 Aug 2018 15:21:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C1E9F6B0283; Thu, 9 Aug 2018 15:21:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A983C6B0284; Thu, 9 Aug 2018 15:21:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id 31CEC6B0280 for ; Thu, 9 Aug 2018 15:21:46 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id p3-v6so630164wmc.7 for ; Thu, 09 Aug 2018 12:21:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KhKme33qvU+GZexOYj4wNnWuiTQyS9w0/hdMlUxENuM=; b=kmC4yH40sJfXrmbJ4Xu1s5N7XcysNOhTDRJmhF7Jwt9MawTaUY8IQujeNBkZ8OLLco c1t6IY6WYw1UZ/WiJofbb2NCN/EzwzckxO8p/lU2tY8lkPJ3XlvBL2Ks1FdFCQa2yp/1 rCmuQOR3WrxwJMKIOPUFgoy1Oz6sN99Vn6cXlOZN+zo8i7VM6g2aHMKssewOeZQtp49g Vv/o03T/RlUm2gClkYKvUdeKzORX0vZAOv0HBh0eATc7NymO23CJmeEJBNgydtB5UL4t lvaiUHLmq8JszczZWzVVzxcE1BCy5DaOsUkNubS9dvpCx8KKhpCPxW9KW08wXrV31bX4 mCbQ== X-Gm-Message-State: AOUpUlH/y1RmnbNy4hKGp/LBEKZsTC5zsgGGXPyYiJ/hfQ1G65rAPHSS 6XND9DpwaZEMWSVW/b/3/PgQ2pF7PtQO+rZ3qeRG6zQWESSGCbgwL7j3G0hoX2GW9SDT763UEKW iHdFf7Jz+s3EtQ3eWQw2gKNlIASAAviBbO1YiULPL9PBdzqt2I8BZ8Prh7j7MZspyB1x12Uhin0 wh0+jNBsMTxluD3EeTUDxWEtGB6Ftf88WgeoR/as3+lGRhW5639wSHTPMOMnqLceWPAz7vhyrge iMD9jZVfA2hSCGVTy46CjRnaliLGh3Fx3+PJMLubKcWpdOAorZWwcIkD8HgMQaVE0qU79bPmD6m Cnatas2wLbAOEiL8pRyUuZ8uNtusk6ioktmHPecvXB7JVYr7iCE1eJgVYYEI2j8Wd7tUmgEDsat x X-Received: by 2002:a1c:3a8f:: with SMTP id h137-v6mr2609962wma.41.1533842505652; Thu, 09 Aug 2018 12:21:45 -0700 (PDT) X-Received: by 2002:a1c:3a8f:: with SMTP id h137-v6mr2609884wma.41.1533842504003; Thu, 09 Aug 2018 12:21:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842503; cv=none; d=google.com; s=arc-20160816; b=JlEUF/rEWg3C08+5f0Jr2rmpqfTRGQC2LX53KgTjkeGm8k9V2wYC13NPJwu8bemKfE AX+2YY/OY/sGJMGXa1HGJ4w2cZECJBdP0Aw0dDIAVOTJjZJoJ6kFp1pRLxYg83F5azoc F66QRV5mEPCG24Ar7IRV7AyknD18tEo3GQkVC1ec+kvLtpL4cOklDeHGrR2sBMq1c/xd DreWgqyy9VKsRWOfrYg5EK2k4ulYBB9Hq722TEMwliQWNNKXlO96c8BIOsvP7SakuXzW VUGoVT78aqxRxJ01vc8/xNRFMT3RyPPLal1dY+94KdawJGdbQlCvMmKaTDAyUNputn72 1mFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=KhKme33qvU+GZexOYj4wNnWuiTQyS9w0/hdMlUxENuM=; b=e1s6mwjsgLJjOd0N2UyAapcfl3qLVHDKMeY9SOMciueoBqteCTM6myEXdKvmc+Qxzu EOpSWHG/nqInIi1iKnqrxZEV4H59ouiqvDkwJsyobVeBS8JDYAZZtbCn6LS6eWd1SRQB 4X3Qnyf1+FTRJOb94tv+8ep3j78mAHxRYCwKKir5+e6geU5hUNm33lzqaIdKKZ8mLfV1 RFw+Tyub2MW4/RSi/5ygjV2sOx8IMwnqvQ/roza6nb/3X8P6YWO0tQpVdIHh3xoExd9k PxoTqYYXpM6es7F2XhdyJc1pvOjKn9QxAh6uJZ+PeUNbKQx5YILkMTIyoCGJMaFRUEd7 p6Kw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=cCrmbQ8g; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id e15-v6sor3128773wrn.18.2018.08.09.12.21.43 for (Google Transport Security); Thu, 09 Aug 2018 12:21:43 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=cCrmbQ8g; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KhKme33qvU+GZexOYj4wNnWuiTQyS9w0/hdMlUxENuM=; b=cCrmbQ8gHab6+bw0JjrURF7s7PSFZNwECp1m8nrVh/XNqBA2KiGwIUjvMAdX83a5B+ ZUnD0m1yAA8KApvEVapmjrNEJZSeF6ECDr/9067/JVl0e3Fs6h8jpo7f3bnDQqLQRJE5 vu4PBfWrHWJhCaOOsRiJQeOjp1eyxapmzygPeZwYtcseauq4z6unaqaQyrvdH8YyuldQ 2qAaZOHblzb+mv9xcjc+NpQRlAYYfA6rKXMuWeBZ7NxZpiBdY1j5mqj8GfgKTbCoMih5 tBjlOde9sM2w0NT2jGz09x+hMsRAc2tg9aoBVwuk8or4pDqeNXPV1y26LXwZTHwwb8WW 8ojw== X-Google-Smtp-Source: AA+uWPwPLIPAkBfgpaTDDCI7hi7RvUk9xxnWQJrjLwo1ddsWNiNx7VadBGigB2kXtwOZ1h30J7CPBw== X-Received: by 2002:a5d:4d8d:: with SMTP id b13-v6mr2157693wru.80.1533842503017; Thu, 09 Aug 2018 12:21:43 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:42 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 17/18] khwasan: update kasan documentation Date: Thu, 9 Aug 2018 21:21:09 +0200 Message-Id: X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch updates KASAN documentation to reflect the addition of KHWASAN. Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 213 +++++++++++++++++------------- 1 file changed, 123 insertions(+), 90 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index aabc8738b3d8..842d95af74d3 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -8,11 +8,19 @@ KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides a fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN has two modes: classic KASAN (a classic version, similar to user space +ASan) and KHWASAN (a version based on memory tagging, similar to user space +HWASan). -Currently KASAN is supported only for the x86_64 and arm64 architectures. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. +For classic KASAN you need GCC version 4.9.2 or later. GCC 5.0 or later is +required for detection of out-of-bounds accesses on stack and global variables. +KHWASAN in turns is only supported in clang and requires revision 330044 or +later. + +Currently classic KASAN is supported for the x86_64, arm64 and xtensa +architectures, and KHWASAN is supported only for arm64. Usage ----- @@ -21,12 +29,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC +and choose between CONFIG_KASAN_GENERIC (to enable classic KASAN) and +CONFIG_KASAN_HW (to enabled KHWASAN). You also need to choose choose between +CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and inline are compiler +instrumentation types. The former produces smaller binary while the latter is +1.1 - 2 times faster. For classic KASAN inline instrumentation requires GCC version 5.0 or later. -KASAN works with both SLUB and SLAB memory allocators. +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line @@ -43,85 +53,80 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access classic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8800696f3d3b by task insmod/2734 + + CPU: 0 PID: 2734 Comm: insmod Not tainted 4.15.0+ #98 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [] dump_stack+0x46/0x58 - [] print_trailer+0xf8/0x160 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] object_err+0x35/0x40 - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kasan_report_error+0x38a/0x3f0 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kasan_unpoison_shadow+0x14/0x40 - [] ? kasan_poison_shadow+0x2f/0x40 - [] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [] __asan_store1+0x75/0xb0 - [] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [] kmalloc_oob_right+0x65/0x75 [test_kasan] - [] init_module+0x9/0x47 [test_kasan] - [] do_one_initcall+0x99/0x200 - [] ? __vunmap+0xec/0x160 - [] load_module+0x2cb3/0x3b20 - [] ? m_show+0x240/0x240 - [] SyS_finit_module+0x76/0x80 - [] system_call_fastpath+0x12/0x17 + __dump_stack lib/dump_stack.c:17 + dump_stack+0x83/0xbc lib/dump_stack.c:53 + print_address_description+0x73/0x280 mm/kasan/report.c:254 + kasan_report_error mm/kasan/report.c:352 + kasan_report+0x10e/0x220 mm/kasan/report.c:410 + __asan_report_store1_noabort+0x17/0x20 mm/kasan/report.c:505 + kmalloc_oob_right+0xa8/0xbc [test_kasan] lib/test_kasan.c:42 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + RIP: 0033:0x7fdd79df99da + RSP: 002b:00007fff2229bdf8 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055c408121190 RCX: 00007fdd79df99da + RDX: 00007fdd7a0b8f88 RSI: 0000000000055670 RDI: 00007fdd7a47e000 + RBP: 000055c4081200b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007fdd79df5d0a R11: 0000000000000202 R12: 00007fdd7a0b8f88 + R13: 000055c408120090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2734: + save_stack+0x43/0xd0 mm/kasan/common.c:176 + set_track+0x20/0x30 mm/kasan/common.c:188 + kasan_kmalloc+0x9a/0xc0 mm/kasan/kasan.c:372 + kmem_cache_alloc_trace+0xcd/0x1a0 mm/slub.c:2761 + kmalloc ./include/linux/slab.h:512 + kmalloc_oob_right+0x56/0xbc [test_kasan] lib/test_kasan.c:36 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + + The buggy address belongs to the object at ffff8800696f3cc0 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8800696f3cc0, ffff8800696f3d40) + The buggy address belongs to the page: + page:ffffea0001a5bcc0 count:1 mapcount:0 mapping: (null) index:0x0 + flags: 0x100000000000100(slab) + raw: 0100000000000100 0000000000000000 0000000000000000 0000000180150015 + raw: ffffea0001a8ce40 0000000300000003 ffff88006d001640 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8800696f3c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc + ffff8800696f3c80: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00 + >ffff8800696f3d00: 00 00 00 00 00 00 00 03 fc fc fc fc fc fc fc fc + ^ + ffff8800696f3d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc fc + ffff8800696f3e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works. @@ -138,18 +143,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For KHWASAN this last report section shows the memory tags around the accessed +address (see Implementation details section). + Implementation details ---------------------- +Classic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Classic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address:: @@ -162,12 +173,34 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +KHWASAN +~~~~~~~ + +KHWASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to store +a pointer tag in the top byte of kernel pointers. KHWASAN also uses shadow +memory to store memory tags associated with each 16-byte memory cell (therefore +it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation KHWASAN generates a random tag, tags allocated memory +with this tag, and embeds this tag into the returned pointer. KHWASAN uses +compile-time instrumentation to insert checks before each memory access. These +checks make sure that tag of the memory that is being accessed is equal to tag +of the pointer that is used to access this memory. In case of a tag mismatch +KHWASAN prints a bug report. + +KHWASAN also has two instrumentation modes (outline, that emits callbacks to +check memory accesses; and inline, that performs the shadow memory checks +inline). With outline instrumentation mode, a bug report is simply printed +from the function that performs the access check. With inline instrumentation +a brk instruction is emitted by the compiler, and a dedicated brk handler is +used to print KHWASAN reports. From patchwork Thu Aug 9 19:21:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10561781 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DF951057 for ; Thu, 9 Aug 2018 19:22:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E95D2B92D for ; Thu, 9 Aug 2018 19:22:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C3EE2B950; Thu, 9 Aug 2018 19:22:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E75542B954 for ; Thu, 9 Aug 2018 19:22:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F8F06B0281; Thu, 9 Aug 2018 15:21:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2B55B6B0283; Thu, 9 Aug 2018 15:21:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 124C76B0285; Thu, 9 Aug 2018 15:21:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wm0-f69.google.com (mail-wm0-f69.google.com [74.125.82.69]) by kanga.kvack.org (Postfix) with ESMTP id A38C06B0281 for ; Thu, 9 Aug 2018 15:21:46 -0400 (EDT) Received: by mail-wm0-f69.google.com with SMTP id v24-v6so639799wmh.5 for ; Thu, 09 Aug 2018 12:21:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=P1b7FpWxMvR27G4AYZ8UwIlFg8OAfRbBoa4KgffyrjU=; b=jZmMqRmrNxQV4z7UyuaNGdAMEb8uF+FPEBEupT1OHMJOYH9y9RK670D9XeCSkzWyaN /XBS9sMxAUprgMyhBB1SETOrl6BTNeoQCbcbCAC5QGcemwSQSA8muZT9PKpbUmI71ACM y6xuZj7aQYOibr1gKOeMh+qxDRqfev1sr1B2SWBI5xaDZUmM/sPxEpOgUvF4YTK4DuMd pZOYm4uc/A/ZVR4biNonlIBOGe/DZEUvmc5jjmmMCuG2HaC+LrUiqdusn5Dd4OcIyIAg opHADaKSSbiMjEhsuPpFEzIjsxKR+DuRMpIkrbFSY0R3qVEfa4yQp+jwOlD1/Vs4Hb4C 5FaA== X-Gm-Message-State: AOUpUlHDraaYDjfw3oFV+IIT6/yina+yNlmuNYqGzSNYLO5pI4Z6RSUc zKQb3SgXVN4U5ZE225/k612AcH29lpQC0Ljsa1oILu+TE0Txyf5I7jsEW/x2oQHa38sy1r52+nS 2ObmM7HYj0plbzkYUUVuz8QdU35593z+NiUnBLoWpb1Xa76ZD/issp5l2AjiEFTUe7aq7bWTF1e 7aCB80+wByCXN46/F0uMOOrdhzPNADrlvIPUnJmRTUfS+8EDp829AY5TrQ4blomqnMfA6hoxt0F u5ov1tVYn2iofyNXC/be4x3adAHyK3AGRCwPz+wb6VKChVYA0jyG9rnTosor36ZODuyf9aMu/QJ r54YNxnoP38GQedoIciyB5/3JdoTJ/MHpN65+cUCttApA+RukaqbMQY0O/ShwDuZU7oe9UVCRIc 2 X-Received: by 2002:adf:8385:: with SMTP id 5-v6mr2301372wre.13.1533842506177; Thu, 09 Aug 2018 12:21:46 -0700 (PDT) X-Received: by 2002:adf:8385:: with SMTP id 5-v6mr2301341wre.13.1533842505329; Thu, 09 Aug 2018 12:21:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533842505; cv=none; d=google.com; s=arc-20160816; b=eWf0GiQctBVT6A1V+cgrlaiKy33912rJNuPaiFPT5xOwn5XVyx8ubE/4DpTVeYG8ov Ncpjg+wCwgN7a/CEA9vu5C6yzqX4QoHLWKEAItt+VPf3LpyJ33hPKVgav6SezRyvO5ok BBbXqI9AIkAucnkjoLi6ksCtc2tjg+VQxhTZiSsugVLqaaN9tSjPIM7HfIjluKyfMVRn TUjToh3VxMDJbr49Oi5ylG0iJXI8B4LWeSD9zyW33f1FVY4JB42eJprwPVd+uxIVvcYO KqwTTt8DxQzO2oPGea4UW0jKrXViphldvDZYfZbAzn1yPU0HnoJoCi8nP4/h8N/llCeW gfJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=P1b7FpWxMvR27G4AYZ8UwIlFg8OAfRbBoa4KgffyrjU=; b=z2I9T+fTGVCPNqRKnFWqrq5RXeVbjid1ldWnpFvWTZIsq5EBL1xKZa+c3qg9SAKwUE cQBvAXcuJ6N1LaxtOCODPq4eRYKFn906FvGCojg3EPEtw0WRKL9hKglwDv3Y5RFl3TUo vPMLC9lrA5b7tLVgofvWQJuIjfQkd3WZ14XE4bfx1foDaJQ4G8ZOFuAkFBBaqF6CFyPj qipQEQbuaZ2pI6v3MZcEHaIDtGub+4PWgVaydNkHj+gOm4Q1G+je+cbOTXh/EKVm631t x8YJZA0n0iuKCvv8ypcdyYDkGZ63898dZg9761cT1bbrKX+eMcAyPvlMTgzCOL+Lc6sU pyVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=mA9+y4wC; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id r1-v6sor3239324wrm.62.2018.08.09.12.21.45 for (Google Transport Security); Thu, 09 Aug 2018 12:21:45 -0700 (PDT) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=mA9+y4wC; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=P1b7FpWxMvR27G4AYZ8UwIlFg8OAfRbBoa4KgffyrjU=; b=mA9+y4wChtcpcT3+ynkCjIMnesx2izOjZNPwE+RYeD0TuJnok9sCs/DsPlPVfaeqXY eoNqfWF3nT7nhh8dV0KZHDU9NgJT0tqYysiErx1jRem/L8bvZGsKkdBMt8f7sM4XoJM7 4OOTJu24TBusxBEYpX3/qu1RXjYsRqgB756pgUyXOHxNnOOJkG1U3+F28HRKq54U9LgW uOwDYLRmsdqC8MfYgKW6vOCg7yc1CiI55MRJbrdCFJJQ1g8QF8jDEh4c++B7D4jeU4Tr bB1fxY21mxk/UMgorlQ/sxiOk82OAcJlqPN7wyb+dcBWrJOt7vIcT22/53BOfx6K5jv/ jMog== X-Google-Smtp-Source: AA+uWPyZymo6JPhQexSC7X2U+waBw209eawNJOMS88Mv336eycjkaHsDoR0YdhFJa2lp/szHqFrJYA== X-Received: by 2002:adf:e190:: with SMTP id k16-v6mr2318826wri.36.1533842504752; Thu, 09 Aug 2018 12:21:44 -0700 (PDT) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:84be:a42a:826d:c530]) by smtp.gmail.com with ESMTPSA id o14-v6sm14738797wmd.35.2018.08.09.12.21.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Aug 2018 12:21:43 -0700 (PDT) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v5 18/18] kasan: add SPDX-License-Identifier mark to source files Date: Thu, 9 Aug 2018 21:21:10 +0200 Message-Id: <7c7356215cab699aaf06b12ef695c41eb2a1857e.1533842385.git.andreyknvl@google.com> X-Mailer: git-send-email 2.18.0.597.ga71716f1ad-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 1 + mm/kasan/kasan.c | 1 + mm/kasan/kasan_init.c | 1 + mm/kasan/kasan_report.c | 1 + mm/kasan/khwasan.c | 1 + mm/kasan/khwasan_report.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + 8 files changed, 8 insertions(+) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index e5648f4218eb..f2576d93e74c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common KASAN and KHWASAN code. * diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index 44ec228de0a2..128a865c9e05 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core KASAN code. * diff --git a/mm/kasan/kasan_init.c b/mm/kasan/kasan_init.c index f436246ccc79..2dfa730a9d43 100644 --- a/mm/kasan/kasan_init.c +++ b/mm/kasan/kasan_init.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. * diff --git a/mm/kasan/kasan_report.c b/mm/kasan/kasan_report.c index fdf2d77e3125..48da73f4ef7c 100644 --- a/mm/kasan/kasan_report.c +++ b/mm/kasan/kasan_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains KASAN specific error reporting code. * diff --git a/mm/kasan/khwasan.c b/mm/kasan/khwasan.c index 6b1309278e39..934f80b2d22e 100644 --- a/mm/kasan/khwasan.c +++ b/mm/kasan/khwasan.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core KHWASAN code. * diff --git a/mm/kasan/khwasan_report.c b/mm/kasan/khwasan_report.c index 51238b404b08..4e193546d94e 100644 --- a/mm/kasan/khwasan_report.c +++ b/mm/kasan/khwasan_report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains KHWASAN specific error reporting code. * diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 3a8ddf8baf7d..0e4dc1a22615 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. * diff --git a/mm/kasan/report.c b/mm/kasan/report.c index e031c78f2e52..633b4b245798 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common KASAN and KHWASAN error reporting code. *