From patchwork Thu Dec 6 12:24:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 10715733 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1156A109C for ; Thu, 6 Dec 2018 12:24:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F29E32CED0 for ; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E651C2D325; Thu, 6 Dec 2018 12:24:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 946302CED0 for ; Thu, 6 Dec 2018 12:24:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD2FF6B79ED; Thu, 6 Dec 2018 07:24:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B84DE6B79EE; Thu, 6 Dec 2018 07:24:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 989E16B79EF; Thu, 6 Dec 2018 07:24:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by kanga.kvack.org (Postfix) with ESMTP id 27EFD6B79ED for ; Thu, 6 Dec 2018 07:24:51 -0500 (EST) Received: by mail-wr1-f69.google.com with SMTP id h11so90076wrs.2 for ; Thu, 06 Dec 2018 04:24:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=u7kkCgM2bR7dpslIwzvVoZC2X1ySw6gfdlixt56XeAA=; b=WgF+aNCi1pS9ukYYld4g747OMsbq9m6H/Dirbz/sLopZ09I0qdTtvgQ/VgJQQ2C5Rf afJGV9CAgXCEiUAZiknerOHmamOT0KxEXrwJ7Ej5DS2zMgpuOJQzBZB0MZDu5VoHaP6Y Bmxd8f0+jbFnuenRe2mgmLla8JsJBpysHz3P/5+Zl33fM8okvqK9FOWWnfyNf0UpGTmj fA+CirzKh7c29MUaRtelzBRl4DHxJue2tEM6vwGEuBgDBGw4+NOKsXcw/9rEXU6LNFHj mTK6nOTmv/I68wgX47ZyofLicPKdzlTwmb1QoV1ggIR6eTsoeRDqcikI0ib3f7m3+4vc 2ogw== X-Gm-Message-State: AA+aEWYiwmmEZWjqC+YVf3OQd8hiHfVO3GnOC6/zODeNPVh+vMjdWb08 8CsyF3kWQRhIe1khbsj51jWGoubOFQJtE/ptqQ7723avIoYg2+kPRTmm+3vQ3wRdxFnjMDtPv/u YnZpwYePMYvs8pPMgSRmSFrzwwZJ1BsKZSKXNEsA51n9wTFH32XzX7DWp+HxKJgQAL4yOCrFkH2 cEE4eOebbERI//zOyK1JdBqRx+zFF2pERt4E+MDx8ZAZwKnT4ZCR8asjb4pwCtbMbQOGM9yg7hx BCeq6xgYYpM6+NSGpbXg9G3uqzhpxm06oWLGXr7NCqklrCkF2W3s2/K5myyTeZBS4qCYrQ9Cupp vuxf8cG8vUHLLLX/2j2GLHg47mHuYjBMlOIl+xsRx1x8SUjeJYn8TsjICYUdftJNHpYGyvXNE1h V X-Received: by 2002:a1c:c008:: with SMTP id q8mr17619738wmf.99.1544099090563; Thu, 06 Dec 2018 04:24:50 -0800 (PST) X-Received: by 2002:a1c:c008:: with SMTP id q8mr17619667wmf.99.1544099089312; Thu, 06 Dec 2018 04:24:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544099089; cv=none; d=google.com; s=arc-20160816; b=nPUbvJkfjBsQY61VXiBrdoZ/0VWWV1/7qEDkpP/DoanPsNEMxOvOlzhtlSn0s1Q/LI JaON5v3TmmK2XWjdH7Obcwts+ZuPyFxo/xCDg1etPnZQzT64znSowETjOlGd60IlC/sZ SiPklTEzytQSPFC8Mlr7V3eK2wn1Mjff9+0fajGjYUdgEksHVvqhEIEJwPPT78F/lB+N o+r2ZU/h7lWZEg8Qgr0kvgKZdlc+dQG9VkPpTEigVDnAXDUbeheb3Lw8oA3aE1LQabUT nuKbOfjvt5puUFrqw49n+qA1h3FnDEtiQrYHr8557ZyGMtnZZR7lhc+w5ujRt/cUjsYb UJqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=u7kkCgM2bR7dpslIwzvVoZC2X1ySw6gfdlixt56XeAA=; b=HQZuD8cdjamUaLi/282L0QuObn1qIWpCx54HsoBQlimSjaPPBZ2344qRNS0dSy0ACy f3T0voPX4Ae8DCn3NWAjvVf/0rscOOQHSTYDahy77V81TVJWLn/VbyWUyqfq4Yca5Xyu gIkxta6HJvHRBdSeYNSuUgVjysmpt6rRXR7PNK7P/6l5z5nf/PbnKJKDs2hLmfQIngNa S4emsch/TamhVqL5Jni20Txgh/tP67Aiw7CKnPVAnxmfNNtDRJqSqCZ57YCpBP0sH9wR f+KTmoRGMMsMz8z1PjyhRU9HQXNWITwfZmp9U2s4NcTs8hYSh23VpjUcNC94HBHJKcKH aKIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Enf16wiL; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id u1sor191570wrr.11.2018.12.06.04.24.49 for (Google Transport Security); Thu, 06 Dec 2018 04:24:49 -0800 (PST) Received-SPF: pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Enf16wiL; spf=pass (google.com: domain of andreyknvl@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=andreyknvl@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u7kkCgM2bR7dpslIwzvVoZC2X1ySw6gfdlixt56XeAA=; b=Enf16wiLZ5gbJcXeaYiv9T1lvd3QlfqZdXV2wcaMmxRbE8VKhpm7vz5gMTdXUDRxqr KCtqw72+zySmDtyFsn+cC0pk5UjavJ97GSAQYQevUz1cTUEbhiEhnEJ4uktbWx0/Kxig x3KFPWluyH+Ov5kOAYDbwp+fQJ1zrdEN/ZxFJWo9/Jjps6twi0JjR19lI1+gn9wxnjd3 /YnUmARwPIqmdyaYp/EYE7RKjSf0iCP5RYloFX9D6CfyhKp6CfJQYJ4RjyaaQbICrJuK QjYAb2YFRySjHVkaiC9dQqebDjzP+V+aX6TQd5aoK5ZzvW6JGQT+z3KnKLahJ2Pq8ky6 cN0A== X-Google-Smtp-Source: AFSGD/UpRIgk9h+y1gmV55lfHXBpLAxVo71KoMj2eiGthQAxJl9eXORdHev95etTM9WXfeqMgeXWhQ== X-Received: by 2002:adf:b243:: with SMTP id y3mr10682390wra.184.1544099088593; Thu, 06 Dec 2018 04:24:48 -0800 (PST) Received: from andreyknvl0.muc.corp.google.com ([2a00:79e0:15:10:3180:41f8:3010:ff61]) by smtp.gmail.com with ESMTPSA id j8sm339988wrt.40.2018.12.06.04.24.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 04:24:47 -0800 (PST) From: Andrey Konovalov To: Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Catalin Marinas , Will Deacon , Christoph Lameter , Andrew Morton , Mark Rutland , Nick Desaulniers , Marc Zyngier , Dave Martin , Ard Biesheuvel , "Eric W . Biederman" , Ingo Molnar , Paul Lawrence , Geert Uytterhoeven , Arnd Bergmann , "Kirill A . Shutemov" , Greg Kroah-Hartman , Kate Stewart , Mike Rapoport , kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-sparse@vger.kernel.org, linux-mm@kvack.org, linux-kbuild@vger.kernel.org Cc: Kostya Serebryany , Evgeniy Stepanov , Lee Smith , Ramana Radhakrishnan , Jacob Bramley , Ruben Ayrapetyan , Jann Horn , Mark Brand , Chintan Pandya , Vishwath Mohan , Andrey Konovalov Subject: [PATCH v13 01/25] kasan, mm: change hooks signatures Date: Thu, 6 Dec 2018 13:24:19 +0100 Message-Id: X-Mailer: git-send-email 2.20.0.rc1.387.gf8505762e3-goog In-Reply-To: References: MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tag-based KASAN changes the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that. Reviewed-by: Andrey Ryabinin Reviewed-by: Dmitry Vyukov Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 43 +++++++++++++++++++++++++++++-------------- include/linux/slab.h | 4 ++-- mm/kasan/kasan.c | 30 ++++++++++++++++++------------ mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 7 files changed, 65 insertions(+), 45 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 46aae129917c..52c86a568a4e 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -51,16 +51,16 @@ void kasan_cache_shutdown(struct kmem_cache *cache); void kasan_poison_slab(struct page *page); void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object); +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags); void kasan_kfree_large(void *ptr, unsigned long ip); void kasan_poison_kfree(void *ptr, unsigned long ip); -void kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags); -void kasan_krealloc(const void *object, size_t new_size, gfp_t flags); +void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags); -void kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); +void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); struct kasan_cache { @@ -105,19 +105,34 @@ static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} static inline void kasan_poison_object_data(struct kmem_cache *cache, void *object) {} -static inline void kasan_init_slab_obj(struct kmem_cache *cache, - const void *object) {} +static inline void *kasan_init_slab_obj(struct kmem_cache *cache, + const void *object) +{ + return (void *)object; +} -static inline void kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) {} +static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +{ + return ptr; +} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} -static inline void kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags) {} -static inline void kasan_krealloc(const void *object, size_t new_size, - gfp_t flags) {} +static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags) +{ + return (void *)object; +} +static inline void *kasan_krealloc(const void *object, size_t new_size, + gfp_t flags) +{ + return (void *)object; +} -static inline void kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) {} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; +} static inline bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip) { diff --git a/include/linux/slab.h b/include/linux/slab.h index 918f374e7156..351ac48dabc4 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -444,7 +444,7 @@ static __always_inline void *kmem_cache_alloc_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc(s, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -455,7 +455,7 @@ kmem_cache_alloc_node_trace(struct kmem_cache *s, { void *ret = kmem_cache_alloc_node(s, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } #endif /* CONFIG_TRACING */ diff --git a/mm/kasan/kasan.c b/mm/kasan/kasan.c index c3bd5209da38..55deff17a4d9 100644 --- a/mm/kasan/kasan.c +++ b/mm/kasan/kasan.c @@ -474,20 +474,22 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache, return (void *)object + cache->kasan_info.free_meta_offset; } -void kasan_init_slab_obj(struct kmem_cache *cache, const void *object) +void *kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_info; if (!(cache->flags & SLAB_KASAN)) - return; + return (void *)object; alloc_info = get_alloc_info(cache, object); __memset(alloc_info, 0, sizeof(*alloc_info)); + + return (void *)object; } -void kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) +void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_kmalloc(cache, object, cache->object_size, flags); + return kasan_kmalloc(cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object, @@ -528,7 +530,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } -void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, +void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags) { unsigned long redzone_start; @@ -538,7 +540,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, quarantine_reduce(); if (unlikely(object == NULL)) - return; + return NULL; redzone_start = round_up((unsigned long)(object + size), KASAN_SHADOW_SCALE_SIZE); @@ -551,10 +553,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, if (cache->flags & SLAB_KASAN) set_track(&get_alloc_info(cache, object)->alloc_track, flags); + + return (void *)object; } EXPORT_SYMBOL(kasan_kmalloc); -void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; unsigned long redzone_start; @@ -564,7 +568,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) quarantine_reduce(); if (unlikely(ptr == NULL)) - return; + return NULL; page = virt_to_page(ptr); redzone_start = round_up((unsigned long)(ptr + size), @@ -574,21 +578,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) kasan_unpoison_shadow(ptr, size); kasan_poison_shadow((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); + + return (void *)ptr; } -void kasan_krealloc(const void *object, size_t size, gfp_t flags) +void *kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; if (unlikely(object == ZERO_SIZE_PTR)) - return; + return ZERO_SIZE_PTR; page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - kasan_kmalloc_large(object, size, flags); + return kasan_kmalloc_large(object, size, flags); else - kasan_kmalloc(page->slab_cache, object, size, flags); + return kasan_kmalloc(page->slab_cache, object, size, flags); } void kasan_poison_kfree(void *ptr, unsigned long ip) diff --git a/mm/slab.c b/mm/slab.c index 2a5654bb3b3f..26f60a22e5e0 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) { void *ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc(_RET_IP_, ret, cachep->object_size, cachep->size, flags); @@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret = slab_alloc(cachep, flags, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(_RET_IP_, ret, size, cachep->size, flags); return ret; @@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) { void *ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_slab_alloc(cachep, ret, flags); + ret = kasan_slab_alloc(cachep, ret, flags); trace_kmem_cache_alloc_node(_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid); @@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret = slab_alloc_node(cachep, flags, nodeid, _RET_IP_); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc_node(_RET_IP_, ret, size, cachep->size, flags, nodeid); @@ -3681,7 +3681,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller) if (unlikely(ZERO_OR_NULL_PTR(cachep))) return cachep; ret = kmem_cache_alloc_node_trace(cachep, flags, node, size); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); return ret; } @@ -3719,7 +3719,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags, return cachep; ret = slab_alloc(cachep, flags, caller); - kasan_kmalloc(cachep, ret, size, flags); + ret = kasan_kmalloc(cachep, ret, size, flags); trace_kmalloc(caller, ret, size, cachep->size, flags); diff --git a/mm/slab.h b/mm/slab.h index 58c6c1c2a78e..4190c24ef0e9 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, kmemleak_alloc_recursive(object, s->object_size, 1, s->flags, flags); - kasan_slab_alloc(s, object, flags); + p[i] = kasan_slab_alloc(s, object, flags); } if (memcg_kmem_enabled()) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7eb8dc136c1c..5f3504e26d4c 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1204,7 +1204,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page = alloc_pages(flags, order); ret = page ? page_address(page) : NULL; kmemleak_alloc(ret, size, 1, flags); - kasan_kmalloc_large(ret, size, flags); + ret = kasan_kmalloc_large(ret, size, flags); return ret; } EXPORT_SYMBOL(kmalloc_order); @@ -1482,7 +1482,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks = ksize(p); if (ks >= new_size) { - kasan_krealloc((void *)p, new_size, flags); + p = kasan_krealloc((void *)p, new_size, flags); return (void *)p; } diff --git a/mm/slub.c b/mm/slub.c index e3629cd7aff1..fdd4a86aa882 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1372,10 +1372,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node, * Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) +static inline void *kmalloc_large_node_hook(void *ptr, size_t size, gfp_t flags) { kmemleak_alloc(ptr, size, 1, flags); - kasan_kmalloc_large(ptr, size, flags); + return kasan_kmalloc_large(ptr, size, flags); } static __always_inline void kfree_hook(void *x) @@ -2768,7 +2768,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) { void *ret = slab_alloc(s, gfpflags, _RET_IP_); trace_kmalloc(_RET_IP_, ret, size, s->size, gfpflags); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_trace); @@ -2796,7 +2796,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s, trace_kmalloc_node(_RET_IP_, ret, size, s->size, gfpflags, node); - kasan_kmalloc(s, ret, size, gfpflags); + ret = kasan_kmalloc(s, ret, size, gfpflags); return ret; } EXPORT_SYMBOL(kmem_cache_alloc_node_trace); @@ -3784,7 +3784,7 @@ void *__kmalloc(size_t size, gfp_t flags) trace_kmalloc(_RET_IP_, ret, size, s->size, flags); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; } @@ -3801,8 +3801,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node) if (page) ptr = page_address(page); - kmalloc_large_node_hook(ptr, size, flags); - return ptr; + return kmalloc_large_node_hook(ptr, size, flags); } void *__kmalloc_node(size_t size, gfp_t flags, int node) @@ -3829,7 +3828,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node) trace_kmalloc_node(_RET_IP_, ret, size, s->size, flags, node); - kasan_kmalloc(s, ret, size, flags); + ret = kasan_kmalloc(s, ret, size, flags); return ret; }