From patchwork Thu Dec 21 20:04:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 13502603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E4D5C46CCD for ; Thu, 21 Dec 2023 20:06:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C1EC6B0099; Thu, 21 Dec 2023 15:06:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F3D776B009A; Thu, 21 Dec 2023 15:06:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF7EA6B009C; Thu, 21 Dec 2023 15:06:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B69E86B009A for ; Thu, 21 Dec 2023 15:06:08 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 91694A053E for ; Thu, 21 Dec 2023 20:06:08 +0000 (UTC) X-FDA: 81591906816.19.BEDD2A9 Received: from out-174.mta0.migadu.com (out-174.mta0.migadu.com [91.218.175.174]) by imf18.hostedemail.com (Postfix) with ESMTP id 8BD431C0034 for ; Thu, 21 Dec 2023 20:06:06 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=b9Brp+Gw; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.174 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703189166; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gSZaD965ObbhfRb2wl16gx9k0NFu/9CcW8O4J7mEceM=; b=ThdUr+GO6nQAY55/nP2UkwgEq1skHP07ToKPww7oF6ndjzSpzpFEMF8clwybyBgLwi4zjc ie5gozSsa2FcIQvHGmmR5n3d7vDHRWudxoK4tLtggaUyGFkbzgYVkWhEsRKmVY5Z21UPH2 pXTerQlrXb8dmSdYyDrXkmSBn3tHML0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=b9Brp+Gw; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.174 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703189166; a=rsa-sha256; cv=none; b=suCkBB5dgoxxAGV+zzF0zf4LBAxRMwueoF05/xADDdAsARuptygbaKFyKMX6guhTqCiGU5 i77BcKaCSsPK10bv5eL7udWBx8irXhjfHham6YLlZNjAIB6dO4Si56A2y1FPsMuyORdAhL 3B3ErwYPG1RizkhzBBuQ9NvrI4NHizU= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1703189164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gSZaD965ObbhfRb2wl16gx9k0NFu/9CcW8O4J7mEceM=; b=b9Brp+GwaAs8486L9eFNPsKBa+ZcAtwpfYQSQNwgNbwsobfM5YdIdTv7QlTk+BZjKdK9i+ vM+1DvB2mrXJOSHWT8Ex00o2C9qkxJuCHnbWABmaNBRFH42VhSFNwm8VJ/Zi/RdPA/9XKa vQ9svxQBNSQqoxiVPrTTZhhN4tftlRM= From: andrey.konovalov@linux.dev To: Marco Elver Cc: Andrey Konovalov , Alexander Potapenko , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH mm 06/11] kasan: clean up is_kfence_address checks Date: Thu, 21 Dec 2023 21:04:48 +0100 Message-Id: <1065732315ef4e141b6177d8f612232d4d5bc0ab.1703188911.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 8BD431C0034 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: p1nqqx9dnu66qxp3tiqxk1et5c6obbfh X-HE-Tag: 1703189166-928846 X-HE-Meta: U2FsdGVkX181khkMI6WUEV1+F2u39m8BzhqyaWuMGZN777RyJO8e3A48SKcX2gBvidsUOqHHH1no32/G0naD7NB8ckkBv6IGf+OdZnucILkDqJ1R2EDu5r8jQId9i7mUtvbkM6FGQAJkEPVTuyL9iNggLzplbfvgNGqZ5Ms5oPjiMmzenikPNgzQnag5StB+SR6mp1Bs9QJv1LAV2VrNRXmNJlvvTNQNdRu1NIaGJbpDIiwueQb0qKSLvBm+sp5NPjBaOmW71ZbulTGKL3mXYDx0bL2I6sHVouk0JdglBrkCmbJ6oRkVmUOHsDuEldLf3bMJjiasD49d16Chq/v4Q0AgMb/IyOIs05ZyTwzx84Agqp6hcnrHwCdqDLXsKTOaalEZid/QN4k9LPtiyCrJPOsi1MnOgxlQ8TEaLK92oGED3cndU5zMATBZ9ItYMt/CfFEZn6Q5ftErMkR17AH8tkjz9g+s9MrKVqh3jVPkP7lrfSAxjxNd2Kp5rIRmJQjG5V7B/pUX7H1Bn40/+M2OJpzccwQHlxsUXyBWPSG3vgfmIuzCi7nEBtybqIbR2ZPIG2x9q1mg/dZHXDRWurYpJ3KdRil9jdOUwc48DCJAE5bnmNETFVXycptLi7X/RETBKyyHvSgWyTIOvfe+znWrKPD1TLBKEPNO2tCLMuNKfepThdYMffUpByGgMZkmmVXvH5XU0mQoKFhhh37fFC/aXErgNcS0G7zo6np32CgIzIKsuWDYihg2Z8LT/NZwIR6OJshkbdXyjI1ihL1YMK0I+1p5i6nYmZw/ZcscPfts925ZHxWxFMGOQrT+JFZV5zWlzN2pbPxTjTSpUh5fpsYsINZCRDVsJ6UNQSSO6s/4Pt+nboSVLs35avJNNd7vP1aPFjAqV2R1qHsCPzEzmPculIC6YVSiIZa+MjcXfaZIxYUnTi9RphFca1440MbdVkSDmmpChx1G5c/d0n11tFX S4q1OGZB 5wddwVuOq1s1HzHfsT92bvE9YCh8HelNfCzCYK66RamESJNK4D2AcFeGkMHhn0wsg0lGr06rCGFJUiji51QSgaDnC8ygys/TwKdDrWV4kJ0LnZTCyEhDXeI1PnLoq1qfN7Gp4dMftLKSJ9cK9SrOrMS+3Vl67rKzrS07MEUrJLrmo0qe2w0cfXQWa9wGRRtTiOASpQ/Zkk22EJK4AY1W8ibqS6xTwcOQi3TlA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Andrey Konovalov 1. Do not untag addresses that are passed to is_kfence_address: it tolerates tagged addresses. 2. Move is_kfence_address checks from internal KASAN functions (kasan_poison/unpoison, etc.) to external-facing ones. Note that kasan_poison/unpoison are never called outside of KASAN/slab code anymore; the comment is wrong, so drop it. 3. Simplify/reorganize the code around the updated checks. Signed-off-by: Andrey Konovalov --- mm/kasan/common.c | 26 +++++++++++++++++--------- mm/kasan/kasan.h | 16 ++-------------- mm/kasan/shadow.c | 12 ------------ 3 files changed, 19 insertions(+), 35 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index f4255e807b74..86adf80cc11a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -79,6 +79,9 @@ EXPORT_SYMBOL(kasan_disable_current); void __kasan_unpoison_range(const void *address, size_t size) { + if (is_kfence_address(address)) + return; + kasan_unpoison(address, size, false); } @@ -218,9 +221,6 @@ static inline bool poison_slab_object(struct kmem_cache *cache, void *object, tagged_object = object; object = kasan_reset_tag(object); - if (is_kfence_address(object)) - return false; - if (unlikely(nearest_obj(cache, virt_to_slab(object), object) != object)) { kasan_report_invalid_free(tagged_object, ip, KASAN_REPORT_INVALID_FREE); return true; @@ -247,7 +247,12 @@ static inline bool poison_slab_object(struct kmem_cache *cache, void *object, bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool init) { - bool buggy_object = poison_slab_object(cache, object, ip, init); + bool buggy_object; + + if (is_kfence_address(object)) + return false; + + buggy_object = poison_slab_object(cache, object, ip, init); return buggy_object ? true : kasan_quarantine_put(cache, object); } @@ -359,7 +364,7 @@ void * __must_check __kasan_kmalloc(struct kmem_cache *cache, const void *object if (unlikely(object == NULL)) return NULL; - if (is_kfence_address(kasan_reset_tag(object))) + if (is_kfence_address(object)) return (void *)object; /* The object has already been unpoisoned by kasan_slab_alloc(). */ @@ -417,7 +422,7 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag if (unlikely(object == ZERO_SIZE_PTR)) return (void *)object; - if (is_kfence_address(kasan_reset_tag(object))) + if (is_kfence_address(object)) return (void *)object; /* @@ -483,6 +488,9 @@ bool __kasan_mempool_poison_object(void *ptr, unsigned long ip) return true; } + if (is_kfence_address(ptr)) + return false; + slab = folio_slab(folio); return !poison_slab_object(slab->slab_cache, ptr, ip, false); } @@ -492,9 +500,6 @@ void __kasan_mempool_unpoison_object(void *ptr, size_t size, unsigned long ip) struct slab *slab; gfp_t flags = 0; /* Might be executing under a lock. */ - if (is_kfence_address(kasan_reset_tag(ptr))) - return; - slab = virt_to_slab(ptr); /* @@ -507,6 +512,9 @@ void __kasan_mempool_unpoison_object(void *ptr, size_t size, unsigned long ip) return; } + if (is_kfence_address(ptr)) + return; + /* Unpoison the object and save alloc info for non-kmalloc() allocations. */ unpoison_slab_object(slab->slab_cache, ptr, size, flags); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 1c34511090d7..5fbcc1b805bc 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -466,35 +466,23 @@ static inline u8 kasan_random_tag(void) { return 0; } static inline void kasan_poison(const void *addr, size_t size, u8 value, bool init) { - addr = kasan_reset_tag(addr); - - /* Skip KFENCE memory if called explicitly outside of sl*b. */ - if (is_kfence_address(addr)) - return; - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) return; if (WARN_ON(size & KASAN_GRANULE_MASK)) return; - hw_set_mem_tag_range((void *)addr, size, value, init); + hw_set_mem_tag_range(kasan_reset_tag(addr), size, value, init); } static inline void kasan_unpoison(const void *addr, size_t size, bool init) { u8 tag = get_tag(addr); - addr = kasan_reset_tag(addr); - - /* Skip KFENCE memory if called explicitly outside of sl*b. */ - if (is_kfence_address(addr)) - return; - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) return; size = round_up(size, KASAN_GRANULE_SIZE); - hw_set_mem_tag_range((void *)addr, size, tag, init); + hw_set_mem_tag_range(kasan_reset_tag(addr), size, tag, init); } static inline bool kasan_byte_accessible(const void *addr) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 0154d200be40..30625303d01a 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -135,10 +135,6 @@ void kasan_poison(const void *addr, size_t size, u8 value, bool init) */ addr = kasan_reset_tag(addr); - /* Skip KFENCE memory if called explicitly outside of sl*b. */ - if (is_kfence_address(addr)) - return; - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) return; if (WARN_ON(size & KASAN_GRANULE_MASK)) @@ -175,14 +171,6 @@ void kasan_unpoison(const void *addr, size_t size, bool init) */ addr = kasan_reset_tag(addr); - /* - * Skip KFENCE memory if called explicitly outside of sl*b. Also note - * that calls to ksize(), where size is not a multiple of machine-word - * size, would otherwise poison the invalid portion of the word. - */ - if (is_kfence_address(addr)) - return; - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) return;