From patchwork Wed Jul 24 16:34:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 13741175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59369C3DA63 for ; Wed, 24 Jul 2024 16:34:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4158F6B0092; Wed, 24 Jul 2024 12:34:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C50C6B0093; Wed, 24 Jul 2024 12:34:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B48B6B0095; Wed, 24 Jul 2024 12:34:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0DABE6B0092 for ; Wed, 24 Jul 2024 12:34:24 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B05F812099A for ; Wed, 24 Jul 2024 16:34:23 +0000 (UTC) X-FDA: 82375194006.29.F20A294 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf23.hostedemail.com (Postfix) with ESMTP id B3B50140020 for ; Wed, 24 Jul 2024 16:34:21 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=PTtq4cjR; spf=pass (imf23.hostedemail.com: domain of jannh@google.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=jannh@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721838814; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7EM+ZpXg0gF/YjekvI/GGKZPQZmJkVX1r6GY58f54Us=; b=ZSNAPrZPamDdhzclGeO/uSvAdEAXPbJEG7EDA379KBk+7XXtE2uI8tZ663bReKZ0tAR7ez 2TBU8WN+398lg72KsokNGDaWKDdOQTM1r1yR41WwaOCT1vyECbjev0yR0jkOrtRaVEToeB B+PLL6uV92V1DfA9o8z1GWC5EtdfZLI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721838814; a=rsa-sha256; cv=none; b=ztLchemjh5LnQJS0WrliW599QQE9Yo3UqTXgZgintZBRm5R4iEvABE1ltjEMSnOx3luzmv N7sHFgZ2wko5Z0zpoDGRIbadQc1CT2d0Ci5Vx/FvnOMaBLozCmWFbOjv04XrB2Idne0mhw 0EWVBaTmKJYyC52HoCV1XH+LCh8wHjQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=PTtq4cjR; spf=pass (imf23.hostedemail.com: domain of jannh@google.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=jannh@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-427fc9834deso1425e9.0 for ; Wed, 24 Jul 2024 09:34:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721838860; x=1722443660; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=7EM+ZpXg0gF/YjekvI/GGKZPQZmJkVX1r6GY58f54Us=; b=PTtq4cjRWN2Ou83Bnddf5oEVVcTskwRZ7Mxhlj4POWTrDfQTeh1zgXmGq8dFtlf0dF WkzAGRqxhoMGVAPLd6l3Icp4USo0lBrY1jSRF1lw75Sf7QLAfwb5oc3ZjYq+iwz6f9tu 2UtoC4OMHKVWZmVyhz5pOq954DOGq3IMNtquR8IzJEeCrspnUcbBeyEodusI51CnKx8Z rnlHMg/gGzIJ9KtVMuawy+TBgmt7UjZZIkEhcwQMgqUGpZfusIsKjLs/yTZiFAnjNlqp sO7YNVRAEDUQsYTDrb3E1MfaR8P2xS4VtwvsShZGjeOfHkLfnvVbRIbGbPd2CIpjCbCU 3Oeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721838860; x=1722443660; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7EM+ZpXg0gF/YjekvI/GGKZPQZmJkVX1r6GY58f54Us=; b=gmGCVLE7RiAbLGh+ICl0AfaFRP0rlnBfV5buFGh8VyPEchXAAwaevMWwFCv2SCnvNW O2MKSOTmaF9ZFFSCfXw8wJ74hBsSw7J3yuTiytGz5DdZI3VjmPhE1jG9TeTn+lCb47Bz E2qNBj1/xrifAl1EMXpclRo+fYKMH7gC54VnFZxFzIHmWdFg5n9N4yKsIa0oLrWWgFKK ZjqZq5+Tu3wfgPs0Wj5e8+i77v99ZMpt+q4eItV0d3Y+eMwRINUTU1C5IFSK4FHdIntc GE6y2qM+lDbsUsms9F3owok7CkoV+naiiVPBx7eKVzQGW09AuGjLZFU/O9kEMQyPTuXQ rYPg== X-Forwarded-Encrypted: i=1; AJvYcCX5fqvfAXs48fdyp6fUiXmj+TpXF5z63uyrH2dXxOkx5goQ5mQu7cKC1ElMQpf0n88Z1helxPIl419vFTQpzXThexE= X-Gm-Message-State: AOJu0Ywk6RM44iNLGJQ2jCotsHhJ7yM545YHr7Bwa5CHsfBYrDwXIaek X10r7qwGxDAe/ulxWkI2NSng0KnaFbLRUhLw+WZses7J+ahWXK8ErGPOCtJpoA== X-Google-Smtp-Source: AGHT+IGRhEir+wlPPgo0EeIMHbQAc+CssNsUlCHg2jKNK1SbGim7OTROv4D0aUO+HlGcU3hVo9zRjg== X-Received: by 2002:a05:600c:4454:b0:426:68ce:c97a with SMTP id 5b1f17b1804b1-427f7c5c3b3mr1464825e9.7.1721838859742; Wed, 24 Jul 2024 09:34:19 -0700 (PDT) Received: from localhost ([2a00:79e0:9d:4:7aec:12da:2527:71ba]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-427f92c7cb3sm37264285e9.0.2024.07.24.09.34.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jul 2024 09:34:19 -0700 (PDT) From: Jann Horn Date: Wed, 24 Jul 2024 18:34:12 +0200 Subject: [PATCH v2 1/2] kasan: catch invalid free before SLUB reinitializes the object MIME-Version: 1.0 Message-Id: <20240724-kasan-tsbrcu-v2-1-45f898064468@google.com> References: <20240724-kasan-tsbrcu-v2-0-45f898064468@google.com> In-Reply-To: <20240724-kasan-tsbrcu-v2-0-45f898064468@google.com> To: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Marco Elver , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jann Horn X-Mailer: b4 0.15-dev X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B3B50140020 X-Stat-Signature: at7nsokrn9gktruokt7e6tapxsqm9sjo X-HE-Tag: 1721838861-149224 X-HE-Meta: U2FsdGVkX18OxiwMlEkQJs8FR2NikJvFPEVZ6aWVSWY2xhpdwSXfMzkuW8v2PfzleiBgOjlJn+oR9Put6qbT6KjSyjMvW/xDkzU0yqRn0lhyFXYL34AtezfhbklaV+fkeIP3l4cYWRqY3H8HrTuBlJltclw0d4ak+4Vf1XGuobn+9TctoTFwnlWT3PRfqr8/2DY1sEOFSw9SzPL6eTPZHdAg0MDgWN7SNoRAjICcqUKg1MSN9AV3+fH90sYxko1z1KMXNm5gBKKsMw4hvYgB/A//jR+g76hGjG58hxYrUUnQE3ZRrIuiNbKJEZcuQ6aeKLTpX0/1tajty+vkTf11PjmRpL2+KHNFXMJuuIR1AXs7zRMqIwCZrWPGxWy1+futoy4CL1ef9ohLOtcSW+wlobK060YlIrpafEBtSgprVmxSzQpiduEFoH01XGoqCm3gmaclJLreY72ucYURKbT7cMJojo+nYwOm5mX5tcKUfR+dvGy7sjj2BHX7EBfqc8deDIiMGXsl9RX1BuNb8VbqAih/6kw5d7oV4FURwhzVqr2ZArT5e3BvUxECN9wg89IKgNS0CA04H3UQJnbyZe19cTWvVzNvURaVg9zovx0GVe/BahI1tALg6x8o8o07oPQjkO6oGpqfKF/3hBqrLDOrWez6RorCatGu9IKksxmxo6Np3lmsqgFEOeyl1VKpdayVJYXH/oFghp4DDWtu53YhFbyeqZaF0sdNVmj0V7qNr6OxVTXplEq4qKuNv+UiMynjUJ9y9VkwmsY1O0M6R8309EPJd8Kue823toRZwQ/WJB+vTpdmVSWbgBcYbd+ZjcOvVqj03dL34AUfWLvp8rnHHYk4vtYyfNJX+AwMXAY03GGp2Ou87NkRtoPKA0W/Kxtw13xIwWm/Sw6quTcHKq2Ft5qy4GChD4Nbo0CZ/0JCxE0e8H00apLlLuF+n97pENMYEoAatGZAWef84DgbOHj dMpJ6Hqs AYj14vyCWErhmz58GxeWM4SxzXMZhwmLJYEcXw9o5aoOgOpSj+vGvW3MucYGaU+hqJ/gd17ODnDKN/p3+KQOVBR55r+mpdQupp/tnHzNJ7bjFVTGqPeVTG/YMWd/rY/lUFlVg+aSdc/qJ0cEzsj2VMKp6W4fQ3CYGYugwl/k1zUOYAMpxA8UpYQnLzUwJ44+7FF0BKswjzWrL8Kw7MqhzBn3YYGW6LhjyiSNRDhAyWjkBcWSqs/RJRxEUDWXuB3ec/qwGGamiYOoD+zBMO1tcIdOxOikxvEljh3WEUF5G0CVGw51D7iOJWOe2ueN8axluKjftacaF74N7zGC0WL/35jOkqobMfYWvLjhrJ7MMB6y0EjY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, when KASAN is combined with init-on-free behavior, the initialization happens before KASAN's "invalid free" checks. More importantly, a subsequent commit will want to use the object metadata region to store an rcu_head, and we should let KASAN check that the object pointer is valid before that. (Otherwise that change will make the existing testcase kmem_cache_invalid_free fail.) So add a new KASAN hook that allows KASAN to pre-validate a kmem_cache_free() operation before SLUB actually starts modifying the object or its metadata. Signed-off-by: Jann Horn Acked-by: Vlastimil Babka #slub --- include/linux/kasan.h | 10 ++++++++++ mm/kasan/common.c | 51 +++++++++++++++++++++++++++++++++++++++------------ mm/slub.c | 7 +++++++ 3 files changed, 56 insertions(+), 12 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 70d6a8f6e25d..eee8ca1dcb40 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -175,6 +175,16 @@ static __always_inline void * __must_check kasan_init_slab_obj( return (void *)object; } +bool __kasan_slab_pre_free(struct kmem_cache *s, void *object, + unsigned long ip); +static __always_inline bool kasan_slab_pre_free(struct kmem_cache *s, + void *object) +{ + if (kasan_enabled()) + return __kasan_slab_pre_free(s, object, _RET_IP_); + return false; +} + bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip, bool init); static __always_inline bool kasan_slab_free(struct kmem_cache *s, diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 85e7c6b4575c..7c7fc6ce7eb7 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -208,31 +208,52 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, return (void *)object; } -static inline bool poison_slab_object(struct kmem_cache *cache, void *object, - unsigned long ip, bool init) +enum free_validation_result { + KASAN_FREE_IS_IGNORED, + KASAN_FREE_IS_VALID, + KASAN_FREE_IS_INVALID +}; + +static enum free_validation_result check_slab_free(struct kmem_cache *cache, + void *object, unsigned long ip) { - void *tagged_object; + void *tagged_object = object; - if (!kasan_arch_is_ready()) - return false; + if (is_kfence_address(object) || !kasan_arch_is_ready()) + return KASAN_FREE_IS_IGNORED; - tagged_object = object; object = kasan_reset_tag(object); if (unlikely(nearest_obj(cache, virt_to_slab(object), object) != object)) { kasan_report_invalid_free(tagged_object, ip, KASAN_REPORT_INVALID_FREE); - return true; + return KASAN_FREE_IS_INVALID; } - /* RCU slabs could be legally used after free within the RCU period. */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - if (!kasan_byte_accessible(tagged_object)) { kasan_report_invalid_free(tagged_object, ip, KASAN_REPORT_DOUBLE_FREE); - return true; + return KASAN_FREE_IS_INVALID; } + return KASAN_FREE_IS_VALID; +} + +static inline bool poison_slab_object(struct kmem_cache *cache, void *object, + unsigned long ip, bool init) +{ + void *tagged_object = object; + enum free_validation_result valid = check_slab_free(cache, object, ip); + + if (valid == KASAN_FREE_IS_IGNORED) + return false; + if (valid == KASAN_FREE_IS_INVALID) + return true; + + object = kasan_reset_tag(object); + + /* RCU slabs could be legally used after free within the RCU period. */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + kasan_poison(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), KASAN_SLAB_FREE, init); @@ -242,6 +263,12 @@ static inline bool poison_slab_object(struct kmem_cache *cache, void *object, return false; } +bool __kasan_slab_pre_free(struct kmem_cache *cache, void *object, + unsigned long ip) +{ + return check_slab_free(cache, object, ip) == KASAN_FREE_IS_INVALID; +} + bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool init) { diff --git a/mm/slub.c b/mm/slub.c index 4927edec6a8c..34724704c52d 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2170,6 +2170,13 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bool init) if (kfence_free(x)) return false; + /* + * Give KASAN a chance to notice an invalid free operation before we + * modify the object. + */ + if (kasan_slab_pre_free(s, x)) + return false; + /* * As memory initialization might be integrated into KASAN, * kasan_slab_free and initialization memset's must be