From patchwork Thu Jul 25 15:31:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jann Horn X-Patchwork-Id: 13742036 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20A5EC3DA49 for ; Thu, 25 Jul 2024 15:32:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2026A6B0088; Thu, 25 Jul 2024 11:32:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 167EE6B0092; Thu, 25 Jul 2024 11:32:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF8A96B0089; Thu, 25 Jul 2024 11:32:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D4E146B0085 for ; Thu, 25 Jul 2024 11:32:15 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3F2521405B4 for ; Thu, 25 Jul 2024 15:32:15 +0000 (UTC) X-FDA: 82378666230.15.5E65A8D Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf01.hostedemail.com (Postfix) with ESMTP id 9DE9540031 for ; Thu, 25 Jul 2024 15:32:11 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NgyVPm8y; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of jannh@google.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=jannh@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721921477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8vTJQJdpFKknSoM6KqvVMqLh+KmmrksaVuFYJyrU9io=; b=y61RPZpld7+esD08DU3H2AS/ISm9GJlJcudRgqrFeAOxAqQQKtN8CAJvnxa0VIVD2U/ULX dWpRnim+mHU6LAmq0XRJlQnTzFgo+EXurUpSiIJtS1Lj3JJrk49A9rcGWnM+WWqbe9jUwV tT6kZZcoBzZQ0bJcbk+t/SA8KmatOHA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721921477; a=rsa-sha256; cv=none; b=S/4GwkfOoHDXFlcn/+E93sCjUaUayWI57XD0WsPl5HzYzkLOXqzwr31ygBc1lfzErMgOTt k94xvTuQIjeuvjdEsoD9pY1r52JR8fp1Fj/E3nIp+nyEg5yCLNMoULg4i5IK4L14vOkt96 ftRn3U72Zj6n92IeYsHcSSzCTDZEqzQ= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NgyVPm8y; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of jannh@google.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=jannh@google.com Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-427fc9834deso55675e9.0 for ; Thu, 25 Jul 2024 08:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721921530; x=1722526330; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=8vTJQJdpFKknSoM6KqvVMqLh+KmmrksaVuFYJyrU9io=; b=NgyVPm8ynlN1oK1Ot+HHMzEwasCPmKmDtdlK1brJ8SIf2l0Itok31ri5G3j3u8yXop bVkL2IqG6G12A6PD4NBD+RwDN/vEgMk4Knb3K4NaKrjKgvg0rjw5St5zCvokh5+hXdxQ fR8GCVFzwfq7O08AaMMjf+Rz3/8B/DtxlqO9JS4zp2rhJmptP0eCTcbliXoNUfQP0Yqg faTvtlJ/1Me4tQItHGkpnuRoHDpnQg2Y9eg5LjjQQrQvnVZPr8ZxK/ushxAH0PAnbjRA Vmr6ECHp9qcPJZcdYzhDdHI3Km3vrVzTAcPehz9hBC8/r9P3IFoOyQm5NdMDvT/iZThS 8G4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721921530; x=1722526330; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8vTJQJdpFKknSoM6KqvVMqLh+KmmrksaVuFYJyrU9io=; b=B0f4wktjanQVltGuh3x+ZAJ7x3U2tuoGGSK0h1l1IYEWBIfQW+/brBqnLLtQGcRV8c F2g9zNcj1h2L6drchowdNzfOLafd93hx8zWL/bwH6lnM+NQoxwsrbfNBmnGMy+G6uMi3 qy3QNMPb9S7j3FIFnkPv+F2ZCet1l00bwX2hO6cF2669gq9MAvba1n+I8Qi7za6Da1M6 Bajlt3cQKC55pFfkKfCYGfd2bk271kZU1oYl2Qc2SW2c5a9L0IfYZOSaRxVC2s6q8nDg 6DElSn/NjX+qBLqkVasw54dI03B8tyBT8eJlneNSuwwfCT79UTek3elfg4zTQETVy2EN P+Gw== X-Forwarded-Encrypted: i=1; AJvYcCXzaoM+3D6Mt5xTmN/QNvXX/70qUHg5JOCf79LpyDokW/6vjnVnv0lPW50YnWQFylcXVrGg+RrmyLB/nzjeNIk3Jeg= X-Gm-Message-State: AOJu0YwtQePT7ucJ8MgdlZxgrMO+Yuj5oJfYJzkadRVWUt3UIxmTWUYt 7uycydwgPs+HWvGFU+ILHQzkDuOAw9u/m4fmcrXjBv5jo7wPe6vflurIhpGMCw== X-Google-Smtp-Source: AGHT+IHg46k8yFhhlKmxiau+po7slN/BdUQxlPTCCg1CDEGtwNC/AbIGdPn3T7W50rDLjeoJZ0IR/g== X-Received: by 2002:a05:600c:5108:b0:426:62a2:dfc with SMTP id 5b1f17b1804b1-42804caec94mr1456125e9.5.1721921527673; Thu, 25 Jul 2024 08:32:07 -0700 (PDT) Received: from localhost ([2a00:79e0:9d:4:8b71:b285:2625:c911]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-427f93e65a7sm83054705e9.33.2024.07.25.08.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jul 2024 08:32:07 -0700 (PDT) From: Jann Horn Date: Thu, 25 Jul 2024 17:31:34 +0200 Subject: [PATCH v3 1/2] kasan: catch invalid free before SLUB reinitializes the object MIME-Version: 1.0 Message-Id: <20240725-kasan-tsbrcu-v3-1-51c92f8f1101@google.com> References: <20240725-kasan-tsbrcu-v3-0-51c92f8f1101@google.com> In-Reply-To: <20240725-kasan-tsbrcu-v3-0-51c92f8f1101@google.com> To: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Marco Elver , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jann Horn X-Mailer: b4 0.15-dev X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9DE9540031 X-Stat-Signature: eyb6ksqeomwyqkqzx3y6aic64y47m6g7 X-Rspam-User: X-HE-Tag: 1721921531-860638 X-HE-Meta: U2FsdGVkX18Bm8sFy8HDfzkutTrspxC4Jk1BsHN9q/WsBlpYNlGsn0ldHYskOruKHZzE3caZZ3axgetZ8i0twwE8CNT9+4lmf45z1VpHOMwUjf1GL5WjTeJWHBKlkPnpHJZVBwS3/bxz9J0bp9QbBXb8hpThiNUqiRXI+uzoFSBh/1tDiWggVYi9W5BzQs9caB6FlR7XFO8JhyHHYD5f7QMjRinrrQiV7tWaeAnwntmmhZLBu+BUCw/nRPGbTCK3bd2uxnHzb73NlSqY2ZWUUJCBwZy7xhOUVwBPJAeOC81LNT5wm0Jq4OhCyR/idaGIC81KS2Z4NL9WV8aba9jYqUbSLoMUkTSGDn69bWyc2u1HHjlUEhha1wNN1fBUk/Bw6ZxitMV5jv7o5Kt5rxkYgo+wHwS+T1THTht54LYOAf2jOzyCdLPl0XVG7kw0RH0BKXJVOE4iYxmI82p0M3wWt71CVJmCVA3QvbuvynCMoYDIXIh96XkTm7AmgfXajXIXnEqOaLBLhgBCAV27WQU0osqpsO3YjwsRDUXAeo3vUparOKbFKEkuxhehnwrCNFQHSGMDF3IB62nz1N6md/kOU3uA7RJ8h+13zaKCOf2gccs1K7BEgcYNpPSw50wCVnMIAJWdMZXgDnig2TyXc/+Bhoqqfo2OqIwLH0SL4Yy9q+VM9Br2JW4SIX1JScrYdFlt2MvQR7DPDIfUMLohrRrMb5sQcRcptY402nMl5baT1Ny1XVR7vfUmI+GzuSyAj2Bhg0wV/OvBE3dJn+tWvDtVebv3G4ubve0I0feVUV6dclSYWPSqIu2sybuGXNfVXCfbhb2+AYyzZbOPFVy7BHXKBRWZn2JFsErHl5H93FPONe09ajdVnXV/JbyCX+wCiad0/TbCmlJTdqBd7QqyEYIyYUcOqu6mLcyc8ZPjARsnlJsRyfS3KZD+ZUqH2H9PV3V70hva82dSiiZ6TPnlYjn LT/DbK5J DUeGYkfdMqjMUqOS0kFp+SP1hkc3orbMMNlSFwY9CRNJ+4nswlex6SPPU3IhULEc95fvlA7H276bcYDxkg0ATLe6gZbR3cZ3XYO8JVylAfl4FOmC23squklHxSjY1b+XWjSIjqLYWSIvDOCfVd9f0nWl/fzLhWPbvm4b39YFFp74WkHBoxWUSJmFM1XO8GV2/G0l2LHHfAHh3N05NW8mfjbYBxn7glqAqnebHCG+loy/Y3xBLFWfvdyMwNEEUWNLlbv5nNqbWp4KJmo+rekpSR2OS5Lkx9BcsbzJ3uPNM4TZjuhhdjka6aNLRO8Tq46IEco5l3/cCjLyX9NovpvzTnwB/N/hij/MXEeaPmVw/PZLqNXf12dP9qNxaaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, when KASAN is combined with init-on-free behavior, the initialization happens before KASAN's "invalid free" checks. More importantly, a subsequent commit will want to use the object metadata region to store an rcu_head, and we should let KASAN check that the object pointer is valid before that. (Otherwise that change will make the existing testcase kmem_cache_invalid_free fail.) So add a new KASAN hook that allows KASAN to pre-validate a kmem_cache_free() operation before SLUB actually starts modifying the object or its metadata. Acked-by: Vlastimil Babka #slub Signed-off-by: Jann Horn --- include/linux/kasan.h | 16 ++++++++++++++++ mm/kasan/common.c | 51 +++++++++++++++++++++++++++++++++++++++------------ mm/slub.c | 7 +++++++ 3 files changed, 62 insertions(+), 12 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 70d6a8f6e25d..ebd93c843e78 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -175,6 +175,16 @@ static __always_inline void * __must_check kasan_init_slab_obj( return (void *)object; } +bool __kasan_slab_pre_free(struct kmem_cache *s, void *object, + unsigned long ip); +static __always_inline bool kasan_slab_pre_free(struct kmem_cache *s, + void *object) +{ + if (kasan_enabled()) + return __kasan_slab_pre_free(s, object, _RET_IP_); + return false; +} + bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip, bool init); static __always_inline bool kasan_slab_free(struct kmem_cache *s, @@ -371,6 +381,12 @@ static inline void *kasan_init_slab_obj(struct kmem_cache *cache, { return (void *)object; } + +static inline bool kasan_slab_pre_free(struct kmem_cache *s, void *object) +{ + return false; +} + static inline bool kasan_slab_free(struct kmem_cache *s, void *object, bool init) { return false; diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 85e7c6b4575c..7c7fc6ce7eb7 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -208,31 +208,52 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, return (void *)object; } -static inline bool poison_slab_object(struct kmem_cache *cache, void *object, - unsigned long ip, bool init) +enum free_validation_result { + KASAN_FREE_IS_IGNORED, + KASAN_FREE_IS_VALID, + KASAN_FREE_IS_INVALID +}; + +static enum free_validation_result check_slab_free(struct kmem_cache *cache, + void *object, unsigned long ip) { - void *tagged_object; + void *tagged_object = object; - if (!kasan_arch_is_ready()) - return false; + if (is_kfence_address(object) || !kasan_arch_is_ready()) + return KASAN_FREE_IS_IGNORED; - tagged_object = object; object = kasan_reset_tag(object); if (unlikely(nearest_obj(cache, virt_to_slab(object), object) != object)) { kasan_report_invalid_free(tagged_object, ip, KASAN_REPORT_INVALID_FREE); - return true; + return KASAN_FREE_IS_INVALID; } - /* RCU slabs could be legally used after free within the RCU period. */ - if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return false; - if (!kasan_byte_accessible(tagged_object)) { kasan_report_invalid_free(tagged_object, ip, KASAN_REPORT_DOUBLE_FREE); - return true; + return KASAN_FREE_IS_INVALID; } + return KASAN_FREE_IS_VALID; +} + +static inline bool poison_slab_object(struct kmem_cache *cache, void *object, + unsigned long ip, bool init) +{ + void *tagged_object = object; + enum free_validation_result valid = check_slab_free(cache, object, ip); + + if (valid == KASAN_FREE_IS_IGNORED) + return false; + if (valid == KASAN_FREE_IS_INVALID) + return true; + + object = kasan_reset_tag(object); + + /* RCU slabs could be legally used after free within the RCU period. */ + if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) + return false; + kasan_poison(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), KASAN_SLAB_FREE, init); @@ -242,6 +263,12 @@ static inline bool poison_slab_object(struct kmem_cache *cache, void *object, return false; } +bool __kasan_slab_pre_free(struct kmem_cache *cache, void *object, + unsigned long ip) +{ + return check_slab_free(cache, object, ip) == KASAN_FREE_IS_INVALID; +} + bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool init) { diff --git a/mm/slub.c b/mm/slub.c index 4927edec6a8c..34724704c52d 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2170,6 +2170,13 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bool init) if (kfence_free(x)) return false; + /* + * Give KASAN a chance to notice an invalid free operation before we + * modify the object. + */ + if (kasan_slab_pre_free(s, x)) + return false; + /* * As memory initialization might be integrated into KASAN, * kasan_slab_free and initialization memset's must be