From patchwork Wed Sep 11 06:45:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13799745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C3E0EE0211 for ; Wed, 11 Sep 2024 06:45:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F54A90000D; Wed, 11 Sep 2024 02:45:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A3E16B0384; Wed, 11 Sep 2024 02:45:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 792C490000D; Wed, 11 Sep 2024 02:45:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 586736B0383 for ; Wed, 11 Sep 2024 02:45:56 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 039B6161038 for ; Wed, 11 Sep 2024 06:45:55 +0000 (UTC) X-FDA: 82551522312.09.1A38792 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) by imf05.hostedemail.com (Postfix) with ESMTP id E29CE10000A for ; Wed, 11 Sep 2024 06:45:53 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=WuLi9yvr; spf=pass (imf05.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726037050; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wZNuj6wBkLP0SnWxYyCLS6aj5zno8YqWIFsVBbPmmqg=; b=tkdQfAh3Q7XAUXRy3bXc9qSc5hXVRQvHfVmLhkjqYEr6AucYEr/33D3nq4FvCQniKEF2Wn 62Ao1cL5c7/iJ9hsDu7bigDVjLEbLxXNYAO+bNYLMgbCjDSX9qSB0ptgcnFJRfWVIjkuW/ QFxuNzQ8eEW1NG55jIogssCjobrzZro= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726037050; a=rsa-sha256; cv=none; b=hxUmzcjHPuVb8K4igKkJTQgAQvPql3y61GCHVmqBRbySVnbZQ6TsYE0o6zXedmEsIwiVNQ MTu0zk7ccHmVqtwTTn6w5Wt+n3xmoyQs4iA1mmA6ZbmiH59mkQeZ8NmhVQ3GoSDuv1lp2w XIILlohquNFlytovN+YAJ6at42TrZV4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=WuLi9yvr; spf=pass (imf05.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726037154; x=1757573154; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dM2adnqSdhHnHwASvqcFoX4V3rfZ14QoQNpiYKbsRgQ=; b=WuLi9yvr32gFRzOLgX151Fjk0jHCZYCi2lwa75m6uX0rFBOPd323K6ZX WG3WQb9kDX0saZoTBZUJOC2NmJ1YsADi6y17QE/8cJcSUsq1qQ3XDqnhn OwF8qmu4cUverB2XjD5L2tAUlEmRV3SI7TbIEM7yPX40tnTe02PD3w9Aw +RVCJ9KcPg2lsDwGISteDaioWcSGWgrudXabxRgn38qJISX1mFLsCk60B 0Q19cMBy6oVXswoilJTBX1WHNzXZI965H1S/FaQqOaDR1OqYsZX2qQsX1 7z/qWQbg4DpdbRoDcYBFLu36wDBFngaLVaCNQrVeHqWWyPKl3+wH4WiDh A==; X-CSE-ConnectionGUID: jgNEGqzPSGK15Ugn3hUElQ== X-CSE-MsgGUID: WB0rTQ+3R2ORsBTyahpbiA== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="36172980" X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="36172980" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 23:45:52 -0700 X-CSE-ConnectionGUID: wcILiiZ3QmaqfhSIVIcfHw== X-CSE-MsgGUID: fx8JhJANRZCRTz/LvKdSHg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="67771485" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa007.jf.intel.com with ESMTP; 10 Sep 2024 23:45:41 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich , Alexander Potapenko , Andrey Ryabinin , Dmitry Vyukov , Vincenzo Frascino Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v2 1/5] mm/kasan: Don't store metadata inside kmalloc object when slub_debug_orig_size is on Date: Wed, 11 Sep 2024 14:45:31 +0800 Message-Id: <20240911064535.557650-2-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240911064535.557650-1-feng.tang@intel.com> References: <20240911064535.557650-1-feng.tang@intel.com> MIME-Version: 1.0 X-Stat-Signature: qs57n6h88ctryithy6xcu3ciakr8eyps X-Rspamd-Queue-Id: E29CE10000A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726037153-326967 X-HE-Meta: U2FsdGVkX1/5qAvM0ZmKkShpjefwMJ5Ox3DaQfX+Uan05EzeZH5zGEwq++As7SrUzueOac3AMEOS+ifQbUlqHBg8lFodOBVold3DplGFRAg9DQLVitNZ8DCkaQ6jk1w//34iNLUVCFuudiy5lendob5qxI19GHYKjtQVnKjpS9LMQAdpGIzDTbFvZsUbRUB2JVapVOmbuepoYZWcOFxf/SWy1Or73wasisq/Fum2OtdKNrITJOzxUfe6UvdyO67E5Q4S00kGn97x1erFRuV/NBSZ0lYTBQ5yExlrtaNML44jr/9tlRfZ9FLJBcpAyXsWuTWP/0gCwoyoALIg4V3fuBYSAIVxFXXZ8Z7TV7Xq1a3qGAQSN3+B/AakVK2hjPuNs/PM7+TmuHkf5ggp48ttb7Z4wVYvrFv4lD0Mmn4rC+q2Ncnml2RucorUY27q/YpbwMeQZJtKetbGZ52FMHkDWGI1sI586itiK0JfJBdQs2AKmU+RjjjVIYt8/50lIxoPilHibf+usXbbEZiX79hjp8XWVy7yYeQvBIIVzTEi2qDZdqjHfyLrFgTI2Yx98DF5ZNKfPDE0MtXK1F1WCh63m5gGOG8WgXp744dXhgUyS3NZvFvyDZSr9R6EUxYkLaTw1kl/tRrlkxTX2D0Q4s0aRTzv/IJqpC9ujVpdCGV8Ruv5fPvpJE1sFjuvI2xeqFlkqncB5TXCYKsqamF5JedR9kL8I1zQnRgq7+20AaJTFLr9xtHWFNns+O04+yiyUMEzMyOPMemlcu1XM8sv3I4dJW5cj6NqjPVQhGoymEKnY/tpTHEGliMA7rNP5QkS9FMHR2VcuuxFNnxjZyf2QjvsZuXwQqFkCj5E9Ya9TcAfRrLDGKWMbRZOSuttJxRsmTYKONrB5GLvwdx/6u2eTjPjxtuyhAjk3YLOfIIOVKCmo4yXsQDADADzJO90yIc6eRUr6rvX2ak93PUylT2eQUv M+i0YL7O qpa0uHcGHpxQyQ6lbRKd/2oUH/2ZBBHOzvVkPEeJ/2VElQwFOW0GT/lJH9Pzq2P3bAQRzSymNnE234KCcSd1YAg8irL7Mh9/pvGXbU5FFlsa5qmByBbQgGnD5FMEoJkKIEcmJ61IhIIfPn8QqRdwlr060j7ezWyv3YqRmGnnS0Vfyl0vEw/6aXMhumAxPETgdQWCWx/pJWTnw9G5QcPDEO48HR8F0M0WBLi8kvNWSi0rkiaa3T3ijr1esc/NdJrE4RnpAfZnXW0aZao5JmAfq6WEGWpgFEAATb9H5zifiRviYt8af1APwtSC++AuAMwvSub0w X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For a kmalloc object, when both kasan and slub redzone sanity check are enabled, they could both manipulate its data space like storing kasan free meta data and setting up kmalloc redzone, and may affect accuracy of that object's 'orig_size'. As an accurate 'orig_size' will be needed by some function like krealloc() soon, save kasan's free meta data in slub's metadata area instead of inside object when 'orig_size' is enabled. This will make it easier to maintain/understand the code. Size wise, when these two options are both enabled, the slub meta data space is already huge, and this just slightly increase the overall size. Signed-off-by: Feng Tang Acked-by: Andrey Konovalov --- mm/kasan/generic.c | 7 +++++-- mm/slab.h | 6 ++++++ mm/slub.c | 17 ----------------- 3 files changed, 11 insertions(+), 19 deletions(-) diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 6310a180278b..8b9e348113b1 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -392,9 +392,12 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that it can * be touched after it was freed, or * 2. Object has a constructor, which means it's expected to - * retain its content until the next allocation. + * retain its content until the next allocation, or + * 3. It is from a kmalloc cache which enables the debug option + * to store original size. */ - if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor) { + if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor || + slub_debug_orig_size(cache)) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); goto free_meta_added; diff --git a/mm/slab.h b/mm/slab.h index f22fb760b286..f72a8849b988 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -689,6 +689,12 @@ void __kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab) void __check_heap_object(const void *ptr, unsigned long n, const struct slab *slab, bool to_user); +static inline bool slub_debug_orig_size(struct kmem_cache *s) +{ + return (kmem_cache_debug_flags(s, SLAB_STORE_USER) && + (s->flags & SLAB_KMALLOC)); +} + #ifdef CONFIG_SLUB_DEBUG void skip_orig_size_check(struct kmem_cache *s, const void *object); #endif diff --git a/mm/slub.c b/mm/slub.c index 21f71cb6cc06..87c95f170f13 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -230,12 +230,6 @@ static inline bool kmem_cache_debug(struct kmem_cache *s) return kmem_cache_debug_flags(s, SLAB_DEBUG_FLAGS); } -static inline bool slub_debug_orig_size(struct kmem_cache *s) -{ - return (kmem_cache_debug_flags(s, SLAB_STORE_USER) && - (s->flags & SLAB_KMALLOC)); -} - void *fixup_red_left(struct kmem_cache *s, void *p) { if (kmem_cache_debug_flags(s, SLAB_RED_ZONE)) @@ -760,21 +754,10 @@ static inline void set_orig_size(struct kmem_cache *s, void *object, unsigned int orig_size) { void *p = kasan_reset_tag(object); - unsigned int kasan_meta_size; if (!slub_debug_orig_size(s)) return; - /* - * KASAN can save its free meta data inside of the object at offset 0. - * If this meta data size is larger than 'orig_size', it will overlap - * the data redzone in [orig_size+1, object_size]. Thus, we adjust - * 'orig_size' to be as at least as big as KASAN's meta data. - */ - kasan_meta_size = kasan_metadata_size(s, true); - if (kasan_meta_size > orig_size) - orig_size = kasan_meta_size; - p += get_info_end(s); p += sizeof(struct track) * 2; From patchwork Wed Sep 11 06:45:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13799750 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FFACEE0212 for ; Wed, 11 Sep 2024 06:46:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F054940008; Wed, 11 Sep 2024 02:46:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 19F2B6B0385; Wed, 11 Sep 2024 02:46:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04195940008; Wed, 11 Sep 2024 02:46:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D23B06B0383 for ; Wed, 11 Sep 2024 02:46:18 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 435A4160FD1 for ; Wed, 11 Sep 2024 06:46:18 +0000 (UTC) X-FDA: 82551523236.27.CA0C28C Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) by imf28.hostedemail.com (Postfix) with ESMTP id 2BE86C0018 for ; Wed, 11 Sep 2024 06:46:15 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=iapW7k+N; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726037072; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zm6mEmpihmiTwCFfdHniUQp/gJG3DwrB6e0k4OJohiA=; b=fgtjbxUvAAsoHm5FIxUHbgnO+7fPLBIryvfWk/4xRiMXRDp2WJXnEKODBvW6jH3mr8317j RV516wm6VbQSl6ANGhO7QjE9NjPfkZst5eqqvCbKcjIJE9lDs6WYYM75DeH8rsfX3+etry 6ALrb603KapNyWtnHGBjDtC9YMJ2rFY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726037072; a=rsa-sha256; cv=none; b=N4xURwmtcZtfQpwVwx/y8mS6jRYMEjKinThHjeU5hkY8A0+CjmeH91UqRYTLbnhGN206HK lijTkSrTMLdlYA0/IG6yfiQLL1C1PJv4liVtaCgYsbs+8dUYgKQPHLh0r4KvEzJGerxe3P 0r3kzr0SF/VGbQylkWyr0X9v59xoi3U= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=iapW7k+N; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726037176; x=1757573176; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Q3PpnMMnKbBe2tGhLr5Y8UQvjMtp65YK3pg5VOQk5Ag=; b=iapW7k+NttK2SdMtxvvZRzfXJ/Q4aRYStPnEDULrkcsgq8deXBQikc9Z Q10Cr+nvFpfSKurkBdL096JQ4ecz+2Lgd1DT20W6TSjkJzvEtoink+0HF LgDMH1ynq/wYzPPoD3C3UXTVpo0CAM/E1KopSVc2O/8MXfC38Nm5yipEd Vbtjr10EfPSjGep2UvEYd4BNVW66LoMn79hsqXTqag7Huvv/nGX2TjOh8 1F+uYOPAw06GMTqlFM3OEbPA+BDGQlXFZpB5wPk6syCIma7BCfQYmEYot nYK9IBSMCSxdr8iuQboZErDRjY8WEYkHQj/Lfo/lTE8Rso1Y08nJu+/qL Q==; X-CSE-ConnectionGUID: GZ7Pal1/Qr+Vbki9wkTEAw== X-CSE-MsgGUID: xZ1okREvRiqilOGubE59GQ== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="36173005" X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="36173005" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 23:46:00 -0700 X-CSE-ConnectionGUID: TlC0q2XjRdmz/RXKu5H2YA== X-CSE-MsgGUID: ouEYEDaZRMK+unGJAsKkGQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="67771491" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa007.jf.intel.com with ESMTP; 10 Sep 2024 23:45:50 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich , Alexander Potapenko , Andrey Ryabinin , Dmitry Vyukov , Vincenzo Frascino Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v2 2/5] mm/slub: Consider kfence case for get_orig_size() Date: Wed, 11 Sep 2024 14:45:32 +0800 Message-Id: <20240911064535.557650-3-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240911064535.557650-1-feng.tang@intel.com> References: <20240911064535.557650-1-feng.tang@intel.com> MIME-Version: 1.0 X-Stat-Signature: x4w8tu96ewnmatan5qziqc3fkwqy6t7f X-Rspamd-Queue-Id: 2BE86C0018 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726037175-566057 X-HE-Meta: U2FsdGVkX1+N05G0jR+gRs4WXynk+BhjRjnk+4im1VFzAUAdqgVjj3jVs4N5LPflnlnzHTXcDUPHd/hjmvl0sb4MxiZvIjDYewrppjchfuugSytLZOMLIaUrqBFSVk3xPRi/jP1s3QEsDB9aKRuUBvQp5VXb97NIfVMa3C+DTu4WkqlbGZhQXUtVeWFARrjqTCsdZARWcmB8SDdGwAve+Cy//eZDfhjxBPxNFckDNqx6efFWiH1+LQwdV5ec25i+NlC95DZQwv7nelan7zno0nCZfThOls6m8fWJoWabYQfb7SYmvPi0pFox0gb1jeR7xA2tAUupmY43pvxOB0vK37+tB8WwVZkCSRqMT1HAImhB2Zks1nqDD89InyROiaskKr6Xn02V+YGlahpp0M9wbz1msOdTVLWS7XYWmaVKQu5dRD+P64hy9i5Y4ayJFU+mQQrBInQGGL0h+/IHItETnJ1fhcqy6Y+3IcR8agTwNfY7tPQVvQkEFI9QVpByPzvNhrvsS+qRPA4aHYMp7Uq23izCQTAGfrM4ztEnDz1ZVgw8ybFiPPqhs9ra0QPgMCZfx9ke6YiSmlpkCqau8OY+wsJ//PCoXULfKoOtcey1GsMgRi8z5uZGCuupuxEhBBTE3AC8wHDYFd4GdeAaXG7d6/ocLAl6dJfwUPTmWUqEdQKU9lhu0fYpRZSZ0A0g3kiTPdPEPhEn6pBQ58XRv/rb7eue6J/LjzXz2y86BINHFd+ZClnSQDxQkezEXtnx4RTcjs4LM5RXo9PMxr3lOa+gee3usyW3j9XT89k/ax0BwRe0beZR5BQulQBEqO7/OgKrQ51cwB8l9zf47i6wIEzn3KgaQNowRKX6TWXg19lV6oV3WetNiAaV70i92qUwT9xiP9SkJyXM7Y1Ctyr98iSG2ZomXWvQHVBMlE0kvHPtmfmVEiM3Z2yVT8S+fgnJuOK2EOSUqpMNX7GTkDwsEob D7o7pSbI WZ+fz6Zf/sj+BuS3OCK5sa5XNX460wqH5hdh0RwcT5Rb0LlF3E4iIo/ROXYD9IpKK3q94dhisi6GusiwBHFhNdlWHn9n6PCS/EpVRbiiUOZ1ivGThIurUrNcPlg7pLOmlRRIO2IzrjRwsSBx12dL/+XlrjvYnf+rD0ICVBIHDq631Uu4/Rtv8R2M0+VihezwtXE7G2LbF5yowq7cv40upmpTs93lZ4wbLscDDvzQxpMEucJtDO/Zn3FZL77v1aoeO9kgNJJ0MmizOGnwnFo1Wxu5ymOchl+08pMDerJ/9yPRTuQ4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When 'orig_size' of kmalloc object is enabled by debug option, it should either contains the actual requested size or the cache's 'object_size'. But it's not true if that object is a kfence-allocated one, and its 'orig_size' in metadata could be zero or other values. This is not a big issue for current 'orig_size' usage, as init_object() and check_object() during alloc/free process will be skipped for kfence addresses. As 'orig_size' will be used by some function block like krealloc(), handle it by returning the 'object_size' in get_orig_size() for kfence addresses. Signed-off-by: Feng Tang --- mm/slub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slub.c b/mm/slub.c index 87c95f170f13..021991e17287 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -768,7 +768,7 @@ static inline unsigned int get_orig_size(struct kmem_cache *s, void *object) { void *p = kasan_reset_tag(object); - if (!slub_debug_orig_size(s)) + if (!slub_debug_orig_size(s) || is_kfence_address(object)) return s->object_size; p += get_info_end(s); From patchwork Wed Sep 11 06:45:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13799752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0F0FEE0211 for ; Wed, 11 Sep 2024 06:47:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 65C2194000A; Wed, 11 Sep 2024 02:47:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 60D986B0387; Wed, 11 Sep 2024 02:47:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ACEC94000A; Wed, 11 Sep 2024 02:47:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 299A16B0177 for ; Wed, 11 Sep 2024 02:47:16 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CDF27812D7 for ; Wed, 11 Sep 2024 06:47:15 +0000 (UTC) X-FDA: 82551525630.16.E327C40 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) by imf28.hostedemail.com (Postfix) with ESMTP id B24ACC0005 for ; Wed, 11 Sep 2024 06:47:13 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="YQO2GKW/"; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726037129; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/S8vF7imCQy4zA2SHeFyV8/wC75Gy306lr1UXEKj3dY=; b=XH6tpPV9tSirwyVQxmb0NYjWjmpUwEZCGknp99OnLKxNFl2tWBcTB8A4Yfem7D5yG5kBcF O5GBm8zOYsVXf55j2hFfFpOzQTCxsk9JkwERIs0iVCvgyVLkpksYTBpd+5ty6WTKobMra0 RAw2behijrXDU10+WD8uRjQF10nm2QI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726037129; a=rsa-sha256; cv=none; b=x34FxcK9O1XIvijM2WgYW+3nrOTB/B5CjBKyi9AJjPl+Egw8WVvmMMS6OjWZB1lucqjSIT 0J4BwzUMt/tRHr1SDM7/1i7qjwIYJNDyzQ8fUqzCnLRsUFDjbMfxJ0f10Yud/xo6W4ut3t 1rfkQIHtHT9fE+l9tAdgp6eRGxiG5aI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="YQO2GKW/"; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726037233; x=1757573233; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ag0o94U2BMb2lFyFC32W4EFoRg3K+ZMunlGHmbIXB5s=; b=YQO2GKW/G5sciCsnPuQWVZtAo3uzFc9OL2TOCMyogI5GozM/KIENJW98 a0mf/ecRaXFDvU60whm1dv20bsx05ZqVaO0/JeW4/VaQjs5Z2CtsOCAjU SGEMnN5vjPmEVDBtY1yLfvIM432Vfh5Cm1jPGZEgy/TB0BWuYAAvvDWE+ y9gO8LtONaR1GhihLYgcyHifxdt/x9MvEETmYjUZU52SmBRdcY5Y2zBol P3A6qvEiOryMqlV7INiE38DOWXChJtztTmBqGqYAT6H/lBTsx7Dr8suEk gZR0+JY7YOujqbcBlfn50i9YbXDIPhF8OojboxuYKQ3RIYEmawyFhzFZO w==; X-CSE-ConnectionGUID: PkpNX5SrSNKhFZ6ABojImg== X-CSE-MsgGUID: xKjUeJfMTgK6Ch99iPF+NQ== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="36173018" X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="36173018" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 23:46:11 -0700 X-CSE-ConnectionGUID: IgyzjmIvSCmBHDIIIAmEIw== X-CSE-MsgGUID: u/rfH00VRWC+1lv/88CKcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="67771497" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa007.jf.intel.com with ESMTP; 10 Sep 2024 23:45:55 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich , Alexander Potapenko , Andrey Ryabinin , Dmitry Vyukov , Vincenzo Frascino Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v2 3/5] mm/slub: Move krealloc() and related code to slub.c Date: Wed, 11 Sep 2024 14:45:33 +0800 Message-Id: <20240911064535.557650-4-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240911064535.557650-1-feng.tang@intel.com> References: <20240911064535.557650-1-feng.tang@intel.com> MIME-Version: 1.0 X-Stat-Signature: 3a5mtsz5j9fogo3y7ugrto9o33tkhumh X-Rspamd-Queue-Id: B24ACC0005 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726037233-477803 X-HE-Meta: U2FsdGVkX19F7PKwySDvIT93Rtl46ZM4AVOcBbLqR7Uf5SgFnNLnwVEXDNLq5k7roYQDiRmIHAJAcOjuSf122+e3IPgsV7XIH5S6/dbsa04wPEJxBUMpzVKpVFLxoCJUFQQ+fcTu1Fo1pRpnKA1/+iZUoAeUJYtau5GtqemxNBkh0s/43Jr0nn+jKTv0TfccTY0xHOTbzy5x9Jvzp/N6ResgkdkR1XJ13Pr5Nlfq9h2Hbg/t0+scCoDuNurUv4421RBog9sgC198qDy42RyhX2SAnSjDjo55AeEzsCI/blNMgTroEa/mVTOIR2JGVnVBWMuI8fEEvSxzZyfSRpapVS7kj3g5UBw2wbRcfvXb0ynzpOrUsaENGiuoT957qynGQ+WfjGsxny/rJaQ5N7iKN8UE/REydZnkDHqpVmtqvSlE6QOQUP99kcP1Sf790OPnqLfSfIAeKxAU8j6hWeSDBMz3ZqO7gbCyN6G7C0HtXyhV4HH+sGNWlkbXNOamDPju7NNZ2xAqY+XVCFG3XAZpYAP0kpkZU58+Xwrv1Sz63a9sfoYY11ApYEVLst6mc01lzN5SizNDboMw13tKBLePJbKyl3Bcm+mN9OVg3mYByGFyzzzqNss2oYPx/tNTvhEfG+wveC9DDnRD8HyD6iX7R1VFuCJ5KDOoagSqyCwbvgvY0tAj98cETKm39k/V0feOHOtEnN4DsKj8igvHIIjC+4klGHD/+XY8ifxiII+LyOP+UGAA506SCRREHluboIwG/9nXBFskt11Cu3dc09KRGeCb+tMbp1U3psUa933Ta44Cu48tv61gly7EPQwE1HKPsp+/iI0yz39gSCWBjum99XnPTiwd1oSSbbkHOgiq9WxHN1WfC8QWZqln+O2viVjwCXoUvZOOOfmWBQpFuu25GdE32LuwKCYlETQDrIi3iTIw5gpBcq66MNQoYE03dnT0VwyrkESOAXUPh9yqChq j3Aefv8+ gCuo1OfWnEJBqPpNx7YKj6lV/6xdnbygtX24wgC3vaeiw7DRQsR8AaRTAZ7u5V6gFJiEtmEnS7QCUzKvl07i2V6G0WRKc5qQdlskDfPYyuBotxtYVo3o59ENiiNpvfbouzHXk2x4IpD0bK0m7yKKHrjGAvsPdRgxoh3b9PWJUrNeh5ViikOgRRJiBH2G4S8UX4FbWf38qFfchW7UPtEM2ZgQRJ3cjfNUaUdfANSFbQIYe+FxiSgrV7EAvX9tqGTGLT8jQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for the following refactoring of krealloc(), for more efficient function calling as it will call some internal functions defined in slub.c. Signed-off-by: Feng Tang --- mm/slab_common.c | 84 ------------------------------------------------ mm/slub.c | 84 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+), 84 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index af6b14769fbd..5734b61a106f 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1185,90 +1185,6 @@ module_init(slab_proc_init); #endif /* CONFIG_SLUB_DEBUG */ -static __always_inline __realloc_size(2) void * -__do_krealloc(const void *p, size_t new_size, gfp_t flags) -{ - void *ret; - size_t ks; - - /* Check for double-free before calling ksize. */ - if (likely(!ZERO_OR_NULL_PTR(p))) { - if (!kasan_check_byte(p)) - return NULL; - ks = ksize(p); - } else - ks = 0; - - /* If the object still fits, repoison it precisely. */ - if (ks >= new_size) { - /* Zero out spare memory. */ - if (want_init_on_alloc(flags)) { - kasan_disable_current(); - memset((void *)p + new_size, 0, ks - new_size); - kasan_enable_current(); - } - - p = kasan_krealloc((void *)p, new_size, flags); - return (void *)p; - } - - ret = kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _RET_IP_); - if (ret && p) { - /* Disable KASAN checks as the object's redzone is accessed. */ - kasan_disable_current(); - memcpy(ret, kasan_reset_tag(p), ks); - kasan_enable_current(); - } - - return ret; -} - -/** - * krealloc - reallocate memory. The contents will remain unchanged. - * @p: object to reallocate memory for. - * @new_size: how many bytes of memory are required. - * @flags: the type of memory to allocate. - * - * If @p is %NULL, krealloc() behaves exactly like kmalloc(). If @new_size - * is 0 and @p is not a %NULL pointer, the object pointed to is freed. - * - * If __GFP_ZERO logic is requested, callers must ensure that, starting with the - * initial memory allocation, every subsequent call to this API for the same - * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that - * __GFP_ZERO is not fully honored by this API. - * - * This is the case, since krealloc() only knows about the bucket size of an - * allocation (but not the exact size it was allocated with) and hence - * implements the following semantics for shrinking and growing buffers with - * __GFP_ZERO. - * - * new bucket - * 0 size size - * |--------|----------------| - * | keep | zero | - * - * In any case, the contents of the object pointed to are preserved up to the - * lesser of the new and old sizes. - * - * Return: pointer to the allocated memory or %NULL in case of error - */ -void *krealloc_noprof(const void *p, size_t new_size, gfp_t flags) -{ - void *ret; - - if (unlikely(!new_size)) { - kfree(p); - return ZERO_SIZE_PTR; - } - - ret = __do_krealloc(p, new_size, flags); - if (ret && kasan_reset_tag(p) != kasan_reset_tag(ret)) - kfree(p); - - return ret; -} -EXPORT_SYMBOL(krealloc_noprof); - /** * kfree_sensitive - Clear sensitive information in memory before freeing * @p: object to free memory of diff --git a/mm/slub.c b/mm/slub.c index 021991e17287..c1796f9dd30f 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4712,6 +4712,90 @@ void kfree(const void *object) } EXPORT_SYMBOL(kfree); +static __always_inline __realloc_size(2) void * +__do_krealloc(const void *p, size_t new_size, gfp_t flags) +{ + void *ret; + size_t ks; + + /* Check for double-free before calling ksize. */ + if (likely(!ZERO_OR_NULL_PTR(p))) { + if (!kasan_check_byte(p)) + return NULL; + ks = ksize(p); + } else + ks = 0; + + /* If the object still fits, repoison it precisely. */ + if (ks >= new_size) { + /* Zero out spare memory. */ + if (want_init_on_alloc(flags)) { + kasan_disable_current(); + memset((void *)p + new_size, 0, ks - new_size); + kasan_enable_current(); + } + + p = kasan_krealloc((void *)p, new_size, flags); + return (void *)p; + } + + ret = kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _RET_IP_); + if (ret && p) { + /* Disable KASAN checks as the object's redzone is accessed. */ + kasan_disable_current(); + memcpy(ret, kasan_reset_tag(p), ks); + kasan_enable_current(); + } + + return ret; +} + +/** + * krealloc - reallocate memory. The contents will remain unchanged. + * @p: object to reallocate memory for. + * @new_size: how many bytes of memory are required. + * @flags: the type of memory to allocate. + * + * If @p is %NULL, krealloc() behaves exactly like kmalloc(). If @new_size + * is 0 and @p is not a %NULL pointer, the object pointed to is freed. + * + * If __GFP_ZERO logic is requested, callers must ensure that, starting with the + * initial memory allocation, every subsequent call to this API for the same + * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that + * __GFP_ZERO is not fully honored by this API. + * + * This is the case, since krealloc() only knows about the bucket size of an + * allocation (but not the exact size it was allocated with) and hence + * implements the following semantics for shrinking and growing buffers with + * __GFP_ZERO. + * + * new bucket + * 0 size size + * |--------|----------------| + * | keep | zero | + * + * In any case, the contents of the object pointed to are preserved up to the + * lesser of the new and old sizes. + * + * Return: pointer to the allocated memory or %NULL in case of error + */ +void *krealloc_noprof(const void *p, size_t new_size, gfp_t flags) +{ + void *ret; + + if (unlikely(!new_size)) { + kfree(p); + return ZERO_SIZE_PTR; + } + + ret = __do_krealloc(p, new_size, flags); + if (ret && kasan_reset_tag(p) != kasan_reset_tag(ret)) + kfree(p); + + return ret; +} +EXPORT_SYMBOL(krealloc_noprof); + struct detached_freelist { struct slab *slab; void *tail; From patchwork Wed Sep 11 06:45:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13799749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2A74EE0211 for ; Wed, 11 Sep 2024 06:46:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FC5F940007; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4ABF36B0384; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3759A940007; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1A4D96B0383 for ; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D04FE1A0F6C for ; Wed, 11 Sep 2024 06:46:15 +0000 (UTC) X-FDA: 82551523110.02.4351113 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) by imf28.hostedemail.com (Postfix) with ESMTP id AB033C000C for ; Wed, 11 Sep 2024 06:46:13 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=mAtxeU2f; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726037069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sQnDdI6wKIFdlJ+Rjok4zmD0BtF30gqRHIfs4m6lyC4=; b=tNe3zYDOQ5p7k7dqMAS+7S8lmXbpOr83N3K4l51/JcgeulzdBUrkB9w6K7TufYppnWc3z9 3ExBPPhh7+ZTAi39C6HhrLJolYVXgPLaMSg95PR9ae+xxSgNbbBVteT/UPhO6WJFncBCWn Bu4EAdVh9TIScBX1/pio0i911D1Y+sQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726037069; a=rsa-sha256; cv=none; b=xI6lgLF2G2wQgtyh2KuV3p53pifE0lgmzIKZToY/SIKZqjyAFsqDCHkrqbxzNKufDVIpFL uRoCJB923IzUqKUHPwQX+dV2gNkFok4+t7GVZdmMW7CFJnFsoHmj0x8unqlAzJ0sCAYZD8 nnuBYkBCLk3Pv8x4g/qQ7fKdYoDjeG8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=mAtxeU2f; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726037173; x=1757573173; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZHBD5nxjCn39VoHzOY+ge2IVDv8unPJPEMRErbHaUA4=; b=mAtxeU2f8Z7WmpquHWY4nyvM1zqRIlqVBl49pGlAl5nPXPdlfSbdv/+M iVjgPTQvxuzlyZYljW594Jgel6PO1SteSfp74AcjfwDE9JbvAECLNZukv btTJu2Zo9qeQRvqlhjlzmdrrILhyp8H/olExK9B0wwyaH+yE/ZYtNXw8r 2I8tTZ3m3HrDZTZBxIP7JzXRBwTL2CaGQ4hS6iamYjLA6QwIjbDc3UWh+ sW2oxqIxZr24oE0RGsoz8X2+00p3zeJ35hYcfmO7LVECPVl0yh0Dj9pOr Rxxn+Yk7t7bQfQsDaGZTWuzzqaPRp5DoJB4U+6I8K3GjwdDIHqBWTaAlH Q==; X-CSE-ConnectionGUID: z8vMWVA3RkuN4fenPk3J6Q== X-CSE-MsgGUID: fe5dW8WhTxeyMMZztKtQqg== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="36173032" X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="36173032" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 23:46:11 -0700 X-CSE-ConnectionGUID: v543qwycT2WvE2geGxwVvA== X-CSE-MsgGUID: Ll8xYtUdRRW3q2YxBIkspA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="67771506" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa007.jf.intel.com with ESMTP; 10 Sep 2024 23:46:00 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich , Alexander Potapenko , Andrey Ryabinin , Dmitry Vyukov , Vincenzo Frascino Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v2 4/5] mm/slub: Improve redzone check and zeroing for krealloc() Date: Wed, 11 Sep 2024 14:45:34 +0800 Message-Id: <20240911064535.557650-5-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240911064535.557650-1-feng.tang@intel.com> References: <20240911064535.557650-1-feng.tang@intel.com> MIME-Version: 1.0 X-Stat-Signature: s41ehnn6c4haw83cdxunuj1fnfo7dq6n X-Rspamd-Queue-Id: AB033C000C X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726037173-368182 X-HE-Meta: U2FsdGVkX1/5rrd+j5+lERaob69+eTF7rv0N2KLvyHLao+b0A7kqFZ+9JsQhnAPM6W13+biXEh+EHiUosjJpd7b2fXE1X5XhiYrB3cBHsZpWQB4AgkzUahOYkExP0z3uphahVvpYhvj8aw3oz7dJHAK5LKgQ3xRXW39AzscuWmaAwAXm8eBOyoMQOMW+cG7sGz4CvkJXCRP2fmpTTMfVRp1nZP+gFMzNxdvVLjLBw3nBqsDrbH12BQERdmRJsbB6KbRw8vFBCkUEk4vZ93yp5kGHdrsDbc3FQNpYOJfsubDA3pF84kjarMuDnHaARY2/EGWUxxzoUOGTkjNpYrKEWmJWZks6pmj1B1Nz8cRm2oSHguHBnW+JOowcwPKaQ9dOXR3IUeJuP+1DO58QLRtSXrHUesBgCoZ/LfOKpaEMiAzPieFBmSVOW3/0EheRo2ulxIGKUNFC2m82syWWjNTIwmMpsiawYSEaIJKXqHgH5w78VjTbJOO+Jis9m3ms+QnputAbrq+qqS4u8d2eecqP4CXXIEHl2X21brQUBCW80mGyPVGoTwCYucY+G9HoKtaGXxPTIkCJqS0L5Rjzdhltbz+r7WKWTmlwNvwYrY/4C9UT1M0iRIrv20WbJpl1MnOg5QSX8Cin1VGUPYcbvwszssx+2QbtkmiQ+pgMUOywJLfj3P6LsKewGvS0K7DIt8S70q7/hjbbBfnzVQ9G0LHUgHUsTa55R5O0D8NlPUVselQF8ZUGlg3zFqYT56z04z+SRleuUCoYq1JvJ0O6s5L+yVTaCE3L+Q7Bnk/+WjEGpQ5OO4GEieaS2fKz7uHlyiCQVPDD/9w0JN9qBreLsV1Yi35fdw1KgVcsF/IDUuaZfSQ+Y3c0tBxFl6hwGZDgDR171KQenEKhuPXrAt0zByKHjqgdPM2LRVyn0Xa4s57RiGiYlCOkPkz6GxJqvDa3DcdRVUziCxcHbnjcQeRwtup MKm+pIzj WCvBERKpVDoXMxi6hLhN/MtijjWpDf+SXXYtVzd3fHMYi3VQie39ZG4+NC2cdA61vb7DwQNH3ypSZLAvPLz1u1O+q6LGypFROymKeh8XyA7OI+IVznSbB51D+BAT7iBpSVniElcG5/oLyC28M8xYWAwQsLSUuHXrJUw5SQ9ojYVvIyo5RZR3mpNz86qzeh2ZuRDe2nXeA4bQALtg6jSC8Dx+ZcMj48ixQu6I++BQMOWvkZ5BXTDCwu7iKefsflDM3TAvb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For current krealloc(), one problem is its caller doesn't pass the old request size, say the object is 64 bytes kmalloc one, but caller may only requested 48 bytes. Then when krealloc() shrinks or grows in the same object, or allocate a new bigger object, it lacks this 'original size' information to do accurate data preserving or zeroing (when __GFP_ZERO is set). Thus with slub debug redzone and object tracking enabled, parts of the object after krealloc() might contain redzone data instead of zeroes, which is violating the __GFP_ZERO guarantees. Good thing is in this case, kmalloc caches do have this 'orig_size' feature. So solve the problem by utilize 'org_size' to do accurate data zeroing and preserving. Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang --- mm/slub.c | 54 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index c1796f9dd30f..e0fb0a26c796 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4717,33 +4717,51 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) { void *ret; size_t ks; + int orig_size = 0; + struct kmem_cache *s; - /* Check for double-free before calling ksize. */ + /* Check for double-free. */ if (likely(!ZERO_OR_NULL_PTR(p))) { if (!kasan_check_byte(p)) return NULL; - ks = ksize(p); + + s = virt_to_cache(p); + orig_size = get_orig_size(s, (void *)p); + ks = s->object_size; } else ks = 0; - /* If the object still fits, repoison it precisely. */ - if (ks >= new_size) { - /* Zero out spare memory. */ - if (want_init_on_alloc(flags)) { - kasan_disable_current(); + /* If the object doesn't fit, allocate a bigger one */ + if (new_size > ks) + goto alloc_new; + + /* Zero out spare memory. */ + if (want_init_on_alloc(flags)) { + kasan_disable_current(); + if (orig_size < new_size) + memset((void *)p + orig_size, 0, new_size - orig_size); + else memset((void *)p + new_size, 0, ks - new_size); - kasan_enable_current(); - } + kasan_enable_current(); + } - p = kasan_krealloc((void *)p, new_size, flags); - return (void *)p; + if (slub_debug_orig_size(s) && !is_kfence_address(p)) { + set_orig_size(s, (void *)p, new_size); + if (s->flags & SLAB_RED_ZONE && new_size < ks) + memset_no_sanitize_memory((void *)p + new_size, + SLUB_RED_ACTIVE, ks - new_size); } + p = kasan_krealloc((void *)p, new_size, flags); + return (void *)p; + +alloc_new: ret = kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _RET_IP_); if (ret && p) { /* Disable KASAN checks as the object's redzone is accessed. */ kasan_disable_current(); - memcpy(ret, kasan_reset_tag(p), ks); + if (orig_size) + memcpy(ret, kasan_reset_tag(p), orig_size); kasan_enable_current(); } @@ -4764,16 +4782,20 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that * __GFP_ZERO is not fully honored by this API. * - * This is the case, since krealloc() only knows about the bucket size of an - * allocation (but not the exact size it was allocated with) and hence - * implements the following semantics for shrinking and growing buffers with - * __GFP_ZERO. + * When slub_debug_orig_size() is off, krealloc() only knows about the bucket + * size of an allocation (but not the exact size it was allocated with) and + * hence implements the following semantics for shrinking and growing buffers + * with __GFP_ZERO. * * new bucket * 0 size size * |--------|----------------| * | keep | zero | * + * Otherwise, the original allocation size 'orig_size' could be used to + * precisely clear the requested size, and the new size will also be stored + * as the new 'orig_size'. + * * In any case, the contents of the object pointed to are preserved up to the * lesser of the new and old sizes. * From patchwork Wed Sep 11 06:45:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13799751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37C30EE0213 for ; Wed, 11 Sep 2024 06:46:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A635940009; Wed, 11 Sep 2024 02:46:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 856AB6B0385; Wed, 11 Sep 2024 02:46:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F64B940009; Wed, 11 Sep 2024 02:46:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4CB8F6B0383 for ; Wed, 11 Sep 2024 02:46:19 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0176C14046E for ; Wed, 11 Sep 2024 06:46:18 +0000 (UTC) X-FDA: 82551523278.23.2C5DC23 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) by imf01.hostedemail.com (Postfix) with ESMTP id CC1EF40004 for ; Wed, 11 Sep 2024 06:46:16 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=X7nZKmfY; spf=pass (imf01.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726037039; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1BHG+Zromux7mCdATmh4gGmA49XGYtvRJwCiGSv6jro=; b=2MUD3qAeIZYRkuDrdehYYMbUg37I6/ZeLnpubjrfvxEDWyPzM+ybVknd2O5qYGNygCmEkx zbZ4MbJ16VOr3eSyJaok9W5HUzEaJJ1rYMJ6fg37REqEqw5ZGP1Q8lUbEq0PsbzqZhNPZp XsTHdAFl2/yW8lQlLlot6Oop5MfYbeA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=X7nZKmfY; spf=pass (imf01.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726037039; a=rsa-sha256; cv=none; b=5u7fVrZD+SEQIIYsD/x/rjX7/w9y5muarnw8N4heSTeAcUC/0ArH+xkOFrv5nlKc7TDo+g IPBnT+gF5PLZhoI1aNlpqsOfvqsFiFq/+97z7AgWsCw/G8bfKHTlzxz5M9MVDKxfFG0jeE aWRWA4ACOS/ih8UfrKw43t8ojlUoCo4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726037177; x=1757573177; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=spRk1KgMCNNaHSmFuRFHc+16C4JN3bxZV6odFONV/Uw=; b=X7nZKmfYhR/YHqK+NFpGhIf2YlNk0gmq0U6cp3NxaOZTX/uUmR8hjyqc OVn+WnnMfQGseaueWMaEa22WJeNh/8KHSeXPg04gcjFclEwAfMEWtWVYa QZls+C/GfkKQtvHxoDy1lUSJJ70B4pumsl/O+S1pkO9q7O3oPzq0jZFqx axoz8EN2xTPFqFzcsqInRruDnEOQeHPKZZfd7hUEtpRAns3HbgPxigi7+ /uyMA3y2E46OO5DKulO6EUGMlLOqnBvXfqKeYJnwxxl8b8nEA49bbxnxF yoHsiIin/MyVeeqVHBnGsCEa/hKSP2vsjjXSSmc+4a+MPDofLPsvxZRKu Q==; X-CSE-ConnectionGUID: fHgyyj0FQtK5PoZPi6jIWA== X-CSE-MsgGUID: sMohAUJ3Qqu8BdOE7S2mmQ== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="36173050" X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="36173050" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 23:46:12 -0700 X-CSE-ConnectionGUID: tvwWZ/Y9QRKy+QIHoKa/xw== X-CSE-MsgGUID: V2Tkj4x9QrGrTc/GWUmeNQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="67771512" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa007.jf.intel.com with ESMTP; 10 Sep 2024 23:46:05 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich , Alexander Potapenko , Andrey Ryabinin , Dmitry Vyukov , Vincenzo Frascino Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v2 5/5] mm/slub, kunit: Add testcase for krealloc redzone and zeroing Date: Wed, 11 Sep 2024 14:45:35 +0800 Message-Id: <20240911064535.557650-6-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240911064535.557650-1-feng.tang@intel.com> References: <20240911064535.557650-1-feng.tang@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CC1EF40004 X-Stat-Signature: mk61rpiky7e3kup5cxj6n8ixhewc9p95 X-Rspam-User: X-HE-Tag: 1726037176-387927 X-HE-Meta: U2FsdGVkX1+7olxtAWuIbVgQb8qN7atbONrmY6JKcrlgkx5EQzngpfS6TiR3v1xQGK8iPZ6lnL6er4CCmkRR6KxUYr044kuAzee9ZP2o5H8+Fymvsw2Vj1PoL/6Ew05wwYwOYpP9v/8zxERkm/0HMfaVAfVACtBDMkPogCwG7t0KJbLv8vmDAD1uixQuzIeAZbBXJOSn8ICKDztqLNSc2VplBzFlDYhj8ZzGpP/kGriKJai6ixfQrKXYB6YFi0n69Vm2v7jAh72ZGYax32hJiB8AUAgQyGY5D500RF95HsuL80X2TylMJEHMSTvqK7AWTV15BbX2CN+jatVDNQj8nzS73XUrJU6SsqMfVd/f3nE//WBgc9X59GmKRcxZvs4AUNv/VTvC3Ob9Tv9lXgAzXEaCkGhsUsZtjInULsxqchWD6UoobDDtsDTIBDaZC/GdSCBI8AJB6Mt4/U/qKEqVC73P7xNpOsaDdLFGWPqnYOh7ftD3BQyy4FQEgj3s61eIiXNOzixzdQ9oHzFYAYCei41qixdiHK1GZosCw/c8KWGRVxrlGTd3H5GAx2U1+fbXLNNQ/ryJx/D8Tgi0KOikApXACBnuoo5sljvFEzSz7mVaGvTzVctBcwjigALZyQcI3wMyQ+9X0qqfPl6rfRnLTUp/suPsBDa+sD8ymX1IriN/WB2s42oH1lqrs5PIJYD7qO5UZDjE4MCAeZOl1RhQ+tu4EyndqKgLB/Xzsz1Z+GOzpCNtR8LBVHmA8vE/2YQ/MBZz6nxK3weF3RO1+eCO+4N0T45bDyPg7RIUwRn1e51AENy5W2QHuewVGFaG/IZJjHd9QfNIVegcJGDmgpiYvSCECJLqv9Bl0XoTmqr9jXgVAjvwo02EI4iJ4i5y7y4vjColqGtZduHvtxxjSRvAlo7eWSzPIQ1QB6rOYrI9lwol9aPbcrFZuF+Q38AY/N5y1MbB/kzyY3/SKtcQ5Px pBuDGMJR fKQnR2PKx44NqBd0/dYnVNZ4jdQh0y1W+Yy0EZNn5bO/x7aX9AZEbw1ymXKk2mpeRAvTV3ByuxHofGeezYHSH+M1968WT/k59Em+i/E3FE4inKH655GmmU6lMurXrgnwc2N9qkp4Vl6u0IVDvIpNpTR0z+xgdXjOi92JBwPDu7PvbzKKHX7fpVA6TUW1F1oYUV91MSlbnrnoP+BVQtd6ll6RLWAZv1YYPUlmObFmEb7P3GOfXGXAEnGATpcKeLBzPf8i0Oe7xv6sI6aK0lFTNlSudeQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Danilo Krummrich raised issue about krealloc+GFP_ZERO [1], and Vlastimil suggested to add some test case which can sanity test the kmalloc-redzone and zeroing by utilizing the kmalloc's 'orig_size' debug feature. It covers the grow and shrink case of krealloc() re-using current kmalloc object, and the case of re-allocating a new bigger object. [1]. https://lore.kernel.org/lkml/20240812223707.32049-1-dakr@kernel.org/ Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang Reviewed-by: Danilo Krummrich --- Hi Danilo, I keep your Reviewed-By tag, as I think this v2 mostly changes what kmalloc slab to be used. Let me know if you want it dropped, thanks. lib/slub_kunit.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/lib/slub_kunit.c b/lib/slub_kunit.c index 6e3a1e5a7142..b3d158f38b98 100644 --- a/lib/slub_kunit.c +++ b/lib/slub_kunit.c @@ -186,6 +186,47 @@ static void test_leak_destroy(struct kunit *test) KUNIT_EXPECT_EQ(test, 1, slab_errors); } +static void test_krealloc_redzone_zeroing(struct kunit *test) +{ + u8 *p; + int i; + struct kmem_cache *s = test_kmem_cache_create("TestSlub_krealloc", 64, + SLAB_KMALLOC|SLAB_STORE_USER|SLAB_RED_ZONE); + + p = __kmalloc_cache_noprof(s, GFP_KERNEL, 48); + memset(p, 0xff, 48); + + kasan_disable_current(); + OPTIMIZER_HIDE_VAR(p); + + /* Test shrink */ + p = krealloc(p, 40, GFP_KERNEL | __GFP_ZERO); + for (i = 40; i < 64; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + + /* Test grow within the same 64B kmalloc object */ + p = krealloc(p, 56, GFP_KERNEL | __GFP_ZERO); + for (i = 40; i < 56; i++) + KUNIT_EXPECT_EQ(test, p[i], 0); + for (i = 56; i < 64; i++) + KUNIT_EXPECT_EQ(test, p[i], SLUB_RED_ACTIVE); + + validate_slab_cache(s); + KUNIT_EXPECT_EQ(test, 0, slab_errors); + + memset(p, 0xff, 56); + /* Test grow with allocating a bigger 128B object */ + p = krealloc(p, 112, GFP_KERNEL | __GFP_ZERO); + for (i = 0; i < 56; i++) + KUNIT_EXPECT_EQ(test, p[i], 0xff); + for (i = 56; i < 112; i++) + KUNIT_EXPECT_EQ(test, p[i], 0); + + kfree(p); + kasan_enable_current(); + kmem_cache_destroy(s); +} + static int test_init(struct kunit *test) { slab_errors = 0; @@ -208,6 +249,7 @@ static struct kunit_case test_cases[] = { KUNIT_CASE(test_kmalloc_redzone_access), KUNIT_CASE(test_kfree_rcu), KUNIT_CASE(test_leak_destroy), + KUNIT_CASE(test_krealloc_redzone_zeroing), {} };