From patchwork Wed Sep 11 06:45:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Tang X-Patchwork-Id: 13799749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2A74EE0211 for ; Wed, 11 Sep 2024 06:46:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FC5F940007; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4ABF36B0384; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3759A940007; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1A4D96B0383 for ; Wed, 11 Sep 2024 02:46:16 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D04FE1A0F6C for ; Wed, 11 Sep 2024 06:46:15 +0000 (UTC) X-FDA: 82551523110.02.4351113 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) by imf28.hostedemail.com (Postfix) with ESMTP id AB033C000C for ; Wed, 11 Sep 2024 06:46:13 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=mAtxeU2f; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726037069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sQnDdI6wKIFdlJ+Rjok4zmD0BtF30gqRHIfs4m6lyC4=; b=tNe3zYDOQ5p7k7dqMAS+7S8lmXbpOr83N3K4l51/JcgeulzdBUrkB9w6K7TufYppnWc3z9 3ExBPPhh7+ZTAi39C6HhrLJolYVXgPLaMSg95PR9ae+xxSgNbbBVteT/UPhO6WJFncBCWn Bu4EAdVh9TIScBX1/pio0i911D1Y+sQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726037069; a=rsa-sha256; cv=none; b=xI6lgLF2G2wQgtyh2KuV3p53pifE0lgmzIKZToY/SIKZqjyAFsqDCHkrqbxzNKufDVIpFL uRoCJB923IzUqKUHPwQX+dV2gNkFok4+t7GVZdmMW7CFJnFsoHmj0x8unqlAzJ0sCAYZD8 nnuBYkBCLk3Pv8x4g/qQ7fKdYoDjeG8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=mAtxeU2f; spf=pass (imf28.hostedemail.com: domain of feng.tang@intel.com designates 198.175.65.12 as permitted sender) smtp.mailfrom=feng.tang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1726037173; x=1757573173; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZHBD5nxjCn39VoHzOY+ge2IVDv8unPJPEMRErbHaUA4=; b=mAtxeU2f8Z7WmpquHWY4nyvM1zqRIlqVBl49pGlAl5nPXPdlfSbdv/+M iVjgPTQvxuzlyZYljW594Jgel6PO1SteSfp74AcjfwDE9JbvAECLNZukv btTJu2Zo9qeQRvqlhjlzmdrrILhyp8H/olExK9B0wwyaH+yE/ZYtNXw8r 2I8tTZ3m3HrDZTZBxIP7JzXRBwTL2CaGQ4hS6iamYjLA6QwIjbDc3UWh+ sW2oxqIxZr24oE0RGsoz8X2+00p3zeJ35hYcfmO7LVECPVl0yh0Dj9pOr Rxxn+Yk7t7bQfQsDaGZTWuzzqaPRp5DoJB4U+6I8K3GjwdDIHqBWTaAlH Q==; X-CSE-ConnectionGUID: z8vMWVA3RkuN4fenPk3J6Q== X-CSE-MsgGUID: fe5dW8WhTxeyMMZztKtQqg== X-IronPort-AV: E=McAfee;i="6700,10204,11191"; a="36173032" X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="36173032" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2024 23:46:11 -0700 X-CSE-ConnectionGUID: v543qwycT2WvE2geGxwVvA== X-CSE-MsgGUID: Ll8xYtUdRRW3q2YxBIkspA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,219,1719903600"; d="scan'208";a="67771506" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by orviesa007.jf.intel.com with ESMTP; 10 Sep 2024 23:46:00 -0700 From: Feng Tang To: Vlastimil Babka , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Andrey Konovalov , Marco Elver , Shuah Khan , David Gow , Danilo Krummrich , Alexander Potapenko , Andrey Ryabinin , Dmitry Vyukov , Vincenzo Frascino Cc: linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, Feng Tang Subject: [PATCH v2 4/5] mm/slub: Improve redzone check and zeroing for krealloc() Date: Wed, 11 Sep 2024 14:45:34 +0800 Message-Id: <20240911064535.557650-5-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240911064535.557650-1-feng.tang@intel.com> References: <20240911064535.557650-1-feng.tang@intel.com> MIME-Version: 1.0 X-Stat-Signature: s41ehnn6c4haw83cdxunuj1fnfo7dq6n X-Rspamd-Queue-Id: AB033C000C X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726037173-368182 X-HE-Meta: U2FsdGVkX1/5rrd+j5+lERaob69+eTF7rv0N2KLvyHLao+b0A7kqFZ+9JsQhnAPM6W13+biXEh+EHiUosjJpd7b2fXE1X5XhiYrB3cBHsZpWQB4AgkzUahOYkExP0z3uphahVvpYhvj8aw3oz7dJHAK5LKgQ3xRXW39AzscuWmaAwAXm8eBOyoMQOMW+cG7sGz4CvkJXCRP2fmpTTMfVRp1nZP+gFMzNxdvVLjLBw3nBqsDrbH12BQERdmRJsbB6KbRw8vFBCkUEk4vZ93yp5kGHdrsDbc3FQNpYOJfsubDA3pF84kjarMuDnHaARY2/EGWUxxzoUOGTkjNpYrKEWmJWZks6pmj1B1Nz8cRm2oSHguHBnW+JOowcwPKaQ9dOXR3IUeJuP+1DO58QLRtSXrHUesBgCoZ/LfOKpaEMiAzPieFBmSVOW3/0EheRo2ulxIGKUNFC2m82syWWjNTIwmMpsiawYSEaIJKXqHgH5w78VjTbJOO+Jis9m3ms+QnputAbrq+qqS4u8d2eecqP4CXXIEHl2X21brQUBCW80mGyPVGoTwCYucY+G9HoKtaGXxPTIkCJqS0L5Rjzdhltbz+r7WKWTmlwNvwYrY/4C9UT1M0iRIrv20WbJpl1MnOg5QSX8Cin1VGUPYcbvwszssx+2QbtkmiQ+pgMUOywJLfj3P6LsKewGvS0K7DIt8S70q7/hjbbBfnzVQ9G0LHUgHUsTa55R5O0D8NlPUVselQF8ZUGlg3zFqYT56z04z+SRleuUCoYq1JvJ0O6s5L+yVTaCE3L+Q7Bnk/+WjEGpQ5OO4GEieaS2fKz7uHlyiCQVPDD/9w0JN9qBreLsV1Yi35fdw1KgVcsF/IDUuaZfSQ+Y3c0tBxFl6hwGZDgDR171KQenEKhuPXrAt0zByKHjqgdPM2LRVyn0Xa4s57RiGiYlCOkPkz6GxJqvDa3DcdRVUziCxcHbnjcQeRwtup MKm+pIzj WCvBERKpVDoXMxi6hLhN/MtijjWpDf+SXXYtVzd3fHMYi3VQie39ZG4+NC2cdA61vb7DwQNH3ypSZLAvPLz1u1O+q6LGypFROymKeh8XyA7OI+IVznSbB51D+BAT7iBpSVniElcG5/oLyC28M8xYWAwQsLSUuHXrJUw5SQ9ojYVvIyo5RZR3mpNz86qzeh2ZuRDe2nXeA4bQALtg6jSC8Dx+ZcMj48ixQu6I++BQMOWvkZ5BXTDCwu7iKefsflDM3TAvb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For current krealloc(), one problem is its caller doesn't pass the old request size, say the object is 64 bytes kmalloc one, but caller may only requested 48 bytes. Then when krealloc() shrinks or grows in the same object, or allocate a new bigger object, it lacks this 'original size' information to do accurate data preserving or zeroing (when __GFP_ZERO is set). Thus with slub debug redzone and object tracking enabled, parts of the object after krealloc() might contain redzone data instead of zeroes, which is violating the __GFP_ZERO guarantees. Good thing is in this case, kmalloc caches do have this 'orig_size' feature. So solve the problem by utilize 'org_size' to do accurate data zeroing and preserving. Suggested-by: Vlastimil Babka Signed-off-by: Feng Tang --- mm/slub.c | 54 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index c1796f9dd30f..e0fb0a26c796 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4717,33 +4717,51 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) { void *ret; size_t ks; + int orig_size = 0; + struct kmem_cache *s; - /* Check for double-free before calling ksize. */ + /* Check for double-free. */ if (likely(!ZERO_OR_NULL_PTR(p))) { if (!kasan_check_byte(p)) return NULL; - ks = ksize(p); + + s = virt_to_cache(p); + orig_size = get_orig_size(s, (void *)p); + ks = s->object_size; } else ks = 0; - /* If the object still fits, repoison it precisely. */ - if (ks >= new_size) { - /* Zero out spare memory. */ - if (want_init_on_alloc(flags)) { - kasan_disable_current(); + /* If the object doesn't fit, allocate a bigger one */ + if (new_size > ks) + goto alloc_new; + + /* Zero out spare memory. */ + if (want_init_on_alloc(flags)) { + kasan_disable_current(); + if (orig_size < new_size) + memset((void *)p + orig_size, 0, new_size - orig_size); + else memset((void *)p + new_size, 0, ks - new_size); - kasan_enable_current(); - } + kasan_enable_current(); + } - p = kasan_krealloc((void *)p, new_size, flags); - return (void *)p; + if (slub_debug_orig_size(s) && !is_kfence_address(p)) { + set_orig_size(s, (void *)p, new_size); + if (s->flags & SLAB_RED_ZONE && new_size < ks) + memset_no_sanitize_memory((void *)p + new_size, + SLUB_RED_ACTIVE, ks - new_size); } + p = kasan_krealloc((void *)p, new_size, flags); + return (void *)p; + +alloc_new: ret = kmalloc_node_track_caller_noprof(new_size, flags, NUMA_NO_NODE, _RET_IP_); if (ret && p) { /* Disable KASAN checks as the object's redzone is accessed. */ kasan_disable_current(); - memcpy(ret, kasan_reset_tag(p), ks); + if (orig_size) + memcpy(ret, kasan_reset_tag(p), orig_size); kasan_enable_current(); } @@ -4764,16 +4782,20 @@ __do_krealloc(const void *p, size_t new_size, gfp_t flags) * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that * __GFP_ZERO is not fully honored by this API. * - * This is the case, since krealloc() only knows about the bucket size of an - * allocation (but not the exact size it was allocated with) and hence - * implements the following semantics for shrinking and growing buffers with - * __GFP_ZERO. + * When slub_debug_orig_size() is off, krealloc() only knows about the bucket + * size of an allocation (but not the exact size it was allocated with) and + * hence implements the following semantics for shrinking and growing buffers + * with __GFP_ZERO. * * new bucket * 0 size size * |--------|----------------| * | keep | zero | * + * Otherwise, the original allocation size 'orig_size' could be used to + * precisely clear the requested size, and the new size will also be stored + * as the new 'orig_size'. + * * In any case, the contents of the object pointed to are preserved up to the * lesser of the new and old sizes. *