From patchwork Fri Jul 19 19:27:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 13737523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 867A5C3DA5D for ; Fri, 19 Jul 2024 19:27:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E02556B0082; Fri, 19 Jul 2024 15:27:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB3666B0083; Fri, 19 Jul 2024 15:27:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7A3F6B0088; Fri, 19 Jul 2024 15:27:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A963D6B0082 for ; Fri, 19 Jul 2024 15:27:56 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E3977140867 for ; Fri, 19 Jul 2024 19:27:55 +0000 (UTC) X-FDA: 82357487310.16.A3C38F7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 3B5C0160007 for ; Fri, 19 Jul 2024 19:27:53 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aATqaNws; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of kees@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kees@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721417238; a=rsa-sha256; cv=none; b=7YlcTOyYCihNIncmnJ6G7ayQdOBetfPfTui1DQebuh7AVs7BP0lZNhrePQeHUcXL3VuXkh 6ddtYnMyVJMWA2k7KTnwISMlDi+wl3XB06YsUY2jCiWttJYFnpplsnsukVlbFgqjDW30vP Ft8A2DJwPA2hC2I3Bu8+CznEljYkKXo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aATqaNws; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of kees@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kees@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721417238; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=TAh0qtpQ41EVMfA3JQ9UR6wMZRfwlStmcxXY7IvZdWg=; b=5MW/qG3uUiRG2mdjyJ/+Fio4mSt83DzmkGo2jYBOPY7yQN8lJaVMlHKnsipxWTR9QHLJb4 p7mj3D9W3K+BZ5AQwcFr+g3oj0ifwxeF/7s8wel1QzoSZMS1nysnXaraZmtssGr9mV26yo 43lbqWgHFy1VuHNPymo27fa+Au8oS7I= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 3F7FE61C03; Fri, 19 Jul 2024 19:27:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DEC5DC32782; Fri, 19 Jul 2024 19:27:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721417271; bh=PEDczIo+N1zc3HDWBH6pVQlbE4VTMIsKTmLki2Kgujg=; h=From:To:Cc:Subject:Date:From; b=aATqaNwsw7zUhMSCxIzcS+PaFt7EMdmq6f1ISFbvRp9EinIRdkadj3apjqOBPt8H2 mM0qZAaRZoW5GGrFgw5BDjxaIEdA+kbr/XkWb35AhKJvwa4bTteCdgeDzoZhyRMGIT y/z/SlqoLUFgV6a1ZbQpcGlrV/AiQMHHmrNiEQH5OzHBAOTEfp0ECgFNmbl3E6W8ZW FcYOWtx3rfm0p4Ftko/7YNZZ3CTMrX10LY5lt0STJ7XD4QG+hL8f9qOXm34N4YWWTO nmtdVtY27l1/HNFMGfZR4MxORr/Ax0LSVDPkhAB4TV+EQBB+t7xl/V7TGhLSYlTkEp ATOrwcoqPg1/g== From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Gustavo A . R . Silva" , Bill Wendling , Justin Stitt , Jann Horn , Przemek Kitszel , Marco Elver , linux-mm@kvack.org, Nathan Chancellor , Nick Desaulniers , linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-hardening@vger.kernel.org Subject: [PATCH] slab: Introduce kmalloc_obj() and family Date: Fri, 19 Jul 2024 12:27:48 -0700 Message-Id: <20240719192744.work.264-kees@kernel.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4424; i=kees@kernel.org; h=from:subject:message-id; bh=PEDczIo+N1zc3HDWBH6pVQlbE4VTMIsKTmLki2Kgujg=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBmmr4zWEhFKb72H7UArt9Y4HKLkvwrT5DByCcol rkyOA/7hJOJAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCZpq+MwAKCRCJcvTf3G3A JkwGEACSNPqi1z0Rbac2tYergCeVTF0BJw90KpQW+JJlLlAtU2q0NcSYmIby+hoV48sYHqwXx2A Te/n3DVId2VY9yV5H9HxORKzXngzVwqHW8L99nwMOPI3ejdy6RGIIg91yLUlDbxY+Ay53ZdFOTF kbFwFSdV7axOIGilb4VWMUJuzG7ioL71ugBTnHfFYEUE2Uw8ssVP0YmAS21oG9XF+wPtOKBisXu PDhtf3/qjvmz6JJcuvNYgb/dig0DqpCxltP+WHVkRzqMgwgxMiTRO6MPJW19yi9CF4aQQtn9JMy 9dvAJ4HhfPfqreSBv9YYMbfapsqqz0X8SKSK+hqtMBFycURsav39aRivbGwi35G+yLFqRXOo8yw ImlwApj+1gUMDuHRwjmMxURyz3pQ1Vtsgwh0MXaNpCiohe5cy9vlhFfvEq86Z/9i4Rh1uRfGyMZ hrpNwb5SIdfzUzGRQs5MNOynY7gCryY+X4ph1qHv0KIXIfEmDhSJSquyoqrstq+SI1sJbJKhSyH kF7p6SDPe8nbj8MmNLMY8Wmpg4Vcujj9MUnuxhIYAwRhn5rY9Mqlf6S43rJ2uMHez4lx99rkaWv YGY4rn/XKmx3u9FzS4N4kQfcDNmLevNyvlVy+QxnEYxSKwxXcrrYjZ/AcrrrHzjSD9UK/n/r4ig gahzp Hb6u7Elb 0w== X-Developer-Key: i=kees@kernel.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3B5C0160007 X-Stat-Signature: hfrou3ionbpc9pum5ndty9crhd9941di X-Rspam-User: X-HE-Tag: 1721417273-499859 X-HE-Meta: U2FsdGVkX1/zLTjLfjN4wvgDtwBBuEMhVFusd8hE8siEdglM6RJYqgcaOzLf9mbQXENnKnoqa+6FuUsaGiXwcG1RmEJgjjUJMslLV1UfNVL5Lk+1rtOpgBWNPjyTf2k86hBk0IggU629c+wVol9Uke3jbfPuOu6+Y7o+JAf/St0nALJd4J5BH3s73WjMWapzwxhBxQwz8H+/+GE+i3c0x9CoC1t6HJXoubQfYKhl8htjH1t1xi1kpDTM34ckJGJMFJ+zvE1r9Jyiq+GMn2Y4js2jE2efB0hpa7/lJXX/pJvGP4m8nlbn/q0YWLwEVQEDQVGjUP7+0/TO0Afh3x90dpsc9vVlGjEYK34r3AQhQfoYtPieN5u7LdLSJWwiVv5l4QFhtxh7NyAay6vruhvOucAk7zG3DchjzyGU3KZ2sxDdxtNxzxghygfcwEDW3ARj/P2VApE7xQU7tKV/ae/gAXIdDtK3Tjm6aJ9Nu1TyfeiQYLPcmq+9BtuCQIEoAEN6OxsCyrCYJHGFLPR7xcPsOgIRwPGRbrtG8FMMoUVy7rz1Od8iuHa1BnxGMCenoPiBGnTA77EI6Us8ew/fTfZyGY1wKlrb8W0g5sD2Rx4KW5pDAdjePtLCrVE/T6vc+HM8A+FuzRsfzLydmFMnl9pp3yi9HmO6eaQhf+52p9/ZqS7GXNxgz5xJPNa3YWTfAE2EXnIJZhJPntNA4clitWkkfN0Y4yya1sSP3cnkyK1c8gO1VYA4ymTlHwLs1P8RA4KDPhfJIu2jy83b0RY5t9ciQjzykY4cq5NPNLAMb+36LtVhaVq7CeE9GELsahFs4lU13wTgSKoYPw9mfmLHAi12QY5gmmTaOnOkNkq3Ee4c2s1PfJ/v5XHXziHbH7r6oT47wgfFwepUr8qPBSp97GB1w6y4AiiQUqSD3Y7OOAvHcvC8U1Au+7Bmgb2KLNCqD/7yxyoJSFAvkK7zrvaRpJl qYXww6Uv Jg7u+zd3hHjIRHewNS37Fk16q/yB1g+cGc0EGsbokRkRc+Py2NOxqFZU1jhdCkKblgAps8W5sLafzdwCXoezpN5OHEnM8BqkatXlkBZyFxlU8kVL0ZPSztIkXqNVDqe0vbEzmWEMrfL9Ru4M8VjSEZ/K9nEPBek8w5mj3tAcsJXbnACSBs6HIfATdsGL35vpNDaa+IxABUXeq7YeqIVu9xyLhFVaT1n9IUKGCDZIl5XWA9TkbA6bfWSbERwUxQePIrL7pgUAzk7p6q2g5+2Wd33PPLeBy9cpnkXfQGd+CjeptAgZPssvpUMQGJocnYrFU4cHrR8IRuiJYwQQrbtQK1c7qOl4fXzpYsh8LCRYUPXGjrJAEJTMU8ldqGeo+3Qj44C6wBD8Kw1iPZGu3UdYT/LFst/1kmI0kkkim9msEyE+irSWexXvrTRFGeP9dbAQaj1xPfwYUIRQxoPCfrZMQ3y/DD3H8agpMm1rBbvhwNCikpd6f8EzN//qwUt92sMVR58lVt0aMqt3GvQv6MrYP7rZUIqSA3ynsI2fmZ9m0rT9I39bZChRJEXmbi+OIo0Ge57hw/GQJXlRPjACUeVeNtjvLnigbPxISoNyYZ7+W3t/bgMuXbS/Bb6am+NtoLl+6VqeOpZtDtq7pUouRQwvHI33vymUDZql7p8mjLsok3vzmXQvFhVEBeIYPgg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce type-aware kmalloc-family helpers to replace the common idioms for single, array, and flexible object allocations: ptr = kmalloc(sizeof(*ptr), gfp); ptr = kcalloc(count, sizeof(*ptr), gfp); ptr = kmalloc_array(count, sizeof(*ptr), gfp); ptr = kcalloc(count, sizeof(*ptr), gfp); ptr = kmalloc(struct_size(ptr, flex_member, count), gfp); These become, respectively: kmalloc_obj(p, gfp); kzalloc_obj(p, count, gfp); kmalloc_obj(p, count, gfp); kzalloc_obj(p, count, gfp); kmalloc_obj(p, flex_member, count, gfp); These each return the size of the allocation, so that other common idioms can be converted easily as well. For example: info->size = struct_size(ptr, flex_member, count); ptr = kmalloc(info->size, gfp); becomes: info->size = kmalloc_obj(ptr, flex_member, count, gfp); Internal introspection of allocated type also becomes possible, allowing for alignment-aware choices and future hardening work. For example, adding __alignof(*ptr) as an argument to the internal allocators so that appropriate/efficient alignment choices can be made, or being able to correctly choose per-allocation offset randomization within a bucket that does not break alignment requirements. Additionally, once __builtin_set_counted_by() (or equivalent) is added by GCC and Clang, it will be possible to automatically set the counted member of a struct with a counted_by FAM, further eliminating open-coded redundant initializations: info->size = struct_size(ptr, flex_member, count); ptr = kmalloc(info->size, gfp); ptr->flex_count = count; becomes (i.e. unchanged from earlier example): info->size = kmalloc_obj(ptr, flex_member, count, gfp); Replacing all existing simple code patterns via Coccinelle[1] shows what could be replaced immediately (saving roughly 1,500 lines): 7040 files changed, 14128 insertions(+), 15557 deletions(-) Link: https://github.com/kees/kernel-tools/blob/trunk/coccinelle/examples/kmalloc_obj-assign-size.cocci [1] Signed-off-by: Kees Cook --- Cc: Vlastimil Babka Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Andrew Morton Cc: Roman Gushchin Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: Gustavo A. R. Silva Cc: Bill Wendling Cc: Justin Stitt Cc: Jann Horn Cc: Przemek Kitszel Cc: Marco Elver Cc: linux-mm@kvack.org --- include/linux/slab.h | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/include/linux/slab.h b/include/linux/slab.h index 7247e217e21b..3817554f2d51 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -665,6 +665,44 @@ static __always_inline __alloc_size(1) void *kmalloc_noprof(size_t size, gfp_t f } #define kmalloc(...) alloc_hooks(kmalloc_noprof(__VA_ARGS__)) +#define __alloc_obj3(ALLOC, P, COUNT, FLAGS) \ +({ \ + size_t __obj_size = size_mul(sizeof(*P), COUNT); \ + void *__obj_ptr; \ + (P) = __obj_ptr = ALLOC(__obj_size, FLAGS); \ + if (!__obj_ptr) \ + __obj_size = 0; \ + __obj_size; \ +}) + +#define __alloc_obj2(ALLOC, P, FLAGS) __alloc_obj3(ALLOC, P, 1, FLAGS) + +#define __alloc_obj4(ALLOC, P, FAM, COUNT, FLAGS) \ +({ \ + size_t __obj_size = struct_size(P, FAM, COUNT); \ + void *__obj_ptr; \ + (P) = __obj_ptr = ALLOC(__obj_size, FLAGS); \ + if (!__obj_ptr) \ + __obj_size = 0; \ + __obj_size; \ +}) + +#define kmalloc_obj(...) \ + CONCATENATE(__alloc_obj, \ + COUNT_ARGS(__VA_ARGS__))(kmalloc, __VA_ARGS__) + +#define kzalloc_obj(...) \ + CONCATENATE(__alloc_obj, \ + COUNT_ARGS(__VA_ARGS__))(kzalloc, __VA_ARGS__) + +#define kvmalloc_obj(...) \ + CONCATENATE(__alloc_obj, \ + COUNT_ARGS(__VA_ARGS__))(kvmalloc, __VA_ARGS__) + +#define kvzalloc_obj(...) \ + CONCATENATE(__alloc_obj, \ + COUNT_ARGS(__VA_ARGS__))(kvzalloc, __VA_ARGS__) + static __always_inline __alloc_size(1) void *kmalloc_node_noprof(size_t size, gfp_t flags, int node) { if (__builtin_constant_p(size) && size) {