From patchwork Thu Dec 21 18:35:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 13502532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 065BEC35274 for ; Thu, 21 Dec 2023 18:35:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 787556B0085; Thu, 21 Dec 2023 13:35:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70EE86B0092; Thu, 21 Dec 2023 13:35:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 449066B008C; Thu, 21 Dec 2023 13:35:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 054886B0088 for ; Thu, 21 Dec 2023 13:35:52 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6A6231A079C for ; Thu, 21 Dec 2023 18:35:51 +0000 (UTC) X-FDA: 81591679302.23.C323DFC Received: from out-172.mta0.migadu.com (out-172.mta0.migadu.com [91.218.175.172]) by imf06.hostedemail.com (Postfix) with ESMTP id 7236018002B for ; Thu, 21 Dec 2023 18:35:49 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=tbPj3uPo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf06.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.172 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703183749; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=DILt3p53uNIRMlyKqLmSfi67XoCveJqLdgVMwZq9yXA=; b=w/pUwTO8Gk9F8kS8Sr3YywgQQWlU2DeWmvcVauzQRqIoFnggxifzibWN0uwAl2PHudyQVh nWDn524btDLDrv2pfO9FBvehGYflFkwLdovEFc6OAzhJ775Ghy32twgqv1qWNvIaFetoIp oLGOzPOdasNMzDr89UtIUakvJUoh2pE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=tbPj3uPo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf06.hostedemail.com: domain of andrey.konovalov@linux.dev designates 91.218.175.172 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703183749; a=rsa-sha256; cv=none; b=7MFLWYAp2lFjqyCHoKB7He7RGm9meeQWZBXZt0yEbCE5RJjYxee0INHmmGLAa+JcTxedud CGozJBsqnD/8NyJdySqG1ax8OAjMEHKAQmo85888nKn1Ri4oSca86gRkOYV1M9/FaKyjSA sGny2AYuEsHjZamTlsFnI7g8XXBtgmk= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1703183747; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=DILt3p53uNIRMlyKqLmSfi67XoCveJqLdgVMwZq9yXA=; b=tbPj3uPoBR5fvyz4IQYqOUCK1dVNmUbtUl682TOpZhrRNOI2YNW915Z6Ra8mnw382p2lYY HAd5JqqGBNdGgKffBGvfA3Jgnw4Cwc00qn3tU0JtUl5qPdxqASFQcj+BykHY8fWr1UcDvc K8EcdixFixDgAws3WeDglq4de+F8pvM= From: andrey.konovalov@linux.dev To: Andrew Morton , Juntong Deng Cc: Andrey Konovalov , Marco Elver , Alexander Potapenko , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH mm 1/4] kasan: clean up kasan_cache_create Date: Thu, 21 Dec 2023 19:35:37 +0100 Message-Id: <20231221183540.168428-1-andrey.konovalov@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 7236018002B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: bxfx1z7k5ycwn6mn819hnnuowfjg5qcm X-HE-Tag: 1703183749-205429 X-HE-Meta: U2FsdGVkX1+tZyRUKKWeU8lbrJbXHVE75It7r8e3u8sV7ME6vW2X/5EmpnYf2b2VJmxe7AvnAALT9L9bfJR18Xii2bDLnPDPLexviHUTI85Sv5COc6DZ6M6HJFF1XHUnKuBR5JQiSLDyOP/IehEJkHGlbJ5XFLb+Eg8jJ7PSiuPw2uxbu8EVNeC1ceBJTun8Q3XtQ3uLJU6wmt4wuxm8Yvb9yNVQYx9f3cuioyJ+jdgwylBFkm+HWAAUVS6xXHmUzQVoxSxS9yBgdU08TnfDY0VEKS1Ar3M4fC37sMOc37xkFVQ1mllo/b56q28tVt7jhnZ9M/0wSueKqJ70ydKOeN8FrcpD+YBJ1oU0XT3SZnznJOWlmmmrwEVyPOvxlsDr1CGTuwnhSiu7yfWIjVqk8CSMHDqZRRJEVnfaC8u3gtQNVFtIXJrY89EWoLUO35rzys9cugig0ofsX+saIJ/NMZi1vsVjUxMPQRFAc1ySL2o1413+kfzn1+386FUdf5nceCnRghINIXalfDb02KnbUjSYfnHFb5Hrfr79NKir4Lu2727LApBbq63ANc+MavVX4qE8e9g2Kvx8wvG8NaWsnCwj3vPT1Czlp6CRWC4QKpPOCAvbWEmU6UKsJlc9mhc0OvzGOcOkXCWanD6TXUf//dYKuFB6RwLwouSY6rEf5PiwHtIOZWxp8mUv4hG0bpj9Jp2bJJZnCVvpfQsLuYyb9NQAuN7KX9eJhneDJkBxnvJqeILirxneF0CaE7fx/FpsH2OCuN/3AlMk29ouevv2AaRVqsQby8PMroEGm3kX12XHwwPlphJ9YPNLCZemJ6qmrb/Gp0BXr7MnvOAHdykbgfQGoF+xdRJDu70l67BpmmvC+2HIcwcdx9HpUiUjbKOED4+hg48GWh023H3t8wM0S9JysCcs0npaYuvQ/Af/RX1MmbX0RmIjz+fc/0sKyGHuyjoc25FkBHUCrrhUQvc M27bBFKH FF28jkUq/Sc0V8xlsRWfeLGF0bDoZjklt5DlQadkdr5EOHdkDUkOVFHTV9PlqgrAe7cWpCbiy3ec8bJwKP8R7Cr1sB/Qqo6L0Od+zZ9mWHmXoHXVjNh5XBeeD+//f2C/GW+4WeMhu58GEFBXlW909ahvwopjE4I04vR8yfwqqonOygDycNnYJuf1skYYDB9sXvIRLXHwiBWRpGAdtJCstlLeyVuQZgEPUYXKyv8yQnkLA513CQ2Oc+z1DgevTCC5Ixlm2cn5LSgvrDgtXj7gwD2RWoQviE70uOyVW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Andrey Konovalov Reorganize the code to avoid nested if/else checks to improve the readability. Also drop the confusing comments about KMALLOC_MAX_SIZE checks: they are relevant for both SLUB and SLAB (originally, the comments likely confused KMALLOC_MAX_SIZE with KMALLOC_MAX_CACHE_SIZE). Fixes: a5989d4ed40c ("kasan: improve free meta storage in Generic KASAN") Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver --- mm/kasan/generic.c | 67 +++++++++++++++++++++++++++------------------- 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 54e20b2bc3e1..769e43e05d0b 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -381,16 +381,11 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, ok_size = *size; - /* Add alloc meta into redzone. */ + /* Add alloc meta into the redzone. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); - /* - * If alloc meta doesn't fit, don't add it. - * This can only happen with SLAB, as it has KMALLOC_MAX_SIZE equal - * to KMALLOC_MAX_CACHE_SIZE and doesn't fall back to page_alloc for - * larger sizes. - */ + /* If alloc meta doesn't fit, don't add it. */ if (*size > KMALLOC_MAX_SIZE) { cache->kasan_info.alloc_meta_offset = 0; *size = ok_size; @@ -401,36 +396,52 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, orig_alloc_meta_offset = cache->kasan_info.alloc_meta_offset; /* - * Add free meta into redzone when it's not possible to store + * Store free meta in the redzone when it's not possible to store * it in the object. This is the case when: * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that it can * be touched after it was freed, or * 2. Object has a constructor, which means it's expected to - * retain its content until the next allocation, or - * 3. Object is too small and SLUB DEBUG is enabled. Avoid - * free meta that exceeds the object size corrupts the - * SLUB DEBUG metadata. - * Otherwise cache->kasan_info.free_meta_offset = 0 is implied. - * If the object is smaller than the free meta and SLUB DEBUG - * is not enabled, it is still possible to store part of the - * free meta in the object. + * retain its content until the next allocation. */ if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor) { cache->kasan_info.free_meta_offset = *size; *size += sizeof(struct kasan_free_meta); - } else if (cache->object_size < sizeof(struct kasan_free_meta)) { - if (__slub_debug_enabled()) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } else { - rem_free_meta_size = sizeof(struct kasan_free_meta) - - cache->object_size; - *size += rem_free_meta_size; - if (cache->kasan_info.alloc_meta_offset != 0) - cache->kasan_info.alloc_meta_offset += rem_free_meta_size; - } + goto free_meta_added; + } + + /* + * Otherwise, if the object is large enough to contain free meta, + * store it within the object. + */ + if (sizeof(struct kasan_free_meta) <= cache->object_size) { + /* cache->kasan_info.free_meta_offset = 0 is implied. */ + goto free_meta_added; } + /* + * For smaller objects, store the beginning of free meta within the + * object and the end in the redzone. And thus shift the location of + * alloc meta to free up space for free meta. + * This is only possible when slub_debug is disabled, as otherwise + * the end of free meta will overlap with slub_debug metadata. + */ + if (!__slub_debug_enabled()) { + rem_free_meta_size = sizeof(struct kasan_free_meta) - + cache->object_size; + *size += rem_free_meta_size; + if (cache->kasan_info.alloc_meta_offset != 0) + cache->kasan_info.alloc_meta_offset += rem_free_meta_size; + goto free_meta_added; + } + + /* + * If the object is small and slub_debug is enabled, store free meta + * in the redzone after alloc meta. + */ + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + +free_meta_added: /* If free meta doesn't fit, don't add it. */ if (*size > KMALLOC_MAX_SIZE) { cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; @@ -440,7 +451,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, /* Calculate size with optimal redzone. */ optimal_size = cache->object_size + optimal_redzone(cache->object_size); - /* Limit it with KMALLOC_MAX_SIZE (relevant for SLAB only). */ + /* Limit it with KMALLOC_MAX_SIZE. */ if (optimal_size > KMALLOC_MAX_SIZE) optimal_size = KMALLOC_MAX_SIZE; /* Use optimal size if the size with added metas is not large enough. */