From patchwork Fri Sep 23 20:28:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12987072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53D99C07E9D for ; Fri, 23 Sep 2022 20:28:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46E028001B; Fri, 23 Sep 2022 16:28:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41CC080016; Fri, 23 Sep 2022 16:28:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BDF88001B; Fri, 23 Sep 2022 16:28:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 17A1880016 for ; Fri, 23 Sep 2022 16:28:35 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DF88780C1A for ; Fri, 23 Sep 2022 20:28:34 +0000 (UTC) X-FDA: 79944488148.18.3BF5117 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf11.hostedemail.com (Postfix) with ESMTP id 91FEB4000F for ; Fri, 23 Sep 2022 20:28:34 +0000 (UTC) Received: by mail-pl1-f180.google.com with SMTP id w20so1143927ply.12 for ; Fri, 23 Sep 2022 13:28:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=09XDqhCmbOxMmk4st3n1VevKxFvu8TBGt9LJYr24nIA=; b=C8a8bBwBCPpYqJC0ttlBfkppX2uMw5viHVMSx4NSUdlu6Mk2tJYnIkpj1O3sRUaf2n YvaKYhZaGs/8xWczPFhL4W6+saiCUf5KQGmW47sQE/VjI9iBzvPK3eyEePf3h3djFnDv 4Xw5J5dvAI2eSiF2SIboitvBfMPVts9ASzBRc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=09XDqhCmbOxMmk4st3n1VevKxFvu8TBGt9LJYr24nIA=; b=oWBp48ml7MHjOXHlHuOIkcyksAZW6o6S5e8L6ux+owCTyprnZZAvNHl1SkEil5NQ38 LyhkSkm41cwok+Cs8siupsZSjUZJ7bziAEbdJ/BxUo6MgGnlmBsoeOpjgiHxUerIhygJ dT3BNuG+XhutJi1MWKF2zzspl5cpCq02u2o1kIzpdHxdEFPxac5hKQVmowH/14dT/xyI w9baytBriAWbjtEjHlqNxG2IaqC9oNxWt6urfwjfpWeMXXSiHlXyV4fJO4zfqZzaY211 B9+Uy9ioALPL+i+lx9ffSPUYnN6Lbr8DuIz/ewHAmcj57UisOd4DwT6XLstRHgOiGeUd VPnw== X-Gm-Message-State: ACrzQf2mq9BKIARNZRhntL+f29gTDU7PRrGmrtnXJS1wpI6WNWenDpRn Ei0LLVajHxSYRqUmEh8ztF1hpQ== X-Google-Smtp-Source: AMsMyM7DXBOfv6ahzpoYpFJ1XA+uXLlf1m80ImgaNTw1lXpXdBLHWAXCKtzP0wavX+dE1aJb2tOr4Q== X-Received: by 2002:a17:903:2015:b0:178:8022:ff1 with SMTP id s21-20020a170903201500b0017880220ff1mr10285638pla.18.1663964913651; Fri, 23 Sep 2022 13:28:33 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id i3-20020aa796e3000000b00535da15a252sm6765031pfq.165.2022.09.23.13.28.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 13:28:32 -0700 (PDT) From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , bpf@vger.kernel.org, "Ruhl, Michael J" , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Greg Kroah-Hartman , Nick Desaulniers , Alex Elder , Josef Bacik , David Sterba , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , Jesse Brandeburg , Daniel Micay , Marco Elver , Miguel Ojeda , linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-fsdevel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, dev@openvswitch.org, x86@kernel.org, llvm@lists.linux.dev, linux-hardening@vger.kernel.org Subject: [PATCH v2 11/16] bpf: Use kmalloc_size_roundup() to match ksize() usage Date: Fri, 23 Sep 2022 13:28:17 -0700 Message-Id: <20220923202822.2667581-12-keescook@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220923202822.2667581-1-keescook@chromium.org> References: <20220923202822.2667581-1-keescook@chromium.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4987; h=from:subject; bh=+umEDwGyJCv3Ovyx4Pj2CYIL4lHo8xavtPhh0E8bcL8=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBjLhblZrOjZBzDi2GSWosP6rXkUyCU3IxBEiC+Kuw9 wTqv/j+JAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYy4W5QAKCRCJcvTf3G3AJjMAEA CfzaZuEjLR7HquBXo8+EM4GS8NWzszH99hX9xWPHebOuW16pmhNIEq+Aju+xJdnrhZANxwQ9Vh+iy8 ymNAeDJdi1cPUvm/XvEjIQdHYJrutoIiSKB2d0AEqLwbsec1LjBhbvhDu0LI7jqNxIfIv2/9Wt0lw+ FLC6qE+PRYixT6MQix4PHQQlKlYq57pj+xWZMmEn7EdqoqjCDsI3K/t90ikcx3WVUqhltKxdMQKBBV x5co1XdfKAeNPVoBf0Q4pGe8YOFn/SbYAQ3SUpg9WyQETPSbnLumwlf27mINUyCKWF27Tg53zEYExy CL2Y77JbZEY8nCSjb5ET/WZ4BJLz8w55BmqKLuG+LHoVxmG9As/KzYhd4wNeF/dVm0OLi7gOh89qzi GEU1kcMn/xDpv1MQ2jDoG8VufwjUDPawkqASsD3lyfggklYm5al7OV0DrKzqFXHwaeBjaWHmM6KLKe 37QfdNfUsWdCuBvcEjoUX7Za+u901T3eO2SKiBvJCpX2sc6vi67INLI4syc1DeZAWI+wioCztNVLto FEw2RcZEC86T5zincgBp5Ofm7ehCRUPa/wet1Ayqkgvsrtnt1X+1xS1o6I1ntxzju/C/TzBCm6nHcb R4gLY+rvdc1IBBznpLWzmV3Cz3ue47gmjjlvg+BX0gGRJVy0rhqVwTrXOpsA== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=C8a8bBwB; spf=pass (imf11.hostedemail.com: domain of keescook@chromium.org designates 209.85.214.180 as permitted sender) smtp.mailfrom=keescook@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663964914; a=rsa-sha256; cv=none; b=kW3tnu6B6/ayzqs4AckYXV4Kb1PcgIlDgBPF6DE4UIraw4SlRZ+t8lV0aPG/g2YyA5DjyD P3jJhaGtcHtDiZ0wQv4Fu5nYaZ+Oy2mdeHJAeF6g74AK7wtHnxLxsKpM9ypVIUS67TKzBV d4lFd0GIUJFT91FOnFnZ6egGDzIbTVM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663964914; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=09XDqhCmbOxMmk4st3n1VevKxFvu8TBGt9LJYr24nIA=; b=s3Jcq4kW3JNjiy/FU95ueF1Fwx63siXZiXJjFPqejDwKBTgVeXphYTYWr4ke5k2Al2jRqZ c2tfTLioiU51IJEIiyk0NGqt9duiHPG8Lg+/EqDv1bh9pSrXruvc3jpiOmvm4HCnS75jSE dhtORoejcjqStuc7H9zIk3GR25D8bdo= Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=C8a8bBwB; spf=pass (imf11.hostedemail.com: domain of keescook@chromium.org designates 209.85.214.180 as permitted sender) smtp.mailfrom=keescook@chromium.org; dmarc=pass (policy=none) header.from=chromium.org X-Stat-Signature: i6z4y6n7m4zng3a9nchcffb6d6n6gzjc X-Rspamd-Queue-Id: 91FEB4000F X-Rspamd-Server: rspam12 X-Rspam-User: X-HE-Tag: 1663964914-175460 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Round up allocations with kmalloc_size_roundup() so that the verifier's use of ksize() is always accurate and no special handling of the memory is needed by KASAN, UBSAN_BOUNDS, nor FORTIFY_SOURCE. Pass the new size information back up to callers so they can use the space immediately, so array resizing to happen less frequently as well. Explicitly zero any trailing bytes in new allocations. Additionally fix a memory allocation leak: if krealloc() fails, "arr" wasn't freed, but NULL was return to the caller of realloc_array() would be writing NULL to the lvalue, losing the reference to the original memory. Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: John Fastabend Cc: Andrii Nakryiko Cc: Martin KaFai Lau Cc: Song Liu Cc: Yonghong Song Cc: KP Singh Cc: Stanislav Fomichev Cc: Hao Luo Cc: Jiri Olsa Cc: bpf@vger.kernel.org Signed-off-by: Kees Cook --- kernel/bpf/verifier.c | 49 +++++++++++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 18 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 096fdac70165..80531f8f0d36 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -978,42 +978,53 @@ static void print_insn_state(struct bpf_verifier_env *env, */ static void *copy_array(void *dst, const void *src, size_t n, size_t size, gfp_t flags) { - size_t bytes; + size_t src_bytes, dst_bytes; if (ZERO_OR_NULL_PTR(src)) goto out; - if (unlikely(check_mul_overflow(n, size, &bytes))) + if (unlikely(check_mul_overflow(n, size, &src_bytes))) return NULL; - if (ksize(dst) < bytes) { + dst_bytes = kmalloc_size_roundup(src_bytes); + if (ksize(dst) < dst_bytes) { kfree(dst); - dst = kmalloc_track_caller(bytes, flags); + dst = kmalloc_track_caller(dst_bytes, flags); if (!dst) return NULL; } - memcpy(dst, src, bytes); + memcpy(dst, src, src_bytes); + memset(dst + src_bytes, 0, dst_bytes - src_bytes); out: return dst ? dst : ZERO_SIZE_PTR; } -/* resize an array from old_n items to new_n items. the array is reallocated if it's too - * small to hold new_n items. new items are zeroed out if the array grows. +/* Resize an array from old_n items to *new_n items. The array is reallocated if it's too + * small to hold *new_n items. New items are zeroed out if the array grows. Allocation + * is rounded up to next kmalloc bucket size to reduce frequency of resizing. *new_n + * contains the new total number of items that will fit. * - * Contrary to krealloc_array, does not free arr if new_n is zero. + * Contrary to krealloc, does not free arr if new_n is zero. */ -static void *realloc_array(void *arr, size_t old_n, size_t new_n, size_t size) +static void *realloc_array(void *arr, size_t old_n, size_t *new_n, size_t size) { - if (!new_n || old_n == new_n) + void *old_arr = arr; + size_t alloc_size; + + if (!new_n || !*new_n || old_n == *new_n) goto out; - arr = krealloc_array(arr, new_n, size, GFP_KERNEL); - if (!arr) + alloc_size = kmalloc_size_roundup(size_mul(*new_n, size)); + arr = krealloc(old_arr, alloc_size, GFP_KERNEL); + if (!arr) { + kfree(old_arr); return NULL; + } - if (new_n > old_n) - memset(arr + old_n * size, 0, (new_n - old_n) * size); + *new_n = alloc_size / size; + if (*new_n > old_n) + memset(arr + old_n * size, 0, (*new_n - old_n) * size); out: return arr ? arr : ZERO_SIZE_PTR; @@ -1045,7 +1056,7 @@ static int copy_stack_state(struct bpf_func_state *dst, const struct bpf_func_st static int resize_reference_state(struct bpf_func_state *state, size_t n) { - state->refs = realloc_array(state->refs, state->acquired_refs, n, + state->refs = realloc_array(state->refs, state->acquired_refs, &n, sizeof(struct bpf_reference_state)); if (!state->refs) return -ENOMEM; @@ -1061,11 +1072,11 @@ static int grow_stack_state(struct bpf_func_state *state, int size) if (old_n >= n) return 0; - state->stack = realloc_array(state->stack, old_n, n, sizeof(struct bpf_stack_state)); + state->stack = realloc_array(state->stack, old_n, &n, sizeof(struct bpf_stack_state)); if (!state->stack) return -ENOMEM; - state->allocated_stack = size; + state->allocated_stack = n * BPF_REG_SIZE; return 0; } @@ -2472,9 +2483,11 @@ static int push_jmp_history(struct bpf_verifier_env *env, { u32 cnt = cur->jmp_history_cnt; struct bpf_idx_pair *p; + size_t size; cnt++; - p = krealloc(cur->jmp_history, cnt * sizeof(*p), GFP_USER); + size = kmalloc_size_roundup(size_mul(cnt, sizeof(*p))); + p = krealloc(cur->jmp_history, size, GFP_USER); if (!p) return -ENOMEM; p[cnt - 1].idx = env->insn_idx;