From patchwork Fri Aug 26 02:44:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 12955494 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61263ECAAA2 for ; Fri, 26 Aug 2022 02:44:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9F2494000A; Thu, 25 Aug 2022 22:44:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C4EA8940007; Thu, 25 Aug 2022 22:44:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA22094000A; Thu, 25 Aug 2022 22:44:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9A56C940007 for ; Thu, 25 Aug 2022 22:44:43 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 78665A070B for ; Fri, 26 Aug 2022 02:44:43 +0000 (UTC) X-FDA: 79840200846.22.BB30A5D Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf31.hostedemail.com (Postfix) with ESMTP id 0E2DD20015 for ; Fri, 26 Aug 2022 02:44:42 +0000 (UTC) Received: by mail-pf1-f181.google.com with SMTP id y141so310842pfb.7 for ; Thu, 25 Aug 2022 19:44:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=+2AV05bdm8xkQpoMCJUPIxLKxzKU4IZaOb+/PeeE9PY=; b=mApOc90L16vuczykhMzLUoCjA8f2iPH3BzfoU0N5IE+LdM22Dc0Gf3KYBlCU1rdV8A LyFIojC2pyWfiab+IlwL6FkedJFe2QC8LQ5EelGv2vuIUXI6Gc7WGrFn6mmIyl3WEoMr wE50BS5y1Y/5o57WY+RKrjyorT6tYqi42m4+02csRgbkfPa85LYhla4h2b4vFdnogwRY qFcuUhYAgrK79nXrrjnb1FtkxC6YMwvwuN0XIK2vVWGATqBFVzrmjLuM5xECbB3Tu1e0 iFQK0JSMJaJoAgdGO/FdcCI/GKXlXQZTv6uP8JNQ0k3q+JBYW4oXHsxSnZnygMBzAzBZ XOVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=+2AV05bdm8xkQpoMCJUPIxLKxzKU4IZaOb+/PeeE9PY=; b=fiKwNAGMBkceCbmjyWqgYTRXDNftQZEl42Itd8VvHI/VyfiZK1Ugu/mObCzuQKjtVB /QuwXgetD0y5niz7RGAQlRst0f7Gcr6aJ8zOS6bXtHhC5XHeyB6MX47xCwJ/g/LIBjSL 5AZzF3zpDw2l7qbT+Sf10OFsfV1Iby+vdzJfQMfD6eR23zAIuo5kJYVWjhRYsy/VZAQ3 FUX7yCZN/79zu1ut/RXTQz1pnJtndIYOsrsGjniXGe6MJml5mDPPTNADbFF7wN1Eqq1p AhuwWKeeyJcSfKlfW0luIgScyQr+RCVqOao1xBP9hi9kGmQgvkUf8g2yCBNJFQTZNF76 Ay6g== X-Gm-Message-State: ACgBeo3GljyNsmxpFQwl9qDJZq0hUABdgSpPVV7QSoQ7q+ugNoQ5l2Ah /Q9gO7sE6fkgns9U+UyCjY0= X-Google-Smtp-Source: AA6agR50gs0JWpOYm9NYHwfXNq0iozZ2+2uX/n9WINT9+ttn6xBahC1wKowr+Ulul7NEIVpMqwyFcA== X-Received: by 2002:a63:491f:0:b0:41d:89d5:b3e7 with SMTP id w31-20020a63491f000000b0041d89d5b3e7mr1588144pga.18.1661481882140; Thu, 25 Aug 2022 19:44:42 -0700 (PDT) Received: from macbook-pro-3.dhcp.thefacebook.com ([2620:10d:c090:400::5:15dc]) by smtp.gmail.com with ESMTPSA id 201-20020a6217d2000000b0052d50e14f1dsm365800pfx.78.2022.08.25.19.44.40 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 25 Aug 2022 19:44:41 -0700 (PDT) From: Alexei Starovoitov To: davem@davemloft.net Cc: daniel@iogearbox.net, andrii@kernel.org, tj@kernel.org, memxor@gmail.com, delyank@fb.com, linux-mm@kvack.org, bpf@vger.kernel.org, kernel-team@fb.com Subject: [PATCH v4 bpf-next 02/15] bpf: Convert hash map to bpf_mem_alloc. Date: Thu, 25 Aug 2022 19:44:17 -0700 Message-Id: <20220826024430.84565-3-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220826024430.84565-1-alexei.starovoitov@gmail.com> References: <20220826024430.84565-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661481883; a=rsa-sha256; cv=none; b=H+B2oSuYMNvKaA2VuM5vbqvdgDAzQm9A9JpAsWs0wA8wEOyb5sX2i2gsMbCw30w8GMUDZH ArOGCI06l6hMVUDCM4bQNhk6oTq25B2uDJFjd+aIdY+pdEBAfmuG6fgargx+AOnWExYRni V9jH4bJZ/wCSj97hZjEBIcS6R3zwz24= ARC-Authentication-Results: i=1; imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=mApOc90L; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf31.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661481883; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+2AV05bdm8xkQpoMCJUPIxLKxzKU4IZaOb+/PeeE9PY=; b=ri3rIBvSev0AtMX0A65OmWQo/Z0gZ8q4kYAAoGQ5l3pBbgREi4zNydkQ/cnpbyk6/cDmes 3omO4j4GBVq80Mt5kKYJJG3eZwG5jaUfq+OXHqnyjd5bXnfKLBBSHGRNFIjyCd4Z/sUH8e tId3GfILqYa0zu82dgUNFE7dXZmAFjY= X-Stat-Signature: 86rio4jwfietau1a8etfxo9rk7e78rdm X-Rspamd-Queue-Id: 0E2DD20015 X-Rspam-User: X-Rspamd-Server: rspam11 Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=mApOc90L; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf31.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com X-HE-Tag: 1661481882-431143 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexei Starovoitov Convert bpf hash map to use bpf memory allocator. Acked-by: Kumar Kartikeya Dwivedi Signed-off-by: Alexei Starovoitov --- kernel/bpf/hashtab.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index b301a63afa2f..bd23c8830d49 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -14,6 +14,7 @@ #include "percpu_freelist.h" #include "bpf_lru_list.h" #include "map_in_map.h" +#include #define HTAB_CREATE_FLAG_MASK \ (BPF_F_NO_PREALLOC | BPF_F_NO_COMMON_LRU | BPF_F_NUMA_NODE | \ @@ -92,6 +93,7 @@ struct bucket { struct bpf_htab { struct bpf_map map; + struct bpf_mem_alloc ma; struct bucket *buckets; void *elems; union { @@ -563,6 +565,10 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) if (err) goto free_prealloc; } + } else { + err = bpf_mem_alloc_init(&htab->ma, htab->elem_size); + if (err) + goto free_map_locked; } return &htab->map; @@ -573,6 +579,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); bpf_map_area_free(htab->buckets); + bpf_mem_alloc_destroy(&htab->ma); free_htab: lockdep_unregister_key(&htab->lockdep_key); bpf_map_area_free(htab); @@ -849,7 +856,7 @@ static void htab_elem_free(struct bpf_htab *htab, struct htab_elem *l) if (htab->map.map_type == BPF_MAP_TYPE_PERCPU_HASH) free_percpu(htab_elem_get_ptr(l, htab->map.key_size)); check_and_free_fields(htab, l); - kfree(l); + bpf_mem_cache_free(&htab->ma, l); } static void htab_elem_free_rcu(struct rcu_head *head) @@ -973,9 +980,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, l_new = ERR_PTR(-E2BIG); goto dec_count; } - l_new = bpf_map_kmalloc_node(&htab->map, htab->elem_size, - GFP_NOWAIT | __GFP_NOWARN, - htab->map.numa_node); + l_new = bpf_mem_cache_alloc(&htab->ma); if (!l_new) { l_new = ERR_PTR(-ENOMEM); goto dec_count; @@ -994,7 +999,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key, pptr = bpf_map_alloc_percpu(&htab->map, size, 8, GFP_NOWAIT | __GFP_NOWARN); if (!pptr) { - kfree(l_new); + bpf_mem_cache_free(&htab->ma, l_new); l_new = ERR_PTR(-ENOMEM); goto dec_count; } @@ -1489,6 +1494,7 @@ static void htab_map_free(struct bpf_map *map) bpf_map_free_kptr_off_tab(map); free_percpu(htab->extra_elems); bpf_map_area_free(htab->buckets); + bpf_mem_alloc_destroy(&htab->ma); for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); lockdep_unregister_key(&htab->lockdep_key);