From patchwork Fri Jul 29 15:23:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5717BC19F2B for ; Fri, 29 Jul 2022 15:23:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7C166B0072; Fri, 29 Jul 2022 11:23:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E28526B0073; Fri, 29 Jul 2022 11:23:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA2638E0001; Fri, 29 Jul 2022 11:23:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BD5BE6B0072 for ; Fri, 29 Jul 2022 11:23:24 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7C25F120AFB for ; Fri, 29 Jul 2022 15:23:24 +0000 (UTC) X-FDA: 79740506328.12.0CE3041 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf01.hostedemail.com (Postfix) with ESMTP id DCC5F400D2 for ; Fri, 29 Jul 2022 15:23:23 +0000 (UTC) Received: by mail-pl1-f173.google.com with SMTP id x10so4199900plb.3 for ; Fri, 29 Jul 2022 08:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N3x5RniCtfbU9X4ilWg0wCJ6uR/I379+KNtOUGQRQWk=; b=MR6BnmL0+/AkdUIl6s2D0Mxj44Qh3mJnK0QhajxPtZyXDzIXOP8IgodIkHLOh5vW6k SuFnhXmgAn+NpmxydQlF2a5pCWk8FIX8YEU6Y+G/fLC1y9eUrXvK4FpC11fyEag0y9yj 4CTFiOMwH/ZcDg0B7spmm3TY3jM0WQG/2GJQVfu5kGbTfSDayJ7G7S62l0lrpHxoAAeL v7AujGGF+n4rnR258DMNk7ITHeA9SxF2rVm6UETeCFHXKFIkGunItcwab3EE5t7t1UyL 93YK/uSh8T5UUYpYYy3f8mynhtlzyPgDz9njO6jiby40lR2jT9ZWCnFNBVw1ZBokfAQC eJNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N3x5RniCtfbU9X4ilWg0wCJ6uR/I379+KNtOUGQRQWk=; b=MlKVK2H/7qL3NNnV4DJzjEJ7WF4Yn1kDwP1Xa0v7asagKt7Q4i1o1AS/P7N2Ux52Mk atXXGifZLzL4ZowTfwYBvY6xGbR6ICSxVaBJ/ebpzuxMbqLWxnbkRDMrgMLOWq34kBWa lvvy9SjNkOSARTPaedPItWhKGcs4ov7bzu0a9JX8UMKc0UUI5k9oky4Oh84/d9/S4ghf TOlRCwhj+JQg4KaG1Yl9PNWpwWXH0JOAOiUxNukGTs0E4ciJINW1raNGUHBBai1hMhql ySKsn9Qz19VEPax83EVHwdinblq11AS/5+8csWOEPubdnfevujWkAHmolz4hdqv30qbx Suxw== X-Gm-Message-State: ACgBeo0XpulpLxfTPL5r20scT0b/hQJu22u2WlZarSJO0CWbE940h+aJ drBmz3CGIFKfOXZhb0N9rqs= X-Google-Smtp-Source: AA6agR6PxNVeqQc1c9TRKFUa0+QWSa9MU7oMGsj7LktTOarVwJpRJ/ZDTOGpZctMMv5u6/+XFxhhZQ== X-Received: by 2002:a17:90b:3555:b0:1f3:259b:b8b3 with SMTP id lt21-20020a17090b355500b001f3259bb8b3mr4635942pjb.91.1659108202921; Fri, 29 Jul 2022 08:23:22 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:21 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 01/15] bpf: Remove unneeded memset in queue_stack_map creation Date: Fri, 29 Jul 2022 15:23:02 +0000 Message-Id: <20220729152316.58205-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MR6BnmL0; spf=pass (imf01.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108204; a=rsa-sha256; cv=none; b=d877MLmVABkr6N/djri6xitl4ZenTwiSodrIW5oZc8HTHY56tk/7cT9u2FyEiDkoPHRkO9 uBNJ2YIrt80EZB7CWfN1OWRKN6eyVF6lqMQ7Mv8dYQiQOR/GOUmD0FPMIvg0iB7gU87FSx IE8GqYs9SAccADPsOQn72g71R5Z0q+g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108203; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N3x5RniCtfbU9X4ilWg0wCJ6uR/I379+KNtOUGQRQWk=; b=eH1A11yT+zspS4WNI6XBZFfhFsRzzu3g7OFCD39rspkoPE+ZO225Mtw9gAtSXr5ywQRhH+ 5YzwW0QhXbntfYHA3b4C3OJ1h6HdFoTiB7xbFgsIC2UUHXKYLM/YuBdULuNJ8W0F7atLRT L5z72h++ej3xZDHGs8G6nwuLBYyJIfk= X-Rspamd-Server: rspam10 X-Rspam-User: Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MR6BnmL0; spf=pass (imf01.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: karoibnp68o19gfksyrdgs3pnz4k5i9n X-Rspamd-Queue-Id: DCC5F400D2 X-HE-Tag: 1659108203-464626 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __GFP_ZERO will clear the memory, so we don't need to memset it. Signed-off-by: Yafang Shao --- kernel/bpf/queue_stack_maps.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index a1c0794ae49d..8a5e060de63b 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -78,8 +78,6 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) if (!qs) return ERR_PTR(-ENOMEM); - memset(qs, 0, sizeof(*qs)); - bpf_map_init_from_attr(&qs->map, attr); qs->size = size; From patchwork Fri Jul 29 15:23:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 927E6C19F2D for ; Fri, 29 Jul 2022 15:23:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B9BE6B0073; Fri, 29 Jul 2022 11:23:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 268D68E0001; Fri, 29 Jul 2022 11:23:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 109C56B0075; Fri, 29 Jul 2022 11:23:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 031626B0073 for ; Fri, 29 Jul 2022 11:23:27 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CED401C6B56 for ; Fri, 29 Jul 2022 15:23:26 +0000 (UTC) X-FDA: 79740506412.22.91ACC22 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf11.hostedemail.com (Postfix) with ESMTP id 4AE97400D9 for ; Fri, 29 Jul 2022 15:23:26 +0000 (UTC) Received: by mail-pl1-f176.google.com with SMTP id x7so4888717pll.7 for ; Fri, 29 Jul 2022 08:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MuUKdiJi+CqUOLmeMIpojQqKJ/GoLF/CYoe+aiGCvRU=; b=BcPEkIG2uX/BG29TRtU20AZAJAcDHBvd7n8X2orpJ1VK7LyYrV54t3WLCKQtHI3Lq3 he6YOMBpMHRqkIk0VFAYSrNUC4idcIZ9FJ/g2LfJ1l+V57Ayoz+Q1kfCtu+hxLHd7Sgx bTDpv7nWAe1KvZ0UiIDZ5lWiH+9117MVCBHE4KhQWbqocrYrQReb0elE1OY7Q/BjVrOe d2qBc1+2L96iDBwFDEcGJCPqm/WgDBiwOEEKwApxPHrv5VpJV3Zbey/78X+QzmY8UbbY tV8j+LKwavkpJ3pfbUdjDsXDGb56sQAR2wh7Z5G9tG7rvqXPJZw72i83w/rqFIrXXzUk PktA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MuUKdiJi+CqUOLmeMIpojQqKJ/GoLF/CYoe+aiGCvRU=; b=ltcC7R1ECG03WQ8gU07IZDA6Vmz9ixrgMkJ05i8rG0JZc6kp5qqDU3FzINol3r/wOG Ui3xmDHWUTv5iTUCNsAiVjRq/FCBp/Lx/ppCT5/iAmiiPcAheBEPE2s4zAkyks1MQ+sU OjVaRTu6UllubQ6dTX09H01gbV6GE705Rfscbv8tq9zyqiSjK03VMeDAJcQFKgvteQIP Cj1/LKxbkzOGKLyj0qsruKaublE0H7HwB3tD60VxYdnuTDqU3WPrxux7ZB89ptvlOzw6 UOSmYQPvsrYjEPe7PyGPhYnGmHjMSCfHwtso0SWLB1fyH5hcjXnsA+11zY6ubfFB9ozw 34Qw== X-Gm-Message-State: ACgBeo1IWtw/AfqckN2aWs2aE2mOe1RLv5GVZggW69f6wwY5QCseXvyh /zcfabMe+2MU+WHz6U1SQwY= X-Google-Smtp-Source: AA6agR7YfoIlbxo8CLs9bnehGJow8QNgQ5mui++Ey30il7CQd0i1TJbbeANlW79BbP8P9qo95ZBFWw== X-Received: by 2002:a17:903:28c:b0:16d:cf30:3b71 with SMTP id j12-20020a170903028c00b0016dcf303b71mr4384084plr.165.1659108205120; Fri, 29 Jul 2022 08:23:25 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:24 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 02/15] bpf: Use bpf_map_area_free instread of kvfree Date: Fri, 29 Jul 2022 15:23:03 +0000 Message-Id: <20220729152316.58205-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108206; a=rsa-sha256; cv=none; b=dwjAjxeVeAwv6sK6BSOo+zd0Jn7NHPNOhx/ehj/o6K2BFxDMoMCR5vP1s9IOHCUWRZTejB qUdl8oNiqrL99Vge0hS7NO/T6uOmomOUymqURTBCa2Aa7PeGY5/UXYt8u8xHojCXsqyK5D nzWNBC7lxeo9+0NUw9q4Occ7Xk+NHao= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=BcPEkIG2; spf=pass (imf11.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108206; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MuUKdiJi+CqUOLmeMIpojQqKJ/GoLF/CYoe+aiGCvRU=; b=UMuiRvdHA68YpAh/6Pux8WHex1HUGR+zRfij8V4mOIlLNtaZ3jxfhraVPmxq07xCYbW0ia NByGlXFUpA2MZYUGlhL13TmjCcLOQJd95wP7iunLuHVt9kwgXp2WncOTF1OorqEGVjfo5r 9fNxhFQG4YSmUliHcXzz8o3TMwHqirA= X-Rspamd-Queue-Id: 4AE97400D9 X-Rspam-User: X-Stat-Signature: krq4zn1w7d8zzzhdmk87n7cj6mgnqwho Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=BcPEkIG2; spf=pass (imf11.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam08 X-HE-Tag: 1659108206-719271 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bpf_map_area_alloc() should be paired with bpf_map_area_free(). Signed-off-by: Yafang Shao --- kernel/bpf/ringbuf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index ded4faeca192..3fb54feb39d4 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -116,7 +116,7 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) err_free_pages: for (i = 0; i < nr_pages; i++) __free_page(pages[i]); - kvfree(pages); + bpf_map_area_free(pages); return NULL; } @@ -190,7 +190,7 @@ static void bpf_ringbuf_free(struct bpf_ringbuf *rb) vunmap(rb); for (i = 0; i < nr_pages; i++) __free_page(pages[i]); - kvfree(pages); + bpf_map_area_free(pages); } static void ringbuf_map_free(struct bpf_map *map) From patchwork Fri Jul 29 15:23:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72D41C00144 for ; Fri, 29 Jul 2022 15:23:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 048A76B0074; Fri, 29 Jul 2022 11:23:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F13DD6B0075; Fri, 29 Jul 2022 11:23:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB4236B0078; Fri, 29 Jul 2022 11:23:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CD99F6B0074 for ; Fri, 29 Jul 2022 11:23:28 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7CC1816117A for ; Fri, 29 Jul 2022 15:23:28 +0000 (UTC) X-FDA: 79740506496.09.31E116E Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 33ED74002C for ; Fri, 29 Jul 2022 15:23:28 +0000 (UTC) Received: by mail-pg1-f172.google.com with SMTP id f11so4262607pgj.7 for ; Fri, 29 Jul 2022 08:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fdHrruL4Gllw7kdBxmFxiRaESQjhqPF3nWXhKnbwkBA=; b=nqKMNPeCDL3zIJ0KAnu7Ubyg6aZdOBBTo2XarSeCo3GTOtWCGPCMrx+WurkaeOe+7j YJD4WZQjhd0Bgig+Sgyho5LrKggMOpX7oqYbSWjUVyGcBBIT0qHCrz32lfHWOfR3mwoe GbypGSPpwb6zZC6O7o5mFNhH9lZPQdl9kEGQ8w1IM76mHpvCZ/JKYqijeizlGjJFmOQt g7mbl1hbonIelI3QG/G2fsrz0d1+Ctwk5e6ld9CgJNzkGmzSGbyaoSqZyDBytdMt8Bbf gxB/4CQI0zb4pcE+RwIe03oyz+9i1OMmxBw9BmClkm8pZAz/xAulvKY1Jv+czNYtQ2vH 20kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fdHrruL4Gllw7kdBxmFxiRaESQjhqPF3nWXhKnbwkBA=; b=DEsccrfWTB0YUCCBZZbewTLpmGIaLRpGgdDENd//RUYcpnuNatNyBfX1tTx2ULlKtR V3i/PaWWCHYviTLAR4aBEI3skmTJGvDHMCbJ3UHTlcsd2kOokP8v8+MyWsm5ZARhh5ua Bpq3n1q12JDgHjHxNElW9WviaNWCBxkQ4Y/DTQrhJpi1HPQtUqCvot+LzNeFgwy0lzgV nQ75CpzeSE09k/oszE7ivs9RuLchUZuGfzwFKFORDd61lNM8v1+xZR10yeWGctqZlcP7 F3hAgS0OLhK/b3VYHW4gg2lqS2dr7hlITe3Jm/l6d6zqZGzwzgjxDFYf7D4pCyV0VAqn UspA== X-Gm-Message-State: AJIora/74LmAvMWJF8QOnrchtUdl0dCpd/SFcE9NiwUb5UrL/WZL97OR f5elYnE4elmqEh71MANNQwg= X-Google-Smtp-Source: AGRyM1sA1x/xZBftR6UikLqMA4lb9NV10asAIBmoGRkZNnjIzlJO7tGpgNuEezQlbu9chOjyiKy4XQ== X-Received: by 2002:a63:8bc8:0:b0:413:9952:6059 with SMTP id j191-20020a638bc8000000b0041399526059mr3293683pge.61.1659108207244; Fri, 29 Jul 2022 08:23:27 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:26 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 03/15] bpf: Make __GFP_NOWARN consistent in bpf map creation Date: Fri, 29 Jul 2022 15:23:04 +0000 Message-Id: <20220729152316.58205-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108208; a=rsa-sha256; cv=none; b=ButuB6f9L+gT54XnKTPWNHgYyMNf/57vubjW/B5gcwjbJ1X/IWSyDvJS8TSd+zH5L+8gkc 9HjNE/tCGv5o7PzZOChOMOX7YrN858Y6KK8dnAgubZzjNdEQ9Fmyb6KZBONH199KvdpoKr faAvd7jsreKcgVnEPqsi6FnVTYsMry8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nqKMNPeC; spf=pass (imf17.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.172 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108208; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fdHrruL4Gllw7kdBxmFxiRaESQjhqPF3nWXhKnbwkBA=; b=4Ldmsh6mUe2qgHNS00pAZlDAXxGjcQ3zl56KK1Pje6Adk4y/nPEn1El/0d7l/O3/tXncZO f7Ggo+/dUaYJ7nfPNr7nCD8gT/ZiD/WQM0AYWh1PZ8BM63SgAdGAR8FnGrtpajrZeeZUfF syiBg05qSaZ8EBfy1DUOL0nFQMXdPJc= Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nqKMNPeC; spf=pass (imf17.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.172 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 33ED74002C X-Stat-Signature: k1a9tf3rqy9twjzd1956tshqihwtq3h1 X-HE-Tag: 1659108208-991943 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some of the bpf maps are created with __GFP_NOWARN, i.e. arraymap, bloom_filter, bpf_local_storage, bpf_struct_ops, lpm_trie, queue_stack_maps, reuseport_array, stackmap and xskmap, while others are created without __GFP_NOWARN, i.e. cpumap, devmap, hashtab, local_storage, offload, ringbuf and sock_map. But there are not key differences between the creation of these maps. So let make this allocation flag consistent in all bpf maps creation. Then we can use a generic helper to alloc all bpf maps. Signed-off-by: Yafang Shao --- kernel/bpf/cpumap.c | 2 +- kernel/bpf/devmap.c | 2 +- kernel/bpf/hashtab.c | 2 +- kernel/bpf/local_storage.c | 4 ++-- kernel/bpf/offload.c | 2 +- kernel/bpf/ringbuf.c | 2 +- net/core/sock_map.c | 4 ++-- 7 files changed, 9 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index f4860ac756cd..b25ca9d603a6 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_ACCOUNT); + cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!cmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index a0e02b009487..88feaa094de8 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -163,7 +163,7 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_ACCOUNT); + dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!dtab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index da7578426a46..f1e5303fe26e 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -495,7 +495,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_ACCOUNT); + htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!htab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 49ef0ce040c7..a64255e20f87 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,8 +313,8 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = kmalloc_node(sizeof(struct bpf_cgroup_storage_map), - __GFP_ZERO | GFP_USER | __GFP_ACCOUNT, numa_node); + map = kzalloc_node(sizeof(struct bpf_cgroup_storage_map), + GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT, numa_node); if (!map) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index bd09290e3648..5a629a1b971c 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -372,7 +372,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) attr->map_type != BPF_MAP_TYPE_HASH) return ERR_PTR(-EINVAL); - offmap = kzalloc(sizeof(*offmap), GFP_USER); + offmap = kzalloc(sizeof(*offmap), GFP_USER | __GFP_NOWARN); if (!offmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 3fb54feb39d4..df8062cb258c 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -164,7 +164,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_ACCOUNT); + rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!rb_map) return ERR_PTR(-ENOMEM); diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 028813dfecb0..763d77162d0c 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,7 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = kzalloc(sizeof(*stab), GFP_USER | __GFP_ACCOUNT); + stab = kzalloc(sizeof(*stab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!stab) return ERR_PTR(-ENOMEM); @@ -1076,7 +1076,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_ACCOUNT); + htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!htab) return ERR_PTR(-ENOMEM); From patchwork Fri Jul 29 15:23:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A97FC19F2A for ; Fri, 29 Jul 2022 15:23:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CE496B0075; Fri, 29 Jul 2022 11:23:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17ECA6B0078; Fri, 29 Jul 2022 11:23:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F18C06B007B; Fri, 29 Jul 2022 11:23:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E4E236B0075 for ; Fri, 29 Jul 2022 11:23:30 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BE4DBAA0DF for ; Fri, 29 Jul 2022 15:23:30 +0000 (UTC) X-FDA: 79740506580.15.A34903C Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf23.hostedemail.com (Postfix) with ESMTP id 6B095140036 for ; Fri, 29 Jul 2022 15:23:30 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id v16-20020a17090abb9000b001f25244c65dso8715180pjr.2 for ; Fri, 29 Jul 2022 08:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BtVcQdl416qb7s4IbcOizRRojtBfwQY2q+CMmuUPEsI=; b=XW6cOXyyNkMTotHKjdFP0DcxFQ2UjvM99NgoZMsgCARdHiVWMjXAgB9+D0I3DfmNC/ OK4mdImWQJvoejmWzAvyCadaGsz9hgmxWu5esmuaaP4YzxOFLdS+3a04L2XM5sBdDX1X gKRnp+iMeJ6kuopt1d4Z9eUpRDoBqf9DhS0lJn1dww7XHUIc9mV4G0Gf1qvZn8OxvciQ pBTlAyuHVCt1bsUT/cBTLXb4kCB7pqjgU6+GlsGEdtVHiJ69Qr/OdRUiLyeXbLp6j+XV sUgG6fjvvqGPCdEwG2wp0M9ne0UTvfPUiidYuoNCjsXlB6pardwUDC3duHBpTfXgD9mv CUvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BtVcQdl416qb7s4IbcOizRRojtBfwQY2q+CMmuUPEsI=; b=qVoUOK9cGP//fmKoEa8tg8jEpQOXVaA3MDWYttFpw3YX8pDxv8bGtuUwJwJosQ/qWi zeMiMuHGw6PQ99uaItsCsffElJrkvxdme2ILH9D6NQM3z6KRHC6VTnr9qU3gwW0yWdc8 hxBgSO2cQ4UR5aIEE3rVlnuZXy0lZugJlGRbRi8qzE0WqXfwPKmhy/o+1G+Jh5cuLwIL jjrGNZENhKvq92RABQrWF3H39IpxPhgZ32b5LSI6bNZSstj+bmY87VtgzsrSSuVwuP7x c0Xy8PItfNMcKF5azigYIBoMTJRQcN4a+9qutjMDG7vHXwGINIglDMuKJn/DaateGTye m+dQ== X-Gm-Message-State: ACgBeo0QfAiWQ3pNx1eC5mMgOWJyfUDfTmne+FLPPbjagpgQjqMYKogC q8eLZQ0D/D3nQ+pw6q5Xe18= X-Google-Smtp-Source: AA6agR44dwd4X5Qy6BOUE7eqSaAT3e8U7sExSR2x3MgA4wDrFfGwNePT26JF4dRQ+knUPvS4ya1gLw== X-Received: by 2002:a17:90a:ea90:b0:1f2:81cd:1948 with SMTP id h16-20020a17090aea9000b001f281cd1948mr4879296pjz.172.1659108209481; Fri, 29 Jul 2022 08:23:29 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:28 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 04/15] bpf: Use bpf_map_area_alloc consistently on bpf map creation Date: Fri, 29 Jul 2022 15:23:05 +0000 Message-Id: <20220729152316.58205-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XW6cOXyy; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108210; a=rsa-sha256; cv=none; b=0OxWK8MWORZ/Pxg4+DsNIfrptAS81z50hF8FPCAH38epqZnEN2I44k7DCTvW5PLlRhq5Tz 35v6ztwAOD1k/zQilI6Wgx1cX+KWDTt9Cd6OyHas4NKaivmDgYwt52eghgU1HsKjevDyXJ VgpsmQarP3l8M9In0EASSP8D7yJKCN8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BtVcQdl416qb7s4IbcOizRRojtBfwQY2q+CMmuUPEsI=; b=F1wRMDAW+rSZQD4SUS16jEwvOVfbIfpocWj0PZUtuVgTt+3T1cDK7qGsK7wHDwmzc0WAic dkmUj+xYfl+JKKSUfaWjCv3619jXAC18hGV65AgrBs76eYyBPOHcxlAQo1wMio4TgOSwUJ A7yokwEjmFia2n/YJIisr4mNPNdb7O4= X-Stat-Signature: m4t6urwu1re6iamw1irekuhj95yfy7t3 X-Rspamd-Queue-Id: 6B095140036 X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XW6cOXyy; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-HE-Tag: 1659108210-334561 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's use the generic helper bpf_map_area_alloc() instead of the open-coded kzalloc helpers in bpf maps creation path. Signed-off-by: Yafang Shao --- kernel/bpf/bpf_local_storage.c | 6 +++--- kernel/bpf/cpumap.c | 6 +++--- kernel/bpf/devmap.c | 6 +++--- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 5 ++--- kernel/bpf/lpm_trie.c | 4 ++-- kernel/bpf/ringbuf.c | 6 +++--- net/core/sock_map.c | 12 ++++++------ 8 files changed, 25 insertions(+), 26 deletions(-) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 8ce40fd869f6..4ee2e7286c23 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -582,7 +582,7 @@ void bpf_local_storage_map_free(struct bpf_local_storage_map *smap, synchronize_rcu(); kvfree(smap->buckets); - kfree(smap); + bpf_map_area_free(smap); } int bpf_local_storage_map_alloc_check(union bpf_attr *attr) @@ -610,7 +610,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) unsigned int i; u32 nbuckets; - smap = kzalloc(sizeof(*smap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap = bpf_map_area_alloc(sizeof(*smap), NUMA_NO_NODE); if (!smap) return ERR_PTR(-ENOMEM); bpf_map_init_from_attr(&smap->map, attr); @@ -623,7 +623,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!smap->buckets) { - kfree(smap); + bpf_map_area_free(smap); return ERR_PTR(-ENOMEM); } diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index b25ca9d603a6..b5ba34ddd4b6 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + cmap = bpf_map_area_alloc(sizeof(*cmap), NUMA_NO_NODE); if (!cmap) return ERR_PTR(-ENOMEM); @@ -118,7 +118,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) return &cmap->map; free_cmap: - kfree(cmap); + bpf_map_area_free(cmap); return ERR_PTR(err); } @@ -623,7 +623,7 @@ static void cpu_map_free(struct bpf_map *map) __cpu_map_entry_replace(cmap, i, NULL); /* call_rcu */ } bpf_map_area_free(cmap->cpu_map); - kfree(cmap); + bpf_map_area_free(cmap); } /* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 88feaa094de8..f9a87dcc5535 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -163,13 +163,13 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + dtab = bpf_map_area_alloc(sizeof(*dtab), NUMA_NO_NODE); if (!dtab) return ERR_PTR(-ENOMEM); err = dev_map_init_map(dtab, attr); if (err) { - kfree(dtab); + bpf_map_area_free(dtab); return ERR_PTR(err); } @@ -240,7 +240,7 @@ static void dev_map_free(struct bpf_map *map) bpf_map_area_free(dtab->netdev_map); } - kfree(dtab); + bpf_map_area_free(dtab); } static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index f1e5303fe26e..8392f7f8a8ac 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -495,7 +495,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); if (!htab) return ERR_PTR(-ENOMEM); @@ -579,7 +579,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) bpf_map_area_free(htab->buckets); free_htab: lockdep_unregister_key(&htab->lockdep_key); - kfree(htab); + bpf_map_area_free(htab); return ERR_PTR(err); } @@ -1496,7 +1496,7 @@ static void htab_map_free(struct bpf_map *map) for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); lockdep_unregister_key(&htab->lockdep_key); - kfree(htab); + bpf_map_area_free(htab); } static void htab_map_seq_show_elem(struct bpf_map *map, void *key, diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index a64255e20f87..098cf336fae6 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,8 +313,7 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = kzalloc_node(sizeof(struct bpf_cgroup_storage_map), - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT, numa_node); + map = bpf_map_area_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node); if (!map) return ERR_PTR(-ENOMEM); @@ -346,7 +345,7 @@ static void cgroup_storage_map_free(struct bpf_map *_map) WARN_ON(!RB_EMPTY_ROOT(&map->root)); WARN_ON(!list_empty(&map->list)); - kfree(map); + bpf_map_area_free(map); } static int cgroup_storage_delete_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index d789e3b831ad..d833496e9e42 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -558,7 +558,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) attr->value_size > LPM_VAL_SIZE_MAX) return ERR_PTR(-EINVAL); - trie = kzalloc(sizeof(*trie), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + trie = bpf_map_area_alloc(sizeof(*trie), NUMA_NO_NODE); if (!trie) return ERR_PTR(-ENOMEM); @@ -609,7 +609,7 @@ static void trie_free(struct bpf_map *map) } out: - kfree(trie); + bpf_map_area_free(trie); } static int trie_get_next_key(struct bpf_map *map, void *_key, void *_next_key) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index df8062cb258c..b483aea35f41 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -164,7 +164,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + rb_map = bpf_map_area_alloc(sizeof(*rb_map), NUMA_NO_NODE); if (!rb_map) return ERR_PTR(-ENOMEM); @@ -172,7 +172,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); if (!rb_map->rb) { - kfree(rb_map); + bpf_map_area_free(rb_map); return ERR_PTR(-ENOMEM); } @@ -199,7 +199,7 @@ static void ringbuf_map_free(struct bpf_map *map) rb_map = container_of(map, struct bpf_ringbuf_map, map); bpf_ringbuf_free(rb_map->rb); - kfree(rb_map); + bpf_map_area_free(rb_map); } static void *ringbuf_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 763d77162d0c..d0c43384d8bf 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,7 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = kzalloc(sizeof(*stab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + stab = bpf_map_area_alloc(sizeof(*stab), NUMA_NO_NODE); if (!stab) return ERR_PTR(-ENOMEM); @@ -52,7 +52,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) sizeof(struct sock *), stab->map.numa_node); if (!stab->sks) { - kfree(stab); + bpf_map_area_free(stab); return ERR_PTR(-ENOMEM); } @@ -361,7 +361,7 @@ static void sock_map_free(struct bpf_map *map) synchronize_rcu(); bpf_map_area_free(stab->sks); - kfree(stab); + bpf_map_area_free(stab); } static void sock_map_release_progs(struct bpf_map *map) @@ -1076,7 +1076,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); if (!htab) return ERR_PTR(-ENOMEM); @@ -1106,7 +1106,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) return &htab->map; free_htab: - kfree(htab); + bpf_map_area_free(htab); return ERR_PTR(err); } @@ -1159,7 +1159,7 @@ static void sock_hash_free(struct bpf_map *map) synchronize_rcu(); bpf_map_area_free(htab->buckets); - kfree(htab); + bpf_map_area_free(htab); } static void *sock_hash_lookup_sys(struct bpf_map *map, void *key) From patchwork Fri Jul 29 15:23:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E730DC19F2B for ; Fri, 29 Jul 2022 15:23:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FEE16B0078; Fri, 29 Jul 2022 11:23:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6AEB46B007B; Fri, 29 Jul 2022 11:23:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 556178E0001; Fri, 29 Jul 2022 11:23:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4817E6B0078 for ; Fri, 29 Jul 2022 11:23:33 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1313A1C6B3E for ; Fri, 29 Jul 2022 15:23:33 +0000 (UTC) X-FDA: 79740506706.31.6EA5BED Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by imf05.hostedemail.com (Postfix) with ESMTP id 6E49C1000BC for ; Fri, 29 Jul 2022 15:23:32 +0000 (UTC) Received: by mail-pj1-f42.google.com with SMTP id a7-20020a17090a008700b001f325db8b90so4153781pja.0 for ; Fri, 29 Jul 2022 08:23:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3d+Vnnj0Wn2TotffdpnitiR4y4GtmpBbmAGTyIaTSkM=; b=idobMl1R4oCtQYLGjlYto1s6UlhcTJgFu3NUkjfEi5j6NUZx/r/8NUrkKFKQkM8u9x pEjnY9LgsdKpeXwc5YSZDJmaxyDUa9ivpvl188U8XXHGn5XzMAVN3T9OutqAU7BFa1tB RHLfjrBhaeV/pigJlqITo6VJ6jwgEuSkMlIKHOx6qfNfvTbBGTisqrwfyhKZjcuvTWj2 ue3m7bUU2oS9KYQvKvEEgqk+WIWEySkGfnHoWRYj2vtQlFrR4QQDbLc2GnWwuYyrnj2p /DU304Bsu8zNJH+SRzWorjTy/+gukbTajj1B+VAk4F9X6tWiejJF+TyNBni7hBaYzHZp lgog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3d+Vnnj0Wn2TotffdpnitiR4y4GtmpBbmAGTyIaTSkM=; b=fSAxx2QIA4OH+n3ghirGuWKGG68wXIVcWLq3at+HWb49WLYR7h5OJFnv4ATHO9NCJZ Nz6mtcCXwNDi0tnI1z2Djz/3MMsWG98wTGHjwYtS9JAmQSWEleab/fuuy9Y31M0/U4xh 0+LMIHOuVSXIYlsN7/3NRWHxzYB0WhkDbD/q0TMxP2HQwhNOPHpOeZPiMfIqIY7hzXER XZexiW/CGWJVFWv5HZXJr91PBeTPSECAx41WoP1M8TQp7JnYB3dRtXl+zhYmDyJ0H3PR VjvUcuVMy3awJ9WW16eUiAgANTZgUv2BYN3a+EhFtDut93pltAelo/JG8V+Ltj5hN1Af gqXw== X-Gm-Message-State: ACgBeo2gbBRWgNuyYd4LzewVtSKkEbD8VJvgQhTy79TNDtD/q8RaXFQJ goAY3HZdkSsrp32yOEEAMLiE/P8lTwp8WBFI X-Google-Smtp-Source: AA6agR5ZlXbmQGLf5QjvBmdvX0UV8lkVnYqG2atAs/tSFEhkNBwSchlIEUS8VEdkWcaJra/qvhUN2w== X-Received: by 2002:a17:902:e8c6:b0:16d:ab3a:1d42 with SMTP id v6-20020a170902e8c600b0016dab3a1d42mr4150046plg.28.1659108211481; Fri, 29 Jul 2022 08:23:31 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:30 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 05/15] bpf: Introduce helpers for container of struct bpf_map Date: Fri, 29 Jul 2022 15:23:06 +0000 Message-Id: <20220729152316.58205-6-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=idobMl1R; spf=pass (imf05.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108212; a=rsa-sha256; cv=none; b=BXxjPDI0NbjN8JhCdo48TFlvoiQ8Srk7yxiQaPYSpXgp19FUhZwOoosHGsm5Kdeenu6v3c AzhWciks0l8E/EFqbm0b23utcNSZ6JQPaSbx+0aytG0m3Ms7b0mHGnCrUMfC8jpmrKxyP8 EgPbYxnFbOz2gUJ2EargAaZxpCEuAVM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108212; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3d+Vnnj0Wn2TotffdpnitiR4y4GtmpBbmAGTyIaTSkM=; b=MqeHV87fvCggGO2MwxwMaG1vPGxrBMOiFpSipAtHLlvvzN/65yqhSYPu3ETCc9bP/e9cfX 1PP/5SjczHFOkQ+8gcgueULeDiU6CZfUsJmtljHdzStXL9aiRjlDj2OIlrLx5aiyIb744G oESJ5STkhrBjfK2syPuD3c7DFETkGG4= Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=idobMl1R; spf=pass (imf05.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 6E49C1000BC X-Stat-Signature: xb868jncudozymbqnxeyk46xa6uxm6ko X-HE-Tag: 1659108212-804689 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently bpf_map_area_alloc() is used to allocate a container of struct bpf_map or members in this container. To distinguish the map creation and other members, let split it into two different helpers, - bpf_map_container_alloc() Used to allocate a container of struct bpf_map, the container is as follows, struct bpf_map_container { struct bpf_map map; // the map must be the first member .... }; Pls. note that the struct bpf_map_contianer is a abstract one, which can be struct bpf_array, struct bpf_bloom_filter and etc. In this helper, it will call bpf_map_save_memcg() to init memcg relevant data in the bpf map. And these data will be cleared in bpf_map_container_free(). - bpf_map_area_alloc() Now it is used to allocate the members in a contianer only. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 4 ++++ kernel/bpf/syscall.c | 56 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 20c26aed7896..2d971b0eb24b 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1634,9 +1634,13 @@ void bpf_map_inc_with_uref(struct bpf_map *map); struct bpf_map * __must_check bpf_map_inc_not_zero(struct bpf_map *map); void bpf_map_put_with_uref(struct bpf_map *map); void bpf_map_put(struct bpf_map *map); +void *bpf_map_container_alloc(u64 size, int numa_node); +void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, + u32 align, u32 offset); void *bpf_map_area_alloc(u64 size, int numa_node); void *bpf_map_area_mmapable_alloc(u64 size, int numa_node); void bpf_map_area_free(void *base); +void bpf_map_container_free(void *base); bool bpf_map_write_active(const struct bpf_map *map); void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); int generic_map_lookup_batch(struct bpf_map *map, diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 83c7136c5788..1a1a81a11b37 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -495,6 +495,62 @@ static void bpf_map_release_memcg(struct bpf_map *map) } #endif +/* + * The return pointer is a bpf_map container, as follow, + * struct bpf_map_container { + * struct bpf_map map; + * ... + * }; + * + * It is used in map creation path. + */ +void *bpf_map_container_alloc(u64 size, int numa_node) +{ + struct bpf_map *map; + void *container; + + container = __bpf_map_area_alloc(size, numa_node, false); + if (!container) + return NULL; + + map = (struct bpf_map *)container; + bpf_map_save_memcg(map); + + return container; +} + +void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, u32 align, + u32 offset) +{ + struct bpf_map *map; + void *container; + void *ptr; + + /* kmalloc'ed memory can't be mmap'ed, use explicit vmalloc */ + ptr = __bpf_map_area_alloc(size, numa_node, true); + if (!ptr) + return NULL; + + container = ptr + align - offset; + map = (struct bpf_map *)container; + bpf_map_save_memcg(map); + + return ptr; +} + +void bpf_map_container_free(void *container) +{ + struct bpf_map *map; + + if (!container) + return; + + map = (struct bpf_map *)container; + bpf_map_release_memcg(map); + + kvfree(container); +} + static int bpf_map_kptr_off_cmp(const void *a, const void *b) { const struct bpf_map_value_off_desc *off_desc1 = a, *off_desc2 = b; From patchwork Fri Jul 29 15:23:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AA78C00144 for ; Fri, 29 Jul 2022 15:23:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E4DD6B007B; Fri, 29 Jul 2022 11:23:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 994B16B007D; Fri, 29 Jul 2022 11:23:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 835D88E0001; Fri, 29 Jul 2022 11:23:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 74AE26B007B for ; Fri, 29 Jul 2022 11:23:35 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4360EC1132 for ; Fri, 29 Jul 2022 15:23:35 +0000 (UTC) X-FDA: 79740506790.05.64FD463 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf26.hostedemail.com (Postfix) with ESMTP id B50A91400C7 for ; Fri, 29 Jul 2022 15:23:34 +0000 (UTC) Received: by mail-pl1-f171.google.com with SMTP id p8so4868844plq.13 for ; Fri, 29 Jul 2022 08:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hWvKtt0XsSyCHiHPOrXuWkS8k5NNSExSS5sGNFOGCEo=; b=qWi0F7JQtGNO72/yD8SADSq0iUTPIzfVVNivxPXcllDpCr8afwl+L1eKtBkdummbGD ol5bniw8M4HC777V2VNaap8wSD5bjlvQGjG5qK/hua8LfNS1zsK9q48aFSAaHEIpav1c /vs+78hWlfzltwsqhVj87O9e9wsbfsutUMsnbe8rloQ7TqR+7HN7LI9Mqph6jAiU7CBo kGRyTFn28jBIzJykLM4ZDq3+BTx6Ev9unZc92F7mF32vhJJ4/iSADoO4fqqxZSL9vGjj M9DUjUpDvf166rl0U/p6EZKbpSDeDbI/WcL3/kPfb/jn+bnLwSAr487UXyUZii141IzA 8xJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hWvKtt0XsSyCHiHPOrXuWkS8k5NNSExSS5sGNFOGCEo=; b=62BrHlcEMpoXaq7wm+lvARqH7mkr6PUPIv8Nq0Cg2EwU2UKHmqe8z6JIF6FC9tDO6E qmBoy66j10OpJeyXv5wQTMAS5SFLY7vh6u1E0ql5xyddtk94row8BS6Bf/+yDLT20A5m AZ7RLTyxb4YHziI7Rh9wj7B/WWi77752zh0V9vJ1Sts6vTBW/BOc6fk//clNvqZEjR/y h+UJPDuAJROTSqvOWqPPZH4P/CcYehCTuTm41ll31Ri5rQckYcmw2DzoDatOHgn8RuAe oMKt/72KgHFCQ16r3v3Hnrrm7OfzYOhq/khWoY4inm+ytRvvGbwwhw0uRUZujfeJ9zBM 0s4A== X-Gm-Message-State: ACgBeo23NoApzp/077sva+nqE40kj/lf+HQ6CuE5Sgc92f6CaqVC9zor 4+dDnfkCE9Dtnq6Fby7D/nJU74i1/gL9c+Pl X-Google-Smtp-Source: AA6agR4CRDOkwFeL7KUH3JCciqczj/ZDeXcs8fab8fsjkB4V9xWoxKgT+VUKmGt2New9nthSlDQ2Zw== X-Received: by 2002:a17:902:7796:b0:16d:41b2:dd36 with SMTP id o22-20020a170902779600b0016d41b2dd36mr4237120pll.137.1659108213626; Fri, 29 Jul 2022 08:23:33 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:32 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 06/15] bpf: Use bpf_map_container_alloc helpers in various bpf maps Date: Fri, 29 Jul 2022 15:23:07 +0000 Message-Id: <20220729152316.58205-7-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qWi0F7JQ; spf=pass (imf26.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108214; a=rsa-sha256; cv=none; b=BT0CpIS6Bv84SgHMeKgI+fKyKAJrXurYaPhOcEY12S2HBnkDhXa+az34UXycsM+Ev2A2vB kN/n4Y37P54O2RW5VUClJxArFHBqZjdgWF67pHnuoaUA7FUSo89hV85e2gQN6RO+ioVUn9 vbkchS8QhYePDZ/SChTyh1zQGWGhqbU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108214; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hWvKtt0XsSyCHiHPOrXuWkS8k5NNSExSS5sGNFOGCEo=; b=yq0hLaH20VA/zkQDu2gP+8bByJSekntsIqPO2pg9UN5KyfsyRgAK7vtBbBY0YZXM2dWm13 PRcvX9dHuKGf8CYONqqOlQBbvirdFBegb5wK/YVOnIzqOoKfaW0tqmr9UmxO5/SqT3hRUZ KrdK2yrOHEwF/DoeqbyX9LmHibJmKUQ= X-Stat-Signature: yggu3jkcy71kpwzqg8ydfjkg87azaeu9 X-Rspamd-Queue-Id: B50A91400C7 X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qWi0F7JQ; spf=pass (imf26.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam12 X-HE-Tag: 1659108214-648162 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the preparation of previous patch, now we can use the helper bpf_map_container_alloc() to allocate all bpf maps. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 1 - kernel/bpf/arraymap.c | 19 +++++++++++-------- kernel/bpf/bloom_filter.c | 4 ++-- kernel/bpf/bpf_local_storage.c | 6 +++--- kernel/bpf/bpf_struct_ops.c | 4 ++-- kernel/bpf/cpumap.c | 6 +++--- kernel/bpf/devmap.c | 6 +++--- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 4 ++-- kernel/bpf/lpm_trie.c | 4 ++-- kernel/bpf/offload.c | 2 +- kernel/bpf/queue_stack_maps.c | 4 ++-- kernel/bpf/reuseport_array.c | 4 ++-- kernel/bpf/ringbuf.c | 6 +++--- kernel/bpf/stackmap.c | 6 +++--- kernel/bpf/syscall.c | 8 -------- net/core/sock_map.c | 12 ++++++------ net/xdp/xskmap.c | 4 ++-- 18 files changed, 50 insertions(+), 56 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 2d971b0eb24b..3f9893e14124 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1638,7 +1638,6 @@ void *bpf_map_container_alloc(u64 size, int numa_node); void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, u32 align, u32 offset); void *bpf_map_area_alloc(u64 size, int numa_node); -void *bpf_map_area_mmapable_alloc(u64 size, int numa_node); void bpf_map_area_free(void *base); void bpf_map_container_free(void *base); bool bpf_map_write_active(const struct bpf_map *map); diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index d3e734bf8056..9517619dbe8d 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -126,16 +126,18 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) /* allocate all map elements and zero-initialize them */ if (attr->map_flags & BPF_F_MMAPABLE) { + u32 align = PAGE_ALIGN(sizeof(struct bpf_array)); + u32 offset = offsetof(struct bpf_array, value); void *data; /* kmalloc'ed memory can't be mmap'ed, use explicit vmalloc */ - data = bpf_map_area_mmapable_alloc(array_size, numa_node); + data = bpf_map_container_mmapable_alloc(array_size, numa_node, + align, offset); if (!data) return ERR_PTR(-ENOMEM); - array = data + PAGE_ALIGN(sizeof(struct bpf_array)) - - offsetof(struct bpf_array, value); + array = data + align - offset; } else { - array = bpf_map_area_alloc(array_size, numa_node); + array = bpf_map_container_alloc(array_size, numa_node); } if (!array) return ERR_PTR(-ENOMEM); @@ -147,7 +149,7 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) array->elem_size = elem_size; if (percpu && bpf_array_alloc_percpu(array)) { - bpf_map_area_free(array); + bpf_map_container_free(array); return ERR_PTR(-ENOMEM); } @@ -420,6 +422,7 @@ static void array_map_free(struct bpf_map *map) struct bpf_array *array = container_of(map, struct bpf_array, map); int i; + if (map_value_has_kptrs(map)) { for (i = 0; i < array->map.max_entries; i++) bpf_map_free_kptrs(map, array_map_elem_ptr(array, i)); @@ -430,9 +433,9 @@ static void array_map_free(struct bpf_map *map) bpf_array_free_percpu(array); if (array->map.map_flags & BPF_F_MMAPABLE) - bpf_map_area_free(array_map_vmalloc_addr(array)); + bpf_map_container_free(array_map_vmalloc_addr(array)); else - bpf_map_area_free(array); + bpf_map_container_free(array); } static void array_map_seq_show_elem(struct bpf_map *map, void *key, @@ -774,7 +777,7 @@ static void fd_array_map_free(struct bpf_map *map) for (i = 0; i < array->map.max_entries; i++) BUG_ON(array->ptrs[i] != NULL); - bpf_map_area_free(array); + bpf_map_container_free(array); } static void *fd_array_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index b9ea539a5561..8b3194903b52 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -142,7 +142,7 @@ static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) } bitset_bytes = roundup(bitset_bytes, sizeof(unsigned long)); - bloom = bpf_map_area_alloc(sizeof(*bloom) + bitset_bytes, numa_node); + bloom = bpf_map_container_alloc(sizeof(*bloom) + bitset_bytes, numa_node); if (!bloom) return ERR_PTR(-ENOMEM); @@ -168,7 +168,7 @@ static void bloom_map_free(struct bpf_map *map) struct bpf_bloom_filter *bloom = container_of(map, struct bpf_bloom_filter, map); - bpf_map_area_free(bloom); + bpf_map_container_free(bloom); } static void *bloom_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 4ee2e7286c23..0b50cb2e1d5b 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -582,7 +582,7 @@ void bpf_local_storage_map_free(struct bpf_local_storage_map *smap, synchronize_rcu(); kvfree(smap->buckets); - bpf_map_area_free(smap); + bpf_map_container_free(smap); } int bpf_local_storage_map_alloc_check(union bpf_attr *attr) @@ -610,7 +610,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) unsigned int i; u32 nbuckets; - smap = bpf_map_area_alloc(sizeof(*smap), NUMA_NO_NODE); + smap = bpf_map_container_alloc(sizeof(*smap), NUMA_NO_NODE); if (!smap) return ERR_PTR(-ENOMEM); bpf_map_init_from_attr(&smap->map, attr); @@ -623,7 +623,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!smap->buckets) { - bpf_map_area_free(smap); + bpf_map_container_free(smap); return ERR_PTR(-ENOMEM); } diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 84b2d9dba79a..66df3059a3fe 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -580,7 +580,7 @@ static void bpf_struct_ops_map_free(struct bpf_map *map) bpf_map_area_free(st_map->links); bpf_jit_free_exec(st_map->image); bpf_map_area_free(st_map->uvalue); - bpf_map_area_free(st_map); + bpf_map_container_free(st_map); } static int bpf_struct_ops_map_alloc_check(union bpf_attr *attr) @@ -618,7 +618,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) */ (vt->size - sizeof(struct bpf_struct_ops_value)); - st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE); + st_map = bpf_map_container_alloc(st_map_size, NUMA_NO_NODE); if (!st_map) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index b5ba34ddd4b6..23e941826eec 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = bpf_map_area_alloc(sizeof(*cmap), NUMA_NO_NODE); + cmap = bpf_map_container_alloc(sizeof(*cmap), NUMA_NO_NODE); if (!cmap) return ERR_PTR(-ENOMEM); @@ -118,7 +118,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) return &cmap->map; free_cmap: - bpf_map_area_free(cmap); + bpf_map_container_free(cmap); return ERR_PTR(err); } @@ -623,7 +623,7 @@ static void cpu_map_free(struct bpf_map *map) __cpu_map_entry_replace(cmap, i, NULL); /* call_rcu */ } bpf_map_area_free(cmap->cpu_map); - bpf_map_area_free(cmap); + bpf_map_container_free(cmap); } /* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index f9a87dcc5535..3e99c10f1729 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -163,13 +163,13 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = bpf_map_area_alloc(sizeof(*dtab), NUMA_NO_NODE); + dtab = bpf_map_container_alloc(sizeof(*dtab), NUMA_NO_NODE); if (!dtab) return ERR_PTR(-ENOMEM); err = dev_map_init_map(dtab, attr); if (err) { - bpf_map_area_free(dtab); + bpf_map_container_free(dtab); return ERR_PTR(err); } @@ -240,7 +240,7 @@ static void dev_map_free(struct bpf_map *map) bpf_map_area_free(dtab->netdev_map); } - bpf_map_area_free(dtab); + bpf_map_container_free(dtab); } static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 8392f7f8a8ac..d2fb144276ab 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -495,7 +495,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); + htab = bpf_map_container_alloc(sizeof(*htab), NUMA_NO_NODE); if (!htab) return ERR_PTR(-ENOMEM); @@ -579,7 +579,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) bpf_map_area_free(htab->buckets); free_htab: lockdep_unregister_key(&htab->lockdep_key); - bpf_map_area_free(htab); + bpf_map_container_free(htab); return ERR_PTR(err); } @@ -1496,7 +1496,7 @@ static void htab_map_free(struct bpf_map *map) for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); lockdep_unregister_key(&htab->lockdep_key); - bpf_map_area_free(htab); + bpf_map_container_free(htab); } static void htab_map_seq_show_elem(struct bpf_map *map, void *key, diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 098cf336fae6..963dc82e3c50 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,7 +313,7 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = bpf_map_area_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node); + map = bpf_map_container_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node); if (!map) return ERR_PTR(-ENOMEM); @@ -345,7 +345,7 @@ static void cgroup_storage_map_free(struct bpf_map *_map) WARN_ON(!RB_EMPTY_ROOT(&map->root)); WARN_ON(!list_empty(&map->list)); - bpf_map_area_free(map); + bpf_map_container_free(map); } static int cgroup_storage_delete_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index d833496e9e42..15ccef2f10a3 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -558,7 +558,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) attr->value_size > LPM_VAL_SIZE_MAX) return ERR_PTR(-EINVAL); - trie = bpf_map_area_alloc(sizeof(*trie), NUMA_NO_NODE); + trie = bpf_map_container_alloc(sizeof(*trie), NUMA_NO_NODE); if (!trie) return ERR_PTR(-ENOMEM); @@ -609,7 +609,7 @@ static void trie_free(struct bpf_map *map) } out: - bpf_map_area_free(trie); + bpf_map_container_free(trie); } static int trie_get_next_key(struct bpf_map *map, void *_key, void *_next_key) diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 5a629a1b971c..02e3f4ef19f5 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -372,7 +372,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) attr->map_type != BPF_MAP_TYPE_HASH) return ERR_PTR(-EINVAL); - offmap = kzalloc(sizeof(*offmap), GFP_USER | __GFP_NOWARN); + offmap = bpf_map_container_alloc(sizeof(*offmap), NUMA_NO_NODE); if (!offmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index 8a5e060de63b..d4c0284c9089 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -74,7 +74,7 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) size = (u64) attr->max_entries + 1; queue_size = sizeof(*qs) + size * attr->value_size; - qs = bpf_map_area_alloc(queue_size, numa_node); + qs = bpf_map_container_alloc(queue_size, numa_node); if (!qs) return ERR_PTR(-ENOMEM); @@ -92,7 +92,7 @@ static void queue_stack_map_free(struct bpf_map *map) { struct bpf_queue_stack *qs = bpf_queue_stack(map); - bpf_map_area_free(qs); + bpf_map_container_free(qs); } static int __queue_map_get(struct bpf_map *map, void *value, bool delete) diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index e2618fb5870e..d36a60ca68f4 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -146,7 +146,7 @@ static void reuseport_array_free(struct bpf_map *map) * Once reaching here, all sk->sk_user_data is not * referencing this "array". "array" can be freed now. */ - bpf_map_area_free(array); + bpf_map_container_free(array); } static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) @@ -158,7 +158,7 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) return ERR_PTR(-EPERM); /* allocate all map elements and zero-initialize them */ - array = bpf_map_area_alloc(struct_size(array, ptrs, attr->max_entries), numa_node); + array = bpf_map_container_alloc(struct_size(array, ptrs, attr->max_entries), numa_node); if (!array) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index b483aea35f41..35258aa45236 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -164,7 +164,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = bpf_map_area_alloc(sizeof(*rb_map), NUMA_NO_NODE); + rb_map = bpf_map_container_alloc(sizeof(*rb_map), NUMA_NO_NODE); if (!rb_map) return ERR_PTR(-ENOMEM); @@ -172,7 +172,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); if (!rb_map->rb) { - bpf_map_area_free(rb_map); + bpf_map_container_free(rb_map); return ERR_PTR(-ENOMEM); } @@ -199,7 +199,7 @@ static void ringbuf_map_free(struct bpf_map *map) rb_map = container_of(map, struct bpf_ringbuf_map, map); bpf_ringbuf_free(rb_map->rb); - bpf_map_area_free(rb_map); + bpf_map_container_free(rb_map); } static void *ringbuf_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 1adbe67cdb95..0c3185406f56 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -100,7 +100,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); cost = n_buckets * sizeof(struct stack_map_bucket *) + sizeof(*smap); - smap = bpf_map_area_alloc(cost, bpf_map_attr_numa_node(attr)); + smap = bpf_map_container_alloc(cost, bpf_map_attr_numa_node(attr)); if (!smap) return ERR_PTR(-ENOMEM); @@ -120,7 +120,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) put_buffers: put_callchain_buffers(); free_smap: - bpf_map_area_free(smap); + bpf_map_container_free(smap); return ERR_PTR(err); } @@ -650,7 +650,7 @@ static void stack_map_free(struct bpf_map *map) bpf_map_area_free(smap->elems); pcpu_freelist_destroy(&smap->freelist); - bpf_map_area_free(smap); + bpf_map_container_free(smap); put_callchain_buffers(); } diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 1a1a81a11b37..a6f68ade200f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -339,11 +339,6 @@ void *bpf_map_area_alloc(u64 size, int numa_node) return __bpf_map_area_alloc(size, numa_node, false); } -void *bpf_map_area_mmapable_alloc(u64 size, int numa_node) -{ - return __bpf_map_area_alloc(size, numa_node, true); -} - void bpf_map_area_free(void *area) { kvfree(area); @@ -669,7 +664,6 @@ static void bpf_map_free_deferred(struct work_struct *work) security_bpf_map_free(map); kfree(map->off_arr); - bpf_map_release_memcg(map); /* implementation dependent freeing, map_free callback also does * bpf_map_free_kptr_off_tab, if needed. */ @@ -1218,8 +1212,6 @@ static int map_create(union bpf_attr *attr) if (err) goto free_map_sec; - bpf_map_save_memcg(map); - err = bpf_map_new_fd(map, f_flags); if (err < 0) { /* failed to allocate fd. diff --git a/net/core/sock_map.c b/net/core/sock_map.c index d0c43384d8bf..4b5876c4a47d 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,7 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = bpf_map_area_alloc(sizeof(*stab), NUMA_NO_NODE); + stab = bpf_map_container_alloc(sizeof(*stab), NUMA_NO_NODE); if (!stab) return ERR_PTR(-ENOMEM); @@ -52,7 +52,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) sizeof(struct sock *), stab->map.numa_node); if (!stab->sks) { - bpf_map_area_free(stab); + bpf_map_container_free(stab); return ERR_PTR(-ENOMEM); } @@ -361,7 +361,7 @@ static void sock_map_free(struct bpf_map *map) synchronize_rcu(); bpf_map_area_free(stab->sks); - bpf_map_area_free(stab); + bpf_map_container_free(stab); } static void sock_map_release_progs(struct bpf_map *map) @@ -1076,7 +1076,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); + htab = bpf_map_container_alloc(sizeof(*htab), NUMA_NO_NODE); if (!htab) return ERR_PTR(-ENOMEM); @@ -1106,7 +1106,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) return &htab->map; free_htab: - bpf_map_area_free(htab); + bpf_map_container_free(htab); return ERR_PTR(err); } @@ -1159,7 +1159,7 @@ static void sock_hash_free(struct bpf_map *map) synchronize_rcu(); bpf_map_area_free(htab->buckets); - bpf_map_area_free(htab); + bpf_map_container_free(htab); } static void *sock_hash_lookup_sys(struct bpf_map *map, void *key) diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index acc8e52a4f5f..6cf98f1c621a 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -75,7 +75,7 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) numa_node = bpf_map_attr_numa_node(attr); size = struct_size(m, xsk_map, attr->max_entries); - m = bpf_map_area_alloc(size, numa_node); + m = bpf_map_container_alloc(size, numa_node); if (!m) return ERR_PTR(-ENOMEM); @@ -90,7 +90,7 @@ static void xsk_map_free(struct bpf_map *map) struct xsk_map *m = container_of(map, struct xsk_map, map); synchronize_net(); - bpf_map_area_free(m); + bpf_map_container_free(m); } static int xsk_map_get_next_key(struct bpf_map *map, void *key, void *next_key) From patchwork Fri Jul 29 15:23:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46CB1C19F2A for ; Fri, 29 Jul 2022 15:23:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D6C006B007D; Fri, 29 Jul 2022 11:23:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D16CF8E0001; Fri, 29 Jul 2022 11:23:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB82C6B0080; Fri, 29 Jul 2022 11:23:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AD7876B007D for ; Fri, 29 Jul 2022 11:23:37 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 73BFB16112E for ; Fri, 29 Jul 2022 15:23:37 +0000 (UTC) X-FDA: 79740506874.13.5283429 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by imf07.hostedemail.com (Postfix) with ESMTP id CF4F4400AC for ; Fri, 29 Jul 2022 15:23:36 +0000 (UTC) Received: by mail-pg1-f171.google.com with SMTP id d7so1228719pgc.13 for ; Fri, 29 Jul 2022 08:23:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XcXG9h+ovnptbv0fZ4hOiyXlQOwZMvCOBp/cMXUb/u8=; b=IWFv+lclDgiPa3ymhPY9q6aCXqaeG/qISakVlu0Nxc7vWmoHPhw+T4sGoi6JRopsnP beNYUbztVH4F05vhDRIEp5hpbNu8la/64Le0T/TATOhuurnQSIn4f4qzv3WX+Zxu5YlA BzTeCxjeYl9BaU8uiZ1pVgzP/UKYZnzLA2YkKOufmRKU7RGSyTMesWuhc7upiOeKpLNr hZtaAoJLyXQgVZ0QK/wkFk1ll2QD+5tmLlkr0eaIISFO6LKcrhBECvUjmQ5GANuf84KW AOsNdYkzh6EcccD3engjSYNrarMZbKYDurIFIbooVl9jO899EeL5EM2L1C6q03lTpL4l o9rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XcXG9h+ovnptbv0fZ4hOiyXlQOwZMvCOBp/cMXUb/u8=; b=XR1gF4C9YOSQoQWmjF90CAUW/q36XH5DikzFqtACj4h2dGgZ9IjU6EPU4Nto6qD3T/ Eay0nYljy28tJkZIh3ydB/9M/Erg06kyYpP1Y2WRFffRyrf3sPZTYCrmNQkgz/prEaN/ CcwKNH2xrcILIkGymHnAAnaVpzDxb50soPZTB6oa+3h6lHjaGM6CsY4zRCuODG4xgmPk p8qgLDKBqTV12Dpo1tRQhIKfSLczPK2x2nFVx86zJ7CSAlH2WAh5VOQ7GtoxDFsjrmLP HWkkyCEvve8UQrcU9yxRsk7/hgfgurzljCbqWsrbu0FBxAlrSO9t9LTwphRVaR/8Ecnu l48A== X-Gm-Message-State: ACgBeo3cKddRCHq8iVFvAVBeF48hVJ8j8JcULJ/5I89WJZ0DGwQsPoE8 MBfLxxREL6enAux9ojRZQMg= X-Google-Smtp-Source: AA6agR7SIQhSbBEU1Bv1FwPss3jvTX+PEYUQva3Awy8g6XTzQUfPeNSOJJ5hWGFT56tk2s0IpV4uIQ== X-Received: by 2002:aa7:8289:0:b0:52c:e97c:dbe4 with SMTP id s9-20020aa78289000000b0052ce97cdbe4mr860524pfm.49.1659108215736; Fri, 29 Jul 2022 08:23:35 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:34 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 07/15] bpf: Define bpf_map_get_memcg for !CONFIG_MEMCG_KMEM Date: Fri, 29 Jul 2022 15:23:08 +0000 Message-Id: <20220729152316.58205-8-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IWFv+lcl; spf=pass (imf07.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108216; a=rsa-sha256; cv=none; b=qFb49vs97kj1bNBjhblsTBUlUnyxSx43+2tFNByhWvA3FDVVpc+TB480AbgTmDgi3mruB4 1gFj4QzHxPqjBcu9lK0fs6OhbygHug0n+4jjtPorrmnDsLiFUMiHIJ5mf2f3lxfm37zfTg xjW0mTRzSzrXtFw37CfAcFb7vhmkh0A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108216; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XcXG9h+ovnptbv0fZ4hOiyXlQOwZMvCOBp/cMXUb/u8=; b=vASMEWdhRNynLquJfNE+eBiAsGfwfPCxamsQNX0DIKJma8OTVuzvt5qAtwKPUI6LiFE9KV QYUbEU3NZ+C1kTlqHLJXGsqLLable8plE4IMWz7c/yMf1GkFwdy3I0wyy4nDoemSrpno+N tQQPSjDy1wuutoKiqUJSRMUkfHrrbZg= X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: CF4F4400AC Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IWFv+lcl; spf=pass (imf07.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: fmjmmojekaz7k1n69xi1qn7hferzq9tt X-Rspam-User: X-HE-Tag: 1659108216-418816 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Then we can use this helper when CONFIG_MEMCG_KMEM is not set. Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 10 ++++++++++ kernel/bpf/syscall.c | 7 ++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9ecead1042b9..2f0a611f12e5 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -361,6 +361,11 @@ struct mem_cgroup { extern struct mem_cgroup *root_mem_cgroup; +static inline struct mem_cgroup *root_memcg(void) +{ + return root_mem_cgroup; +} + enum page_memcg_data_flags { /* page->memcg_data is a pointer to an objcgs vector */ MEMCG_DATA_OBJCGS = (1UL << 0), @@ -1138,6 +1143,11 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, #define MEM_CGROUP_ID_SHIFT 0 #define MEM_CGROUP_ID_MAX 0 +static inline struct mem_cgroup *root_memcg(void) +{ + return NULL; +} + static inline struct mem_cgroup *folio_memcg(struct folio *folio) { return NULL; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index a6f68ade200f..7289ee1a300a 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -433,7 +433,7 @@ static struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) if (map->objcg) return get_mem_cgroup_from_objcg(map->objcg); - return root_mem_cgroup; + return root_memcg(); } void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, @@ -488,6 +488,11 @@ static void bpf_map_save_memcg(struct bpf_map *map) static void bpf_map_release_memcg(struct bpf_map *map) { } + +static struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) +{ + return root_memcg(); +} #endif /* From patchwork Fri Jul 29 15:23:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8048FC00144 for ; Fri, 29 Jul 2022 15:23:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 180EB6B007E; Fri, 29 Jul 2022 11:23:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1303E6B0080; Fri, 29 Jul 2022 11:23:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3C3D6B0081; Fri, 29 Jul 2022 11:23:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E2E9B6B007E for ; Fri, 29 Jul 2022 11:23:39 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B887814117C for ; Fri, 29 Jul 2022 15:23:39 +0000 (UTC) X-FDA: 79740506958.26.D5670D8 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf21.hostedemail.com (Postfix) with ESMTP id EA5D31C00DE for ; Fri, 29 Jul 2022 15:23:38 +0000 (UTC) Received: by mail-pl1-f177.google.com with SMTP id x7so4889268pll.7 for ; Fri, 29 Jul 2022 08:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yY5QSV/StxaGhIVU47wKOjMZAWLb0lpndrHiTZxnvTs=; b=PVhPRrUg7ts2cUfPBRcTovmnlKW7Z5k7eaG8OzXLVvnfJOvHgyu+i8x7VKhygLtv0r YD3O8X/bmJOLVfIibZhs2g0Oh4Q0u2gPzYtb3CSldJZBSI8bltEPv3oYgBT3wohii4aN GA6Bh2xyG7YoSAZxnPRssoav9dUyYFHVEnwsc6mvYjBl6RHluaYd/yGYx5DrUGrSku4w LO++RYVwgK4A+0X8GEFqU4G6yNqq2xuDDdvLgO5TITKmKvXoOB6g95Mo1gwnhGFV15uE gLHS+9YqT2h0PbERhHCnS4uDsW/Fe9QWBsQyUwADGKdNaegJFma3lcge7hyCC9a5khQZ 3Buw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yY5QSV/StxaGhIVU47wKOjMZAWLb0lpndrHiTZxnvTs=; b=sxuYeqVkeNzbulBukFBg7x/kB5AjECkihKpXmS7eoV/AtIgYnilpnJv47qIHCR/CYd OKqeYBVT9whn2T5UpgqVGK9u79dfnaWYy+ZoV9dVQW0VzQ6JEfv+NQ85ko3dhBNBqljO lQZL68Bxq7iC8JVpaTqs54UaR45hoSvj/5vdA5rWWvaQXl/MMRmxFDK8y4ERSnaodhQF 8r+4xqrmwEJ35v6k4NBETRC7GV7GLx3k/ZfeFk6xfNXBJ5V/9Ktqi4vxa5jyK4FB9AzC ww/5Z06l9k7eJzmdWqWj4zXBqth4iYeD5KYz6M9lHrhFxZP/nxC6nGQ79YFCxZXcpBp+ NbtQ== X-Gm-Message-State: ACgBeo0mHHFwagnbvw+cSvKtWcKIpyqTHhPQiw9u6veENOsBVUqiYmhy 0OFsNyBZpKLEgw7cIv0bXd0= X-Google-Smtp-Source: AA6agR6a0ENsUe9oe9ps5/wlhZJjT9cX4gsopZOaAdU3UCGEo0KTZUOFx9XY0Hyud2hzLvkqfJ6qwg== X-Received: by 2002:a17:902:e791:b0:16d:7f5d:d29e with SMTP id cp17-20020a170902e79100b0016d7f5dd29emr4344400plb.14.1659108217878; Fri, 29 Jul 2022 08:23:37 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:36 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 08/15] bpf: Use scope-based charge for bpf_map_area_alloc Date: Fri, 29 Jul 2022 15:23:09 +0000 Message-Id: <20220729152316.58205-9-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108219; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yY5QSV/StxaGhIVU47wKOjMZAWLb0lpndrHiTZxnvTs=; b=pCN/SghtOq8FRpgq+kacA6VGYKq4KLVu4TEH/JCsJpO7e6aptxGVgSpo+LhBZSTdlOJsxs 1CHcKIrf/xiKWAtinM8gvmVwY671Fz+uH3K/CL0GOZcjFT42vxtVNoz4hxm9sOZFSyRgtf 22Go2k9mZdeIE1MixjBF3/bYBHQtf7w= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PVhPRrUg; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108219; a=rsa-sha256; cv=none; b=oj+/2I1hHMk8J6jTWA63gockIVtogxEsj8ZrVB0fp7F+PHJey6wOnUKIWggSgTFUjjdmCl 5Pcr21FGsWSBAWaofGAWM5lu172aRHaxeXzvE9Wdua9+kGIFOUFth6wFXORLNlDPR90vki dx7cX7SdIq28OGnBCDU5SK+EqkzaxnQ= X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EA5D31C00DE X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PVhPRrUg; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: x3oxzpjt3bkbzb8i7ytnq85t7w3nssg5 X-HE-Tag: 1659108218-554234 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's also get memcg from the bpf map in bpf_map_area_alloc() instead of using the current memcg. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 2 +- kernel/bpf/bpf_struct_ops.c | 4 ++-- kernel/bpf/cpumap.c | 2 +- kernel/bpf/devmap.c | 12 ++++++++---- kernel/bpf/hashtab.c | 5 +++-- kernel/bpf/ringbuf.c | 14 +++++++++----- kernel/bpf/stackmap.c | 3 ++- kernel/bpf/syscall.c | 28 ++++++++++++++++++---------- net/core/sock_map.c | 7 ++++--- 9 files changed, 48 insertions(+), 29 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 3f9893e14124..711d9b1829d4 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1637,7 +1637,7 @@ void bpf_map_put(struct bpf_map *map); void *bpf_map_container_alloc(u64 size, int numa_node); void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, u32 align, u32 offset); -void *bpf_map_area_alloc(u64 size, int numa_node); +void *bpf_map_area_alloc(struct bpf_map *map, u64 size, int numa_node); void bpf_map_area_free(void *base); void bpf_map_container_free(void *base); bool bpf_map_write_active(const struct bpf_map *map); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 66df3059a3fe..874fda7e2b8b 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -625,9 +625,9 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) st_map->st_ops = st_ops; map = &st_map->map; - st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE); + st_map->uvalue = bpf_map_area_alloc(map, vt->size, NUMA_NO_NODE); st_map->links = - bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_links *), + bpf_map_area_alloc(map, btf_type_vlen(t) * sizeof(struct bpf_links *), NUMA_NO_NODE); st_map->image = bpf_jit_alloc_exec(PAGE_SIZE); if (!st_map->uvalue || !st_map->links || !st_map->image) { diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 23e941826eec..95c1642deaf6 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -110,7 +110,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) } /* Alloc array for possible remote "destination" CPUs */ - cmap->cpu_map = bpf_map_area_alloc(cmap->map.max_entries * + cmap->cpu_map = bpf_map_area_alloc(&cmap->map, cmap->map.max_entries * sizeof(struct bpf_cpu_map_entry *), cmap->map.numa_node); if (!cmap->cpu_map) diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 3e99c10f1729..b625d578bc93 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -88,13 +88,15 @@ static DEFINE_PER_CPU(struct list_head, dev_flush_list); static DEFINE_SPINLOCK(dev_map_lock); static LIST_HEAD(dev_map_list); -static struct hlist_head *dev_map_create_hash(unsigned int entries, +static struct hlist_head *dev_map_create_hash(struct bpf_map *map, + unsigned int entries, int numa_node) { int i; struct hlist_head *hash; - hash = bpf_map_area_alloc((u64) entries * sizeof(*hash), numa_node); + hash = bpf_map_area_alloc(map, (u64) entries * sizeof(*hash), + numa_node); if (hash != NULL) for (i = 0; i < entries; i++) INIT_HLIST_HEAD(&hash[i]); @@ -138,14 +140,16 @@ static int dev_map_init_map(struct bpf_dtab *dtab, union bpf_attr *attr) } if (attr->map_type == BPF_MAP_TYPE_DEVMAP_HASH) { - dtab->dev_index_head = dev_map_create_hash(dtab->n_buckets, + dtab->dev_index_head = dev_map_create_hash(&dtab->map, + dtab->n_buckets, dtab->map.numa_node); if (!dtab->dev_index_head) return -ENOMEM; spin_lock_init(&dtab->index_lock); } else { - dtab->netdev_map = bpf_map_area_alloc((u64) dtab->map.max_entries * + dtab->netdev_map = bpf_map_area_alloc(&dtab->map, + (u64) dtab->map.max_entries * sizeof(struct bpf_dtab_netdev *), dtab->map.numa_node); if (!dtab->netdev_map) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index d2fb144276ab..2a34a115e14f 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -331,7 +331,8 @@ static int prealloc_init(struct bpf_htab *htab) if (htab_has_extra_elems(htab)) num_entries += num_possible_cpus(); - htab->elems = bpf_map_area_alloc((u64)htab->elem_size * num_entries, + htab->elems = bpf_map_area_alloc(&htab->map, + (u64)htab->elem_size * num_entries, htab->map.numa_node); if (!htab->elems) return -ENOMEM; @@ -532,7 +533,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) goto free_htab; err = -ENOMEM; - htab->buckets = bpf_map_area_alloc(htab->n_buckets * + htab->buckets = bpf_map_area_alloc(&htab->map, htab->n_buckets * sizeof(struct bucket), htab->map.numa_node); if (!htab->buckets) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 35258aa45236..7c875d4d5b2f 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -59,7 +59,9 @@ struct bpf_ringbuf_hdr { u32 pg_off; }; -static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) +static struct bpf_ringbuf *bpf_ringbuf_area_alloc(struct bpf_map *map, + size_t data_sz, + int numa_node) { const gfp_t flags = GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL | __GFP_NOWARN | __GFP_ZERO; @@ -89,7 +91,7 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) * user-space implementations significantly. */ array_size = (nr_meta_pages + 2 * nr_data_pages) * sizeof(*pages); - pages = bpf_map_area_alloc(array_size, numa_node); + pages = bpf_map_area_alloc(map, array_size, numa_node); if (!pages) return NULL; @@ -127,11 +129,12 @@ static void bpf_ringbuf_notify(struct irq_work *work) wake_up_all(&rb->waitq); } -static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node) +static struct bpf_ringbuf *bpf_ringbuf_alloc(struct bpf_map *map, + size_t data_sz, int numa_node) { struct bpf_ringbuf *rb; - rb = bpf_ringbuf_area_alloc(data_sz, numa_node); + rb = bpf_ringbuf_area_alloc(map, data_sz, numa_node); if (!rb) return NULL; @@ -170,7 +173,8 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) bpf_map_init_from_attr(&rb_map->map, attr); - rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); + rb_map->rb = bpf_ringbuf_alloc(&rb_map->map, attr->max_entries, + rb_map->map.numa_node); if (!rb_map->rb) { bpf_map_container_free(rb_map); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 0c3185406f56..c9a91ca05a03 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -48,7 +48,8 @@ static int prealloc_elems_and_freelist(struct bpf_stack_map *smap) (u64)smap->map.value_size; int err; - smap->elems = bpf_map_area_alloc(elem_size * smap->map.max_entries, + smap->elems = bpf_map_area_alloc(&smap->map, + elem_size * smap->map.max_entries, smap->map.numa_node); if (!smap->elems) return -ENOMEM; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 7289ee1a300a..4f893d2ac4fd 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -334,16 +334,6 @@ static void *__bpf_map_area_alloc(u64 size, int numa_node, bool mmapable) flags, numa_node, __builtin_return_address(0)); } -void *bpf_map_area_alloc(u64 size, int numa_node) -{ - return __bpf_map_area_alloc(size, numa_node, false); -} - -void bpf_map_area_free(void *area) -{ - kvfree(area); -} - static u32 bpf_map_flags_retain_permanent(u32 flags) { /* Some map creation flags are not tied to the map object but @@ -495,6 +485,24 @@ static struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) } #endif +void *bpf_map_area_alloc(struct bpf_map *map, u64 size, int numa_node) +{ + struct mem_cgroup *memcg, *old_memcg; + void *ptr; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + ptr = __bpf_map_area_alloc(size, numa_node, false); + set_active_memcg(old_memcg); + + return ptr; +} + +void bpf_map_area_free(void *area) +{ + kvfree(area); +} + /* * The return pointer is a bpf_map container, as follow, * struct bpf_map_container { diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 4b5876c4a47d..1f49dc89822c 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -48,8 +48,9 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) bpf_map_init_from_attr(&stab->map, attr); raw_spin_lock_init(&stab->lock); - stab->sks = bpf_map_area_alloc((u64) stab->map.max_entries * - sizeof(struct sock *), + stab->sks = bpf_map_area_alloc(&stab->map, + (u64)stab->map.max_entries * + sizeof(struct sock *), stab->map.numa_node); if (!stab->sks) { bpf_map_container_free(stab); @@ -1091,7 +1092,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) goto free_htab; } - htab->buckets = bpf_map_area_alloc(htab->buckets_num * + htab->buckets = bpf_map_area_alloc(&htab->map, htab->buckets_num * sizeof(struct bpf_shtab_bucket), htab->map.numa_node); if (!htab->buckets) { From patchwork Fri Jul 29 15:23:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DB36C19F2A for ; Fri, 29 Jul 2022 15:23:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C30036B0080; Fri, 29 Jul 2022 11:23:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE0D46B0081; Fri, 29 Jul 2022 11:23:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0B616B0082; Fri, 29 Jul 2022 11:23:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 90D586B0080 for ; Fri, 29 Jul 2022 11:23:41 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7147A1C696F for ; Fri, 29 Jul 2022 15:23:41 +0000 (UTC) X-FDA: 79740507042.16.7B3D32F Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf21.hostedemail.com (Postfix) with ESMTP id 1F4A31C00D5 for ; Fri, 29 Jul 2022 15:23:40 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id w17-20020a17090a8a1100b001f326c73df6so3942907pjn.3 for ; Fri, 29 Jul 2022 08:23:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/QlHQZkaHYaQUqr8wiHtqQx5hHw1sh9z9adPB7Jsz2k=; b=lvJP+1FG/ePBYe0WKUD8cI1RjJ3OLvTLP05nfMyRIXF9Cn5hW4SsClBM2ZTxhdhyfB VS+tbTDnauOxdE9A1b+U/wKrsxMm0JUSfjMQ4OA7ReBBgDPNlXNXM/4Exj46MrO6U2Qq eY+YxM/Dme8BG0xRfuw5ibtuq5M3bLSS6EQpip4wui5yyZk7p9sdOIa1gMhLu3jthyIV EE4K9ni/8hThPsbrOO+blVLo2ShKZpLSqE2i7c0Irm39kaiXJudhoRJRkOEQUBD8FCsU qKmoWH6ZmEaZWXbCwGh/RmnkYwg38/zFJ9PX38EE1F0XpJrJc9vnUIPcHcCtJZoBWkos VIIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/QlHQZkaHYaQUqr8wiHtqQx5hHw1sh9z9adPB7Jsz2k=; b=YHqYFh8DyAakab3UBF1x61wjnqLXD6WkQDz93sONKJMTxa546gOY0unaErRFV72nwm BYYiwBZZoXmXrY/KfJsor8lxJ0NLfb7YTvLD2KPJc63rOguA2a9Rb9+88CQg4MlJ64wu D1tL+WeHR4MKBSeAPJhHglL8KEfUluwmG611cTVGiL2bM/LAV5NtL+fq9fVzX88ncaXy lIFyyUzAo5E7trr5K9jUt062pe7J7sFq7vsRkuUgP5Vxl86YTaPTxcdw5UoAc0wiqZNf 7OVmRkllYBte1spjC05uEhogVQnS4TUDAP9vVZDgTEuvnjcceFdSJMNG+1AuyZgH56Zl G52A== X-Gm-Message-State: ACgBeo3xl8YAuSfmO7SHKauf2N0XG5ZpQw1HMacZZ2Eqlx83NE+Bl1g4 HJk+NtaQ+fPJo64LC2hx0BWYzEb4ja7+P797 X-Google-Smtp-Source: AA6agR5CclJJZ7kgrC33kQLUVREUZ4UgNBzC2JWzJideJc20Jf2HFETAMi58PCicfJv3XCN7cAM58g== X-Received: by 2002:a17:903:2343:b0:16c:1efe:1fe0 with SMTP id c3-20020a170903234300b0016c1efe1fe0mr4542669plh.106.1659108220144; Fri, 29 Jul 2022 08:23:40 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:39 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 09/15] bpf: Use bpf_map_kzalloc in arraymap Date: Fri, 29 Jul 2022 15:23:10 +0000 Message-Id: <20220729152316.58205-10-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=lvJP+1FG; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108221; a=rsa-sha256; cv=none; b=mDltUAlrfIDGjuHE/dLspiz8TSABeJPzJiZKD7F6c8PXLypqoOCOg1C0wtEtZIjfU1sm5M eDGOnwy1UhRT/o9x2CCURUAmfamyG2PzK3OVvwZmg9/AcbqIbAnBtNEiqjoKnVR6WJjBMH msB4tyDnxDhuwJOYNGc7Sra57MpsouA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108221; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/QlHQZkaHYaQUqr8wiHtqQx5hHw1sh9z9adPB7Jsz2k=; b=2ve/E1htnmQlomznOZTngi/YWkWw7oMsE73FTTXbeSiSBVIgduzAB80o2xgsLnJfKlXCUS sMkUJIFvY2SI2Mgu/xWj+aA7W5MGRE+ROAPTCtyGjZ+Z84vzo5UIkyxMMpkiWBigORrDaM 9R8YEAEEtscsCAz6P8bAa5+bV9cA8ys= Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=lvJP+1FG; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Rspam-User: X-Stat-Signature: kjns6xk59k61d4nbafsph9tmdtdagxrw X-Rspamd-Queue-Id: 1F4A31C00D5 X-HE-Tag: 1659108220-596897 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allocates memory after map creation, then we can use the generic helper bpf_map_kzalloc() instead of the open-coded kzalloc(). Signed-off-by: Yafang Shao --- kernel/bpf/arraymap.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 9517619dbe8d..bfcfc5df9983 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -1093,20 +1093,20 @@ static struct bpf_map *prog_array_map_alloc(union bpf_attr *attr) struct bpf_array_aux *aux; struct bpf_map *map; - aux = kzalloc(sizeof(*aux), GFP_KERNEL_ACCOUNT); - if (!aux) + map = array_map_alloc(attr); + if (IS_ERR(map)) return ERR_PTR(-ENOMEM); + aux = bpf_map_kzalloc(map, sizeof(*aux), GFP_KERNEL); + if (!aux) { + array_map_free(map); + return ERR_PTR(-ENOMEM); + } + INIT_WORK(&aux->work, prog_array_map_clear_deferred); INIT_LIST_HEAD(&aux->poke_progs); mutex_init(&aux->poke_mutex); - map = array_map_alloc(attr); - if (IS_ERR(map)) { - kfree(aux); - return map; - } - container_of(map, struct bpf_array, map)->aux = aux; aux->map = map; From patchwork Fri Jul 29 15:23:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 500C3C19F2A for ; Fri, 29 Jul 2022 15:23:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E51A06B0081; Fri, 29 Jul 2022 11:23:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E02486B0082; Fri, 29 Jul 2022 11:23:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C559C8E0001; Fri, 29 Jul 2022 11:23:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B3CDB6B0081 for ; Fri, 29 Jul 2022 11:23:43 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 88CF98118F for ; Fri, 29 Jul 2022 15:23:43 +0000 (UTC) X-FDA: 79740507126.11.5118207 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf14.hostedemail.com (Postfix) with ESMTP id 1EFC81000CA for ; Fri, 29 Jul 2022 15:23:42 +0000 (UTC) Received: by mail-pj1-f54.google.com with SMTP id q7-20020a17090a7a8700b001f300db8677so5565753pjf.5 for ; Fri, 29 Jul 2022 08:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jAUWseH3fRr8riszGReq2qXDLxnVVqoEg86tgbmI8NE=; b=fmrI2QyU7IHbIWzCz94cxxmolN8UgJDWb3QaJ65Tp00KLnhEL9aZ2FVu4/xeQFvkxm Vf6GZuOm6d/T0FvqGvBzpq/iwTAPES9t1kDoGxhPm8QpjxLmP/5UJGNNb0u+Sw5CvaJU w3rQTlHgYH+YBmFCql1y8JfODcl23WsL8RFZh1+IX1gF88xfPVACbPoI7yeFykfu30rd ds2nhZtOma9CpW8KYv3890dSigOkV30IpMYMN9A/3owa6ayTK+XMB4ZF0bjQO7p0A0kv 6ONIyiiNrA6VEQXauRHRTLAUlWQCeR3w+k0iKKKdb6wLCGF+rBN/rkb2Yv9HkOoqjBOs 25yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jAUWseH3fRr8riszGReq2qXDLxnVVqoEg86tgbmI8NE=; b=UeBorp+CysLyIFoZt5mBQ6a8gqCFDuhz7WfHvB6QU/pxu8FgWap49t0Ma3JRF371bk OUjug/ISrYok+eXutvaPPM9+F3OLXPFu2xN60wJLxF9YIuZtAq60aXq2oORaQbeuItGt MdXwCUrm4CBPsHihjEwtXMbGvRRiIGi4QLt8BHEL3Dt62opHyV2K4xlq76myrVRg+gmn sILlbM1BpENuPcmBN9GWGfFsofOXmhPE0TeEqqLfnZQ8+xrsufGx2PAQN6VbGIHAYJno vdSuSbi6JCU4Xd40AYso65opczLpGtz6ZSzhZNmMMWL1DPTmYQyotzSasdwh5opWIo/f f+gQ== X-Gm-Message-State: ACgBeo0hOKtSgw6QZb2ujwyqhwl2q7Sa/4iGW3kuouBze8hU+EVrnCdy ik44OG6Y53rpOtHxbwq7nCs= X-Google-Smtp-Source: AA6agR73Gj5Asvp4W0xUgGklQRFVIg49NlQKP3kuhqIKi/rewhM/3O8FzmBipmh7MLHSJUTpYlMo1A== X-Received: by 2002:a17:903:11c9:b0:16b:8293:c5a1 with SMTP id q9-20020a17090311c900b0016b8293c5a1mr4435414plh.72.1659108222266; Fri, 29 Jul 2022 08:23:42 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:41 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 10/15] bpf: Use bpf_map_pages_alloc in ringbuf Date: Fri, 29 Jul 2022 15:23:11 +0000 Message-Id: <20220729152316.58205-11-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108223; a=rsa-sha256; cv=none; b=7WC47il8Qeu6oYqrCw9wrr0796kFjQ/NL4CFtTH7KlFqY1Ct4Jg+ciSP9Or+cwKxlqY9H/ XnhacLoDeNkcj02lqBwjtw/y/0TWcCb4/+B1veT2d9FVEx+xPS21PO+fQuC8e9mSy+t9A3 yDrDjdcBnBYWmiCOLaB+QRoz4LcYQSg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=fmrI2QyU; spf=pass (imf14.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108223; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jAUWseH3fRr8riszGReq2qXDLxnVVqoEg86tgbmI8NE=; b=a/njWthfbbQGBOsMTadIMlZ9ZYwa3fQN34JWal7RTpyO1WeBJThr+IAZ1x0syVbc75bf2z 9tWmXHOx2OakXeivaPEv2D1dHfF4sC/ee0gUnyoQe72tQColoIooQbafEhx5BBrt+OxWkl UeNQwIQN6ktE3iBzBmwUl6TTv6Gu7ao= Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=fmrI2QyU; spf=pass (imf14.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1EFC81000CA X-Stat-Signature: brtb1ttdjcmomg8oj3b4jiwsie4iy1qs X-HE-Tag: 1659108222-213320 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper bpf_map_pages_alloc() for this memory allocation. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 4 ++++ kernel/bpf/ringbuf.c | 27 +++++++++------------------ kernel/bpf/syscall.c | 41 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 18 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 711d9b1829d4..4af72d2b6d73 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1638,8 +1638,12 @@ void *bpf_map_container_alloc(u64 size, int numa_node); void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, u32 align, u32 offset); void *bpf_map_area_alloc(struct bpf_map *map, u64 size, int numa_node); +void *bpf_map_pages_alloc(struct bpf_map *map, struct page **pages, + int nr_meta_pages, int nr_data_pages, int nid, + gfp_t flags, unsigned int order); void bpf_map_area_free(void *base); void bpf_map_container_free(void *base); +void bpf_map_pages_free(struct page **pages, int nr_pages); bool bpf_map_write_active(const struct bpf_map *map); void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); int generic_map_lookup_batch(struct bpf_map *map, diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 7c875d4d5b2f..25973cab251d 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -63,15 +63,15 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(struct bpf_map *map, size_t data_sz, int numa_node) { - const gfp_t flags = GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL | + const gfp_t flags = GFP_KERNEL | __GFP_RETRY_MAYFAIL | __GFP_NOWARN | __GFP_ZERO; int nr_meta_pages = RINGBUF_PGOFF + RINGBUF_POS_PAGES; int nr_data_pages = data_sz >> PAGE_SHIFT; int nr_pages = nr_meta_pages + nr_data_pages; - struct page **pages, *page; struct bpf_ringbuf *rb; + struct page **pages; size_t array_size; - int i; + void *ptr; /* Each data page is mapped twice to allow "virtual" * continuous read of samples wrapping around the end of ring @@ -95,16 +95,10 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(struct bpf_map *map, if (!pages) return NULL; - for (i = 0; i < nr_pages; i++) { - page = alloc_pages_node(numa_node, flags, 0); - if (!page) { - nr_pages = i; - goto err_free_pages; - } - pages[i] = page; - if (i >= nr_meta_pages) - pages[nr_data_pages + i] = page; - } + ptr = bpf_map_pages_alloc(map, pages, nr_meta_pages, nr_data_pages, + numa_node, flags, 0); + if (!ptr) + goto err_free_pages; rb = vmap(pages, nr_meta_pages + 2 * nr_data_pages, VM_MAP | VM_USERMAP, PAGE_KERNEL); @@ -116,8 +110,6 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(struct bpf_map *map, } err_free_pages: - for (i = 0; i < nr_pages; i++) - __free_page(pages[i]); bpf_map_area_free(pages); return NULL; } @@ -189,11 +181,10 @@ static void bpf_ringbuf_free(struct bpf_ringbuf *rb) * to unmap rb itself with vunmap() below */ struct page **pages = rb->pages; - int i, nr_pages = rb->nr_pages; + int nr_pages = rb->nr_pages; vunmap(rb); - for (i = 0; i < nr_pages; i++) - __free_page(pages[i]); + bpf_map_pages_free(pages, nr_pages); bpf_map_area_free(pages); } diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 4f893d2ac4fd..5c13782839f3 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -559,6 +559,47 @@ void bpf_map_container_free(void *container) kvfree(container); } +void *bpf_map_pages_alloc(struct bpf_map *map, struct page **pages, + int nr_meta_pages, int nr_data_pages, int nid, + gfp_t flags, unsigned int order) +{ + int nr_pages = nr_meta_pages + nr_data_pages; + struct mem_cgroup *memcg, *old_memcg; + struct page *page; + int i; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + for (i = 0; i < nr_pages; i++) { + page = alloc_pages_node(nid, flags | __GFP_ACCOUNT, order); + if (!page) { + nr_pages = i; + set_active_memcg(old_memcg); + goto err_free_pages; + } + pages[i] = page; + if (i >= nr_meta_pages) + pages[nr_data_pages + i] = page; + } + set_active_memcg(old_memcg); + + return pages; + +err_free_pages: + for (i = 0; i < nr_pages; i++) + __free_page(pages[i]); + + return NULL; +} + +void bpf_map_pages_free(struct page **pages, int nr_pages) +{ + int i; + + for (i = 0; i < nr_pages; i++) + __free_page(pages[i]); +} + static int bpf_map_kptr_off_cmp(const void *a, const void *b) { const struct bpf_map_value_off_desc *off_desc1 = a, *off_desc2 = b; From patchwork Fri Jul 29 15:23:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69A64C00144 for ; Fri, 29 Jul 2022 15:23:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3C9B6B0082; Fri, 29 Jul 2022 11:23:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9F786B0083; Fri, 29 Jul 2022 11:23:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1C378E0001; Fri, 29 Jul 2022 11:23:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C3A896B0082 for ; Fri, 29 Jul 2022 11:23:45 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9B0DCAB332 for ; Fri, 29 Jul 2022 15:23:45 +0000 (UTC) X-FDA: 79740507210.17.E82FB33 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf11.hostedemail.com (Postfix) with ESMTP id 3C4EC40041 for ; Fri, 29 Jul 2022 15:23:45 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id w63-20020a17090a6bc500b001f3160a6011so6814536pjj.5 for ; Fri, 29 Jul 2022 08:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z/tZbl83Zz5RN5ZtLnfBVgtLVcF0QM8whQKykqg7AFs=; b=Odn0BIB7DUWvvQYR+O232jYQnTPeV1M0T4uHezkKllPsrtVCWuN6AzQ9mmkzucvR59 NX5o5yZSdNiIqSyZzPWhgHumWOjIJL+SKOtj8RlgwBtnTHMgLQhYipVDoMc5ijk8yzqj 8VrpnerKwh4TL5/YY4y9Q3NIOzSrvZoT2ZrhHkofgszttpTtHSgak+49Gyl7kIyMrLo5 g72j99GhmqgfeRYM72OCw+fhCkgTDLLIEUg36nSIcn58t1eyOlb4mYDULoC8UEz12ewD eC5PlwZh3NoqqPlJW8eZedRUR/d9FhBuaStavi861xJUBbKYcZykH8Kgu2gga/dsQNdg d2QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z/tZbl83Zz5RN5ZtLnfBVgtLVcF0QM8whQKykqg7AFs=; b=XH6BSJrVV2LWliq/5dOPOipwRfJi5la3d/PzCsVz0fuuHEtGZ0XRzpwOMU3IAwdXke zNI6hfi1Xqer119kVS7frWuccHZFApXVZFQMIk4bCPL4uY8GKTCndTT0dstlmDtdv4tP 18/gi1tfFtwU2tLk0CCM5tp4S6tqJPeO479GFMJQBS+KSMOIHDm4wnU/5hH+Vl9j5BiR 7z7UcqTbTEQgxd7wZlspsOoCG+hLPE3nFl3UuKtfNeqw/leqSMqQ3t6ZVzd42tMYQKqM bayNm534nXI6xhCb51kf4+ln1+TpM7BbEkKzUVgNyCN7NZirKMP9sFjgFQsCtnKcXtsq PmEw== X-Gm-Message-State: ACgBeo0CeCIEYAeCz45ZW4YO5f9Xezq0NCekjoOecfMQhVtU6hLQtarV HGK2zPMZ2JsPiq8DTg8W2EA= X-Google-Smtp-Source: AA6agR7BcQrR/Mgac/I67FnC6AY4rn3lQ3Gnc2LbnKyRmp4ej47lpNm/jWkrdbAsyc1iR3dwwKRDNQ== X-Received: by 2002:a17:902:8502:b0:16c:c5c5:a198 with SMTP id bj2-20020a170902850200b0016cc5c5a198mr4366289plb.88.1659108224367; Fri, 29 Jul 2022 08:23:44 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:43 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 11/15] bpf: Use bpf_map_kvcalloc in bpf_local_storage Date: Fri, 29 Jul 2022 15:23:12 +0000 Message-Id: <20220729152316.58205-12-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Odn0BIB7; spf=pass (imf11.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108225; a=rsa-sha256; cv=none; b=JSJFEXIWfjxGe3/hYt47ms/TFTc3tHAxS98miImNlJJLUj0h0LOQ9fab+cAMPLsY2KTW3y snOgv5sxrP837GrdF3ktPo8tiXj28mDuSTL15God132QrqXo4SCsW7wESuLIhn667ebGbu UWxgVkKPelfE/qup638sil43IljtQcQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108225; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z/tZbl83Zz5RN5ZtLnfBVgtLVcF0QM8whQKykqg7AFs=; b=UCdK1y24QG7mr0O0D33A4dwuve9Q50DpTQyoPFDGuIJh6/yzsw5fYPdvDrAo7mvOzIKqKO /q96AcMogDMYoVSu/b1DHIrHUouKdWl6RdOygDm00Ybnf5j97xJ/rhEyfEz5eiSU9Krfeo LEW1PBTe17eq5R/SQPTGHCK69iA7v/8= Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Odn0BIB7; spf=pass (imf11.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3C4EC40041 X-Stat-Signature: defqgaqtdsd41p1gmrooofy8ukeq89ri X-HE-Tag: 1659108225-800088 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper bpf_map_kvcalloc() for this memory allocation. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 8 ++++++++ kernel/bpf/bpf_local_storage.c | 4 ++-- kernel/bpf/syscall.c | 14 ++++++++++++++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 4af72d2b6d73..605214fedd6d 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1662,6 +1662,8 @@ struct bpf_prog *bpf_prog_get_curr_or_next(u32 *id); void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node); void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags); +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags); void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags); #else @@ -1678,6 +1680,12 @@ bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) return kzalloc(size, flags); } +static inline void * +bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, gfp_t flags) +{ + return kvcalloc(n, size, flags); +} + static inline void __percpu * bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 0b50cb2e1d5b..3440135c3612 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -620,8 +620,8 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) nbuckets = max_t(u32, 2, nbuckets); smap->bucket_log = ilog2(nbuckets); - smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap->buckets = bpf_map_kvcalloc(&smap->map, sizeof(*smap->buckets), + nbuckets, GFP_USER | __GFP_NOWARN); if (!smap->buckets) { bpf_map_container_free(smap); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 5c13782839f3..f7210fa8c3be 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -455,6 +455,20 @@ void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) return ptr; } +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags) +{ + struct mem_cgroup *memcg, *old_memcg; + void *ptr; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + ptr = kvcalloc(n, size, flags | __GFP_ACCOUNT); + set_active_memcg(old_memcg); + + return ptr; +} + void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) { From patchwork Fri Jul 29 15:23:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD4D4C19F2C for ; Fri, 29 Jul 2022 15:23:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B72D6B0083; Fri, 29 Jul 2022 11:23:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4655E6B0085; Fri, 29 Jul 2022 11:23:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3071E8E0001; Fri, 29 Jul 2022 11:23:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 201566B0083 for ; Fri, 29 Jul 2022 11:23:48 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id ECE00141610 for ; Fri, 29 Jul 2022 15:23:47 +0000 (UTC) X-FDA: 79740507294.08.E119E23 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf21.hostedemail.com (Postfix) with ESMTP id ED44F1C00D5 for ; Fri, 29 Jul 2022 15:23:46 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id w17-20020a17090a8a1100b001f326c73df6so3943240pjn.3 for ; Fri, 29 Jul 2022 08:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pNoGfDj1V1TnWI6qBdBSp31pmoo67RNKkb0LUb2vNLg=; b=L5yhSIt8gjsSmoj1o/cBW9AzyQHq5mIC+9at26SQ6hcp6bPNjtuU8B409uMpIMZulG QPxCn+2t9lkamdw4jzEf0G7zH9NHRcpwqv16qIJXXhJYDy5yKOj47BUrDDSVxTl9UkwT v9/v1FmerqehsorOy6r6RTfjjTPHyTpxQIEeQPNzVU2JKPvYvoBgpEtGQZpUiAJ1nqLi 3wkIv92UxZWulnBVltosq9eXBkLyzfnsQMRY6U+plLOjEM5r9evSaEkkJ1O4Vvl56u0l znIJhCMahnLYJnPpQcR8ECUaR1SL/ePMj83pAVrBVaC8Der/7J67vC6fQmZFhSNf1Wn4 c3/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pNoGfDj1V1TnWI6qBdBSp31pmoo67RNKkb0LUb2vNLg=; b=U/CtZwe3aVndtttDdOjO36Oqvuwaxt5mQBWXDWHFCsBZBiYR77oOkLda45CjJnyqs4 xhqtxNnnfGWeZnAuFvnkIDK1tqcvDzAIoU8WqUD0RB6E1adjiwQ3YFkbqhR4y2VBpGNc dibs8s9PljHs4Wb01qVLEXzpGipVSQXcTfNpYLgD6xFxIjuk+lCx3b4/N6Qtywf9Qpth MDB0RkthM7w9wFHEuSEByp/1ufHWRSG2Dsy6Qa9+nR/uefSpQRMVNwB/oz0B4RKCpDhM PjssxC2qyqp22MMEJXvB7W8M/ZRjhoSF642oPFAi/pnBhvNbLQIK/IM3J10WMDO8/NXO Suiw== X-Gm-Message-State: ACgBeo2gl7qzc+hwaX0oaCdQCJAZOOX5YCQFzJBXxUxF5Y9/ltQeXxbq 9LLGBGapzy9cbmmfEdTL41E= X-Google-Smtp-Source: AA6agR6NHf7CpfGmFU7zV1EhZPCbkoe+KGvzl+qrAN+Jzzpvjgr18XBqTeZdgzhLr65Hfxz2R6a99Q== X-Received: by 2002:a17:902:e5cd:b0:16d:b92c:6848 with SMTP id u13-20020a170902e5cd00b0016db92c6848mr4335575plf.56.1659108226544; Fri, 29 Jul 2022 08:23:46 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:45 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 12/15] mm, memcg: Add new helper get_obj_cgroup_from_cgroup Date: Fri, 29 Jul 2022 15:23:13 +0000 Message-Id: <20220729152316.58205-13-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=L5yhSIt8; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108227; a=rsa-sha256; cv=none; b=PsKhT9Yr01WXtzetb14HLNcPh/G0H7nXoleFsGUTOPZEOJYozA9wKCwmJd+MuhmqtCOy1f iwzofnAhuOnuhBqUv2WrmdGYgTQPhi+rOni6FJItovpkzetpS6VnROO9DGITZjixNV+sT1 RQCyaiZn4aPXm22QCJsSWI+gAzIIZWI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pNoGfDj1V1TnWI6qBdBSp31pmoo67RNKkb0LUb2vNLg=; b=lB8nMum191d6RYk4+ofJo4Udw9DBamrBpQB0YoZ1G5xuHRDPwSSEkQow2pj/b6UwK43asm GByWIIoJuGalbb4N0ZA9QVfg2ehzVWcL5orUcRKI/qCDDeruyr8aaSGpu4Nxswsv/8Mz3v F6+uvcdbE0pXjwMPDYgmiOwzhqWb3kw= Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=L5yhSIt8; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Rspam-User: X-Stat-Signature: 7mmk4nziwxd597rbz94uupan1nysymym X-Rspamd-Queue-Id: ED44F1C00D5 X-HE-Tag: 1659108226-277174 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper get_obj_cgroup_from_cgroup() to get obj_cgroup from a specific cgroup. Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 1 + mm/memcontrol.c | 41 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2f0a611f12e5..901a9219fe9a 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1713,6 +1713,7 @@ bool mem_cgroup_kmem_disabled(void); int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order); void __memcg_kmem_uncharge_page(struct page *page, int order); +struct obj_cgroup *get_obj_cgroup_from_cgroup(struct cgroup *cgrp); struct obj_cgroup *get_obj_cgroup_from_current(void); struct obj_cgroup *get_obj_cgroup_from_page(struct page *page); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 618c366a2f07..762cffae0320 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2908,6 +2908,47 @@ static struct obj_cgroup *__get_obj_cgroup_from_memcg(struct mem_cgroup *memcg) return objcg; } +static struct obj_cgroup *get_obj_cgroup_from_memcg(struct mem_cgroup *memcg) +{ + struct obj_cgroup *objcg; + + if (memcg_kmem_bypass()) + return NULL; + + rcu_read_lock(); + objcg = __get_obj_cgroup_from_memcg(memcg); + rcu_read_unlock(); + return objcg; +} + +struct obj_cgroup *get_obj_cgroup_from_cgroup(struct cgroup *cgrp) +{ + struct cgroup_subsys_state *css; + struct mem_cgroup *memcg; + struct obj_cgroup *objcg; + + rcu_read_lock(); + css = rcu_dereference(cgrp->subsys[memory_cgrp_id]); + if (!css || !css_tryget_online(css)) { + rcu_read_unlock(); + cgroup_put(cgrp); + return ERR_PTR(-EINVAL); + } + rcu_read_unlock(); + cgroup_put(cgrp); + + memcg = mem_cgroup_from_css(css); + if (!memcg) { + css_put(css); + return ERR_PTR(-EINVAL); + } + + objcg = get_obj_cgroup_from_memcg(memcg); + css_put(css); + + return objcg; +} + __always_inline struct obj_cgroup *get_obj_cgroup_from_current(void) { struct obj_cgroup *objcg = NULL; From patchwork Fri Jul 29 15:23:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A666DC19F2B for ; Fri, 29 Jul 2022 15:23:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 440AE6B0085; Fri, 29 Jul 2022 11:23:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F07F8E0001; Fri, 29 Jul 2022 11:23:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 242E76B0088; Fri, 29 Jul 2022 11:23:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 162DF6B0085 for ; Fri, 29 Jul 2022 11:23:50 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E4F5C1A1217 for ; Fri, 29 Jul 2022 15:23:49 +0000 (UTC) X-FDA: 79740507378.07.6DB682D Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf19.hostedemail.com (Postfix) with ESMTP id 8C68C1A000E for ; Fri, 29 Jul 2022 15:23:49 +0000 (UTC) Received: by mail-pj1-f45.google.com with SMTP id o14-20020a17090a4b4e00b001f2f2b61be5so5564720pjl.4 for ; Fri, 29 Jul 2022 08:23:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y6e0+BfhzTEhaMIObPp/tIKhd/5AtWlTaOTlEQZf2GY=; b=SJW46zqW4liJjC9RNghwcRe0UqCgDmtGwf0+wgFVhsSYnDwkrwQvucNeZDu6lBJst/ SBdVYgeE1jdq62I4rQUd9bo1swjbyOvDVX/52WFozqx5IJPiN+iwGaUfIMxVbkIEsC8d fNgaNPPA5dVUY8bVep4mBvFq2UuSXnstMV+DNS1befoCuzaKYx3ikh9dqTaNRfePWsVR 28AfNQyzFgbONkq/KwkafQ0TDjTyh6Uzao1VSo8jAk+AWQhyg/RyPkYjwYm+RS7yyqVp ukZk3zQ+oGwr6mebFVTAn0Y4hPEomr7WsfwllOyUPFRHs0NCeiSgwTbGLoUL9DwYhREI ti7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y6e0+BfhzTEhaMIObPp/tIKhd/5AtWlTaOTlEQZf2GY=; b=uK8FLbl8EgVvI1DVUynD1+Hzmla1yd1b7phaDYqd7bMsoWxaJ/BnpNTLpmCeqc/nib /SS/Hwd0yVbxX8OFj3CHwTjSNtS1U6FebNVvFTbxQ+/opJnT7OW/8Y//EUGUK2Gu6EUr rDgWSg1f+CXggyth17gz5L8YB2F8G3x05ZP2MOrcNWu5QfiZkH3pdwja2p141e6583Pv ZqufCGRd1K+sR4m7PiM/i5FoPVMeVtF5kVmQwOrL1RI40LcsGdABoeIZ6ieX6RDo7j2P 1JT5zU9Shs5sAZOoTNKLQ5++rmGvhO8u69/0IpMiCKstHIML2O1zNGFB+pIxOpT2CaIx tx0A== X-Gm-Message-State: ACgBeo23KlAZfDsS3fqlM+COv092/1kFEyVPb7JUSxMXOcj0XBlPp9Ge 1GkxHx9Nlp4OsRBUEJoYy+4= X-Google-Smtp-Source: AA6agR6T+aMpQScA3MSKZ91rVos74j/v0J7O4lrbrTzSPJ+SJjFr9+lQ5CwAO6uGkd9QeUEsZZ73KA== X-Received: by 2002:a17:902:c40a:b0:16d:2dcb:9b8c with SMTP id k10-20020a170902c40a00b0016d2dcb9b8cmr4267480plk.61.1659108228525; Fri, 29 Jul 2022 08:23:48 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:47 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 13/15] bpf: Add new parameter into bpf_map_container_alloc Date: Fri, 29 Jul 2022 15:23:14 +0000 Message-Id: <20220729152316.58205-14-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SJW46zqW; spf=pass (imf19.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.45 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108229; a=rsa-sha256; cv=none; b=g8Q3sMJa7AsjM/UJ7OapzHPG/DCqTosAaGTPIbUWRvLbsr6D6Bd5X8h+8vaTc3xUsrWtmq UaZuAppzkDZdQx9wi/8o/IGIEdIe/U9VvjSXE1YcRD/pfhqUmstnxNpXkhoQctxjdr7ZL7 fnR27Qb+wsHQtAImp4RRVHudGgGltyY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108229; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y6e0+BfhzTEhaMIObPp/tIKhd/5AtWlTaOTlEQZf2GY=; b=FetDmyNg5/8SWxHkgKdDRlL9pzOYICmZJ+yarC1HhvjQJKZQg2FPqi0pMxqhiqXr4ZGV/D Jfv+vlSlKGvABUE9pAPsl6GVf41wb6iDcYOAe7im3bhyq18pf95+iBMv6OEKbVZ1Pv7Pbj cf7BkibxBIobL4sIfxFGO14wFZypXnY= Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SJW46zqW; spf=pass (imf19.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.45 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Rspam-User: X-Stat-Signature: 5synnzqhr7kcew6skwjgczo8z6hjd35b X-Rspamd-Queue-Id: 8C68C1A000E X-HE-Tag: 1659108229-574996 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add bpf attr into bpf_map_container_alloc(), then we can get map creation related attributes in these function. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 6 +++--- kernel/bpf/arraymap.c | 15 ++++++++------- kernel/bpf/bloom_filter.c | 7 ++++--- kernel/bpf/bpf_local_storage.c | 6 +++--- kernel/bpf/bpf_struct_ops.c | 6 +++--- kernel/bpf/cpumap.c | 6 +++--- kernel/bpf/devmap.c | 6 +++--- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 6 +++--- kernel/bpf/lpm_trie.c | 6 +++--- kernel/bpf/offload.c | 6 +++--- kernel/bpf/queue_stack_maps.c | 6 +++--- kernel/bpf/reuseport_array.c | 7 ++++--- kernel/bpf/ringbuf.c | 6 +++--- kernel/bpf/stackmap.c | 6 +++--- kernel/bpf/syscall.c | 10 +++++----- net/core/sock_map.c | 12 ++++++------ net/xdp/xskmap.c | 6 +++--- 18 files changed, 66 insertions(+), 63 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 605214fedd6d..2bd941279e6c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1634,9 +1634,9 @@ void bpf_map_inc_with_uref(struct bpf_map *map); struct bpf_map * __must_check bpf_map_inc_not_zero(struct bpf_map *map); void bpf_map_put_with_uref(struct bpf_map *map); void bpf_map_put(struct bpf_map *map); -void *bpf_map_container_alloc(u64 size, int numa_node); -void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, - u32 align, u32 offset); +void *bpf_map_container_alloc(union bpf_attr *attr, u64 size, int numa_node); +void *bpf_map_container_mmapable_alloc(union bpf_attr *attr, u64 size, + int numa_node, u32 align, u32 offset); void *bpf_map_area_alloc(struct bpf_map *map, u64 size, int numa_node); void *bpf_map_pages_alloc(struct bpf_map *map, struct page **pages, int nr_meta_pages, int nr_data_pages, int nid, diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index bfcfc5df9983..883905c6c845 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -131,16 +131,17 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) void *data; /* kmalloc'ed memory can't be mmap'ed, use explicit vmalloc */ - data = bpf_map_container_mmapable_alloc(array_size, numa_node, - align, offset); - if (!data) - return ERR_PTR(-ENOMEM); + data = bpf_map_container_mmapable_alloc(attr, array_size, + numa_node, align, + offset); + if (IS_ERR(data)) + return data; array = data + align - offset; } else { - array = bpf_map_container_alloc(array_size, numa_node); + array = bpf_map_container_alloc(attr, array_size, numa_node); } - if (!array) - return ERR_PTR(-ENOMEM); + if (IS_ERR(array)) + return ERR_CAST(array); array->index_mask = index_mask; array->map.bypass_spec_v1 = bypass_spec_v1; diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index 8b3194903b52..9fe3c6774c40 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -142,10 +142,11 @@ static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) } bitset_bytes = roundup(bitset_bytes, sizeof(unsigned long)); - bloom = bpf_map_container_alloc(sizeof(*bloom) + bitset_bytes, numa_node); + bloom = bpf_map_container_alloc(attr, sizeof(*bloom) + bitset_bytes, + numa_node); - if (!bloom) - return ERR_PTR(-ENOMEM); + if (IS_ERR(bloom)) + return ERR_CAST(bloom); bpf_map_init_from_attr(&bloom->map, attr); diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 3440135c3612..e12891dcf2a9 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -610,9 +610,9 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) unsigned int i; u32 nbuckets; - smap = bpf_map_container_alloc(sizeof(*smap), NUMA_NO_NODE); - if (!smap) - return ERR_PTR(-ENOMEM); + smap = bpf_map_container_alloc(attr, sizeof(*smap), NUMA_NO_NODE); + if (IS_ERR(smap)) + return ERR_CAST(smap); bpf_map_init_from_attr(&smap->map, attr); nbuckets = roundup_pow_of_two(num_possible_cpus()); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 874fda7e2b8b..51b7ce9902a8 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -618,9 +618,9 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) */ (vt->size - sizeof(struct bpf_struct_ops_value)); - st_map = bpf_map_container_alloc(st_map_size, NUMA_NO_NODE); - if (!st_map) - return ERR_PTR(-ENOMEM); + st_map = bpf_map_container_alloc(attr, st_map_size, NUMA_NO_NODE); + if (IS_ERR(st_map)) + return ERR_CAST(st_map); st_map->st_ops = st_ops; map = &st_map->map; diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 95c1642deaf6..1e915a3bd1a2 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,9 +97,9 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = bpf_map_container_alloc(sizeof(*cmap), NUMA_NO_NODE); - if (!cmap) - return ERR_PTR(-ENOMEM); + cmap = bpf_map_container_alloc(attr, sizeof(*cmap), NUMA_NO_NODE); + if (IS_ERR(cmap)) + return ERR_CAST(cmap); bpf_map_init_from_attr(&cmap->map, attr); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index b625d578bc93..11c7b8411b03 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -167,9 +167,9 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = bpf_map_container_alloc(sizeof(*dtab), NUMA_NO_NODE); - if (!dtab) - return ERR_PTR(-ENOMEM); + dtab = bpf_map_container_alloc(attr, sizeof(*dtab), NUMA_NO_NODE); + if (IS_ERR(dtab)) + return ERR_CAST(dtab); err = dev_map_init_map(dtab, attr); if (err) { diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 2a34a115e14f..3cb9486eb313 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -496,9 +496,9 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = bpf_map_container_alloc(sizeof(*htab), NUMA_NO_NODE); - if (!htab) - return ERR_PTR(-ENOMEM); + htab = bpf_map_container_alloc(attr, sizeof(*htab), NUMA_NO_NODE); + if (IS_ERR(htab)) + return ERR_CAST(htab); lockdep_register_key(&htab->lockdep_key); diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 963dc82e3c50..b2bd031aba79 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,9 +313,9 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = bpf_map_container_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node); - if (!map) - return ERR_PTR(-ENOMEM); + map = bpf_map_container_alloc(attr, sizeof(*map), numa_node); + if (IS_ERR(map)) + return ERR_CAST(map); /* copy mandatory map attributes */ bpf_map_init_from_attr(&map->map, attr); diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 15ccef2f10a3..e55327ad9c3a 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -558,9 +558,9 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) attr->value_size > LPM_VAL_SIZE_MAX) return ERR_PTR(-EINVAL); - trie = bpf_map_container_alloc(sizeof(*trie), NUMA_NO_NODE); - if (!trie) - return ERR_PTR(-ENOMEM); + trie = bpf_map_container_alloc(attr, sizeof(*trie), NUMA_NO_NODE); + if (IS_ERR(trie)) + return ERR_CAST(trie); /* copy mandatory map attributes */ bpf_map_init_from_attr(&trie->map, attr); diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 02e3f4ef19f5..34fb5261ff95 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -372,9 +372,9 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) attr->map_type != BPF_MAP_TYPE_HASH) return ERR_PTR(-EINVAL); - offmap = bpf_map_container_alloc(sizeof(*offmap), NUMA_NO_NODE); - if (!offmap) - return ERR_PTR(-ENOMEM); + offmap = bpf_map_container_alloc(attr, sizeof(*offmap), NUMA_NO_NODE); + if (IS_ERR(offmap)) + return ERR_CAST(offmap); bpf_map_init_from_attr(&offmap->map, attr); diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index d4c0284c9089..9425df0695ac 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -74,9 +74,9 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) size = (u64) attr->max_entries + 1; queue_size = sizeof(*qs) + size * attr->value_size; - qs = bpf_map_container_alloc(queue_size, numa_node); - if (!qs) - return ERR_PTR(-ENOMEM); + qs = bpf_map_container_alloc(attr, queue_size, numa_node); + if (IS_ERR(qs)) + return ERR_CAST(qs); bpf_map_init_from_attr(&qs->map, attr); diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index d36a60ca68f4..0d3091866e1d 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -153,14 +153,15 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) { int numa_node = bpf_map_attr_numa_node(attr); struct reuseport_array *array; + size_t size = struct_size(array, ptrs, attr->max_entries); if (!bpf_capable()) return ERR_PTR(-EPERM); /* allocate all map elements and zero-initialize them */ - array = bpf_map_container_alloc(struct_size(array, ptrs, attr->max_entries), numa_node); - if (!array) - return ERR_PTR(-ENOMEM); + array = bpf_map_container_alloc(attr, size, numa_node); + if (IS_ERR(array)) + return ERR_CAST(array); /* copy mandatory map attributes */ bpf_map_init_from_attr(&array->map, attr); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 25973cab251d..3be472fd55da 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -159,9 +159,9 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = bpf_map_container_alloc(sizeof(*rb_map), NUMA_NO_NODE); - if (!rb_map) - return ERR_PTR(-ENOMEM); + rb_map = bpf_map_container_alloc(attr, sizeof(*rb_map), NUMA_NO_NODE); + if (IS_ERR(rb_map)) + return ERR_CAST(rb_map); bpf_map_init_from_attr(&rb_map->map, attr); diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index c9a91ca05a03..c952c7547279 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -101,9 +101,9 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); cost = n_buckets * sizeof(struct stack_map_bucket *) + sizeof(*smap); - smap = bpf_map_container_alloc(cost, bpf_map_attr_numa_node(attr)); - if (!smap) - return ERR_PTR(-ENOMEM); + smap = bpf_map_container_alloc(attr, cost, bpf_map_attr_numa_node(attr)); + if (IS_ERR(smap)) + return ERR_CAST(smap); bpf_map_init_from_attr(&smap->map, attr); smap->n_buckets = n_buckets; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index f7210fa8c3be..6401cc417fa9 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -526,14 +526,14 @@ void bpf_map_area_free(void *area) * * It is used in map creation path. */ -void *bpf_map_container_alloc(u64 size, int numa_node) +void *bpf_map_container_alloc(union bpf_attr *attr, u64 size, int numa_node) { struct bpf_map *map; void *container; container = __bpf_map_area_alloc(size, numa_node, false); if (!container) - return NULL; + return ERR_PTR(-ENOMEM); map = (struct bpf_map *)container; bpf_map_save_memcg(map); @@ -541,8 +541,8 @@ void *bpf_map_container_alloc(u64 size, int numa_node) return container; } -void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, u32 align, - u32 offset) +void *bpf_map_container_mmapable_alloc(union bpf_attr *attr, u64 size, + int numa_node, u32 align, u32 offset) { struct bpf_map *map; void *container; @@ -551,7 +551,7 @@ void *bpf_map_container_mmapable_alloc(u64 size, int numa_node, u32 align, /* kmalloc'ed memory can't be mmap'ed, use explicit vmalloc */ ptr = __bpf_map_area_alloc(size, numa_node, true); if (!ptr) - return NULL; + return ERR_PTR(-ENOMEM); container = ptr + align - offset; map = (struct bpf_map *)container; diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 1f49dc89822c..4d9b730aa27f 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,9 +41,9 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = bpf_map_container_alloc(sizeof(*stab), NUMA_NO_NODE); - if (!stab) - return ERR_PTR(-ENOMEM); + stab = bpf_map_container_alloc(attr, sizeof(*stab), NUMA_NO_NODE); + if (IS_ERR(stab)) + return ERR_CAST(stab); bpf_map_init_from_attr(&stab->map, attr); raw_spin_lock_init(&stab->lock); @@ -1077,9 +1077,9 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = bpf_map_container_alloc(sizeof(*htab), NUMA_NO_NODE); - if (!htab) - return ERR_PTR(-ENOMEM); + htab = bpf_map_container_alloc(attr, sizeof(*htab), NUMA_NO_NODE); + if (IS_ERR(htab)) + return ERR_CAST(htab); bpf_map_init_from_attr(&htab->map, attr); diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 6cf98f1c621a..161fc569ed88 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -75,9 +75,9 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) numa_node = bpf_map_attr_numa_node(attr); size = struct_size(m, xsk_map, attr->max_entries); - m = bpf_map_container_alloc(size, numa_node); - if (!m) - return ERR_PTR(-ENOMEM); + m = bpf_map_container_alloc(attr, size, numa_node); + if (IS_ERR(m)) + return ERR_CAST(m); bpf_map_init_from_attr(&m->map, attr); spin_lock_init(&m->lock); From patchwork Fri Jul 29 15:23:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAE00C00144 for ; Fri, 29 Jul 2022 15:23:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 419926B0087; Fri, 29 Jul 2022 11:23:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CA706B0088; Fri, 29 Jul 2022 11:23:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 21B406B0089; Fri, 29 Jul 2022 11:23:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 141B96B0087 for ; Fri, 29 Jul 2022 11:23:52 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E6194C1133 for ; Fri, 29 Jul 2022 15:23:51 +0000 (UTC) X-FDA: 79740507462.15.1E09E6B Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf20.hostedemail.com (Postfix) with ESMTP id 8E4FC1C000B for ; Fri, 29 Jul 2022 15:23:51 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id v16-20020a17090abb9000b001f25244c65dso8716330pjr.2 for ; Fri, 29 Jul 2022 08:23:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vYnEy8nKJWMROwA3MmvoCcmRLFYdVgR55TwlItJ2+GU=; b=Mkrl04Y5bR3wO1T3rRf4pjfQ6zW0Au+4rSRFHRqhmyLikvX+6Mhx2b47cSG2rgEaCW XPCcWAbWgbq/cxdH/xP1L7oWnHwfbF3+SmoqM4Y0/peTnhwEjmCVKDBmISQceZn7IKy9 4lfLn41fhG7TI/xqGA3YRfVmEJm9MsyqXXoZx5tdaye0L+S/j5TIrnsFXlcDu95kwb41 TGYhUGnPF1jX3PWn0L6lRgwISXK/D3AJat0IY1I5m3UfRy8hYosZNVkHJnVdWd63sEqv /nmRZxOjXyPAdnEfbjEmIkT4HHGTyNMGVyLGhY2EoIaQ7L/HS5meKFFLAGP+s+knjQHL iKAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vYnEy8nKJWMROwA3MmvoCcmRLFYdVgR55TwlItJ2+GU=; b=yvXQTRYbVHeq3jVaB3yPHGK6zBpCQzDkALPme7swFKZgs53rDWl4kI9qgsl2KzBKtp XbryT72CJ+GIp3IIwQyffyYst2QjwxRXahH6BYvzCUwisCUFXN4LWDaPHxkGRbIYJZK7 h9vhPgcQwjVGl0s0qS7ys+D1Qld9XG7Mz7cnMqXTsEbEcbTDbRPtxeEPA7Am8DURx4xl 6BFX4mrk4ACK45xVj4EPHJtYUWCLzpa85pd29IzPiDdWNfYJKys7SWF2IkSfnhWfZlw1 X/OHFLkLk8+dGAuxxHvZlOM+N01pFrfWNTGCgulTl4JKxEC/k+1HICGtjt79+CLyzRpV 097w== X-Gm-Message-State: ACgBeo2d5U3BwfSUe0f9bX+cuIPrK4Ulwh3rmd5JDkFBp0iFEk14fqba jCY2mC2uI7BCkMj5PBH5Dbg= X-Google-Smtp-Source: AA6agR5/5myi/pHhqOGN+umenk6xWAgqpqvDsItrls1rKZv/ueQaUf6O4B1azHULpfbNK/hwB1PyTg== X-Received: by 2002:a17:902:d48a:b0:16d:30f4:ca5a with SMTP id c10-20020a170902d48a00b0016d30f4ca5amr4369743plg.50.1659108230655; Fri, 29 Jul 2022 08:23:50 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:49 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 14/15] bpf: Add new map flag BPF_F_SELECTABLE_MEMCG Date: Fri, 29 Jul 2022 15:23:15 +0000 Message-Id: <20220729152316.58205-15-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108231; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vYnEy8nKJWMROwA3MmvoCcmRLFYdVgR55TwlItJ2+GU=; b=hUcJ3tESvwC8VsdxSjve/3duFcqKuquRIZ6OK77DqZ73FvtZVhqEwPY9Gqnkh/EoAYsV9x Z2zQ7SIy7bAxH8rU3G++KOh6Bg4BeFxGHc8Q20Q0ke7o1va9eAAprgtcBHfQiaYFTSFAQz FAfkQFz35nbxj/t+RGScyJiQQC7Qfdo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Mkrl04Y5; spf=pass (imf20.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108231; a=rsa-sha256; cv=none; b=SvnHGzI/BFn387rFYD0KUXaf53J925LKBwQCl23EapGYiSM/OjI/boCku2SYgeg4Dp3I78 fyQkwpml6Qcl3qNkwmf/0HydAJNbD+5kpvIBUgN3tNYmAMWOCs/DUskmGPC/FfGjSXBjth 5GiojLAlT7HDd4mYkqJSJDrsGCIrf24= X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8E4FC1C000B X-Rspam-User: Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Mkrl04Y5; spf=pass (imf20.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: rt69tsrxd9hq4wchkfuw36118c4j3s5g X-HE-Tag: 1659108231-748199 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new map flag BPF_F_SELECTABLE_MEMCG for map creation. This flag is supported for all bpf maps. This is a preparation for followup patch. Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 3 +++ kernel/bpf/arraymap.c | 2 +- kernel/bpf/bloom_filter.c | 4 ++-- kernel/bpf/bpf_local_storage.c | 3 ++- kernel/bpf/bpf_struct_ops.c | 5 ++++- kernel/bpf/devmap.c | 4 ++-- kernel/bpf/hashtab.c | 2 +- kernel/bpf/local_storage.c | 2 +- kernel/bpf/queue_stack_maps.c | 2 +- kernel/bpf/ringbuf.c | 2 +- kernel/bpf/stackmap.c | 2 +- net/core/sock_map.c | 4 ++-- tools/include/uapi/linux/bpf.h | 3 +++ 13 files changed, 24 insertions(+), 14 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 59a217ca2dfd..d5fc1ea70b59 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1227,6 +1227,9 @@ enum { /* Create a map that is suitable to be an inner map with dynamic max entries */ BPF_F_INNER_MAP = (1U << 12), + +/* Selectable memcg */ + BPF_F_SELECTABLE_MEMCG = (1U << 13), }; /* Flags for BPF_PROG_QUERY. */ diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 883905c6c845..eb8deac529ac 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -17,7 +17,7 @@ #define ARRAY_CREATE_FLAG_MASK \ (BPF_F_NUMA_NODE | BPF_F_MMAPABLE | BPF_F_ACCESS_MASK | \ - BPF_F_PRESERVE_ELEMS | BPF_F_INNER_MAP) + BPF_F_PRESERVE_ELEMS | BPF_F_INNER_MAP | BPF_F_SELECTABLE_MEMCG) static void bpf_array_free_percpu(struct bpf_array *array) { diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index 9fe3c6774c40..3714aebc9ed6 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -9,8 +9,8 @@ #include #include -#define BLOOM_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_ZERO_SEED | BPF_F_ACCESS_MASK) +#define BLOOM_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | \ + BPF_F_ZERO_SEED | BPF_F_ACCESS_MASK | BPF_F_SELECTABLE_MEMCG) struct bpf_bloom_filter { struct bpf_map map; diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index e12891dcf2a9..7798235f311e 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -15,7 +15,8 @@ #include #include -#define BPF_LOCAL_STORAGE_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_CLONE) +#define BPF_LOCAL_STORAGE_CREATE_FLAG_MASK \ + (BPF_F_NO_PREALLOC | BPF_F_CLONE | BPF_F_SELECTABLE_MEMCG) static struct bpf_local_storage_map_bucket * select_bucket(struct bpf_local_storage_map *smap, diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 51b7ce9902a8..208d593e6a44 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -12,6 +12,8 @@ #include #include +#define STRUCT_OPS_CREATE_FLAG_MASK (BPF_F_SELECTABLE_MEMCG) + enum bpf_struct_ops_state { BPF_STRUCT_OPS_STATE_INIT, BPF_STRUCT_OPS_STATE_INUSE, @@ -586,7 +588,8 @@ static void bpf_struct_ops_map_free(struct bpf_map *map) static int bpf_struct_ops_map_alloc_check(union bpf_attr *attr) { if (attr->key_size != sizeof(unsigned int) || attr->max_entries != 1 || - attr->map_flags || !attr->btf_vmlinux_value_type_id) + (attr->map_flags & ~STRUCT_OPS_CREATE_FLAG_MASK) || + !attr->btf_vmlinux_value_type_id) return -EINVAL; return 0; } diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 11c7b8411b03..52858963373c 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -50,8 +50,8 @@ #include #include -#define DEV_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) +#define DEV_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | \ + BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_SELECTABLE_MEMCG) struct xdp_dev_bulk_queue { struct xdp_frame *q[DEV_MAP_BULK_SIZE]; diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 3cb9486eb313..532c8ee89d58 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -17,7 +17,7 @@ #define HTAB_CREATE_FLAG_MASK \ (BPF_F_NO_PREALLOC | BPF_F_NO_COMMON_LRU | BPF_F_NUMA_NODE | \ - BPF_F_ACCESS_MASK | BPF_F_ZERO_SEED) + BPF_F_ACCESS_MASK | BPF_F_ZERO_SEED | BPF_F_SELECTABLE_MEMCG) #define BATCH_OPS(_name) \ .map_lookup_batch = \ diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index b2bd031aba79..009d6f43a099 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -16,7 +16,7 @@ #include "../cgroup/cgroup-internal.h" #define LOCAL_STORAGE_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK | BPF_F_SELECTABLE_MEMCG) struct bpf_cgroup_storage_map { struct bpf_map map; diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index 9425df0695ac..c4aab43198ad 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -12,7 +12,7 @@ #include "percpu_freelist.h" #define QUEUE_STACK_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK | BPF_F_SELECTABLE_MEMCG) struct bpf_queue_stack { struct bpf_map map; diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 3be472fd55da..53a7eb8db257 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -12,7 +12,7 @@ #include #include -#define RINGBUF_CREATE_FLAG_MASK (BPF_F_NUMA_NODE) +#define RINGBUF_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | BPF_F_SELECTABLE_MEMCG) /* non-mmap()'able part of bpf_ringbuf (everything up to consumer page) */ #define RINGBUF_PGOFF \ diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index c952c7547279..007b10d2da7d 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -14,7 +14,7 @@ #define STACK_CREATE_FLAG_MASK \ (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | \ - BPF_F_STACK_BUILD_ID) + BPF_F_STACK_BUILD_ID | BPF_F_SELECTABLE_MEMCG) struct stack_map_bucket { struct pcpu_freelist_node fnode; diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 4d9b730aa27f..0310b00e19b8 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -21,8 +21,8 @@ struct bpf_stab { raw_spinlock_t lock; }; -#define SOCK_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) +#define SOCK_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | \ + BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_SELECTABLE_MEMCG) static int sock_map_prog_update(struct bpf_map *map, struct bpf_prog *prog, struct bpf_prog *old, u32 which); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 59a217ca2dfd..d5fc1ea70b59 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1227,6 +1227,9 @@ enum { /* Create a map that is suitable to be an inner map with dynamic max entries */ BPF_F_INNER_MAP = (1U << 12), + +/* Selectable memcg */ + BPF_F_SELECTABLE_MEMCG = (1U << 13), }; /* Flags for BPF_PROG_QUERY. */ From patchwork Fri Jul 29 15:23:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12932545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA2F5C19F2B for ; Fri, 29 Jul 2022 15:23:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D8696B0088; Fri, 29 Jul 2022 11:23:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6610D6B0089; Fri, 29 Jul 2022 11:23:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B3708E0001; Fri, 29 Jul 2022 11:23:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3E6866B0088 for ; Fri, 29 Jul 2022 11:23:54 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 22039C0DA6 for ; Fri, 29 Jul 2022 15:23:54 +0000 (UTC) X-FDA: 79740507588.20.54EB2F1 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf21.hostedemail.com (Postfix) with ESMTP id AA0CB1C00D5 for ; Fri, 29 Jul 2022 15:23:53 +0000 (UTC) Received: by mail-pl1-f178.google.com with SMTP id o3so4902850ple.5 for ; Fri, 29 Jul 2022 08:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7YsE4RcubCx0MNNfTDgzCPEblKVY7zNgxkQBDs4eXwk=; b=O5fg25+RPCgr8YZ1DZBPRlYRypSoSLhhZ+s3atPcrFK25aDQy6CL5rwsw3m5+llAGw wxrSeU2pkwin6ICTg/I3esczDOPduHDhcISG8YjXkEl7nhA+/yeiVyiBanmd9vhcuN1y E/hSC/t7jj5rA1Meyi7gYlgtiHotJd1DdRmBBK0p2WnPK0R9+EwD+M2bpgHZ6x0q5OWX gAoKVgnTZLygdyqobNOJdDpPJffXYepcW9t/NfVuXa2B2frGIHDd/tdQ9M5E3HJK8EFL Tr3Rt7EnK4zC3dObgNYBgZgpxW/coISdKL/i9f6FAYcMOrnO+oDj7WvlZZBTzgVAy+YH QTxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7YsE4RcubCx0MNNfTDgzCPEblKVY7zNgxkQBDs4eXwk=; b=2Gmylp/hVuOhz6zwRyl0UWUzYeN6EzO7xe9JL0tLtvpfObwuNtkScA3nYz6k7ThiY6 eQWb1ifXefCnqGb6u+clO5gOzP81rlbt7PgBztp/DN0iW4WFy+ZX7+NGhFsxOvBPvbko FbcL/v4YHF/t6bgJb6+OlCA/vx870+OqksqnHXVZMRiUKZrZ5ohtuo4DPRHTPxOkCzBY 6VryQlq7ohwExcMs2luCEA/CHci+8T/cCobNW0B58YoQnhZUdcArF9kxMaCzHGGMw7UA Pr5cW9wspZEm8dB6Rog/4OLicEIjUlkf2Q2SE80lkhAe/489Kdw8xI06pqAR9vERU1e/ e6Yw== X-Gm-Message-State: ACgBeo0+QkxcJP2xvr6pje50n81w2YyTouM4Vjz22Aeg3TECPFf/lQ4O oT/Kq1utOEsdCXB9T5V+KIU= X-Google-Smtp-Source: AA6agR7ytKHuTcGgGPxk083gm4DArTadtq9szQO+g7w3QFR6tTBNSGjLIDUSkmlzzv7MzA+JPSsXSg== X-Received: by 2002:a17:902:a502:b0:15e:c251:b769 with SMTP id s2-20020a170902a50200b0015ec251b769mr4526184plq.115.1659108232767; Fri, 29 Jul 2022 08:23:52 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:2912:5400:4ff:fe16:4344]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b0016d3a354cffsm3714219plm.89.2022.07.29.08.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Jul 2022 08:23:51 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [RFC PATCH bpf-next 15/15] bpf: Introduce selectable memcg for bpf map Date: Fri, 29 Jul 2022 15:23:16 +0000 Message-Id: <20220729152316.58205-16-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220729152316.58205-1-laoar.shao@gmail.com> References: <20220729152316.58205-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=O5fg25+R; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659108233; a=rsa-sha256; cv=none; b=bk70w4Ofl4jPvMG/itqnibn4hShdDMVISNdKdE3fDNhPd2p7PDFVZ3WgodelGUJ0p5AXp6 OA8Ip9CeSaecbA/zMgdpzfQA8kd68+TR6F2mO7ry2LlayXU74Q1n64YcJX5giffJkPn0wX wUCIs5Hl1x2aPpRQLuDyBY6bRlC94aE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659108233; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7YsE4RcubCx0MNNfTDgzCPEblKVY7zNgxkQBDs4eXwk=; b=x8U8Xtzc9hq384rP6MB4tXYfznn72+Z1edNdj1tB4uJgGfQlExLI+9jD3eW4uQ0EhtUOSf 4gZb34cvjeW72v8LvSmaXNS9obHuD/6rP3n+NzaGYv5WqaFDa4qif2JwPyaD+r5zHmPwXD gqCjlcN7brFeQyu68H9rL0xKyxYaREI= X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: AA0CB1C00D5 Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=O5fg25+R; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: k7uemtgy4cmmu98ck7xxzoowqk7c39kx X-Rspam-User: X-HE-Tag: 1659108233-156835 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A new member memcg_fd is introduced into bpf attr of BPF_MAP_CREATE command, which is the fd of an opened cgroup directory. In this cgroup, the memory subsystem must be enabled. This value is valid only when BPF_F_SELECTABLE_MEMCG is set in map_flags. Once the kernel get the memory cgroup from this fd, it will set this memcg into bpf map, then all the subsequent memory allocation of this map will be charge to the memcg. The map creation paths in libbpf are also changed consequently. Currently it is only supported for cgroup2 directory. The usage of this new member as follows, struct bpf_map_create_opts map_opts = { .sz = sizeof(map_opts), .map_flags = BPF_F_SELECTABLE_MEMCG, }; int memcg_fd, int map_fd; int key, value; memcg_fd = open("/cgroup2", O_DIRECTORY); if (memcg_fd < 0) { perror("memcg dir open"); return -1; } map_opts.memcg_fd = memcg_fd; map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, "map_for_memcg", sizeof(key), sizeof(value), 1024, &map_opts); if (map_fd <= 0) { perror("map create"); return -1; } Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 2 ++ kernel/bpf/syscall.c | 47 ++++++++++++++++++++++++++-------- tools/include/uapi/linux/bpf.h | 2 ++ tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/bpf.h | 3 ++- tools/lib/bpf/libbpf.c | 2 ++ 6 files changed, 46 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index d5fc1ea70b59..a6e02c8be924 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1296,6 +1296,8 @@ union bpf_attr { * struct stored as the * map value */ + __s32 memcg_fd; /* selectable memcg */ + __s32 :32; /* hole */ /* Any per-map-type extra fields * * BPF_MAP_TYPE_BLOOM_FILTER - the lowest 4 bits indicate the diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 6401cc417fa9..9900e2b87315 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -402,14 +402,30 @@ void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock) } #ifdef CONFIG_MEMCG_KMEM -static void bpf_map_save_memcg(struct bpf_map *map) +static int bpf_map_save_memcg(struct bpf_map *map, union bpf_attr *attr) { - /* Currently if a map is created by a process belonging to the root - * memory cgroup, get_obj_cgroup_from_current() will return NULL. - * So we have to check map->objcg for being NULL each time it's - * being used. - */ - map->objcg = get_obj_cgroup_from_current(); + struct obj_cgroup *objcg; + struct cgroup *cgrp; + + if (attr->map_flags & BPF_F_SELECTABLE_MEMCG) { + cgrp = cgroup_get_from_fd(attr->memcg_fd); + if (IS_ERR(cgrp)) + return -EINVAL; + + objcg = get_obj_cgroup_from_cgroup(cgrp); + if (IS_ERR(objcg)) + return PTR_ERR(objcg); + } else { + /* Currently if a map is created by a process belonging to the root + * memory cgroup, get_obj_cgroup_from_current() will return NULL. + * So we have to check map->objcg for being NULL each time it's + * being used. + */ + objcg = get_obj_cgroup_from_current(); + } + + map->objcg = objcg; + return 0; } static void bpf_map_release_memcg(struct bpf_map *map) @@ -485,8 +501,9 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, } #else -static void bpf_map_save_memcg(struct bpf_map *map) +static int bpf_map_save_memcg(struct bpf_map *map, union bpf_attr *attr) { + return 0; } static void bpf_map_release_memcg(struct bpf_map *map) @@ -530,13 +547,18 @@ void *bpf_map_container_alloc(union bpf_attr *attr, u64 size, int numa_node) { struct bpf_map *map; void *container; + int ret; container = __bpf_map_area_alloc(size, numa_node, false); if (!container) return ERR_PTR(-ENOMEM); map = (struct bpf_map *)container; - bpf_map_save_memcg(map); + ret = bpf_map_save_memcg(map, attr); + if (ret) { + bpf_map_area_free(container); + return ERR_PTR(ret); + } return container; } @@ -547,6 +569,7 @@ void *bpf_map_container_mmapable_alloc(union bpf_attr *attr, u64 size, struct bpf_map *map; void *container; void *ptr; + int ret; /* kmalloc'ed memory can't be mmap'ed, use explicit vmalloc */ ptr = __bpf_map_area_alloc(size, numa_node, true); @@ -555,7 +578,11 @@ void *bpf_map_container_mmapable_alloc(union bpf_attr *attr, u64 size, container = ptr + align - offset; map = (struct bpf_map *)container; - bpf_map_save_memcg(map); + ret = bpf_map_save_memcg(map, attr); + if (ret) { + bpf_map_area_free(ptr); + return ERR_PTR(ret); + } return ptr; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index d5fc1ea70b59..a6e02c8be924 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1296,6 +1296,8 @@ union bpf_attr { * struct stored as the * map value */ + __s32 memcg_fd; /* selectable memcg */ + __s32 :32; /* hole */ /* Any per-map-type extra fields * * BPF_MAP_TYPE_BLOOM_FILTER - the lowest 4 bits indicate the diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 5eb0df90eb2b..662ce5808386 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -199,6 +199,7 @@ int bpf_map_create(enum bpf_map_type map_type, attr.map_extra = OPTS_GET(opts, map_extra, 0); attr.numa_node = OPTS_GET(opts, numa_node, 0); attr.map_ifindex = OPTS_GET(opts, map_ifindex, 0); + attr.memcg_fd = OPTS_GET(opts, memcg_fd, 0); fd = sys_bpf_fd(BPF_MAP_CREATE, &attr, attr_sz); return libbpf_err_errno(fd); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index 88a7cc4bd76f..481aad49422b 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -51,8 +51,9 @@ struct bpf_map_create_opts { __u32 numa_node; __u32 map_ifindex; + __u32 memcg_fd; }; -#define bpf_map_create_opts__last_field map_ifindex +#define bpf_map_create_opts__last_field memcg_fd LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 50d41815f431..86916d550031 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -505,6 +505,7 @@ struct bpf_map { bool pinned; bool reused; bool autocreate; + __s32 memcg_fd; __u64 map_extra; }; @@ -4928,6 +4929,7 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b create_attr.map_ifindex = map->map_ifindex; create_attr.map_flags = def->map_flags; create_attr.numa_node = map->numa_node; + create_attr.memcg_fd = map->memcg_fd; create_attr.map_extra = map->map_extra; if (bpf_map__is_struct_ops(map))