From patchwork Sat Mar 19 17:30:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786248 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA050C43219 for ; Sat, 19 Mar 2022 17:30:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243052AbiCSRcE (ORCPT ); Sat, 19 Mar 2022 13:32:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234986AbiCSRcD (ORCPT ); Sat, 19 Mar 2022 13:32:03 -0400 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA39745506; Sat, 19 Mar 2022 10:30:41 -0700 (PDT) Received: by mail-pg1-x536.google.com with SMTP id o23so7348920pgk.13; Sat, 19 Mar 2022 10:30:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DaQ7tIpYP0FN+ysFq8lKyeNWnVlaqiQkKqC0wh+aN8U=; b=gIIgdf3++Eq+4vAKrm3Bmzg6y37nT58dS7sKmznA/0Z0he8n3pQlzCSRFAZMUFkFUA a27Yi/AC9krchK6+tcCHfx+fOHarQKIpX/4zr3dJjpv/4iG8fDQcXDV2w1ehhqEfpzwz Tv2WW1/NT238AIev0argguYbMWsJVobYGPInR2KvXuJ8TVOe87qHdUckLRmNXaDEjJp7 p46jaVoNgeVwj5kXnrBpCbh3Gmc5lVNo78u8b/t18KVBwcqEQrclITzz5vOht+Ru1GNc tQ2vtQsBFanMpYrxzVmrMViPI4TSIiqZJI5migoLRX6ngj+g6DHtyXKYGTYF1wiYqb9/ TbjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DaQ7tIpYP0FN+ysFq8lKyeNWnVlaqiQkKqC0wh+aN8U=; b=znoPK8vllSaTYi6VaUSeWwXjxm1ZEincK+XnOWnnWV9A+B6xuzZU6T8m+MLkjX5zJf El5oIEEUGJdKPV/rnNw6h6Nhq6/EMbx3KE1AMMWQSRd8g9J1bJhSmXreVKN2Vczakona nRg4njX4T/8soN8eIKm82FV3ARDi869c2ru6Z+mKIOdNDQaikLDnpy1NyMoqI4tRBLcO t4+E8bZMSfvixWxMCSH7yYCY+gmlN7GSs2iW2RGCp+6s/MZYFctT46kmMtznh0mBQrne BxvfD+Vo2HKRpmHhhMNW1F5yxstipFw00SMc+T//W7vEh8d+OLtnmn6bRN8ot0rxndzG sTOA== X-Gm-Message-State: AOAM5322iDhWIughPAiMvn/m/he2Ohrn9AYf5DM4oZKyU4zKbXhxnBYh wmxCD1AkkKjQVm3e/J5GmEo= X-Google-Smtp-Source: ABdhPJzJfpVsPFMvRvZxlinLbP2ikPf+r/4KhMLR5qGL9f8tBztSFPKGq/Oaa6WjOipKpgwMZJ0XWQ== X-Received: by 2002:a63:ef4f:0:b0:375:6898:f6b8 with SMTP id c15-20020a63ef4f000000b003756898f6b8mr12183943pgk.59.1647711041349; Sat, 19 Mar 2022 10:30:41 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:41 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 01/14] bpf: Introduce no charge flag for bpf map Date: Sat, 19 Mar 2022 17:30:23 +0000 Message-Id: <20220319173036.23352-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net A new map flag BPF_F_NO_CHARGE is introduced in bpf_attr, with which we can choose not to charge map memory while account it to root memcg only. At the map creation time, we can get the no charge flag from struct bpf_attr directly, while for other paths we can get it from struct bpf_map. The usecase of this flag is that sometimes we may create bpf maps with a process running in a container (with memcg) but these maps are targeted to the whole system, so we don't want to charge these memory into this container. That will be good for memory resource management for this container, as these shared bpf maps are always pinned which should belong to the system rather than this container. That can also help to make the charging behavior consistent, for example, if we charge the pinned memory into this container, after the contianer restarts these memory will not belong to it any more. Two helpers are introduced for followup usage. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 15 ++++++++++++++- include/uapi/linux/bpf.h | 3 +++ kernel/bpf/syscall.c | 1 + tools/include/uapi/linux/bpf.h | 3 +++ 4 files changed, 21 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 88449fbbe063..07c6603a6c81 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -184,7 +184,8 @@ struct bpf_map { char name[BPF_OBJ_NAME_LEN]; bool bypass_spec_v1; bool frozen; /* write-once; write-protected by freeze_mutex */ - /* 14 bytes hole */ + bool no_charge; /* Don't charge to memcg */ + /* 13 bytes hole */ /* The 3rd and 4th cacheline with misc members to avoid false sharing * particularly with refcounting. @@ -207,6 +208,18 @@ struct bpf_map { } owner; }; +static inline gfp_t +map_flags_no_charge(gfp_t flags, union bpf_attr *attr) +{ + return flags |= (attr->map_flags & BPF_F_NO_CHARGE) ? 0 : __GFP_ACCOUNT; +} + +static inline gfp_t +bpf_flags_no_charge(gfp_t flags, bool no_charge) +{ + return flags |= no_charge ? 0 : __GFP_ACCOUNT; +} + static inline bool map_value_has_spin_lock(const struct bpf_map *map) { return map->spin_lock_off >= 0; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 7604e7d5438f..e2dba6cdd88d 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1225,6 +1225,9 @@ enum { /* Create a map that is suitable to be an inner map with dynamic max entries */ BPF_F_INNER_MAP = (1U << 12), + +/* Don't charge memory to memcg */ + BPF_F_NO_CHARGE = (1U << 13), }; /* Flags for BPF_PROG_QUERY. */ diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index cdaa1152436a..029f04588b1a 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -368,6 +368,7 @@ void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) map->map_flags = bpf_map_flags_retain_permanent(attr->map_flags); map->numa_node = bpf_map_attr_numa_node(attr); map->map_extra = attr->map_extra; + map->no_charge = attr->map_flags & BPF_F_NO_CHARGE; } static int bpf_map_alloc_id(struct bpf_map *map) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 7604e7d5438f..e2dba6cdd88d 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1225,6 +1225,9 @@ enum { /* Create a map that is suitable to be an inner map with dynamic max entries */ BPF_F_INNER_MAP = (1U << 12), + +/* Don't charge memory to memcg */ + BPF_F_NO_CHARGE = (1U << 13), }; /* Flags for BPF_PROG_QUERY. */ From patchwork Sat Mar 19 17:30:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786249 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 773B7C433F5 for ; Sat, 19 Mar 2022 17:30:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243110AbiCSRcF (ORCPT ); Sat, 19 Mar 2022 13:32:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243028AbiCSRcE (ORCPT ); Sat, 19 Mar 2022 13:32:04 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 06EF64550E; Sat, 19 Mar 2022 10:30:43 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id h5so9468051plf.7; Sat, 19 Mar 2022 10:30:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SbeZr/I+CalicDsBM6Lp+v2L/E+i5mvTRbU/FduAJzc=; b=QlkPs7KKYZp/Vt3mckJyR+0PM1YuL5+7nltt6dNtOiHh8ERRVBjGf9kNzg+dsCk1iQ 0yLLxrqsVlHtzV5Vw9ohcLqy7ZGSTA9UDNmE44KNDq3qg0R836FTm+vIEvjUXlOdpHqn lfuVupHZKfZynl/zXu/R6GrM8tSf4xhBKJJzR6G9p7Wr7CKYFWMyvr1x1rXPGWkVnYRi +O29Rngeovdqpp1gj0wIG0p9eDROkQQtL6oAlqK2HVVoGAsE4KSKddEcmAsJWfmzgMgh 8OBxFs244+jpjfsaiYjXoP9WHiCR9DpnP7JVWQgPCGKq4GieqiFrA38N6mWdjbXmFfMc 1JEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SbeZr/I+CalicDsBM6Lp+v2L/E+i5mvTRbU/FduAJzc=; b=wxh6ZGnbnFUY7MLq741RkTcGKHQHeBqD9U8e+/TRieox+0p0UlirxZZW6VSuzzv1VO iEZ+oyZM/RVqXEv1Qn3aMW4kJKOS5g19EMl1e8VbskuElWeKT1pJco9DvX7STkfjMOCC PWgKjAFnDfv0FzLuHhOK/MRhWb0Z7NpYJ0FmUgphEPODUv3oFLw2ABj4YCTHbjwt44CC +l0QPGbjKDSnWbrsm+Fa+1HCn40ZS3bdcEwHfEnbUuKxDlrwBFiA9QxNR7uzEtmwIhJ0 UvqhH2VL+BgPFMl1y0G5yRGK3SRpRYJgn4Ct19v+Zo+shcbVIb/ifGuiLw7oRmJdNnHO 83BA== X-Gm-Message-State: AOAM530Rz8uk6g9aAjbLXB+ZIJV2Svs5Mz8cDRQEqyAW8AI679ClcRHG HDIZ4sHqnZiLmbEKE6HB2Co= X-Google-Smtp-Source: ABdhPJyhjleiQ+4zT/uhoE3mMf1ZeubMi5rFuwyYwjBW0IfV8A8vwfM99q/1OpYQE8OfdMPNvRBg9A== X-Received: by 2002:a17:902:d4c8:b0:154:2416:218b with SMTP id o8-20020a170902d4c800b001542416218bmr5180946plg.139.1647711042563; Sat, 19 Mar 2022 10:30:42 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:42 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 02/14] bpf: Only sys admin can set no charge flag Date: Sat, 19 Mar 2022 17:30:24 +0000 Message-Id: <20220319173036.23352-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Only the sys admin has the privilege to account the bpf map memory into root memcg only. Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 029f04588b1a..0cca3d7d0d84 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -852,6 +852,9 @@ static int map_create(union bpf_attr *attr) attr->map_extra != 0) return -EINVAL; + if (attr->map_flags & BPF_F_NO_CHARGE && !capable(CAP_SYS_ADMIN)) + return -EPERM; + f_flags = bpf_get_file_flag(attr->map_flags); if (f_flags < 0) return f_flags; From patchwork Sat Mar 19 17:30:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786250 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 831A1C433FE for ; Sat, 19 Mar 2022 17:30:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243393AbiCSRcN (ORCPT ); Sat, 19 Mar 2022 13:32:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243300AbiCSRcF (ORCPT ); Sat, 19 Mar 2022 13:32:05 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 564CB45505; Sat, 19 Mar 2022 10:30:44 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id s11so12095356pfu.13; Sat, 19 Mar 2022 10:30:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pfPDh91liqZm+BP4mghw0y9pBJj8lD1CecCVSBxX9RQ=; b=LiVGSCcXA6fHEXuuBuyhZr2cUPqAXeiwwy8Is9dF8j216O3XiRVmmBHNuvfaDGVv19 741ttud2pKa+CpCpdY27U034+eAiYDX4yOgjoX7e8VxXXWL+ze7fsqT7HfKNQgfXKH2o zZejoBD6UhxhZjz+912GqD8nwU+A/+GM99pTo1lSHZp8b9yP0z5AV3BUytsevTo742tr xeWn53M5qe+Uw2hkQIxmpQp22r6PCBMPEjxQd9HbJoQ7BcJYu1Hob0OSCGKWGveTpxK3 GJTitbn7tITZXV9f8dAQTy2ibHHjr/ZTfwzMoR8TAsnCAC7G1rhRfkwpbnyn99q9FD9X AcHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pfPDh91liqZm+BP4mghw0y9pBJj8lD1CecCVSBxX9RQ=; b=nQcSaYaWPeSuc6mQNWPA9HNgd18/JEYQmNt9VcEB7Lj5tqFjRghvhdB7N9HOzkLmEd Df56jECbHxtGz2LekvYFoXkuoZ/5NCOOB4EjTQwY5iVDValkTKPYtVFZH9RGx9qA9/Ms E0yfdRP+qy/B8vSIJe8L/xUwQBcbwGqFYujD89K16wkX4NClOrotOujHRwFhFbrKKFIV ysbyChz1t3+MSEe/8JMrRkhxRBtkvsdsG+Pf8H0oIrk3gDUyYOY9otPY2LldC30M5R96 JGz2cP6hBlNVhcPSnCHo4KbHaFPytN1uPcAzPBqA6lcbrgnmLMDzB0tFpe7/NfXO12ba QIdA== X-Gm-Message-State: AOAM530Y2UdL8bbWk1k6/DflN9E9U58H0gDXXkll1vKfZ7RvKQYVADyQ 0GF6z1jfypFe7N8UMwjrLPQ= X-Google-Smtp-Source: ABdhPJwXxdHfQIbhiS31nJsRloSgUOj73eXxsJggkaIikMTT44n46uK1Cl3+YoW4QIW7h+1LM3C1Zw== X-Received: by 2002:a63:af06:0:b0:378:3582:a49f with SMTP id w6-20020a63af06000000b003783582a49fmr12325169pge.125.1647711043808; Sat, 19 Mar 2022 10:30:43 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:43 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 03/14] bpf: Enable no charge in map _CREATE_FLAG_MASK Date: Sat, 19 Mar 2022 17:30:25 +0000 Message-Id: <20220319173036.23352-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Many maps have their create masks to warn the invalid map flag. Set BPF_F_NO_CHARGE in all these masks to enable it. Signed-off-by: Yafang Shao --- kernel/bpf/arraymap.c | 2 +- kernel/bpf/bloom_filter.c | 4 ++-- kernel/bpf/bpf_local_storage.c | 3 ++- kernel/bpf/bpf_struct_ops.c | 5 ++++- kernel/bpf/cpumap.c | 5 ++++- kernel/bpf/devmap.c | 2 +- kernel/bpf/hashtab.c | 2 +- kernel/bpf/local_storage.c | 2 +- kernel/bpf/lpm_trie.c | 2 +- kernel/bpf/queue_stack_maps.c | 2 +- kernel/bpf/ringbuf.c | 2 +- kernel/bpf/stackmap.c | 2 +- net/core/sock_map.c | 2 +- net/xdp/xskmap.c | 5 ++++- 14 files changed, 25 insertions(+), 15 deletions(-) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 7f145aefbff8..ac123747303c 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -16,7 +16,7 @@ #define ARRAY_CREATE_FLAG_MASK \ (BPF_F_NUMA_NODE | BPF_F_MMAPABLE | BPF_F_ACCESS_MASK | \ - BPF_F_PRESERVE_ELEMS | BPF_F_INNER_MAP) + BPF_F_PRESERVE_ELEMS | BPF_F_INNER_MAP | BPF_F_NO_CHARGE) static void bpf_array_free_percpu(struct bpf_array *array) { diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index b141a1346f72..f8ebfb4831e5 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -8,8 +8,8 @@ #include #include -#define BLOOM_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_ZERO_SEED | BPF_F_ACCESS_MASK) +#define BLOOM_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | \ + BPF_F_ZERO_SEED | BPF_F_ACCESS_MASK | BPF_F_NO_CHARGE) struct bpf_bloom_filter { struct bpf_map map; diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 092a1ac772d7..a92d3032fcde 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -15,7 +15,8 @@ #include #include -#define BPF_LOCAL_STORAGE_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_CLONE) +#define BPF_LOCAL_STORAGE_CREATE_FLAG_MASK \ + (BPF_F_NO_PREALLOC | BPF_F_CLONE | BPF_F_NO_CHARGE) static struct bpf_local_storage_map_bucket * select_bucket(struct bpf_local_storage_map *smap, diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 21069dbe9138..09eb848e6d12 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -21,6 +21,8 @@ enum bpf_struct_ops_state { refcount_t refcnt; \ enum bpf_struct_ops_state state +#define STRUCT_OPS_CREATE_FLAG_MASK (BPF_F_NO_CHARGE) + struct bpf_struct_ops_value { BPF_STRUCT_OPS_COMMON_VALUE; char data[] ____cacheline_aligned_in_smp; @@ -556,7 +558,8 @@ static void bpf_struct_ops_map_free(struct bpf_map *map) static int bpf_struct_ops_map_alloc_check(union bpf_attr *attr) { if (attr->key_size != sizeof(unsigned int) || attr->max_entries != 1 || - attr->map_flags || !attr->btf_vmlinux_value_type_id) + attr->map_flags & ~STRUCT_OPS_CREATE_FLAG_MASK || + !attr->btf_vmlinux_value_type_id) return -EINVAL; return 0; } diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 650e5d21f90d..201226fc652b 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -39,6 +39,9 @@ */ #define CPU_MAP_BULK_SIZE 8 /* 8 == one cacheline on 64-bit archs */ + +#define CPU_MAP_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | BPF_F_NO_CHARGE) + struct bpf_cpu_map_entry; struct bpf_cpu_map; @@ -93,7 +96,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) if (attr->max_entries == 0 || attr->key_size != 4 || (value_size != offsetofend(struct bpf_cpumap_val, qsize) && value_size != offsetofend(struct bpf_cpumap_val, bpf_prog.fd)) || - attr->map_flags & ~BPF_F_NUMA_NODE) + attr->map_flags & ~CPU_MAP_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_ACCOUNT); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 038f6d7a83e4..39bf8b521f27 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -50,7 +50,7 @@ #include #define DEV_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) + (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_NO_CHARGE) struct xdp_dev_bulk_queue { struct xdp_frame *q[DEV_MAP_BULK_SIZE]; diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 65877967f414..5c6ec8780b09 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -16,7 +16,7 @@ #define HTAB_CREATE_FLAG_MASK \ (BPF_F_NO_PREALLOC | BPF_F_NO_COMMON_LRU | BPF_F_NUMA_NODE | \ - BPF_F_ACCESS_MASK | BPF_F_ZERO_SEED) + BPF_F_ACCESS_MASK | BPF_F_ZERO_SEED | BPF_F_NO_CHARGE) #define BATCH_OPS(_name) \ .map_lookup_batch = \ diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 497916060ac7..865766c240d6 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -15,7 +15,7 @@ #include "../cgroup/cgroup-internal.h" #define LOCAL_STORAGE_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK | BPF_F_NO_CHARGE) struct bpf_cgroup_storage_map { struct bpf_map map; diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 5763cc7ac4f1..f42edf613624 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -537,7 +537,7 @@ static int trie_delete_elem(struct bpf_map *map, void *_key) #define LPM_KEY_SIZE_MIN LPM_KEY_SIZE(LPM_DATA_SIZE_MIN) #define LPM_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_NUMA_NODE | \ - BPF_F_ACCESS_MASK) + BPF_F_ACCESS_MASK | BPF_F_NO_CHARGE) static struct bpf_map *trie_alloc(union bpf_attr *attr) { diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index f9c734aaa990..c78eed4659ce 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -11,7 +11,7 @@ #include "percpu_freelist.h" #define QUEUE_STACK_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK | BPF_F_NO_CHARGE) struct bpf_queue_stack { struct bpf_map map; diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 710ba9de12ce..88779f688679 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -11,7 +11,7 @@ #include #include -#define RINGBUF_CREATE_FLAG_MASK (BPF_F_NUMA_NODE) +#define RINGBUF_CREATE_FLAG_MASK (BPF_F_NUMA_NODE | BPF_F_NO_CHARGE) /* non-mmap()'able part of bpf_ringbuf (everything up to consumer page) */ #define RINGBUF_PGOFF \ diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 38bdfcd06f55..b2e7dc1d9f5a 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -14,7 +14,7 @@ #define STACK_CREATE_FLAG_MASK \ (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | \ - BPF_F_STACK_BUILD_ID) + BPF_F_STACK_BUILD_ID | BPF_F_NO_CHARGE) struct stack_map_bucket { struct pcpu_freelist_node fnode; diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 2d213c4011db..7b0215bea413 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -22,7 +22,7 @@ struct bpf_stab { }; #define SOCK_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) + (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_NO_CHARGE) static int sock_map_prog_update(struct bpf_map *map, struct bpf_prog *prog, struct bpf_prog *old, u32 which); diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 65b53fb3de13..10a5ae727bd5 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -12,6 +12,9 @@ #include "xsk.h" +#define XSK_MAP_CREATE_FLAG_MASK \ + (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_NO_CHARGE) + static struct xsk_map_node *xsk_map_node_alloc(struct xsk_map *map, struct xdp_sock __rcu **map_entry) { @@ -68,7 +71,7 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) if (attr->max_entries == 0 || attr->key_size != 4 || attr->value_size != 4 || - attr->map_flags & ~(BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY)) + attr->map_flags & ~XSK_MAP_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); numa_node = bpf_map_attr_numa_node(attr); From patchwork Sat Mar 19 17:30:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786251 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46871C433EF for ; Sat, 19 Mar 2022 17:30:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243673AbiCSRcO (ORCPT ); Sat, 19 Mar 2022 13:32:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243028AbiCSRcH (ORCPT ); Sat, 19 Mar 2022 13:32:07 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99DAA45506; Sat, 19 Mar 2022 10:30:45 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id bx24-20020a17090af49800b001c6872a9e4eso6908966pjb.5; Sat, 19 Mar 2022 10:30:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=59FYyOnE/ZfEE+/nWxi+Uw2ttiMNAftCWsX5hGEEb68=; b=K5CK3XbPteaXT9NC/Kt7ZweGODe0q1YbJP/or7LyRRCogBUhlcCxGunIOv4Rr42z0/ aLYZeU9LGqsD2ly1kCsXG2KH/njEAXgvUo7OFEPCN54Tt0Zr5+PJWhLCcfHxmHllqrbo OvYkhSz1Zt9vq3YSTHiOwkQ+EPq4fEUQpo+YzQi5mecwY3YEuY9svC4mQzse8AUJ73MC mXOYHHRQuA6b3X8nUzE29jOe5XMxu9boxYVfyxrwYQD1vREplesaUPQzaYD1RG7KKVh+ jgdbaOZG6hi18NmezjFFyTBr/gl3vOquUADRvmeQ2/zIUxidC0vgBglFYGGwP7jKC30A AekA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=59FYyOnE/ZfEE+/nWxi+Uw2ttiMNAftCWsX5hGEEb68=; b=DLS5R3pOCn/7yPgZ+UUybAO3WD3467kTpMpO1/4wbCieZjZ9Oi0pQGx7hWQyxtdAaK sW0vhM4zLIiVGP3WZ4JMO2Y0JEsv7IG3hUMHkAtLcUI4gv7CRhYwDguui2R2hkMHZPM4 4cnRnSm79XN3acuqlrjxRypLvp3PZxZR1D1PCsSGsIm8/xp2BODoCFPVFzG2xp3Cpyf9 +rr4ny8o5tKToGF/oSTnxNmrJyG2NfPbh6/jcf/IjHxF5yOFuY76CudmsbosyfvgUwQm CJSSIdyhzoRpZrXUBfSRs+4CfaSscdL5tNb7c3Qq42N4ZhaeEbLexgS0H5EUZhSFSgbe fNGA== X-Gm-Message-State: AOAM530ZMQLNiLTE/HXL/kbKxIF9ENvsflutpHhageYuHSMWjHQCDLOk 4A+bNRRPhqphWc8SSeUBb3E= X-Google-Smtp-Source: ABdhPJwfmnqCjaYsNruPjkN8VzGPjXY5rVHbzV6iD/dhuKWThJxTIsUb4037rl7jXLCT9K9X1zDU9Q== X-Received: by 2002:a17:902:db0c:b0:154:1b36:6a6b with SMTP id m12-20020a170902db0c00b001541b366a6bmr5091539plx.7.1647711045029; Sat, 19 Mar 2022 10:30:45 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:44 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 04/14] bpf: Introduce new parameter bpf_attr in bpf_map_area_alloc Date: Sat, 19 Mar 2022 17:30:26 +0000 Message-Id: <20220319173036.23352-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add a new parameter bpf_attr in bpf_map_area_alloc(), then we can get no charge flag from it. Currently there're two parameters, one of which is also got from bpf_attr, so we can remove it after this change. No functional change. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 4 ++-- kernel/bpf/arraymap.c | 5 ++--- kernel/bpf/bloom_filter.c | 3 +-- kernel/bpf/bpf_struct_ops.c | 8 ++++---- kernel/bpf/cpumap.c | 3 +-- kernel/bpf/devmap.c | 10 ++++------ kernel/bpf/hashtab.c | 10 ++++------ kernel/bpf/queue_stack_maps.c | 3 +-- kernel/bpf/reuseport_array.c | 3 +-- kernel/bpf/ringbuf.c | 11 ++++++----- kernel/bpf/stackmap.c | 11 ++++++----- kernel/bpf/syscall.c | 13 +++++++------ net/core/sock_map.c | 6 ++---- net/xdp/xskmap.c | 4 +--- 14 files changed, 42 insertions(+), 52 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 07c6603a6c81..90a542d5a411 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1518,8 +1518,8 @@ void bpf_map_inc_with_uref(struct bpf_map *map); struct bpf_map * __must_check bpf_map_inc_not_zero(struct bpf_map *map); void bpf_map_put_with_uref(struct bpf_map *map); void bpf_map_put(struct bpf_map *map); -void *bpf_map_area_alloc(u64 size, int numa_node); -void *bpf_map_area_mmapable_alloc(u64 size, int numa_node); +void *bpf_map_area_alloc(u64 size, union bpf_attr *attr); +void *bpf_map_area_mmapable_alloc(u64 size, union bpf_attr *attr); void bpf_map_area_free(void *base); bool bpf_map_write_active(const struct bpf_map *map); void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index ac123747303c..e26aef906392 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -81,7 +81,6 @@ int array_map_alloc_check(union bpf_attr *attr) static struct bpf_map *array_map_alloc(union bpf_attr *attr) { bool percpu = attr->map_type == BPF_MAP_TYPE_PERCPU_ARRAY; - int numa_node = bpf_map_attr_numa_node(attr); u32 elem_size, index_mask, max_entries; bool bypass_spec_v1 = bpf_bypass_spec_v1(); u64 array_size, mask64; @@ -130,13 +129,13 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) void *data; /* kmalloc'ed memory can't be mmap'ed, use explicit vmalloc */ - data = bpf_map_area_mmapable_alloc(array_size, numa_node); + data = bpf_map_area_mmapable_alloc(array_size, attr); if (!data) return ERR_PTR(-ENOMEM); array = data + PAGE_ALIGN(sizeof(struct bpf_array)) - offsetof(struct bpf_array, value); } else { - array = bpf_map_area_alloc(array_size, numa_node); + array = bpf_map_area_alloc(array_size, attr); } if (!array) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index f8ebfb4831e5..a35c664b4a02 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -90,7 +90,6 @@ static int bloom_map_get_next_key(struct bpf_map *map, void *key, void *next_key static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) { u32 bitset_bytes, bitset_mask, nr_hash_funcs, nr_bits; - int numa_node = bpf_map_attr_numa_node(attr); struct bpf_bloom_filter *bloom; if (!bpf_capable()) @@ -141,7 +140,7 @@ static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) } bitset_bytes = roundup(bitset_bytes, sizeof(unsigned long)); - bloom = bpf_map_area_alloc(sizeof(*bloom) + bitset_bytes, numa_node); + bloom = bpf_map_area_alloc(sizeof(*bloom) + bitset_bytes, attr); if (!bloom) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 09eb848e6d12..1ca1407ae5e6 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -591,17 +591,17 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) */ (vt->size - sizeof(struct bpf_struct_ops_value)); - st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE); + attr->map_flags &= ~BPF_F_NUMA_NODE; + st_map = bpf_map_area_alloc(st_map_size, attr); if (!st_map) return ERR_PTR(-ENOMEM); st_map->st_ops = st_ops; map = &st_map->map; - st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE); + st_map->uvalue = bpf_map_area_alloc(vt->size, attr); st_map->progs = - bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_prog *), - NUMA_NO_NODE); + bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_prog *), attr); st_map->image = bpf_jit_alloc_exec(PAGE_SIZE); if (!st_map->uvalue || !st_map->progs || !st_map->image) { bpf_struct_ops_map_free(map); diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 201226fc652b..5a5b40e986ff 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -113,8 +113,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) /* Alloc array for possible remote "destination" CPUs */ cmap->cpu_map = bpf_map_area_alloc(cmap->map.max_entries * - sizeof(struct bpf_cpu_map_entry *), - cmap->map.numa_node); + sizeof(struct bpf_cpu_map_entry *), attr); if (!cmap->cpu_map) goto free_cmap; diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 39bf8b521f27..2857176c82bb 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -88,12 +88,12 @@ static DEFINE_SPINLOCK(dev_map_lock); static LIST_HEAD(dev_map_list); static struct hlist_head *dev_map_create_hash(unsigned int entries, - int numa_node) + union bpf_attr *attr) { int i; struct hlist_head *hash; - hash = bpf_map_area_alloc((u64) entries * sizeof(*hash), numa_node); + hash = bpf_map_area_alloc((u64) entries * sizeof(*hash), attr); if (hash != NULL) for (i = 0; i < entries; i++) INIT_HLIST_HEAD(&hash[i]); @@ -137,16 +137,14 @@ static int dev_map_init_map(struct bpf_dtab *dtab, union bpf_attr *attr) } if (attr->map_type == BPF_MAP_TYPE_DEVMAP_HASH) { - dtab->dev_index_head = dev_map_create_hash(dtab->n_buckets, - dtab->map.numa_node); + dtab->dev_index_head = dev_map_create_hash(dtab->n_buckets, attr); if (!dtab->dev_index_head) return -ENOMEM; spin_lock_init(&dtab->index_lock); } else { dtab->netdev_map = bpf_map_area_alloc((u64) dtab->map.max_entries * - sizeof(struct bpf_dtab_netdev *), - dtab->map.numa_node); + sizeof(struct bpf_dtab_netdev *), attr); if (!dtab->netdev_map) return -ENOMEM; } diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 5c6ec8780b09..2c84045ff8e1 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -303,7 +303,7 @@ static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key, return NULL; } -static int prealloc_init(struct bpf_htab *htab) +static int prealloc_init(union bpf_attr *attr, struct bpf_htab *htab) { u32 num_entries = htab->map.max_entries; int err = -ENOMEM, i; @@ -311,8 +311,7 @@ static int prealloc_init(struct bpf_htab *htab) if (htab_has_extra_elems(htab)) num_entries += num_possible_cpus(); - htab->elems = bpf_map_area_alloc((u64)htab->elem_size * num_entries, - htab->map.numa_node); + htab->elems = bpf_map_area_alloc((u64)htab->elem_size * num_entries, attr); if (!htab->elems) return -ENOMEM; @@ -513,8 +512,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) err = -ENOMEM; htab->buckets = bpf_map_area_alloc(htab->n_buckets * - sizeof(struct bucket), - htab->map.numa_node); + sizeof(struct bucket), attr); if (!htab->buckets) goto free_htab; @@ -535,7 +533,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) htab_init_buckets(htab); if (prealloc) { - err = prealloc_init(htab); + err = prealloc_init(attr, htab); if (err) goto free_map_locked; diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index c78eed4659ce..0ff93c5bc184 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -66,14 +66,13 @@ static int queue_stack_map_alloc_check(union bpf_attr *attr) static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) { - int numa_node = bpf_map_attr_numa_node(attr); struct bpf_queue_stack *qs; u64 size, queue_size; size = (u64) attr->max_entries + 1; queue_size = sizeof(*qs) + size * attr->value_size; - qs = bpf_map_area_alloc(queue_size, numa_node); + qs = bpf_map_area_alloc(queue_size, attr); if (!qs) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index 8251243022a2..b19fb70118a4 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -150,14 +150,13 @@ static void reuseport_array_free(struct bpf_map *map) static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) { - int numa_node = bpf_map_attr_numa_node(attr); struct reuseport_array *array; if (!bpf_capable()) return ERR_PTR(-EPERM); /* allocate all map elements and zero-initialize them */ - array = bpf_map_area_alloc(struct_size(array, ptrs, attr->max_entries), numa_node); + array = bpf_map_area_alloc(struct_size(array, ptrs, attr->max_entries), attr); if (!array) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 88779f688679..a3b4d2a0a2c7 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -58,13 +58,14 @@ struct bpf_ringbuf_hdr { u32 pg_off; }; -static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) +static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, union bpf_attr *attr) { const gfp_t flags = GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL | __GFP_NOWARN | __GFP_ZERO; int nr_meta_pages = RINGBUF_PGOFF + RINGBUF_POS_PAGES; int nr_data_pages = data_sz >> PAGE_SHIFT; int nr_pages = nr_meta_pages + nr_data_pages; + int numa_node = bpf_map_attr_numa_node(attr); struct page **pages, *page; struct bpf_ringbuf *rb; size_t array_size; @@ -88,7 +89,7 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) * user-space implementations significantly. */ array_size = (nr_meta_pages + 2 * nr_data_pages) * sizeof(*pages); - pages = bpf_map_area_alloc(array_size, numa_node); + pages = bpf_map_area_alloc(array_size, attr); if (!pages) return NULL; @@ -126,11 +127,11 @@ static void bpf_ringbuf_notify(struct irq_work *work) wake_up_all(&rb->waitq); } -static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node) +static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, union bpf_attr *attr) { struct bpf_ringbuf *rb; - rb = bpf_ringbuf_area_alloc(data_sz, numa_node); + rb = bpf_ringbuf_area_alloc(data_sz, attr); if (!rb) return NULL; @@ -169,7 +170,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) bpf_map_init_from_attr(&rb_map->map, attr); - rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); + rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, attr); if (!rb_map->rb) { kfree(rb_map); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index b2e7dc1d9f5a..ed6bebef0132 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -42,14 +42,15 @@ static inline int stack_map_data_size(struct bpf_map *map) sizeof(struct bpf_stack_build_id) : sizeof(u64); } -static int prealloc_elems_and_freelist(struct bpf_stack_map *smap) +static int prealloc_elems_and_freelist(union bpf_attr *attr, + struct bpf_stack_map *smap) { u64 elem_size = sizeof(struct stack_map_bucket) + (u64)smap->map.value_size; int err; - smap->elems = bpf_map_area_alloc(elem_size * smap->map.max_entries, - smap->map.numa_node); + smap->elems = bpf_map_area_alloc(elem_size * + smap->map.max_entries, attr); if (!smap->elems) return -ENOMEM; @@ -101,7 +102,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) cost = n_buckets * sizeof(struct stack_map_bucket *) + sizeof(*smap); cost += n_buckets * (value_size + sizeof(struct stack_map_bucket)); - smap = bpf_map_area_alloc(cost, bpf_map_attr_numa_node(attr)); + smap = bpf_map_area_alloc(cost, attr); if (!smap) return ERR_PTR(-ENOMEM); @@ -113,7 +114,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) if (err) goto free_smap; - err = prealloc_elems_and_freelist(smap); + err = prealloc_elems_and_freelist(attr, smap); if (err) goto put_buffers; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 0cca3d7d0d84..f70a7067ef4a 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -295,7 +295,7 @@ static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, * (e.g. in map update path) without taking care of setting the active * memory cgroup (see at bpf_map_kmalloc_node() for example). */ -static void *__bpf_map_area_alloc(u64 size, int numa_node, bool mmapable) +static void *__bpf_map_area_alloc(u64 size, union bpf_attr *attr, bool mmapable) { /* We really just want to fail instead of triggering OOM killer * under memory pressure, therefore we set __GFP_NORETRY to kmalloc, @@ -308,8 +308,9 @@ static void *__bpf_map_area_alloc(u64 size, int numa_node, bool mmapable) */ const gfp_t gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_ACCOUNT; - unsigned int flags = 0; + int numa_node = bpf_map_attr_numa_node(attr); unsigned long align = 1; + unsigned int flags = 0; void *area; if (size >= SIZE_MAX) @@ -332,14 +333,14 @@ static void *__bpf_map_area_alloc(u64 size, int numa_node, bool mmapable) flags, numa_node, __builtin_return_address(0)); } -void *bpf_map_area_alloc(u64 size, int numa_node) +void *bpf_map_area_alloc(u64 size, union bpf_attr *attr) { - return __bpf_map_area_alloc(size, numa_node, false); + return __bpf_map_area_alloc(size, attr, false); } -void *bpf_map_area_mmapable_alloc(u64 size, int numa_node) +void *bpf_map_area_mmapable_alloc(u64 size, union bpf_attr *attr) { - return __bpf_map_area_alloc(size, numa_node, true); + return __bpf_map_area_alloc(size, attr, true); } void bpf_map_area_free(void *area) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 7b0215bea413..26b89d37944d 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -49,8 +49,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) raw_spin_lock_init(&stab->lock); stab->sks = bpf_map_area_alloc((u64) stab->map.max_entries * - sizeof(struct sock *), - stab->map.numa_node); + sizeof(struct sock *), attr); if (!stab->sks) { kfree(stab); return ERR_PTR(-ENOMEM); @@ -1093,8 +1092,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) } htab->buckets = bpf_map_area_alloc(htab->buckets_num * - sizeof(struct bpf_shtab_bucket), - htab->map.numa_node); + sizeof(struct bpf_shtab_bucket), attr); if (!htab->buckets) { err = -ENOMEM; goto free_htab; diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 10a5ae727bd5..50795c0c9b81 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -63,7 +63,6 @@ static void xsk_map_sock_delete(struct xdp_sock *xs, static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) { struct xsk_map *m; - int numa_node; u64 size; if (!capable(CAP_NET_ADMIN)) @@ -74,10 +73,9 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) attr->map_flags & ~XSK_MAP_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - numa_node = bpf_map_attr_numa_node(attr); size = struct_size(m, xsk_map, attr->max_entries); - m = bpf_map_area_alloc(size, numa_node); + m = bpf_map_area_alloc(size, attr); if (!m) return ERR_PTR(-ENOMEM); From patchwork Sat Mar 19 17:30:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786252 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A99AC433F5 for ; Sat, 19 Mar 2022 17:30:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243676AbiCSRcP (ORCPT ); Sat, 19 Mar 2022 13:32:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243386AbiCSRcH (ORCPT ); Sat, 19 Mar 2022 13:32:07 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF9CB4550E; Sat, 19 Mar 2022 10:30:46 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id gb19so9875707pjb.1; Sat, 19 Mar 2022 10:30:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3WuzE/uAtr18lGVaEOIViWaXXDGLmE8cOvzc322Vqig=; b=U3gnt1odSsXS96u0KSUsMa4JdEK+c88qDiumrnoW9VlEI1drFOGmZFEJk8GSvB/T6x qa99pBmwoBsTYcODdLG0t69lxdoFOF5JsrU5lR4OYVfWG/dqOWi8GzPw93QkywOcu0BK Zr+I2xakib/C99HXmI0AH8SjCqKug9WUTf1MLAcMiG7bfjZp1jfLgUwez3Zf4kgdGKq1 b+h+fwMAdWHAIMzPxwpm2/hWyM7NvwOz/+kMxwpRQ7A7ojh4BAailix4UWwAMZGFPBPe z7jjQmG/12K/lofHz2tTdldjKW2Q0G+Lb6wfZSDVEBtNCKzj7QYL0oofFaQkPOBVFJmh czpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3WuzE/uAtr18lGVaEOIViWaXXDGLmE8cOvzc322Vqig=; b=6u7PDM9UyOjlPRW90+IVnixC4C46GcPg2IcRPFnI8eZOw7I6VaqGGUYGohyE/IePCy xiVok9jvPXduqe37c0W60XmDQt1YXI8wa/lX7g+d4H0afA7IUXbkIL6qlIIl0rDz1bLi tuUNM1DdD4Gms2Sn5TPkww6tM5TIJY/+UfmmwjRY6UqtF5d2+1p2lzF3x9CcLvFHkUGg 2V6+WlRYQDu5xqY5Gq1vrnlLkfCYibUXF1k2RpOt08Y1PsAOS7Jr0mCWxPqZAqjmeRay 2bq8yClBSP+leYfhORa+IogImJQa8ZFUb+9IsRYf9nYIoFwpXB7x/ldHbuTE+ln2pKMY fMLA== X-Gm-Message-State: AOAM532p7l3xohrres5+zHpPyRlKv3tTT+Ir+Cb6nOHLkYhUD7zCqURa 30JBrWTH/6EHJbXaFY5velc= X-Google-Smtp-Source: ABdhPJw9FIF9vgwyN8OREh52A/jGghm9PrNkyABokhdKcaIawAJarKGKDk7JTox59oPleyl7Um+K3w== X-Received: by 2002:a17:90a:d584:b0:1b8:7864:1735 with SMTP id v4-20020a17090ad58400b001b878641735mr17400048pju.126.1647711046284; Sat, 19 Mar 2022 10:30:46 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:45 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 05/14] bpf: Allow no charge in bpf_map_area_alloc Date: Sat, 19 Mar 2022 17:30:27 +0000 Message-Id: <20220319173036.23352-6-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Use the helper we introduced before to decide whether set the __GFP_ACCOUNT or not. Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index f70a7067ef4a..add3b4045b4d 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -307,7 +307,7 @@ static void *__bpf_map_area_alloc(u64 size, union bpf_attr *attr, bool mmapable) * __GFP_RETRY_MAYFAIL to avoid such situations. */ - const gfp_t gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_ACCOUNT; + const gfp_t gfp = map_flags_no_charge(__GFP_NOWARN | __GFP_ZERO, attr); int numa_node = bpf_map_attr_numa_node(attr); unsigned long align = 1; unsigned int flags = 0; From patchwork Sat Mar 19 17:30:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786253 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C119C433EF for ; Sat, 19 Mar 2022 17:31:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243679AbiCSRcW (ORCPT ); Sat, 19 Mar 2022 13:32:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243429AbiCSRcL (ORCPT ); Sat, 19 Mar 2022 13:32:11 -0400 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7E104552B; Sat, 19 Mar 2022 10:30:47 -0700 (PDT) Received: by mail-pj1-x1033.google.com with SMTP id l4-20020a17090a49c400b001c6840df4a3so7230301pjm.0; Sat, 19 Mar 2022 10:30:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tlUVvnpalHf+NQDEGRuVRqs1mWkMR7ChiLJEV3o+2Gs=; b=b+0WOzv5QB5+n/S20HOa2mzV0FPpHcvm5Mx9dRPMPi7QXc74QJdQSxv7OA8ZqCgIiK iBpzwwFyifkOo5bTzlngp0xZ3JGhEx9QkudeAMe0+40BuN5YHkTzX2v3ZX9a0EfZl2aW 8Tqci4BTDrs36dy2i6vFKfuL8bfCcDESXEQA6ypm5CkAgDotEzGaU0ENq0rrVBTIypOU R0FagYGZDwo0mw7n/YF6JJ7hUhpLf406llT0/cbYv2YXgZA4uwFG2lLa60lFtirQn7Ja hLmQascreXw/6d3tZzPiP5rSkjByzIFCDTYKYiC3NWfGxzlTOzmOTGistXW2i1CCmARZ vh9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tlUVvnpalHf+NQDEGRuVRqs1mWkMR7ChiLJEV3o+2Gs=; b=rd5vu5cuR5CdYHbb3wSTNOhmwP+lZH1KHFtQNxcxgNH79SRyzp+5TgaZ14ekkcEWqs A+VVUQ4W72AZeC85AIh5KSkPUlMkNPrFmDQsSup6NH0cHN6mVeeaGQzUXQTm11WTE1Sg 5CUsqg4M58S2kwkmDblnaMbnybAieqN0n231sPSZCqhPFCSTW2W1RPVJxp2f4sZfeSp7 hVt3mJSsfCDdgbuX2xgwXHrIWnRxyt0IENuOZX0oq0VcjugD7iD1FnBsD9IBQSo/gS4L 24BYxK/5MTIE5Ahwc2R0sq+1trkWCw6+rN54x8JPcea2rsWCVuZNK1qF8Bx7xNG9Hqf3 QfxQ== X-Gm-Message-State: AOAM530Hb0rNG1U2wISbhQUqNm6p941PguE0oRQ12wi5yykCS2Amg3Lo RCocWVX3Dylprh5KAB212ZY= X-Google-Smtp-Source: ABdhPJwJRkOwuhK3vq0vx0S4/e1H1jq1fI2TUfzFUYVOzgYTg/Db5VgKFU+zPydyVvPXiNi64ge4+Q== X-Received: by 2002:a17:902:be18:b0:153:2444:9c1a with SMTP id r24-20020a170902be1800b0015324449c1amr5212078pls.152.1647711047452; Sat, 19 Mar 2022 10:30:47 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:47 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 06/14] bpf: Allow no charge for allocation not at map creation time Date: Sat, 19 Mar 2022 17:30:28 +0000 Message-Id: <20220319173036.23352-7-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Below three functions are used for memory allocation which is not at map creation time, - bpf_map_kmalloc_node() - bpf_map_kzalloc() - bpf_map_alloc_percpu() For this kind of path, we can get the no charge flag from bpf_map struct we set before. Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index add3b4045b4d..e84aeefa05f4 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -434,7 +434,8 @@ void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, void *ptr; old_memcg = set_active_memcg(map->memcg); - ptr = kmalloc_node(size, flags | __GFP_ACCOUNT, node); + ptr = kmalloc_node(size, bpf_flags_no_charge(flags, map->no_charge), + node); set_active_memcg(old_memcg); return ptr; @@ -446,7 +447,7 @@ void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) void *ptr; old_memcg = set_active_memcg(map->memcg); - ptr = kzalloc(size, flags | __GFP_ACCOUNT); + ptr = kzalloc(size, bpf_flags_no_charge(flags, map->no_charge)); set_active_memcg(old_memcg); return ptr; @@ -459,7 +460,8 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, void __percpu *ptr; old_memcg = set_active_memcg(map->memcg); - ptr = __alloc_percpu_gfp(size, align, flags | __GFP_ACCOUNT); + ptr = __alloc_percpu_gfp(size, align, + bpf_flags_no_charge(flags, map->no_charge)); set_active_memcg(old_memcg); return ptr; From patchwork Sat Mar 19 17:30:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786256 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91EDBC433F5 for ; Sat, 19 Mar 2022 17:31:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243709AbiCSRc1 (ORCPT ); Sat, 19 Mar 2022 13:32:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243632AbiCSRcM (ORCPT ); Sat, 19 Mar 2022 13:32:12 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B9D445789; Sat, 19 Mar 2022 10:30:49 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id z3so9462505plg.8; Sat, 19 Mar 2022 10:30:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fbHuw2wN1RGMdDQanAXcScwfI02r7zqf8nuc7Y7i6So=; b=EftifGYUFgc8p+r1eKYESSfOtKu9MWFg12bUtv26Rl9uF8geVk4/r0RujVw831C6Lr T67W6T2AueI9nWi7cQR7+n+uc21vhyz/KzxW3obbga4ubV8ZHtNCvwkJGAKLOPgGPSIk 721tpCeodYzGhqsZqTUNTRUprE5xNnkes80HIOv2xps7OOSbI4Lp98xb3bUz+SxWyy92 XNIySUhtFQKYVe9ZqsF9YqZ/ri7ZiNx8cT+U6sIVIkPOO1xM20Jh5k9ZoQMAg7+S7/uQ WqQnWm1PEr6vu2BoH16fWCxx2Vvfw4H+Bd4T5qAkZRbbgyAi95RPz7rgZokXSTrDnvfG V2Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fbHuw2wN1RGMdDQanAXcScwfI02r7zqf8nuc7Y7i6So=; b=FfMgD7Y2QRcpoAOOJPFsnrXCpG+kOCPqnTfYg3ywLd4w0doJkE20pVBm/7y8Z1+qv9 khp1JskFdQolG8IDu/Mfv3RTf6BZWNhg4NZ8ED1uvjtGZQ4tHMS5XuZYYksn/734rFBs Wpch0Rf0yy1PtfnjqQfZJcxVU0lMS9QjXCXS1IKuTXERjEVYr4PW54ZTh8elAXuRNiS/ 6I7prBoOs8NgEYSVpbwt1Brp53Bj6rTfJWZE1tO9LYiJmlTAQjKv4TkJ1OCbYDjeLmYY h2XPH3+4cVF7ZMclhwbnx3SeYRbKjQ+66ppbqh6XTmbsiSI1T+AKtKqe4ve6k5axVneG RBzg== X-Gm-Message-State: AOAM5317qW4LAEB1d+Atg9Nj1o/ylXm6T4D+UB6BKXtqmDHWr3EB4+e3 PG+ODC85qUhddwaKj/RKk3V3cG2YornQN7z752Y= X-Google-Smtp-Source: ABdhPJwJva28WJ7jWfa33Znt81ZHpX/1QIXtCHZnxe+533hjN5affi9hym5cjzvBQn/oJ+7rhn5haA== X-Received: by 2002:a17:90a:1f4d:b0:1bb:a657:ace5 with SMTP id y13-20020a17090a1f4d00b001bba657ace5mr27796079pjy.39.1647711048648; Sat, 19 Mar 2022 10:30:48 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:48 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 07/14] bpf: Allow no charge in map specific allocation Date: Sat, 19 Mar 2022 17:30:29 +0000 Message-Id: <20220319173036.23352-8-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Some maps have their own ->map_alloc, in which the no charge should also be allowed. Signed-off-by: Yafang Shao --- kernel/bpf/arraymap.c | 2 +- kernel/bpf/bpf_local_storage.c | 5 +++-- kernel/bpf/cpumap.c | 2 +- kernel/bpf/devmap.c | 2 +- kernel/bpf/hashtab.c | 2 +- kernel/bpf/local_storage.c | 2 +- kernel/bpf/lpm_trie.c | 2 +- kernel/bpf/ringbuf.c | 6 +++--- 8 files changed, 12 insertions(+), 11 deletions(-) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index e26aef906392..9df425ad769c 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -1062,7 +1062,7 @@ static struct bpf_map *prog_array_map_alloc(union bpf_attr *attr) struct bpf_array_aux *aux; struct bpf_map *map; - aux = kzalloc(sizeof(*aux), GFP_KERNEL_ACCOUNT); + aux = kzalloc(sizeof(*aux), map_flags_no_charge(GFP_KERNEL, attr)); if (!aux) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index a92d3032fcde..b626546d384d 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -582,11 +582,12 @@ int bpf_local_storage_map_alloc_check(union bpf_attr *attr) struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) { + gfp_t gfp_flags = map_flags_no_charge(GFP_USER | __GFP_NOWARN, attr); struct bpf_local_storage_map *smap; unsigned int i; u32 nbuckets; - smap = kzalloc(sizeof(*smap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap = kzalloc(sizeof(*smap), gfp_flags); if (!smap) return ERR_PTR(-ENOMEM); bpf_map_init_from_attr(&smap->map, attr); @@ -597,7 +598,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) smap->bucket_log = ilog2(nbuckets); smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + map_flags_no_charge(GFP_USER | __GFP_NOWARN, attr)); if (!smap->buckets) { kfree(smap); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 5a5b40e986ff..fd3b3f05e76a 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -99,7 +99,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~CPU_MAP_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_ACCOUNT); + cmap = kzalloc(sizeof(*cmap), map_flags_no_charge(GFP_USER, attr)); if (!cmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 2857176c82bb..6aaa2e3ce795 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -160,7 +160,7 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_ACCOUNT); + dtab = kzalloc(sizeof(*dtab), map_flags_no_charge(GFP_USER, attr)); if (!dtab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 2c84045ff8e1..74696d8196a5 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -474,7 +474,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_ACCOUNT); + htab = kzalloc(sizeof(*htab), map_flags_no_charge(GFP_USER, attr)); if (!htab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 865766c240d6..741ab7cf3626 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,7 +313,7 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) return ERR_PTR(-EINVAL); map = kmalloc_node(sizeof(struct bpf_cgroup_storage_map), - __GFP_ZERO | GFP_USER | __GFP_ACCOUNT, numa_node); + map_flags_no_charge(__GFP_ZERO | GFP_USER, attr), numa_node); if (!map) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index f42edf613624..9673f8e98c58 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -557,7 +557,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) attr->value_size > LPM_VAL_SIZE_MAX) return ERR_PTR(-EINVAL); - trie = kzalloc(sizeof(*trie), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + trie = kzalloc(sizeof(*trie), map_flags_no_charge(GFP_USER | __GFP_NOWARN, attr)); if (!trie) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index a3b4d2a0a2c7..3db07cd0ab60 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -60,8 +60,8 @@ struct bpf_ringbuf_hdr { static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, union bpf_attr *attr) { - const gfp_t flags = GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL | - __GFP_NOWARN | __GFP_ZERO; + const gfp_t flags = map_flags_no_charge(__GFP_RETRY_MAYFAIL | + __GFP_NOWARN | __GFP_ZERO, attr); int nr_meta_pages = RINGBUF_PGOFF + RINGBUF_POS_PAGES; int nr_data_pages = data_sz >> PAGE_SHIFT; int nr_pages = nr_meta_pages + nr_data_pages; @@ -164,7 +164,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_ACCOUNT); + rb_map = kzalloc(sizeof(*rb_map), map_flags_no_charge(GFP_USER, attr)); if (!rb_map) return ERR_PTR(-ENOMEM); From patchwork Sat Mar 19 17:30:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786255 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21644C433EF for ; Sat, 19 Mar 2022 17:31:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243708AbiCSRc0 (ORCPT ); Sat, 19 Mar 2022 13:32:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243386AbiCSRcU (ORCPT ); Sat, 19 Mar 2022 13:32:20 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56D594578A; Sat, 19 Mar 2022 10:30:50 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id mm17-20020a17090b359100b001c6da62a559so1303235pjb.3; Sat, 19 Mar 2022 10:30:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cDbZmnyIct7dh5oDHwzWtZU+1KNpHzcJsMgS0/FqC04=; b=qSUGKlq9jr+L42BLcgqJwPR3lzz7NGXgeJc3Ei2E1+bUYjo0YyU+vCBYGWCPEhjQZC VvMCBJMqT/taNLtjl93gvIrB1/F7XpPn2Zb1inz+Lm+jQXUHszDGKJHFo4CfzE/i+22I cEXJVHx9YcusE6ibDBjce99Df8IAw19ABmTNMKjX0u3iNaDmjW2WdWmVAxYx9nY42pZZ jHVT3wqvwOn/+fdvWV1z+drzYTj/5QmjJbwIBVjymZA8B/iz7Z2TW8NpHoP8ZKsmq7D1 s80vFs2WJQB8qg/6iNqif2OKYOm1tBmGisy7CEaXUnqQFK8DDSiBo/UKnLRzvVdV9uzz EPJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cDbZmnyIct7dh5oDHwzWtZU+1KNpHzcJsMgS0/FqC04=; b=hxGOmfQpwgkSJ83rsr10ijvu+BqRc/cuLjRzFyPCr96iER4Hux4nWbUF4DY65rNMxZ IM5W9fypPVV7a0uQTTny8ZpxPDfuTi9N8hYwVo5MEby+b0cObuUfsFVvaU1aAUClxrFS xhwMqx4wWQAGW2iLSOcO7wiCx8Zg0ZdhG4aMo5aNtUJwGokEQUGOfLepHWQneTdK8Kc1 QcnMcyA7gY3RwXbHjLLum+rFzYkPeS6Hn31pCeTRu7DYgn7Pt0ca1YHKcaVir1iifayt 6FOkW4r7J+s7JvwGPQj1fcsyy6wZ+vyKompXUWhvMrGk8mstHHsCM9+ypVOdOlbdxHOi e5LQ== X-Gm-Message-State: AOAM532i/yRK2W61/81o3i2NiVaeqgqZ3aIHyG507ZS/32U6WHGt1pyA gmOXopbkNXgqUHWjzLERQwU= X-Google-Smtp-Source: ABdhPJzfqmK2Zh6x7voOpbJTgq3UTIo40u8y9v1MPFqLp/68HUc5AdKYX5mafm02kCF7iYc3MKftCg== X-Received: by 2002:a17:90b:3b43:b0:1c6:f878:ea52 with SMTP id ot3-20020a17090b3b4300b001c6f878ea52mr459431pjb.68.1647711049850; Sat, 19 Mar 2022 10:30:49 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:49 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 08/14] bpf: Aggregate flags for BPF_PROG_LOAD command Date: Sat, 19 Mar 2022 17:30:30 +0000 Message-Id: <20220319173036.23352-9-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net It will be easy to read if we aggregate the flags for BPF_PROG_LOAD into Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 15 +++++++++------ tools/include/uapi/linux/bpf.h | 15 +++++++++------ 2 files changed, 18 insertions(+), 12 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index e2dba6cdd88d..93ee04fb8c62 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1065,12 +1065,14 @@ enum bpf_link_type { #define BPF_F_ALLOW_MULTI (1U << 1) #define BPF_F_REPLACE (1U << 2) +/* flags for BPF_PROG_LOAD command */ +enum { /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the * verifier will perform strict alignment checking as if the kernel * has been built with CONFIG_EFFICIENT_UNALIGNED_ACCESS not set, * and NET_IP_ALIGN defined to 2. */ -#define BPF_F_STRICT_ALIGNMENT (1U << 0) + BPF_F_STRICT_ALIGNMENT = (1U << 0), /* If BPF_F_ANY_ALIGNMENT is used in BPF_PROF_LOAD command, the * verifier will allow any alignment whatsoever. On platforms @@ -1084,7 +1086,7 @@ enum bpf_link_type { * of an unaligned access the alignment check would trigger before * the one we are interested in. */ -#define BPF_F_ANY_ALIGNMENT (1U << 1) + BPF_F_ANY_ALIGNMENT = (1U << 1), /* BPF_F_TEST_RND_HI32 is used in BPF_PROG_LOAD command for testing purpose. * Verifier does sub-register def/use analysis and identifies instructions whose @@ -1102,10 +1104,10 @@ enum bpf_link_type { * Then, if verifier is not doing correct analysis, such randomization will * regress tests to expose bugs. */ -#define BPF_F_TEST_RND_HI32 (1U << 2) + BPF_F_TEST_RND_HI32 = (1U << 2), /* The verifier internal test flag. Behavior is undefined */ -#define BPF_F_TEST_STATE_FREQ (1U << 3) + BPF_F_TEST_STATE_FREQ = (1U << 3), /* If BPF_F_SLEEPABLE is used in BPF_PROG_LOAD command, the verifier will * restrict map and helper usage for such programs. Sleepable BPF programs can @@ -1113,12 +1115,13 @@ enum bpf_link_type { * Such programs are allowed to use helpers that may sleep like * bpf_copy_from_user(). */ -#define BPF_F_SLEEPABLE (1U << 4) + BPF_F_SLEEPABLE = (1U << 4), /* If BPF_F_XDP_HAS_FRAGS is used in BPF_PROG_LOAD command, the loaded program * fully support xdp frags. */ -#define BPF_F_XDP_HAS_FRAGS (1U << 5) + BPF_F_XDP_HAS_FRAGS = (1U << 5), +}; /* link_create.kprobe_multi.flags used in LINK_CREATE command for * BPF_TRACE_KPROBE_MULTI attach type to create return probe. diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index e2dba6cdd88d..71a4d8fdc880 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1065,12 +1065,14 @@ enum bpf_link_type { #define BPF_F_ALLOW_MULTI (1U << 1) #define BPF_F_REPLACE (1U << 2) +/* flags for BPF_PROG_LOAD */ +enum { /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the * verifier will perform strict alignment checking as if the kernel * has been built with CONFIG_EFFICIENT_UNALIGNED_ACCESS not set, * and NET_IP_ALIGN defined to 2. */ -#define BPF_F_STRICT_ALIGNMENT (1U << 0) + BPF_F_STRICT_ALIGNMENT = (1U << 0), /* If BPF_F_ANY_ALIGNMENT is used in BPF_PROF_LOAD command, the * verifier will allow any alignment whatsoever. On platforms @@ -1084,7 +1086,7 @@ enum bpf_link_type { * of an unaligned access the alignment check would trigger before * the one we are interested in. */ -#define BPF_F_ANY_ALIGNMENT (1U << 1) + BPF_F_ANY_ALIGNMENT = (1U << 1), /* BPF_F_TEST_RND_HI32 is used in BPF_PROG_LOAD command for testing purpose. * Verifier does sub-register def/use analysis and identifies instructions whose @@ -1102,10 +1104,10 @@ enum bpf_link_type { * Then, if verifier is not doing correct analysis, such randomization will * regress tests to expose bugs. */ -#define BPF_F_TEST_RND_HI32 (1U << 2) + BPF_F_TEST_RND_HI32 = (1U << 2), /* The verifier internal test flag. Behavior is undefined */ -#define BPF_F_TEST_STATE_FREQ (1U << 3) + BPF_F_TEST_STATE_FREQ = (1U << 3), /* If BPF_F_SLEEPABLE is used in BPF_PROG_LOAD command, the verifier will * restrict map and helper usage for such programs. Sleepable BPF programs can @@ -1113,12 +1115,13 @@ enum bpf_link_type { * Such programs are allowed to use helpers that may sleep like * bpf_copy_from_user(). */ -#define BPF_F_SLEEPABLE (1U << 4) + BPF_F_SLEEPABLE = (1U << 4), /* If BPF_F_XDP_HAS_FRAGS is used in BPF_PROG_LOAD command, the loaded program * fully support xdp frags. */ -#define BPF_F_XDP_HAS_FRAGS (1U << 5) + BPF_F_XDP_HAS_FRAGS = (1U << 5), +}; /* link_create.kprobe_multi.flags used in LINK_CREATE command for * BPF_TRACE_KPROBE_MULTI attach type to create return probe. From patchwork Sat Mar 19 17:30:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786261 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2D1EC433EF for ; Sat, 19 Mar 2022 17:31:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243689AbiCSRct (ORCPT ); Sat, 19 Mar 2022 13:32:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243717AbiCSRcl (ORCPT ); Sat, 19 Mar 2022 13:32:41 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52754457B1; Sat, 19 Mar 2022 10:31:03 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id o13so7338313pgc.12; Sat, 19 Mar 2022 10:31:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cJVyZjeOEvx3Lfsyy6KEMt0pbjay35w3CRFKsCROdt0=; b=OK4rLjPj7lNzt5BEEa9L1Z4QKJn9pYIcC9ajSWth0exTWw1+1u7BCl8D0VakqT6IAU XiqRdOH1SNNrB82Q/huswOP4Djic+w+i/ATCAF0iFRKnuYzWMSlmLeCFj+p8H0mkCwll uGaN1fq/gOyLCrQc4QFUX6yg3FDa/+Tm8Duqp/jK7xhrm8G0YJkeUdwydfI0Hi7h21o+ 1MJA/7VNowNk3kjz8Dy3jqxyDhKGoWNqga1/bSRDlahTOhvLyJeVxSAv1CRN/qd7W+8a umByxC08emMOptRp0tA0lz9BeVjkJFX2rlDaErusVMQqbeg1dGyzSRL9RLgIOBdgGOsk fFGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cJVyZjeOEvx3Lfsyy6KEMt0pbjay35w3CRFKsCROdt0=; b=0WgSZSfd7Q+RZ3juNq/VVRMe7XxC4aqt4z7JcIYl+VXs6D7VHjA18VZMkBMCzmtP1a HGm3Ddi4EScJp2UDzOV2FI3HxEivOt7NhaGsfDBKD6JUplmvZMfp2KjC07jEawV23kiJ wrKiQln+pFcf5V5bFIpRjloNcx5/r0lgkKzu7ikCeck79sODl7GvYCovsbmBhR8SMLJO xoEnD8YCGMvFHc8srHV+OUhTAqNrpE/3jLiU1IucExHTWzuySOnQ2pSWVKeaVf+0YRLZ 8Iae0PSSSyM081Y5PE9J60r8lZBApoFIA+zePxxQjjHY9On5/vH7nG1BriUCelnYNs34 3WTw== X-Gm-Message-State: AOAM532kEcy7t/uIctCnvWkYAcl30UsZLe6HLYKfw28gOyODD09LgMIm HQJsoeNdP3FrP3YXtjZ5QwEwIoLrd2zJEpYVgbQ= X-Google-Smtp-Source: ABdhPJwmiilJbVdPYG4/QB253/81b+H5bYLkd9Pn7OxLgGgpM8MfUoP7wNhIy4RR8R9itochanvoZA== X-Received: by 2002:a62:684:0:b0:4f7:803:d1b0 with SMTP id 126-20020a620684000000b004f70803d1b0mr16231429pfg.10.1647711051443; Sat, 19 Mar 2022 10:30:51 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:50 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 09/14] bpf: Add no charge flag for bpf prog Date: Sat, 19 Mar 2022 17:30:31 +0000 Message-Id: <20220319173036.23352-10-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net A no charge flag is also introduced for bpf prog, which is similar with the no charge flag introduced for bpf map. The usecase of it is the same too. It is added in bpf_attr for BPF_PROG_LOAD command, and then set in bpf_prog_aux for the memory allocation which is not at the loading path. There're 4B holes after the member max_rdwr_access in struct bpf_prog_aux, so we can place the new member there. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 8 ++++++++ include/uapi/linux/bpf.h | 3 +++ kernel/bpf/syscall.c | 4 +++- tools/include/uapi/linux/bpf.h | 3 +++ 4 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 90a542d5a411..69ff3e35b8f2 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -214,6 +214,13 @@ map_flags_no_charge(gfp_t flags, union bpf_attr *attr) return flags |= (attr->map_flags & BPF_F_NO_CHARGE) ? 0 : __GFP_ACCOUNT; } +static inline gfp_t +prog_flags_no_charge(gfp_t flags, union bpf_attr *attr) +{ + return flags |= (attr->prog_flags & BPF_F_PROG_NO_CHARGE) ? + 0 : __GFP_ACCOUNT; +} + static inline gfp_t bpf_flags_no_charge(gfp_t flags, bool no_charge) { @@ -958,6 +965,7 @@ struct bpf_prog_aux { u32 ctx_arg_info_size; u32 max_rdonly_access; u32 max_rdwr_access; + bool no_charge; /* dont' charge memory to memcg */ struct btf *attach_btf; const struct bpf_ctx_arg_aux *ctx_arg_info; struct mutex dst_mutex; /* protects dst_* pointers below, *after* prog becomes visible */ diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 93ee04fb8c62..3c98b1b77db6 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1121,6 +1121,9 @@ enum { * fully support xdp frags. */ BPF_F_XDP_HAS_FRAGS = (1U << 5), + +/* Don't charge memory to memcg */ + BPF_F_PROG_NO_CHARGE = (1U << 6), }; /* link_create.kprobe_multi.flags used in LINK_CREATE command for diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index e84aeefa05f4..346f3df9fa1d 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2230,7 +2230,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) BPF_F_TEST_STATE_FREQ | BPF_F_SLEEPABLE | BPF_F_TEST_RND_HI32 | - BPF_F_XDP_HAS_FRAGS)) + BPF_F_XDP_HAS_FRAGS | + BPF_F_PROG_NO_CHARGE)) return -EINVAL; if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && @@ -2317,6 +2318,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) prog->aux->offload_requested = !!attr->prog_ifindex; prog->aux->sleepable = attr->prog_flags & BPF_F_SLEEPABLE; prog->aux->xdp_has_frags = attr->prog_flags & BPF_F_XDP_HAS_FRAGS; + prog->aux->no_charge = attr->prog_flags & BPF_F_PROG_NO_CHARGE; err = security_bpf_prog_alloc(prog->aux); if (err) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 71a4d8fdc880..89752e5c11c0 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1121,6 +1121,9 @@ enum { * fully support xdp frags. */ BPF_F_XDP_HAS_FRAGS = (1U << 5), + +/* Don't charge memory to memcg */ + BPF_F_PROG_NO_CHARGE = (1U << 6), }; /* link_create.kprobe_multi.flags used in LINK_CREATE command for From patchwork Sat Mar 19 17:30:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786254 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65EE9C4321E for ; Sat, 19 Mar 2022 17:31:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243706AbiCSRcY (ORCPT ); Sat, 19 Mar 2022 13:32:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243683AbiCSRcV (ORCPT ); Sat, 19 Mar 2022 13:32:21 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC52845ADC; Sat, 19 Mar 2022 10:30:53 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id b8so9842126pjb.4; Sat, 19 Mar 2022 10:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qHQu4l0DNA359MKDX3wgS7bak31X8wuVL1Qxy9vcN30=; b=aVlQw/X30t3guhPlzexjfi4ZuV1qG2ftDBMaM2cxYxqwAi3vbYD6Qvz749eSnnmN3I dcjSrNHmbXjAqWtWH9tlty2IMJT5ilANT557Z9fyWcqL3VhW8p+AWj02tlbZx2cPrTVs b1ipu9ey3X/BQRhLYasrQJrjczm7Qpmz3cCvBHfN8DtM/l5g0NO1OCC7+VT+o5NJ9HcD FW2ZjEo7o8iZXK9cbz6wNQ+H4kITaPS7gbBuVDOIm5JLoDEbKsZMAthE037au1hACQBN kATbhDZARdTdhdL0FTjfcGN0oNMFTSTSZqrbMuUJc6K4XFgMwQEKQ8PAzDUSE+aqN1U1 XiRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qHQu4l0DNA359MKDX3wgS7bak31X8wuVL1Qxy9vcN30=; b=1kKUSuEIBEri2WuLOsrtKm710NPF3xUsbNafiTV+CVzZPsLsQ8x+2uH3cttHl32XPk 4CrsIYelW1Qv1VI1Tt1By59hAnxoTcDGaDlZEkFkbAffkZWPWOEc59ZSkM3c4rrG8V04 yj9oSDSSvY8Y9o7JUqI+e6iCS3esG1gIGnijtRySEDJ3s4qLYs7u3Y67cc3/83nTwQAY 7UmT+//Q+W3DGhCFpWyXOPnUrCMOWzjg+Ucs4k2TdEMUvbegKQH/oUaYU3SCcAhJ/NRS 38QXWlRxNME9kJFm637ONboXRjANDcdjYnGbz1ohEKBWtaGIbN0VU+M9iWDH5xkoIF7j 6X2Q== X-Gm-Message-State: AOAM532CQb/VFS5CefiXp/3uO8Xozw16/QOditIROZz79oGiXuJVQtYA tEoIBWdEfglqWmtehVnMfw95sZkR4r1somi7sgU= X-Google-Smtp-Source: ABdhPJxbI/bKQUsvIS+pdtOIBg2Ba5EngaSH06pOcvVbNR2cQU79iWIhoSG40JbwcKqwIDkohnV3Ig== X-Received: by 2002:a17:903:230c:b0:151:93d0:5608 with SMTP id d12-20020a170903230c00b0015193d05608mr5013516plh.167.1647711053379; Sat, 19 Mar 2022 10:30:53 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:52 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 10/14] bpf: Only sys admin can set no charge flag for bpf prog Date: Sat, 19 Mar 2022 17:30:32 +0000 Message-Id: <20220319173036.23352-11-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net When a bpf prog is loaded by a proccess running in a container (with memcg), only sys admin has privilege not to charge bpf prog memory into this container while account it to root memcg only. Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 346f3df9fa1d..ecc5de216f50 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2234,6 +2234,9 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) BPF_F_PROG_NO_CHARGE)) return -EINVAL; + if (attr->prog_flags & BPF_F_PROG_NO_CHARGE && !capable(CAP_SYS_ADMIN)) + return -EPERM; + if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && (attr->prog_flags & BPF_F_ANY_ALIGNMENT) && !bpf_capable()) From patchwork Sat Mar 19 17:30:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786257 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5535C433FE for ; Sat, 19 Mar 2022 17:31:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243733AbiCSRcm (ORCPT ); Sat, 19 Mar 2022 13:32:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243685AbiCSRcV (ORCPT ); Sat, 19 Mar 2022 13:32:21 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3237945AC7; Sat, 19 Mar 2022 10:30:55 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id m22so9903224pja.0; Sat, 19 Mar 2022 10:30:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qIBKd9AzBGUka0TmRgPVm4pjAnkawvLN160pSTlVjSE=; b=H7xrz3r2dp+CX42pfgNSHej70Eh5wyqJphEaiZENqivoqpjc79U2gclxAcbGZjAIuv 5Y8O43UrciWrCCUOt9KdHlNRyc7rN93C35SjixmxJkeZTjQSuk37xYncjmBD8p5AonVb W0iC2+dkxSjHqTAz/6HcOLTUKErUS5ywoGoGpwT22lpNjCsG2hGxXE8kFtyvbPTaKBcd SRhUek8dmnMSH7uDxqj7bxvX9gQzD/xnzZUPZ3K+XxOE9tpG6YSujT3IzuSFX2uLlf1D vPfkW8wFitJHolvspFnk7phCrR7Viy+p3PF3JfEdeU8XP4AVlet8rQ7/Y4GOrOdJm0Iq QMJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qIBKd9AzBGUka0TmRgPVm4pjAnkawvLN160pSTlVjSE=; b=PvBrBkPeeq6JkF+RiCZekGVP5s3fMmro2cqJZl8F0oF19Av3UfgX29qxsfLwvE4LHs nNZyzR/tl/dMk/UYz0fWh0WmYNGg9pqreJdac5TDJAQ6ABUv9stWknYmbBCaj8ZkqD+9 XX8BDgPCt5fBwxkyodcvcRYDFyXvVzyOrEAcQXyeao982bDKPKL6IXoo81uxb+bvySBC 6XcJGeJeL3VaBeeMSqder2IRzrbv/NK4ejjPt0VlJ/jUlbzSV/K4opvIHxlVh+DchAxc fmbCq+/wYUqDBh2NkuhJ9A3bx02+/HADszL/uT7IH859EdAG11m6tmDp4TT6TfJ+wICk CAUg== X-Gm-Message-State: AOAM532sddZ0M4A6CFZb7eYx8M8ITY/Ckf7fQJ65EDsh4WNXMANN8VqO 70z52UHQqo4/0sWN4zy7jsQ= X-Google-Smtp-Source: ABdhPJy073V4vKdfGyENPJElILxuQy13uwmc0gTYcOqVJ0M4/j1gSgN4unBpWn9RRK1DossTTufgnA== X-Received: by 2002:a17:903:32c4:b0:151:c6ae:e24b with SMTP id i4-20020a17090332c400b00151c6aee24bmr5195960plr.85.1647711054660; Sat, 19 Mar 2022 10:30:54 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:54 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 11/14] bpf: Set __GFP_ACCOUNT at the callsite of bpf_prog_alloc Date: Sat, 19 Mar 2022 17:30:33 +0000 Message-Id: <20220319173036.23352-12-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Instead of setting __GFP_ACCOUNT inside bpf_prog_alloc(), let's set it at the callsite. No functional change. It is a preparation for followup patch. Signed-off-by: Yafang Shao --- kernel/bpf/core.c | 8 ++++---- kernel/bpf/syscall.c | 2 +- kernel/bpf/verifier.c | 2 +- net/core/filter.c | 6 +++--- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 1324f9523e7c..0f68b8203c18 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -80,7 +80,7 @@ void *bpf_internal_load_pointer_neg_helper(const struct sk_buff *skb, int k, uns struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flags) { - gfp_t gfp_flags = GFP_KERNEL_ACCOUNT | __GFP_ZERO | gfp_extra_flags; + gfp_t gfp_flags = __GFP_ZERO | gfp_extra_flags; struct bpf_prog_aux *aux; struct bpf_prog *fp; @@ -89,12 +89,12 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flag if (fp == NULL) return NULL; - aux = kzalloc(sizeof(*aux), GFP_KERNEL_ACCOUNT | gfp_extra_flags); + aux = kzalloc(sizeof(*aux), gfp_extra_flags); if (aux == NULL) { vfree(fp); return NULL; } - fp->active = alloc_percpu_gfp(int, GFP_KERNEL_ACCOUNT | gfp_extra_flags); + fp->active = alloc_percpu_gfp(int, gfp_flags); if (!fp->active) { vfree(fp); kfree(aux); @@ -116,7 +116,7 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flag struct bpf_prog *bpf_prog_alloc(unsigned int size, gfp_t gfp_extra_flags) { - gfp_t gfp_flags = GFP_KERNEL_ACCOUNT | __GFP_ZERO | gfp_extra_flags; + gfp_t gfp_flags = __GFP_ZERO | gfp_extra_flags; struct bpf_prog *prog; int cpu; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index ecc5de216f50..fdfbb4d0d5e0 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2305,7 +2305,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) } /* plain bpf_prog allocation */ - prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER); + prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER | __GFP_ACCOUNT); if (!prog) { if (dst_prog) bpf_prog_put(dst_prog); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 0287176bfe9a..fe989cc08391 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12991,7 +12991,7 @@ static int jit_subprogs(struct bpf_verifier_env *env) * subprogs don't have IDs and not reachable via prog_get_next_id * func[i]->stats will never be accessed and stays NULL */ - func[i] = bpf_prog_alloc_no_stats(bpf_prog_size(len), GFP_USER); + func[i] = bpf_prog_alloc_no_stats(bpf_prog_size(len), GFP_USER | __GFP_ACCOUNT); if (!func[i]) goto out_free; memcpy(func[i]->insnsi, &prog->insnsi[subprog_start], diff --git a/net/core/filter.c b/net/core/filter.c index 03655f2074ae..6466a1e0ed4d 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -1363,7 +1363,7 @@ int bpf_prog_create(struct bpf_prog **pfp, struct sock_fprog_kern *fprog) if (!bpf_check_basics_ok(fprog->filter, fprog->len)) return -EINVAL; - fp = bpf_prog_alloc(bpf_prog_size(fprog->len), 0); + fp = bpf_prog_alloc(bpf_prog_size(fprog->len), __GFP_ACCOUNT); if (!fp) return -ENOMEM; @@ -1410,7 +1410,7 @@ int bpf_prog_create_from_user(struct bpf_prog **pfp, struct sock_fprog *fprog, if (!bpf_check_basics_ok(fprog->filter, fprog->len)) return -EINVAL; - fp = bpf_prog_alloc(bpf_prog_size(fprog->len), 0); + fp = bpf_prog_alloc(bpf_prog_size(fprog->len), __GFP_ACCOUNT); if (!fp) return -ENOMEM; @@ -1488,7 +1488,7 @@ struct bpf_prog *__get_filter(struct sock_fprog *fprog, struct sock *sk) if (!bpf_check_basics_ok(fprog->filter, fprog->len)) return ERR_PTR(-EINVAL); - prog = bpf_prog_alloc(bpf_prog_size(fprog->len), 0); + prog = bpf_prog_alloc(bpf_prog_size(fprog->len), __GFP_ACCOUNT); if (!prog) return ERR_PTR(-ENOMEM); From patchwork Sat Mar 19 17:30:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786258 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1866BC433F5 for ; Sat, 19 Mar 2022 17:31:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243753AbiCSRcn (ORCPT ); Sat, 19 Mar 2022 13:32:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243686AbiCSRcV (ORCPT ); Sat, 19 Mar 2022 13:32:21 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AEFB46141; Sat, 19 Mar 2022 10:30:56 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id t22so9504713plo.0; Sat, 19 Mar 2022 10:30:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UnzpRSS/vLYZGmFXrB6nYaZPHuwQlW9x/vmkhMBVTEQ=; b=RsczAP8C3Ig4HKDvm6X86NOreFdEhMYHBpRBoeKF/94wlV9+u9Egr4V/TBqS1FX8Ue lFFPQpIMJS4G5lpfiEw4KvUr8o5fqRs/XJsZMcZPpxldyQLPMhxPur699k75MNBgyaQ5 4dS5qDSS0tsiNXWZjsakb9ETsLt+JEePQaPATpMIr3Z/+Z86EFP3EGI9zrujz75CkY4V ahN24nCYzn1MoHhVLuq5xcKyvqaej5JUbg49KNmekfTtZuNF73XEQW4iYhSAIFbHtXCu LVoMb7GX+KBO8hWssyjZ8SSU8gCSwnDRoVlaBZRSYuVFbQtS176El/2RnVtm0bGcXaqC U0VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UnzpRSS/vLYZGmFXrB6nYaZPHuwQlW9x/vmkhMBVTEQ=; b=xLhZ7OQ4E6dLm+gjHuIVZ2MS49DrlOAVmjIgXa1A96EXuTQMTvebZH4s0481xqTIZu Or9v6PerpPiTa4ZBYDutUrTG+zoflmqi9+KpILbleBtqfy52Q8cZrvAZHsfZvUn7x9ef MvIjltl/m2RaNV/Kcy33DtLgxe2UPR0D5FIW0Nol8OB5LmroNVITfNNaVu4jZQw/UnOe DJe4ng0XU3pSns8S3Bmy8EV3eFYKSaAm6YY0+xNvYuUSH7kqDazdTLP21gJ4EyOFYHTZ KSjUm+AJw2kCDqVldmT2MAL+djaViqmhuCeTVL/6meo94dHmTaWpWhqTGosOsdzb8HX/ mhKA== X-Gm-Message-State: AOAM532H6H9vW6Ab0EturzBedYh3O25Z0cr2tQpSE8ejF5BqcOciPdFI 9+DBsIQiUBEc+xuMkoYpFVQ= X-Google-Smtp-Source: ABdhPJzDsThUZW6p6KlCe6SKrOblLrhRM+o0ogBcfGoBqkKzlliLEkvGKbHYLlwwOKXRc+RIyMwfZw== X-Received: by 2002:a17:903:246:b0:153:87f0:a93e with SMTP id j6-20020a170903024600b0015387f0a93emr5130014plh.171.1647711055893; Sat, 19 Mar 2022 10:30:55 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:55 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 12/14] bpf: Allow no charge for bpf prog Date: Sat, 19 Mar 2022 17:30:34 +0000 Message-Id: <20220319173036.23352-13-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow not to charge memory used by bpf progs. This includes the memory used by bpf progs itself, auxxiliary data, statistics and bpf line info. Signed-off-by: Yafang Shao --- kernel/bpf/core.c | 12 ++++++++++-- kernel/bpf/syscall.c | 6 ++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 0f68b8203c18..7aa750e8bd7d 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -144,12 +144,17 @@ EXPORT_SYMBOL_GPL(bpf_prog_alloc); int bpf_prog_alloc_jited_linfo(struct bpf_prog *prog) { + gfp_t gfp_flags = GFP_KERNEL | __GFP_NOWARN; + if (!prog->aux->nr_linfo || !prog->jit_requested) return 0; + if (!prog->aux->no_charge) + gfp_flags |= __GFP_ACCOUNT; + prog->aux->jited_linfo = kvcalloc(prog->aux->nr_linfo, sizeof(*prog->aux->jited_linfo), - GFP_KERNEL_ACCOUNT | __GFP_NOWARN); + gfp_flags); if (!prog->aux->jited_linfo) return -ENOMEM; @@ -224,7 +229,7 @@ void bpf_prog_fill_jited_linfo(struct bpf_prog *prog, struct bpf_prog *bpf_prog_realloc(struct bpf_prog *fp_old, unsigned int size, gfp_t gfp_extra_flags) { - gfp_t gfp_flags = GFP_KERNEL_ACCOUNT | __GFP_ZERO | gfp_extra_flags; + gfp_t gfp_flags = GFP_KERNEL | __GFP_ZERO | gfp_extra_flags; struct bpf_prog *fp; u32 pages; @@ -233,6 +238,9 @@ struct bpf_prog *bpf_prog_realloc(struct bpf_prog *fp_old, unsigned int size, if (pages <= fp_old->pages) return fp_old; + if (!fp_old->aux->no_charge) + gfp_flags |= __GFP_ACCOUNT; + fp = __vmalloc(size, gfp_flags); if (fp) { memcpy(fp, fp_old, fp_old->pages * PAGE_SIZE); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index fdfbb4d0d5e0..e386b549fafc 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2218,9 +2218,10 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) enum bpf_prog_type type = attr->prog_type; struct bpf_prog *prog, *dst_prog = NULL; struct btf *attach_btf = NULL; - int err; + gfp_t gfp_flags = GFP_USER; char license[128]; bool is_gpl; + int err; if (CHECK_ATTR(BPF_PROG_LOAD)) return -EINVAL; @@ -2305,7 +2306,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) } /* plain bpf_prog allocation */ - prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER | __GFP_ACCOUNT); + prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), + prog_flags_no_charge(gfp_flags, attr)); if (!prog) { if (dst_prog) bpf_prog_put(dst_prog); From patchwork Sat Mar 19 17:30:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786259 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E61FC433EF for ; Sat, 19 Mar 2022 17:31:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243769AbiCSRcp (ORCPT ); Sat, 19 Mar 2022 13:32:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243692AbiCSRcV (ORCPT ); Sat, 19 Mar 2022 13:32:21 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93FB946664; Sat, 19 Mar 2022 10:30:57 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id mj15-20020a17090b368f00b001c637aa358eso13051871pjb.0; Sat, 19 Mar 2022 10:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZbZuWBU2sc499QxzBGg0bo+ElRsdQRuA/wKM02GiSt0=; b=JDzhcgxLaOZ0V5gQyGN26pppLmf+IEka5KXyc+sYktuWOVw1iyjcmYwMhfau6fSgW0 fr6qpqXP4nz1I9Wi3AdFDsrYzUoKQ5rcPKtdleHoqnbiEA1BR6lY5jMhpxNh/DZJLx1D pJqny+cak/5s+YheGaxZCKKrSId/jE5qRPEjX8k2Tsw4yLAiheUuhGnXs4JQDY/do6N4 qD8kHpB00mbFSTf7EW7F2iGL8zO5+tc6YGyRebwiSnoNQLpxHnD9c17jbvdeE8S9rXFQ gECgFIUCTgYZav51XtHHKH3EI32BVcZ6u1LuyVTOByOIkBHO0ac7oQxNpfi5+1K0agaS IdJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZbZuWBU2sc499QxzBGg0bo+ElRsdQRuA/wKM02GiSt0=; b=az9JXMl7RWECzVh8MG4/gdJDAdREzkgGWssiFEW51oi/REl67+09nfvWWQzGVlxPXY oVrVo1FdDDXk61vz432KH9oKCpo4km7cvhIzcplcj3qCWog8M4RHI4bwg1y8t1L82UoR d0JRbLB4zZE60tHsgAUB1IabwfZpQjIjlxo2fh/ouTPYG4Xxsgr63dBwx6uq/FMPuoJT CjcBy0NejG+uscm3h4NjRsu7x1FwbQrmfNSbRqONM5kJseD+p/TZ7r4WBGnbrbPMXG1t ch6dd97JXCSw4H/+5w7rJ78adnO4mAHMptL0KMJl9OR99cdqG00NLb0pS0sGxuErhmtA Q8pQ== X-Gm-Message-State: AOAM532bsDYBSp31dU+cjO22bmjQ02HkfSCqkxgQWDi76EHt8F/OVxBP lL1bIHPlbd/XNPVmLDlsHxg= X-Google-Smtp-Source: ABdhPJxy2a+rHVqh1VHAdTv82D7TgvdaSm91ZkM3za+b6cl7Kt8elH7cuNexrX827Jw5FearElHdtg== X-Received: by 2002:a17:902:8f96:b0:153:62bb:c4a3 with SMTP id z22-20020a1709028f9600b0015362bbc4a3mr5131018plo.154.1647711057087; Sat, 19 Mar 2022 10:30:57 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:56 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 13/14] bpf: selftests: Add test case for BPF_F_NO_CHARTE Date: Sat, 19 Mar 2022 17:30:35 +0000 Message-Id: <20220319173036.23352-14-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net BPF_F_NO_CHARTE test case for various maps. Below is the result, $ ./test_maps ... test_no_charge:PASS ... Signed-off-by: Yafang Shao --- .../selftests/bpf/map_tests/no_charg.c | 79 +++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100644 tools/testing/selftests/bpf/map_tests/no_charg.c diff --git a/tools/testing/selftests/bpf/map_tests/no_charg.c b/tools/testing/selftests/bpf/map_tests/no_charg.c new file mode 100644 index 000000000000..db18685a53f7 --- /dev/null +++ b/tools/testing/selftests/bpf/map_tests/no_charg.c @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include +#include +#include +#include + +#include + +struct map_attr { + __u32 map_type; + char *map_name; + __u32 key_size; + __u32 value_size; + __u32 max_entries; + __u32 map_flags; +}; + +static struct map_attr attrs[] = { + {BPF_MAP_TYPE_HASH, "BPF_MAP_TYPE_HASH", 4, 4, 10}, + {BPF_MAP_TYPE_ARRAY, "BPF_MAP_TYPE_ARRAY", 4, 4, 10}, + {BPF_MAP_TYPE_PROG_ARRAY, "BPF_MAP_TYPE_PROG_ARRAY", 4, 4, 10}, + {BPF_MAP_TYPE_PERF_EVENT_ARRAY, "BPF_MAP_TYPE_PERF_EVENT_ARRAY", 4, 4, 10}, + {BPF_MAP_TYPE_PERCPU_HASH, "BPF_MAP_TYPE_PERCPU_HASH", 4, 4, 10}, + {BPF_MAP_TYPE_PERCPU_ARRAY, "BPF_MAP_TYPE_PERCPU_ARRAY", 4, 4, 10}, + {BPF_MAP_TYPE_STACK_TRACE, "BPF_MAP_TYPE_STACK_TRACE", 4, 8, 10}, + {BPF_MAP_TYPE_CGROUP_ARRAY, "BPF_MAP_TYPE_CGROUP_ARRAY", 4, 4, 10}, + {BPF_MAP_TYPE_LRU_HASH, "BPF_MAP_TYPE_LRU_HASH", 4, 4, 10}, + {BPF_MAP_TYPE_LRU_PERCPU_HASH, "BPF_MAP_TYPE_LRU_PERCPU_HASH", 4, 4, 10}, + {BPF_MAP_TYPE_LPM_TRIE, "BPF_MAP_TYPE_LPM_TRIE", 32, 4, 10, BPF_F_NO_PREALLOC}, + {BPF_MAP_TYPE_DEVMAP, "BPF_MAP_TYPE_DEVMAP", 4, 4, 10}, + {BPF_MAP_TYPE_SOCKMAP, "BPF_MAP_TYPE_SOCKMAP", 4, 4, 10}, + {BPF_MAP_TYPE_CPUMAP, "BPF_MAP_TYPE_CPUMAP", 4, 4, 10}, + {BPF_MAP_TYPE_XSKMAP, "BPF_MAP_TYPE_XSKMAP", 4, 4, 10}, + {BPF_MAP_TYPE_SOCKHASH, "BPF_MAP_TYPE_SOCKHASH", 4, 4, 10}, + {BPF_MAP_TYPE_REUSEPORT_SOCKARRAY, "BPF_MAP_TYPE_REUSEPORT_SOCKARRAY", 4, 4, 10}, + {BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE, "BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE", 8, 4, 0}, + {BPF_MAP_TYPE_QUEUE, "BPF_MAP_TYPE_QUEUE", 0, 4, 10}, + {BPF_MAP_TYPE_DEVMAP_HASH, "BPF_MAP_TYPE_DEVMAP_HASH", 4, 4, 10}, + {BPF_MAP_TYPE_RINGBUF, "BPF_MAP_TYPE_RINGBUF", 0, 0, 4096}, + {BPF_MAP_TYPE_BLOOM_FILTER, "BPF_MAP_TYPE_BLOOM_FILTER", 0, 4, 10}, +}; + +static __u32 flags[] = { + BPF_F_NO_CHARGE, +}; + +void test_map_flags(union bpf_attr *attr, char *name) +{ + int mfd; + + mfd = syscall(SYS_bpf, BPF_MAP_CREATE, attr, sizeof(*attr)); + CHECK(mfd <= 0 && mfd != -EPERM, "no_charge", "%s error: %s\n", + name, strerror(errno)); + + if (mfd > 0) + close(mfd); +} + +void test_no_charge(void) +{ + union bpf_attr attr; + int i, j; + + memset(&attr, 0, sizeof(attr)); + for (i = 0; i < sizeof(flags) / sizeof(__u32); i++) { + for (j = 0; j < sizeof(attrs) / sizeof(struct map_attr); j++) { + attr.map_type = attrs[j].map_type; + attr.key_size = attrs[j].key_size; + attr.value_size = attrs[j].value_size; + attr.max_entries = attrs[j].max_entries; + attr.map_flags = attrs[j].map_flags | flags[i]; + test_map_flags(&attr, attrs[j].map_name); + } + } + + printf("%s:PASS\n", __func__); +} From patchwork Sat Mar 19 17:30:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12786260 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A43EC433EF for ; Sat, 19 Mar 2022 17:31:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243784AbiCSRcr (ORCPT ); Sat, 19 Mar 2022 13:32:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243702AbiCSRcW (ORCPT ); Sat, 19 Mar 2022 13:32:22 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9AC646B04; Sat, 19 Mar 2022 10:30:58 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id mz9-20020a17090b378900b001c657559290so10657015pjb.2; Sat, 19 Mar 2022 10:30:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gNHwzX1XXzyuZEkRQLK5ljL7FKM2KTEIhvt/+nJ9w5k=; b=cWsSBz1wcJLbkEhvadWb1z178lDBINSMILOr0KYVKwhDbun9djaH9TCmc3V/X47pym DuDLtLZW9Mw1cTrnsjsKmlH7ZvuFPFkJ4rsarGoModU082cK1VA+dDJ+GNIc5CwoKx34 wvQQTZSYyrSVI7T6FfY1aefOUuwaZmC5wiCWxKZqWxV9RepJVsKZh34D2LgSsyFjKZuK BUthR6o8mnnz61PINJHyIrwp2uDJw5Xecw1vVn5dz30SLOTpiWnp0p0ojMl2ldLivLyo awA5jMvuzOhSmBdGifSsHR9p9ndyhKd83qk3MlNvtLjcTPqaODL9fz9n1vXPSd4moFCf +X/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gNHwzX1XXzyuZEkRQLK5ljL7FKM2KTEIhvt/+nJ9w5k=; b=XfqoWp/LEYVCUwVdRZ2WhIQ+Oe72Bao3Y5nw/uWn9wt5DQg37rXBw4HCYoiwAXQbeJ BdjOFxOaVUMdu+HvTOVe+YYbrKHX1umguWqnKcbzgJIiSeR7eGWkfE02SH3qdv97fbso yhcWy9uBR5/A3srgOvB2HvaKAFhW5Qi8KhvaQHZwhlUPCaRCzvKrxIxLdFQTVh5ulBsu x3FGU9WUOpuUy/HtDg/0wbAPLKuLSEgyyJawO8HM2oBWCpaaEJ8DSqibsxgFfdinHQAO /kdno53tPsg0EomsNp+0HDearF+h/eFraJH3BCfLHtS62FN2bH65P9V55df+o7qXTx17 cE3Q== X-Gm-Message-State: AOAM532GiV0mk9FaiBL1V2+9fnUIB8tBHshL6wI/ohMViHWoL5P5Q6VF RUHtTYgPhR5Y0KFSIj1jfS8= X-Google-Smtp-Source: ABdhPJx0WWiyXY4tSlfUjYz1EUtkAZiBSBQOTjQ9TIbe9waIxqXbbGnwv+SSon9nwqQjPI9mhc08Og== X-Received: by 2002:a17:90a:7305:b0:1bd:6972:faf5 with SMTP id m5-20020a17090a730500b001bd6972faf5mr17517556pjk.131.1647711058332; Sat, 19 Mar 2022 10:30:58 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:4ab8:5400:3ff:fee9:a154]) by smtp.gmail.com with ESMTPSA id k21-20020aa788d5000000b004f71bff2893sm12722136pff.67.2022.03.19.10.30.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Mar 2022 10:30:57 -0700 (PDT) From: Yafang Shao To: roman.gushchin@linux.dev, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, shuah@kernel.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, Yafang Shao Subject: [PATCH 14/14] bpf: selftests: Add test case for BPF_F_PROG_NO_CHARGE Date: Sat, 19 Mar 2022 17:30:36 +0000 Message-Id: <20220319173036.23352-15-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220319173036.23352-1-laoar.shao@gmail.com> References: <20220319173036.23352-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Test case to check if BPF_F_PROG_NO_CHARGE valid. The result as follows, $ ./test_progs ... #103 no_charge:OK ... Signed-off-by: Yafang Shao --- .../selftests/bpf/prog_tests/no_charge.c | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/no_charge.c diff --git a/tools/testing/selftests/bpf/prog_tests/no_charge.c b/tools/testing/selftests/bpf/prog_tests/no_charge.c new file mode 100644 index 000000000000..85fbd2ccbc71 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/no_charge.c @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include + +#include "test_progs.h" + +#define BPF_ALU64_IMM(OP, DST, IMM) \ + ((struct bpf_insn) { \ + .code = BPF_ALU64 | BPF_OP(OP) | BPF_K, \ + .dst_reg = DST, \ + .src_reg = 0, \ + .off = 0, \ + .imm = IMM }) + +#define BPF_EXIT_INSN() \ + ((struct bpf_insn) { \ + .code = BPF_JMP | BPF_EXIT, \ + .dst_reg = 0, \ + .src_reg = 0, \ + .off = 0, \ + .imm = 0 }) + +void test_no_charge(void) +{ + struct bpf_insn prog[] = { + BPF_ALU64_IMM(BPF_MOV, BPF_REG_0, 0), + BPF_EXIT_INSN(), + }; + union bpf_attr attr; + int duration = 0; + int fd; + + bzero(&attr, sizeof(attr)); + attr.prog_type = BPF_PROG_TYPE_SCHED_CLS; + attr.insn_cnt = 2; + attr.insns = (__u64)prog; + attr.license = (__u64)("GPL"); + attr.prog_flags |= BPF_F_PROG_NO_CHARGE; + + fd = syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr)); + CHECK(fd < 0 && fd != -EPERM, "no_charge", "error: %s\n", + strerror(errno)); + + if (fd > 0) + close(fd); +}