From patchwork Tue Mar 8 13:10:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773788 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18991C433EF for ; Tue, 8 Mar 2022 13:11:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DCCA8D0005; Tue, 8 Mar 2022 08:11:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 78CBE8D0001; Tue, 8 Mar 2022 08:11:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 655028D0005; Tue, 8 Mar 2022 08:11:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0198.hostedemail.com [216.40.44.198]) by kanga.kvack.org (Postfix) with ESMTP id 578B18D0001 for ; Tue, 8 Mar 2022 08:11:16 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0EC46A5A6F for ; Tue, 8 Mar 2022 13:11:16 +0000 (UTC) X-FDA: 79221254952.27.87D7DD0 Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf22.hostedemail.com (Postfix) with ESMTP id 98384C000A for ; Tue, 8 Mar 2022 13:11:15 +0000 (UTC) Received: by mail-pj1-f48.google.com with SMTP id mg21-20020a17090b371500b001bef9e4657cso2231306pjb.0 for ; Tue, 08 Mar 2022 05:11:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XZ5/JOgeRQIev0F6iGnllvaw2Xpd0Vkw5vT4hctm9LE=; b=MTzlU8i+d5WWYrMR/nWgGDPhI0Em4aOWdkZfDtdNOw00rdj00RUcnaXU6V2LyUUEZs uytZymJYyJKFq5tGJQgjprgAXFsu5QRkBF6OWo/euJ16VdTVnjZkGDnyqLTR0lyrZSBw h5Arb4qABcrL/BaNajXPaUMfzI0qkN+KfnpEmZf2rvVifTSkWAL04xDLWGsrz8lc8yW8 ObYqT2lgYVecoJi2s3+Ein2cLceICEFBpFBM5B0xC/pXtTx84q823bHG2PZMUn7MZC5s pETcwE5r9uCYDYvA6rwRvvrqxfKH9ePkVqlQ/SyzRW5YaU2C80F/hj29fFRayl00Txkb H72Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XZ5/JOgeRQIev0F6iGnllvaw2Xpd0Vkw5vT4hctm9LE=; b=wzq6fVxHAWOYKbvQNpJmTcjqjajYhceMiLbj3n6fKS7ti6+upjMR9vfQpigaFd23im 0rD3l/msnAGAFEWilvjAS4d2hVjRxxkz7kCuUHhzlqMv6ZGagratmM5LYsXsCeyquCIe A9HZKh7Rvo0N5r6B/lKHX1x2dTAH8K3vBaCecxh76LDmQXL0fG86q3vQxk7+sVJu8/63 yPO5S3mpHQbx3ZvM41efuahjmN4P9AU+qIh0QY1T0Km3An3oM6nj10Bcmza7mi70ncQz cb7c/ggDrfai+j4ZNXSppy4wkttPFXRs1g1TzzGiQaE0PTgck+k7ykAVVLJW6j9syLBK 7V5g== X-Gm-Message-State: AOAM531W8kgLhv/jhsKN/kE8TQ/Zzz7mYhuiMQ3OZGk/k/sqnB/CI65H sWJtbQd6N8et0+S2nQUI19M= X-Google-Smtp-Source: ABdhPJzP43L1ogB3N4skFJGzR1Jg8A1fcMH9gZ1HrLSlGaD86J9Ijh+wfhfk/iUZ4nBV95hjZmnXAA== X-Received: by 2002:a17:90a:6508:b0:1be:d59c:1f10 with SMTP id i8-20020a17090a650800b001bed59c1f10mr4518554pjj.229.1646745074658; Tue, 08 Mar 2022 05:11:14 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:14 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao , Joanne Koong Subject: [PATCH RFC 1/9] bpftool: fix print error when show bpf man Date: Tue, 8 Mar 2022 13:10:48 +0000 Message-Id: <20220308131056.6732-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 98384C000A X-Stat-Signature: ycd1opz7motrun5dgw816t6aifount1a Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MTzlU8i+; spf=pass (imf22.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1646745075-693693 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If there is no btf_id or frozen, it will not show the pids, but the pids doesn't depends on any one of them. Fixes: 9330986c0300 ("bpf: Add bloom filter map implementation") Signed-off-by: Yafang Shao Cc: Joanne Koong --- tools/bpf/bpftool/map.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index e746642..0bba337 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -620,17 +620,14 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info) u32_as_hash_field(info->id)) printf("\n\tpinned %s", (char *)entry->value); } - printf("\n"); if (frozen_str) { frozen = atoi(frozen_str); free(frozen_str); } - if (!info->btf_id && !frozen) - return 0; - - printf("\t"); + if (info->btf_id || frozen) + printf("\n\t"); if (info->btf_id) printf("btf_id %d", info->btf_id); From patchwork Tue Mar 8 13:10:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93740C433FE for ; Tue, 8 Mar 2022 13:11:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1284D8D0006; Tue, 8 Mar 2022 08:11:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FFC08D0001; Tue, 8 Mar 2022 08:11:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E921E8D0006; Tue, 8 Mar 2022 08:11:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id D73158D0001 for ; Tue, 8 Mar 2022 08:11:17 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id AA4EE6064D for ; Tue, 8 Mar 2022 13:11:17 +0000 (UTC) X-FDA: 79221254994.08.5F62903 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf20.hostedemail.com (Postfix) with ESMTP id 3BCDA1C000D for ; Tue, 8 Mar 2022 13:11:17 +0000 (UTC) Received: by mail-pj1-f45.google.com with SMTP id m22so17169877pja.0 for ; Tue, 08 Mar 2022 05:11:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7qC9ZJnuyUGolXYQcUY/Tf83TQ1ETOj72VlDmZ6p6Ws=; b=bbq65S48n9rZTrLzmf2FQhuouyrA3lYX5oOGHOk8bQ0lmGrPYhUO3lK998y9GjF9bu vCvJg7DISbQI18AnpD6LwKkqHVPzuAawwcKFxpQBY7Jci46AOIY27abot/Y7c0pUmleb sSIvOx+rD9KQ+NLDELdCNfBxcG6Qadgfg55KmuFg9UK6S/oZbf5oExnqnphyWYxQVIoS jR5tuUxa1B1WNKBTpwq5YAq2N3gU0+gQWQfDZEXc0n5mrrV/O6xvBRth6wLThNOjgQ1E /S8xUomQXGSIxjE/689sHKrC/06BzAG4JA7+MSvXUqThXLw7VNa0D6TLyxTKTWgMWzJF 1huQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7qC9ZJnuyUGolXYQcUY/Tf83TQ1ETOj72VlDmZ6p6Ws=; b=CkMqnd9mzk2k7jToBVd9lAwPScDMrichjAudBMmnVdA6uYYcK3/QGsrW21EGMVXhKL kjrZ+fc1xAEWu066fBBffxr0gLK664egF2eNUJqu12yXSRSQfgoqaIixNfisMFLUE4kf 6H+zlwa5t/SLNB47tUFm6s6QUzuFQWRih7/MgfHf3/cu1njBC2kS5K5bPTxrdZRPEdGe AfDbDBaHKaw6J1yjqR7trfi0Yz5LjY/Y/BnbhSyqI3co5FctlQJ53fmYqWlmCMcGLOly LaLo0ss7si5UAuJV0XJNJ+uqvciCHWJHtQV0rW8J0He3XurBgZFkDkUdNNIEdQaaKccj JWeA== X-Gm-Message-State: AOAM530v8I/w9zXB5mq9RKYCwFWsVOfu/R6/AZ1nlaGiwlGGktaUcSRT iuUR6CchlecVYS+K9jOicQ0= X-Google-Smtp-Source: ABdhPJxRT96hUCFt+OLq3AvoUmcskuUoVsObUNWdtEx8FWULrshXYxLLY3NlZtrK0hmbq5kssZXcTg== X-Received: by 2002:a17:90b:1809:b0:1bf:7e9:bdf5 with SMTP id lw9-20020a17090b180900b001bf07e9bdf5mr4642772pjb.52.1646745076176; Tue, 08 Mar 2022 05:11:16 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:15 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 2/9] bpftool: show memcg info of bpf map Date: Tue, 8 Mar 2022 13:10:49 +0000 Message-Id: <20220308131056.6732-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3BCDA1C000D X-Stat-Signature: hbcgc5szipt6w5k6665ywfd1zhxrxiy9 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=bbq65S48; spf=pass (imf20.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.45 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1646745077-730281 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bpf map can be charged to a memcg, so we'd better show the memcg info to do better bpf memory management. This patch adds a new field "memcg_state" to show whether a bpf map is charged to a memcg and whether the memcg is offlined. Currently it has three values, 0 : not charged -1 : the charged memcg is offline 1 : the charged memcg is online For instance, $ bpftool map show 2: array name iterator.rodata flags 0x480 key 4B value 98B max_entries 1 memlock 4096B btf_id 240 frozen memcg_state 0 3: hash name calico_failsafe flags 0x1 key 4B value 1B max_entries 65535 memlock 524288B memcg_state 1 6: lru_hash name access_record flags 0x0 key 8B value 24B max_entries 102400 memlock 3276800B btf_id 256 memcg_state -1 Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 11 +++++++++++ tools/bpf/bpftool/map.c | 2 ++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 15 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 4eebea8..a448b06 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -5864,6 +5864,7 @@ struct bpf_map_info { __u32 btf_value_type_id; __u32 :32; /* alignment pad */ __u64 map_extra; + __s8 memcg_state; } __attribute__((aligned(8))); struct bpf_btf_info { diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index db402eb..3b50fcb 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3939,6 +3939,17 @@ static int bpf_map_get_info_by_fd(struct file *file, } info.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id; +#ifdef CONFIG_MEMCG_KMEM + if (map->memcg) { + struct mem_cgroup *memcg = map->memcg; + + if (memcg == root_mem_cgroup) + info.memcg_state = 0; + else + info.memcg_state = memcg->kmemcg_id < 0 ? -1 : 1; + } +#endif + if (bpf_map_is_dev_bound(map)) { err = bpf_map_offload_info_fill(&info, map); if (err) diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index 0bba337..fe8322f 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -550,6 +550,7 @@ static int show_map_close_json(int fd, struct bpf_map_info *info) jsonw_end_array(json_wtr); } + jsonw_int_field(json_wtr, "memcg_state", info->memcg_state); emit_obj_refs_json(refs_table, info->id, json_wtr); jsonw_end_object(json_wtr); @@ -635,6 +636,7 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info) if (frozen) printf("%sfrozen", info->btf_id ? " " : ""); + printf("\n\tmemcg_state %d", info->memcg_state); emit_obj_refs_plain(refs_table, info->id, "\n\tpids "); printf("\n"); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 4eebea8..41e65b3 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -5864,6 +5864,7 @@ struct bpf_map_info { __u32 btf_value_type_id; __u32 :32; /* alignment pad */ __u64 map_extra; + __s8 memcg_state; } __attribute__((aligned(8))); struct bpf_btf_info { From patchwork Tue Mar 8 13:10:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773790 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1BE7C433EF for ; Tue, 8 Mar 2022 13:11:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 967A28D0007; Tue, 8 Mar 2022 08:11:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 89FE38D0001; Tue, 8 Mar 2022 08:11:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CA208D0007; Tue, 8 Mar 2022 08:11:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 5DB198D0001 for ; Tue, 8 Mar 2022 08:11:19 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2FB21248BB for ; Tue, 8 Mar 2022 13:11:19 +0000 (UTC) X-FDA: 79221255078.08.5B2FB31 Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by imf15.hostedemail.com (Postfix) with ESMTP id 9ECB7A000A for ; Tue, 8 Mar 2022 13:11:18 +0000 (UTC) Received: by mail-pg1-f169.google.com with SMTP id t14so16437896pgr.3 for ; Tue, 08 Mar 2022 05:11:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BdKzZoxcCHUk05nIKgEhaGedb/vHkDzRSP6HrqUzWkw=; b=mVG368BLiMOTpvYHAUUF4Hl0SvELdZmvTum03FPpeGhXzLHWktVasfzvvXjRTjOz8v OeRQAhrdpy137X55NmR+Q/Kuzq9xiVfXNfSvJp32pazR2MPbIlJ8Vzb0Bd7V7G/na2UQ /kfik/XTfzvUUpTBTd1qB8DMG2kpuKdtEfRyYh14NVfqPCAXvfHKDJ+64jnLsvC2UHCd Q3sUYvjb4wA56QqTFnZWhIKlsDgOcCCK5NzY5cMPJAHGwN6RbCmGB1z9nZM4JR4/0Zny plLD1LQTq3acxLCjuPV6GmozEA9rAtVuiBPQKKhaxmdk4ZZHaK0851qNleWCQ6+8DyHd neGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BdKzZoxcCHUk05nIKgEhaGedb/vHkDzRSP6HrqUzWkw=; b=NKjwMrri7/EaHVmKk6e/JMv/WEZDeR2R0RyZLz3c4E3JN7B+k4844tTE4ujyBdV1Nj E2LCnVgGLBULe3GSRKwWDr+2UFslxtLH9QbG/uMvXYaT+EnOx5rKPazyJbbsQtozevGd dxF1UtXppjXLIo7CjyaVeXZryN5IYE0TeKqiJ9H2x9hre+swYSFuXgaUTjaXZc6/vNnW zqBpf5L9v6ASSsid0sBImzWZOPb+3LvrXys8l7Do6t0u4YNKiRpUJtlafxp/6ATZzzfW We5WnFvi1z5ziwDtkDnaFZ7LoJzGtcZDOmhqwqa0yh2IpG4TXw4D7cs6U/cgvCeTb1VX 0FFg== X-Gm-Message-State: AOAM531XoDwszq/HAk8+1u7Lw3QtroVpk2e6im4pfTBcjJrWHRgbSJ/V V4n7OYSTkllB3zWAEeltS0U= X-Google-Smtp-Source: ABdhPJxi1T75LgLTxomuRmkgtIa5sVE1omWzFmGT1bjc4dtbt/cmnHAw3/PxWVByPMfUm079lFsD0Q== X-Received: by 2002:a63:61d3:0:b0:37f:f9c5:bbb8 with SMTP id v202-20020a6361d3000000b0037ff9c5bbb8mr13005281pgb.332.1646745077687; Tue, 08 Mar 2022 05:11:17 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:17 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 3/9] mm: add methord to charge kmalloc-ed address Date: Tue, 8 Mar 2022 13:10:50 +0000 Message-Id: <20220308131056.6732-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9ECB7A000A X-Stat-Signature: wc4o9tskyamzdntz8w75zbwjzo4rrmc4 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=mVG368BL; spf=pass (imf15.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-HE-Tag: 1646745078-980822 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch implements a methord to charge or uncharge related pages or objects from a given kmalloc-ed address. It is similar to kfree, except that it doesn't touch the pages or objects while does account only. Signed-off-by: Yafang Shao --- include/linux/slab.h | 1 + mm/slab.c | 6 ++++++ mm/slob.c | 6 ++++++ mm/slub.c | 32 ++++++++++++++++++++++++++++++++ 4 files changed, 45 insertions(+) diff --git a/include/linux/slab.h b/include/linux/slab.h index 5b6193f..ae82e23 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -182,6 +182,7 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name, void * __must_check krealloc(const void *objp, size_t new_size, gfp_t flags) __alloc_size(2); void kfree(const void *objp); void kfree_sensitive(const void *objp); +void kcharge(const void *objp, bool charge); size_t __ksize(const void *objp); size_t ksize(const void *objp); #ifdef CONFIG_PRINTK diff --git a/mm/slab.c b/mm/slab.c index ddf5737..fbff613 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3796,6 +3796,12 @@ void kfree(const void *objp) } EXPORT_SYMBOL(kfree); +void kcharge(const void *objp, bool charge) +{ + /* Not implemented yet */ +} +EXPORT_SYMBOL(kfree); + /* * This initializes kmem_cache_node or resizes various caches for all nodes. */ diff --git a/mm/slob.c b/mm/slob.c index 60c5842..d3a789f 100644 --- a/mm/slob.c +++ b/mm/slob.c @@ -569,6 +569,12 @@ void kfree(const void *block) } EXPORT_SYMBOL(kfree); +void kcharge(const void *block, bool charge) +{ + /* not implemented yet. */ +} +EXPORT_SYMBOL(kcharge); + /* can't use ksize for kmem_cache_alloc memory, only kmalloc */ size_t __ksize(const void *block) { diff --git a/mm/slub.c b/mm/slub.c index 2614740..e933d45 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4563,6 +4563,38 @@ void kfree(const void *x) } EXPORT_SYMBOL(kfree); +void kcharge(const void *x, bool charge) +{ + void *object = (void *)x; + struct folio *folio; + struct slab *slab; + + WARN_ON(!in_task()); + + if (unlikely(ZERO_OR_NULL_PTR(x))) + return; + + folio = virt_to_folio(x); + if (unlikely(!folio_test_slab(folio))) { + unsigned int order = folio_order(folio); + int sign = charge ? 1 : -1; + + mod_lruvec_page_state(folio_page(folio, 0), NR_SLAB_UNRECLAIMABLE_B, + sign * (PAGE_SIZE << order)); + + return; + } + + slab = folio_slab(folio); + if (charge) + memcg_slab_post_alloc_hook(slab->slab_cache, + get_obj_cgroup_from_current(), GFP_KERNEL, 1, &object); + else + memcg_slab_free_hook(slab->slab_cache, &object, 1); + +} +EXPORT_SYMBOL(kcharge); + #define SHRINK_PROMOTE_MAX 32 /* From patchwork Tue Mar 8 13:10:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06571C433F5 for ; Tue, 8 Mar 2022 13:11:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE6118D0008; Tue, 8 Mar 2022 08:11:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E94B68D0001; Tue, 8 Mar 2022 08:11:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0BA88D0008; Tue, 8 Mar 2022 08:11:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id C3CE68D0001 for ; Tue, 8 Mar 2022 08:11:20 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9EA01248BB for ; Tue, 8 Mar 2022 13:11:20 +0000 (UTC) X-FDA: 79221255120.05.FFEF707 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf21.hostedemail.com (Postfix) with ESMTP id 2D6F51C0012 for ; Tue, 8 Mar 2022 13:11:19 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id z16so17294183pfh.3 for ; Tue, 08 Mar 2022 05:11:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z98OQ6AdalG5oMDKVv5aA3FO3T1l3cK2nxdIAVKfKFQ=; b=YdbOfZYq1mOC7LrS/ckJnfEBQ+MAblMP+dx7AWe6RKa98jAA9r1n44yEApW3RKGMd5 zRC97/VTsM7EsdnHMDO7hE6yzmYjY+E4jfZQL8n8DTe/cfFkXDN5zun4HeZTuag2VUiI 1uVbJRLZgyBFHI3LI09GiKz5UK1PMeMrNKAI4k6YNc4E1md3TxXvNZpzuFtYIbfkIFOJ i797KhWbBX2INTNoR3rd6zalgu7CvEkKZbVypUJ8AUSn5KpA5CEVOMo06zFbjaj2nm8t BELs1qacF+x+Y92yqHN5epNG5LlOKvH5ZD7MfFAEnqr8Z3Ff6F3wXJTKxppyuwulT8mY r62Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z98OQ6AdalG5oMDKVv5aA3FO3T1l3cK2nxdIAVKfKFQ=; b=OunBFut7I9vhCf4kaTblQR7RwJgT5AP5ZxA5gTl96D4bCnwiMf9lCv6ywjxfpq6PDh yXFwJhPkZ6cbdEBWeUE0uk/om/VqjLscd1bbPRd1rXi0Yg40VySjYsqXNFOAOtRw3w9I giEvAOR+E/MCn3Emlit3nIujLt3umy7RrBlhGEU4Ml57KEai92eQ6jCv722MUWLgEVvv uLcLOdgDy2Tv0wEnY1IqH0KD3QiYtAOAhctNUqFd6csHVUP0vfzhxlJhtPj4oidttrQu F0wt8NuN66aODz/74mL0H0i1VJw2zn9frTHAQLNAh8arQPpXKnh/mPHNK9tCO7g6Fkmq x51A== X-Gm-Message-State: AOAM531tKxb6koGTkdVonU/dfahYKOQ1vKn6GrkaOScaOgsPXyrH9mRl q7SqviEyFUZj/CepfWyl9nQ= X-Google-Smtp-Source: ABdhPJxrM0tl1/gYxlXQIsaOe+AjwvJr57dfg0zKANngu7XWWfwFpew1AJotBmNiE7RcN7WhSYaheA== X-Received: by 2002:a63:b30d:0:b0:378:c5ee:afda with SMTP id i13-20020a63b30d000000b00378c5eeafdamr14297243pgf.385.1646745079189; Tue, 08 Mar 2022 05:11:19 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:18 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 4/9] mm: add methord to charge vmalloc-ed address Date: Tue, 8 Mar 2022 13:10:51 +0000 Message-Id: <20220308131056.6732-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2D6F51C0012 X-Stat-Signature: ojub6535pk4pdug7si7grx37wxt5owj5 X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YdbOfZYq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Server: rspam03 X-HE-Tag: 1646745079-40586 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch adds a methord to charge or uncharge a given vmalloc-ed address. It is similar to vfree, except that it doesn't touch the related pages while does account only. Signed-off-by: Yafang Shao --- include/linux/slab.h | 1 + include/linux/vmalloc.h | 1 + mm/util.c | 9 +++++++++ mm/vmalloc.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 40 insertions(+) diff --git a/include/linux/slab.h b/include/linux/slab.h index ae82e23..7173354 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -759,6 +759,7 @@ extern void *kvrealloc(const void *p, size_t oldsize, size_t newsize, gfp_t flag __alloc_size(3); extern void kvfree(const void *addr); extern void kvfree_sensitive(const void *addr, size_t len); +void kvcharge(const void *addr, bool charge); unsigned int kmem_cache_size(struct kmem_cache *s); void __init kmem_cache_init_late(void); diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 880227b..b48d941 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -161,6 +161,7 @@ void *__vmalloc_node(unsigned long size, unsigned long align, gfp_t gfp_mask, extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); +void vcharge(const void *addr, bool charge); extern void *vmap(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot); diff --git a/mm/util.c b/mm/util.c index 7e433690..f5f5e05 100644 --- a/mm/util.c +++ b/mm/util.c @@ -614,6 +614,15 @@ void kvfree(const void *addr) } EXPORT_SYMBOL(kvfree); +void kvcharge(const void *addr, bool charge) +{ + if (is_vmalloc_addr(addr)) + vcharge(addr, charge); + else + kcharge(addr, charge); +} +EXPORT_SYMBOL(kvcharge); + /** * kvfree_sensitive - Free a data object containing sensitive information. * @addr: address of the data object to be freed. diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 4165304..6fc2295 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2715,6 +2715,35 @@ void vfree(const void *addr) } EXPORT_SYMBOL(vfree); +void vcharge(const void *addr, bool charge) +{ + unsigned int page_order; + struct vm_struct *area; + int i; + + WARN_ON(!in_task()); + + if (!addr) + return; + + area = find_vm_area(addr); + if (unlikely(!area)) + return; + + page_order = vm_area_page_order(area); + for (i = 0; i < area->nr_pages; i += 1U << page_order) { + struct page *page = area->pages[i]; + + WARN_ON(!page); + if (charge) + memcg_kmem_charge_page(page, GFP_KERNEL, page_order); + else + memcg_kmem_uncharge_page(page, page_order); + cond_resched(); + } +} +EXPORT_SYMBOL(vcharge); + /** * vunmap - release virtual mapping obtained by vmap() * @addr: memory base address From patchwork Tue Mar 8 13:10:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 777ABC433F5 for ; Tue, 8 Mar 2022 13:11:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1F3C8D0009; Tue, 8 Mar 2022 08:11:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DCF978D0001; Tue, 8 Mar 2022 08:11:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C23748D0009; Tue, 8 Mar 2022 08:11:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id B593B8D0001 for ; Tue, 8 Mar 2022 08:11:22 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 73FA8972E6 for ; Tue, 8 Mar 2022 13:11:22 +0000 (UTC) X-FDA: 79221255204.23.0BE169B Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf30.hostedemail.com (Postfix) with ESMTP id E057C80003 for ; Tue, 8 Mar 2022 13:11:21 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id c16-20020a17090aa61000b001befad2bfaaso2370579pjq.1 for ; Tue, 08 Mar 2022 05:11:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6X23AJzOcszEUYA+Oc8WYhr7w/7aeKWJdCFlxFdfbxc=; b=QzDOs9yQq9R1CZFP5kH/i37OmN2DriXgh+Uc29/28MK52/n2BNL5T+LpOEtCkDH1wT SArng9TL/tXBv+Z80bt9GxJnPhLPtlJphCtIvRp0gdbabeClqLN5OJJWkW3w2ZTfWwQa Rnjfw7mAFLLj9wTWH+Eh6SOwOz0vOv1IZ2Ez9Hdj3Lib06/AoNebKC9gEtTDUdNrv4NQ 9DcZx2G8SWEzn6bao5aEaUSzUNDhlZZ1d3NO0Rldf6bomrFxx1CFs1Xa9YnCHp8zH9iB t5155kTpOmXdPuCwsVILsy5c1hsx6nSVXhy04JPx1GbVRQYFwe+KylhvR1CA514F2xZd 3C/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6X23AJzOcszEUYA+Oc8WYhr7w/7aeKWJdCFlxFdfbxc=; b=8QsrUQwmFpCp8JQeVgtzHi3xI4cAIn+5lOs/KQNzcLolcuUusb23jW85OlfN95rPwZ ReWQYtlMi9s4i/375OznuP2s99JFNBi1m2fv0XvSPtjRpwiO9I7A6f5w+fNCbc5qvugR ZDzzMz1uEvyfYoJgieAM+qjG5DUAKV1VGZiahXSG4zrANAI/+8dIf/funK/9cuW9bGmG yl/pMCY/nK82I8Yo0R6WT0Lur/XBks3mX7vsjQDW4s3m2Y1gkNefR4gcdNTxsKjwd2FO 45PETF3U5OUTv40bjjxNoo1qeW1zM7PQDkE5CGbKYsWgtphz4irpg+fN6GW9jjcajELl oBrg== X-Gm-Message-State: AOAM530N59GRVDnWk41nMIudkPEzSz4R+gLj9HLAwfHnzDe1HPALmsaa 7DOGLf7F91A2b26Hvo+DLW4= X-Google-Smtp-Source: ABdhPJxKOq3e10aQ1RmNw8ArFeArmumS9bRFdOU8RkbwmtWaG0AV0PF5C3HqteolvWr6aJIng6XTbQ== X-Received: by 2002:a17:90a:4fa2:b0:1bd:383a:2b6b with SMTP id q31-20020a17090a4fa200b001bd383a2b6bmr4727275pjh.108.1646745080990; Tue, 08 Mar 2022 05:11:20 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:20 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 5/9] mm: add methord to charge percpu address Date: Tue, 8 Mar 2022 13:10:52 +0000 Message-Id: <20220308131056.6732-6-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: E057C80003 X-Stat-Signature: itf56r139asywpec6gkkhkxr4t5qwrcg X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=QzDOs9yQ; spf=pass (imf30.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam07 X-HE-Tag: 1646745081-398803 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch adds a methord to charge or uncharge a percpu address. It is similar to free_percpu except that it doesn't touch the related pages while does account only. Signed-off-by: Yafang Shao --- include/linux/percpu.h | 1 + mm/percpu.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/include/linux/percpu.h b/include/linux/percpu.h index f1ec5ad..1a65221 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -128,6 +128,7 @@ extern int __init pcpu_page_first_chunk(size_t reserved_size, extern void __percpu *__alloc_percpu_gfp(size_t size, size_t align, gfp_t gfp) __alloc_size(1); extern void __percpu *__alloc_percpu(size_t size, size_t align) __alloc_size(1); extern void free_percpu(void __percpu *__pdata); +void charge_percpu(void __percpu *__pdata, bool charge); extern phys_addr_t per_cpu_ptr_to_phys(void *addr); #define alloc_percpu_gfp(type, gfp) \ diff --git a/mm/percpu.c b/mm/percpu.c index ea28db2..22fc0ff 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -2309,6 +2309,56 @@ void free_percpu(void __percpu *ptr) } EXPORT_SYMBOL_GPL(free_percpu); +void charge_percpu(void __percpu *ptr, bool charge) +{ + int bit_off, off, bits, size, end; + struct obj_cgroup *objcg; + struct pcpu_chunk *chunk; + unsigned long flags; + void *addr; + + WARN_ON(!in_task()); + + if (!ptr) + return; + + addr = __pcpu_ptr_to_addr(ptr); + spin_lock_irqsave(&pcpu_lock, flags); + chunk = pcpu_chunk_addr_search(addr); + off = addr - chunk->base_addr; + objcg = chunk->obj_cgroups[off >> PCPU_MIN_ALLOC_SHIFT]; + if (!objcg) { + spin_unlock_irqrestore(&pcpu_lock, flags); + return; + } + + bit_off = off / PCPU_MIN_ALLOC_SIZE; + /* find end index */ + end = find_next_bit(chunk->bound_map, pcpu_chunk_map_bits(chunk), + bit_off + 1); + bits = end - bit_off; + size = bits * PCPU_MIN_ALLOC_SIZE; + + if (charge) { + obj_cgroup_get(objcg); + obj_cgroup_charge(objcg, GFP_KERNEL, size * num_possible_cpus()); + rcu_read_lock(); + mod_memcg_state(obj_cgroup_memcg(objcg), MEMCG_PERCPU_B, + (size * num_possible_cpus())); + rcu_read_unlock(); + } else { + obj_cgroup_uncharge(objcg, size * num_possible_cpus()); + rcu_read_lock(); + mod_memcg_state(obj_cgroup_memcg(objcg), MEMCG_PERCPU_B, + -(size * num_possible_cpus())); + rcu_read_unlock(); + obj_cgroup_put(objcg); + } + + spin_unlock_irqrestore(&pcpu_lock, flags); +} +EXPORT_SYMBOL(charge_percpu); + bool __is_kernel_percpu_address(unsigned long addr, unsigned long *can_addr) { #ifdef CONFIG_SMP From patchwork Tue Mar 8 13:10:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0086DC433F5 for ; Tue, 8 Mar 2022 13:11:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A17F8D000A; Tue, 8 Mar 2022 08:11:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 828E28D0001; Tue, 8 Mar 2022 08:11:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 719CE8D000A; Tue, 8 Mar 2022 08:11:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 60E588D0001 for ; Tue, 8 Mar 2022 08:11:24 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4053E618DA for ; Tue, 8 Mar 2022 13:11:24 +0000 (UTC) X-FDA: 79221255288.09.B02843D Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf16.hostedemail.com (Postfix) with ESMTP id AC51B18000B for ; Tue, 8 Mar 2022 13:11:23 +0000 (UTC) Received: by mail-pl1-f180.google.com with SMTP id q11so17005224pln.11 for ; Tue, 08 Mar 2022 05:11:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3u3VsA3Fi5wDkVLbct/DeOE8XIO9r5bCMklz39tC+30=; b=d3xMfPnSj9DUQxDjtQLyi+itK0WX3JodAeZCw3xPBtkpGMEM+aD8bzj6wZwCzhWcHZ kt5w7qKQK3TtStKQ3oT5qOYPzVp+cM1eosOYmB4siu0vBKluYV91Do7HgnXDEzhnVvWj /WVDgwGVTkUWcEoOPTIW8t2SQAfmnpXJiVWedbMFSLl9wZeb1olWm/3CDgl9nqVXGaK7 1l6LV05YKWWBgXJBMIoPk8gJTEYnCZFusR5ixA0BtXVZJAfPp6gJtnzkg3aQPwjfKaqG 3o+ON2B1FO4YimaKwPZj1bBFt+nJn4+/9+rHZp7yv1OvuJ13QO4UOZGSb9B2++pmQ4vx 9ALg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3u3VsA3Fi5wDkVLbct/DeOE8XIO9r5bCMklz39tC+30=; b=JY9HWeuQxZE+GXBuPKPmsNF9oRYhA6hKImL69B+ybALd98XULi9E6kO68a8kd7CtZW gEUAmfALo8NjtrBVDrzg198ZhQ05h4a8xweiRDDBTaZvFOD3FMD/yzptCls8CG5AkICc 7evmDfHOzwNE8M0LDCNxOo6LLWsBPSYg2DgLqrnMNG4pn5E/DwGFnpwreBgpKJGd7ves OD5jHOx72eFoH831PguR/mlUXr3uBreh4r57N2c3eXoc+AL4n7ZdZC+fAwTY5z3Fvxrl B2rBiH0DUmYqC4NgGRf8Nzt+to0FHrStHnZ/ZHl/c7BR0vz+Xhq6BJdIJDNDWUbF0ET9 MFlw== X-Gm-Message-State: AOAM531RtQanfG9ESFF/AmazJX3FBLerdDtP6qXGfgYFL4GNCIpSTBv3 F7ZS6Ugb35+zhN93uNadPykFu486QyYg426/eaE= X-Google-Smtp-Source: ABdhPJzwC/En9lunCH8+wURsQQ1/FmGWLsc3MnOR1uRl1atGtzR7v2u8iVEpo0GJyAqsYIshW6NEaQ== X-Received: by 2002:a17:902:f68f:b0:151:fbf5:db45 with SMTP id l15-20020a170902f68f00b00151fbf5db45mr4536477plg.60.1646745082822; Tue, 08 Mar 2022 05:11:22 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:22 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 6/9] bpf: add a helper to find map by id Date: Tue, 8 Mar 2022 13:10:53 +0000 Message-Id: <20220308131056.6732-7-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: AC51B18000B X-Stat-Signature: 3hqq3wmkgudash5ndwsfedai5ug4i73k Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=d3xMfPnS; spf=pass (imf16.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-HE-Tag: 1646745083-167994 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A new helper bpf_map_idr_find() is introduced for later use. Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 3b50fcb..68fea3b 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3474,6 +3474,21 @@ static int bpf_prog_get_fd_by_id(const union bpf_attr *attr) return fd; } +static struct bpf_map *bpf_map_idr_find(unsigned long id) +{ + void *map; + + spin_lock_bh(&map_idr_lock); + map = idr_find(&map_idr, id); + if (map) + map = __bpf_map_inc_not_zero(map, true); + else + map = ERR_PTR(-ENOENT); + spin_unlock_bh(&map_idr_lock); + + return map; +} + #define BPF_MAP_GET_FD_BY_ID_LAST_FIELD open_flags static int bpf_map_get_fd_by_id(const union bpf_attr *attr) @@ -3494,14 +3509,7 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) if (f_flags < 0) return f_flags; - spin_lock_bh(&map_idr_lock); - map = idr_find(&map_idr, id); - if (map) - map = __bpf_map_inc_not_zero(map, true); - else - map = ERR_PTR(-ENOENT); - spin_unlock_bh(&map_idr_lock); - + map = bpf_map_idr_find(id); if (IS_ERR(map)) return PTR_ERR(map); From patchwork Tue Mar 8 13:10:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6F2BC433FE for ; Tue, 8 Mar 2022 13:11:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71F7E8D000C; Tue, 8 Mar 2022 08:11:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A47C8D0001; Tue, 8 Mar 2022 08:11:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 543C08D000C; Tue, 8 Mar 2022 08:11:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 459E98D0001 for ; Tue, 8 Mar 2022 08:11:28 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0A4E41A8F for ; Tue, 8 Mar 2022 13:11:28 +0000 (UTC) X-FDA: 79221255456.13.FF8DF4A Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf28.hostedemail.com (Postfix) with ESMTP id 6E6BAC0007 for ; Tue, 8 Mar 2022 13:11:27 +0000 (UTC) Received: by mail-pf1-f173.google.com with SMTP id s11so17253130pfu.13 for ; Tue, 08 Mar 2022 05:11:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3y4pmtErmcUQqTEOzusAX2pzTuUiyQeDCSeMBuC6VMQ=; b=SiVmiDuO/2M2Xzol8AGb7jc3+OznGw2J3+JiAFGq0IgPtRBs+B5WCzg555zf8AFxw/ SvYvYKlfzqYxdYqt6HLWTZDy8hkMR7VnTuVpC45UG+9hWjGQT82Z1lZGXZz5u5ZkI1Sb T8wWUD1Ign+8sEYvUjXqv3kCkXpo2pRUaADUOJC50otN7MQwMr6DhUs3+5lXu5EZyZlv 2hNDqCxhpUMTgCqyUWHe6LD6wxoxG2PbB4oUB14AjukT5kPNQVRsOtgTr1FmYzxJeIYq pq7ceeU7QUXEBoLpCRr4f2m0Wt8L3ogseVrdawWE9L5U3W4JXEjIhDfPwNpqxZEx+Kkp TdMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3y4pmtErmcUQqTEOzusAX2pzTuUiyQeDCSeMBuC6VMQ=; b=a48vnQ3q8tAwJQY6uQnWYi/CoEugIrEurnX9ijFT4aosgje1EA9wCa2uN26oIs9DkK 3T9085xM3+kbCCMby2tCHnpNVhEYAANU3CgNupynvsQc2A4221lrAb9arsaqa02nD7TQ Iso6QbY8KY8AOJt9Od3N1RD3H59l9s7+ny/qAD07hOwKDbl3GM+ocjMoHnfYEbfEzHK1 UXlRGF5W8USaDG3PvF9Fz+OsjKS5P21UYmjdFrSORgh7FWMM5tqU6vIeScy8hGYztIGG rTPBB9S8Ig4dInnVKRLKnwR64c8NRdAABEOvwtwqz8FWJpHhCTlJjPGpcQXHLcb0T/JI 2T5w== X-Gm-Message-State: AOAM532640hbIMa8hV+rEIENy5vlKWQQvwWuE6rQyxI2Kefc6gbwD8tS +fWaUX4Db70EIsvn7qXDNWk= X-Google-Smtp-Source: ABdhPJxozHq59VkUFv288fl2mix6RMNdONpReM8rlehkbq7w0YTK49AhZHAdBosK85EByFXP+tX5JQ== X-Received: by 2002:a63:d253:0:b0:380:3aee:8437 with SMTP id t19-20020a63d253000000b003803aee8437mr9002618pgi.382.1646745084332; Tue, 08 Mar 2022 05:11:24 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:23 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 7/9] bpf: add BPF_MAP_RECHARGE syscall Date: Tue, 8 Mar 2022 13:10:54 +0000 Message-Id: <20220308131056.6732-8-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6E6BAC0007 X-Stat-Signature: 3njedn16p6g5ndx96ez1r1749kyuxdss Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SiVmiDuO; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-HE-Tag: 1646745087-473895 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch adds a new bpf syscall BPF_MAP_RECHARGE, which means to recharge the allocated memory of a bpf map from an offline memcg to the current memcg. The recharge methord for each map will be implemented in the follow-up patches. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 9 +++++++++ kernel/bpf/syscall.c | 19 ++++++++++++++++++- 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 88449fb..fca274e 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -147,6 +147,8 @@ struct bpf_map_ops { bpf_callback_t callback_fn, void *callback_ctx, u64 flags); + bool (*map_recharge_memcg)(struct bpf_map *map); + /* BTF name and id of struct allocated by map_alloc */ const char * const map_btf_name; int *map_btf_id; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index a448b06..290ea67 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -821,6 +821,14 @@ struct bpf_cgroup_storage_key { * Returns zero on success. On error, -1 is returned and *errno* * is set appropriately. * + * BPF_MAP_RECHARGE + * Description + * Recharge bpf memory from an offline memcg + * + * Return + * Returns zero on success. On error, -1 is returned and *errno* + * is set appropriately. + * * NOTES * eBPF objects (maps and programs) can be shared between processes. * @@ -875,6 +883,7 @@ enum bpf_cmd { BPF_ITER_CREATE, BPF_LINK_DETACH, BPF_PROG_BIND_MAP, + BPF_MAP_RECHARGE, }; enum bpf_map_type { diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 68fea3b..85456f1 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1128,7 +1128,6 @@ static int map_lookup_elem(union bpf_attr *attr) return err; } - #define BPF_MAP_UPDATE_ELEM_LAST_FIELD flags static int map_update_elem(union bpf_attr *attr, bpfptr_t uattr) @@ -4621,6 +4620,21 @@ static int bpf_prog_bind_map(union bpf_attr *attr) return ret; } +static int map_recharge_elem(union bpf_attr *attr) +{ + int id = attr->map_id; + struct bpf_map *map; + + map = bpf_map_idr_find(id); + if (IS_ERR(map)) + return PTR_ERR(map); + + if (map->ops->map_recharge_memcg) + map->ops->map_recharge_memcg(map); + + return 0; +} + static int __sys_bpf(int cmd, bpfptr_t uattr, unsigned int size) { union bpf_attr attr; @@ -4757,6 +4771,9 @@ static int __sys_bpf(int cmd, bpfptr_t uattr, unsigned int size) case BPF_PROG_BIND_MAP: err = bpf_prog_bind_map(&attr); break; + case BPF_MAP_RECHARGE: + err = map_recharge_elem(&attr); + break; default: err = -EINVAL; break; From patchwork Tue Mar 8 13:10:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773794 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B91BC433FE for ; Tue, 8 Mar 2022 13:11:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03BDE8D000B; Tue, 8 Mar 2022 08:11:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F2F498D0001; Tue, 8 Mar 2022 08:11:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCF868D000B; Tue, 8 Mar 2022 08:11:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0025.hostedemail.com [216.40.44.25]) by kanga.kvack.org (Postfix) with ESMTP id CE1B78D0001 for ; Tue, 8 Mar 2022 08:11:27 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 90D911829201B for ; Tue, 8 Mar 2022 13:11:27 +0000 (UTC) X-FDA: 79221255414.17.635A86B Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf26.hostedemail.com (Postfix) with ESMTP id 19CAC14000F for ; Tue, 8 Mar 2022 13:11:26 +0000 (UTC) Received: by mail-pg1-f174.google.com with SMTP id z4so16408043pgh.12 for ; Tue, 08 Mar 2022 05:11:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lr/CFIp00pEF/BEp0dXxqdNW/AI45azIAndxyHO9fEc=; b=JB0q4IHcXTozLToE7J/d1F6oTslE6YdOVpdWGhqprivSKxigjke87oe7Q/SAuyYUeH WGJhQaBuZ3HVQClkwYFjPKSRzgp4emV2L1P9SSaiybKpW76EN96/bTPE61IvacYCCbPu 2tnyx9h8AlAdJPTWgHaw0h6UrUPrZ543f9QNEfkMR0fTECuDmx4tQ3zxfQqOJsCk8sVk DYKKS+Xgd/h2MU2mxDUrjhR88QnzZGpZMmDwkqIWWmQrDT1ib7gRT6txscX8hGM0D7a6 uqP4aOAJ9FLNxpLxK1WiRTAVv86fU6Cfs15ePd+M+iOUp7u+I0QzoTVPdqbLiMfI4jIW wpSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lr/CFIp00pEF/BEp0dXxqdNW/AI45azIAndxyHO9fEc=; b=JobsRQQXwl9+2aw3X0KE0VKGLRITxXIykim6Z6Qxl84TbIQax+aJlmDjiTrQ98j61u h1kAOEUYCcEBpXbCqOCuTaIgfZhJrBRfAZWZLLhE9v2SgTm/j0KnmeJ3x4aEELlxT22Z H3UGBPtvXEHvnHF9BnS3vIOzY/04b2UQkOlN+w9T2yk0w/P0p/xxhPYCepRMew1iFynu r5tmgrBMvY88ixBfWfBfe3rBHt7uqM/NMWUNTfP8sEAjDShRhW+OP+oxHtYBItI1lR6N J3xhwJFcxY2Mi7M/0FRzldpXhD4+q2JKbJm5b8EjyieyceQWB1k5mSbCVJe9FZelf5sY 442g== X-Gm-Message-State: AOAM532/EeN0kAaXow9N+bdUQbjHANUd5HdEzYKvqjavOvfdN+rHxTnF wActMlE78oZymya4ePzuY7M= X-Google-Smtp-Source: ABdhPJyOlCtmyFx3M63Qe/Lo+ROqOvjtLMDDMPcSoi5Quoxet+iI8DibJx/wB3rOKVChv0jQxhwF4w== X-Received: by 2002:a05:6a00:198f:b0:4f6:c0e4:207d with SMTP id d15-20020a056a00198f00b004f6c0e4207dmr17968238pfl.82.1646745086059; Tue, 08 Mar 2022 05:11:26 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:25 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 8/9] bpf: make bpf_map_{save, release}_memcg public Date: Tue, 8 Mar 2022 13:10:55 +0000 Message-Id: <20220308131056.6732-9-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 19CAC14000F X-Stat-Signature: 8387k5euixh5cs3wcco74xzqy7t7h6m3 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=JB0q4IHc; spf=pass (imf26.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1646745086-994736 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: These two helpers will be used in map specific files later. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 21 +++++++++++++++++++++ kernel/bpf/syscall.c | 19 ------------------- 2 files changed, 21 insertions(+), 19 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index fca274e..2f3f092 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -23,6 +23,7 @@ #include #include #include +#include struct bpf_verifier_env; struct bpf_verifier_log; @@ -209,6 +210,26 @@ struct bpf_map { } owner; }; +#ifdef CONFIG_MEMCG_KMEM +static inline void bpf_map_save_memcg(struct bpf_map *map) +{ + map->memcg = get_mem_cgroup_from_mm(current->mm); +} + +static inline void bpf_map_release_memcg(struct bpf_map *map) +{ + mem_cgroup_put(map->memcg); +} +#else +static inline void bpf_map_save_memcg(struct bpf_map *map) +{ +} + +static inline void bpf_map_release_memcg(struct bpf_map *map) +{ +} +#endif + static inline bool map_value_has_spin_lock(const struct bpf_map *map) { return map->spin_lock_off >= 0; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 85456f1..7b4cbe7 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -414,16 +414,6 @@ void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock) } #ifdef CONFIG_MEMCG_KMEM -static void bpf_map_save_memcg(struct bpf_map *map) -{ - map->memcg = get_mem_cgroup_from_mm(current->mm); -} - -static void bpf_map_release_memcg(struct bpf_map *map) -{ - mem_cgroup_put(map->memcg); -} - void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node) { @@ -461,15 +451,6 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, return ptr; } - -#else -static void bpf_map_save_memcg(struct bpf_map *map) -{ -} - -static void bpf_map_release_memcg(struct bpf_map *map) -{ -} #endif /* called from workqueue */ From patchwork Tue Mar 8 13:10:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 289CBC43219 for ; Tue, 8 Mar 2022 13:11:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5BA7B8D000D; Tue, 8 Mar 2022 08:11:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 56ADE8D0001; Tue, 8 Mar 2022 08:11:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40AC28D000D; Tue, 8 Mar 2022 08:11:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 334538D0001 for ; Tue, 8 Mar 2022 08:11:29 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 02236618DA for ; Tue, 8 Mar 2022 13:11:28 +0000 (UTC) X-FDA: 79221255498.12.0766810 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf09.hostedemail.com (Postfix) with ESMTP id 7A0F514000F for ; Tue, 8 Mar 2022 13:11:28 +0000 (UTC) Received: by mail-pf1-f179.google.com with SMTP id z15so17292663pfe.7 for ; Tue, 08 Mar 2022 05:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bgJJlFnbXaRCIAYL5SQZwRZJIW6MmhaiowbOp4O3M1Y=; b=BTNwJVlZtiizxAYf1GqUM418mm4KpUgPRAZTqG8USzvhC/zyC+HUN65jCkTRLcPE2y HC/PH3SHk9dp5WMPyxiDG5XVzxxSlmltP5UJbf0EkQf2S+Fe/dbRT+enxXvZFxiN82Ow wtRn7eF5edcKUjmFS8ZTggAfNGUZIjzpxJkhs9p29fupdSUaEbpaDnj9IFnIf9Oxvwsh ZyaZyxh5UI7oo1CvxNcIyaVnuS8Z/p8OJadCY6UL6fhmjwrhb0OmPoQ35tw6WiG7nTCm KzeezESmY6izJJ+jz5rSydmReEB289qVpYWBJ6imgVnyOlsLHG0l1gGjNvR2wWVYoxix t9DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bgJJlFnbXaRCIAYL5SQZwRZJIW6MmhaiowbOp4O3M1Y=; b=ToBnbHnuG9DFAsZos3Tal3e938EHmdZr3oSzhm6vLSC2jSWJJeWzcOpNJIL6qJgoOy 5I+QD6iD7KwrQhSMOiy1yddrRe7gWKuxuBwKFfbflDkuviS5/JZnYxjbG4qd9qUp36rA 9B2B5D0Yz1UwOR7axOQo2GUsRFuvXEesgkTNg3+6FSjuo9XkX2X93VBz0YKkIIUngBFq VIH8l0Cip09qcR1VT/RR/RfzWu/S9Be7+3la5DPo/6PyThfKnxloaVrBiIH1TnwTLJ0A 0kvEa29piDWLNX3DPIqBmlAR+d+T2cOpTyMHfC1ISCSdGLnX1tLa/zaiZScbOPFnmR+b XE+A== X-Gm-Message-State: AOAM533ETSoeMeFTYNavlIbKbz2vImZ487U8VNHbPX24A9DAN3Y8zkYT 5sWQbOfnumAhAgLx+zlzq3g= X-Google-Smtp-Source: ABdhPJxcfF7dzcGyqjiKeRaKXA6W9nGPA0uFJq/5jcaGXg7/O7wJRtUT++VsWLtNLmrzXp07AhJYEg== X-Received: by 2002:aa7:8882:0:b0:4df:7b9e:1ccb with SMTP id z2-20020aa78882000000b004df7b9e1ccbmr18056424pfe.41.1646745087617; Tue, 08 Mar 2022 05:11:27 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:27 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 9/9] bpf: support recharge for hash map Date: Tue, 8 Mar 2022 13:10:56 +0000 Message-Id: <20220308131056.6732-10-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 7A0F514000F X-Stat-Signature: okudnromynkdaecybo4ksk3aj8a56aex Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=BTNwJVlZ; spf=pass (imf09.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-HE-Tag: 1646745088-452581 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch supports recharge for hash map. We have already known how the hash map is allocated and freed, we can also know how to charge and uncharge the hash map. Firstly, we need to uncharge it from the old memcg, then charge it to the current memcg. The old memcg must be an offline memcg. Signed-off-by: Yafang Shao --- kernel/bpf/hashtab.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 6587796..4d103f1 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "percpu_freelist.h" #include "bpf_lru_list.h" #include "map_in_map.h" @@ -1466,6 +1467,36 @@ static void htab_map_free(struct bpf_map *map) kfree(htab); } +static bool htab_map_recharge_memcg(struct bpf_map *map) +{ + struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + struct mem_cgroup *old = map->memcg; + int i; + + if (!old) + return false; + + /* Only process offline memcg */ + if (old == root_mem_cgroup || old->kmemcg_id >= 0) + return false; + + bpf_map_release_memcg(map); + kcharge(htab, false); + kvcharge(htab->buckets, false); + charge_percpu(htab->extra_elems, false); + for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) + charge_percpu(htab->map_locked[i], false); + + kcharge(htab, true); + kvcharge(htab->buckets, true); + charge_percpu(htab->extra_elems, true); + for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) + charge_percpu(htab->map_locked[i], true); + bpf_map_save_memcg(map); + + return true; +} + static void htab_map_seq_show_elem(struct bpf_map *map, void *key, struct seq_file *m) { @@ -2111,6 +2142,7 @@ static int bpf_for_each_hash_elem(struct bpf_map *map, bpf_callback_t callback_f .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_release_uref = htab_map_free_timers, .map_lookup_elem = htab_map_lookup_elem, @@ -2133,6 +2165,7 @@ static int bpf_for_each_hash_elem(struct bpf_map *map, bpf_callback_t callback_f .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_release_uref = htab_map_free_timers, .map_lookup_elem = htab_lru_map_lookup_elem, @@ -2258,6 +2291,7 @@ static void htab_percpu_map_seq_show_elem(struct bpf_map *map, void *key, .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_percpu_map_lookup_elem, .map_lookup_and_delete_elem = htab_percpu_map_lookup_and_delete_elem, @@ -2278,6 +2312,7 @@ static void htab_percpu_map_seq_show_elem(struct bpf_map *map, void *key, .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_lru_percpu_map_lookup_elem, .map_lookup_and_delete_elem = htab_lru_percpu_map_lookup_and_delete_elem,