From patchwork Tue Mar 8 13:10:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773778 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C4B6C4332F for ; Tue, 8 Mar 2022 13:11:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241614AbiCHNMV (ORCPT ); Tue, 8 Mar 2022 08:12:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346781AbiCHNMR (ORCPT ); Tue, 8 Mar 2022 08:12:17 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3017F48884; Tue, 8 Mar 2022 05:11:15 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id mv5-20020a17090b198500b001bf2a039831so2148694pjb.5; Tue, 08 Mar 2022 05:11:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XZ5/JOgeRQIev0F6iGnllvaw2Xpd0Vkw5vT4hctm9LE=; b=MTzlU8i+d5WWYrMR/nWgGDPhI0Em4aOWdkZfDtdNOw00rdj00RUcnaXU6V2LyUUEZs uytZymJYyJKFq5tGJQgjprgAXFsu5QRkBF6OWo/euJ16VdTVnjZkGDnyqLTR0lyrZSBw h5Arb4qABcrL/BaNajXPaUMfzI0qkN+KfnpEmZf2rvVifTSkWAL04xDLWGsrz8lc8yW8 ObYqT2lgYVecoJi2s3+Ein2cLceICEFBpFBM5B0xC/pXtTx84q823bHG2PZMUn7MZC5s pETcwE5r9uCYDYvA6rwRvvrqxfKH9ePkVqlQ/SyzRW5YaU2C80F/hj29fFRayl00Txkb H72Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XZ5/JOgeRQIev0F6iGnllvaw2Xpd0Vkw5vT4hctm9LE=; b=fu1jrF2LbS1rJY0sQim6E9nf/b8B+aLoxDwlQsLRoCiFVriZVwNMOYH8QtgiJ0yQ+j Yie3ONS9wQ9NWbwa83ykaikOdoNmi/P1CKvyGroVgtWgOkcyJ1CmVxiASb/Nc6XZi4At +Yjg1aQkDj9Nso4V7QQmB9uee5VXR2wXvgLk6haOFAvAaB6RiHtBHVRj8Nc5JmeOiApw RjxqMkr+rV0dewUAu4Ckw2I4frjiiUQkyrUj413+zq4U2YnKXzzHzOme0LmL3NLanVdL vYa8vDijvr/HEeVDhVHe0wRVvRIITc+INAcP2UePnKmka+xYdsVoZPgrDmQNpbIiwf2e dbfQ== X-Gm-Message-State: AOAM5304b2nGV872V3hPdfGcvmtZSyGks0U2eUQdRAQXBlZ9LJxUvtiY LmziJNShjJicQMybGlBcwqc= X-Google-Smtp-Source: ABdhPJzP43L1ogB3N4skFJGzR1Jg8A1fcMH9gZ1HrLSlGaD86J9Ijh+wfhfk/iUZ4nBV95hjZmnXAA== X-Received: by 2002:a17:90a:6508:b0:1be:d59c:1f10 with SMTP id i8-20020a17090a650800b001bed59c1f10mr4518554pjj.229.1646745074658; Tue, 08 Mar 2022 05:11:14 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:14 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao , Joanne Koong Subject: [PATCH RFC 1/9] bpftool: fix print error when show bpf man Date: Tue, 8 Mar 2022 13:10:48 +0000 Message-Id: <20220308131056.6732-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC If there is no btf_id or frozen, it will not show the pids, but the pids doesn't depends on any one of them. Fixes: 9330986c0300 ("bpf: Add bloom filter map implementation") Signed-off-by: Yafang Shao Cc: Joanne Koong --- tools/bpf/bpftool/map.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index e746642..0bba337 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -620,17 +620,14 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info) u32_as_hash_field(info->id)) printf("\n\tpinned %s", (char *)entry->value); } - printf("\n"); if (frozen_str) { frozen = atoi(frozen_str); free(frozen_str); } - if (!info->btf_id && !frozen) - return 0; - - printf("\t"); + if (info->btf_id || frozen) + printf("\n\t"); if (info->btf_id) printf("btf_id %d", info->btf_id); From patchwork Tue Mar 8 13:10:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773779 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8659C433EF for ; Tue, 8 Mar 2022 13:11:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233480AbiCHNMX (ORCPT ); Tue, 8 Mar 2022 08:12:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347025AbiCHNMR (ORCPT ); Tue, 8 Mar 2022 08:12:17 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C21548890; Tue, 8 Mar 2022 05:11:16 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id k92so8021295pjh.5; Tue, 08 Mar 2022 05:11:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7qC9ZJnuyUGolXYQcUY/Tf83TQ1ETOj72VlDmZ6p6Ws=; b=bbq65S48n9rZTrLzmf2FQhuouyrA3lYX5oOGHOk8bQ0lmGrPYhUO3lK998y9GjF9bu vCvJg7DISbQI18AnpD6LwKkqHVPzuAawwcKFxpQBY7Jci46AOIY27abot/Y7c0pUmleb sSIvOx+rD9KQ+NLDELdCNfBxcG6Qadgfg55KmuFg9UK6S/oZbf5oExnqnphyWYxQVIoS jR5tuUxa1B1WNKBTpwq5YAq2N3gU0+gQWQfDZEXc0n5mrrV/O6xvBRth6wLThNOjgQ1E /S8xUomQXGSIxjE/689sHKrC/06BzAG4JA7+MSvXUqThXLw7VNa0D6TLyxTKTWgMWzJF 1huQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7qC9ZJnuyUGolXYQcUY/Tf83TQ1ETOj72VlDmZ6p6Ws=; b=Xbp6oA13NMU/HfSz0h5hdRm9X0Nq5D0W5gSPOSlTrW6Lpu6yYqNXWDpamc68NlGRoY glKmuxG56vyuvEpq+vuKZ5srFczcDAo2hKYJpmTsfh9ekrfiwN0QGPMbcYcnQAM7dsOr Rj9NYiVV9RY9Q4ET0edG2INg6WftUye8/WGgH0AWnCzNOU/5je4DwCL08+rkSJdZ+E8X cqvsKfD/YNXLoo9xqDas+1rDiKfCmhUefoP5kl1xxz93cggU9GHNPPRQuyjBlFvmayq6 xX410S9TqAlpPYL8TcRTiTmL87afoSkeTw+kxvKwhnROonN/Wb7QzwkYJLYPLRY2e9Lg krOQ== X-Gm-Message-State: AOAM530QQqKSEUnH03sI2gMxHPslKI5eFE1TQ736DLVqKuAOPWsQfohu U4gLSJdASl3Gu4VRgZC6ys8= X-Google-Smtp-Source: ABdhPJxRT96hUCFt+OLq3AvoUmcskuUoVsObUNWdtEx8FWULrshXYxLLY3NlZtrK0hmbq5kssZXcTg== X-Received: by 2002:a17:90b:1809:b0:1bf:7e9:bdf5 with SMTP id lw9-20020a17090b180900b001bf07e9bdf5mr4642772pjb.52.1646745076176; Tue, 08 Mar 2022 05:11:16 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:15 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 2/9] bpftool: show memcg info of bpf map Date: Tue, 8 Mar 2022 13:10:49 +0000 Message-Id: <20220308131056.6732-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC bpf map can be charged to a memcg, so we'd better show the memcg info to do better bpf memory management. This patch adds a new field "memcg_state" to show whether a bpf map is charged to a memcg and whether the memcg is offlined. Currently it has three values, 0 : not charged -1 : the charged memcg is offline 1 : the charged memcg is online For instance, $ bpftool map show 2: array name iterator.rodata flags 0x480 key 4B value 98B max_entries 1 memlock 4096B btf_id 240 frozen memcg_state 0 3: hash name calico_failsafe flags 0x1 key 4B value 1B max_entries 65535 memlock 524288B memcg_state 1 6: lru_hash name access_record flags 0x0 key 8B value 24B max_entries 102400 memlock 3276800B btf_id 256 memcg_state -1 Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 11 +++++++++++ tools/bpf/bpftool/map.c | 2 ++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 15 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 4eebea8..a448b06 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -5864,6 +5864,7 @@ struct bpf_map_info { __u32 btf_value_type_id; __u32 :32; /* alignment pad */ __u64 map_extra; + __s8 memcg_state; } __attribute__((aligned(8))); struct bpf_btf_info { diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index db402eb..3b50fcb 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3939,6 +3939,17 @@ static int bpf_map_get_info_by_fd(struct file *file, } info.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id; +#ifdef CONFIG_MEMCG_KMEM + if (map->memcg) { + struct mem_cgroup *memcg = map->memcg; + + if (memcg == root_mem_cgroup) + info.memcg_state = 0; + else + info.memcg_state = memcg->kmemcg_id < 0 ? -1 : 1; + } +#endif + if (bpf_map_is_dev_bound(map)) { err = bpf_map_offload_info_fill(&info, map); if (err) diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index 0bba337..fe8322f 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -550,6 +550,7 @@ static int show_map_close_json(int fd, struct bpf_map_info *info) jsonw_end_array(json_wtr); } + jsonw_int_field(json_wtr, "memcg_state", info->memcg_state); emit_obj_refs_json(refs_table, info->id, json_wtr); jsonw_end_object(json_wtr); @@ -635,6 +636,7 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info) if (frozen) printf("%sfrozen", info->btf_id ? " " : ""); + printf("\n\tmemcg_state %d", info->memcg_state); emit_obj_refs_plain(refs_table, info->id, "\n\tpids "); printf("\n"); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 4eebea8..41e65b3 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -5864,6 +5864,7 @@ struct bpf_map_info { __u32 btf_value_type_id; __u32 :32; /* alignment pad */ __u64 map_extra; + __s8 memcg_state; } __attribute__((aligned(8))); struct bpf_btf_info { From patchwork Tue Mar 8 13:10:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1515AC433F5 for ; Tue, 8 Mar 2022 13:11:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346249AbiCHNM3 (ORCPT ); Tue, 8 Mar 2022 08:12:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234899AbiCHNMV (ORCPT ); Tue, 8 Mar 2022 08:12:21 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48C7948E41; Tue, 8 Mar 2022 05:11:18 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id a5so5710929pfv.2; Tue, 08 Mar 2022 05:11:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BdKzZoxcCHUk05nIKgEhaGedb/vHkDzRSP6HrqUzWkw=; b=mVG368BLiMOTpvYHAUUF4Hl0SvELdZmvTum03FPpeGhXzLHWktVasfzvvXjRTjOz8v OeRQAhrdpy137X55NmR+Q/Kuzq9xiVfXNfSvJp32pazR2MPbIlJ8Vzb0Bd7V7G/na2UQ /kfik/XTfzvUUpTBTd1qB8DMG2kpuKdtEfRyYh14NVfqPCAXvfHKDJ+64jnLsvC2UHCd Q3sUYvjb4wA56QqTFnZWhIKlsDgOcCCK5NzY5cMPJAHGwN6RbCmGB1z9nZM4JR4/0Zny plLD1LQTq3acxLCjuPV6GmozEA9rAtVuiBPQKKhaxmdk4ZZHaK0851qNleWCQ6+8DyHd neGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BdKzZoxcCHUk05nIKgEhaGedb/vHkDzRSP6HrqUzWkw=; b=T+tv+erLOYUodKRwrQo799/Hy2Bm0asGnoCfttiq/1f/epUXuMud0exijiAeqiEUJU JwRSdzdN2PuMjDoXew68KQ5nWgBxhO8DMqcNu0vlOoB80whLHuNIRRZRJOgyITxJbBoX 9bezDaoXJIsED/lqaIPKwuDSIQLHdyYtudBbFZ6R/lJbqdYV5QRShuKtaoqebuNa6fXU loNRWiRvSfbnYtwsBWIKpEu0tQD9LM/Bn08uwA5TH8qQcE6e0ukocZenul7SwrQcwkJd sMVeHzx3QHmhVLfZgYebpmLryrHlpAGV31aXlcGXjeV2Mi/GtbXHnihvEiTGb2QPGaRc EClw== X-Gm-Message-State: AOAM532JsQ8lhZQy/V3p/kDDMT+msbEyHYK1XRVrhSeHVQqgKhp+8lgU F/TwurWx3op8jXm683GOmq1I6gtz/VSjhHsGw6o= X-Google-Smtp-Source: ABdhPJxi1T75LgLTxomuRmkgtIa5sVE1omWzFmGT1bjc4dtbt/cmnHAw3/PxWVByPMfUm079lFsD0Q== X-Received: by 2002:a63:61d3:0:b0:37f:f9c5:bbb8 with SMTP id v202-20020a6361d3000000b0037ff9c5bbb8mr13005281pgb.332.1646745077687; Tue, 08 Mar 2022 05:11:17 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:17 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 3/9] mm: add methord to charge kmalloc-ed address Date: Tue, 8 Mar 2022 13:10:50 +0000 Message-Id: <20220308131056.6732-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-State: RFC This patch implements a methord to charge or uncharge related pages or objects from a given kmalloc-ed address. It is similar to kfree, except that it doesn't touch the pages or objects while does account only. Signed-off-by: Yafang Shao --- include/linux/slab.h | 1 + mm/slab.c | 6 ++++++ mm/slob.c | 6 ++++++ mm/slub.c | 32 ++++++++++++++++++++++++++++++++ 4 files changed, 45 insertions(+) diff --git a/include/linux/slab.h b/include/linux/slab.h index 5b6193f..ae82e23 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -182,6 +182,7 @@ struct kmem_cache *kmem_cache_create_usercopy(const char *name, void * __must_check krealloc(const void *objp, size_t new_size, gfp_t flags) __alloc_size(2); void kfree(const void *objp); void kfree_sensitive(const void *objp); +void kcharge(const void *objp, bool charge); size_t __ksize(const void *objp); size_t ksize(const void *objp); #ifdef CONFIG_PRINTK diff --git a/mm/slab.c b/mm/slab.c index ddf5737..fbff613 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -3796,6 +3796,12 @@ void kfree(const void *objp) } EXPORT_SYMBOL(kfree); +void kcharge(const void *objp, bool charge) +{ + /* Not implemented yet */ +} +EXPORT_SYMBOL(kfree); + /* * This initializes kmem_cache_node or resizes various caches for all nodes. */ diff --git a/mm/slob.c b/mm/slob.c index 60c5842..d3a789f 100644 --- a/mm/slob.c +++ b/mm/slob.c @@ -569,6 +569,12 @@ void kfree(const void *block) } EXPORT_SYMBOL(kfree); +void kcharge(const void *block, bool charge) +{ + /* not implemented yet. */ +} +EXPORT_SYMBOL(kcharge); + /* can't use ksize for kmem_cache_alloc memory, only kmalloc */ size_t __ksize(const void *block) { diff --git a/mm/slub.c b/mm/slub.c index 2614740..e933d45 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4563,6 +4563,38 @@ void kfree(const void *x) } EXPORT_SYMBOL(kfree); +void kcharge(const void *x, bool charge) +{ + void *object = (void *)x; + struct folio *folio; + struct slab *slab; + + WARN_ON(!in_task()); + + if (unlikely(ZERO_OR_NULL_PTR(x))) + return; + + folio = virt_to_folio(x); + if (unlikely(!folio_test_slab(folio))) { + unsigned int order = folio_order(folio); + int sign = charge ? 1 : -1; + + mod_lruvec_page_state(folio_page(folio, 0), NR_SLAB_UNRECLAIMABLE_B, + sign * (PAGE_SIZE << order)); + + return; + } + + slab = folio_slab(folio); + if (charge) + memcg_slab_post_alloc_hook(slab->slab_cache, + get_obj_cgroup_from_current(), GFP_KERNEL, 1, &object); + else + memcg_slab_free_hook(slab->slab_cache, &object, 1); + +} +EXPORT_SYMBOL(kcharge); + #define SHRINK_PROMOTE_MAX 32 /* From patchwork Tue Mar 8 13:10:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC992C43217 for ; Tue, 8 Mar 2022 13:11:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347061AbiCHNM2 (ORCPT ); Tue, 8 Mar 2022 08:12:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232181AbiCHNMV (ORCPT ); Tue, 8 Mar 2022 08:12:21 -0500 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8A2B488A0; Tue, 8 Mar 2022 05:11:19 -0800 (PST) Received: by mail-pf1-x42c.google.com with SMTP id p8so17287237pfh.8; Tue, 08 Mar 2022 05:11:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z98OQ6AdalG5oMDKVv5aA3FO3T1l3cK2nxdIAVKfKFQ=; b=YdbOfZYq1mOC7LrS/ckJnfEBQ+MAblMP+dx7AWe6RKa98jAA9r1n44yEApW3RKGMd5 zRC97/VTsM7EsdnHMDO7hE6yzmYjY+E4jfZQL8n8DTe/cfFkXDN5zun4HeZTuag2VUiI 1uVbJRLZgyBFHI3LI09GiKz5UK1PMeMrNKAI4k6YNc4E1md3TxXvNZpzuFtYIbfkIFOJ i797KhWbBX2INTNoR3rd6zalgu7CvEkKZbVypUJ8AUSn5KpA5CEVOMo06zFbjaj2nm8t BELs1qacF+x+Y92yqHN5epNG5LlOKvH5ZD7MfFAEnqr8Z3Ff6F3wXJTKxppyuwulT8mY r62Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z98OQ6AdalG5oMDKVv5aA3FO3T1l3cK2nxdIAVKfKFQ=; b=5OS99iRwHy9HfFc0WKwaUfIkhwaA1cCcwGFrCwHfPfcbQnzYgpjO/fYOiOpl/vIpWK pOkVQdPSDJ94VIAFu59NN8UPWvxC4xXwWV3j5E1+O6q0HC79S5ia0Jb/WkLTzgsyDS7u JBh0l8N5boa6q/dav6PFOTJN9NpEzRS45394UKX7AZGlLcMVTK6zFA7Cr2M1c+cFH+5Q 54EF7BZiJg2L1hhHj04VCAcSq9NhAhdmS7A6RAdU9nstAif0e7JzKNwtH81sJ5fpoOdT wGO3PqNgn6gJRXEO69tG1DbHmmGjR1Aq4/K+Zh0WL/+XA/r4peAir+I7EBi2xzHKysXT UbZQ== X-Gm-Message-State: AOAM533C+pzsWm+kTgsWz/G7KZwv0G+5YE2ehPw8kUtc1OPGhSEmLOEn jqdGAwwn8lyAGiq9uEPwApY= X-Google-Smtp-Source: ABdhPJxrM0tl1/gYxlXQIsaOe+AjwvJr57dfg0zKANngu7XWWfwFpew1AJotBmNiE7RcN7WhSYaheA== X-Received: by 2002:a63:b30d:0:b0:378:c5ee:afda with SMTP id i13-20020a63b30d000000b00378c5eeafdamr14297243pgf.385.1646745079189; Tue, 08 Mar 2022 05:11:19 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:18 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 4/9] mm: add methord to charge vmalloc-ed address Date: Tue, 8 Mar 2022 13:10:51 +0000 Message-Id: <20220308131056.6732-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-State: RFC This patch adds a methord to charge or uncharge a given vmalloc-ed address. It is similar to vfree, except that it doesn't touch the related pages while does account only. Signed-off-by: Yafang Shao --- include/linux/slab.h | 1 + include/linux/vmalloc.h | 1 + mm/util.c | 9 +++++++++ mm/vmalloc.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 40 insertions(+) diff --git a/include/linux/slab.h b/include/linux/slab.h index ae82e23..7173354 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -759,6 +759,7 @@ extern void *kvrealloc(const void *p, size_t oldsize, size_t newsize, gfp_t flag __alloc_size(3); extern void kvfree(const void *addr); extern void kvfree_sensitive(const void *addr, size_t len); +void kvcharge(const void *addr, bool charge); unsigned int kmem_cache_size(struct kmem_cache *s); void __init kmem_cache_init_late(void); diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 880227b..b48d941 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -161,6 +161,7 @@ void *__vmalloc_node(unsigned long size, unsigned long align, gfp_t gfp_mask, extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); +void vcharge(const void *addr, bool charge); extern void *vmap(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot); diff --git a/mm/util.c b/mm/util.c index 7e433690..f5f5e05 100644 --- a/mm/util.c +++ b/mm/util.c @@ -614,6 +614,15 @@ void kvfree(const void *addr) } EXPORT_SYMBOL(kvfree); +void kvcharge(const void *addr, bool charge) +{ + if (is_vmalloc_addr(addr)) + vcharge(addr, charge); + else + kcharge(addr, charge); +} +EXPORT_SYMBOL(kvcharge); + /** * kvfree_sensitive - Free a data object containing sensitive information. * @addr: address of the data object to be freed. diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 4165304..6fc2295 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2715,6 +2715,35 @@ void vfree(const void *addr) } EXPORT_SYMBOL(vfree); +void vcharge(const void *addr, bool charge) +{ + unsigned int page_order; + struct vm_struct *area; + int i; + + WARN_ON(!in_task()); + + if (!addr) + return; + + area = find_vm_area(addr); + if (unlikely(!area)) + return; + + page_order = vm_area_page_order(area); + for (i = 0; i < area->nr_pages; i += 1U << page_order) { + struct page *page = area->pages[i]; + + WARN_ON(!page); + if (charge) + memcg_kmem_charge_page(page, GFP_KERNEL, page_order); + else + memcg_kmem_uncharge_page(page, page_order); + cond_resched(); + } +} +EXPORT_SYMBOL(vcharge); + /** * vunmap - release virtual mapping obtained by vmap() * @addr: memory base address From patchwork Tue Mar 8 13:10:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85344C433EF for ; Tue, 8 Mar 2022 13:11:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347039AbiCHNMh (ORCPT ); Tue, 8 Mar 2022 08:12:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234606AbiCHNMY (ORCPT ); Tue, 8 Mar 2022 08:12:24 -0500 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DAB6488B6; Tue, 8 Mar 2022 05:11:21 -0800 (PST) Received: by mail-pl1-x62b.google.com with SMTP id m2so11046481pll.0; Tue, 08 Mar 2022 05:11:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6X23AJzOcszEUYA+Oc8WYhr7w/7aeKWJdCFlxFdfbxc=; b=QzDOs9yQq9R1CZFP5kH/i37OmN2DriXgh+Uc29/28MK52/n2BNL5T+LpOEtCkDH1wT SArng9TL/tXBv+Z80bt9GxJnPhLPtlJphCtIvRp0gdbabeClqLN5OJJWkW3w2ZTfWwQa Rnjfw7mAFLLj9wTWH+Eh6SOwOz0vOv1IZ2Ez9Hdj3Lib06/AoNebKC9gEtTDUdNrv4NQ 9DcZx2G8SWEzn6bao5aEaUSzUNDhlZZ1d3NO0Rldf6bomrFxx1CFs1Xa9YnCHp8zH9iB t5155kTpOmXdPuCwsVILsy5c1hsx6nSVXhy04JPx1GbVRQYFwe+KylhvR1CA514F2xZd 3C/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6X23AJzOcszEUYA+Oc8WYhr7w/7aeKWJdCFlxFdfbxc=; b=G7uo/XXQehS2HJszpfZTnuqtAcr8oCVBolH9/xMu3y05Lb0tnx1km5sNug3rSo2c5e Cuh2U+3Zm5/FoO63TXJVV7QYkzHIMAwe6s8Gs91UdnpkGGrWJJQSn9KcBhI0FEfw6g4G OgFWJWlSm8YK5ZBU6yE0JCsxBjTpvd1nGceWK8YCpvu8Lyuo3E/Ez1LMrQxSnZHswSJq s8+3SZXKZSUXAIAMQPzmV7P6rLV+iSOCq60RNRZh3nnm82TrxWgOfq8IYvlWie+Vz8qr n80qEXyLk6i64dQuXcOdHlqK6O04TlMrs1gkBLSFzqTgYaxQcSPlGk5ssGz1IHiXNLCZ XHaw== X-Gm-Message-State: AOAM530SSWe8Jun2y/rDFnh6IWe5wl2t2seGzqPp1TOMYbqjnDcL51Nj sub75A0zUunVWmfqqZiVRms= X-Google-Smtp-Source: ABdhPJxKOq3e10aQ1RmNw8ArFeArmumS9bRFdOU8RkbwmtWaG0AV0PF5C3HqteolvWr6aJIng6XTbQ== X-Received: by 2002:a17:90a:4fa2:b0:1bd:383a:2b6b with SMTP id q31-20020a17090a4fa200b001bd383a2b6bmr4727275pjh.108.1646745080990; Tue, 08 Mar 2022 05:11:20 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:20 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 5/9] mm: add methord to charge percpu address Date: Tue, 8 Mar 2022 13:10:52 +0000 Message-Id: <20220308131056.6732-6-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-State: RFC This patch adds a methord to charge or uncharge a percpu address. It is similar to free_percpu except that it doesn't touch the related pages while does account only. Signed-off-by: Yafang Shao --- include/linux/percpu.h | 1 + mm/percpu.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/include/linux/percpu.h b/include/linux/percpu.h index f1ec5ad..1a65221 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -128,6 +128,7 @@ extern int __init pcpu_page_first_chunk(size_t reserved_size, extern void __percpu *__alloc_percpu_gfp(size_t size, size_t align, gfp_t gfp) __alloc_size(1); extern void __percpu *__alloc_percpu(size_t size, size_t align) __alloc_size(1); extern void free_percpu(void __percpu *__pdata); +void charge_percpu(void __percpu *__pdata, bool charge); extern phys_addr_t per_cpu_ptr_to_phys(void *addr); #define alloc_percpu_gfp(type, gfp) \ diff --git a/mm/percpu.c b/mm/percpu.c index ea28db2..22fc0ff 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -2309,6 +2309,56 @@ void free_percpu(void __percpu *ptr) } EXPORT_SYMBOL_GPL(free_percpu); +void charge_percpu(void __percpu *ptr, bool charge) +{ + int bit_off, off, bits, size, end; + struct obj_cgroup *objcg; + struct pcpu_chunk *chunk; + unsigned long flags; + void *addr; + + WARN_ON(!in_task()); + + if (!ptr) + return; + + addr = __pcpu_ptr_to_addr(ptr); + spin_lock_irqsave(&pcpu_lock, flags); + chunk = pcpu_chunk_addr_search(addr); + off = addr - chunk->base_addr; + objcg = chunk->obj_cgroups[off >> PCPU_MIN_ALLOC_SHIFT]; + if (!objcg) { + spin_unlock_irqrestore(&pcpu_lock, flags); + return; + } + + bit_off = off / PCPU_MIN_ALLOC_SIZE; + /* find end index */ + end = find_next_bit(chunk->bound_map, pcpu_chunk_map_bits(chunk), + bit_off + 1); + bits = end - bit_off; + size = bits * PCPU_MIN_ALLOC_SIZE; + + if (charge) { + obj_cgroup_get(objcg); + obj_cgroup_charge(objcg, GFP_KERNEL, size * num_possible_cpus()); + rcu_read_lock(); + mod_memcg_state(obj_cgroup_memcg(objcg), MEMCG_PERCPU_B, + (size * num_possible_cpus())); + rcu_read_unlock(); + } else { + obj_cgroup_uncharge(objcg, size * num_possible_cpus()); + rcu_read_lock(); + mod_memcg_state(obj_cgroup_memcg(objcg), MEMCG_PERCPU_B, + -(size * num_possible_cpus())); + rcu_read_unlock(); + obj_cgroup_put(objcg); + } + + spin_unlock_irqrestore(&pcpu_lock, flags); +} +EXPORT_SYMBOL(charge_percpu); + bool __is_kernel_percpu_address(unsigned long addr, unsigned long *can_addr) { #ifdef CONFIG_SMP From patchwork Tue Mar 8 13:10:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773783 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EDAEC433EF for ; Tue, 8 Mar 2022 13:11:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234899AbiCHNMi (ORCPT ); Tue, 8 Mar 2022 08:12:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347064AbiCHNMa (ORCPT ); Tue, 8 Mar 2022 08:12:30 -0500 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43FCA48E4C; Tue, 8 Mar 2022 05:11:23 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id m2so11046567pll.0; Tue, 08 Mar 2022 05:11:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3u3VsA3Fi5wDkVLbct/DeOE8XIO9r5bCMklz39tC+30=; b=d3xMfPnSj9DUQxDjtQLyi+itK0WX3JodAeZCw3xPBtkpGMEM+aD8bzj6wZwCzhWcHZ kt5w7qKQK3TtStKQ3oT5qOYPzVp+cM1eosOYmB4siu0vBKluYV91Do7HgnXDEzhnVvWj /WVDgwGVTkUWcEoOPTIW8t2SQAfmnpXJiVWedbMFSLl9wZeb1olWm/3CDgl9nqVXGaK7 1l6LV05YKWWBgXJBMIoPk8gJTEYnCZFusR5ixA0BtXVZJAfPp6gJtnzkg3aQPwjfKaqG 3o+ON2B1FO4YimaKwPZj1bBFt+nJn4+/9+rHZp7yv1OvuJ13QO4UOZGSb9B2++pmQ4vx 9ALg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3u3VsA3Fi5wDkVLbct/DeOE8XIO9r5bCMklz39tC+30=; b=LA3uWsLE1HvBLU200/beBDdNvg1cYENKHzlSJ3uLVt9JOI59YrQzRkLHdNRPJ/+RII FFHifp477D8PijtZ7+rHiwYA03JiRF9bgOnL0KxTMSMm1/UyO5rnxwE3c9M6Yr9Qs6Lo P3GHWscWM7QaX/bbYTSwrl9IBFkbSf3OSAEucXqlZLdZ45S0rYLN4fTYRPlHvd10sveL ebYgtcAKIIHbup7iDA4bzOryRXQgqcNFR1l4vGCEN8JOo2OuqrGeDD4kAgFoXrS0D+Er wrsK+6R2Bo9tNyW5NzNTubdrQVgC4Aph8SSuAkpbNChKWoQCGJ8rgejxqtQyM0Is+4Ny 5uag== X-Gm-Message-State: AOAM531iKbNTYPgm8Zdu4pKldvBXhVUrw4hjqbT18yEL2WAF4N/YcNxI zOdm8Tk5/tp7EP2ClNjrsRo= X-Google-Smtp-Source: ABdhPJzwC/En9lunCH8+wURsQQ1/FmGWLsc3MnOR1uRl1atGtzR7v2u8iVEpo0GJyAqsYIshW6NEaQ== X-Received: by 2002:a17:902:f68f:b0:151:fbf5:db45 with SMTP id l15-20020a170902f68f00b00151fbf5db45mr4536477plg.60.1646745082822; Tue, 08 Mar 2022 05:11:22 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:22 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 6/9] bpf: add a helper to find map by id Date: Tue, 8 Mar 2022 13:10:53 +0000 Message-Id: <20220308131056.6732-7-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC A new helper bpf_map_idr_find() is introduced for later use. Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 3b50fcb..68fea3b 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -3474,6 +3474,21 @@ static int bpf_prog_get_fd_by_id(const union bpf_attr *attr) return fd; } +static struct bpf_map *bpf_map_idr_find(unsigned long id) +{ + void *map; + + spin_lock_bh(&map_idr_lock); + map = idr_find(&map_idr, id); + if (map) + map = __bpf_map_inc_not_zero(map, true); + else + map = ERR_PTR(-ENOENT); + spin_unlock_bh(&map_idr_lock); + + return map; +} + #define BPF_MAP_GET_FD_BY_ID_LAST_FIELD open_flags static int bpf_map_get_fd_by_id(const union bpf_attr *attr) @@ -3494,14 +3509,7 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) if (f_flags < 0) return f_flags; - spin_lock_bh(&map_idr_lock); - map = idr_find(&map_idr, id); - if (map) - map = __bpf_map_inc_not_zero(map, true); - else - map = ERR_PTR(-ENOENT); - spin_unlock_bh(&map_idr_lock); - + map = bpf_map_idr_find(id); if (IS_ERR(map)) return PTR_ERR(map); From patchwork Tue Mar 8 13:10:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773786 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB6C0C433FE for ; Tue, 8 Mar 2022 13:11:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347068AbiCHNMn (ORCPT ); Tue, 8 Mar 2022 08:12:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347056AbiCHNM1 (ORCPT ); Tue, 8 Mar 2022 08:12:27 -0500 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF4A448E53; Tue, 8 Mar 2022 05:11:24 -0800 (PST) Received: by mail-pf1-x42e.google.com with SMTP id e15so15706682pfv.11; Tue, 08 Mar 2022 05:11:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3y4pmtErmcUQqTEOzusAX2pzTuUiyQeDCSeMBuC6VMQ=; b=SiVmiDuO/2M2Xzol8AGb7jc3+OznGw2J3+JiAFGq0IgPtRBs+B5WCzg555zf8AFxw/ SvYvYKlfzqYxdYqt6HLWTZDy8hkMR7VnTuVpC45UG+9hWjGQT82Z1lZGXZz5u5ZkI1Sb T8wWUD1Ign+8sEYvUjXqv3kCkXpo2pRUaADUOJC50otN7MQwMr6DhUs3+5lXu5EZyZlv 2hNDqCxhpUMTgCqyUWHe6LD6wxoxG2PbB4oUB14AjukT5kPNQVRsOtgTr1FmYzxJeIYq pq7ceeU7QUXEBoLpCRr4f2m0Wt8L3ogseVrdawWE9L5U3W4JXEjIhDfPwNpqxZEx+Kkp TdMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3y4pmtErmcUQqTEOzusAX2pzTuUiyQeDCSeMBuC6VMQ=; b=59WBCR/M7mUmLkcvVN2HJC+C5GXYhoXLbljRsebVoJh2k3FIhaoGRzoc4X2AwiqemQ N4k5qXzdb+cGJGhvJnhaArs1KWS4kRLDFSQAjSFFgX2nsGNPvlT8yFnBvYVfkojVa+eR tTU/og8wXCoXC+kgqpgL2GsZZlzdJqxjS9ihqQzg9JPTt7R8OcbHNhNVfLg44JKb0QkB xuFRXEdMltpfip4Z3swaneonms3U7TBqQlkarjLQyfJk6h16+63e4/PKs2jSfYCH+B4P R3jZSyUuYAJCUlvcA431qWYhzLOPZEqFJpfYy5lJmX+1ffLCzyBXaa/eN3ioaNzq7N4C DIcA== X-Gm-Message-State: AOAM530PpzRgancpePRIeXmbUT2rO3gJLB0+DmgDLWjA1HJmDmZg08sH Sew2+ygnirwOVxw0wamQZtE= X-Google-Smtp-Source: ABdhPJxozHq59VkUFv288fl2mix6RMNdONpReM8rlehkbq7w0YTK49AhZHAdBosK85EByFXP+tX5JQ== X-Received: by 2002:a63:d253:0:b0:380:3aee:8437 with SMTP id t19-20020a63d253000000b003803aee8437mr9002618pgi.382.1646745084332; Tue, 08 Mar 2022 05:11:24 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:23 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 7/9] bpf: add BPF_MAP_RECHARGE syscall Date: Tue, 8 Mar 2022 13:10:54 +0000 Message-Id: <20220308131056.6732-8-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC This patch adds a new bpf syscall BPF_MAP_RECHARGE, which means to recharge the allocated memory of a bpf map from an offline memcg to the current memcg. The recharge methord for each map will be implemented in the follow-up patches. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 9 +++++++++ kernel/bpf/syscall.c | 19 ++++++++++++++++++- 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 88449fb..fca274e 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -147,6 +147,8 @@ struct bpf_map_ops { bpf_callback_t callback_fn, void *callback_ctx, u64 flags); + bool (*map_recharge_memcg)(struct bpf_map *map); + /* BTF name and id of struct allocated by map_alloc */ const char * const map_btf_name; int *map_btf_id; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index a448b06..290ea67 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -821,6 +821,14 @@ struct bpf_cgroup_storage_key { * Returns zero on success. On error, -1 is returned and *errno* * is set appropriately. * + * BPF_MAP_RECHARGE + * Description + * Recharge bpf memory from an offline memcg + * + * Return + * Returns zero on success. On error, -1 is returned and *errno* + * is set appropriately. + * * NOTES * eBPF objects (maps and programs) can be shared between processes. * @@ -875,6 +883,7 @@ enum bpf_cmd { BPF_ITER_CREATE, BPF_LINK_DETACH, BPF_PROG_BIND_MAP, + BPF_MAP_RECHARGE, }; enum bpf_map_type { diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 68fea3b..85456f1 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1128,7 +1128,6 @@ static int map_lookup_elem(union bpf_attr *attr) return err; } - #define BPF_MAP_UPDATE_ELEM_LAST_FIELD flags static int map_update_elem(union bpf_attr *attr, bpfptr_t uattr) @@ -4621,6 +4620,21 @@ static int bpf_prog_bind_map(union bpf_attr *attr) return ret; } +static int map_recharge_elem(union bpf_attr *attr) +{ + int id = attr->map_id; + struct bpf_map *map; + + map = bpf_map_idr_find(id); + if (IS_ERR(map)) + return PTR_ERR(map); + + if (map->ops->map_recharge_memcg) + map->ops->map_recharge_memcg(map); + + return 0; +} + static int __sys_bpf(int cmd, bpfptr_t uattr, unsigned int size) { union bpf_attr attr; @@ -4757,6 +4771,9 @@ static int __sys_bpf(int cmd, bpfptr_t uattr, unsigned int size) case BPF_PROG_BIND_MAP: err = bpf_prog_bind_map(&attr); break; + case BPF_MAP_RECHARGE: + err = map_recharge_elem(&attr); + break; default: err = -EINVAL; break; From patchwork Tue Mar 8 13:10:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773785 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5220C433F5 for ; Tue, 8 Mar 2022 13:11:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347067AbiCHNMl (ORCPT ); Tue, 8 Mar 2022 08:12:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347040AbiCHNM1 (ORCPT ); Tue, 8 Mar 2022 08:12:27 -0500 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9ADF148884; Tue, 8 Mar 2022 05:11:26 -0800 (PST) Received: by mail-pg1-x536.google.com with SMTP id c11so4023354pgu.11; Tue, 08 Mar 2022 05:11:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lr/CFIp00pEF/BEp0dXxqdNW/AI45azIAndxyHO9fEc=; b=JB0q4IHcXTozLToE7J/d1F6oTslE6YdOVpdWGhqprivSKxigjke87oe7Q/SAuyYUeH WGJhQaBuZ3HVQClkwYFjPKSRzgp4emV2L1P9SSaiybKpW76EN96/bTPE61IvacYCCbPu 2tnyx9h8AlAdJPTWgHaw0h6UrUPrZ543f9QNEfkMR0fTECuDmx4tQ3zxfQqOJsCk8sVk DYKKS+Xgd/h2MU2mxDUrjhR88QnzZGpZMmDwkqIWWmQrDT1ib7gRT6txscX8hGM0D7a6 uqP4aOAJ9FLNxpLxK1WiRTAVv86fU6Cfs15ePd+M+iOUp7u+I0QzoTVPdqbLiMfI4jIW wpSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lr/CFIp00pEF/BEp0dXxqdNW/AI45azIAndxyHO9fEc=; b=DEKs1ZUQhBacW2MSbjDCwWwcwbyLZ5gH21aaIHH9LKqa+cXaHnCtK4Oi2MiYbGcSe4 1PANkP6q7k2F47ixA7tl53z+xVt3NAtQkj208q0r/ufh1sDi8ri7uQEDF8dkC73sHN6o N/stkV4lgxqNND9fQFhnqZ/FORIsFo2iAeJZoh9UrXZYJDCFnBJ0Ru3FL79m6J07eVbZ ZHgfwbZcgdjacpsBFvkDtmhY9bYs8xgItBJQltPwVIWxbmluibGtZOEDafcSK2LvrpU/ Bxs14dfZpfsyPFevVoxdDP3oagF+Sg7DYdI8a7yGdOEsWXoeVduBU1H1vfWBq5vxdfjH SFtg== X-Gm-Message-State: AOAM5323NzjLIKGrKJ6ak/c1HBvw85zRKNn4HICA2L6dqD7CXuxgXyih IdFr4F8BjLfz7qifptcexKsMvwF96pJNXzVUZxg= X-Google-Smtp-Source: ABdhPJyOlCtmyFx3M63Qe/Lo+ROqOvjtLMDDMPcSoi5Quoxet+iI8DibJx/wB3rOKVChv0jQxhwF4w== X-Received: by 2002:a05:6a00:198f:b0:4f6:c0e4:207d with SMTP id d15-20020a056a00198f00b004f6c0e4207dmr17968238pfl.82.1646745086059; Tue, 08 Mar 2022 05:11:26 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:25 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 8/9] bpf: make bpf_map_{save, release}_memcg public Date: Tue, 8 Mar 2022 13:10:55 +0000 Message-Id: <20220308131056.6732-9-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC These two helpers will be used in map specific files later. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 21 +++++++++++++++++++++ kernel/bpf/syscall.c | 19 ------------------- 2 files changed, 21 insertions(+), 19 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index fca274e..2f3f092 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -23,6 +23,7 @@ #include #include #include +#include struct bpf_verifier_env; struct bpf_verifier_log; @@ -209,6 +210,26 @@ struct bpf_map { } owner; }; +#ifdef CONFIG_MEMCG_KMEM +static inline void bpf_map_save_memcg(struct bpf_map *map) +{ + map->memcg = get_mem_cgroup_from_mm(current->mm); +} + +static inline void bpf_map_release_memcg(struct bpf_map *map) +{ + mem_cgroup_put(map->memcg); +} +#else +static inline void bpf_map_save_memcg(struct bpf_map *map) +{ +} + +static inline void bpf_map_release_memcg(struct bpf_map *map) +{ +} +#endif + static inline bool map_value_has_spin_lock(const struct bpf_map *map) { return map->spin_lock_off >= 0; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 85456f1..7b4cbe7 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -414,16 +414,6 @@ void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock) } #ifdef CONFIG_MEMCG_KMEM -static void bpf_map_save_memcg(struct bpf_map *map) -{ - map->memcg = get_mem_cgroup_from_mm(current->mm); -} - -static void bpf_map_release_memcg(struct bpf_map *map) -{ - mem_cgroup_put(map->memcg); -} - void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node) { @@ -461,15 +451,6 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, return ptr; } - -#else -static void bpf_map_save_memcg(struct bpf_map *map) -{ -} - -static void bpf_map_release_memcg(struct bpf_map *map) -{ -} #endif /* called from workqueue */ From patchwork Tue Mar 8 13:10:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12773784 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6D3DC433FE for ; Tue, 8 Mar 2022 13:11:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346651AbiCHNMk (ORCPT ); Tue, 8 Mar 2022 08:12:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229804AbiCHNM2 (ORCPT ); Tue, 8 Mar 2022 08:12:28 -0500 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17F1C488A0; Tue, 8 Mar 2022 05:11:28 -0800 (PST) Received: by mail-pg1-x529.google.com with SMTP id 132so16430349pga.5; Tue, 08 Mar 2022 05:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bgJJlFnbXaRCIAYL5SQZwRZJIW6MmhaiowbOp4O3M1Y=; b=BTNwJVlZtiizxAYf1GqUM418mm4KpUgPRAZTqG8USzvhC/zyC+HUN65jCkTRLcPE2y HC/PH3SHk9dp5WMPyxiDG5XVzxxSlmltP5UJbf0EkQf2S+Fe/dbRT+enxXvZFxiN82Ow wtRn7eF5edcKUjmFS8ZTggAfNGUZIjzpxJkhs9p29fupdSUaEbpaDnj9IFnIf9Oxvwsh ZyaZyxh5UI7oo1CvxNcIyaVnuS8Z/p8OJadCY6UL6fhmjwrhb0OmPoQ35tw6WiG7nTCm KzeezESmY6izJJ+jz5rSydmReEB289qVpYWBJ6imgVnyOlsLHG0l1gGjNvR2wWVYoxix t9DQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bgJJlFnbXaRCIAYL5SQZwRZJIW6MmhaiowbOp4O3M1Y=; b=aTSeHa5vF2f8fiPxIRCqR9TnMXjw89rUEr3C0uXg9XT93uiEEnRz2u5J2K+mAM6U/G Or5DpH4djKBCMP8rKEmSHOH2qbDmw319x3gUY8WQXJ6TroTMXMDLJeA1bIjO0j+tCgT+ T4bW3Yex8Si8amxY1DMib5ETk/3rj1kmzek5jopqQKNJctN/VFSndsaJrsL/+EU5MQnX ceEpXyQj67GnpWa1RzxjlAVOPlNR4AUYxD08quxck07RuSELCrflZJBbCWaOhOzf8jgz NffOF2/8YlTZlKoi7IxMF7yiSdId/DLiQSeh6QM3pFlJhiTowsYhO/HGde+MQvOPnko1 3xYA== X-Gm-Message-State: AOAM530AJTtoqkpPGBHYzWlEBiAQUQp4kgO16BKC49jLcCFceOXEXjiZ qT/MGI6+6iFucbuoOz7v48E= X-Google-Smtp-Source: ABdhPJxcfF7dzcGyqjiKeRaKXA6W9nGPA0uFJq/5jcaGXg7/O7wJRtUT++VsWLtNLmrzXp07AhJYEg== X-Received: by 2002:aa7:8882:0:b0:4df:7b9e:1ccb with SMTP id z2-20020aa78882000000b004df7b9e1ccbmr18056424pfe.41.1646745087617; Tue, 08 Mar 2022 05:11:27 -0800 (PST) Received: from vultr.guest ([149.248.19.67]) by smtp.gmail.com with ESMTPSA id s20-20020a056a00179400b004f709998d13sm7378598pfg.10.2022.03.08.05.11.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Mar 2022 05:11:27 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, akpm@linux-foundation.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com Cc: linux-mm@kvack.org, netdev@vger.kernel.org, bpf@vger.kernel.org, Yafang Shao Subject: [PATCH RFC 9/9] bpf: support recharge for hash map Date: Tue, 8 Mar 2022 13:10:56 +0000 Message-Id: <20220308131056.6732-10-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220308131056.6732-1-laoar.shao@gmail.com> References: <20220308131056.6732-1-laoar.shao@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC This patch supports recharge for hash map. We have already known how the hash map is allocated and freed, we can also know how to charge and uncharge the hash map. Firstly, we need to uncharge it from the old memcg, then charge it to the current memcg. The old memcg must be an offline memcg. Signed-off-by: Yafang Shao --- kernel/bpf/hashtab.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 6587796..4d103f1 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "percpu_freelist.h" #include "bpf_lru_list.h" #include "map_in_map.h" @@ -1466,6 +1467,36 @@ static void htab_map_free(struct bpf_map *map) kfree(htab); } +static bool htab_map_recharge_memcg(struct bpf_map *map) +{ + struct bpf_htab *htab = container_of(map, struct bpf_htab, map); + struct mem_cgroup *old = map->memcg; + int i; + + if (!old) + return false; + + /* Only process offline memcg */ + if (old == root_mem_cgroup || old->kmemcg_id >= 0) + return false; + + bpf_map_release_memcg(map); + kcharge(htab, false); + kvcharge(htab->buckets, false); + charge_percpu(htab->extra_elems, false); + for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) + charge_percpu(htab->map_locked[i], false); + + kcharge(htab, true); + kvcharge(htab->buckets, true); + charge_percpu(htab->extra_elems, true); + for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) + charge_percpu(htab->map_locked[i], true); + bpf_map_save_memcg(map); + + return true; +} + static void htab_map_seq_show_elem(struct bpf_map *map, void *key, struct seq_file *m) { @@ -2111,6 +2142,7 @@ static int bpf_for_each_hash_elem(struct bpf_map *map, bpf_callback_t callback_f .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_release_uref = htab_map_free_timers, .map_lookup_elem = htab_map_lookup_elem, @@ -2133,6 +2165,7 @@ static int bpf_for_each_hash_elem(struct bpf_map *map, bpf_callback_t callback_f .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_release_uref = htab_map_free_timers, .map_lookup_elem = htab_lru_map_lookup_elem, @@ -2258,6 +2291,7 @@ static void htab_percpu_map_seq_show_elem(struct bpf_map *map, void *key, .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_percpu_map_lookup_elem, .map_lookup_and_delete_elem = htab_percpu_map_lookup_and_delete_elem, @@ -2278,6 +2312,7 @@ static void htab_percpu_map_seq_show_elem(struct bpf_map *map, void *key, .map_alloc_check = htab_map_alloc_check, .map_alloc = htab_map_alloc, .map_free = htab_map_free, + .map_recharge_memcg = htab_map_recharge_memcg, .map_get_next_key = htab_map_get_next_key, .map_lookup_elem = htab_lru_percpu_map_lookup_elem, .map_lookup_and_delete_elem = htab_lru_percpu_map_lookup_and_delete_elem,