From patchwork Wed Aug 10 15:18:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F40BEC19F2A for ; Wed, 10 Aug 2022 15:18:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CCD86B0072; Wed, 10 Aug 2022 11:18:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 854F88E0001; Wed, 10 Aug 2022 11:18:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A7F66B0074; Wed, 10 Aug 2022 11:18:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 583A76B0072 for ; Wed, 10 Aug 2022 11:18:47 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2CB8A40E8F for ; Wed, 10 Aug 2022 15:18:47 +0000 (UTC) X-FDA: 79784040294.15.02CF659 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf13.hostedemail.com (Postfix) with ESMTP id A928320192 for ; Wed, 10 Aug 2022 15:18:46 +0000 (UTC) Received: by mail-pf1-f176.google.com with SMTP id y141so13986575pfb.7 for ; Wed, 10 Aug 2022 08:18:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=iWaAjku+9DHBSIX7N6Ue4B+MVNq84IC+6QjMUqI3TB0=; b=CPgLyVnidMMk4GxlhvJN9csE3zFaEsw0RkDcky9edtzCViXQGvkkolE7C4TauVH1QK 2nwbPpGvXiEawLGVclFMmxHrdzGXGedWIq7VVtZ5LJZV4uDTybEQXg2lZo1vIrf119fs 5a+PGRzH49LiiCC6cxeXW2R8+oPDi4Yqx67nioX52IQnf6F7a7PqSh3xeeN3GOKYuAGv UoZusZOzrP/kcTKXc+27w4CadHPevkiaH6PG5MPg4w589WghL5lDmrgT3NjzroR+Eo8x gORQJLAFVP5SqSrnPdA7oiaRP1IG1vN1c0YUxc1BT1Ff8ORvSMv7a3Vlh0p57unUFgrA YLPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=iWaAjku+9DHBSIX7N6Ue4B+MVNq84IC+6QjMUqI3TB0=; b=HTCMlOLBcmKGKxdyQ4kGPoQd0B/OSJjrEegIqzxYq/sn6PzLHV/5vuUSrOklKTrn16 IPzKUoHO2We1ONpjlonTCWagjrB6mMBoqaJYPlxD5CshU0RgWI/fOWw+A4Sw4RLCS5TD mWSH9cr84egG+6hp+v8AVjsdmk1sPznrTQ1+dApMuoaKm5vuAqk7mCfyHNAJ3fPAV8Td 9IBNZKDHi7QoPe9gkhZJkeWTOa1fBucxOs99WftYquDc/sqSmV7TlT/EJT9eyBdCwT4+ SiYm170a/sZdoF1O/L1AicbQ3XAe7unAiaFULHIvkdlF5K1yA8EuDR3LLz9Kr0cLOKSc XkiQ== X-Gm-Message-State: ACgBeo2oGAU0YGtcLLXDjaclv918TJXqpu1jUrHgSoMTSzJSCdvBwSDj 4dov/lANBXDgAiNXa7XpWHU= X-Google-Smtp-Source: AA6agR7sx3QNeFXPD4mnbqj/uQJmqwZMeBLMKJlcJXBo6ebE+MWM5vHtGHaBxs5FWyjJha+MVy6bOg== X-Received: by 2002:a63:494a:0:b0:41c:f29e:2a2e with SMTP id y10-20020a63494a000000b0041cf29e2a2emr22391660pgk.477.1660144725756; Wed, 10 Aug 2022 08:18:45 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:44 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 01/15] bpf: Remove unneeded memset in queue_stack_map creation Date: Wed, 10 Aug 2022 15:18:26 +0000 Message-Id: <20220810151840.16394-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144726; a=rsa-sha256; cv=none; b=A6i2Z/mhgpz7hLSqXNv5B6PN9HZqW7zvY0MnocQM6NFiGuruhypTdCbaTpgZAfV64Eeyl1 1xvxEqPeK0/UXbKHVHyHXsMnu73Xxz7N/fSqTtZCvpiTTnosQ2n1sIBUv9P7bdl8Y4Ig/8 lCJC8DSy+LIS38CKjViefo978vM/Ph0= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=CPgLyVni; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iWaAjku+9DHBSIX7N6Ue4B+MVNq84IC+6QjMUqI3TB0=; b=iY/0KoJO4R8K3KT6SLHk8iCi5dMhApsg+DBTiq7mWTMOaKgvQcRiYfg1KCbyr0TkWZWRj3 ERZFUc29HSWqmsNVtYQlRXWXEWfUrXBvqx6xQssjphsHowrl5BvRwkrW6vbqF5/t6Nue4X WrpoQXujM7FYDweqPXDc4vCVQi8eO6o= X-Rspamd-Server: rspam12 X-Rspam-User: X-Rspamd-Queue-Id: A928320192 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=CPgLyVni; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Stat-Signature: mwqqpj8cqryjx8qz8wbemf3r5a7jbzph X-HE-Tag: 1660144726-417280 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __GFP_ZERO will clear the memory, so we don't need to memset it. Signed-off-by: Yafang Shao --- kernel/bpf/queue_stack_maps.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index a1c0794..8a5e060 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -78,8 +78,6 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) if (!qs) return ERR_PTR(-ENOMEM); - memset(qs, 0, sizeof(*qs)); - bpf_map_init_from_attr(&qs->map, attr); qs->size = size; From patchwork Wed Aug 10 15:18:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EB3DC00140 for ; Wed, 10 Aug 2022 15:18:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 80A026B0073; Wed, 10 Aug 2022 11:18:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 792366B0074; Wed, 10 Aug 2022 11:18:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5EC876B0075; Wed, 10 Aug 2022 11:18:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 43E506B0073 for ; Wed, 10 Aug 2022 11:18:49 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1C5FF4140C for ; Wed, 10 Aug 2022 15:18:49 +0000 (UTC) X-FDA: 79784040378.01.4D8DA65 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf10.hostedemail.com (Postfix) with ESMTP id AA6A8C0177 for ; Wed, 10 Aug 2022 15:18:48 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id q7-20020a17090a7a8700b001f300db8677so2420740pjf.5 for ; Wed, 10 Aug 2022 08:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=0qneriZlJB8MgUasF3oViKnlhzvyQw+rN/9EeLAKF4M=; b=TgGNLJw+BG2sALPoudkYDLIisbvrlTy7Epjw7sJucu8XOeUAww+Zrw5V/QbGPxzlU7 6vC7+2w+TOmeVTZFe11FTHUd6TscOL6cLwB9twxvsid65cnQwY8QMvDPU5RlAs5w5HG/ LDWjih8myCE17+PiotdM4circp79KSLTaDGRicB+ncaRWmUNgnvxzFIj6Omvt/VulW+O 5GrPcLdp1iuxBHPNCMPXiquLzESc2ppKZzn+UGYo0S2ZK2dwURl1qiO6jYgKilJZCn0r bu5f1sV2rCaGn6uVYMsZMOdgjocRuH/r+Cvd4ueX2q8A67notVve/t7FCBdKT6pXOk5s t7mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=0qneriZlJB8MgUasF3oViKnlhzvyQw+rN/9EeLAKF4M=; b=qAnPPnMvndxVSQc/nu3H11tlXUPME1glXDuEUtQFx7CEZH83jaNyL06Z53WeiXjn8j r8yGTfYdngVhV7BvyIIs5Bb/RZC/2Q6XIw8TDy6hGa0YCInLytJVI4eK0tqtwV0o4CCw /Wln3EcvGoRO6zH4eoJAxb9Ff7J3x9fpkTRNVejtD9jju8cM16oQ7Roco2GXHJBdMtO5 i8uAGUPVs+2soYO1xPx7ZZO0tHwNz9hGjeEbMV4FUj2/qYR/dA7RJhq9CWlexm0RWGTI LzNpLRsPRcMzs26GVdEedVMg8pyPp+Mkib+qFFSmLbNvELg3+E/mjR7U2O0RmY+krr6e 1gvQ== X-Gm-Message-State: ACgBeo0q4siLiDXXqsxsNRHFjW64MiSknX8LmGQb8tSxbWRNqxb1LxHo xYm1J+OKuC25feTmn0XIsoA= X-Google-Smtp-Source: AA6agR4d1Bxd/15ISidjIj+TY1Sb3aM1iWdWnxazQxG6tDo5K9JirOOAXyUItlqAGwtIQ5VXLWdDRg== X-Received: by 2002:a17:902:7d89:b0:16e:f604:31b8 with SMTP id a9-20020a1709027d8900b0016ef60431b8mr28284781plm.0.1660144727674; Wed, 10 Aug 2022 08:18:47 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:46 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 02/15] bpf: Use bpf_map_area_free instread of kvfree Date: Wed, 10 Aug 2022 15:18:27 +0000 Message-Id: <20220810151840.16394-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144728; a=rsa-sha256; cv=none; b=nS7MD5jwL4s7smVKuHEmwtuFI4HgPgKCR+zSU1rtZPd0pgQyasQMk0NY8ddT+cl74iJH1t aOo85xal0ae/sxw0NqWn7DYOiXLi5rEU7sh6vLQcHHFJBWyqzFguu0sC3OOrmqYzNNJmTi QR3nVggHojNUk1SYQBemVnBYkUWx7nU= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=TgGNLJw+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144728; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0qneriZlJB8MgUasF3oViKnlhzvyQw+rN/9EeLAKF4M=; b=36ShdX7Yail3ExOegOJU3/D5vjeVGg1b7BxpU4O7JVd/lghGZGoivLS69vQZwfBfLXKAgY 8Fh6qTTCcQlWIX/lp27D8u8qPdeSq8LuE8pJZsYTUNmcc1EVlkA03fEBMBQEacR+zNWVcA 8bdIt8+OtAwqnvHPY+8l/SISz7QVIs8= Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=TgGNLJw+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspam-User: X-Stat-Signature: ppkique4g889zfgqmybffafy8htmsuyr X-Rspamd-Queue-Id: AA6A8C0177 X-Rspamd-Server: rspam03 X-HE-Tag: 1660144728-726648 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: bpf_map_area_alloc() should be paired with bpf_map_area_free(). Signed-off-by: Yafang Shao --- kernel/bpf/ringbuf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index ded4fae..3fb54fe 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -116,7 +116,7 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) err_free_pages: for (i = 0; i < nr_pages; i++) __free_page(pages[i]); - kvfree(pages); + bpf_map_area_free(pages); return NULL; } @@ -190,7 +190,7 @@ static void bpf_ringbuf_free(struct bpf_ringbuf *rb) vunmap(rb); for (i = 0; i < nr_pages; i++) __free_page(pages[i]); - kvfree(pages); + bpf_map_area_free(pages); } static void ringbuf_map_free(struct bpf_map *map) From patchwork Wed Aug 10 15:18:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A55F8C25B07 for ; Wed, 10 Aug 2022 15:18:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4AE006B0074; Wed, 10 Aug 2022 11:18:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 438006B0075; Wed, 10 Aug 2022 11:18:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ED7C6B0078; Wed, 10 Aug 2022 11:18:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0B0A96B0074 for ; Wed, 10 Aug 2022 11:18:51 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id DD246413DF for ; Wed, 10 Aug 2022 15:18:50 +0000 (UTC) X-FDA: 79784040420.20.7268701 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by imf17.hostedemail.com (Postfix) with ESMTP id 757D54017A for ; Wed, 10 Aug 2022 15:18:50 +0000 (UTC) Received: by mail-pj1-f42.google.com with SMTP id o3-20020a17090a0a0300b001f7649cd317so2545678pjo.0 for ; Wed, 10 Aug 2022 08:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=HeJKAK+ncLW0QpMLm4IDJ4E+/yPQHa8h30kUW/Dj5Mc=; b=XtiAQU2TKOS4SFca4YmsB1FpaWpYM2uRBKRRXuKoFk8dmJQVocIHaqj3ViB9MwYNUP NBs9Rmgm25dnnSiiqjPij5pMXxWkvby2d/sDHS2LrRilzeYCq1WQrg6ulCSBcaqqH8nI ObPSlTLt7qhtGBFV6wm8DFWftcaVT0BgLNWu9ZnchCHKrai7RndnuJwifJfnWPyieeWV DX+GuMMHDCxpM1jo7NfqSnIFXUpfqlbCw1Q6dyqzLRwJy+8vVF8+SaPwsrvx52kKG4d9 uj0X2EiNlDGextq0j9+3TYfBYgoAaVHcJRh3kMVYCui7IgZxR0Jzu/tPF3dY7BsQahJx FrXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=HeJKAK+ncLW0QpMLm4IDJ4E+/yPQHa8h30kUW/Dj5Mc=; b=VeHR0NJOLLTfBG+bCcpXsNY20NJo5ayNfumiVtjOjEJxEQTLNnfB2l2yGCCp22iP/O VAEDZFOnjmfsenTz9TPGXX4D/lY0kRpBwqvh30GCWrrB7966Rq1rn2XnJrK90xosaXvy KpZWeZDujh6H0A9FBOgFzG52LFjjREGf4FTB/cFSuSRS0PLF9HlT0N6xGYOav5FVYFmw L4wubVak4w741e86LIALTRYzQiGQ4JquMrh2O1/IZmb3L1bDMxlSNbM6KQUdoSBOeiam r0d0x6J3jRIvJoykzmAJpJ/as757uRj/4TwHUvpKzLhAtQqPZ7oUhGlcUy23+q+wFeYH FRrA== X-Gm-Message-State: ACgBeo0I1/Tw/tAobW+oZn3JTGxjjr8SXPCaddzknYSBnxOyow+xsjbV zGc9pnw4MRrXhLrLf7lqkIk= X-Google-Smtp-Source: AA6agR7VMBFj+U1/fWThTC6WnTI2GpLS9sPyCIsBQYq/J78+ZCvY7sLBGx8dOi6DBEUy14frCPJ19g== X-Received: by 2002:a17:902:e5c6:b0:16e:f3b6:ddb5 with SMTP id u6-20020a170902e5c600b0016ef3b6ddb5mr28394737plf.122.1660144729610; Wed, 10 Aug 2022 08:18:49 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:48 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 03/15] bpf: Make __GFP_NOWARN consistent in bpf map creation Date: Wed, 10 Aug 2022 15:18:28 +0000 Message-Id: <20220810151840.16394-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144730; a=rsa-sha256; cv=none; b=XsHeYS+2lhoW+9D3CUmw9YGmUglqvW1Ob+DbtMHj6KD4veqJF9yeHkiNaq5KBfw+IqsGWf QeZ0KvvfItS89xxn3SPlVQzoNhpYfsZLdPz5bMdiBnBfSKlwNuSXO+c72wb+GdjOdSzRKz ekcH2YqxWtWJ1RTYzF08XcrwrXA9wQ4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XtiAQU2T; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144730; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HeJKAK+ncLW0QpMLm4IDJ4E+/yPQHa8h30kUW/Dj5Mc=; b=zRB5fP1467GREb+1ryQjg1xfwh5/C16fCaHi/GpNiovDpQfLVgQOPphaigudsZBIXCjrlD rSVWGGugQKOQoT0zJHCecVK8v/O3gR5lgW5HiwhDp2zM7L5g+qqwshJYbuqiBu2VDzNowJ IQ7SYCk4T0ruSvhkCmXKCEMlAh4oxC4= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 757D54017A X-Rspam-User: X-Stat-Signature: rg85mskcdbnhj5u4ez3m4iykma1ig5nu Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XtiAQU2T; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf17.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-HE-Tag: 1660144730-814421 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some of the bpf maps are created with __GFP_NOWARN, i.e. arraymap, bloom_filter, bpf_local_storage, bpf_struct_ops, lpm_trie, queue_stack_maps, reuseport_array, stackmap and xskmap, while others are created without __GFP_NOWARN, i.e. cpumap, devmap, hashtab, local_storage, offload, ringbuf and sock_map. But there are not key differences between the creation of these maps. So let make this allocation flag consistent in all bpf maps creation. Then we can use a generic helper to alloc all bpf maps. Signed-off-by: Yafang Shao --- kernel/bpf/cpumap.c | 2 +- kernel/bpf/devmap.c | 2 +- kernel/bpf/hashtab.c | 2 +- kernel/bpf/local_storage.c | 4 ++-- kernel/bpf/offload.c | 2 +- kernel/bpf/ringbuf.c | 2 +- net/core/sock_map.c | 4 ++-- 7 files changed, 9 insertions(+), 9 deletions(-) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index f4860ac..b25ca9d 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_ACCOUNT); + cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!cmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index a0e02b0..88feaa0 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -163,7 +163,7 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_ACCOUNT); + dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!dtab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index da75784..f1e5303 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -495,7 +495,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_ACCOUNT); + htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!htab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 49ef0ce..a64255e 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,8 +313,8 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = kmalloc_node(sizeof(struct bpf_cgroup_storage_map), - __GFP_ZERO | GFP_USER | __GFP_ACCOUNT, numa_node); + map = kzalloc_node(sizeof(struct bpf_cgroup_storage_map), + GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT, numa_node); if (!map) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index bd09290..5a629a1 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -372,7 +372,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) attr->map_type != BPF_MAP_TYPE_HASH) return ERR_PTR(-EINVAL); - offmap = kzalloc(sizeof(*offmap), GFP_USER); + offmap = kzalloc(sizeof(*offmap), GFP_USER | __GFP_NOWARN); if (!offmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 3fb54fe..df8062c 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -164,7 +164,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_ACCOUNT); + rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!rb_map) return ERR_PTR(-ENOMEM); diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 028813d..763d771 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,7 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = kzalloc(sizeof(*stab), GFP_USER | __GFP_ACCOUNT); + stab = kzalloc(sizeof(*stab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!stab) return ERR_PTR(-ENOMEM); @@ -1076,7 +1076,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_ACCOUNT); + htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!htab) return ERR_PTR(-ENOMEM); From patchwork Wed Aug 10 15:18:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 270B7C00140 for ; Wed, 10 Aug 2022 15:18:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B8C706B0075; Wed, 10 Aug 2022 11:18:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B14066B0078; Wed, 10 Aug 2022 11:18:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 967BD8E0001; Wed, 10 Aug 2022 11:18:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8646A6B0075 for ; Wed, 10 Aug 2022 11:18:53 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 507541C6CE8 for ; Wed, 10 Aug 2022 15:18:53 +0000 (UTC) X-FDA: 79784040546.01.0C78B32 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf06.hostedemail.com (Postfix) with ESMTP id 75346180034 for ; Wed, 10 Aug 2022 15:18:52 +0000 (UTC) Received: by mail-pl1-f180.google.com with SMTP id 17so14503114plj.10 for ; Wed, 10 Aug 2022 08:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=kdwwAaZR+asNE4VplBgkNqPcX927kxyG8S7wtrO1tJQ=; b=DtMIAdfYNd+6krVLUt0b4W/LijFJKwIGp5JOfCRmrJJiDP2+FSE8vqIXJXTQFl+LYe STuuDOcSIZPTH3lx0BuajtH5E6c6X347lrKXrVdd0FeiWDf8CqFkjbdjHuvUiB8pFwIU WZH8a3UrSr/nbWO5ADjfEVtStTblyelcn5ZnblDGovut7xU8YCkIY73LbjWk0W7voxPh w4uufLb0JKRjOSAwPLaZIMigZMnp8piOdHTxt9t87eEzu766zAKQyGvWjtujB8sLUzRI e152lm7FCAJii4mblD/u42vu51bEDhWS2V13RtGmiCWjKgcwt9uudhoMpdbzSNxVy5Sc u7vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=kdwwAaZR+asNE4VplBgkNqPcX927kxyG8S7wtrO1tJQ=; b=knBQUx/dqtrlTfdD1so4bqtDUuZe/XF4Z2ijQnrmPHhrurTzNM5UXC3tQESQWzEkB6 H9TgjpSWhiW84gjBWXS0B8fBGPoC1PPIe8plcEsyhJxO4AA6YCUdg3D0vkSdCxszxMTd 73g4bchkuwvg5dYn/3cU2CrEhX5D/EJOEOpXDgfIhyqBU4D5LMORy0Pqft2u2uQspCpV hE/6TElGOzMBT7lZ3mAuM1uAWI5z8KF6wcvHoa41dCjwWjwGCP6J9swwkRhymZuPraCG Lv9JTpHdZDjoK0lQ9VLOX4vLgDpwtwLwtxEhfiCpUub2BpXhHIepmT4M51aALXpdQefv htVg== X-Gm-Message-State: ACgBeo1xNifafsgzgkd9ij60cI349H7plMVjJgsvPBWC6u4dINpsHF7l 5lUJEsB3UOGRgKDdnJvxUJPilR7ZAQs3cEzoQx4= X-Google-Smtp-Source: AA6agR5ZP1E2zsKWFzCfFu47w/Pccm+vqmJRsO9y2UBclWBS6HzwGq8nnjbZmM8oMmz9dZEFYXG+2w== X-Received: by 2002:a17:90b:17c9:b0:1f3:3a7c:a3a7 with SMTP id me9-20020a17090b17c900b001f33a7ca3a7mr4437962pjb.76.1660144731574; Wed, 10 Aug 2022 08:18:51 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:50 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 04/15] bpf: Use bpf_map_area_alloc consistently on bpf map creation Date: Wed, 10 Aug 2022 15:18:29 +0000 Message-Id: <20220810151840.16394-5-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DtMIAdfY; spf=pass (imf06.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144732; a=rsa-sha256; cv=none; b=5RS7PVTOlz2S03bV+fe4to7W1Emmi+vPHo7IbJgiEyz/JupJyAxVUiWL9s77/DUhF4VA37 NzoPRV09BeCKdqwp4XEiHW2TX5UZXG2M7XjyxSY2N/7WKEkL9VoXA2nHstperNjtAbuvIo 452BFPNSmdA6W/qOmzW544S9IFZLTc4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144732; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kdwwAaZR+asNE4VplBgkNqPcX927kxyG8S7wtrO1tJQ=; b=tuYKrZQfLqH469wiBCsmggi+nsoFk0AUuHM/LvIZ822uFDiVxQRRLRchgZw5FTW0a9YqKT LGNDuakgxLNO7ZHyQE8VfxPIsRRdkWvesMIpQFbWqLscTPZlUYBBi55uFFyv/G09RoTgba GSQL7ip2e/P5PiXYYZ/LnArfkrGT2do= Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DtMIAdfY; spf=pass (imf06.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: rdsodc1ygpuozsrnq1m4p456jjawotec X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 75346180034 X-Rspam-User: X-HE-Tag: 1660144732-537443 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's use the generic helper bpf_map_area_alloc() instead of the open-coded kzalloc helpers in bpf maps creation path. Signed-off-by: Yafang Shao --- kernel/bpf/bpf_local_storage.c | 6 +++--- kernel/bpf/cpumap.c | 6 +++--- kernel/bpf/devmap.c | 6 +++--- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 5 ++--- kernel/bpf/lpm_trie.c | 4 ++-- kernel/bpf/offload.c | 6 +++--- kernel/bpf/ringbuf.c | 6 +++--- net/core/sock_map.c | 12 ++++++------ 9 files changed, 28 insertions(+), 29 deletions(-) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 8ce40fd..4ee2e72 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -582,7 +582,7 @@ void bpf_local_storage_map_free(struct bpf_local_storage_map *smap, synchronize_rcu(); kvfree(smap->buckets); - kfree(smap); + bpf_map_area_free(smap); } int bpf_local_storage_map_alloc_check(union bpf_attr *attr) @@ -610,7 +610,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) unsigned int i; u32 nbuckets; - smap = kzalloc(sizeof(*smap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap = bpf_map_area_alloc(sizeof(*smap), NUMA_NO_NODE); if (!smap) return ERR_PTR(-ENOMEM); bpf_map_init_from_attr(&smap->map, attr); @@ -623,7 +623,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!smap->buckets) { - kfree(smap); + bpf_map_area_free(smap); return ERR_PTR(-ENOMEM); } diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index b25ca9d..b5ba34d 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = kzalloc(sizeof(*cmap), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + cmap = bpf_map_area_alloc(sizeof(*cmap), NUMA_NO_NODE); if (!cmap) return ERR_PTR(-ENOMEM); @@ -118,7 +118,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) return &cmap->map; free_cmap: - kfree(cmap); + bpf_map_area_free(cmap); return ERR_PTR(err); } @@ -623,7 +623,7 @@ static void cpu_map_free(struct bpf_map *map) __cpu_map_entry_replace(cmap, i, NULL); /* call_rcu */ } bpf_map_area_free(cmap->cpu_map); - kfree(cmap); + bpf_map_area_free(cmap); } /* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 88feaa0..f9a87dc 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -163,13 +163,13 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = kzalloc(sizeof(*dtab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + dtab = bpf_map_area_alloc(sizeof(*dtab), NUMA_NO_NODE); if (!dtab) return ERR_PTR(-ENOMEM); err = dev_map_init_map(dtab, attr); if (err) { - kfree(dtab); + bpf_map_area_free(dtab); return ERR_PTR(err); } @@ -240,7 +240,7 @@ static void dev_map_free(struct bpf_map *map) bpf_map_area_free(dtab->netdev_map); } - kfree(dtab); + bpf_map_area_free(dtab); } static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index f1e5303..8392f7f 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -495,7 +495,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); if (!htab) return ERR_PTR(-ENOMEM); @@ -579,7 +579,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) bpf_map_area_free(htab->buckets); free_htab: lockdep_unregister_key(&htab->lockdep_key); - kfree(htab); + bpf_map_area_free(htab); return ERR_PTR(err); } @@ -1496,7 +1496,7 @@ static void htab_map_free(struct bpf_map *map) for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); lockdep_unregister_key(&htab->lockdep_key); - kfree(htab); + bpf_map_area_free(htab); } static void htab_map_seq_show_elem(struct bpf_map *map, void *key, diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index a64255e..098cf33 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,8 +313,7 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = kzalloc_node(sizeof(struct bpf_cgroup_storage_map), - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT, numa_node); + map = bpf_map_area_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node); if (!map) return ERR_PTR(-ENOMEM); @@ -346,7 +345,7 @@ static void cgroup_storage_map_free(struct bpf_map *_map) WARN_ON(!RB_EMPTY_ROOT(&map->root)); WARN_ON(!list_empty(&map->list)); - kfree(map); + bpf_map_area_free(map); } static int cgroup_storage_delete_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index d789e3b..d833496 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -558,7 +558,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) attr->value_size > LPM_VAL_SIZE_MAX) return ERR_PTR(-EINVAL); - trie = kzalloc(sizeof(*trie), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + trie = bpf_map_area_alloc(sizeof(*trie), NUMA_NO_NODE); if (!trie) return ERR_PTR(-ENOMEM); @@ -609,7 +609,7 @@ static void trie_free(struct bpf_map *map) } out: - kfree(trie); + bpf_map_area_free(trie); } static int trie_get_next_key(struct bpf_map *map, void *_key, void *_next_key) diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 5a629a1..13e4efc 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -372,7 +372,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) attr->map_type != BPF_MAP_TYPE_HASH) return ERR_PTR(-EINVAL); - offmap = kzalloc(sizeof(*offmap), GFP_USER | __GFP_NOWARN); + offmap = bpf_map_area_alloc(sizeof(*offmap), NUMA_NO_NODE); if (!offmap) return ERR_PTR(-ENOMEM); @@ -404,7 +404,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) err_unlock: up_write(&bpf_devs_lock); rtnl_unlock(); - kfree(offmap); + bpf_map_area_free(offmap); return ERR_PTR(err); } @@ -428,7 +428,7 @@ void bpf_map_offload_map_free(struct bpf_map *map) up_write(&bpf_devs_lock); rtnl_unlock(); - kfree(offmap); + bpf_map_area_free(offmap); } int bpf_map_offload_lookup_elem(struct bpf_map *map, void *key, void *value) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index df8062c..b483aea 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -164,7 +164,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = kzalloc(sizeof(*rb_map), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + rb_map = bpf_map_area_alloc(sizeof(*rb_map), NUMA_NO_NODE); if (!rb_map) return ERR_PTR(-ENOMEM); @@ -172,7 +172,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); if (!rb_map->rb) { - kfree(rb_map); + bpf_map_area_free(rb_map); return ERR_PTR(-ENOMEM); } @@ -199,7 +199,7 @@ static void ringbuf_map_free(struct bpf_map *map) rb_map = container_of(map, struct bpf_ringbuf_map, map); bpf_ringbuf_free(rb_map->rb); - kfree(rb_map); + bpf_map_area_free(rb_map); } static void *ringbuf_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 763d771..d0c4338 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,7 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = kzalloc(sizeof(*stab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + stab = bpf_map_area_alloc(sizeof(*stab), NUMA_NO_NODE); if (!stab) return ERR_PTR(-ENOMEM); @@ -52,7 +52,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) sizeof(struct sock *), stab->map.numa_node); if (!stab->sks) { - kfree(stab); + bpf_map_area_free(stab); return ERR_PTR(-ENOMEM); } @@ -361,7 +361,7 @@ static void sock_map_free(struct bpf_map *map) synchronize_rcu(); bpf_map_area_free(stab->sks); - kfree(stab); + bpf_map_area_free(stab); } static void sock_map_release_progs(struct bpf_map *map) @@ -1076,7 +1076,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = kzalloc(sizeof(*htab), GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); if (!htab) return ERR_PTR(-ENOMEM); @@ -1106,7 +1106,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) return &htab->map; free_htab: - kfree(htab); + bpf_map_area_free(htab); return ERR_PTR(err); } @@ -1159,7 +1159,7 @@ static void sock_hash_free(struct bpf_map *map) synchronize_rcu(); bpf_map_area_free(htab->buckets); - kfree(htab); + bpf_map_area_free(htab); } static void *sock_hash_lookup_sys(struct bpf_map *map, void *key) From patchwork Wed Aug 10 15:18:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22F4DC25B0C for ; Wed, 10 Aug 2022 15:18:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2BAD6B0078; Wed, 10 Aug 2022 11:18:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B5FB6B007B; Wed, 10 Aug 2022 11:18:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B9888E0001; Wed, 10 Aug 2022 11:18:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 683B06B0078 for ; Wed, 10 Aug 2022 11:18:55 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3B4CF1C385F for ; Wed, 10 Aug 2022 15:18:55 +0000 (UTC) X-FDA: 79784040630.20.4D21CC9 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf25.hostedemail.com (Postfix) with ESMTP id E1AC9A0062 for ; Wed, 10 Aug 2022 15:18:54 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id q9-20020a17090a2dc900b001f58bcaca95so2495102pjm.3 for ; Wed, 10 Aug 2022 08:18:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=bZ/Wn/pmwYaPcD9cZLDVbMlygpn2TOkNAIPIYOEl18Y=; b=UZqCbeSc22sRQWZd+RhChsE4dWDPqNKHL76pEEfMY+wqnZlJvi6cRD7ua3Gaa60Hx2 CAe+jsbYkYFPiO2urZ+gNwzPkswhX8P0vMpj8kKJGHPQ+3whiPae54/J0ZRSQ/HUIw4B GITX56ZlyJ5jtzQDQa12XN0EVx46NDz6G7X3il4CfvEWFw0pzGMhzGX0FQfo2O4I5fl0 6vwdFXlgkQtZdunmCOOYvrWeJucbUnlSiHYu5+Qqa+6oExySt+Nbsc0oR8SefojbHSD9 iIEse4WYr2CbqGdES+DpvJs+NgjMIojCwVyx/xCV9KBNBYjgwRrPd6c0jRxVHhGKeHt+ M59Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=bZ/Wn/pmwYaPcD9cZLDVbMlygpn2TOkNAIPIYOEl18Y=; b=bqmo7T9RJusjRbBm+laDfJZ9sZWtt9jRLMXeaK3yPVnY5kNQMMCrXULBMtl54fskOe jtN9hVckOUEVApnGJvj2UON5Ho+ubD3XxWB37EZvloJ3AaD9jGM+a7Y2TmgHQMci+jgR w+8URGXIueve0GNSxjBZStErYS8Ln9H4Bp8Pd9Q7HCriytm2b4U+BmglPwueM1JcCi0Y d5FByaJ1f6caV/O0d2DPU7FjTLvVxD4X2Z47gLdGReVDpgRfNERvgwHK6BZMXnbS9Mvv hWCoFh3EbISOTxG/XmlsdDiqgJJ1AZSbSgjCoc1d+regCQBhpwz1cBLP75wVo/pkClHu CDtg== X-Gm-Message-State: ACgBeo1vl0oHsN8q4EQn0KHQ5fk8B9CIfOhGd5sLyrh0Sh23gyZgZc9P 2CHsmNzM1C8mHQfvhMYe4P4= X-Google-Smtp-Source: AA6agR7zMPKqwoHrBCMJ/QFxg4BZHomKAMIsFVdSEpWqsvP46BAFZwZnz8InOPJ0E3RrAROE/EQdgA== X-Received: by 2002:a17:902:ccd0:b0:16c:5d4f:99f3 with SMTP id z16-20020a170902ccd000b0016c5d4f99f3mr27988073ple.139.1660144733978; Wed, 10 Aug 2022 08:18:53 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:53 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 05/15] bpf: Fix incorrect mem_cgroup_put Date: Wed, 10 Aug 2022 15:18:30 +0000 Message-Id: <20220810151840.16394-6-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144735; a=rsa-sha256; cv=none; b=RXlclK2+6n58a+m94WmQqXe5E2N9UcgmPbLcoolXYZmurHXHWQd2VzF+pNAekU4u/YYUZz T//LdvtkE3phE/tIcQFtBiaJPgkIN7g33ovHgUDzp6J5rCxP70Ot4ORyrzC1mU5gZghkO7 Asl1K5Fb8ZkWM23SLMfl9+zBTWrgCMk= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UZqCbeSc; spf=pass (imf25.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144735; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bZ/Wn/pmwYaPcD9cZLDVbMlygpn2TOkNAIPIYOEl18Y=; b=TTkwwI9/x4QtbNIYyRQYFhTDYy83ceTGMsp7hU7HrNgTqmQmPlj4JS9v0ZbcjiJyUN7PBL 0b0/+To9hvdl7IsjWawSoYg9nvgf9lBzUGkcILWdmUcFn1zJD0YTPLPQsL5lPryaytSLrK cIqkxoQaWJLB6TnGzcgfoqcLiJHHlz4= X-Rspamd-Queue-Id: E1AC9A0062 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UZqCbeSc; spf=pass (imf25.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Stat-Signature: ubkgg1n5fgy7inqmkfzozfas3ai4xoae X-Rspamd-Server: rspam05 X-HE-Tag: 1660144734-900506 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The memcg may be the root_mem_cgroup, in which case we shouldn't put it. So a new helper bpf_map_put_memcg() is introduced to pair with bpf_map_get_memcg(). Fixes: 4201d9ab3e42 ("bpf: reparent bpf maps on memcg offlining") Cc: Roman Gushchin Cc: Shakeel Butt Signed-off-by: Yafang Shao --- kernel/bpf/syscall.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 83c7136..51ab8b1 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -441,6 +441,14 @@ static struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) return root_mem_cgroup; } +static void bpf_map_put_memcg(struct mem_cgroup *memcg) +{ + if (mem_cgroup_is_root(memcg)) + return; + + mem_cgroup_put(memcg); +} + void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node) { @@ -451,7 +459,7 @@ void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, old_memcg = set_active_memcg(memcg); ptr = kmalloc_node(size, flags | __GFP_ACCOUNT, node); set_active_memcg(old_memcg); - mem_cgroup_put(memcg); + bpf_map_put_memcg(memcg); return ptr; } @@ -465,7 +473,7 @@ void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) old_memcg = set_active_memcg(memcg); ptr = kzalloc(size, flags | __GFP_ACCOUNT); set_active_memcg(old_memcg); - mem_cgroup_put(memcg); + bpf_map_put_memcg(memcg); return ptr; } @@ -480,7 +488,7 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, old_memcg = set_active_memcg(memcg); ptr = __alloc_percpu_gfp(size, align, flags | __GFP_ACCOUNT); set_active_memcg(old_memcg); - mem_cgroup_put(memcg); + bpf_map_put_memcg(memcg); return ptr; } From patchwork Wed Aug 10 15:18:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06082C00140 for ; Wed, 10 Aug 2022 15:19:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9AB706B0071; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F3216B0080; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6331A8E0001; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 48B796B0071 for ; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3BBE5120602 for ; Wed, 10 Aug 2022 15:18:57 +0000 (UTC) X-FDA: 79784040714.15.85C2DA4 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf14.hostedemail.com (Postfix) with ESMTP id DB08110005B for ; Wed, 10 Aug 2022 15:18:56 +0000 (UTC) Received: by mail-pf1-f176.google.com with SMTP id b133so13999023pfb.6 for ; Wed, 10 Aug 2022 08:18:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=ZnOgYzkZxt9WSL7A8CyXVeY/jkqQttbvdChy+LL/htY=; b=RdpQ34Fx0/VGaTzpvtFOmvGQm29Ipm2n0EBOGETRHgm3A09EU/Tb8O7IXin+zzvmEY PO6QZnDySzHolaWexsbO1IBZsdUkmtWLOTzpu3gJ3yJCFgYjuMfkBHsY7LQ6Q320Ja9o kXdGZ17ElSsWPXl9XcLt+P87a7rEonVkYIttScWZ5tVpGLSgw4r3Q3B1IaydFzkvDmSy XHE8q5rZGfq60l/BXlweZ54O5esZbCdWdiB7dCUd2e2Ut+usySL9vix+BUwtqNnrxUGS RtsdZeWMXd91o9UeL3YQURQ+byH41xnKdPyiwplFrghjMbdaROGzdiU7YelzziDYMQcG N2Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=ZnOgYzkZxt9WSL7A8CyXVeY/jkqQttbvdChy+LL/htY=; b=L/oAkXU3QkkgpOz9hr5IJN3pO56FXvcMMJTdVfTd5QG++ozhYeYVuwnwfX4HLfOtD9 RzYpBjEZMYoXQg2so/iUue9P5f+xImGotiOd68wg6K1MDL/7c50OZWao8AYq/MzBh/Mc EVegaDyguFQ0FG1KhyxylebAzoaIqVfKeDU9qixqSXa/EnUOE0CjWcRpobGyQA+Pz44e QfAtjfvHof1rjwbsiUIBsOjQXvDqEDbsUQaFLMAzE9DZ7/HRR4UvHTqtYEApLTnaXiUl V/9mh4tKyDErtWpigILTlnoLrJdC5iHdlLtjKUZql5zg5L2kCgyOEBoRJpkmoiYZILUX v8OA== X-Gm-Message-State: ACgBeo2n4aKSCtOWw/miSp5MBTC5Cq/hBUgD0lGL9FtxWR9ATgseDZc0 SsWbW/Y1uy6lxgl0uImoMKM= X-Google-Smtp-Source: AA6agR7T+mXtw1Yp9VTWXrsQVNEQkxkQW0ET8/XsMOMA6C0eOOdQPQx2ixg3OaV1FWIqCJc3hFW1fw== X-Received: by 2002:a63:4c0d:0:b0:41a:77fe:2bc8 with SMTP id z13-20020a634c0d000000b0041a77fe2bc8mr23165203pga.82.1660144735958; Wed, 10 Aug 2022 08:18:55 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:55 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 06/15] bpf: Define bpf_map_{get,put}_memcg for !CONFIG_MEMCG_KMEM Date: Wed, 10 Aug 2022 15:18:31 +0000 Message-Id: <20220810151840.16394-7-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=RdpQ34Fx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144736; a=rsa-sha256; cv=none; b=OmXI9dowPbFPgqeHBT0MCDJ0mhtHOTLq3T4e6S57kxwXmdKtWPPwnqrmqhuPfnYLKvYQQt 4JGKFmII1JwEQ5ba25jgwc/l3hupxLQqOJZ8rC0J5TydbUUBxaGT+yXXf0jMoxQfgiuH5r 41cK/bP/ifnnwXWrIJJB5kyCw7iUNBE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZnOgYzkZxt9WSL7A8CyXVeY/jkqQttbvdChy+LL/htY=; b=pMcJs8f125yqADIHB2yRqXpkiYg+4yvAGxM1NPZ3n/ELKXHe1sKaFgYoUoQwP3oFSdueVu tTlRAH2WA3bxh//DRg0ptAOBwEDD5VFP5nDHvjkvlRCX7oE3PK9r/w0+nfTAljax3d34/P mrrO6IkL6076W1+lbavIyclwT5J1qoo= X-Rspamd-Server: rspam10 X-Rspam-User: Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=RdpQ34Fx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Queue-Id: DB08110005B X-Stat-Signature: 1tk79ogb67deucte3jqswru1cec7fhm4 X-HE-Tag: 1660144736-186478 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can use this helper when CONFIG_MEMCG_KMEM or CONFIG_MEMCG is not set. It also moves bpf_map_{get,put}_memcg into include/linux/bpf.h, so these two helpers can be used in other source files. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 29 +++++++++++++++++++++++++++++ include/linux/memcontrol.h | 10 ++++++++++ kernel/bpf/syscall.c | 16 ---------------- 3 files changed, 39 insertions(+), 16 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 20c26ae..fe3b565 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -27,6 +27,7 @@ #include #include #include +#include struct bpf_verifier_env; struct bpf_verifier_log; @@ -2571,4 +2572,32 @@ static inline void bpf_cgroup_atype_get(u32 attach_btf_id, int cgroup_atype) {} static inline void bpf_cgroup_atype_put(int cgroup_atype) {} #endif /* CONFIG_BPF_LSM */ +#ifdef CONFIG_MEMCG_KMEM +static inline struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) +{ + if (map->objcg) + return get_mem_cgroup_from_objcg(map->objcg); + + return root_mem_cgroup; +} + +static inline void bpf_map_put_memcg(struct mem_cgroup *memcg) +{ + if (mem_cgroup_is_root(memcg)) + return; + + mem_cgroup_put(memcg); +} + +#else +static inline struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) +{ + return root_memcg(); +} + +static inline void bpf_map_put_memcg(struct mem_cgroup *memcg) +{ +} +#endif + #endif /* _LINUX_BPF_H */ diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9ecead1..2f0a611 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -361,6 +361,11 @@ struct mem_cgroup { extern struct mem_cgroup *root_mem_cgroup; +static inline struct mem_cgroup *root_memcg(void) +{ + return root_mem_cgroup; +} + enum page_memcg_data_flags { /* page->memcg_data is a pointer to an objcgs vector */ MEMCG_DATA_OBJCGS = (1UL << 0), @@ -1138,6 +1143,11 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order, #define MEM_CGROUP_ID_SHIFT 0 #define MEM_CGROUP_ID_MAX 0 +static inline struct mem_cgroup *root_memcg(void) +{ + return NULL; +} + static inline struct mem_cgroup *folio_memcg(struct folio *folio) { return NULL; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 51ab8b1..19c3a81 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -433,22 +433,6 @@ static void bpf_map_release_memcg(struct bpf_map *map) obj_cgroup_put(map->objcg); } -static struct mem_cgroup *bpf_map_get_memcg(const struct bpf_map *map) -{ - if (map->objcg) - return get_mem_cgroup_from_objcg(map->objcg); - - return root_mem_cgroup; -} - -static void bpf_map_put_memcg(struct mem_cgroup *memcg) -{ - if (mem_cgroup_is_root(memcg)) - return; - - mem_cgroup_put(memcg); -} - void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node) { From patchwork Wed Aug 10 15:18:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E116C00140 for ; Wed, 10 Aug 2022 15:19:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA1F36B007B; Wed, 10 Aug 2022 11:18:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A29D18E0001; Wed, 10 Aug 2022 11:18:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87CC66B007E; Wed, 10 Aug 2022 11:18:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 76A806B007B for ; Wed, 10 Aug 2022 11:18:59 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 55836413DF for ; Wed, 10 Aug 2022 15:18:59 +0000 (UTC) X-FDA: 79784040798.15.6082CE8 Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf24.hostedemail.com (Postfix) with ESMTP id C316E180085 for ; Wed, 10 Aug 2022 15:18:58 +0000 (UTC) Received: by mail-pg1-f180.google.com with SMTP id h132so14617828pgc.10 for ; Wed, 10 Aug 2022 08:18:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=ZuUr0Lmb/ZOxk9PY5AfEZVtv5LOeYubF7wGlzoETmHE=; b=OHEQuhGa2E+9meTBDvo4R2MaXiYLv46EQQrHAGQ3JbFSFXuMFRDQWOZD23mUAky4KU OdJtLuJZjuzRxOu088twYy4nJQYluaHRMZjvhG9loN/vIeefkKoHQU2tcLgaFdnStZzO R6v480SZuZVpcD9ZCiG9YqtXNbWy/GenPH0Ngh4lznaoOhClBYV13p7DV3XBf36nhrjx N4tXmzGujKFH87N9VHq7kDTXRcyBcicH1oNBbzayC8dGYYh5E2Ay8a9MmCxQJcwAA70S tpQosUDU+WzgHIpv/YGR4TygyIZnYDWpks/QOSEm5L3D8BKwmyqxQYdESo8Pq/KKJr7e qb9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=ZuUr0Lmb/ZOxk9PY5AfEZVtv5LOeYubF7wGlzoETmHE=; b=qkBujrpO/gX/u0VmmIcrBK/TjZAx2aCyWHPuCiVFr1xlHgoeEJWG1Dcb0OCZWkiKl6 +iD13ZKoirgRsXuiTzYlQ21Qi0mnoLViOaH4GdLbMLOH4GuFyg3j5yflMhUw6aViDmo3 3L8VM5GGEGjGftbRWaKWejF0UWW2z25zBrCINYQN6zq4J/z85i0TFQ0MibcsWNICqtQN xq0VvefhpfGCFJvgLI5P0NVJ9zZNu7SpSLCbXERdo2FjDIG19uHRvADvQrU936G9WUNr 5gFNv8AxOF9Wc6GTMVk1L1z/9ctQRNOWbpaR56Duc5JqwZ4b2Q2hW4VpRSDItf/c+5ct WZAA== X-Gm-Message-State: ACgBeo2UxswYOG2aKYCImaUIE80Q6S90HIizbDQjeuGeyAhdmvikjuEf B/38j1pyAVIPyPNx2gEf2n0= X-Google-Smtp-Source: AA6agR6ilBNkZuPCRzWfdrpskFMZWXABEGPKHGKS0BRmL0vru/PZc53aw592+RCpI5B3o2SiI/u8mA== X-Received: by 2002:a63:1624:0:b0:41a:9dea:1c80 with SMTP id w36-20020a631624000000b0041a9dea1c80mr23479619pgl.400.1660144737875; Wed, 10 Aug 2022 08:18:57 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:57 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 07/15] bpf: Call bpf_map_init_from_attr() immediately after map creation Date: Wed, 10 Aug 2022 15:18:32 +0000 Message-Id: <20220810151840.16394-8-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144738; a=rsa-sha256; cv=none; b=BEc8j9wYI5uAP83P3nfVmkbWvYglRjPQ6VpbRqMYHiDnUivd1CoE8k0C1JziUzpKy4B6gs 0kF/3TQhavGwGRqXqwG2Cg/g+9w7hniRWNsTNRLVjU1rYQM1yJCzp7CTQnoUhpGYHV4LHD m2tMbsT+GWhaSAEOa/xLmJo5QAG0bqo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=OHEQuhGa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZuUr0Lmb/ZOxk9PY5AfEZVtv5LOeYubF7wGlzoETmHE=; b=A6XPAU4TcoxopVPlu5x6x8uNAmkCfDATqzRVftMtOqVsbMKfWI3N5rr5bXxcf6kBIMZgdS sZeHQE/YrrBrFLPfisHB6YthHtCLGxH1AwkByITFoU1FN/1o7LFRrYm1xD/B8549juOHRS Wr1BmZAdItEF72i4M0DZycJVAtyNzfU= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C316E180085 X-Rspam-User: X-Stat-Signature: ba143aisjb6feewpwcuyasrhg19m7jju Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=OHEQuhGa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-HE-Tag: 1660144738-211549 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In order to make all other map related memory allocations been allocated after memcg is saved in the map, we should save the memcg immediately after map creation. But the map is created in bpf_map_area_alloc(), within which we can't get the related bpf_map (except with a pointer casting which may be error prone), so we can do it in bpf_map_init_from_attr(), which is used by all bpf maps. bpf_map_init_from_attr() is executed immediately after bpf_map_area_alloc() for almost all bpf maps except bpf_struct_ops, devmap and hashmap, so this patch changes these three maps. In the future we will change the return type of bpf_map_init_from_attr() from void to int for error cases, so put it immediately after bpf_map_area_alloc() will make it eary to handle the error case. Signed-off-by: Yafang Shao --- kernel/bpf/bpf_struct_ops.c | 2 +- kernel/bpf/devmap.c | 5 ++--- kernel/bpf/hashtab.c | 4 ++-- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 84b2d9d..36f24f8 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -624,6 +624,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) st_map->st_ops = st_ops; map = &st_map->map; + bpf_map_init_from_attr(map, attr); st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE); st_map->links = @@ -637,7 +638,6 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) mutex_init(&st_map->lock); set_vm_flush_reset_perms(st_map->image); - bpf_map_init_from_attr(map, attr); return map; } diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index f9a87dc..20decc7 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -127,9 +127,6 @@ static int dev_map_init_map(struct bpf_dtab *dtab, union bpf_attr *attr) */ attr->map_flags |= BPF_F_RDONLY_PROG; - - bpf_map_init_from_attr(&dtab->map, attr); - if (attr->map_type == BPF_MAP_TYPE_DEVMAP_HASH) { dtab->n_buckets = roundup_pow_of_two(dtab->map.max_entries); @@ -167,6 +164,8 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!dtab) return ERR_PTR(-ENOMEM); + bpf_map_init_from_attr(&dtab->map, attr); + err = dev_map_init_map(dtab, attr); if (err) { bpf_map_area_free(dtab); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 8392f7f..48dc04c 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -499,10 +499,10 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) if (!htab) return ERR_PTR(-ENOMEM); - lockdep_register_key(&htab->lockdep_key); - bpf_map_init_from_attr(&htab->map, attr); + lockdep_register_key(&htab->lockdep_key); + if (percpu_lru) { /* ensure each CPU's lru list has >=1 elements. * since we are at it, make each lru list has the same From patchwork Wed Aug 10 15:18:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1068C19F2A for ; Wed, 10 Aug 2022 15:19:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 742156B007D; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A7E06B0071; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C2006B007E; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 287876B0071 for ; Wed, 10 Aug 2022 11:19:02 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id F0BDF8129D for ; Wed, 10 Aug 2022 15:19:01 +0000 (UTC) X-FDA: 79784040882.22.EDAB69B Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by imf28.hostedemail.com (Postfix) with ESMTP id D51EEC017D for ; Wed, 10 Aug 2022 15:19:00 +0000 (UTC) Received: by mail-pg1-f173.google.com with SMTP id c24so9771284pgg.11 for ; Wed, 10 Aug 2022 08:19:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=pNR85emlY/Gqguy9oN4UnMeNs/m0sbawgAOOJ626Rfw=; b=HmdVUKvwrT1UViXpAx6fWYM+0ZDNfYOxmrnB30CiRN2rC2E1QjFvwLfOCJQU/FCenV BzKXikqkfl0qqYG9gE8ni0q2iC6kuVJwzGwrebWErVSXHJaTzNxPEhwkX5ohAL8Km3Mo mppRiY0aRuDL5ZlpvTfwn5wri7Q8DMUvQASlbYgftGXuLidGg30oyMO+mknusLYRoFVF LtVqKnuHCluZ3fshofQbMDpVszhy0KftU/O4yHE8vQ1Ulvv+xUAvcx2CcSDn2XyOJAiU NtUB+yuZ8JDzYVGpsRhjygJ5QnvQjC3jIT0hUlSE3qc91rGlE5O3lHKdWSlUVi7Cake7 7u9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=pNR85emlY/Gqguy9oN4UnMeNs/m0sbawgAOOJ626Rfw=; b=BciG22q0RvTBHgZTv07l2ZiS6BLiLUWOfOj6pxZubbaePPt7ZnZcRhyGyP6VL0orz6 5e9inlaLHaxZ/fqupW6v3ofbDEIcAmO2Pil2p7Sv4p9qGxpUYB6xPVMkG/BW/84cgOlU h2xIoSRotFIZdgeZyewMfAOOe7MloKtZDIfj37IVzPB4z6Phkt7iKuWaRhDn4L2oYmTu DyH6jzsHjChkx8cyQbEXE5iuIUG4H6WReTfD+scuTiGsY0YioMJYDyXPynTrbBW6Lbi3 r7+Sww9RjngzYH2UE06N31oN6gnwB1hS0duIpfRsEEpkTyqB6ovhEN/iMaB2Cd6WNm8a 1G2Q== X-Gm-Message-State: ACgBeo1HMXfjtOkit/7jQ7x+ueeGVWd42luDFipi+P2nd7E5gM5EeTMJ yNqJdtRCwGw+FYXj98WuO0s= X-Google-Smtp-Source: AA6agR4KzuyCbK60wPw25WLaEb7jmGLkH6an6vqpHHKoY4X/G3A61HKOHAwjgQDk0pQUhYHI9GJjkA== X-Received: by 2002:a63:ef54:0:b0:41a:56e7:aeb3 with SMTP id c20-20020a63ef54000000b0041a56e7aeb3mr22908648pgk.49.1660144739831; Wed, 10 Aug 2022 08:18:59 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.18.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:18:58 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 08/15] bpf: Save memcg in bpf_map_init_from_attr() Date: Wed, 10 Aug 2022 15:18:33 +0000 Message-Id: <20220810151840.16394-9-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144741; a=rsa-sha256; cv=none; b=5h4t08WdPWr7Jds1Z7/+C0V4xLE8nlTThb65zNuwAp40dzQBI/G4x64co9dkc2cfpw5Vok 1ulXCLW04xs2FiUA79cDkAOgMN/YwWvmFjqJLxnaME9E7Aw2br65QVvzEm1m/X0cRGf3PW Wv5esOajHom7jFfSQ/dGurTzjV+HRPE= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=HmdVUKvw; spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144741; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pNR85emlY/Gqguy9oN4UnMeNs/m0sbawgAOOJ626Rfw=; b=fsbHSTVPUhGdVIQHHVVgkMUIw7/d49y0C8x8LXE4qT6gEvKUqLG+aRl4pXph1A/1kKiNy3 HKfmye64Az8BMV61801EKyWHGYoWWDHrNYCsLXh+HgybTBLfnOfJinZNqO09D48cfi5j/Z Eaqm2cXOWDIhLBPAIAzvFNmqd0pvi4o= X-Rspamd-Queue-Id: D51EEC017D Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=HmdVUKvw; spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Stat-Signature: 76chjjopon64xtn3hur9493reseiaq61 X-Rspamd-Server: rspam05 X-HE-Tag: 1660144740-31324 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move bpf_map_save_memcg() into bpf_map_init_from_attr(), then all other map related memory allocation will be allocated after saving the memcg. And then we can get memcg from the map in the followup memory allocation. To pair with this change, bpf_map_release_memcg() is moved into bpf_map_area_free(). A new parameter struct bpf_map is introduced into bpf_map_area_free() for this purpose. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 2 +- kernel/bpf/arraymap.c | 8 +++--- kernel/bpf/bloom_filter.c | 2 +- kernel/bpf/bpf_local_storage.c | 4 +-- kernel/bpf/bpf_struct_ops.c | 6 ++--- kernel/bpf/cpumap.c | 6 ++--- kernel/bpf/devmap.c | 8 +++--- kernel/bpf/hashtab.c | 10 +++---- kernel/bpf/local_storage.c | 2 +- kernel/bpf/lpm_trie.c | 2 +- kernel/bpf/offload.c | 4 +-- kernel/bpf/queue_stack_maps.c | 2 +- kernel/bpf/reuseport_array.c | 2 +- kernel/bpf/ringbuf.c | 8 +++--- kernel/bpf/stackmap.c | 8 +++--- kernel/bpf/syscall.c | 60 ++++++++++++++++++++++-------------------- net/core/sock_map.c | 12 ++++----- net/xdp/xskmap.c | 2 +- 18 files changed, 76 insertions(+), 72 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index fe3b565..414f8b7 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1637,7 +1637,7 @@ struct bpf_prog *bpf_prog_get_type_dev(u32 ufd, enum bpf_prog_type type, void bpf_map_put(struct bpf_map *map); void *bpf_map_area_alloc(u64 size, int numa_node); void *bpf_map_area_mmapable_alloc(u64 size, int numa_node); -void bpf_map_area_free(void *base); +void bpf_map_area_free(void *base, struct bpf_map *map); bool bpf_map_write_active(const struct bpf_map *map); void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); int generic_map_lookup_batch(struct bpf_map *map, diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index d3e734b..9ce4d1b 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -147,7 +147,7 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) array->elem_size = elem_size; if (percpu && bpf_array_alloc_percpu(array)) { - bpf_map_area_free(array); + bpf_map_area_free(array, &array->map); return ERR_PTR(-ENOMEM); } @@ -430,9 +430,9 @@ static void array_map_free(struct bpf_map *map) bpf_array_free_percpu(array); if (array->map.map_flags & BPF_F_MMAPABLE) - bpf_map_area_free(array_map_vmalloc_addr(array)); + bpf_map_area_free(array_map_vmalloc_addr(array), map); else - bpf_map_area_free(array); + bpf_map_area_free(array, map); } static void array_map_seq_show_elem(struct bpf_map *map, void *key, @@ -774,7 +774,7 @@ static void fd_array_map_free(struct bpf_map *map) for (i = 0; i < array->map.max_entries; i++) BUG_ON(array->ptrs[i] != NULL); - bpf_map_area_free(array); + bpf_map_area_free(array, map); } static void *fd_array_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index b9ea539..e59064d 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -168,7 +168,7 @@ static void bloom_map_free(struct bpf_map *map) struct bpf_bloom_filter *bloom = container_of(map, struct bpf_bloom_filter, map); - bpf_map_area_free(bloom); + bpf_map_area_free(bloom, map); } static void *bloom_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 4ee2e72..77e075b 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -582,7 +582,7 @@ void bpf_local_storage_map_free(struct bpf_local_storage_map *smap, synchronize_rcu(); kvfree(smap->buckets); - bpf_map_area_free(smap); + bpf_map_area_free(smap, &smap->map); } int bpf_local_storage_map_alloc_check(union bpf_attr *attr) @@ -623,7 +623,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); if (!smap->buckets) { - bpf_map_area_free(smap); + bpf_map_area_free(smap, &smap->map); return ERR_PTR(-ENOMEM); } diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 36f24f8..9fb8ad1 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -577,10 +577,10 @@ static void bpf_struct_ops_map_free(struct bpf_map *map) if (st_map->links) bpf_struct_ops_map_put_progs(st_map); - bpf_map_area_free(st_map->links); + bpf_map_area_free(st_map->links, NULL); bpf_jit_free_exec(st_map->image); - bpf_map_area_free(st_map->uvalue); - bpf_map_area_free(st_map); + bpf_map_area_free(st_map->uvalue, NULL); + bpf_map_area_free(st_map, map); } static int bpf_struct_ops_map_alloc_check(union bpf_attr *attr) diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index b5ba34d..7de2ae6 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -118,7 +118,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) return &cmap->map; free_cmap: - bpf_map_area_free(cmap); + bpf_map_area_free(cmap, &cmap->map); return ERR_PTR(err); } @@ -622,8 +622,8 @@ static void cpu_map_free(struct bpf_map *map) /* bq flush and cleanup happens after RCU grace-period */ __cpu_map_entry_replace(cmap, i, NULL); /* call_rcu */ } - bpf_map_area_free(cmap->cpu_map); - bpf_map_area_free(cmap); + bpf_map_area_free(cmap->cpu_map, NULL); + bpf_map_area_free(cmap, map); } /* Elements are kept alive by RCU; either by rcu_read_lock() (from syscall) or diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 20decc7..3268ce7 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -168,7 +168,7 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) err = dev_map_init_map(dtab, attr); if (err) { - bpf_map_area_free(dtab); + bpf_map_area_free(dtab, &dtab->map); return ERR_PTR(err); } @@ -221,7 +221,7 @@ static void dev_map_free(struct bpf_map *map) } } - bpf_map_area_free(dtab->dev_index_head); + bpf_map_area_free(dtab->dev_index_head, NULL); } else { for (i = 0; i < dtab->map.max_entries; i++) { struct bpf_dtab_netdev *dev; @@ -236,10 +236,10 @@ static void dev_map_free(struct bpf_map *map) kfree(dev); } - bpf_map_area_free(dtab->netdev_map); + bpf_map_area_free(dtab->netdev_map, NULL); } - bpf_map_area_free(dtab); + bpf_map_area_free(dtab, &dtab->map); } static int dev_map_get_next_key(struct bpf_map *map, void *key, void *next_key) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 48dc04c..1b3653d 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -290,7 +290,7 @@ static void htab_free_elems(struct bpf_htab *htab) cond_resched(); } free_elems: - bpf_map_area_free(htab->elems); + bpf_map_area_free(htab->elems, NULL); } /* The LRU list has a lock (lru_lock). Each htab bucket has a lock @@ -576,10 +576,10 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) free_map_locked: for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); - bpf_map_area_free(htab->buckets); + bpf_map_area_free(htab->buckets, NULL); free_htab: lockdep_unregister_key(&htab->lockdep_key); - bpf_map_area_free(htab); + bpf_map_area_free(htab, &htab->map); return ERR_PTR(err); } @@ -1492,11 +1492,11 @@ static void htab_map_free(struct bpf_map *map) bpf_map_free_kptr_off_tab(map); free_percpu(htab->extra_elems); - bpf_map_area_free(htab->buckets); + bpf_map_area_free(htab->buckets, NULL); for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) free_percpu(htab->map_locked[i]); lockdep_unregister_key(&htab->lockdep_key); - bpf_map_area_free(htab); + bpf_map_area_free(htab, map); } static void htab_map_seq_show_elem(struct bpf_map *map, void *key, diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index 098cf33..c705d66 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -345,7 +345,7 @@ static void cgroup_storage_map_free(struct bpf_map *_map) WARN_ON(!RB_EMPTY_ROOT(&map->root)); WARN_ON(!list_empty(&map->list)); - bpf_map_area_free(map); + bpf_map_area_free(map, _map); } static int cgroup_storage_delete_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index d833496..fd99360 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -609,7 +609,7 @@ static void trie_free(struct bpf_map *map) } out: - bpf_map_area_free(trie); + bpf_map_area_free(trie, map); } static int trie_get_next_key(struct bpf_map *map, void *_key, void *_next_key) diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 13e4efc..c9941a9 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -404,7 +404,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) err_unlock: up_write(&bpf_devs_lock); rtnl_unlock(); - bpf_map_area_free(offmap); + bpf_map_area_free(offmap, &offmap->map); return ERR_PTR(err); } @@ -428,7 +428,7 @@ void bpf_map_offload_map_free(struct bpf_map *map) up_write(&bpf_devs_lock); rtnl_unlock(); - bpf_map_area_free(offmap); + bpf_map_area_free(offmap, map); } int bpf_map_offload_lookup_elem(struct bpf_map *map, void *key, void *value) diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index 8a5e060..f2ec0c4 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -92,7 +92,7 @@ static void queue_stack_map_free(struct bpf_map *map) { struct bpf_queue_stack *qs = bpf_queue_stack(map); - bpf_map_area_free(qs); + bpf_map_area_free(qs, map); } static int __queue_map_get(struct bpf_map *map, void *value, bool delete) diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index e2618fb..594cdb0 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -146,7 +146,7 @@ static void reuseport_array_free(struct bpf_map *map) * Once reaching here, all sk->sk_user_data is not * referencing this "array". "array" can be freed now. */ - bpf_map_area_free(array); + bpf_map_area_free(array, map); } static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index b483aea..74dd8dc 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -116,7 +116,7 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) err_free_pages: for (i = 0; i < nr_pages; i++) __free_page(pages[i]); - bpf_map_area_free(pages); + bpf_map_area_free(pages, NULL); return NULL; } @@ -172,7 +172,7 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); if (!rb_map->rb) { - bpf_map_area_free(rb_map); + bpf_map_area_free(rb_map, &rb_map->map); return ERR_PTR(-ENOMEM); } @@ -190,7 +190,7 @@ static void bpf_ringbuf_free(struct bpf_ringbuf *rb) vunmap(rb); for (i = 0; i < nr_pages; i++) __free_page(pages[i]); - bpf_map_area_free(pages); + bpf_map_area_free(pages, NULL); } static void ringbuf_map_free(struct bpf_map *map) @@ -199,7 +199,7 @@ static void ringbuf_map_free(struct bpf_map *map) rb_map = container_of(map, struct bpf_ringbuf_map, map); bpf_ringbuf_free(rb_map->rb); - bpf_map_area_free(rb_map); + bpf_map_area_free(rb_map, map); } static void *ringbuf_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 1adbe67..042b7d2 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -62,7 +62,7 @@ static int prealloc_elems_and_freelist(struct bpf_stack_map *smap) return 0; free_elems: - bpf_map_area_free(smap->elems); + bpf_map_area_free(smap->elems, NULL); return err; } @@ -120,7 +120,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) put_buffers: put_callchain_buffers(); free_smap: - bpf_map_area_free(smap); + bpf_map_area_free(smap, &smap->map); return ERR_PTR(err); } @@ -648,9 +648,9 @@ static void stack_map_free(struct bpf_map *map) { struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map); - bpf_map_area_free(smap->elems); + bpf_map_area_free(smap->elems, NULL); pcpu_freelist_destroy(&smap->freelist); - bpf_map_area_free(smap); + bpf_map_area_free(smap, map); put_callchain_buffers(); } diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 19c3a81..01d8d4a 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -293,6 +293,34 @@ static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, return err; } +#ifdef CONFIG_MEMCG_KMEM +static void bpf_map_save_memcg(struct bpf_map *map) +{ + /* Currently if a map is created by a process belonging to the root + * memory cgroup, get_obj_cgroup_from_current() will return NULL. + * So we have to check map->objcg for being NULL each time it's + * being used. + */ + map->objcg = get_obj_cgroup_from_current(); +} + +static void bpf_map_release_memcg(struct bpf_map *map) +{ + if (map->objcg) + obj_cgroup_put(map->objcg); +} + +#else +static void bpf_map_save_memcg(struct bpf_map *map) +{ +} + +static void bpf_map_release_memcg(struct bpf_map *map) +{ +} + +#endif + /* Please, do not use this function outside from the map creation path * (e.g. in map update path) without taking care of setting the active * memory cgroup (see at bpf_map_kmalloc_node() for example). @@ -344,8 +372,10 @@ void *bpf_map_area_mmapable_alloc(u64 size, int numa_node) return __bpf_map_area_alloc(size, numa_node, true); } -void bpf_map_area_free(void *area) +void bpf_map_area_free(void *area, struct bpf_map *map) { + if (map) + bpf_map_release_memcg(map); kvfree(area); } @@ -363,6 +393,7 @@ static u32 bpf_map_flags_retain_permanent(u32 flags) void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) { + bpf_map_save_memcg(map); map->map_type = attr->map_type; map->key_size = attr->key_size; map->value_size = attr->value_size; @@ -417,22 +448,6 @@ void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock) } #ifdef CONFIG_MEMCG_KMEM -static void bpf_map_save_memcg(struct bpf_map *map) -{ - /* Currently if a map is created by a process belonging to the root - * memory cgroup, get_obj_cgroup_from_current() will return NULL. - * So we have to check map->objcg for being NULL each time it's - * being used. - */ - map->objcg = get_obj_cgroup_from_current(); -} - -static void bpf_map_release_memcg(struct bpf_map *map) -{ - if (map->objcg) - obj_cgroup_put(map->objcg); -} - void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node) { @@ -477,14 +492,6 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, return ptr; } -#else -static void bpf_map_save_memcg(struct bpf_map *map) -{ -} - -static void bpf_map_release_memcg(struct bpf_map *map) -{ -} #endif static int bpf_map_kptr_off_cmp(const void *a, const void *b) @@ -605,7 +612,6 @@ static void bpf_map_free_deferred(struct work_struct *work) security_bpf_map_free(map); kfree(map->off_arr); - bpf_map_release_memcg(map); /* implementation dependent freeing, map_free callback also does * bpf_map_free_kptr_off_tab, if needed. */ @@ -1154,8 +1160,6 @@ static int map_create(union bpf_attr *attr) if (err) goto free_map_sec; - bpf_map_save_memcg(map); - err = bpf_map_new_fd(map, f_flags); if (err < 0) { /* failed to allocate fd. diff --git a/net/core/sock_map.c b/net/core/sock_map.c index d0c4338..e8f414a 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -52,7 +52,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) sizeof(struct sock *), stab->map.numa_node); if (!stab->sks) { - bpf_map_area_free(stab); + bpf_map_area_free(stab, &stab->map); return ERR_PTR(-ENOMEM); } @@ -360,8 +360,8 @@ static void sock_map_free(struct bpf_map *map) /* wait for psock readers accessing its map link */ synchronize_rcu(); - bpf_map_area_free(stab->sks); - bpf_map_area_free(stab); + bpf_map_area_free(stab->sks, NULL); + bpf_map_area_free(stab, map); } static void sock_map_release_progs(struct bpf_map *map) @@ -1106,7 +1106,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) return &htab->map; free_htab: - bpf_map_area_free(htab); + bpf_map_area_free(htab, &htab->map); return ERR_PTR(err); } @@ -1158,8 +1158,8 @@ static void sock_hash_free(struct bpf_map *map) /* wait for psock readers accessing its map link */ synchronize_rcu(); - bpf_map_area_free(htab->buckets); - bpf_map_area_free(htab); + bpf_map_area_free(htab->buckets, NULL); + bpf_map_area_free(htab, map); } static void *sock_hash_lookup_sys(struct bpf_map *map, void *key) diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index acc8e52..5abb87e 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -90,7 +90,7 @@ static void xsk_map_free(struct bpf_map *map) struct xsk_map *m = container_of(map, struct xsk_map, map); synchronize_net(); - bpf_map_area_free(m); + bpf_map_area_free(m, map); } static int xsk_map_get_next_key(struct bpf_map *map, void *key, void *next_key) From patchwork Wed Aug 10 15:18:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9480C25B07 for ; Wed, 10 Aug 2022 15:19:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD9436B007E; Wed, 10 Aug 2022 11:19:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D37686B0080; Wed, 10 Aug 2022 11:19:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8A826B0081; Wed, 10 Aug 2022 11:19:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9D72E6B007E for ; Wed, 10 Aug 2022 11:19:04 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 76F9612143E for ; Wed, 10 Aug 2022 15:19:04 +0000 (UTC) X-FDA: 79784041008.17.2D9136F Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf21.hostedemail.com (Postfix) with ESMTP id CFFCE1C01AE for ; Wed, 10 Aug 2022 15:19:02 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id ha11so15084319pjb.2 for ; Wed, 10 Aug 2022 08:19:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=LSKn9EI6EEnZFa2cfczgY3SUVwQTryiMyelz91Pffy4=; b=PN2pGwpdHxlb6yB5Yp5TxvpbSVZBz+GlgHanyE0vM5zFB4vn5e+6ac+M8a+98ai6m1 ZNGLHKA0/LgIx0zBqxxguzGa7AbOBdkXzRx6w4rokOio316MobYA5RlDRt08BXar8vG1 zTV3wru95o/9NOREx99SHCX9+QPmtT1SiLzgxFgM6lsekcLi7GGPGnHZm2eDQM6BnQtQ bJfssLS56sSqQnYEwAkDZ+V3QlCaxMo4s5NK+/vA/qrTJJ5Nf6KITtoFs8zYFWH8KYXR 5jy5oXGdGlSXy9fyhFL+b2x1ato+URFcybDeip3pLv5s6UPbC8AfNiQIotVqZoUKdDgu Iidg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=LSKn9EI6EEnZFa2cfczgY3SUVwQTryiMyelz91Pffy4=; b=6e8MLoRklqbstg9YEiIJ5ZlNlQC8Rt60j9WsOr6lGZPDbBRkZ/0C77kYt2c671Xs/q zhaCn9Gc4nPFPU+istpKypeLW/nFuwgK2gRs13RtPn+/4iDQpRiNEmdjL74ZDFLM1hBa EKld6HPf7/HekyQwUUdMaS54/BrMyZVVena4IyjE4eBWccuu8UTYlSSdTOmssWjWzZSM zGsX6N8oWCSzW5zZZN4zfos2ebnJ0tmIcxh7fcAPiYSQoH6mUUrUN+QOT9bq1TPkCfre yYyA5eT2TBVQqpZPD+zth3iQZwWlWIXGKZEeWEV2vVxmCte+ynziF9JEguVaSx6hy1yM CnNw== X-Gm-Message-State: ACgBeo3zxBs16+QNeOVJEcd4Kmf5GErB7CmdnCThZ54SQCpXVH/BfULX fm0WySDzOI6EhtmzsKfsSlE= X-Google-Smtp-Source: AA6agR5gel1asRtrz0hDm33HYMmdHPEeeNLkKS/zcYxXY+zSnn3ot6p8qUQVUW8uOclZEm8cl6AVyw== X-Received: by 2002:a17:90b:1b0c:b0:1f5:1041:a4f3 with SMTP id nu12-20020a17090b1b0c00b001f51041a4f3mr4359739pjb.234.1660144741883; Wed, 10 Aug 2022 08:19:01 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:01 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 09/15] bpf: Use scoped-based charge in bpf_map_area_alloc Date: Wed, 10 Aug 2022 15:18:34 +0000 Message-Id: <20220810151840.16394-10-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144743; a=rsa-sha256; cv=none; b=Sfsf5nbUXgnybdD3Dyu9lfVlqLfH+Xz1Y75HWNgSA0Wr1yPFxQ/yei4CzrgHJ1skv9gmVS o/XhPBCslABlO4Q4ihfl8Y1aC7NqPuGzN2688QqNel8QR7vXyCFDu0nEgbsVimk3Ii+QnU JbkubcchWOcvpy7a3bQsXjinhDoTgwc= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PN2pGwpd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144743; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LSKn9EI6EEnZFa2cfczgY3SUVwQTryiMyelz91Pffy4=; b=OwlX3k142C6hA7r7J/7BY+bJj78W4ogFX2h3Rt0a5/Kjt9ZqXgYoSMWbHLvqd7EEUsseW+ gq4g6YX+3tsUlXOe3cHklFK/iyF/SBi5DtzdVc2Q1o3f9AdS5qsCkG6u35Jo+nPQMUoSe2 3UAWLeYKf0voTGkoVIlS4b3okO4rc2U= X-Stat-Signature: e9ou5464gz99i8p3hb76y1w87h75fw6u X-Rspamd-Queue-Id: CFFCE1C01AE Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PN2pGwpd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf21.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1660144742-72018 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently bpf_map_area_alloc() is used to allocate a container of struct bpf_map or members in this container. To distinguish the map creation and the other case, a new parameter struct bpf_map is added into bpf_map_area_alloc(). Then for the non-map-creation case, we could get the memcg from the map instead of using the current memcg. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 2 +- kernel/bpf/arraymap.c | 2 +- kernel/bpf/bloom_filter.c | 2 +- kernel/bpf/bpf_local_storage.c | 2 +- kernel/bpf/bpf_struct_ops.c | 6 +++--- kernel/bpf/cpumap.c | 5 +++-- kernel/bpf/devmap.c | 13 ++++++++----- kernel/bpf/hashtab.c | 8 +++++--- kernel/bpf/local_storage.c | 2 +- kernel/bpf/lpm_trie.c | 2 +- kernel/bpf/offload.c | 2 +- kernel/bpf/queue_stack_maps.c | 2 +- kernel/bpf/reuseport_array.c | 2 +- kernel/bpf/ringbuf.c | 15 +++++++++------ kernel/bpf/stackmap.c | 5 +++-- kernel/bpf/syscall.c | 16 ++++++++++++++-- net/core/sock_map.c | 10 ++++++---- net/xdp/xskmap.c | 2 +- 18 files changed, 61 insertions(+), 37 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 414f8b7..d7485b7 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1635,7 +1635,7 @@ struct bpf_prog *bpf_prog_get_type_dev(u32 ufd, enum bpf_prog_type type, struct bpf_map * __must_check bpf_map_inc_not_zero(struct bpf_map *map); void bpf_map_put_with_uref(struct bpf_map *map); void bpf_map_put(struct bpf_map *map); -void *bpf_map_area_alloc(u64 size, int numa_node); +void *bpf_map_area_alloc(u64 size, int numa_node, struct bpf_map *map); void *bpf_map_area_mmapable_alloc(u64 size, int numa_node); void bpf_map_area_free(void *base, struct bpf_map *map); bool bpf_map_write_active(const struct bpf_map *map); diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 9ce4d1b..80974c5 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -135,7 +135,7 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) array = data + PAGE_ALIGN(sizeof(struct bpf_array)) - offsetof(struct bpf_array, value); } else { - array = bpf_map_area_alloc(array_size, numa_node); + array = bpf_map_area_alloc(array_size, numa_node, NULL); } if (!array) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index e59064d..6691f79 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -142,7 +142,7 @@ static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) } bitset_bytes = roundup(bitset_bytes, sizeof(unsigned long)); - bloom = bpf_map_area_alloc(sizeof(*bloom) + bitset_bytes, numa_node); + bloom = bpf_map_area_alloc(sizeof(*bloom) + bitset_bytes, numa_node, NULL); if (!bloom) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 77e075b..67ab249 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -610,7 +610,7 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) unsigned int i; u32 nbuckets; - smap = bpf_map_area_alloc(sizeof(*smap), NUMA_NO_NODE); + smap = bpf_map_area_alloc(sizeof(*smap), NUMA_NO_NODE, NULL); if (!smap) return ERR_PTR(-ENOMEM); bpf_map_init_from_attr(&smap->map, attr); diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 9fb8ad1..37ba5c0 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -618,7 +618,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) */ (vt->size - sizeof(struct bpf_struct_ops_value)); - st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE); + st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE, NULL); if (!st_map) return ERR_PTR(-ENOMEM); @@ -626,10 +626,10 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) map = &st_map->map; bpf_map_init_from_attr(map, attr); - st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE); + st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE, map); st_map->links = bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_links *), - NUMA_NO_NODE); + NUMA_NO_NODE, map); st_map->image = bpf_jit_alloc_exec(PAGE_SIZE); if (!st_map->uvalue || !st_map->links || !st_map->image) { bpf_struct_ops_map_free(map); diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index 7de2ae6..b593157 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -97,7 +97,7 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) attr->map_flags & ~BPF_F_NUMA_NODE) return ERR_PTR(-EINVAL); - cmap = bpf_map_area_alloc(sizeof(*cmap), NUMA_NO_NODE); + cmap = bpf_map_area_alloc(sizeof(*cmap), NUMA_NO_NODE, NULL); if (!cmap) return ERR_PTR(-ENOMEM); @@ -112,7 +112,8 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) /* Alloc array for possible remote "destination" CPUs */ cmap->cpu_map = bpf_map_area_alloc(cmap->map.max_entries * sizeof(struct bpf_cpu_map_entry *), - cmap->map.numa_node); + cmap->map.numa_node, + &cmap->map); if (!cmap->cpu_map) goto free_cmap; diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 3268ce7..807a4cd 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -89,12 +89,13 @@ struct bpf_dtab { static LIST_HEAD(dev_map_list); static struct hlist_head *dev_map_create_hash(unsigned int entries, - int numa_node) + int numa_node, + struct bpf_map *map) { int i; struct hlist_head *hash; - hash = bpf_map_area_alloc((u64) entries * sizeof(*hash), numa_node); + hash = bpf_map_area_alloc((u64) entries * sizeof(*hash), numa_node, map); if (hash != NULL) for (i = 0; i < entries; i++) INIT_HLIST_HEAD(&hash[i]); @@ -136,7 +137,8 @@ static int dev_map_init_map(struct bpf_dtab *dtab, union bpf_attr *attr) if (attr->map_type == BPF_MAP_TYPE_DEVMAP_HASH) { dtab->dev_index_head = dev_map_create_hash(dtab->n_buckets, - dtab->map.numa_node); + dtab->map.numa_node, + &dtab->map); if (!dtab->dev_index_head) return -ENOMEM; @@ -144,7 +146,8 @@ static int dev_map_init_map(struct bpf_dtab *dtab, union bpf_attr *attr) } else { dtab->netdev_map = bpf_map_area_alloc((u64) dtab->map.max_entries * sizeof(struct bpf_dtab_netdev *), - dtab->map.numa_node); + dtab->map.numa_node, + &dtab->map); if (!dtab->netdev_map) return -ENOMEM; } @@ -160,7 +163,7 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); - dtab = bpf_map_area_alloc(sizeof(*dtab), NUMA_NO_NODE); + dtab = bpf_map_area_alloc(sizeof(*dtab), NUMA_NO_NODE, NULL); if (!dtab) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 1b3653d..e9a6d2c4 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -332,7 +332,8 @@ static int prealloc_init(struct bpf_htab *htab) num_entries += num_possible_cpus(); htab->elems = bpf_map_area_alloc((u64)htab->elem_size * num_entries, - htab->map.numa_node); + htab->map.numa_node, + &htab->map); if (!htab->elems) return -ENOMEM; @@ -495,7 +496,7 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) struct bpf_htab *htab; int err, i; - htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); + htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE, NULL); if (!htab) return ERR_PTR(-ENOMEM); @@ -534,7 +535,8 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) err = -ENOMEM; htab->buckets = bpf_map_area_alloc(htab->n_buckets * sizeof(struct bucket), - htab->map.numa_node); + htab->map.numa_node, + &htab->map); if (!htab->buckets) goto free_htab; diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index c705d66..fcc7ece 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -313,7 +313,7 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) /* max_entries is not used and enforced to be 0 */ return ERR_PTR(-EINVAL); - map = bpf_map_area_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node); + map = bpf_map_area_alloc(sizeof(struct bpf_cgroup_storage_map), numa_node, NULL); if (!map) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index fd99360..3d329ae 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -558,7 +558,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) attr->value_size > LPM_VAL_SIZE_MAX) return ERR_PTR(-EINVAL); - trie = bpf_map_area_alloc(sizeof(*trie), NUMA_NO_NODE); + trie = bpf_map_area_alloc(sizeof(*trie), NUMA_NO_NODE, NULL); if (!trie) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index c9941a9..87c59da 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -372,7 +372,7 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) attr->map_type != BPF_MAP_TYPE_HASH) return ERR_PTR(-EINVAL); - offmap = bpf_map_area_alloc(sizeof(*offmap), NUMA_NO_NODE); + offmap = bpf_map_area_alloc(sizeof(*offmap), NUMA_NO_NODE, NULL); if (!offmap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index f2ec0c4..bf57e45 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -74,7 +74,7 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) size = (u64) attr->max_entries + 1; queue_size = sizeof(*qs) + size * attr->value_size; - qs = bpf_map_area_alloc(queue_size, numa_node); + qs = bpf_map_area_alloc(queue_size, numa_node, NULL); if (!qs) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index 594cdb0..52c7e77 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -158,7 +158,7 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) return ERR_PTR(-EPERM); /* allocate all map elements and zero-initialize them */ - array = bpf_map_area_alloc(struct_size(array, ptrs, attr->max_entries), numa_node); + array = bpf_map_area_alloc(struct_size(array, ptrs, attr->max_entries), numa_node, NULL); if (!array) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 74dd8dc..5eb7820 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -59,7 +59,8 @@ struct bpf_ringbuf_hdr { u32 pg_off; }; -static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) +static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node, + struct bpf_map *map) { const gfp_t flags = GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL | __GFP_NOWARN | __GFP_ZERO; @@ -89,7 +90,7 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) * user-space implementations significantly. */ array_size = (nr_meta_pages + 2 * nr_data_pages) * sizeof(*pages); - pages = bpf_map_area_alloc(array_size, numa_node); + pages = bpf_map_area_alloc(array_size, numa_node, map); if (!pages) return NULL; @@ -127,11 +128,12 @@ static void bpf_ringbuf_notify(struct irq_work *work) wake_up_all(&rb->waitq); } -static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node) +static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node, + struct bpf_map *map) { struct bpf_ringbuf *rb; - rb = bpf_ringbuf_area_alloc(data_sz, numa_node); + rb = bpf_ringbuf_area_alloc(data_sz, numa_node, map); if (!rb) return NULL; @@ -164,13 +166,14 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); #endif - rb_map = bpf_map_area_alloc(sizeof(*rb_map), NUMA_NO_NODE); + rb_map = bpf_map_area_alloc(sizeof(*rb_map), NUMA_NO_NODE, NULL); if (!rb_map) return ERR_PTR(-ENOMEM); bpf_map_init_from_attr(&rb_map->map, attr); - rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node); + rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node, + &rb_map->map); if (!rb_map->rb) { bpf_map_area_free(rb_map, &rb_map->map); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 042b7d2..9440fab 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -49,7 +49,8 @@ static int prealloc_elems_and_freelist(struct bpf_stack_map *smap) int err; smap->elems = bpf_map_area_alloc(elem_size * smap->map.max_entries, - smap->map.numa_node); + smap->map.numa_node, + &smap->map); if (!smap->elems) return -ENOMEM; @@ -100,7 +101,7 @@ static struct bpf_map *stack_map_alloc(union bpf_attr *attr) return ERR_PTR(-E2BIG); cost = n_buckets * sizeof(struct stack_map_bucket *) + sizeof(*smap); - smap = bpf_map_area_alloc(cost, bpf_map_attr_numa_node(attr)); + smap = bpf_map_area_alloc(cost, bpf_map_attr_numa_node(attr), NULL); if (!smap) return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 01d8d4a..02ce7e9 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -362,9 +362,21 @@ static void *__bpf_map_area_alloc(u64 size, int numa_node, bool mmapable) flags, numa_node, __builtin_return_address(0)); } -void *bpf_map_area_alloc(u64 size, int numa_node) +void *bpf_map_area_alloc(u64 size, int numa_node, struct bpf_map *map) { - return __bpf_map_area_alloc(size, numa_node, false); + struct mem_cgroup *memcg, *old_memcg; + void *ptr; + + if (!map) + return __bpf_map_area_alloc(size, numa_node, false); + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + ptr = __bpf_map_area_alloc(size, numa_node, false); + set_active_memcg(old_memcg); + bpf_map_put_memcg(memcg); + + return ptr; } void *bpf_map_area_mmapable_alloc(u64 size, int numa_node) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index e8f414a..2b3b24e 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -41,7 +41,7 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) attr->map_flags & ~SOCK_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); - stab = bpf_map_area_alloc(sizeof(*stab), NUMA_NO_NODE); + stab = bpf_map_area_alloc(sizeof(*stab), NUMA_NO_NODE, NULL); if (!stab) return ERR_PTR(-ENOMEM); @@ -50,7 +50,8 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) stab->sks = bpf_map_area_alloc((u64) stab->map.max_entries * sizeof(struct sock *), - stab->map.numa_node); + stab->map.numa_node, + &stab->map); if (!stab->sks) { bpf_map_area_free(stab, &stab->map); return ERR_PTR(-ENOMEM); @@ -1076,7 +1077,7 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) if (attr->key_size > MAX_BPF_STACK) return ERR_PTR(-E2BIG); - htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE); + htab = bpf_map_area_alloc(sizeof(*htab), NUMA_NO_NODE, NULL); if (!htab) return ERR_PTR(-ENOMEM); @@ -1093,7 +1094,8 @@ static struct bpf_map *sock_hash_alloc(union bpf_attr *attr) htab->buckets = bpf_map_area_alloc(htab->buckets_num * sizeof(struct bpf_shtab_bucket), - htab->map.numa_node); + htab->map.numa_node, + &htab->map); if (!htab->buckets) { err = -ENOMEM; goto free_htab; diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index 5abb87e..beb11fd 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -75,7 +75,7 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) numa_node = bpf_map_attr_numa_node(attr); size = struct_size(m, xsk_map, attr->max_entries); - m = bpf_map_area_alloc(size, numa_node); + m = bpf_map_area_alloc(size, numa_node, NULL); if (!m) return ERR_PTR(-ENOMEM); From patchwork Wed Aug 10 15:18:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1B83C19F2A for ; Wed, 10 Aug 2022 15:19:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 463636B0072; Wed, 10 Aug 2022 11:19:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 325A56B0080; Wed, 10 Aug 2022 11:19:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1031C8E0001; Wed, 10 Aug 2022 11:19:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E931D6B0075 for ; Wed, 10 Aug 2022 11:19:06 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C6A6A1C6D01 for ; Wed, 10 Aug 2022 15:19:06 +0000 (UTC) X-FDA: 79784041092.08.F6B7A7B Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf10.hostedemail.com (Postfix) with ESMTP id 50C8CC0177 for ; Wed, 10 Aug 2022 15:19:06 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id 15-20020a17090a098f00b001f305b453feso2515792pjo.1 for ; Wed, 10 Aug 2022 08:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=wfVdiCm3ki6hY+6umh5GbzZ8nNKi0kDY1+Bd1ej/jM8=; b=UdXfowzBOTp4IdbmLV0WXeQAcY/vsJ/7uo+iK0cbB9dRpy21/lcf/YanTi3shCIAU0 Q8rfIZTnd0sFO2eOAJg8bQRhBlA5UyajIW7JKq6kChqbxIqBjxer1gwpkQzfLIAI5B/5 VUeN+qVeFxShxqCMoW4Wr+biXvlCI+ndPPS5CnKurUdGwbSf4MtK3rdRBQ45hNXX9V5z JAQTTSrfQv7hQ3PFf8Jj36ClWIoJ5YqlWfCZdHmguctdIvHlIOMV3/PBphph5bd/SDbY GC2bpylDRY7EkNbki9of3Gdh3iWuOlFezWsDa+jJ5+bZZ4AXwuCNcMB0hstIpQyEK8cV +YZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=wfVdiCm3ki6hY+6umh5GbzZ8nNKi0kDY1+Bd1ej/jM8=; b=SFUmyBKxZeRA8nbxY9Ln1bcTtIKrr2PknGa9l01tdfmHMk01MZGbUXwmceJ0KCrFzT lLKfSES28COD72qiw0jQcNp8v2tGvD/Vyo6k7AvugX9eMUkJOHwAOHsafwqQIraiDOtO oRPtYBxWYPDn9fiTN5yQm1+mk9P/rO74nBkYeN/5NZjImSjYUfNO2AjaVM1nZ0EzxoQN mMa7nNJ3r96yWQwAkIH+Gnr4s5N1tmV8w+ymmiao2ioMVN1Sg4W8+t68bA8F77oxhP3d YnTMM3rmhhSqinJT7WZByRdyhcerk4hbXYkRbOnUt5a6t1/jWtirXGM7kY7xOfGQEjOy kuvA== X-Gm-Message-State: ACgBeo0iHXMD5gUui+Q9TLl2iwNMt7b0ACgskNGv7RHLx4yi/Ev0wTWZ gcJM0Gx8+I6eTwsp+v5eT1M= X-Google-Smtp-Source: AA6agR6Zk/PLReQ+MJ1KpoaibPWZe0SiORmjhQNLojFx1LN3y6d44aIAK00GwGL2TEAU0VSXMfvqcQ== X-Received: by 2002:a17:902:e80c:b0:16f:8663:6c57 with SMTP id u12-20020a170902e80c00b0016f86636c57mr28145450plg.102.1660144743943; Wed, 10 Aug 2022 08:19:03 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:03 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao , Andrii Nakryiko Subject: [PATCH bpf-next 10/15] bpf: Introduce new helpers bpf_ringbuf_pages_{alloc,free} Date: Wed, 10 Aug 2022 15:18:35 +0000 Message-Id: <20220810151840.16394-11-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UdXfowzB; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144746; a=rsa-sha256; cv=none; b=1v5IgQGm1/jmcmV8oKJtmzseXLqYziZiXIIyKxTLXG7hkuN19XTjFH+mZbcsKXzMUiPb6Z PPiEJ/60KCz5urpw3z6nNWNzleETkiVgY9dMxmnbPYw1C8UrgiuR2RLBFnTyvNcXScBHUo +WzMmDAPVKm7QpiIxG1gEDfRVU4bHOA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144746; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wfVdiCm3ki6hY+6umh5GbzZ8nNKi0kDY1+Bd1ej/jM8=; b=kc3jZMLLBGUjqbzqY941cf5iZH4M95GWjGhDM5H/5A/KHNwOvoUn4VL44wtCZqpvERDqFu ifL36B0TDk5I1XU35lyVbqfZRwEUfA88jBCNG0wefFnoP8pNO2vQICte8g/2zOpFAG4Tjz jFYMFvxYOps1Du7J+6o4e3W1cVYpLJw= X-Rspamd-Server: rspam10 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UdXfowzB; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Queue-Id: 50C8CC0177 X-Stat-Signature: 8ighy4fsspxifmjaouwt1qnz5857i611 X-HE-Tag: 1660144746-521428 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allocate pages related memory into the new helper bpf_ringbuf_pages_alloc(), then it can be handled as a single unit. Suggested-by: Andrii Nakryiko Signed-off-by: Yafang Shao --- kernel/bpf/ringbuf.c | 80 ++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 56 insertions(+), 24 deletions(-) diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 5eb7820..1e7284c 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -59,6 +59,57 @@ struct bpf_ringbuf_hdr { u32 pg_off; }; +static void bpf_ringbuf_pages_free(struct page **pages, int nr_pages) +{ + int i; + + for (i = 0; i < nr_pages; i++) + __free_page(pages[i]); + bpf_map_area_free(pages, NULL); +} + +static struct page **bpf_ringbuf_pages_alloc(struct bpf_map *map, + int nr_meta_pages, + int nr_data_pages, + int numa_node, + const gfp_t flags) +{ + int nr_pages = nr_meta_pages + nr_data_pages; + struct mem_cgroup *memcg, *old_memcg; + struct page **pages, *page; + int array_size; + int i; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + array_size = (nr_meta_pages + 2 * nr_data_pages) * sizeof(*pages); + pages = bpf_map_area_alloc(array_size, numa_node, NULL); + if (!pages) + goto err; + + for (i = 0; i < nr_pages; i++) { + page = alloc_pages_node(numa_node, flags, 0); + if (!page) { + nr_pages = i; + goto err_free_pages; + } + pages[i] = page; + if (i >= nr_meta_pages) + pages[nr_data_pages + i] = page; + } + set_active_memcg(old_memcg); + bpf_map_put_memcg(memcg); + + return pages; + +err_free_pages: + bpf_ringbuf_pages_free(pages, nr_pages); +err: + set_active_memcg(old_memcg); + bpf_map_put_memcg(memcg); + return NULL; +} + static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node, struct bpf_map *map) { @@ -67,10 +118,8 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node, int nr_meta_pages = RINGBUF_PGOFF + RINGBUF_POS_PAGES; int nr_data_pages = data_sz >> PAGE_SHIFT; int nr_pages = nr_meta_pages + nr_data_pages; - struct page **pages, *page; struct bpf_ringbuf *rb; - size_t array_size; - int i; + struct page **pages; /* Each data page is mapped twice to allow "virtual" * continuous read of samples wrapping around the end of ring @@ -89,22 +138,11 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node, * when mmap()'ed in user-space, simplifying both kernel and * user-space implementations significantly. */ - array_size = (nr_meta_pages + 2 * nr_data_pages) * sizeof(*pages); - pages = bpf_map_area_alloc(array_size, numa_node, map); + pages = bpf_ringbuf_pages_alloc(map, nr_meta_pages, nr_data_pages, + numa_node, flags); if (!pages) return NULL; - for (i = 0; i < nr_pages; i++) { - page = alloc_pages_node(numa_node, flags, 0); - if (!page) { - nr_pages = i; - goto err_free_pages; - } - pages[i] = page; - if (i >= nr_meta_pages) - pages[nr_data_pages + i] = page; - } - rb = vmap(pages, nr_meta_pages + 2 * nr_data_pages, VM_MAP | VM_USERMAP, PAGE_KERNEL); if (rb) { @@ -114,10 +152,6 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node, return rb; } -err_free_pages: - for (i = 0; i < nr_pages; i++) - __free_page(pages[i]); - bpf_map_area_free(pages, NULL); return NULL; } @@ -188,12 +222,10 @@ static void bpf_ringbuf_free(struct bpf_ringbuf *rb) * to unmap rb itself with vunmap() below */ struct page **pages = rb->pages; - int i, nr_pages = rb->nr_pages; + int nr_pages = rb->nr_pages; vunmap(rb); - for (i = 0; i < nr_pages; i++) - __free_page(pages[i]); - bpf_map_area_free(pages, NULL); + bpf_ringbuf_pages_free(pages, nr_pages); } static void ringbuf_map_free(struct bpf_map *map) From patchwork Wed Aug 10 15:18:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BEC3C19F2A for ; Wed, 10 Aug 2022 15:19:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 104B26B0082; Wed, 10 Aug 2022 11:19:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 090D06B0083; Wed, 10 Aug 2022 11:19:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D86406B0085; Wed, 10 Aug 2022 11:19:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C399B6B0082 for ; Wed, 10 Aug 2022 11:19:15 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 276B9C146F for ; Wed, 10 Aug 2022 15:19:15 +0000 (UTC) X-FDA: 79784041470.16.0894E14 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf27.hostedemail.com (Postfix) with ESMTP id C50EB4018D for ; Wed, 10 Aug 2022 15:19:06 +0000 (UTC) Received: by mail-pf1-f179.google.com with SMTP id u133so13973411pfc.10 for ; Wed, 10 Aug 2022 08:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=6MxaJVkVqzqq1Kknlzxc81EzIkRTYeOwRhc2SFg8N/c=; b=UHSEQt3XIXfznt3a+ubZbeDFz66AINIJwJOLhdv8iQaZ/pSRMV7wThITwBmxC05Fi0 KKI8ViRUZnWN2KuTrILsThD1VM2522RlvqfVJELqcRjRUicVfOQfC1eBlnF0HgmxorHD vl95KCLj2Ar0+XkoQAI+ZGTub8AgH9LzsfIrnQXZnnNWR03KVxsc6Hv+RdwxpxYSByBy +dY3DLa7rRuktr93pPg1qRZKRoti9NSbEe6MJ1ahuzweyNR1JyoBmJ2qb3GgDosw2Rui mYhGhRIMsnPH3pvNlWCsv3Ohr+xNk7HyIOInDG8mzySuN1pRyzsK15T19wUk14N/rvsi Y4NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=6MxaJVkVqzqq1Kknlzxc81EzIkRTYeOwRhc2SFg8N/c=; b=lmcrhU5Pmil916Z60JAPHgNA9H02kOl9a+oaEkAXKeIDTTn3Yk/DYFZXAotzrpW+nu zptTd6nbsvpmDPyUI38uyQF/qGIaB/nOMPkolnjyDfoBs/uKq6G8JBx7+Oi+G8IKZ8rp Sfxh4fbdt7ZYPro4Wo4LW6XL5oBnx6jR7ubnlbxrSvFKRF+kCtSiCEUc3+Fz57jc0VdY jtbaBe9QKP1NxMMk5/T/8099SK1oYoaRUYDcqDdVyAPyWO9rBJOQiAe6YJr6NE/XFDue 6BUh74rIRhglVkJ1A4XyHu2Wn9JJZt7FEf1XY+FCDI8fBZodGeVOvhnp2VrYNZxD9FjP teEA== X-Gm-Message-State: ACgBeo290pQ426rXDxVJmaP4El7HDXKpoeAqTKsZqhSuQCZLXvBXB8Rb K1JH+0++xPbZnlZ3NkkT7c8= X-Google-Smtp-Source: AA6agR5ytZN6hdOuMqFW2+9K97jQ/CcpaM1qMww3s46sJZmj8BHJWlMkVC0swgs0Kd2CpWegMFqhmA== X-Received: by 2002:aa7:88d1:0:b0:52f:8fb8:5ec6 with SMTP id k17-20020aa788d1000000b0052f8fb85ec6mr10316333pff.34.1660144745882; Wed, 10 Aug 2022 08:19:05 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:05 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 11/15] bpf: Use bpf_map_kzalloc in arraymap Date: Wed, 10 Aug 2022 15:18:36 +0000 Message-Id: <20220810151840.16394-12-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UHSEQt3X; spf=pass (imf27.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144753; a=rsa-sha256; cv=none; b=nncT+EsZQB5waLrG3U6BAHExAYj0fmj+AQd3qPqOUwjyTQZN8qvaRNrc6n+PJ1Y/8UYb3v CUohUGNCFmJ3QL2ykV1E0FL/EpIi7oCCxlQQmjBU/1qvAkcqpP5fwNH7/fTlETN1g+n1JE 5AwIoRQVGUszkvYtayQfZg0LEx8fwj0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144753; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6MxaJVkVqzqq1Kknlzxc81EzIkRTYeOwRhc2SFg8N/c=; b=NC/Cmh+N7dGZ0GFzYfSvTY/DvJhs87Fv7z95E7aUJjJBSwWIUa9NqwmtGEAmMFW2drb62a t291dyVlzX/o+2N9/yjlNB1s2scKkDfD3vOQhMM9W7ftqkdS9b95CIWg7VUN5TU0JRlfcc PWOHp6Q3f+ZB7aPET/khyiTUco5KCDw= Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UHSEQt3X; spf=pass (imf27.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: fwgjradr5bpb4r1ojup3cjwah4gr3bgi X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: C50EB4018D X-Rspam-User: X-HE-Tag: 1660144746-169001 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allocates memory after map creation, then we can use the generic helper bpf_map_kzalloc() instead of the open-coded kzalloc(). Signed-off-by: Yafang Shao --- kernel/bpf/arraymap.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 80974c5..3039832 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -1090,20 +1090,20 @@ static struct bpf_map *prog_array_map_alloc(union bpf_attr *attr) struct bpf_array_aux *aux; struct bpf_map *map; - aux = kzalloc(sizeof(*aux), GFP_KERNEL_ACCOUNT); - if (!aux) + map = array_map_alloc(attr); + if (IS_ERR(map)) return ERR_PTR(-ENOMEM); + aux = bpf_map_kzalloc(map, sizeof(*aux), GFP_KERNEL); + if (!aux) { + array_map_free(map); + return ERR_PTR(-ENOMEM); + } + INIT_WORK(&aux->work, prog_array_map_clear_deferred); INIT_LIST_HEAD(&aux->poke_progs); mutex_init(&aux->poke_mutex); - map = array_map_alloc(attr); - if (IS_ERR(map)) { - kfree(aux); - return map; - } - container_of(map, struct bpf_array, map)->aux = aux; aux->map = map; From patchwork Wed Aug 10 15:18:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2587C00140 for ; Wed, 10 Aug 2022 15:19:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 439936B0072; Wed, 10 Aug 2022 11:19:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C17F6B0075; Wed, 10 Aug 2022 11:19:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 213DD6B0078; Wed, 10 Aug 2022 11:19:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0E34F6B0072 for ; Wed, 10 Aug 2022 11:19:23 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CBD211C6D2D for ; Wed, 10 Aug 2022 15:19:22 +0000 (UTC) X-FDA: 79784041764.08.436322C Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf28.hostedemail.com (Postfix) with ESMTP id B1001C0161 for ; Wed, 10 Aug 2022 15:19:08 +0000 (UTC) Received: by mail-pf1-f169.google.com with SMTP id p125so10453535pfp.2 for ; Wed, 10 Aug 2022 08:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=L21h4+LhKzVbOgymwJqIY9/nxSMZ9tbw+WqBvebpAHo=; b=KeXYWGzf22r6tp6AGlXKgfkYSLL92qFn/v0xzLOawzsAZ7dKaX1F920h1kIiDeKqy0 tIHMdiZFS147YlQvLzXvGD77IukqdqITM58J2OHO510EF6687XIPaU1ye2Teqep5Rcmj 2wR4anFZfqV3I/hi2S8GmdWxvXYkIQKh4Assq6E3oFz8i40BkflACjOeQKb6uw/dppMd KicCGJhhWGMlTNJUfj25v8cpn/3/8GdksrHDBln/s9HeC2TLRj7cvg11mtdNrvVrLQtm FYJtK1KD+fEhK0mwIqsAjP9QcBv4TY3vPmhmiMxdZ2nooUKbEuseAnZ20wQYspkh1wNG YYqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=L21h4+LhKzVbOgymwJqIY9/nxSMZ9tbw+WqBvebpAHo=; b=UcLMnIIlQd1SrlhQol81h2NNBKlmy53U2udrGYI2fgv5DOSaSAYLzTPUw+Xg7KK2YF C6WEw8GeU8WrjJdkCXLJCObXUwcFdLU865Ng9q8ZGBp9jOOW+Z3yYCV6e6HM7hGpjrCx rtR8ywxqzJZOVmL9tB3S3QqPfmQmeR4bOq60gHdzHwR3luuEposcVE7Wm8hQOFuP1sjE qZGDtROF21hMpi1QyDYTWfk0XRh1/hG8RgnZbpQROiASKsvfbKRVMU0+B2zWBvc7+9Wm /4UKK2glfxkU3fPx9EPoICWqjim1tmt12pi+R44793+h/McYBb8sC4hxUo28t8RMsAk2 SwcQ== X-Gm-Message-State: ACgBeo3zy1wHmBVaxzAfy2Wss7O+MWQEvRdbkKsJDCG9z0An04Bb8PX/ bo1cpJVra+uElpiYJC1s8So= X-Google-Smtp-Source: AA6agR4Qm1piYZUT+NZjX322Lv/JWEoiqSZroQJTNfAvHnmXAGUoEZ2XBS2sQ0tRlxEBO/vP2ie4Ag== X-Received: by 2002:a05:6a00:1706:b0:52f:6f75:991b with SMTP id h6-20020a056a00170600b0052f6f75991bmr12147620pfc.34.1660144747830; Wed, 10 Aug 2022 08:19:07 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:07 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 12/15] bpf: Use bpf_map_kvcalloc in bpf_local_storage Date: Wed, 10 Aug 2022 15:18:37 +0000 Message-Id: <20220810151840.16394-13-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144760; a=rsa-sha256; cv=none; b=7Gjny7RZCKXIDff7Rs/2HFGWaiSVxnnL7OSkAqbrbuFn/UYoAEgvC5aONpqYJodZPV7eRr Ux3mgnkQTYODIg/LWDe1JoWlbc28P3Ev/dGMLt1Q7IrZnEL8xF33AD/S5vXR392crQH57G dxDly1Gj83RjoZQflcrYDWTIw31GoZ4= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KeXYWGzf; dmarc=temperror reason="query timed out" header.from=gmail.com (policy=temperror); spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144760; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L21h4+LhKzVbOgymwJqIY9/nxSMZ9tbw+WqBvebpAHo=; b=nzomqzGdnCdFBx/v+eWAjvXs3/FXNLSVrijObyKwZUj5QqGyyj3E/EjZvJ2Ka8a7Kw78pe 92max22ahjuA54OgSz9K04mgi8bsdVqMEWAUOHLdXLe6Wv4RNhlJVRphGRXXkq7N7r4MiK ZVyCvDjf+khGmiGUGiX2/oRtX8yoLHU= X-Rspamd-Server: rspam12 X-Rspam-User: X-Rspamd-Queue-Id: B1001C0161 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KeXYWGzf; dmarc=temperror reason="query timed out" header.from=gmail.com (policy=temperror); spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Stat-Signature: 7rhtaast7mq15rf4b5hkp65mdydpzngk X-HE-Tag: 1660144748-649979 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper bpf_map_kvcalloc() for this memory allocation. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 8 ++++++++ kernel/bpf/bpf_local_storage.c | 4 ++-- kernel/bpf/syscall.c | 15 +++++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index d7485b7..ebb6ed4 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1656,6 +1656,8 @@ int generic_map_delete_batch(struct bpf_map *map, void *bpf_map_kmalloc_node(const struct bpf_map *map, size_t size, gfp_t flags, int node); void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags); +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags); void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags); #else @@ -1672,6 +1674,12 @@ void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, return kzalloc(size, flags); } +static inline void * +bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, gfp_t flags) +{ + return kvcalloc(n, size, flags); +} + static inline void __percpu * bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 67ab249..71d5bf1 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -620,8 +620,8 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) nbuckets = max_t(u32, 2, nbuckets); smap->bucket_log = ilog2(nbuckets); - smap->buckets = kvcalloc(sizeof(*smap->buckets), nbuckets, - GFP_USER | __GFP_NOWARN | __GFP_ACCOUNT); + smap->buckets = bpf_map_kvcalloc(&smap->map, sizeof(*smap->buckets), + nbuckets, GFP_USER | __GFP_NOWARN); if (!smap->buckets) { bpf_map_area_free(smap, &smap->map); return ERR_PTR(-ENOMEM); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 02ce7e9..2bd3bcf 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -489,6 +489,21 @@ void *bpf_map_kzalloc(const struct bpf_map *map, size_t size, gfp_t flags) return ptr; } +void *bpf_map_kvcalloc(struct bpf_map *map, size_t n, size_t size, + gfp_t flags) +{ + struct mem_cgroup *memcg, *old_memcg; + void *ptr; + + memcg = bpf_map_get_memcg(map); + old_memcg = set_active_memcg(memcg); + ptr = kvcalloc(n, size, flags | __GFP_ACCOUNT); + set_active_memcg(old_memcg); + bpf_map_put_memcg(memcg); + + return ptr; +} + void __percpu *bpf_map_alloc_percpu(const struct bpf_map *map, size_t size, size_t align, gfp_t flags) { From patchwork Wed Aug 10 15:18:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C33E9C00140 for ; Wed, 10 Aug 2022 15:19:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 598506B0075; Wed, 10 Aug 2022 11:19:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 520E06B0080; Wed, 10 Aug 2022 11:19:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 39DB08E0001; Wed, 10 Aug 2022 11:19:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 26E226B0075 for ; Wed, 10 Aug 2022 11:19:11 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ED5B38057B for ; Wed, 10 Aug 2022 15:19:10 +0000 (UTC) X-FDA: 79784041260.18.921DBDC Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf07.hostedemail.com (Postfix) with ESMTP id 9D33740177 for ; Wed, 10 Aug 2022 15:19:10 +0000 (UTC) Received: by mail-pg1-f174.google.com with SMTP id q16so14623364pgq.6 for ; Wed, 10 Aug 2022 08:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=Clyu36Jre2FnqWFd0IUB464wOplrravQ4IfQGIbyf7A=; b=YEil+EOUwJvQIvFBvsL8BPyBTpmCr/QRfJvhndnQRuC2ZQ0+oQiSl9+jCjuJsBhbCT t9ZCx1gUCzZYRJYYHoPNdbmW/+0irKUw1pjI7jNKTtBINlmPaSWFy91p5RHjUFl6XNgM G1xgYpU4pkXPTphUNnsdL/jS18recBewUX0nzUnJ39ufiGnGv3a8FxQFoEMVMyp3kgSa 8T/+jAcFoP51aFusdVra1eIzo7qi0alJ1JstrFTAj/ESY+9O6OVrVAGYqLnrtB4TLz0C DfbqqKV2N0K862CAJDiK+bNn0Q5z9kegp3Ck5xrlgC3xb4U2WxfDoDWCvmvJf39os0HE xeww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=Clyu36Jre2FnqWFd0IUB464wOplrravQ4IfQGIbyf7A=; b=nwfu1+TKL7L5Qd+8vEDqVfjQ5iVnJp14dslmuNt+t//B7MKjYrkReYOznYhgGvETlw Po/+GJu+oiyR5m7F9Qs9L8zIhLckFDzaeOqwjHtXjvbkujqRmF2TX8U2XV61mLf48fMQ H5mCpMx2dmdEFPcot2fOsB+EddLIzNIS9p/U10TWnAoi5kvBM4/QvSGiLTi1GIqLwc8s jKlfaBRuCibB0aLW4mrYBFrfh7j35jwqTIMCkzs175UndWI0tqs3qZB3Ksw5cq44mlgJ zuVU/Zy0iS2xbZS1bR0BINEqTRB8cdB75bJrS/mikI5T3MNWKcSZ9epDj35+tIPeEy1f UCAg== X-Gm-Message-State: ACgBeo20rdti4H4qLlzs5PDYcYL8iNCANIMKTFpWWLqoaUGWJbV3U0G5 5Wk9qcBsbXhKekk5UCtgvuE= X-Google-Smtp-Source: AA6agR7Xkytp2XXN8I2Qcu6v3KLTmoDxQ3RCtj8xr4Gi8KDkaVZMWgLR+utyFISJ9K3PkHwewHU68g== X-Received: by 2002:a05:6a00:21c8:b0:52b:ffc0:15e7 with SMTP id t8-20020a056a0021c800b0052bffc015e7mr28079599pfj.29.1660144749771; Wed, 10 Aug 2022 08:19:09 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:08 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 13/15] mm, memcg: Add new helper get_obj_cgroup_from_cgroup Date: Wed, 10 Aug 2022 15:18:38 +0000 Message-Id: <20220810151840.16394-14-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144750; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Clyu36Jre2FnqWFd0IUB464wOplrravQ4IfQGIbyf7A=; b=lqaSbOr8RcrC51lUozHJJ0r/AZSQ+VL19tGANr88FUxtaG8xWx97fEBslR0/+KeEr61Y8L xu/iXpZwFdehRQ/rgG15onDX2e5SBDIRgq/oTc2uASEBe/XeIailV8vCCoI60e+h5eHPP/ M0g4Ll+epNDXkSsVqQEtRiaKY69IJhs= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YEil+EOU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144750; a=rsa-sha256; cv=none; b=ULuH4ebXrJHfd63VwAUUy3TpqqKniMJTTBydCkmak4cSpXtt2ifuQcbpXDixWTbCOIC6aU /D6uQQk0ilx/D0giFnKxPmJimou/jrZR1OL00tlUPmyUhErm7or9KnbXNzzHVoVa+ab9h7 4PkHTnTOzjZlyFlgAY5PP7Jty49iVUw= X-Stat-Signature: hm3u36no8ptup5kwqswm8h33z19yjqez X-Rspamd-Queue-Id: 9D33740177 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YEil+EOU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1660144750-296743 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce new helper get_obj_cgroup_from_cgroup() to get obj_cgroup from a specific cgroup. Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 1 + mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 2f0a611..901a921 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1713,6 +1713,7 @@ static inline void set_shrinker_bit(struct mem_cgroup *memcg, int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order); void __memcg_kmem_uncharge_page(struct page *page, int order); +struct obj_cgroup *get_obj_cgroup_from_cgroup(struct cgroup *cgrp); struct obj_cgroup *get_obj_cgroup_from_current(void); struct obj_cgroup *get_obj_cgroup_from_page(struct page *page); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 618c366..762cffa 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2908,6 +2908,47 @@ static struct obj_cgroup *__get_obj_cgroup_from_memcg(struct mem_cgroup *memcg) return objcg; } +static struct obj_cgroup *get_obj_cgroup_from_memcg(struct mem_cgroup *memcg) +{ + struct obj_cgroup *objcg; + + if (memcg_kmem_bypass()) + return NULL; + + rcu_read_lock(); + objcg = __get_obj_cgroup_from_memcg(memcg); + rcu_read_unlock(); + return objcg; +} + +struct obj_cgroup *get_obj_cgroup_from_cgroup(struct cgroup *cgrp) +{ + struct cgroup_subsys_state *css; + struct mem_cgroup *memcg; + struct obj_cgroup *objcg; + + rcu_read_lock(); + css = rcu_dereference(cgrp->subsys[memory_cgrp_id]); + if (!css || !css_tryget_online(css)) { + rcu_read_unlock(); + cgroup_put(cgrp); + return ERR_PTR(-EINVAL); + } + rcu_read_unlock(); + cgroup_put(cgrp); + + memcg = mem_cgroup_from_css(css); + if (!memcg) { + css_put(css); + return ERR_PTR(-EINVAL); + } + + objcg = get_obj_cgroup_from_memcg(memcg); + css_put(css); + + return objcg; +} + __always_inline struct obj_cgroup *get_obj_cgroup_from_current(void) { struct obj_cgroup *objcg = NULL; From patchwork Wed Aug 10 15:18:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECA7DC25B0C for ; Wed, 10 Aug 2022 15:19:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 857896B0080; Wed, 10 Aug 2022 11:19:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E0976B0081; Wed, 10 Aug 2022 11:19:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E46F8E0001; Wed, 10 Aug 2022 11:19:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 45FBB6B0080 for ; Wed, 10 Aug 2022 11:19:13 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1A01016128A for ; Wed, 10 Aug 2022 15:19:13 +0000 (UTC) X-FDA: 79784041386.07.79D04F7 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf24.hostedemail.com (Postfix) with ESMTP id BFC4C180195 for ; Wed, 10 Aug 2022 15:19:12 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id m2so14536183pls.4 for ; Wed, 10 Aug 2022 08:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=hi6fhDArphhgg9aTfUb0vjQeqS917l/JsjjKe9ELdYY=; b=LobkAxEJbP7pA2IheEsLEktxc27J3Jlqo+GWd6XiM7ro3ZtOPuAkH+u4fl216wBeSH ro/JqMg4lzxhv0ozcRJg2A5/Ik1++m9/gjWG4fniTUc0iKEvBN3NOf0YE5E3+5IrmIch 1pGsg70RYwGBVfQeqthvTvOU9jGGBK10QtCpdzVyOoluoKkZjwxggrOIQL65zZsecNS9 Hb5CbWKv2LTmwjV3ccWawKv8bKpRNEuqtKQBBAFuVMwrGzbM30orihLe6Oivka56BYw1 /DGACSWG0DmMB833wNVFlFcL9c78TkwK3PoE896C6lnk1kIKHU7Xyf1VnyFz9sKnXmz5 /+dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=hi6fhDArphhgg9aTfUb0vjQeqS917l/JsjjKe9ELdYY=; b=Dyjqx6miaWDg1JCEejKUHmeS30HS9TvDcEelvuuZPSGHUM3euwhZkbhicK0AdCMDPz HMGga3XG5DgKEop7+lQXJsoOAkSNL8z9QhITXyqWr2atxQ+7PyBeIujsAH1Ttp3OieM6 ydQcrWEa3i0swb61zb+IU1jE6rl/RKMGRny3jueUYTp8iPdQuMvaSR0NkoQ05bXc6nuY WOHfolbpwyA31926yR1fOmxns1n1SJTTFO2Lvd3n15E4/GLWfJrCKe2srVoU+5OVZipW 3cO5n8kqN+NdyPZzEZPRvYA3zbBlK2szrFEfQM+WMoi7mOtVMVQOz3K/1DebF7lrQNPd wjPA== X-Gm-Message-State: ACgBeo1XCXKK8ZObrR89TXuS767ELzwyQpKf7XZimaVq2WvjXlUXy/ch LCdvIZ2qESEuTxU699+vUqM= X-Google-Smtp-Source: AA6agR5ecGXWYh+DnciSiUU7slYVzGDc4/JNSVixTZ38bgdGybTX0xKmTso+KhWg4BAY0qpZKBmCzQ== X-Received: by 2002:a17:902:7044:b0:170:d739:8cb9 with SMTP id h4-20020a170902704400b00170d7398cb9mr9282442plt.120.1660144751773; Wed, 10 Aug 2022 08:19:11 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:10 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 14/15] bpf: Add return value for bpf_map_init_from_attr Date: Wed, 10 Aug 2022 15:18:39 +0000 Message-Id: <20220810151840.16394-15-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LobkAxEJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144752; a=rsa-sha256; cv=none; b=4JPYE6YcrcsyoLNb4S7qBjR9DlZfCBiFr0fxMoNAKVtex3UYQtM6d9ktYj4DEwQMWSRx9z DQq9Ahuehh4yxJCKqsNosx1LbrkZ8YLHY4qiyDR6XusTBZTXUVNpVf0eLisUyUGExi+y5K 1eXhLapzb7LHRjnn8U0z/0BsNxDlX0U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144752; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hi6fhDArphhgg9aTfUb0vjQeqS917l/JsjjKe9ELdYY=; b=gr9Ca/i4QuCmb82kGwgZQk4WtEVJtYkJJh0iQIU2207FtK9eyWvpTyVtUz31mcqqo0AiqG skBljNQ9HatddWoajHiogJeqwfxrzHc5c6Pjbb0pQ0wXkNM8xC3o5/1mL6MYAsPpoUABR3 b/4GHirS7/xlleDUcUH+AcbwADeGyUQ= X-Rspamd-Server: rspam10 X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LobkAxEJ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Queue-Id: BFC4C180195 X-Stat-Signature: k3hbzifz1jkmhi87xhfqzg6gzdyyt6gn X-HE-Tag: 1660144752-373270 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add return value for bpf_map_init_from_attr() to indicate whether it init successfully. This is a preparation of the followup patch. Signed-off-by: Yafang Shao --- include/linux/bpf.h | 2 +- kernel/bpf/arraymap.c | 8 +++++++- kernel/bpf/bloom_filter.c | 7 ++++++- kernel/bpf/bpf_local_storage.c | 7 ++++++- kernel/bpf/bpf_struct_ops.c | 7 ++++++- kernel/bpf/cpumap.c | 6 +++++- kernel/bpf/devmap.c | 6 +++++- kernel/bpf/hashtab.c | 6 +++++- kernel/bpf/local_storage.c | 7 ++++++- kernel/bpf/lpm_trie.c | 8 +++++++- kernel/bpf/offload.c | 6 +++++- kernel/bpf/queue_stack_maps.c | 7 ++++++- kernel/bpf/reuseport_array.c | 7 ++++++- kernel/bpf/ringbuf.c | 7 ++++++- kernel/bpf/syscall.c | 4 +++- net/core/sock_map.c | 8 +++++++- net/xdp/xskmap.c | 8 +++++++- 17 files changed, 94 insertions(+), 17 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index ebb6ed4..a60a68d 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1639,7 +1639,7 @@ struct bpf_prog *bpf_prog_get_type_dev(u32 ufd, enum bpf_prog_type type, void *bpf_map_area_mmapable_alloc(u64 size, int numa_node); void bpf_map_area_free(void *base, struct bpf_map *map); bool bpf_map_write_active(const struct bpf_map *map); -void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); +int bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr); int generic_map_lookup_batch(struct bpf_map *map, const union bpf_attr *attr, union bpf_attr __user *uattr); diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 3039832..28d0310 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -85,6 +85,7 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) bool bypass_spec_v1 = bpf_bypass_spec_v1(); u64 array_size, mask64; struct bpf_array *array; + int err; elem_size = round_up(attr->value_size, 8); @@ -143,7 +144,12 @@ static struct bpf_map *array_map_alloc(union bpf_attr *attr) array->map.bypass_spec_v1 = bypass_spec_v1; /* copy mandatory map attributes */ - bpf_map_init_from_attr(&array->map, attr); + err = bpf_map_init_from_attr(&array->map, attr); + if (err) { + bpf_map_area_free(array, NULL); + return ERR_PTR(err); + } + array->elem_size = elem_size; if (percpu && bpf_array_alloc_percpu(array)) { diff --git a/kernel/bpf/bloom_filter.c b/kernel/bpf/bloom_filter.c index 6691f79..be64227 100644 --- a/kernel/bpf/bloom_filter.c +++ b/kernel/bpf/bloom_filter.c @@ -93,6 +93,7 @@ static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) u32 bitset_bytes, bitset_mask, nr_hash_funcs, nr_bits; int numa_node = bpf_map_attr_numa_node(attr); struct bpf_bloom_filter *bloom; + int err; if (!bpf_capable()) return ERR_PTR(-EPERM); @@ -147,7 +148,11 @@ static struct bpf_map *bloom_map_alloc(union bpf_attr *attr) if (!bloom) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&bloom->map, attr); + err = bpf_map_init_from_attr(&bloom->map, attr); + if (err) { + bpf_map_area_free(bloom, NULL); + return ERR_PTR(err); + } bloom->nr_hash_funcs = nr_hash_funcs; bloom->bitset_mask = bitset_mask; diff --git a/kernel/bpf/bpf_local_storage.c b/kernel/bpf/bpf_local_storage.c index 71d5bf1..b82d124 100644 --- a/kernel/bpf/bpf_local_storage.c +++ b/kernel/bpf/bpf_local_storage.c @@ -609,11 +609,16 @@ struct bpf_local_storage_map *bpf_local_storage_map_alloc(union bpf_attr *attr) struct bpf_local_storage_map *smap; unsigned int i; u32 nbuckets; + int err; smap = bpf_map_area_alloc(sizeof(*smap), NUMA_NO_NODE, NULL); if (!smap) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&smap->map, attr); + err = bpf_map_init_from_attr(&smap->map, attr); + if (err) { + bpf_map_area_free(&smap->map, NULL); + return ERR_PTR(err); + } nbuckets = roundup_pow_of_two(num_possible_cpus()); /* Use at least 2 buckets, select_bucket() is undefined behavior with 1 bucket */ diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 37ba5c0..7cfbabc 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -598,6 +598,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) struct bpf_struct_ops_map *st_map; const struct btf_type *t, *vt; struct bpf_map *map; + int err; if (!bpf_capable()) return ERR_PTR(-EPERM); @@ -624,7 +625,11 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) st_map->st_ops = st_ops; map = &st_map->map; - bpf_map_init_from_attr(map, attr); + err = bpf_map_init_from_attr(map, attr); + if (err) { + bpf_map_area_free(st_map, NULL); + return ERR_PTR(err); + } st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE, map); st_map->links = diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index b593157..e672e62 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -101,7 +101,11 @@ static struct bpf_map *cpu_map_alloc(union bpf_attr *attr) if (!cmap) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&cmap->map, attr); + err = bpf_map_init_from_attr(&cmap->map, attr); + if (err) { + bpf_map_area_free(cmap, NULL); + return ERR_PTR(err); + } /* Pre-limit array size based on NR_CPUS, not final CPU check */ if (cmap->map.max_entries > NR_CPUS) { diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 807a4cd..10f038d 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -167,7 +167,11 @@ static struct bpf_map *dev_map_alloc(union bpf_attr *attr) if (!dtab) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&dtab->map, attr); + err = bpf_map_init_from_attr(&dtab->map, attr); + if (err) { + bpf_map_area_free(dtab, NULL); + return ERR_PTR(err); + } err = dev_map_init_map(dtab, attr); if (err) { diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index e9a6d2c4..f059ae0 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -500,7 +500,11 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr) if (!htab) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&htab->map, attr); + err = bpf_map_init_from_attr(&htab->map, attr); + if (err) { + bpf_map_area_free(htab, NULL); + return ERR_PTR(err); + } lockdep_register_key(&htab->lockdep_key); diff --git a/kernel/bpf/local_storage.c b/kernel/bpf/local_storage.c index fcc7ece..1901195 100644 --- a/kernel/bpf/local_storage.c +++ b/kernel/bpf/local_storage.c @@ -287,6 +287,7 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) __u32 max_value_size = BPF_LOCAL_STORAGE_MAX_VALUE_SIZE; int numa_node = bpf_map_attr_numa_node(attr); struct bpf_cgroup_storage_map *map; + int err; /* percpu is bound by PCPU_MIN_UNIT_SIZE, non-percu * is the same as other local storages. @@ -318,7 +319,11 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr) return ERR_PTR(-ENOMEM); /* copy mandatory map attributes */ - bpf_map_init_from_attr(&map->map, attr); + err = bpf_map_init_from_attr(&map->map, attr); + if (err) { + bpf_map_area_free(map, NULL); + return ERR_PTR(err); + } spin_lock_init(&map->lock); map->root = RB_ROOT; diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 3d329ae..38d7b00 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -543,6 +543,7 @@ static int trie_delete_elem(struct bpf_map *map, void *_key) static struct bpf_map *trie_alloc(union bpf_attr *attr) { struct lpm_trie *trie; + int err; if (!bpf_capable()) return ERR_PTR(-EPERM); @@ -563,7 +564,12 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) return ERR_PTR(-ENOMEM); /* copy mandatory map attributes */ - bpf_map_init_from_attr(&trie->map, attr); + err = bpf_map_init_from_attr(&trie->map, attr); + if (err) { + bpf_map_area_free(trie, NULL); + return ERR_PTR(err); + } + trie->data_size = attr->key_size - offsetof(struct bpf_lpm_trie_key, data); trie->max_prefixlen = trie->data_size * 8; diff --git a/kernel/bpf/offload.c b/kernel/bpf/offload.c index 87c59da..dba7ed2 100644 --- a/kernel/bpf/offload.c +++ b/kernel/bpf/offload.c @@ -376,7 +376,11 @@ struct bpf_map *bpf_map_offload_map_alloc(union bpf_attr *attr) if (!offmap) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&offmap->map, attr); + err = bpf_map_init_from_attr(&offmap->map, attr); + if (err) { + bpf_map_area_free(offmap, NULL); + return ERR_PTR(err); + } rtnl_lock(); down_write(&bpf_devs_lock); diff --git a/kernel/bpf/queue_stack_maps.c b/kernel/bpf/queue_stack_maps.c index bf57e45..f231897 100644 --- a/kernel/bpf/queue_stack_maps.c +++ b/kernel/bpf/queue_stack_maps.c @@ -70,6 +70,7 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) int numa_node = bpf_map_attr_numa_node(attr); struct bpf_queue_stack *qs; u64 size, queue_size; + int err; size = (u64) attr->max_entries + 1; queue_size = sizeof(*qs) + size * attr->value_size; @@ -78,7 +79,11 @@ static struct bpf_map *queue_stack_map_alloc(union bpf_attr *attr) if (!qs) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&qs->map, attr); + err = bpf_map_init_from_attr(&qs->map, attr); + if (err) { + bpf_map_area_free(qs, NULL); + return ERR_PTR(err); + } qs->size = size; diff --git a/kernel/bpf/reuseport_array.c b/kernel/bpf/reuseport_array.c index 52c7e77..4da969b 100644 --- a/kernel/bpf/reuseport_array.c +++ b/kernel/bpf/reuseport_array.c @@ -153,6 +153,7 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) { int numa_node = bpf_map_attr_numa_node(attr); struct reuseport_array *array; + int err; if (!bpf_capable()) return ERR_PTR(-EPERM); @@ -163,7 +164,11 @@ static struct bpf_map *reuseport_array_alloc(union bpf_attr *attr) return ERR_PTR(-ENOMEM); /* copy mandatory map attributes */ - bpf_map_init_from_attr(&array->map, attr); + err = bpf_map_init_from_attr(&array->map, attr); + if (err) { + bpf_map_area_free(array, NULL); + return ERR_PTR(err); + } return &array->map; } diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 1e7284c..3edefd3 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -185,6 +185,7 @@ static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node, static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) { struct bpf_ringbuf_map *rb_map; + int err; if (attr->map_flags & ~RINGBUF_CREATE_FLAG_MASK) return ERR_PTR(-EINVAL); @@ -204,7 +205,11 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr) if (!rb_map) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&rb_map->map, attr); + err = bpf_map_init_from_attr(&rb_map->map, attr); + if (err) { + bpf_map_area_free(rb_map, NULL); + return ERR_PTR(err); + } rb_map->rb = bpf_ringbuf_alloc(attr->max_entries, rb_map->map.numa_node, &rb_map->map); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 2bd3bcf..5f5cade4 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -403,7 +403,7 @@ static u32 bpf_map_flags_retain_permanent(u32 flags) return flags & ~(BPF_F_RDONLY | BPF_F_WRONLY); } -void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) +int bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) { bpf_map_save_memcg(map); map->map_type = attr->map_type; @@ -413,6 +413,8 @@ void bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) map->map_flags = bpf_map_flags_retain_permanent(attr->map_flags); map->numa_node = bpf_map_attr_numa_node(attr); map->map_extra = attr->map_extra; + + return 0; } static int bpf_map_alloc_id(struct bpf_map *map) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 2b3b24e..766a260 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -31,6 +31,7 @@ static int sock_map_prog_update(struct bpf_map *map, struct bpf_prog *prog, static struct bpf_map *sock_map_alloc(union bpf_attr *attr) { struct bpf_stab *stab; + int err; if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); @@ -45,7 +46,12 @@ static struct bpf_map *sock_map_alloc(union bpf_attr *attr) if (!stab) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&stab->map, attr); + err = bpf_map_init_from_attr(&stab->map, attr); + if (err) { + bpf_map_area_free(stab, NULL); + return ERR_PTR(err); + } + raw_spin_lock_init(&stab->lock); stab->sks = bpf_map_area_alloc((u64) stab->map.max_entries * diff --git a/net/xdp/xskmap.c b/net/xdp/xskmap.c index beb11fd..8e7a5a6 100644 --- a/net/xdp/xskmap.c +++ b/net/xdp/xskmap.c @@ -63,6 +63,7 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) struct xsk_map *m; int numa_node; u64 size; + int err; if (!capable(CAP_NET_ADMIN)) return ERR_PTR(-EPERM); @@ -79,7 +80,12 @@ static struct bpf_map *xsk_map_alloc(union bpf_attr *attr) if (!m) return ERR_PTR(-ENOMEM); - bpf_map_init_from_attr(&m->map, attr); + err = bpf_map_init_from_attr(&m->map, attr); + if (err) { + bpf_map_area_free(m, NULL); + return ERR_PTR(err); + } + spin_lock_init(&m->lock); return &m->map; From patchwork Wed Aug 10 15:18:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 12940674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 750C1C25B07 for ; Wed, 10 Aug 2022 15:19:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A5546B0081; Wed, 10 Aug 2022 11:19:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12C838E0001; Wed, 10 Aug 2022 11:19:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD5206B0083; Wed, 10 Aug 2022 11:19:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C56A06B0081 for ; Wed, 10 Aug 2022 11:19:14 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9A792140EE3 for ; Wed, 10 Aug 2022 15:19:14 +0000 (UTC) X-FDA: 79784041428.22.AEDDED7 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf24.hostedemail.com (Postfix) with ESMTP id 3FDAB18019D for ; Wed, 10 Aug 2022 15:19:14 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id m2so14536285pls.4 for ; Wed, 10 Aug 2022 08:19:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=j0vArd5kCmiXrID4XqlmDliohUWxtZHgIa+7cLj7rn4=; b=MMqCZhZzn3sPBSjB7ZDkJJIzBlptDVnfvWBJBi7wzuaf50HQkzpq0n1tWWQQphwNyJ ZklkxnQA8koYN0EJXc+3L66T4Z7Q4/utfybyMzW/726CwjF6qxJRRhjGxZLL/UFOCJpy TbLLn8wPu5HHYALnaIL6Zr1cVcQhfLODb3U5fKDoct69VMSPVdkMR02c4MZ4pbaOGL88 bsrNxcSN9J2IbjjHDyneHYAO1vR1Fk8CzPkrwABFfjOsMeoee/lSNk1rOo7iZrmr9mwX neOuRzkK/8bEk/+EYUnY/xmdqOrtEX7z889sH2Ud4PBHNpxb5htGNjQTuNIdLiCtBobn EDYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=j0vArd5kCmiXrID4XqlmDliohUWxtZHgIa+7cLj7rn4=; b=qTLH4Aqu125cVCJ27icpET+bkgsL5QL2cBDVr9ceCXLSuBBYOfdSE9ymJlGBfgYsar +URly3YFfMvSmLcL6qEIIofQY41vDJrsx4YgG2P53spemf0zNe96kgngBaddmR9rNcF8 HnY4by0BDEvLZyiXoPuKnolPVtj9Tw/rlvY+rZN6V+/M6L+wmlI1EKeppI9KRseOa3Fr 3/IO/iuHlvP0VbwRA+i0cb+mFT+da3Alf19rnfAqev9/AfaajPJ+9ZnupoGYFKMNN8Na xHEwFyvhpu/z1cuw4wyOtqXSTESedBLJD/n1r7ZzzvDcEDuaeTK8ObiAFPJC3z1lEav4 +ZRw== X-Gm-Message-State: ACgBeo3f2ANe9yEQx28JqwMWF/N4N9aNax7g7+buZQw7ZhfcyNFV7IPA 5sE4EB26Wu6PtwxBf8P6d1s= X-Google-Smtp-Source: AA6agR7KJ1PJ7r6sNN9BQifYoxntxi66pFLvj/s4h7cp7gKDAa5B4ZqS0oSQrdeDzsOIO2LhhJwkrA== X-Received: by 2002:a17:90b:33c6:b0:1f4:f595:b0f1 with SMTP id lk6-20020a17090b33c600b001f4f595b0f1mr4349009pjb.230.1660144753765; Wed, 10 Aug 2022 08:19:13 -0700 (PDT) Received: from vultr.guest ([2001:19f0:6001:5c3e:5400:4ff:fe19:c3bc]) by smtp.gmail.com with ESMTPSA id 85-20020a621558000000b0052b6ed5ca40sm2071935pfv.192.2022.08.10.08.19.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 08:19:12 -0700 (PDT) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, songmuchun@bytedance.com, akpm@linux-foundation.org Cc: netdev@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, Yafang Shao Subject: [PATCH bpf-next 15/15] bpf: Introduce selectable memcg for bpf map Date: Wed, 10 Aug 2022 15:18:40 +0000 Message-Id: <20220810151840.16394-16-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220810151840.16394-1-laoar.shao@gmail.com> References: <20220810151840.16394-1-laoar.shao@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MMqCZhZz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660144754; a=rsa-sha256; cv=none; b=6sbo32RqXYCcHJ76CDFyV7BFli428hQo1A1FSEZtmV1Rg8oG5Ri21a3i1CrfuBmiIK+T/G IfK3o/Bc4QwtLaRjtTbXTLGsaImOq9rcf9MZDOqnLJsC4nBghTF6kUZi0J/aaUVf7IYoX1 7uhLQ0+HeEg+Qqg8SEXXLsr3wL3MRME= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660144754; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j0vArd5kCmiXrID4XqlmDliohUWxtZHgIa+7cLj7rn4=; b=TjHuy6LEMhZO/5AP2+LmXpkWbJ3IUNhfvzSAES5EAxMck2K/Yyo6Ina2A8YcbU1gtNu2i6 oa+Y6w6OsNTHN8VgRnY9n7wYkTQox+yKf3R6jd2WmvlVau8/6qpM0jL4YHmppZKDcU18KW XezaAjCsjRtf0yY5QsduLsYjPAFr0XY= X-Rspamd-Server: rspam10 X-Rspam-User: Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MMqCZhZz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com X-Rspamd-Queue-Id: 3FDAB18019D X-Stat-Signature: bhowng59xt9i5byyr6nk8ymy6kumbg7n X-HE-Tag: 1660144754-679634 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A new bpf attr memcg_fd is introduced for map creation. The map creation path in libbpf is changed consequently to set this attr. A new member memcg_fd is introduced into bpf attr of BPF_MAP_CREATE command, which is the fd of an opened cgroup directory. In this cgroup, the memory subsystem must be enabled. The valid memcg_fd must be a postive number, that means it can't be zero(a valid return value of open(2)). Once the kernel get the memory cgroup from this fd, it will set this memcg into bpf map, then all the subsequent memory allocation of this map will be charged to the memcg. The map creation paths in libbpf are also changed consequently. Currently it is only supported for cgroup2 directory. The usage of this new member as follows, struct bpf_map_create_opts map_opts = { .sz = sizeof(map_opts), }; int memcg_fd, map_fd, old_fd; int key, value; memcg_fd = open("/cgroup2", O_DIRECTORY); if (memcg_fd < 0) { perror("memcg dir open"); return -1; } /* 0 is a invalid fd */ if (memcg_fd == 0) { old_fd = memcg_fd; memcg_fd = fcntl(memcg_fd, F_DUPFD_CLOEXEC, 3); close(old_fd); if (memcg_fd < 0) { perror("fcntl"); return -1; } } map_opts.memcg_fd = memcg_fd; map_fd = bpf_map_create(BPF_MAP_TYPE_HASH, "map_for_memcg", sizeof(key), sizeof(value), 1024, &map_opts); if (map_fd <= 0) { close(memcg_fd); perror("map create"); return -1; } Signed-off-by: Yafang Shao --- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 42 ++++++++++++++++++++++++++++++++---------- tools/include/uapi/linux/bpf.h | 1 + tools/lib/bpf/bpf.c | 3 ++- tools/lib/bpf/bpf.h | 3 ++- tools/lib/bpf/gen_loader.c | 2 +- tools/lib/bpf/libbpf.c | 2 ++ tools/lib/bpf/skel_internal.h | 2 +- 8 files changed, 42 insertions(+), 14 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 534e33f..d46acbb 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1300,6 +1300,7 @@ struct bpf_stack_build_id { * to using 5 hash functions). */ __u64 map_extra; + __u32 memcg_fd; /* selectable memcg */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 5f5cade4..91a6e15 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -294,14 +294,30 @@ static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, } #ifdef CONFIG_MEMCG_KMEM -static void bpf_map_save_memcg(struct bpf_map *map) +static int bpf_map_save_memcg(struct bpf_map *map, u32 memcg_fd) { - /* Currently if a map is created by a process belonging to the root - * memory cgroup, get_obj_cgroup_from_current() will return NULL. - * So we have to check map->objcg for being NULL each time it's - * being used. - */ - map->objcg = get_obj_cgroup_from_current(); + struct obj_cgroup *objcg; + struct cgroup *cgrp; + + if (memcg_fd) { + cgrp = cgroup_get_from_fd(memcg_fd); + if (IS_ERR(cgrp)) + return -EINVAL; + + objcg = get_obj_cgroup_from_cgroup(cgrp); + if (IS_ERR(objcg)) + return PTR_ERR(objcg); + } else { + /* Currently if a map is created by a process belonging to the root + * memory cgroup, get_obj_cgroup_from_current() will return NULL. + * So we have to check map->objcg for being NULL each time it's + * being used. + */ + objcg = get_obj_cgroup_from_current(); + } + + map->objcg = objcg; + return 0; } static void bpf_map_release_memcg(struct bpf_map *map) @@ -311,8 +327,9 @@ static void bpf_map_release_memcg(struct bpf_map *map) } #else -static void bpf_map_save_memcg(struct bpf_map *map) +static int bpf_map_save_memcg(struct bpf_map *map, u32 memcg_fd) { + return 0; } static void bpf_map_release_memcg(struct bpf_map *map) @@ -405,7 +422,12 @@ static u32 bpf_map_flags_retain_permanent(u32 flags) int bpf_map_init_from_attr(struct bpf_map *map, union bpf_attr *attr) { - bpf_map_save_memcg(map); + int err; + + err = bpf_map_save_memcg(map, attr->memcg_fd); + if (err) + return err; + map->map_type = attr->map_type; map->key_size = attr->key_size; map->value_size = attr->value_size; @@ -1091,7 +1113,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, return ret; } -#define BPF_MAP_CREATE_LAST_FIELD map_extra +#define BPF_MAP_CREATE_LAST_FIELD memcg_fd /* called via syscall */ static int map_create(union bpf_attr *attr) { diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index f58d58e..12203406 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1300,6 +1300,7 @@ struct bpf_stack_build_id { * to using 5 hash functions). */ __u64 map_extra; + __u32 memcg_fd; /* selectable memcg */ }; struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */ diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index efcc06d..9c613b2 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -171,7 +171,7 @@ int bpf_map_create(enum bpf_map_type map_type, __u32 max_entries, const struct bpf_map_create_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, map_extra); + const size_t attr_sz = offsetofend(union bpf_attr, memcg_fd); union bpf_attr attr; int fd; @@ -199,6 +199,7 @@ int bpf_map_create(enum bpf_map_type map_type, attr.map_extra = OPTS_GET(opts, map_extra, 0); attr.numa_node = OPTS_GET(opts, numa_node, 0); attr.map_ifindex = OPTS_GET(opts, map_ifindex, 0); + attr.memcg_fd = OPTS_GET(opts, memcg_fd, 0); fd = sys_bpf_fd(BPF_MAP_CREATE, &attr, attr_sz); return libbpf_err_errno(fd); diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h index 9c50bea..dd0d929 100644 --- a/tools/lib/bpf/bpf.h +++ b/tools/lib/bpf/bpf.h @@ -51,8 +51,9 @@ struct bpf_map_create_opts { __u32 numa_node; __u32 map_ifindex; + __u32 memcg_fd; }; -#define bpf_map_create_opts__last_field map_ifindex +#define bpf_map_create_opts__last_field memcg_fd LIBBPF_API int bpf_map_create(enum bpf_map_type map_type, const char *map_name, diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 23f5c46..f35b014 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -451,7 +451,7 @@ void bpf_gen__map_create(struct bpf_gen *gen, __u32 key_size, __u32 value_size, __u32 max_entries, struct bpf_map_create_opts *map_attr, int map_idx) { - int attr_size = offsetofend(union bpf_attr, map_extra); + int attr_size = offsetofend(union bpf_attr, memcg_fd); bool close_inner_map_fd = false; int map_create_attr, idx; union bpf_attr attr; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index f7364ea..88c65e5 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -506,6 +506,7 @@ struct bpf_map { bool reused; bool autocreate; __u64 map_extra; + __u32 memcg_fd; }; enum extern_type { @@ -4931,6 +4932,7 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b create_attr.map_flags = def->map_flags; create_attr.numa_node = map->numa_node; create_attr.map_extra = map->map_extra; + create_attr.memcg_fd = map->memcg_fd; if (bpf_map__is_struct_ops(map)) create_attr.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id; diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index bd6f450..83403cc 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -222,7 +222,7 @@ static inline int skel_map_create(enum bpf_map_type map_type, __u32 value_size, __u32 max_entries) { - const size_t attr_sz = offsetofend(union bpf_attr, map_extra); + const size_t attr_sz = offsetofend(union bpf_attr, memcg_fd); union bpf_attr attr; memset(&attr, 0, attr_sz);