From patchwork Tue Feb 6 22:04:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 13547851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72AFDC4828D for ; Tue, 6 Feb 2024 22:05:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0171D6B00A7; Tue, 6 Feb 2024 17:05:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EE1246B00A8; Tue, 6 Feb 2024 17:05:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0CEC6B00A9; Tue, 6 Feb 2024 17:05:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B95646B00A7 for ; Tue, 6 Feb 2024 17:05:51 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 94D921C1047 for ; Tue, 6 Feb 2024 22:05:51 +0000 (UTC) X-FDA: 81762762102.22.AB24FDD Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf29.hostedemail.com (Postfix) with ESMTP id BBA6312000C for ; Tue, 6 Feb 2024 22:05:49 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EDCzCIb6; spf=pass (imf29.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707257149; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z5sXl6wfPkVpJxXcJSxvXuOvb+1sMHHR6BvnXTDIECA=; b=nVmn/+ADqSaVCTPShLvMfBV/i9OqQhwpFKAl1HIiWisnep2oUKCoEb3NY/CrSCCk7qmKqs FWINSJhUGt7IJJD+SLp2drB3NE3bVQFKQ0WQZP7kE+C4Sbz9FI+SoduyGXW5MNeLXuOY4S BeYRiH8NSVJuDCzDOUD/KpPBDF44ug8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707257149; a=rsa-sha256; cv=none; b=yTQbANnJ4C0gUG3ddNPSauM0BwkdjomyMo3UGP3f8pCBFmsqCo6JgGhh4uYvWIEndpk4uc OFum1AkbnEPRAQQf+16HdwyIjj2X9aH/FyG61biW69pXJ0W0hYy+cekKJQwBvmjDWgw4CS jGS9/0iheS/FmOqeH1W+eiOiRs4bnoc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EDCzCIb6; spf=pass (imf29.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1d93edfa76dso135495ad.1 for ; Tue, 06 Feb 2024 14:05:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707257149; x=1707861949; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=z5sXl6wfPkVpJxXcJSxvXuOvb+1sMHHR6BvnXTDIECA=; b=EDCzCIb657eGigimPthC0rMybLVFId5JHq4ZRVJLUDKTugUzmjNEqYuj+3Cy6otWcL DEuC3Hf9uA7rqHMp4/hKV26dcE5d8GlASUSXT/V/lCbx0HGH8kW1jCGXd6nZRDSMqBWg k0DaMJ4C/tZa4ISRZD82J6fozP580JfkGepBpk3rMTXwwE/T4F91hJIFd4wSi7pRqNzk jQFz9d7AA6kVOzffX6Qkj9BC3ueirlJhdR5M5ruf3F/eAppT6nJKuDM8otxToqk7zgTr WSgsux4UxrtetxP8eHQWnu4qiYFmVrAmbrVfxGaF10w8G7SIKl/keTzzyOpGsqvDO3XB lAIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707257149; x=1707861949; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z5sXl6wfPkVpJxXcJSxvXuOvb+1sMHHR6BvnXTDIECA=; b=fBgIFiSw+pPuK5dUbxo7gpBQNzaTffZAykr3SwYlZoYrrBTJmghPnPtKo+AFJ4X6RP tAeSKeS3gp50Qt+OAMu6SE2Pgsft7yfD0joGBAkxHEC2OnFTEMgGdKP25ROreqeZw/iu hq8dPebaHNOGRcFFMc7utGOY1CDzJBwSB61PuvLQTIXkqWHqc3z4iHKinBStkjkDwCss d/heu6jLGS6PQjOhpJXcK6922UC7Oi/757fnBkSDZJ0ZcSReqIDXRY9KlZHYAYjM53Tx cTlV53uULz0lCN8ft241rWHrPPlIR+qjyKIUJoZ2bupo/pBI87KSGZOtbMHOGbwh0uxv CXtQ== X-Gm-Message-State: AOJu0YwKx42nTawkCua4LlxTFmxDGLCfaNiTBTmxNFgGWliURDl10kIR CxvA7KMWg5ZL8rb7lBPKq/NCHJJv6DWJQSvCqNuzWQqo3wCdyv07 X-Google-Smtp-Source: AGHT+IF9U+BtdYCt6M8dYZtgQt/dJeYpcJAyzhX1n//KSQC1NX0NDBwdE7WV9Rte853ceTYT7aKzoQ== X-Received: by 2002:a17:902:650f:b0:1d9:edf5:c86b with SMTP id b15-20020a170902650f00b001d9edf5c86bmr59997plk.67.1707257148740; Tue, 06 Feb 2024 14:05:48 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCUwrnDaQTz88yxH7R10WszU4AcGQqpT0AnT3dwaaN4OHwpmdUYYkayQ0PtrkBsBSO9Wjx5ZBjmxmqDBYbHr2gYsJUzr8QjyyEEQvgAvG79SyCGECLR+KK1BoD8xCyc5PEBoX01aS0vrV5oCmEn0oAq8Cv79v4fg0YQg3/5TU0uAa9YA/zGgqQufx4ZMBU3oiWF3pbssnjT3Rt3yOG3GlgiyLxeTvaM8xicjLTdO62TKE10RjLGBtI7VQRG0Ytm9FHSObaQjTVnSsh5LfPEi9eM4mstvz9dj7/i0 Received: from localhost.localdomain ([2620:10d:c090:400::4:27bf]) by smtp.gmail.com with ESMTPSA id r1-20020a170903410100b001d8f81ecebesm3718pld.192.2024.02.06.14.05.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 06 Feb 2024 14:05:48 -0800 (PST) From: Alexei Starovoitov To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, memxor@gmail.com, eddyz87@gmail.com, tj@kernel.org, brho@google.com, hannes@cmpxchg.org, linux-mm@kvack.org, kernel-team@fb.com Subject: [PATCH bpf-next 16/16] selftests/bpf: Add bpf_arena_htab test. Date: Tue, 6 Feb 2024 14:04:41 -0800 Message-Id: <20240206220441.38311-17-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20240206220441.38311-1-alexei.starovoitov@gmail.com> References: <20240206220441.38311-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: BBA6312000C X-Rspam-User: X-Stat-Signature: ig7ieo7qdx3dikcbexsr9shwxj4g7tf9 X-Rspamd-Server: rspam03 X-HE-Tag: 1707257149-160980 X-HE-Meta: U2FsdGVkX1+zuOoJYjDPXJUnYlZOnZ8CD4D7IlTGb9y/3miX02K+QPawue954wtKSnmauuGOcVUgU1OHydN209hVkZCgSTpk3L/vLTFL9m7GSF0SoqOEs8J3N69tK1Ft2Df7gBv2V99umk39BcluJjXvkDWkD/hKw9eMWd5RKUZHGr3MCW/iRBFo9Bb31qFwstpQ6Qtf/2uLtbllgeevgM3E/3hvqX+U7w92AqWr+v1ofYzimiCabInZbLXosfhZOW8f38YJU8y7IBXKzGidzU5XO2tGUr5Y5pZs+vZmseKJv2X8Yws7ZKltZOM27jCRzdn5xGmpPDC6lQ1BSY9I4s98E4YUCfo875z8z/x9PX1yBLK7ylmiwXOvjMxudCKsYfSEI5ESsJNNLzoMyXHWtL6elmY/qf2jpCNfUYeb0Okf8JvNw4bnROCPWSMzU0XUU/NaQam5fsyR6SKb0LUCzypAJfOfFBCXbVGD9V33hsSaM+fY5WcSXpQ2OmP4rMwZirrxkJszjZOoFrJdED9sB/k739U4ECZY82SG/675FVtufhRLMPEeyBNfJPpQSATcwHIwyNdFH6zm7Sr9w2261Pk+RY6LaU3SjUGLRRFFHympRrsASN0Uu0a3l0vDDmvqFEQmao75J1rx7HBlbiMNKWHRasDtoVV+HdRECCWMbYBJiQv4SSi5ysaOphpgDsHTOLyUSQjqq7QCIbUEpFal942USlYoxFi3PQXDmIduzpl+MMjc2aP9+RN97/r60GuXvl56RiuZA/4g1tBL5rWXCczAbS1mzYA/RP2uz2jkZ82EfTkSivj7OpAkqrCwKlf29X+A4vUBKZsTYXYFL/sxLyLTKky9vFjCCiZ+d4TrnV76+YPkcHfdEJEY/XOm+vm9Xlqny2ri5YgjqJR+WraH8Pe/jugFHS0Kw0xkKzE0AYIYtOqIS1rIs1BDWqbF4kI7w7yw2eN/T0A3uTesZAb bIpmkRf0 G8suS6wRG53Ao9D7n7cPBYu4M3kZY4oZdf7C7Mxy6/XVnCRgGsHD4ysuxM1zHX/gW75sjRXjgHGd7ZUB5BFpHklRhcEg0mYVA1RC4lJHDvUL7X6yxzD+xp2Dx62GzI3CwFAOpodhDlkIYJ9tSSsaqTLfIUghqLja1V0C6gAMUvrHFYpeRHJIPDfd5GokJo329FGZMM+MgrAJNH4p8jwcCdoADG18MruUovWRttXKrWZ7h2QIpkmbI0z6Wm7l7vcf7enmccXMzCYlMR5XaZcG+TkLi6x6b8muFYpQFfhS/H3yfhKjDKsDIToWoI4ecqdtBTpmssrt9JKrMNYd1UAq3D3q4Ey/DHR8SndPqEe8dXIcLKgbLV61QttCOWLpHDLVZ4AM3A3jFC1bDs45JNcOu9EBz+10d/oHFqlUS3P054eQ2ZcPxKEf4i/BRncuL6XyE5wxBuhrHAnPEwWQll3lxjuY9RkfxCcvBgePD4GU1oFHq4IyER4Pl4Sdz6hIYxIHBezNJPnqDkvoK/0m3fA3gM9SaRjCn/gOMwuP8XD0/NT+WtlrcYYiEhkwhFgaUkf1H+E2heQquF2/Wl/cuut6CfJGBocDhTSmuN76V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov bpf_arena_htab.h - hash table implemented as bpf program Signed-off-by: Alexei Starovoitov --- tools/testing/selftests/bpf/bpf_arena_htab.h | 100 ++++++++++++++++++ .../selftests/bpf/prog_tests/arena_htab.c | 88 +++++++++++++++ .../testing/selftests/bpf/progs/arena_htab.c | 48 +++++++++ .../selftests/bpf/progs/arena_htab_asm.c | 5 + 4 files changed, 241 insertions(+) create mode 100644 tools/testing/selftests/bpf/bpf_arena_htab.h create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_htab.c create mode 100644 tools/testing/selftests/bpf/progs/arena_htab.c create mode 100644 tools/testing/selftests/bpf/progs/arena_htab_asm.c diff --git a/tools/testing/selftests/bpf/bpf_arena_htab.h b/tools/testing/selftests/bpf/bpf_arena_htab.h new file mode 100644 index 000000000000..acc01a876668 --- /dev/null +++ b/tools/testing/selftests/bpf/bpf_arena_htab.h @@ -0,0 +1,100 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#pragma once +#include +#include "bpf_arena_alloc.h" +#include "bpf_arena_list.h" + +struct htab_bucket { + struct arena_list_head head; +}; +typedef struct htab_bucket __arena htab_bucket_t; + +struct htab { + htab_bucket_t *buckets; + int n_buckets; +}; +typedef struct htab __arena htab_t; + +static inline htab_bucket_t *__select_bucket(htab_t *htab, __u32 hash) +{ + htab_bucket_t *b = htab->buckets; + + cast_kern(b); + return &b[hash & (htab->n_buckets - 1)]; +} + +static inline arena_list_head_t *select_bucket(htab_t *htab, __u32 hash) +{ + return &__select_bucket(htab, hash)->head; +} + +struct hashtab_elem { + int hash; + int key; + int value; + struct arena_list_node hash_node; +}; +typedef struct hashtab_elem __arena hashtab_elem_t; + +static hashtab_elem_t *lookup_elem_raw(arena_list_head_t *head, __u32 hash, int key) +{ + hashtab_elem_t *l; + + list_for_each_entry(l, head, hash_node) + if (l->hash == hash && l->key == key) + return l; + + return NULL; +} + +static int htab_hash(int key) +{ + return key; +} + +__weak int htab_lookup_elem(htab_t *htab __arg_arena, int key) +{ + hashtab_elem_t *l_old; + arena_list_head_t *head; + + cast_kern(htab); + head = select_bucket(htab, key); + l_old = lookup_elem_raw(head, htab_hash(key), key); + if (l_old) + return l_old->value; + return 0; +} + +__weak int htab_update_elem(htab_t *htab __arg_arena, int key, int value) +{ + hashtab_elem_t *l_new = NULL, *l_old; + arena_list_head_t *head; + + cast_kern(htab); + head = select_bucket(htab, key); + l_old = lookup_elem_raw(head, htab_hash(key), key); + + l_new = bpf_alloc(sizeof(*l_new)); + if (!l_new) + return -ENOMEM; + l_new->key = key; + l_new->hash = htab_hash(key); + l_new->value = value; + + list_add_head(&l_new->hash_node, head); + if (l_old) { + list_del(&l_old->hash_node); + bpf_free(l_old); + } + return 0; +} + +void htab_init(htab_t *htab) +{ + void __arena *buckets = bpf_arena_alloc_pages(&arena, NULL, 2, NUMA_NO_NODE, 0); + + cast_user(buckets); + htab->buckets = buckets; + htab->n_buckets = 2 * PAGE_SIZE / sizeof(struct htab_bucket); +} diff --git a/tools/testing/selftests/bpf/prog_tests/arena_htab.c b/tools/testing/selftests/bpf/prog_tests/arena_htab.c new file mode 100644 index 000000000000..0766702de846 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/arena_htab.c @@ -0,0 +1,88 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include + +#include "arena_htab_asm.skel.h" +#include "arena_htab.skel.h" + +#define PAGE_SIZE 4096 + +#include "bpf_arena_htab.h" + +static void test_arena_htab_common(struct htab *htab) +{ + int i; + + printf("htab %p buckets %p n_buckets %d\n", htab, htab->buckets, htab->n_buckets); + ASSERT_OK_PTR(htab->buckets, "htab->buckets shouldn't be NULL"); + for (i = 0; htab->buckets && i < 16; i += 4) { + /* + * Walk htab buckets and link lists since all pointers are correct, + * though they were written by bpf program. + */ + int val = htab_lookup_elem(htab, i); + + ASSERT_EQ(i, val, "key == value"); + } +} + +static void test_arena_htab_llvm(void) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + struct arena_htab *skel; + struct htab *htab; + size_t arena_sz; + void *area; + int ret; + + skel = arena_htab__open_and_load(); + if (!ASSERT_OK_PTR(skel, "arena_htab__open_and_load")) + return; + + area = bpf_map__initial_value(skel->maps.arena, &arena_sz); + /* fault-in a page with pgoff == 0 as sanity check */ + *(volatile int *)area = 0x55aa; + + /* bpf prog will allocate more pages */ + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.arena_htab_llvm), &opts); + ASSERT_OK(ret, "ret"); + ASSERT_OK(opts.retval, "retval"); + if (skel->bss->skip) { + printf("%s:SKIP:compiler doesn't support arena_cast\n", __func__); + test__skip(); + goto out; + } + htab = skel->bss->htab_for_user; + test_arena_htab_common(htab); +out: + arena_htab__destroy(skel); +} + +static void test_arena_htab_asm(void) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + struct arena_htab_asm *skel; + struct htab *htab; + int ret; + + skel = arena_htab_asm__open_and_load(); + if (!ASSERT_OK_PTR(skel, "arena_htab_asm__open_and_load")) + return; + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.arena_htab_asm), &opts); + ASSERT_OK(ret, "ret"); + ASSERT_OK(opts.retval, "retval"); + htab = skel->bss->htab_for_user; + test_arena_htab_common(htab); + arena_htab_asm__destroy(skel); +} + +void test_arena_htab(void) +{ + if (test__start_subtest("arena_htab_llvm")) + test_arena_htab_llvm(); + if (test__start_subtest("arena_htab_asm")) + test_arena_htab_asm(); +} diff --git a/tools/testing/selftests/bpf/progs/arena_htab.c b/tools/testing/selftests/bpf/progs/arena_htab.c new file mode 100644 index 000000000000..51a9eeb3df5a --- /dev/null +++ b/tools/testing/selftests/bpf/progs/arena_htab.c @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include +#include "bpf_experimental.h" + +struct { + __uint(type, BPF_MAP_TYPE_ARENA); + __uint(map_flags, BPF_F_MMAPABLE); + __uint(max_entries, 1u << 20); /* max_entries * value_size == size of mmap() region */ + __type(key, __u64); + __type(value, __u64); +} arena SEC(".maps"); + +#include "bpf_arena_htab.h" + +void __arena *htab_for_user; +bool skip = false; + +SEC("syscall") +int arena_htab_llvm(void *ctx) +{ +#if defined(__BPF_FEATURE_ARENA_CAST) || defined(BPF_ARENA_FORCE_ASM) + struct htab __arena *htab; + __u64 i; + + htab = bpf_alloc(sizeof(*htab)); + cast_kern(htab); + htab_init(htab); + + /* first run. No old elems in the table */ + bpf_for(i, 0, 1000) + htab_update_elem(htab, i, i); + + /* should replace all elems with new ones */ + bpf_for(i, 0, 1000) + htab_update_elem(htab, i, i); + cast_user(htab); + htab_for_user = htab; +#else + skip = true; +#endif + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/arena_htab_asm.c b/tools/testing/selftests/bpf/progs/arena_htab_asm.c new file mode 100644 index 000000000000..6cd70ea12f0d --- /dev/null +++ b/tools/testing/selftests/bpf/progs/arena_htab_asm.c @@ -0,0 +1,5 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#define BPF_ARENA_FORCE_ASM +#define arena_htab_llvm arena_htab_asm +#include "arena_htab.c"