From patchwork Tue Aug 25 18:29:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: KP Singh X-Patchwork-Id: 11736163 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E6664109B for ; Tue, 25 Aug 2020 18:30:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C6A772074D for ; Tue, 25 Aug 2020 18:30:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="jDfTkfnu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726759AbgHYSaG (ORCPT ); Tue, 25 Aug 2020 14:30:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726551AbgHYS31 (ORCPT ); Tue, 25 Aug 2020 14:29:27 -0400 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2932C061795 for ; Tue, 25 Aug 2020 11:29:26 -0700 (PDT) Received: by mail-ej1-x644.google.com with SMTP id m22so17813691eje.10 for ; Tue, 25 Aug 2020 11:29:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7EGajeDiJIxrlPr3Y0NRvAuPtl3AEsFbIm2NTrH6ZEc=; b=jDfTkfnuojbMaFUK6N1gavlgXoz0UOitqddOW0kO7XGsBtyCIdWvVGP4Ali167FpVl +8fZXzYunItxxegZYSeFNDr+lSOHwnPn/FI8sdZL/Jmm+9TIQS005cX+6+E9SOoujGw1 aSz2+ffiizsI2Lg0F9RC26uoeZVLjUSIq012E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7EGajeDiJIxrlPr3Y0NRvAuPtl3AEsFbIm2NTrH6ZEc=; b=F5r2nDeXI+XFdLNAQaOajubMtZd4qJWpvzwmW1HqUOuLyCYPn4UaPqrvlG8rYFLE6c 6Y/IUN/C2CQmJK539brO/FseM61KBlfTAormEKL6b7gdiOQDNlgfpOpD/qJ9pqPHJ9mS NaiFmKmtL2UEqYnGrTnJoIEQuxJdDKfn9wMsYZqJEPiiEpCiEThVa8M2kvN1WwiIbK+6 t0C7cPfZ/CVHvf6/ZEv4BUXRHo5WstUIp+3SlNS032aWy0oXj0KMjypP9cOQfDuV7+LS nlp5vuAk3B1BNrq5PdkZYqaeSxnBrzTnBY15ZmC3QcyiTuvRQtUiKVppb+VZZnVh7W9N lXpg== X-Gm-Message-State: AOAM533lwXE8dqJjUnmTNDJb3BSLsM3We0DhSaBm+xKMa1FuAOBdeFac Y+HM6O/O/W2QxxzJmXzxs6MROA== X-Google-Smtp-Source: ABdhPJxwWmTdSKRTZoliBYv1yxoma3Uae22mPp0EOYaM+udhulIxdQLSMBipzg5LcWTHCcC3UvKnbg== X-Received: by 2002:a17:906:4f11:: with SMTP id t17mr8838115eju.371.1598380165443; Tue, 25 Aug 2020 11:29:25 -0700 (PDT) Received: from kpsingh.zrh.corp.google.com ([81.6.44.51]) by smtp.gmail.com with ESMTPSA id dr21sm15323286ejc.112.2020.08.25.11.29.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 11:29:24 -0700 (PDT) From: KP Singh To: linux-kernel@vger.kernel.org, bpf@vger.kernel.org, linux-security-module@vger.kernel.org Cc: Martin KaFai Lau , Alexei Starovoitov , Daniel Borkmann , Paul Turner , Jann Horn , Florent Revest Subject: [PATCH bpf-next v10 2/7] bpf: Generalize caching for sk_storage. Date: Tue, 25 Aug 2020 20:29:14 +0200 Message-Id: <20200825182919.1118197-3-kpsingh@chromium.org> X-Mailer: git-send-email 2.28.0.297.g1956fa8f8d-goog In-Reply-To: <20200825182919.1118197-1-kpsingh@chromium.org> References: <20200825182919.1118197-1-kpsingh@chromium.org> MIME-Version: 1.0 Sender: owner-linux-security-module@vger.kernel.org Precedence: bulk List-ID: From: KP Singh Provide the a ability to define local storage caches on a per-object type basis. The caches and caching indices for different objects should not be inter-mixed as suggested in: https://lore.kernel.org/bpf/20200630193441.kdwnkestulg5erii@kafai-mbp.dhcp.thefacebook.com/ "Caching a sk-storage at idx=0 of a sk should not stop an inode-storage to be cached at the same idx of a inode." Acked-by: Martin KaFai Lau Signed-off-by: KP Singh --- include/net/bpf_sk_storage.h | 19 +++++++++++++++++++ net/core/bpf_sk_storage.c | 31 +++++++++++++++---------------- 2 files changed, 34 insertions(+), 16 deletions(-) diff --git a/include/net/bpf_sk_storage.h b/include/net/bpf_sk_storage.h index 5036c94c0503..950c5aaba15e 100644 --- a/include/net/bpf_sk_storage.h +++ b/include/net/bpf_sk_storage.h @@ -3,6 +3,9 @@ #ifndef _BPF_SK_STORAGE_H #define _BPF_SK_STORAGE_H +#include +#include + struct sock; void bpf_sk_storage_free(struct sock *sk); @@ -15,6 +18,22 @@ struct sk_buff; struct nlattr; struct sock; +#define BPF_LOCAL_STORAGE_CACHE_SIZE 16 + +struct bpf_local_storage_cache { + spinlock_t idx_lock; + u64 idx_usage_counts[BPF_LOCAL_STORAGE_CACHE_SIZE]; +}; + +#define DEFINE_BPF_STORAGE_CACHE(name) \ +static struct bpf_local_storage_cache name = { \ + .idx_lock = __SPIN_LOCK_UNLOCKED(name.idx_lock), \ +} + +u16 bpf_local_storage_cache_idx_get(struct bpf_local_storage_cache *cache); +void bpf_local_storage_cache_idx_free(struct bpf_local_storage_cache *cache, + u16 idx); + #ifdef CONFIG_BPF_SYSCALL int bpf_sk_storage_clone(const struct sock *sk, struct sock *newsk); struct bpf_sk_storage_diag * diff --git a/net/core/bpf_sk_storage.c b/net/core/bpf_sk_storage.c index f975e2d01207..ec61ee7c7ee4 100644 --- a/net/core/bpf_sk_storage.c +++ b/net/core/bpf_sk_storage.c @@ -14,6 +14,8 @@ #define BPF_LOCAL_STORAGE_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_CLONE) +DEFINE_BPF_STORAGE_CACHE(sk_cache); + struct bpf_local_storage_map_bucket { struct hlist_head list; raw_spinlock_t lock; @@ -78,10 +80,6 @@ struct bpf_local_storage_elem { #define SELEM(_SDATA) \ container_of((_SDATA), struct bpf_local_storage_elem, sdata) #define SDATA(_SELEM) (&(_SELEM)->sdata) -#define BPF_LOCAL_STORAGE_CACHE_SIZE 16 - -static DEFINE_SPINLOCK(cache_idx_lock); -static u64 cache_idx_usage_counts[BPF_LOCAL_STORAGE_CACHE_SIZE]; struct bpf_local_storage { struct bpf_local_storage_data __rcu *cache[BPF_LOCAL_STORAGE_CACHE_SIZE]; @@ -521,16 +519,16 @@ static int sk_storage_delete(struct sock *sk, struct bpf_map *map) return 0; } -static u16 cache_idx_get(void) +u16 bpf_local_storage_cache_idx_get(struct bpf_local_storage_cache *cache) { u64 min_usage = U64_MAX; u16 i, res = 0; - spin_lock(&cache_idx_lock); + spin_lock(&cache->idx_lock); for (i = 0; i < BPF_LOCAL_STORAGE_CACHE_SIZE; i++) { - if (cache_idx_usage_counts[i] < min_usage) { - min_usage = cache_idx_usage_counts[i]; + if (cache->idx_usage_counts[i] < min_usage) { + min_usage = cache->idx_usage_counts[i]; res = i; /* Found a free cache_idx */ @@ -538,18 +536,19 @@ static u16 cache_idx_get(void) break; } } - cache_idx_usage_counts[res]++; + cache->idx_usage_counts[res]++; - spin_unlock(&cache_idx_lock); + spin_unlock(&cache->idx_lock); return res; } -static void cache_idx_free(u16 idx) +void bpf_local_storage_cache_idx_free(struct bpf_local_storage_cache *cache, + u16 idx) { - spin_lock(&cache_idx_lock); - cache_idx_usage_counts[idx]--; - spin_unlock(&cache_idx_lock); + spin_lock(&cache->idx_lock); + cache->idx_usage_counts[idx]--; + spin_unlock(&cache->idx_lock); } /* Called by __sk_destruct() & bpf_sk_storage_clone() */ @@ -601,7 +600,7 @@ static void bpf_local_storage_map_free(struct bpf_map *map) smap = (struct bpf_local_storage_map *)map; - cache_idx_free(smap->cache_idx); + bpf_local_storage_cache_idx_free(&sk_cache, smap->cache_idx); /* Note that this map might be concurrently cloned from * bpf_sk_storage_clone. Wait for any existing bpf_sk_storage_clone @@ -718,7 +717,7 @@ static struct bpf_map *bpf_local_storage_map_alloc(union bpf_attr *attr) smap->elem_size = sizeof(struct bpf_local_storage_elem) + attr->value_size; - smap->cache_idx = cache_idx_get(); + smap->cache_idx = bpf_local_storage_cache_idx_get(&sk_cache); return &smap->map; }