From patchwork Tue May 7 12:42:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leone Fernando X-Patchwork-Id: 13656799 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1A5E6CDB1; Tue, 7 May 2024 12:43:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.196 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085832; cv=none; b=oPu9J7XrET+aHkR85gVwNtZ2pRQ9QKeM8yjnAo9hV52LEyjse6ucM7XKTrCd6+e5LbBrruSkhiYoOYmSewiaQu1Mc6APOV5TfK+nIvXarJtFniJ95nUn2Q+w6bHKWPbKr8qpXkzwBQOXlmqJPrKvfr0uyxQd1CBr/RFTudID/JE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085832; c=relaxed/simple; bh=Fdw3Uq6FEPRXufspQMbgG3TS8Lw8taqj/1zCK6girO8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RNo345F7dW3X0IPDpfXLoD9COpCMqnw/eGSKaMnKXWblGQIkfgQ8L/F6+D8Gzzz3KoK1PaFyuKWQsLRO44vWiC+AGgwCY5d0xItVKI/wWfNcNUq5z5DUGoQAnXLwYXYxwhUjw7xWYN87Pr3C6c78pKsDe/8L2wmFINyd3MEWqfg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BWjsXC49; arc=none smtp.client-ip=209.85.208.196 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BWjsXC49" Received: by mail-lj1-f196.google.com with SMTP id 38308e7fff4ca-2e242b1df60so37348921fa.1; Tue, 07 May 2024 05:43:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715085829; x=1715690629; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=N8GyKjDI+jVJzoUqihNVdIo6Wq2mpBnHDTxCj86/HJU=; b=BWjsXC49kuNvJiiE1c5BPKNckopzi9aW8nibdwhzN5x4i+VZz2t2ZZky2hMPXgkQyi PIK+UGL/W+2F9HKPXerAAW1hhgycsN70UgEJmoyJ90pLvNq678KGipT4cAm8cqwuKw4Q bXWs6Cliix1k033utyibEv65bYZVwWBFxDnzSe0/je9p3avnoYY9OxScAtucfDOZtF7A bgENjft3RajX68x4g7MOKZxboRGDaH+6QNjw13GVn3/3c19sIs9/MAZF7gt3WIWdD27g 9kHKGYIpeEmPFgG7KH1R9JVhqfLqfAyko4yg21Rl15UCB7ZSo6ui+1t9wZz5UNG6KWUG sqlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715085829; x=1715690629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N8GyKjDI+jVJzoUqihNVdIo6Wq2mpBnHDTxCj86/HJU=; b=AWjr8YSM+mteuocwTBgm7XEvz9MBqGvZCkzzCE85tc5Fu6NMTRvDDNsYMYDpp+QRCz EvZWA7lA7ff7vjteeiWssriAWNenBAbguIFJeUCaxidmtt7SJGf0YUYnlXjWKHZOgYCR 5S3ID565VJ2vA7JuHV0LR0Wl1XMi7jFUkJNtH9JL+7nCtLTKnB1j1o7T78bNlD/2a8sY K1JyAcTmn2mxllv6stK60fNacgmO/wZS98dWdM7nmN/gUhEiqacjoPmnN1fLH7KmCEEO 7VPlRBcypzPHO0ep3pMyPCYKnk+vMWLSSFYYrHlqCNwO4BPNNe0OdTlsXCG9S9+i0d5S 3P+A== X-Forwarded-Encrypted: i=1; AJvYcCUQ3h0Et+cIo8fPdQDsxbWCGzsOfYpTHwa6mNrxXg1AIfWeTClpPpfnO+ZqpePpT70jFnPl+5e8wqZhd0ogVJykA2M1wCTywT08jwMwsgw5aR+HG/3iT5Im76zAcvgFfpSs8uLs X-Gm-Message-State: AOJu0YxBwEBL1Xncp2m8ZHSJVzTjDJnIXZHty+mtKYSkdtYZ7ZPJekkY HiVx/b9v/5DmQRAbCLFfXHweqaLW0O9UKonqGmAEGwBb6Vt3aTLN X-Google-Smtp-Source: AGHT+IFHc+0z1+JCQJPFoXA3akwq8HNYDjHWGqqrWzoTJ1buSar8nUO1akR4p6xn3GSzCitM+ru0Bw== X-Received: by 2002:a2e:a6a1:0:b0:2df:faf0:c5e4 with SMTP id q33-20020a2ea6a1000000b002dffaf0c5e4mr11218713lje.23.1715085828649; Tue, 07 May 2024 05:43:48 -0700 (PDT) Received: from localhost ([45.130.85.2]) by smtp.gmail.com with ESMTPSA id j13-20020a05600c1c0d00b0041be58cdf83sm19555059wms.4.2024.05.07.05.43.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 May 2024 05:43:48 -0700 (PDT) From: Leone Fernando To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemb@google.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Leone Fernando Subject: [PATCH net-next v2 1/4] net: route: expire rt if the dst it holds is expired Date: Tue, 7 May 2024 14:42:26 +0200 Message-Id: <20240507124229.446802-2-leone4fernando@gmail.com> In-Reply-To: <20240507124229.446802-1-leone4fernando@gmail.com> References: <20240507124229.446802-1-leone4fernando@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org The function rt_is_expired is used to verify that a cached dst is valid. Currently, this function ignores the rt.dst->expires value. Add a check to rt_is_expired that validates that the dst is not expired. Signed-off-by: Leone Fernando --- net/ipv4/route.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 5fd54103174f..4e6b7a67f177 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -389,7 +389,8 @@ static inline int ip_rt_proc_init(void) static inline bool rt_is_expired(const struct rtable *rth) { - return rth->rt_genid != rt_genid_ipv4(dev_net(rth->dst.dev)); + return rth->rt_genid != rt_genid_ipv4(dev_net(rth->dst.dev)) || + (rth->dst.expires && time_after(jiffies, rth->dst.expires)); } void rt_cache_flush(struct net *net) From patchwork Tue May 7 12:42:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leone Fernando X-Patchwork-Id: 13656800 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BC04158D9C; Tue, 7 May 2024 12:44:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.65 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085860; cv=none; b=PNxTrclI+TtC4aJ+eCqctHncLilKAILfC/lc5yq6X+yErLefMOx7al9makha2p51kGg4gsIV47AuKBT7NQR2SARaPri/F8UE3nw3SsUcFhKVXz4GOREkSzDuXrKtMJW8T4c/Nbv9o2oOnU4XGVmWz89RAHO1YBkyF3exOtpx9KQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085860; c=relaxed/simple; bh=uM5sJnWrOplndwtlhh0T7d3Eeti4vJVik1sqvT2i43A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pjIUxWz7Fn7Zg0Ln+kHfWvplmCWlbl37Zam9s2swpGRqDfnzhyGo5A0KrkRN+vvMtqg7UD07MG1Mf6b/RnjhnT7p0glJydN+thhTr9TmBsgL61WKNQxXyVPADQDw9Jq+jAlFTGObkpjSdhStVbbTChgJIJoRR5tAjwFSTVNGTY8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JV0pymax; arc=none smtp.client-ip=209.85.221.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JV0pymax" Received: by mail-wr1-f65.google.com with SMTP id ffacd0b85a97d-34db6a29a1eso1996057f8f.1; Tue, 07 May 2024 05:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715085857; x=1715690657; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lWpqb3t6CANxEDWhaC7kiYcjYm92T0oXtF4rwdCOiDo=; b=JV0pymaxgFKwP2tlaOKWQ2DUoQ9ZW0aRvOM6w1YCEzKbI+LvZXIXf1yP8xQBIqqUZO Hj//BAwgECR3AV1xNSO9+NjLSBKooJnK099P/VTE7BDzV6gmPLT3W0Zy9z7CjWSLk9VJ DFo/Egvkqe4dLUa7KUUegz1uCQAOpxntiURYzsE/eoF1sxlWuHiXcarSQF0VPvw4kzyO x3TFsNpbxlzVxOeeaWJ+Cb/tqoeSwVqrRex36lkbS/aaaYHqDOU3sd1M/tiSShVL2efA rZ85TnVRy7UaobFQakyFPE42Lxrx2WfR8xNj8LXL4LUnF08nPv4En8SlzGRIIO7uMNVV TDZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715085857; x=1715690657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lWpqb3t6CANxEDWhaC7kiYcjYm92T0oXtF4rwdCOiDo=; b=soePD6Ef5kGSQySnRNdAoV8vh4ZYOHHIkKv7A6GR+ro16LPaufpflXNQitUz6VMlal DVeCSMvP8DfCjoSocyo1he1VtBKdYqPFfUMjCJaQ92uCKP+r17fTFJlgaYH6XxUXlG6a hQRroIQCZ8C+OvnY2vipRd5VgRdHTxVnvvfN5LmoqokRp3jy0a+dxJpU/ob8snLxQBd9 /Z5Xp6EcHKoJl78I30541JSdBt5VWEy5FzWNspFCCnEIUTv7JGVEaS4XfgL/RBfU0Xtd S4Iqw/t6sQWctatsTqFCGZ1BFRmBwJc4IDMUTUOX9LWqI+XiG2OlA/p3PscR2lwKUM7d MQyw== X-Forwarded-Encrypted: i=1; AJvYcCX4tipqnd2kpUguqtQ0tukOcZs4y+HCKH+gcrmH5dxVVUQCdIPekZR4KFo5XTVzfqamu2QuJ8h9RIzEV3XUZ3FDA3i1CTxL+/H+BiWSg/HVdjbIG1/vrxvTskULyDEULjnH9FTW X-Gm-Message-State: AOJu0YyuNZhlbbKMXskwhSnzFNyl0HMvI8iwV4ZPkZSo2mUPXGKJt6PL ty7hNou0FQIt/lCDkGkj3fHcshziwQ1zXgIynqwrREAR4l0ar7Jx X-Google-Smtp-Source: AGHT+IHgawRJtasMMLApQc5OHFyDRvuX4yFM6a3R+XcG8MbgkY8XrJ4Xajdc8fdCyowexgppBftf2w== X-Received: by 2002:adf:ebd2:0:b0:34f:824:17b2 with SMTP id v18-20020adfebd2000000b0034f082417b2mr3978192wrn.65.1715085856605; Tue, 07 May 2024 05:44:16 -0700 (PDT) Received: from localhost ([45.130.85.2]) by smtp.gmail.com with ESMTPSA id n8-20020adffe08000000b0034df2d0bd71sm12864608wrr.12.2024.05.07.05.44.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 May 2024 05:44:16 -0700 (PDT) From: Leone Fernando To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemb@google.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Leone Fernando Subject: [PATCH net-next v2 2/4] net: dst_cache: add input_dst_cache API Date: Tue, 7 May 2024 14:42:27 +0200 Message-Id: <20240507124229.446802-3-leone4fernando@gmail.com> In-Reply-To: <20240507124229.446802-1-leone4fernando@gmail.com> References: <20240507124229.446802-1-leone4fernando@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org The input_dst_cache allows fast lookup of frequently encountered dsts. In order to provide stable results, I implemented a simple linear hashtable with each bucket containing a constant amount of entries (DST_CACHE_INPUT_BUCKET_SIZE). Similarly to how the route hint is used, I defined the hashtable key to contain the daddr and the tos of the IP header. Lookup is performed in a straightforward manner: start at the bucket head corresponding the hashed key and search the following DST_CACHE_INPUT_BUCKET_SIZE entries of the array for a matching key. When inserting a new dst to the cache, if all the bucket entries are full, the oldest one is deleted to make room for the new dst. Signed-off-by: Leone Fernando --- include/net/dst_cache.h | 68 +++++++++++++++++++++ net/core/dst_cache.c | 132 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 200 insertions(+) diff --git a/include/net/dst_cache.h b/include/net/dst_cache.h index df6622a5fe98..b6f012ffeb86 100644 --- a/include/net/dst_cache.h +++ b/include/net/dst_cache.h @@ -7,12 +7,40 @@ #if IS_ENABLED(CONFIG_IPV6) #include #endif +#include + +#define DST_CACHE_INPUT_SHIFT (9) +#define DST_CACHE_INPUT_SIZE (1 << DST_CACHE_INPUT_SHIFT) +#define DST_CACHE_INPUT_BUCKET_SIZE (4) +#define DST_CACHE_INPUT_HASH_MASK (~(DST_CACHE_INPUT_BUCKET_SIZE - 1)) +#define INVALID_DST_CACHE_INPUT_KEY (~(u64)(0)) struct dst_cache { struct dst_cache_pcpu __percpu *cache; unsigned long reset_ts; }; +extern unsigned int dst_cache_net_id __read_mostly; + +/** + * idst_for_each_in_bucket - iterate over a dst cache bucket + * @pos: the type * to use as a loop cursor + * @head: the head of the cpu dst cache. + * @hash: the hash of the bucket + */ +#define idst_for_each_in_bucket(pos, head, hash) \ + for (pos = &head[hash]; \ + pos < &head[hash + DST_CACHE_INPUT_BUCKET_SIZE]; \ + pos++) + +/** + * idst_for_each_in_cache - iterate over the dst cache + * @pos: the type * to use as a loop cursor + * @head: the head of the cpu dst cache. + */ +#define idst_for_each_in_cache(pos, head) \ + for (pos = head; pos < head + DST_CACHE_INPUT_SIZE; pos++) + /** * dst_cache_get - perform cache lookup * @dst_cache: the cache @@ -106,4 +134,44 @@ int dst_cache_init(struct dst_cache *dst_cache, gfp_t gfp); */ void dst_cache_destroy(struct dst_cache *dst_cache); +/** + * dst_cache_input_get_noref - perform lookup in the input cache, + * return a noref dst + * @dst_cache: the input cache + * @skb: the packet according to which the dst entry will be searched + * local BH must be disabled. + */ +struct dst_entry *dst_cache_input_get_noref(struct dst_cache *dst_cache, + struct sk_buff *skb); + +/** + * dst_cache_input_add - add the dst of the given skb to the input cache. + * + * in case the cache bucket is full, the oldest entry will be deleted + * and replaced with the new one. + * @dst_cache: the input cache + * @skb: The packet according to which the dst entry will be searched + * + * local BH must be disabled. + */ +void dst_cache_input_add(struct dst_cache *dst_cache, + const struct sk_buff *skb); + +/** + * dst_cache_input_init - initialize the input cache, + * allocating the required storage + */ +int __init dst_cache_input_init(void); + +static inline u64 create_dst_cache_key_ip4(const struct sk_buff *skb) +{ + struct iphdr *iphdr = ip_hdr(skb); + + return (((u64)ntohl(iphdr->daddr)) << 8) | iphdr->tos; +} + +static inline u32 hash_dst_cache_key(u64 key) +{ + return hash_64(key, DST_CACHE_INPUT_SHIFT) & DST_CACHE_INPUT_HASH_MASK; +} #endif diff --git a/net/core/dst_cache.c b/net/core/dst_cache.c index 0c0bdb058c5b..843da4679488 100644 --- a/net/core/dst_cache.c +++ b/net/core/dst_cache.c @@ -13,6 +13,7 @@ #include #endif #include +#include struct dst_cache_pcpu { unsigned long refresh_ts; @@ -21,9 +22,12 @@ struct dst_cache_pcpu { union { struct in_addr in_saddr; struct in6_addr in6_saddr; + u64 key; }; }; +unsigned int dst_cache_net_id __read_mostly; + static void dst_cache_per_cpu_dst_set(struct dst_cache_pcpu *dst_cache, struct dst_entry *dst, u32 cookie) { @@ -181,3 +185,131 @@ void dst_cache_reset_now(struct dst_cache *dst_cache) } } EXPORT_SYMBOL_GPL(dst_cache_reset_now); + +static void dst_cache_input_set(struct dst_cache_pcpu *idst, + struct dst_entry *dst, u64 key) +{ + dst_cache_per_cpu_dst_set(idst, dst, 0); + idst->key = key; + idst->refresh_ts = jiffies; +} + +static struct dst_entry *__dst_cache_input_get_noref(struct dst_cache_pcpu *idst) +{ + struct dst_entry *dst = idst->dst; + + if (unlikely(dst->obsolete && !dst->ops->check(dst, idst->cookie))) { + dst_cache_input_set(idst, NULL, INVALID_DST_CACHE_INPUT_KEY); + goto fail; + } + + idst->refresh_ts = jiffies; + return dst; + +fail: + return NULL; +} + +struct dst_entry *dst_cache_input_get_noref(struct dst_cache *dst_cache, + struct sk_buff *skb) +{ + struct dst_entry *out_dst = NULL; + struct dst_cache_pcpu *pcpu_cache; + struct dst_cache_pcpu *idst; + u32 hash; + u64 key; + + pcpu_cache = this_cpu_ptr(dst_cache->cache); + key = create_dst_cache_key_ip4(skb); + hash = hash_dst_cache_key(key); + idst_for_each_in_bucket(idst, pcpu_cache, hash) { + if (key == idst->key) { + out_dst = __dst_cache_input_get_noref(idst); + goto out; + } + } +out: + return out_dst; +} + +static void dst_cache_input_reset_now(struct dst_cache *dst_cache) +{ + struct dst_cache_pcpu *caches; + struct dst_cache_pcpu *idst; + struct dst_entry *dst; + int i; + + for_each_possible_cpu(i) { + caches = per_cpu_ptr(dst_cache->cache, i); + idst_for_each_in_cache(idst, caches) { + idst->key = INVALID_DST_CACHE_INPUT_KEY; + dst = idst->dst; + if (dst) + dst_release(dst); + } + } +} + +static int __net_init dst_cache_input_net_init(struct net *net) +{ + struct dst_cache *dst_cache = net_generic(net, dst_cache_net_id); + + dst_cache->cache = (struct dst_cache_pcpu __percpu *)alloc_percpu_gfp(struct dst_cache_pcpu[DST_CACHE_INPUT_SIZE], + GFP_KERNEL | __GFP_ZERO); + if (!dst_cache->cache) + return -ENOMEM; + + dst_cache_input_reset_now(dst_cache); + return 0; +} + +static void __net_exit dst_cache_input_net_exit(struct net *net) +{ + struct dst_cache *dst_cache = net_generic(net, dst_cache_net_id); + + dst_cache_input_reset_now(dst_cache); + free_percpu(dst_cache->cache); + dst_cache->cache = NULL; +} + +static bool idst_empty(struct dst_cache_pcpu *idst) +{ + return idst->key == INVALID_DST_CACHE_INPUT_KEY; +} + +void dst_cache_input_add(struct dst_cache *dst_cache, const struct sk_buff *skb) +{ + struct dst_cache_pcpu *entry = NULL; + struct dst_cache_pcpu *pcpu_cache; + struct dst_cache_pcpu *idst; + u32 hash; + u64 key; + + pcpu_cache = this_cpu_ptr(dst_cache->cache); + key = create_dst_cache_key_ip4(skb); + hash = hash_dst_cache_key(key); + idst_for_each_in_bucket(idst, pcpu_cache, hash) { + if (idst_empty(idst)) { + entry = idst; + goto add_to_cache; + } + if (!entry || time_before(idst->refresh_ts, entry->refresh_ts)) + entry = idst; + } + +add_to_cache: + dst_cache_input_set(entry, skb_dst(skb), key); +} + +static struct pernet_operations dst_cache_input_ops __net_initdata = { + .init = dst_cache_input_net_init, + .exit = dst_cache_input_net_exit, + .id = &dst_cache_net_id, + .size = sizeof(struct dst_cache), +}; + +int __init dst_cache_input_init(void) +{ + return register_pernet_subsys(&dst_cache_input_ops); +} +subsys_initcall(dst_cache_input_init); From patchwork Tue May 7 12:42:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leone Fernando X-Patchwork-Id: 13656801 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E8A1158D6D; Tue, 7 May 2024 12:44:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.67 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085890; cv=none; b=deSbdvcTzio9Jy8LsQbqnRUjS0B98QFYcHf2BA/TCxMmR+F85pwe8Xs/TOgr3egQ/u3yH3//Cp87/8lWY1wf3dt+QxJyzC1ZVs18z3ZVP3H4R+DBczPPUMGU0g7PAoY4PkcF7r7AmCW5FgTWVHPeVjSlZHkBL9oEqx7d8CGxIoU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085890; c=relaxed/simple; bh=VHaDqy4uCN9hdVx42KaFqF/f4VYZwWy/OWJxc5HEReo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cvz/hcPSvHABCySsPKKVh3TrhJ/LcTDem2HNdiGoKVDnSBWy33sfQuavuKTeWPw4hWmGUacnUEdlzpO+0cCv/2YMH534c2ygHtHYFz6eh5iIq0Seu2EVV3uDIsjqItm+qiE6YFR7gFc1c/WHo9zuWIA+AJcVeQVqgB+wpm69uXg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=h0prA9tn; arc=none smtp.client-ip=209.85.221.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h0prA9tn" Received: by mail-wr1-f67.google.com with SMTP id ffacd0b85a97d-34e0d8b737eso3163011f8f.1; Tue, 07 May 2024 05:44:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715085887; x=1715690687; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5/QYkjOUDr+8s3NA0sZvLBYnsJKMzaMJ2QY9V4Fij30=; b=h0prA9tnc5qAi4RFThtGQfyYgXS4k0t3NlX4PKum4s+fvPrFqh0PsmgW0Z/yaVUR/p iKfcYPPNznUuWVBPQpBmk3VirE8RjJT+5LscKjZKXULG+EYpAHkZ8b0IuaTU4IDpW653 fdSkK+bMwBiyuNAK14Rkf38XgwToXf2pfNYs14GVlutWHViyw/tXrToVJho5iObtYrxN SulDqx3TXtgKPfZVssMiQRXoQ9WoI3LKnUMKqNe0JSOJDg83z7P/2R3llck6USjSBenr 7MmJOLE1S30JwVIG0N6UT7MGZpwB4O0SJt2hdGkxWfILV0ryr7nE5ifGt+eaIeXUGGjy tgSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715085887; x=1715690687; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5/QYkjOUDr+8s3NA0sZvLBYnsJKMzaMJ2QY9V4Fij30=; b=cJGki1AkxevfmCFitXx06jFnL1P604Xxucs1iZSXDPYt7OoeCU5JmBA6Jf1oOgu0wx B6zUCumwUJWiDwWlG2n9PczotAHu6ZOHHZuX7Ct2V7YE3DNGKEq+I2gR/5IFVkla/ljn P6vOq2vW5Ywcc90I8vSvgJNv3cqCVDo90JviMhIkl+24PnAskzZccatl4jPXXPntgW10 MCpCY4WHs8Gz3X/4FvV6aypFmfIIPny9P5e51T7nehqMF1RWgnAE2qINMFKvxkC13ygK cfxOKneOtpfjR1gs0G0sEOp8llo5nX53BZ9kz9ipGkSOaZC+Hq78u8AMNLdOJySFcrSh b2Sw== X-Forwarded-Encrypted: i=1; AJvYcCViOq6kLIHEhdqLaidi4I0ZXN2h3wAWXr93DeS7eRIvXTDlWlrN+VnRc0izXyX10EPt3VUrwvcokhPgpPIBM1+HAXvNZJAAeNSotkhI6mvhiRTq+QKRWW8DeRqClihkQNAPOI0A X-Gm-Message-State: AOJu0Ywd5wefTm2ibGq0kT5eg10cxu12Ru/Y/MlAwTCcbqcBtCDfsl0H /k+s7ZMdnEycPfcnLidG3qaykCmXn0M0stqI3BMbXHyHle5WdTyE X-Google-Smtp-Source: AGHT+IHC3qNccXCFnuRqIob+JAHZIErqZ51mXsd8Hb+SDMcwyF3PJXzjvhA11tgjYLlzsd6n7fsdpg== X-Received: by 2002:a5d:4304:0:b0:34d:75f3:4a77 with SMTP id ffacd0b85a97d-34f8174bb68mr2199109f8f.6.1715085886570; Tue, 07 May 2024 05:44:46 -0700 (PDT) Received: from localhost ([45.130.85.2]) by smtp.gmail.com with ESMTPSA id h9-20020a05600c350900b00417ee886977sm23467416wmq.4.2024.05.07.05.44.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 May 2024 05:44:46 -0700 (PDT) From: Leone Fernando To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemb@google.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Leone Fernando Subject: [PATCH net-next v2 3/4] net: route: always compile dst_cache Date: Tue, 7 May 2024 14:42:28 +0200 Message-Id: <20240507124229.446802-4-leone4fernando@gmail.com> In-Reply-To: <20240507124229.446802-1-leone4fernando@gmail.com> References: <20240507124229.446802-1-leone4fernando@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org make dst_cache to always compile, delete all relevant ifdefs Signed-off-by: Leone Fernando --- drivers/net/Kconfig | 1 - include/net/dst_metadata.h | 2 -- include/net/ip_tunnels.h | 2 -- net/Kconfig | 4 ---- net/core/Makefile | 3 +-- net/core/dst.c | 4 ---- net/ipv4/Kconfig | 1 - net/ipv4/ip_tunnel_core.c | 4 ---- net/ipv4/udp_tunnel_core.c | 4 ---- net/ipv6/Kconfig | 4 ---- net/ipv6/ip6_udp_tunnel.c | 4 ---- net/netfilter/nft_tunnel.c | 2 -- net/openvswitch/Kconfig | 1 - net/sched/act_tunnel_key.c | 2 -- 14 files changed, 1 insertion(+), 37 deletions(-) diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig index 9920b3a68ed1..6819ce6db863 100644 --- a/drivers/net/Kconfig +++ b/drivers/net/Kconfig @@ -78,7 +78,6 @@ config WIREGUARD depends on IPV6 || !IPV6 depends on !KMSAN # KMSAN doesn't support the crypto configs below select NET_UDP_TUNNEL - select DST_CACHE select CRYPTO select CRYPTO_LIB_CURVE25519 select CRYPTO_LIB_CHACHA20POLY1305 diff --git a/include/net/dst_metadata.h b/include/net/dst_metadata.h index 4160731dcb6e..46cebd8ea374 100644 --- a/include/net/dst_metadata.h +++ b/include/net/dst_metadata.h @@ -165,7 +165,6 @@ static inline struct metadata_dst *tun_dst_unclone(struct sk_buff *skb) memcpy(&new_md->u.tun_info, &md_dst->u.tun_info, sizeof(struct ip_tunnel_info) + md_size); -#ifdef CONFIG_DST_CACHE /* Unclone the dst cache if there is one */ if (new_md->u.tun_info.dst_cache.cache) { int ret; @@ -176,7 +175,6 @@ static inline struct metadata_dst *tun_dst_unclone(struct sk_buff *skb) return ERR_PTR(ret); } } -#endif skb_dst_drop(skb); skb_dst_set(skb, &new_md->dst); diff --git a/include/net/ip_tunnels.h b/include/net/ip_tunnels.h index 9a6a08ec7713..ed1b36a5ae52 100644 --- a/include/net/ip_tunnels.h +++ b/include/net/ip_tunnels.h @@ -102,9 +102,7 @@ struct ip_tunnel_encap { struct ip_tunnel_info { struct ip_tunnel_key key; struct ip_tunnel_encap encap; -#ifdef CONFIG_DST_CACHE struct dst_cache dst_cache; -#endif u8 options_len; u8 mode; }; diff --git a/net/Kconfig b/net/Kconfig index d5ab791f7afa..cd5304e6cba2 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -441,10 +441,6 @@ config LWTUNNEL_BPF Allows to run BPF programs as a nexthop action following a route lookup for incoming and outgoing packets. -config DST_CACHE - bool - default n - config GRO_CELLS bool default n diff --git a/net/core/Makefile b/net/core/Makefile index 21d6fbc7e884..cfd327ae389e 100644 --- a/net/core/Makefile +++ b/net/core/Makefile @@ -13,7 +13,7 @@ obj-y += dev.o dev_addr_lists.o dst.o netevent.o \ neighbour.o rtnetlink.o utils.o link_watch.o filter.o \ sock_diag.o dev_ioctl.o tso.o sock_reuseport.o \ fib_notifier.o xdp.o flow_offload.o gro.o \ - netdev-genl.o netdev-genl-gen.o gso.o + netdev-genl.o netdev-genl-gen.o gso.o dst_cache.o obj-$(CONFIG_NETDEV_ADDR_LIST_TEST) += dev_addr_lists_test.o @@ -33,7 +33,6 @@ obj-$(CONFIG_CGROUP_NET_PRIO) += netprio_cgroup.o obj-$(CONFIG_CGROUP_NET_CLASSID) += netclassid_cgroup.o obj-$(CONFIG_LWTUNNEL) += lwtunnel.o obj-$(CONFIG_LWTUNNEL_BPF) += lwt_bpf.o -obj-$(CONFIG_DST_CACHE) += dst_cache.o obj-$(CONFIG_HWBM) += hwbm.o obj-$(CONFIG_GRO_CELLS) += gro_cells.o obj-$(CONFIG_FAILOVER) += failover.o diff --git a/net/core/dst.c b/net/core/dst.c index 95f533844f17..f035c39be104 100644 --- a/net/core/dst.c +++ b/net/core/dst.c @@ -291,10 +291,8 @@ EXPORT_SYMBOL_GPL(metadata_dst_alloc); void metadata_dst_free(struct metadata_dst *md_dst) { -#ifdef CONFIG_DST_CACHE if (md_dst->type == METADATA_IP_TUNNEL) dst_cache_destroy(&md_dst->u.tun_info.dst_cache); -#endif if (md_dst->type == METADATA_XFRM) dst_release(md_dst->u.xfrm_info.dst_orig); kfree(md_dst); @@ -326,10 +324,8 @@ void metadata_dst_free_percpu(struct metadata_dst __percpu *md_dst) for_each_possible_cpu(cpu) { struct metadata_dst *one_md_dst = per_cpu_ptr(md_dst, cpu); -#ifdef CONFIG_DST_CACHE if (one_md_dst->type == METADATA_IP_TUNNEL) dst_cache_destroy(&one_md_dst->u.tun_info.dst_cache); -#endif if (one_md_dst->type == METADATA_XFRM) dst_release(one_md_dst->u.xfrm_info.dst_orig); } diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig index 8e94ed7c56a0..189f716b03e8 100644 --- a/net/ipv4/Kconfig +++ b/net/ipv4/Kconfig @@ -185,7 +185,6 @@ config NET_IPGRE_DEMUX config NET_IP_TUNNEL tristate - select DST_CACHE select GRO_CELLS default n diff --git a/net/ipv4/ip_tunnel_core.c b/net/ipv4/ip_tunnel_core.c index a3676155be78..05fbc40c5d16 100644 --- a/net/ipv4/ip_tunnel_core.c +++ b/net/ipv4/ip_tunnel_core.c @@ -683,13 +683,11 @@ static int ip_tun_build_state(struct net *net, struct nlattr *attr, return err; } -#ifdef CONFIG_DST_CACHE err = dst_cache_init(&tun_info->dst_cache, GFP_KERNEL); if (err) { lwtstate_free(new_state); return err; } -#endif if (tb[LWTUNNEL_IP_ID]) tun_info->key.tun_id = nla_get_be64(tb[LWTUNNEL_IP_ID]); @@ -727,11 +725,9 @@ static int ip_tun_build_state(struct net *net, struct nlattr *attr, static void ip_tun_destroy_state(struct lwtunnel_state *lwtstate) { -#ifdef CONFIG_DST_CACHE struct ip_tunnel_info *tun_info = lwt_tun_info(lwtstate); dst_cache_destroy(&tun_info->dst_cache); -#endif } static int ip_tun_fill_encap_opts_geneve(struct sk_buff *skb, diff --git a/net/ipv4/udp_tunnel_core.c b/net/ipv4/udp_tunnel_core.c index e4e0fa869fa4..576ab973d1f4 100644 --- a/net/ipv4/udp_tunnel_core.c +++ b/net/ipv4/udp_tunnel_core.c @@ -216,13 +216,11 @@ struct rtable *udp_tunnel_dst_lookup(struct sk_buff *skb, struct rtable *rt = NULL; struct flowi4 fl4; -#ifdef CONFIG_DST_CACHE if (dst_cache) { rt = dst_cache_get_ip4(dst_cache, saddr); if (rt) return rt; } -#endif memset(&fl4, 0, sizeof(fl4)); fl4.flowi4_mark = skb->mark; @@ -245,10 +243,8 @@ struct rtable *udp_tunnel_dst_lookup(struct sk_buff *skb, ip_rt_put(rt); return ERR_PTR(-ELOOP); } -#ifdef CONFIG_DST_CACHE if (dst_cache) dst_cache_set_ip4(dst_cache, &rt->dst, fl4.saddr); -#endif *saddr = fl4.saddr; return rt; } diff --git a/net/ipv6/Kconfig b/net/ipv6/Kconfig index 08d4b7132d4c..093c768d41ab 100644 --- a/net/ipv6/Kconfig +++ b/net/ipv6/Kconfig @@ -124,7 +124,6 @@ config IPV6_MIP6 config IPV6_ILA tristate "IPv6: Identifier Locator Addressing (ILA)" depends on NETFILTER - select DST_CACHE select LWTUNNEL help Support for IPv6 Identifier Locator Addressing (ILA). @@ -203,7 +202,6 @@ config IPV6_NDISC_NODETYPE config IPV6_TUNNEL tristate "IPv6: IP-in-IPv6 tunnel (RFC2473)" select INET6_TUNNEL - select DST_CACHE select GRO_CELLS help Support for IPv6-in-IPv6 and IPv4-in-IPv6 tunnels described in @@ -291,7 +289,6 @@ config IPV6_SEG6_LWTUNNEL bool "IPv6: Segment Routing Header encapsulation support" depends on IPV6 select LWTUNNEL - select DST_CACHE select IPV6_MULTIPLE_TABLES help Support for encapsulation of packets within an outer IPv6 @@ -333,7 +330,6 @@ config IPV6_IOAM6_LWTUNNEL bool "IPv6: IOAM Pre-allocated Trace insertion support" depends on IPV6 select LWTUNNEL - select DST_CACHE help Support for the insertion of IOAM Pre-allocated Trace Header using the lightweight tunnels mechanism. diff --git a/net/ipv6/ip6_udp_tunnel.c b/net/ipv6/ip6_udp_tunnel.c index c99053189ea8..de92aea01cfc 100644 --- a/net/ipv6/ip6_udp_tunnel.c +++ b/net/ipv6/ip6_udp_tunnel.c @@ -145,13 +145,11 @@ struct dst_entry *udp_tunnel6_dst_lookup(struct sk_buff *skb, struct dst_entry *dst = NULL; struct flowi6 fl6; -#ifdef CONFIG_DST_CACHE if (dst_cache) { dst = dst_cache_get_ip6(dst_cache, saddr); if (dst) return dst; } -#endif memset(&fl6, 0, sizeof(fl6)); fl6.flowi6_mark = skb->mark; fl6.flowi6_proto = IPPROTO_UDP; @@ -173,10 +171,8 @@ struct dst_entry *udp_tunnel6_dst_lookup(struct sk_buff *skb, dst_release(dst); return ERR_PTR(-ELOOP); } -#ifdef CONFIG_DST_CACHE if (dst_cache) dst_cache_set_ip6(dst_cache, dst, &fl6.saddr); -#endif *saddr = fl6.saddr; return dst; } diff --git a/net/netfilter/nft_tunnel.c b/net/netfilter/nft_tunnel.c index 60a76e6e348e..aa4a872edae2 100644 --- a/net/netfilter/nft_tunnel.c +++ b/net/netfilter/nft_tunnel.c @@ -514,13 +514,11 @@ static int nft_tunnel_obj_init(const struct nft_ctx *ctx, return -ENOMEM; memcpy(&md->u.tun_info, &info, sizeof(info)); -#ifdef CONFIG_DST_CACHE err = dst_cache_init(&md->u.tun_info.dst_cache, GFP_KERNEL); if (err < 0) { metadata_dst_free(md); return err; } -#endif ip_tunnel_info_opts_set(&md->u.tun_info, &priv->opts.u, priv->opts.len, priv->opts.flags); priv->md = md; diff --git a/net/openvswitch/Kconfig b/net/openvswitch/Kconfig index 29a7081858cd..b7a5ab6374b8 100644 --- a/net/openvswitch/Kconfig +++ b/net/openvswitch/Kconfig @@ -13,7 +13,6 @@ config OPENVSWITCH select LIBCRC32C select MPLS select NET_MPLS_GSO - select DST_CACHE select NET_NSH select NF_CONNTRACK_OVS if NF_CONNTRACK select NF_NAT_OVS if NF_NAT diff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c index af7c99845948..9d673b642b7c 100644 --- a/net/sched/act_tunnel_key.c +++ b/net/sched/act_tunnel_key.c @@ -476,11 +476,9 @@ static int tunnel_key_init(struct net *net, struct nlattr *nla, goto err_out; } -#ifdef CONFIG_DST_CACHE ret = dst_cache_init(&metadata->u.tun_info.dst_cache, GFP_KERNEL); if (ret) goto release_tun_meta; -#endif if (opts_len) { ret = tunnel_key_opts_set(tb[TCA_TUNNEL_KEY_ENC_OPTS], From patchwork Tue May 7 12:42:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leone Fernando X-Patchwork-Id: 13656802 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82528158D9C; Tue, 7 May 2024 12:45:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.194 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085919; cv=none; b=CdhFupgN8TADLkK7YqzXyEZF4I9C0A5APExh8a7kSiLwulsV3/VpAZvqmxMpa0QTbSv+ywUuTcXPO82D9ZyG+AiN5kT2dm0AOEL8vgUwxNzKiMax1sh6TGuzdBYIV9l6eR4iimB6KvLPahjmwO8hGbOOaT8xbTcZyUMQgd7661c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715085919; c=relaxed/simple; bh=YN3/mWGuvT+JhJ7JR7zHSioayWmea4yIowhWbMS/Cnc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K/BLPtc0WOjX3C/1uU2rHRHyIW1sZdzoqFOeMJlYEujo0mEwTEV4VoiwG7AOUxnby9Gb1L2dnt9CK3GDGWGvc074IzPg/JfaUibQW1yRa4Lh3e+xRW7euaMuIA5nXDoRw9CA9sqyBWOuWHnN/QbJPCqZf85n0E/1Mt0HYs8iiHU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AHGGJRNx; arc=none smtp.client-ip=209.85.208.194 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AHGGJRNx" Received: by mail-lj1-f194.google.com with SMTP id 38308e7fff4ca-2e38a7ebdb6so20224041fa.2; Tue, 07 May 2024 05:45:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715085915; x=1715690715; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hIMeLvgtOJaPGTq4xRu76ebVHmu+7aXjg26c5XHdzoo=; b=AHGGJRNxMlguVGdnExvN89zV0Wl6OGtiwxFHSdt5DmESNy05u/CpCR5QZbnPPD3v0a d8FLp7YXzRMjdU3gHmCYYpNz6kGZPCwLGytL0sKlf1ovF5qmIKtQhevU08lkraksGT2f Di0iLeeMvgkIkrRwRu4/8rX6qEQrEEUERP9tN41PJSWyQSrQk0g5jNTME59oPiFtO90f +sa5NUr/SqdF+Tr5OsUpu6tdxu3OoG1atX4H4ob47CEtDdhYFyUiDcnj/mTOS/+FssFM 7pa74ZhXcFhW4mjHSHpdu93sfkQlVdUUahb9whKMyYrw+aW4gqWb2VRJMm340TVIusJf eahw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715085915; x=1715690715; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hIMeLvgtOJaPGTq4xRu76ebVHmu+7aXjg26c5XHdzoo=; b=FNNn2TVNfMqd5L3wPo2hUe0/YswxhQ+ilELuegskG4eHxAVZ3gQJtocf8VURl8Xxm8 FJHB3w263xhwyNThFw0DaJo2ToiJj1GUTcfAJ9MV+Xcy7i1Iu4YRTK/qa+OPvt8bYPN1 C0l2my5zCi+Yaco9ladqnxB+olIYLAuqpnp7xLIaOwM2plCMdWQnNnn6c0TSRu2r+otw PFURLrGOAq4YC3DbEnczfUwYFihDQLjSbN8l+KBdpJ9MnSedlPe3KRPa9g77LTfWBsTs qMDhPddNkIJ7fNpZWHI9sFsE7kpkdlGiSoDs/6NTjxaq6G6P33EnoSr9HnWhbKeq7jDj jDkg== X-Forwarded-Encrypted: i=1; AJvYcCWcxcjhlo3eW055fdIAHyCCuZnU3L+0tBSK15NgE1v+BxK9fZ+7E/b2dz/O31yFufpctdH5N67ZN01xuhqHwsVmnY4MayK1cmiYmArgwqF4ZIYZM8an4HHP/W9GPk5Cw62/1868 X-Gm-Message-State: AOJu0Yz9540Wk72IhCjGTUF646bWoliuNpXVyr8yRVxqVeH83oayo19+ TJXbiDGNO+Ss4U7EloelRFovPZP3ycZXzsfS4UmmnR0qDoHOHl4G X-Google-Smtp-Source: AGHT+IGJknlOWDT5IDGO4g0Vg0FdOXT2CEmuZn3OgVH7ZbPZIqsVk5ocA1HPsRJkRC1UmsGaqx80tw== X-Received: by 2002:a05:6512:3709:b0:51b:812:3c82 with SMTP id z9-20020a056512370900b0051b08123c82mr7417480lfr.5.1715085915239; Tue, 07 May 2024 05:45:15 -0700 (PDT) Received: from localhost ([45.130.85.2]) by smtp.gmail.com with ESMTPSA id j17-20020adfff91000000b00349ac818326sm12914844wrr.43.2024.05.07.05.45.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 May 2024 05:45:14 -0700 (PDT) From: Leone Fernando To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemb@google.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Leone Fernando Subject: [PATCH net-next v2 4/4] net: route: replace route hints with input_dst_cache Date: Tue, 7 May 2024 14:42:29 +0200 Message-Id: <20240507124229.446802-5-leone4fernando@gmail.com> In-Reply-To: <20240507124229.446802-1-leone4fernando@gmail.com> References: <20240507124229.446802-1-leone4fernando@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org Replace route hints with cached dsts - ip_rcv_finish_core will first try to use the cache and only then fall back to the demux or perform a full lookup. Only add newly found dsts to the cache after all the checks have passed successfully to avoid adding a dropped packet's dst to the cache. Multicast dsts are not added to the dst_cache as it will require additional checks and multicast packets are rarer and a slower path anyway. A check was added to ip_route_use_dst_cache that prevents forwarding packets received by devices for which forwarding is disabled. Relevant checks were added to ip_route_use_dst_cache to make sure the dst can be used and to ensure IPCB(skb) flags are correct. Signed-off-by: Leone Fernando --- include/net/route.h | 6 ++-- net/ipv4/ip_input.c | 58 +++++++++++++++++++----------------- net/ipv4/route.c | 72 +++++++++++++++++++++++++++++++++------------ 3 files changed, 88 insertions(+), 48 deletions(-) diff --git a/include/net/route.h b/include/net/route.h index 93833cfe9c96..c9433b8b9417 100644 --- a/include/net/route.h +++ b/include/net/route.h @@ -202,9 +202,9 @@ int ip_mc_validate_source(struct sk_buff *skb, __be32 daddr, __be32 saddr, struct in_device *in_dev, u32 *itag); int ip_route_input_noref(struct sk_buff *skb, __be32 dst, __be32 src, u8 tos, struct net_device *devin); -int ip_route_use_hint(struct sk_buff *skb, __be32 dst, __be32 src, - u8 tos, struct net_device *devin, - const struct sk_buff *hint); +int ip_route_use_dst_cache(struct sk_buff *skb, __be32 daddr, __be32 saddr, + u8 tos, struct net_device *dev, + struct dst_entry *dst); static inline int ip_route_input(struct sk_buff *skb, __be32 dst, __be32 src, u8 tos, struct net_device *devin) diff --git a/net/ipv4/ip_input.c b/net/ipv4/ip_input.c index d6fbcbd2358a..35c8b122d62f 100644 --- a/net/ipv4/ip_input.c +++ b/net/ipv4/ip_input.c @@ -305,30 +305,44 @@ static inline bool ip_rcv_options(struct sk_buff *skb, struct net_device *dev) return true; } -static bool ip_can_use_hint(const struct sk_buff *skb, const struct iphdr *iph, - const struct sk_buff *hint) +static bool ip_can_add_dst_cache(struct sk_buff *skb, __u16 rt_type) { - return hint && !skb_dst(skb) && ip_hdr(hint)->daddr == iph->daddr && - ip_hdr(hint)->tos == iph->tos; + return skb_valid_dst(skb) && + rt_type != RTN_BROADCAST && + rt_type != RTN_MULTICAST && + !(IPCB(skb)->flags & IPSKB_MULTIPATH); +} + +static bool ip_can_use_dst_cache(const struct net *net, struct sk_buff *skb) +{ + return !skb_dst(skb) && !fib4_has_custom_rules(net); } int tcp_v4_early_demux(struct sk_buff *skb); int udp_v4_early_demux(struct sk_buff *skb); static int ip_rcv_finish_core(struct net *net, struct sock *sk, - struct sk_buff *skb, struct net_device *dev, - const struct sk_buff *hint) + struct sk_buff *skb, struct net_device *dev) { + struct dst_cache *dst_cache = net_generic(net, dst_cache_net_id); const struct iphdr *iph = ip_hdr(skb); + struct dst_entry *dst; int err, drop_reason; struct rtable *rt; + bool do_cache; drop_reason = SKB_DROP_REASON_NOT_SPECIFIED; - if (ip_can_use_hint(skb, iph, hint)) { - err = ip_route_use_hint(skb, iph->daddr, iph->saddr, iph->tos, - dev, hint); - if (unlikely(err)) - goto drop_error; + do_cache = ip_can_use_dst_cache(net, skb); + if (do_cache) { + dst = dst_cache_input_get_noref(dst_cache, skb); + if (dst) { + err = ip_route_use_dst_cache(skb, iph->daddr, + iph->saddr, iph->tos, + dev, dst); + if (unlikely(err)) + goto drop_error; + do_cache = false; + } } if (READ_ONCE(net->ipv4.sysctl_ip_early_demux) && @@ -418,6 +432,9 @@ static int ip_rcv_finish_core(struct net *net, struct sock *sk, } } + if (do_cache && ip_can_add_dst_cache(skb, rt->rt_type)) + dst_cache_input_add(dst_cache, skb); + return NET_RX_SUCCESS; drop: @@ -444,7 +461,7 @@ static int ip_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb) if (!skb) return NET_RX_SUCCESS; - ret = ip_rcv_finish_core(net, sk, skb, dev, NULL); + ret = ip_rcv_finish_core(net, sk, skb, dev); if (ret != NET_RX_DROP) ret = dst_input(skb); return ret; @@ -581,21 +598,11 @@ static void ip_sublist_rcv_finish(struct list_head *head) } } -static struct sk_buff *ip_extract_route_hint(const struct net *net, - struct sk_buff *skb, int rt_type) -{ - if (fib4_has_custom_rules(net) || rt_type == RTN_BROADCAST || - IPCB(skb)->flags & IPSKB_MULTIPATH) - return NULL; - - return skb; -} - static void ip_list_rcv_finish(struct net *net, struct sock *sk, struct list_head *head) { - struct sk_buff *skb, *next, *hint = NULL; struct dst_entry *curr_dst = NULL; + struct sk_buff *skb, *next; struct list_head sublist; INIT_LIST_HEAD(&sublist); @@ -610,14 +617,11 @@ static void ip_list_rcv_finish(struct net *net, struct sock *sk, skb = l3mdev_ip_rcv(skb); if (!skb) continue; - if (ip_rcv_finish_core(net, sk, skb, dev, hint) == NET_RX_DROP) + if (ip_rcv_finish_core(net, sk, skb, dev) == NET_RX_DROP) continue; dst = skb_dst(skb); if (curr_dst != dst) { - hint = ip_extract_route_hint(net, skb, - dst_rtable(dst)->rt_type); - /* dispatch old sublist */ if (!list_empty(&sublist)) ip_sublist_rcv_finish(&sublist); diff --git a/net/ipv4/route.c b/net/ipv4/route.c index 4e6b7a67f177..6d88d1f4969b 100644 --- a/net/ipv4/route.c +++ b/net/ipv4/route.c @@ -1771,6 +1771,24 @@ static void ip_handle_martian_source(struct net_device *dev, #endif } +static void ip_route_set_doredirect(struct in_device *in_dev, + struct in_device *out_dev, + struct sk_buff *skb, + u8 gw_family, + __be32 gw4, + __be32 saddr) +{ + if (out_dev == in_dev && IN_DEV_TX_REDIRECTS(out_dev) && + skb->protocol == htons(ETH_P_IP)) { + __be32 gw; + + gw = gw_family == AF_INET ? gw4 : 0; + if (IN_DEV_SHARED_MEDIA(out_dev) || + inet_addr_onlink(out_dev, saddr, gw)) + IPCB(skb)->flags |= IPSKB_DOREDIRECT; + } +} + /* called in rcu_read_lock() section */ static int __mkroute_input(struct sk_buff *skb, const struct fib_result *res, @@ -1803,15 +1821,10 @@ static int __mkroute_input(struct sk_buff *skb, } do_cache = res->fi && !itag; - if (out_dev == in_dev && err && IN_DEV_TX_REDIRECTS(out_dev) && - skb->protocol == htons(ETH_P_IP)) { - __be32 gw; - - gw = nhc->nhc_gw_family == AF_INET ? nhc->nhc_gw.ipv4 : 0; - if (IN_DEV_SHARED_MEDIA(out_dev) || - inet_addr_onlink(out_dev, saddr, gw)) - IPCB(skb)->flags |= IPSKB_DOREDIRECT; - } + if (err) + ip_route_set_doredirect(in_dev, out_dev, skb, + nhc->nhc_gw_family, + nhc->nhc_gw.ipv4, saddr); if (skb->protocol != htons(ETH_P_IP)) { /* Not IP (i.e. ARP). Do not create route, if it is @@ -2141,14 +2154,15 @@ static int ip_mkroute_input(struct sk_buff *skb, /* Implements all the saddr-related checks as ip_route_input_slow(), * assuming daddr is valid and the destination is not a local broadcast one. - * Uses the provided hint instead of performing a route lookup. + * Uses the provided dst from dst_cache instead of performing a route lookup. */ -int ip_route_use_hint(struct sk_buff *skb, __be32 daddr, __be32 saddr, - u8 tos, struct net_device *dev, - const struct sk_buff *hint) +int ip_route_use_dst_cache(struct sk_buff *skb, __be32 daddr, __be32 saddr, + u8 tos, struct net_device *dev, + struct dst_entry *dst) { + struct in_device *out_dev = __in_dev_get_rcu(dst->dev); struct in_device *in_dev = __in_dev_get_rcu(dev); - struct rtable *rt = skb_rtable(hint); + struct rtable *rt = (struct rtable *)dst; struct net *net = dev_net(dev); int err = -EINVAL; u32 tag = 0; @@ -2165,21 +2179,43 @@ int ip_route_use_hint(struct sk_buff *skb, __be32 daddr, __be32 saddr, if (ipv4_is_loopback(saddr) && !IN_DEV_NET_ROUTE_LOCALNET(in_dev, net)) goto martian_source; - if (rt->rt_type != RTN_LOCAL) - goto skip_validate_source; + if (ipv4_is_loopback(daddr) && !IN_DEV_NET_ROUTE_LOCALNET(in_dev, net)) + goto martian_destination; + if (rt->rt_type != RTN_LOCAL) { + if (!IN_DEV_FORWARD(in_dev)) { + err = -EHOSTUNREACH; + goto out_err; + } + goto skip_validate_source; + } tos &= IPTOS_RT_MASK; err = fib_validate_source(skb, saddr, daddr, tos, 0, dev, in_dev, &tag); if (err < 0) goto martian_source; + if (err) + ip_route_set_doredirect(in_dev, out_dev, skb, rt->rt_gw_family, + rt->rt_gw4, saddr); + skip_validate_source: - skb_dst_copy(skb, hint); + skb_dst_set_noref(skb, dst); return 0; martian_source: ip_handle_martian_source(dev, in_dev, skb, daddr, saddr); +out_err: return err; + +martian_destination: + RT_CACHE_STAT_INC(in_martian_dst); +#ifdef CONFIG_IP_ROUTE_VERBOSE + if (IN_DEV_LOG_MARTIANS(in_dev)) + net_warn_ratelimited("martian destination %pI4 from %pI4, dev %s\n", + &daddr, &saddr, dev->name); +#endif + err = -EINVAL; + goto out_err; } /* get device for dst_alloc with local routes */ @@ -2200,7 +2236,7 @@ static struct net_device *ip_rt_get_dev(struct net *net, * addresses, because every properly looped back packet * must have correct destination already attached by output routine. * Changes in the enforced policies must be applied also to - * ip_route_use_hint(). + * ip_route_use_dst_cache(). * * Such approach solves two big problems: * 1. Not simplex devices are handled properly.