From patchwork Fri Jul 29 10:35:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Mikhalitsyn X-Patchwork-Id: 12932305 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 013D2C3F6B0 for ; Fri, 29 Jul 2022 10:37:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232892AbiG2KhT (ORCPT ); Fri, 29 Jul 2022 06:37:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234550AbiG2KhR (ORCPT ); Fri, 29 Jul 2022 06:37:17 -0400 Received: from relay.virtuozzo.com (relay.virtuozzo.com [130.117.225.111]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10E5080F67; Fri, 29 Jul 2022 03:37:16 -0700 (PDT) Received: from dev010.ch-qa.sw.ru ([172.29.1.15]) by relay.virtuozzo.com with esmtp (Exim 4.95) (envelope-from ) id 1oHNL9-00Cf1E-If; Fri, 29 Jul 2022 12:35:55 +0200 From: Alexander Mikhalitsyn To: netdev@vger.kernel.org Cc: "Denis V. Lunev" , Alexander Mikhalitsyn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Daniel Borkmann , David Ahern , Yajun Deng , Roopa Prabhu , linux-kernel@vger.kernel.org, Alexey Kuznetsov , Konstantin Khorenko , kernel@openvz.org Subject: [PATCH 1/2] neigh: fix possible DoS due to net iface start/stop loop Date: Fri, 29 Jul 2022 13:35:58 +0300 Message-Id: <20220729103559.215140-2-alexander.mikhalitsyn@virtuozzo.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220729103559.215140-1-alexander.mikhalitsyn@virtuozzo.com> References: <20220729103559.215140-1-alexander.mikhalitsyn@virtuozzo.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: "Denis V. Lunev" Normal processing of ARP request (usually this is Ethernet broadcast packet) coming to the host is looking like the following: * the packet comes to arp_process() call and is passed through routing procedure * the request is put into the queue using pneigh_enqueue() if corresponding ARP record is not local (common case for container records on the host) * the request is processed by timer (within 80 jiffies by default) and ARP reply is sent from the same arp_process() using NEIGH_CB(skb)->flags & LOCALLY_ENQUEUED condition (flag is set inside pneigh_enqueue()) And here the problem comes. Linux kernel calls pneigh_queue_purge() which destroys the whole queue of ARP requests on ANY network interface start/stop event through __neigh_ifdown(). This is actually not a problem within the original world as network interface start/stop was accessible to the host 'root' only, which could do more destructive things. But the world is changed and there are Linux containers available. Here container 'root' has an access to this API and could be considered as untrusted user in the hosting (container's) world. Thus there is an attack vector to other containers on node when container's root will endlessly start/stop interfaces. We have observed similar situation on a real production node when docker container was doing such activity and thus other containers on the node become not accessible. The patch proposed doing very simple thing. It drops only packets from the same namespace in the pneigh_queue_purge() where network interface state change is detected. This is enough to prevent the problem for the whole node preserving original semantics of the code. Investigated-by: Alexander Mikhalitsyn Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Daniel Borkmann Cc: David Ahern Cc: Yajun Deng Cc: Roopa Prabhu Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: Alexey Kuznetsov Cc: Alexander Mikhalitsyn Cc: Konstantin Khorenko Cc: kernel@openvz.org Signed-off-by: Denis V. Lunev --- net/core/neighbour.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 54625287ee5b..213ec0be800b 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -307,14 +307,24 @@ static int neigh_del_timer(struct neighbour *n) return 0; } -static void pneigh_queue_purge(struct sk_buff_head *list) +static void pneigh_queue_purge(struct sk_buff_head *list, struct net *net) { + unsigned long flags; struct sk_buff *skb; - while ((skb = skb_dequeue(list)) != NULL) { - dev_put(skb->dev); - kfree_skb(skb); + spin_lock_irqsave(&list->lock, flags); + skb = skb_peek(list); + while (skb) { + struct sk_buff *skb_next = skb_peek_next(skb, list); + + if (!net || net_eq(dev_net(skb->dev), net)) { + __skb_unlink(skb, list); + dev_put(skb->dev); + kfree_skb(skb); + } + skb = skb_next; } + spin_unlock_irqrestore(&list->lock, flags); } static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, @@ -386,8 +396,7 @@ static int __neigh_ifdown(struct neigh_table *tbl, struct net_device *dev, neigh_flush_dev(tbl, dev, skip_perm); pneigh_ifdown_and_unlock(tbl, dev); - del_timer_sync(&tbl->proxy_timer); - pneigh_queue_purge(&tbl->proxy_queue); + pneigh_queue_purge(&tbl->proxy_queue, dev_net(dev)); return 0; } @@ -1787,7 +1796,7 @@ int neigh_table_clear(int index, struct neigh_table *tbl) cancel_delayed_work_sync(&tbl->managed_work); cancel_delayed_work_sync(&tbl->gc_work); del_timer_sync(&tbl->proxy_timer); - pneigh_queue_purge(&tbl->proxy_queue); + pneigh_queue_purge(&tbl->proxy_queue, NULL); neigh_ifdown(tbl, NULL); if (atomic_read(&tbl->entries)) pr_crit("neighbour leakage\n"); From patchwork Fri Jul 29 10:35:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Mikhalitsyn X-Patchwork-Id: 12932304 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AEC1C00144 for ; Fri, 29 Jul 2022 10:37:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234698AbiG2KhS (ORCPT ); Fri, 29 Jul 2022 06:37:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232892AbiG2KhR (ORCPT ); Fri, 29 Jul 2022 06:37:17 -0400 Received: from relay.virtuozzo.com (relay.virtuozzo.com [130.117.225.111]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B8FF13F05; Fri, 29 Jul 2022 03:37:16 -0700 (PDT) Received: from dev010.ch-qa.sw.ru ([172.29.1.15]) by relay.virtuozzo.com with esmtp (Exim 4.95) (envelope-from ) id 1oHNLB-00Cf1E-IH; Fri, 29 Jul 2022 12:35:57 +0200 From: Alexander Mikhalitsyn To: netdev@vger.kernel.org Cc: Alexander Mikhalitsyn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Daniel Borkmann , David Ahern , Yajun Deng , Roopa Prabhu , linux-kernel@vger.kernel.org, Alexey Kuznetsov , Konstantin Khorenko , kernel@openvz.org, "Denis V . Lunev" Subject: [PATCH 2/2] neighbour: make proxy_queue.qlen limit per-device Date: Fri, 29 Jul 2022 13:35:59 +0300 Message-Id: <20220729103559.215140-3-alexander.mikhalitsyn@virtuozzo.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20220729103559.215140-1-alexander.mikhalitsyn@virtuozzo.com> References: <20220729103559.215140-1-alexander.mikhalitsyn@virtuozzo.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Right now we have a neigh_param PROXY_QLEN which specifies maximum length of neigh_table->proxy_queue. But in fact, this limitation doesn't work well because check condition looks like: tbl->proxy_queue.qlen > NEIGH_VAR(p, PROXY_QLEN) The problem is that p (struct neigh_parms) is a per-device thing, but tbl (struct neigh_table) is a system-wide global thing. It seems reasonable to make proxy_queue limit per-device based. Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Daniel Borkmann Cc: David Ahern Cc: Yajun Deng Cc: Roopa Prabhu Cc: netdev@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: Alexey Kuznetsov Cc: Alexander Mikhalitsyn Cc: Konstantin Khorenko Cc: kernel@openvz.org Suggested-by: Denis V. Lunev Signed-off-by: Alexander Mikhalitsyn Reviewed-by: Denis V. Lunev --- include/net/neighbour.h | 1 + net/core/neighbour.c | 25 ++++++++++++++++++++++--- 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 87419f7f5421..bc3fbec70d10 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -82,6 +82,7 @@ struct neigh_parms { struct rcu_head rcu_head; int reachable_time; + int qlen; int data[NEIGH_VAR_DATA_MAX]; DECLARE_BITMAP(data_state, NEIGH_VAR_DATA_MAX); }; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 213ec0be800b..d98923e3ff02 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -316,9 +316,19 @@ static void pneigh_queue_purge(struct sk_buff_head *list, struct net *net) skb = skb_peek(list); while (skb != NULL) { struct sk_buff *skb_next = skb_peek_next(skb, list); - if (net == NULL || net_eq(dev_net(skb->dev), net)) { + struct net_device *dev = skb->dev; + + if (!net || net_eq(dev_net(dev), net)) { + struct in_device *in_dev; + + rcu_read_lock(); + in_dev = __in_dev_get_rcu(dev); + if (in_dev) + in_dev->arp_parms->qlen--; + rcu_read_unlock(); __skb_unlink(skb, list); - dev_put(skb->dev); + + dev_put(dev); kfree_skb(skb); } skb = skb_next; @@ -1605,8 +1615,15 @@ static void neigh_proxy_process(struct timer_list *t) if (tdif <= 0) { struct net_device *dev = skb->dev; + struct in_device *in_dev; + rcu_read_lock(); + in_dev = __in_dev_get_rcu(dev); + if (in_dev) + in_dev->arp_parms->qlen--; + rcu_read_unlock(); __skb_unlink(skb, &tbl->proxy_queue); + if (tbl->proxy_redo && netif_running(dev)) { rcu_read_lock(); tbl->proxy_redo(skb); @@ -1631,7 +1648,7 @@ void pneigh_enqueue(struct neigh_table *tbl, struct neigh_parms *p, unsigned long sched_next = jiffies + prandom_u32_max(NEIGH_VAR(p, PROXY_DELAY)); - if (tbl->proxy_queue.qlen > NEIGH_VAR(p, PROXY_QLEN)) { + if (p->qlen > NEIGH_VAR(p, PROXY_QLEN)) { kfree_skb(skb); return; } @@ -1647,6 +1664,7 @@ void pneigh_enqueue(struct neigh_table *tbl, struct neigh_parms *p, skb_dst_drop(skb); dev_hold(skb->dev); __skb_queue_tail(&tbl->proxy_queue, skb); + p->qlen++; mod_timer(&tbl->proxy_timer, sched_next); spin_unlock(&tbl->proxy_queue.lock); } @@ -1679,6 +1697,7 @@ struct neigh_parms *neigh_parms_alloc(struct net_device *dev, refcount_set(&p->refcnt, 1); p->reachable_time = neigh_rand_reach_time(NEIGH_VAR(p, BASE_REACHABLE_TIME)); + p->qlen = 0; dev_hold_track(dev, &p->dev_tracker, GFP_KERNEL); p->dev = dev; write_pnet(&p->net, net); @@ -1744,6 +1763,7 @@ void neigh_table_init(int index, struct neigh_table *tbl) refcount_set(&tbl->parms.refcnt, 1); tbl->parms.reachable_time = neigh_rand_reach_time(NEIGH_VAR(&tbl->parms, BASE_REACHABLE_TIME)); + tbl->parms.qlen = 0; tbl->stats = alloc_percpu(struct neigh_statistics); if (!tbl->stats)