From patchwork Tue Jan 14 20:55:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13939525 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-qk1-f201.google.com (mail-qk1-f201.google.com [209.85.222.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E358A19149F for ; Tue, 14 Jan 2025 20:55:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888137; cv=none; b=c/94SvFuj2j/dosBigEWZvVs5A6Hc4PX+SaPydMO0CoQXAeQmVT1JQW1PCpzC/rmkezErVUfFDgz4f66TljOqv6+Yw7JWdebEhHgYf9N+oVfKbnNvr9JU0YUhU0ud8UXME6JgwJvcgr1fTHIvWGhY6dR8GYnFABmCfLvcLs0068= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888137; c=relaxed/simple; bh=+jiNcGCIk7HzAY4joxHdvs51wDmgqi5k4h8rVtt9YN0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=a7ndS0nyQXILgQRfrUC4jc2hy2AODmgP8I8LBxvSDAtvxpzgUdS0EKFBcx88Rztha75VpXt0LeUrVJWqrPQl9M3Q3uXDUWJv8bcFH2K19UemWSCu2RwbissMk/PmJVfcdyTfcXo90iDPvFEh+o9ija6FKb/Yd8OLTdI1pfTVE5g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=X2mf+yTA; arc=none smtp.client-ip=209.85.222.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="X2mf+yTA" Received: by mail-qk1-f201.google.com with SMTP id af79cd13be357-7b6f482362aso948429585a.1 for ; Tue, 14 Jan 2025 12:55:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736888135; x=1737492935; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Wk+fZj4sfaYNKt3et6GAqarEN+Xwd4BiVEI/0rxxZqk=; b=X2mf+yTA49CgXr+iNqoZ879DcyhRPJ+Cs01AmswVEnHk0IDQKEjItNQUp6RqbeGw5d +ruuDRpw42ABlpIl3360QcAL3m75OU7RkO2ulMPwId+knUYP+UCXAE8zUOqC1y76Pixh QZXwONe22nG7c4Dt49sPRgHbZNLvkf9nGZR2oGUwSZjL2FtPBOvlbGltrcEOt2tOMKk4 UaUKPZR15dvVrXbXRrTL5K1jj6MgGxMmyv+vB2GsDve/kgCFp2e9NpY7AR0Ul9SACvKj oHArlA1mOsqM0wAfTQ99AeoadJPgE1VfPdWFT8MSkZXqX6Fv8rf7F8kAW0dxR7tw4Ngm 3BDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736888135; x=1737492935; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wk+fZj4sfaYNKt3et6GAqarEN+Xwd4BiVEI/0rxxZqk=; b=RyxNhBO7jhRINF9YcIdHqIycagxlXxxqpNFP7R/pM9f8EeswMfWiUL+m+sUkGxaVxt X7CVs4CiubgL2ndHjE8s9XPWLspu7DDhkKd5zGC7c1Ffc0kEU81jSQhvvA+2XedJrWp7 6So0NrGNTeFvpEJ35cEF68SrK2vjdgKNjBvL6LOzyvG/6of4g72tiVjPN1u1b1sDN07P cndfIZ9T2uyIotj+rgQYbNxPW41jSYhtLncy85gvl8IghpICLDoEUhDkP18OIHaRlfGQ lAVLAq16c11hUkqkVSHzQykaKVbvd1XUSu+1V3+v6Ahq9lkalyKm8QKRFrcObmXvuZgQ qd+g== X-Gm-Message-State: AOJu0Yx4wwBc4s6tK0M6Z82XiXS8lTRRJnq326Dlm2pS+NsGEr/4jDdP /aNKyX8wTzuYVFy0BJ2Cz0NJ9MquSlhS34L34y4fZAcKZ6wzhGrxQbZNMy/2xyiLChv1zUOltzo U9+27LNjjIw== X-Google-Smtp-Source: AGHT+IGnVqbdwJggzt2aWlemeuMHD8lrwBER1fTkLyfxxL3TVzFB3zQlhRpUVDOt/GLiF6Qn0by+zO9kHAdwxA== X-Received: from qkmy29.prod.google.com ([2002:a05:620a:e1d:b0:7bc:dee1:94a3]) (user=edumazet job=prod-delivery.src-stubby-dispatcher) by 2002:a05:620a:1b8f:b0:7b6:6ffc:e972 with SMTP id af79cd13be357-7bcd96fa261mr4460129185a.5.1736888134879; Tue, 14 Jan 2025 12:55:34 -0800 (PST) Date: Tue, 14 Jan 2025 20:55:27 +0000 In-Reply-To: <20250114205531.967841-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250114205531.967841-1-edumazet@google.com> X-Mailer: git-send-email 2.48.0.rc2.279.g1de40edade-goog Message-ID: <20250114205531.967841-2-edumazet@google.com> Subject: [PATCH v3 net-next 1/5] net: expedite synchronize_net() for cleanup_net() From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, Simon Horman , eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org cleanup_net() is the single thread responsible for netns dismantles, and a serious bottleneck. Before we can get per-netns RTNL, make sure all synchronize_net() called from this thread are using rcu_synchronize_expedited(). v3: deal with CONFIG_NET_NS=n Signed-off-by: Eric Dumazet Reviewed-by: Jesse Brandeburg --- include/net/net_namespace.h | 2 ++ net/core/dev.c | 11 ++++++++++- net/core/net_namespace.c | 5 +++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h index 5a2a0df8ad91b677b515b392869c6c755be5c868..0f5eb9db0c6264efc1ac83ab577511fd6823f4fe 100644 --- a/include/net/net_namespace.h +++ b/include/net/net_namespace.h @@ -210,6 +210,8 @@ void net_ns_barrier(void); struct ns_common *get_net_ns(struct ns_common *ns); struct net *get_net_ns_by_fd(int fd); +extern struct task_struct *cleanup_net_task; + #else /* CONFIG_NET_NS */ #include #include diff --git a/net/core/dev.c b/net/core/dev.c index fda4e1039bf01d46cfaa5f134d20e1d2bcdcfdfc..0542346a403c2602f94d8bc61f7be0ea0c64c33a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10072,6 +10072,15 @@ static void dev_index_release(struct net *net, int ifindex) WARN_ON(xa_erase(&net->dev_by_index, ifindex)); } +static bool from_cleanup_net(void) +{ +#ifdef CONFIG_NET_NS + return current == cleanup_net_task; +#else + return false; +#endif +} + /* Delayed registration/unregisteration */ LIST_HEAD(net_todo_list); DECLARE_WAIT_QUEUE_HEAD(netdev_unregistering_wq); @@ -11447,7 +11456,7 @@ EXPORT_SYMBOL_GPL(alloc_netdev_dummy); void synchronize_net(void) { might_sleep(); - if (rtnl_is_locked()) + if (from_cleanup_net() || rtnl_is_locked()) synchronize_rcu_expedited(); else synchronize_rcu(); diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c index b5cd3ae4f04cf28d43f8401a3dafebac4a297123..cb39a12b2f8295c605f08b5589932932150a1644 100644 --- a/net/core/net_namespace.c +++ b/net/core/net_namespace.c @@ -588,6 +588,8 @@ static void unhash_nsid(struct net *net, struct net *last) static LLIST_HEAD(cleanup_list); +struct task_struct *cleanup_net_task; + static void cleanup_net(struct work_struct *work) { const struct pernet_operations *ops; @@ -596,6 +598,8 @@ static void cleanup_net(struct work_struct *work) LIST_HEAD(net_exit_list); LIST_HEAD(dev_kill_list); + cleanup_net_task = current; + /* Atomically snapshot the list of namespaces to cleanup */ net_kill_list = llist_del_all(&cleanup_list); @@ -670,6 +674,7 @@ static void cleanup_net(struct work_struct *work) put_user_ns(net->user_ns); net_free(net); } + cleanup_net_task = NULL; } /** From patchwork Tue Jan 14 20:55:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13939526 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-qt1-f202.google.com (mail-qt1-f202.google.com [209.85.160.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8154620C47F for ; Tue, 14 Jan 2025 20:55:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888139; cv=none; b=q4m38cxcwTJm/v0tyVwchijp4k9FSDhw85MupxcTMXOXlFiHYT5+LamT5k16j59bkyqKF628XmXe7/vlZVSZI3Zn/lwHu+bTwWWvzWs8MvldgFNTJtdH987trXlGiozKYlxeQKm5+C1gABdOz2PiUIj6FBJMputr3oUkSoMut+8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888139; c=relaxed/simple; bh=kE6Ol122z9shZe0ALiEEiiN0Fxw+ppJCaKhQ7pRqGws=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ert+/GC3UHYr3LYwtpvsPOXDmhyedULK4E7P0qGElPEdfQDu3Ehzgdv+moElCMLQlB4geNtLQ5TaHtFSWhBBp/TOim9YfqQjSSMhEgim/y+B7NbDijo/0SZJoORDDmklnaAmSxAvD2Ndwy6+asE6DiIsPldx2uAyRDBMYPffSeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jTWq1smH; arc=none smtp.client-ip=209.85.160.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jTWq1smH" Received: by mail-qt1-f202.google.com with SMTP id d75a77b69052e-46790c5b1a5so148995641cf.2 for ; Tue, 14 Jan 2025 12:55:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736888136; x=1737492936; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=umZ8c1cXwPdHXyHK5U4mqkUbZzV6gnO5gDr3Vw2yphM=; b=jTWq1smH9A8+EY+42EagmYDOpf6LkuwiBoUyNMqFmAC96J0TSfBbmmykcpNjqWHUIT tMC7WnzUuSVFLq1pNqS9odADnVCOJLrjEpI20T6WhyQSzF6IiugBoPoZoMSMKtw16Ut6 BeGY/E8zxs/ezzg0TlJZDbJUeLyZLQYYSVCp24/e2Q7gF1CZxvnHQTt3PEl69mLOeDTO tZXHpwjX1IyGUYZKsOeSL15UafivQIEFpwL+GZKDRaXOrultwBtIP2QetkgFpuGM6LlT HQLsc7Te/9ouDgaXyRQ80eEYt3lX6Vqi2VLW5Eh5tela8tRdFWnqQPA5eCU9KqaU5qvd TWFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736888136; x=1737492936; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=umZ8c1cXwPdHXyHK5U4mqkUbZzV6gnO5gDr3Vw2yphM=; b=fEWcIRk/S24N/gRW8XfNVyxvKG/4XOEjqv1KMWnEy/3vj1mVikxf/j6kp+BrOnOpVM xuVOrzzrwGZYuq2u5J3LH59ZX/jp8M3TzG0/sHOU86RzsCYdwrJlkIQF9bA8ETL8UCB+ vetGvVyW/6xvlAkY9lHgF/rWQ9eox+y7RosLXCImEiAJrbvq0dcuv3NrchurrPza4j3K K+e+5z04D4YTVleR7xvsm+kgf6B5APz+8nftUQhdxpYjspSmJBOinkaDwK6yfiiF95Cr YXAiRMk/1hg/5znlkXmmqCnJjR1PhHuFqD+UMiVU6CTAMoW8mz4uZH9AdpTT1XIjuBTN uF4g== X-Gm-Message-State: AOJu0YwpwL6hCabq+F3XfCehlkk/PdH897D6R2or1jZvJ3TSJ0tiHszc qUzJQAAs9iSV1pV2ealWeGlm1+1RASe8rNMEeE2kXyNO1M0TSWu0XSu42mvqUTNYyf2iSpIYFLa mPjUHR+WtUQ== X-Google-Smtp-Source: AGHT+IHZb8FjDrSKYsKTBOOsJGQ/sz3/O9gAwZTKJIFYgaapyiksM3yX466gULZHwF1AewuZiJCajY5B+u0Hew== X-Received: from qtbbq15.prod.google.com ([2002:a05:622a:1c0f:b0:46c:9f17:12ee]) (user=edumazet job=prod-delivery.src-stubby-dispatcher) by 2002:ac8:58c4:0:b0:467:87f4:a29f with SMTP id d75a77b69052e-46c7108f7acmr412157281cf.45.1736888136441; Tue, 14 Jan 2025 12:55:36 -0800 (PST) Date: Tue, 14 Jan 2025 20:55:28 +0000 In-Reply-To: <20250114205531.967841-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250114205531.967841-1-edumazet@google.com> X-Mailer: git-send-email 2.48.0.rc2.279.g1de40edade-goog Message-ID: <20250114205531.967841-3-edumazet@google.com> Subject: [PATCH v3 net-next 2/5] net: no longer assume RTNL is held in flush_all_backlogs() From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, Simon Horman , eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org flush_all_backlogs() uses per-cpu and static data to hold its temporary data, on the assumption it is called under RTNL protection. Following patch in the series will break this assumption. Use instead a dynamically allocated piece of memory. In the unlikely case the allocation fails, use a boot-time allocated memory. Signed-off-by: Eric Dumazet Reviewed-by: Jesse Brandeburg --- net/core/dev.c | 53 +++++++++++++++++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 0542346a403c2602f94d8bc61f7be0ea0c64c33a..b0e05e44d771bee2721d054ddbd03166cc676680 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -6012,8 +6012,6 @@ void netif_receive_skb_list(struct list_head *head) } EXPORT_SYMBOL(netif_receive_skb_list); -static DEFINE_PER_CPU(struct work_struct, flush_works); - /* Network device is going away, flush any packets still pending */ static void flush_backlog(struct work_struct *work) { @@ -6070,36 +6068,54 @@ static bool flush_required(int cpu) return true; } +struct flush_backlogs { + cpumask_t flush_cpus; + struct work_struct w[]; +}; + +static struct flush_backlogs *flush_backlogs_alloc(void) +{ + return kmalloc(struct_size_t(struct flush_backlogs, w, nr_cpu_ids), + GFP_KERNEL); +} + +static struct flush_backlogs *flush_backlogs_fallback; +static DEFINE_MUTEX(flush_backlogs_mutex); + static void flush_all_backlogs(void) { - static cpumask_t flush_cpus; + struct flush_backlogs *ptr = flush_backlogs_alloc(); unsigned int cpu; - /* since we are under rtnl lock protection we can use static data - * for the cpumask and avoid allocating on stack the possibly - * large mask - */ - ASSERT_RTNL(); + if (!ptr) { + mutex_lock(&flush_backlogs_mutex); + ptr = flush_backlogs_fallback; + } + cpumask_clear(&ptr->flush_cpus); cpus_read_lock(); - cpumask_clear(&flush_cpus); for_each_online_cpu(cpu) { if (flush_required(cpu)) { - queue_work_on(cpu, system_highpri_wq, - per_cpu_ptr(&flush_works, cpu)); - cpumask_set_cpu(cpu, &flush_cpus); + INIT_WORK(&ptr->w[cpu], flush_backlog); + queue_work_on(cpu, system_highpri_wq, &ptr->w[cpu]); + __cpumask_set_cpu(cpu, &ptr->flush_cpus); } } /* we can have in flight packet[s] on the cpus we are not flushing, * synchronize_net() in unregister_netdevice_many() will take care of - * them + * them. */ - for_each_cpu(cpu, &flush_cpus) - flush_work(per_cpu_ptr(&flush_works, cpu)); + for_each_cpu(cpu, &ptr->flush_cpus) + flush_work(&ptr->w[cpu]); cpus_read_unlock(); + + if (ptr != flush_backlogs_fallback) + kfree(ptr); + else + mutex_unlock(&flush_backlogs_mutex); } static void net_rps_send_ipi(struct softnet_data *remsd) @@ -12286,12 +12302,13 @@ static int __init net_dev_init(void) * Initialise the packet receive queues. */ + flush_backlogs_fallback = flush_backlogs_alloc(); + if (!flush_backlogs_fallback) + goto out; + for_each_possible_cpu(i) { - struct work_struct *flush = per_cpu_ptr(&flush_works, i); struct softnet_data *sd = &per_cpu(softnet_data, i); - INIT_WORK(flush, flush_backlog); - skb_queue_head_init(&sd->input_pkt_queue); skb_queue_head_init(&sd->process_queue); #ifdef CONFIG_XFRM_OFFLOAD From patchwork Tue Jan 14 20:55:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13939527 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-qk1-f202.google.com (mail-qk1-f202.google.com [209.85.222.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B78020D4F7 for ; Tue, 14 Jan 2025 20:55:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888140; cv=none; b=UhKSl19Oi14yo9UZTPz5nsHG+odSq6zt2psjJYWczS7p5FKppUblS7rUum0eD2l/OiWMNBha5noRwACk3mxtwOaIta7w5irgZ0EzhndPy3QOEiM3RV4+N+pthQd4q0evhJIUkBsJ6HOZ1f8p5BirmpdeGA+Ft0NlprP4iIorT1Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888140; c=relaxed/simple; bh=5QT1SwStzSyi+YNhhlTqVLWEfrv2zApNwOucVt1F6Yw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=L0495AgiF+5F9Q1p6RcyZnZ7pRxwYhFu1exqY3ogmrUyLDPVgdJThWdiglwNCUV67xM//zk+vIDtbDtjnaBt8JbGwz8d5NXdGfq3mZ3yQ2UFw0l8+2Pz0IN6SdDxfGryF+VcwCz3WllguImvL5VVFLTjAeWUSg71O2jWOgw1ixY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KshQJXnl; arc=none smtp.client-ip=209.85.222.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KshQJXnl" Received: by mail-qk1-f202.google.com with SMTP id af79cd13be357-7b9e433351dso847687185a.3 for ; Tue, 14 Jan 2025 12:55:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736888137; x=1737492937; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9dKCl1aYvxo7o5A/HIHfWMpJChOiFK8BNQYQtUV/oww=; b=KshQJXnlp5XImBZ/9U1CVsWf1SBZrvAxOwZCjhjtVtTROPylrJOxeeXS77t26ZMFHf plc5I+aLCmJ/Q+p9XP2x48/rltwKsCutsgniwDh84FyXszMbPxVPynE42+2sNHL0IkaN JmrFM8eF2qYzLp21hn/Xr8nj0DQ8TK2aUBLXXpElfm/axhL4IekL74AdqGQpFxsmB0/Z id7uCjoWcMgx48amvaFHpYaOuOIxX/UsSUprK/eKwgt1HS1KXXBqqjv04qg1VFkHG8R5 Xc7e/2fQUxm2olAxb1TGpvg3foSgh1MjaH06U2WbCjYTQ0lMfjJWpDKbJqW/lDw/fylq aLiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736888137; x=1737492937; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9dKCl1aYvxo7o5A/HIHfWMpJChOiFK8BNQYQtUV/oww=; b=VrAJ6MTW5hZSgs7mcObfo9AMuDGNrzRVJEKG+ljFNtvtNt/sTf/gsQN+NWI9ndhAYr /z1Zf/Y0ugbo8P7wse9EDuUmllyi+ojgjtudLDO/wHBJRnQ5FZ473FHbjXHcIthCB7hW 52PLhq3lc/u/5mr1D8q5HMkIaeEnLihxPWWPJyKW+dgn9ppId0BFbzHiuAN3hRWU+vG+ AtnqUbWwetxijhRoVZR9qMmg/ef8WcPz20TlHeo76d41jw+rPYddgZe/1886+e9R8PAt 4BKLBqDP7X1N1Sg1yEDGpi11emY0cqbf9OkEuwVQ+Ry8PsRyAQT1MO0+P7tqKot97pcG +2aA== X-Gm-Message-State: AOJu0Yy3sxuLp1q/GcJEi+/4yZSQGxK4B7lm15esXXZhvKPdoXoVcrW/ z5v0FDYZGq0mvH1e+FHskT7H0puOhDb9NHNDdhlKQrfl9T9m6PfrXewKuWscOXURpxsUwY8AVkB K+CmX63BkpA== X-Google-Smtp-Source: AGHT+IGzXMrVgkYfLQ9kC2i05J6xA0Cc2RmVMuXE4QQT8TkUyprXdr6XKWZiP+iiXxEa/mD/p7LjWD/JLtV52A== X-Received: from qknrb5.prod.google.com ([2002:a05:620a:8d05:b0:7b6:e209:1c29]) (user=edumazet job=prod-delivery.src-stubby-dispatcher) by 2002:a05:620a:841:b0:7be:3f22:2d99 with SMTP id af79cd13be357-7be3f222e8dmr1076319085a.31.1736888137491; Tue, 14 Jan 2025 12:55:37 -0800 (PST) Date: Tue, 14 Jan 2025 20:55:29 +0000 In-Reply-To: <20250114205531.967841-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250114205531.967841-1-edumazet@google.com> X-Mailer: git-send-email 2.48.0.rc2.279.g1de40edade-goog Message-ID: <20250114205531.967841-4-edumazet@google.com> Subject: [PATCH v3 net-next 3/5] net: no longer hold RTNL while calling flush_all_backlogs() From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, Simon Horman , eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org flush_all_backlogs() is called from unregister_netdevice_many_notify() as part of netdevice dismantles. This is currently called under RTNL, and can last up to 50 ms on busy hosts. There is no reason to hold RTNL at this stage, if our caller is cleanup_net() : netns are no more visible, devices are in NETREG_UNREGISTERING state and no other thread could mess our state while RTNL is temporarily released. In order to provide isolation, this patch provides a separate 'net_todo_list' for cleanup_net(). Signed-off-by: Eric Dumazet Reviewed-by: Jesse Brandeburg --- net/core/dev.c | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index b0e05e44d771bee2721d054ddbd03166cc676680..f4dd92bed2223269053b6576e4954fcce218a2e5 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10097,14 +10097,37 @@ static bool from_cleanup_net(void) #endif } +static void rtnl_drop_if_cleanup_net(void) +{ + if (from_cleanup_net()) + __rtnl_unlock(); +} + +static void rtnl_acquire_if_cleanup_net(void) +{ + if (from_cleanup_net()) + rtnl_lock(); +} + /* Delayed registration/unregisteration */ LIST_HEAD(net_todo_list); +static LIST_HEAD(net_todo_list_for_cleanup_net); + +/* TODO: net_todo_list/net_todo_list_for_cleanup_net should probably + * be provided by callers, instead of being static, rtnl protected. + */ +static struct list_head *todo_list(void) +{ + return from_cleanup_net() ? &net_todo_list_for_cleanup_net : + &net_todo_list; +} + DECLARE_WAIT_QUEUE_HEAD(netdev_unregistering_wq); atomic_t dev_unreg_count = ATOMIC_INIT(0); static void net_set_todo(struct net_device *dev) { - list_add_tail(&dev->todo_list, &net_todo_list); + list_add_tail(&dev->todo_list, todo_list()); } static netdev_features_t netdev_sync_upper_features(struct net_device *lower, @@ -10952,7 +10975,7 @@ void netdev_run_todo(void) #endif /* Snapshot list, allow later requests */ - list_replace_init(&net_todo_list, &list); + list_replace_init(todo_list(), &list); __rtnl_unlock(); @@ -11575,8 +11598,10 @@ void unregister_netdevice_many_notify(struct list_head *head, unlist_netdevice(dev); WRITE_ONCE(dev->reg_state, NETREG_UNREGISTERING); } - flush_all_backlogs(); + rtnl_drop_if_cleanup_net(); + flush_all_backlogs(); + rtnl_acquire_if_cleanup_net(); synchronize_net(); list_for_each_entry(dev, head, unreg_list) { From patchwork Tue Jan 14 20:55:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13939528 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-qk1-f202.google.com (mail-qk1-f202.google.com [209.85.222.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA10820E030 for ; Tue, 14 Jan 2025 20:55:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888141; cv=none; b=sUkxNv5xu9y2bUPcWKSzTakiB7J39vd5GjMztwW1Fhc07Vr6+znKe9tMwSCruRYYcC/x0xr3mbOORtO4GHNZsfFTNB6CWI/RdgeANx7Bk+fE2y09sEv1ZveSE9n0uuMgzFC7bUx5Uf4dnhSPWGD94osLmSYIiGHEVcPVFiS9o5M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888141; c=relaxed/simple; bh=rQSLxR8KbIxeQ2MuKcauUf63uJ3Gtr7BoD8Hx1aKMQU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qatN9Pa2cHZ8mTphwZDt8IlH+Mazz6bFElNR4tYcJwFEzCzEIx2Tj1YRaBSgATou6WnmtNSIApVk81ll9uzGPgWC5W/bjZD3WxtmxdPOSc77tHkrqdk45bE7dpm5/c3+FvkgubxZwCQopNXPg1ShuFyj17OtLkJ6yvGOK8tYbzQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EOWISYhV; arc=none smtp.client-ip=209.85.222.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EOWISYhV" Received: by mail-qk1-f202.google.com with SMTP id af79cd13be357-7b9e433351dso847692285a.3 for ; Tue, 14 Jan 2025 12:55:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736888139; x=1737492939; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=EPbS0Ap1+J6rIE2hC9wrsURuYYqX/S/O4Nkt98Y22n0=; b=EOWISYhVHma2GDV91IQ/HZuEXggSn5lQcJNBCso0/Cl3H1lmbkjjW86eUdf0tHqhu7 dsxhMNllUbnxlHJ79zZuCa7Hlx3Qcpx+LFRUYRo2EN6+oh8G9wcRCEYCKEiD4O/PwQwS teX90IqBkzTNoy1ijUAxS39wwlfVDu2FRIOOegLrw2v+WTQirlB61HlWUudXssp5U3Dg cfBUIOl7FU9wCmTfv/7rOlVgaiSroX2K/i2oLB+E6W1s4p6gCg3PX3ARrtKKWDQSKhm8 bBnVOTj9d4PDu3KlzGap657HQUH4AUV74S/bK3rSFxs/EQK99Y0BKWSg755MjqjPMA3m i11w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736888139; x=1737492939; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EPbS0Ap1+J6rIE2hC9wrsURuYYqX/S/O4Nkt98Y22n0=; b=CibDBRIA3J6PwpfBEtsgRA1pTNtnVy0OOeHC9Ty4QQEMhEudBFeAleydtcEitJizYn Y9GfSHZkioti0bs3GfFZdhuSwFnG/LkynormJED/k4Hz6IyzECFYUae1SjK2dsWASbG4 taIG6yqv6zXl3WnwTJI4YFLm6M9DrK8E6MeCpIU6DR8hCaomYHLFyxep6lTb2s5db7qx 6VjN8AkZZfKmEM+jWPAh1He/wLwbNuLjWkck7ZXHoBv13n3Ca3SMvZ52F6NFiybEcrDo RGCInMrCpp4bW1ZYatxmEwK9JyZToIRhncDEoYI9E7lsrTms2Xhm0Y2nbZ0NckW46ZvT NICg== X-Gm-Message-State: AOJu0Yzb42npFR24ejkj/NSY0JtLBhePDt3XfPh410QJInVTNIfvz+S3 R36hK/B+GvvZFhwqpr67Kc9l1GuVMbJXhGHM5xHsvkflEeyStchSqNCml8nzg5yfNrL7ebJwxmd BZlF6gk+1oA== X-Google-Smtp-Source: AGHT+IG93FrNRSnt60s1uDStXVAC9Yeuy3Lhrf0vOkYTx2SfiLBgvGDdhpr1IaJOMs5Q6lQ4DDsGhJBXcP+JWA== X-Received: from qknoo23.prod.google.com ([2002:a05:620a:5317:b0:7b6:c486:8de9]) (user=edumazet job=prod-delivery.src-stubby-dispatcher) by 2002:a05:620a:4556:b0:7a9:aac8:f244 with SMTP id af79cd13be357-7bcd96e56d0mr3747912685a.13.1736888138809; Tue, 14 Jan 2025 12:55:38 -0800 (PST) Date: Tue, 14 Jan 2025 20:55:30 +0000 In-Reply-To: <20250114205531.967841-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250114205531.967841-1-edumazet@google.com> X-Mailer: git-send-email 2.48.0.rc2.279.g1de40edade-goog Message-ID: <20250114205531.967841-5-edumazet@google.com> Subject: [PATCH v3 net-next 4/5] net: reduce RTNL hold duration in unregister_netdevice_many_notify() (part 1) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, Simon Horman , eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org Two synchronize_net() calls are currently done while holding RTNL. This is source of RTNL contention in workloads adding and deleting many network namespaces per second, because synchronize_rcu() and synchronize_rcu_expedited() can use 60+ ms in some cases. For cleanup_net() use, temporarily release RTNL while calling the last synchronize_net(). This should be safe, because devices are no longer visible to other threads at this point. In any case, the new netdev_lock() / netdev_unlock() infrastructure that we are adding should allow to fix potential issues, with a combination of a per-device mutex and dev->reg_state awareness. Signed-off-by: Eric Dumazet Reviewed-by: Jesse Brandeburg --- net/core/dev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/net/core/dev.c b/net/core/dev.c index f4dd92bed2223269053b6576e4954fcce218a2e5..574bd40f3a2bfcc6e43300fad669b1579d48039a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -11602,6 +11602,7 @@ void unregister_netdevice_many_notify(struct list_head *head, rtnl_drop_if_cleanup_net(); flush_all_backlogs(); rtnl_acquire_if_cleanup_net(); + /* TODO: move this before the prior rtnl_acquire_if_cleanup_net() */ synchronize_net(); list_for_each_entry(dev, head, unreg_list) { @@ -11662,7 +11663,9 @@ void unregister_netdevice_many_notify(struct list_head *head, #endif } + rtnl_drop_if_cleanup_net(); synchronize_net(); + rtnl_acquire_if_cleanup_net(); list_for_each_entry(dev, head, unreg_list) { netdev_put(dev, &dev->dev_registered_tracker); From patchwork Tue Jan 14 20:55:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13939529 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-qv1-f74.google.com (mail-qv1-f74.google.com [209.85.219.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E92C20F077 for ; Tue, 14 Jan 2025 20:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888142; cv=none; b=LhWGmy1iYDjNxvF0kYusz35pk1RWyDiwMhrslaJ70GEeOhNg0iWrocNIFptUu7ER/KRjnYaRAZ1Mfld+Vf7lb4MxlT4uITrWkC7L/+LMy+jZo9chf6fJizdu9TzJcSx1GTGbEL8h9LrDYgYziOp3i3rd/kU06R3bLJXRzL92F4c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736888142; c=relaxed/simple; bh=tT/u3IIvDrQGFqxuwLN9DETqDeVfLw9jgy72mii2fk8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=t7axUzyEoIu5w5mYBR+ZqUhrTDIN7CTOgxbx5cOVsRWBSaRO/z5Ejnc/1+rsd8rq5d/W1tThEwwsP8+CtYGnwQbXCevFfUyA145DWPBERUp5w6gaV7KHeVZSi6yJM5DgWR8HAmnOIyYVFVSUm30SUTNFkKeO8Ayvr/CkrWt4e+k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zxXi4D28; arc=none smtp.client-ip=209.85.219.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--edumazet.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zxXi4D28" Received: by mail-qv1-f74.google.com with SMTP id 6a1803df08f44-6d89a389ee9so130017286d6.2 for ; Tue, 14 Jan 2025 12:55:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736888140; x=1737492940; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UdRt2MenkusSF76OGzgxiaoJEpU/B6XIKvkKO5Wni2c=; b=zxXi4D28vWuVR2sfGWT1oNhoeqdXWcidWmmK37fLO0Me0unmioGiO8Fdsop66ALMP+ yvtiY3DpdAHI7AhxOJRg4EsAlIyBWRP2gOrxwnGsYDoWLfIusjrz1p6Pf9UnRub4UHsN 1sT0/ENxz/yHLKtxmRFyBbUeT/iZz5Kjxvk+EYjw2jXAuXUsSf7EjmyxqHhzTWSchm+E ph6/E5dTYLTSC/XnAwla8PZ27tXmj2k6PFZXJeVbrrx+VYDTnBIlcMCmpzbhUGuKItBM /qWQTe5UAnmFUrWqr/TlW+43uME5yVI7TyN2dQFuvzDtgJx1M8KZJFQiEdeSVkGxXiV0 5sYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736888140; x=1737492940; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UdRt2MenkusSF76OGzgxiaoJEpU/B6XIKvkKO5Wni2c=; b=bATXwwi0cj9altu9UrJ5XdGfz5gIdJ66jfMMNQxwqlWcpMcL/wH7bBf/vjbAZvoo9w /QRpIjkMmhLD9LfRqKeHN7U/jdiKOawwL0QZQsUAWLX7VrWriOs0tiGpJIn3ZSlOscli gkRDl6Mz4CRtEEsNhaCxoj3Xp1fFoh8IXr7ciJPQelybOoY3VJBDQyo2tqylZfw3JUZu q8DYRdbdBlwg0FYPNGEAOqyyiaWEyrvSq+LPbdbaJNgd5P+f//8qch9B9oDLLHFby/Dr F845kVY1HxI31eOwZGmb5V/kqTfM1zbIojb+TjjKutGYQ4ZTNslf0UYX8XaHcgS3qCWm 38mw== X-Gm-Message-State: AOJu0YyOcSvRWYkJHEHYWSRFigcq07v/MEtQj3SU+xs+nxZYonDJL4nV pjNIqo+IyI91THtgqczWv3s+GGxQBdb1hHoRfsfmW95KJIvfu889G/ewOHq1ou2sEdiGTMl177O /EmBHI0xrsw== X-Google-Smtp-Source: AGHT+IFhzDvDEuef4XgbiDkCrEpXZ3fOyfaYxd12cBEQbBPPUf2wKyKOFKyFiRs6DqSFe7O1/LcHxu4kzovhvg== X-Received: from qvbow4.prod.google.com ([2002:a05:6214:3f84:b0:6d8:aa06:d4bf]) (user=edumazet job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6214:400d:b0:6d8:7d63:f424 with SMTP id 6a1803df08f44-6df9b1ea89amr387929836d6.12.1736888140030; Tue, 14 Jan 2025 12:55:40 -0800 (PST) Date: Tue, 14 Jan 2025 20:55:31 +0000 In-Reply-To: <20250114205531.967841-1-edumazet@google.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250114205531.967841-1-edumazet@google.com> X-Mailer: git-send-email 2.48.0.rc2.279.g1de40edade-goog Message-ID: <20250114205531.967841-6-edumazet@google.com> Subject: [PATCH v3 net-next 5/5] net: reduce RTNL hold duration in unregister_netdevice_many_notify() (part 2) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski , Paolo Abeni Cc: netdev@vger.kernel.org, Simon Horman , eric.dumazet@gmail.com, Eric Dumazet X-Patchwork-Delegate: kuba@kernel.org One synchronize_net() call is currently done while holding RTNL. This is source of RTNL contention in workloads adding and deleting many network namespaces per second, because synchronize_rcu() and synchronize_rcu_expedited() can use 60+ ms in some cases. For cleanup_net() use, temporarily release RTNL while calling the last synchronize_net(). This should be safe, because devices are no longer visible to other threads after unlist_netdevice() call and setting dev->reg_state to NETREG_UNREGISTERING. In any case, the new netdev_lock() / netdev_unlock() infrastructure that we are adding should allow to fix potential issues, with a combination of a per-device mutex and dev->reg_state awareness. Signed-off-by: Eric Dumazet Reviewed-by: Jesse Brandeburg --- net/core/dev.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 574bd40f3a2bfcc6e43300fad669b1579d48039a..0bb97bb392dcdfa82292fe1ae2ee8290b036df60 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -11601,9 +11601,8 @@ void unregister_netdevice_many_notify(struct list_head *head, rtnl_drop_if_cleanup_net(); flush_all_backlogs(); - rtnl_acquire_if_cleanup_net(); - /* TODO: move this before the prior rtnl_acquire_if_cleanup_net() */ synchronize_net(); + rtnl_acquire_if_cleanup_net(); list_for_each_entry(dev, head, unreg_list) { struct sk_buff *skb = NULL;