From patchwork Fri Feb 4 18:36:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12735451 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1BACC433F5 for ; Fri, 4 Feb 2022 18:36:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236465AbiBDSgl (ORCPT ); Fri, 4 Feb 2022 13:36:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243054AbiBDSgh (ORCPT ); Fri, 4 Feb 2022 13:36:37 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F010C061714 for ; Fri, 4 Feb 2022 10:36:37 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id d5so6378033pjk.5 for ; Fri, 04 Feb 2022 10:36:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U8mvndYc5GiIgFu7fMQsqr6+5Dn5yDgnXOI/6CxhF5c=; b=lIW5JCQemPN/GJOX43CN0V/vd3xcbZjg9iCYnMmljRO8dIUPpAsFFdu51OpCrl8Tbp kXQm0Aoi4I+Un2k+wl+hspNRqtkNCZ/LIuV79waIGwdLfEPPWA4iSUfuf1fK3TVaVIW2 qjHk02tW6Q8mOuky6sKfAwYhbw3RwKuAtn9pYMJRj4Mf2z7D5SEOAJjACgllhmd7/9Yg daxLLs0tIoeo1s3bJHUfSy118810mzsAkulaMVq22mOFOcEvfVfTp2fZxWEzz8X6BoSl dGn12v1NDZl0UVFLBPT/m/ipi9foRMCX2KzbADXeck0UMzvxEoEY5dJ7vsi5mdZxHj1O YY5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U8mvndYc5GiIgFu7fMQsqr6+5Dn5yDgnXOI/6CxhF5c=; b=pur+mDDVw6G+Qq1yZCmLCxgaLaqR0oumjX+jjy0B8Lc5s4hG8rqXIG0OqtbDBP2EjC WnCUj/AphgKZVkof8ER3j8R3qpR+hkzOm/YFTDM7Ygxwfxrxm+TC+BIzTHHLIXPTuz9K bVKaPfXg2uOOgkOr63rYDlNNNMAHjbLDmURnbZE0cDatrgZu6Jwq7I4VIdjgPV+cBzH+ tfx6wupfrKglYgHIDJtriir3NyqUtjR17z2/Mp5O4+RyWcPeqiWnntQSVJBj7OoTL1Dz JtNI4wjEJuGsc7fSYJLY6FQ2axbKX73MpCFgMaOQiJKpdn1wQEvX/W3Gor530u1OmqQT SQew== X-Gm-Message-State: AOAM531COWvhsyUMfh9bQuVw94bSeWIi0P2MUrH/Vz7Lac3AW6FZQL2s fMkY+4Ea41we6fbK0vV1MMk= X-Google-Smtp-Source: ABdhPJzUHFIAoELeSeIGy9wvLaEJQUFpCtrV8cmZ4P8h/BMQtmMb87HnibhjuxwaIOFjiXhdEoswCA== X-Received: by 2002:a17:902:7c02:: with SMTP id x2mr4238733pll.47.1643999796674; Fri, 04 Feb 2022 10:36:36 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:e0d3:6ec9:bd06:3e67]) by smtp.gmail.com with ESMTPSA id f15sm3483142pfv.189.2022.02.04.10.36.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Feb 2022 10:36:36 -0800 (PST) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet Subject: [PATCH net-next 1/3] ref_tracker: implement use-after-free detection Date: Fri, 4 Feb 2022 10:36:28 -0800 Message-Id: <20220204183630.2376998-2-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220204183630.2376998-1-eric.dumazet@gmail.com> References: <20220204183630.2376998-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet Whenever ref_tracker_dir_init() is called, mark the struct ref_tracker_dir as dead. Test the dead status from ref_tracker_alloc() and ref_tracker_free() This should detect buggy dev_put()/dev_hold() happening too late in netdevice dismantle process. Signed-off-by: Eric Dumazet --- include/linux/ref_tracker.h | 2 ++ lib/ref_tracker.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/ref_tracker.h b/include/linux/ref_tracker.h index 60f3453be23e6881725d383c55f93143fda1e7a2..a443abda937d86ff534225bf16b958a9da295a7d 100644 --- a/include/linux/ref_tracker.h +++ b/include/linux/ref_tracker.h @@ -13,6 +13,7 @@ struct ref_tracker_dir { spinlock_t lock; unsigned int quarantine_avail; refcount_t untracked; + bool dead; struct list_head list; /* List of active trackers */ struct list_head quarantine; /* List of dead trackers */ #endif @@ -26,6 +27,7 @@ static inline void ref_tracker_dir_init(struct ref_tracker_dir *dir, INIT_LIST_HEAD(&dir->quarantine); spin_lock_init(&dir->lock); dir->quarantine_avail = quarantine_count; + dir->dead = false; refcount_set(&dir->untracked, 1); stack_depot_init(); } diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c index a6789c0c626b0f68ad67c264cd19177a63fb82d2..32ff6bd497f8e464eeb51a3628cb24bded0547da 100644 --- a/lib/ref_tracker.c +++ b/lib/ref_tracker.c @@ -20,6 +20,7 @@ void ref_tracker_dir_exit(struct ref_tracker_dir *dir) unsigned long flags; bool leak = false; + dir->dead = true; spin_lock_irqsave(&dir->lock, flags); list_for_each_entry_safe(tracker, n, &dir->quarantine, head) { list_del(&tracker->head); @@ -72,6 +73,8 @@ int ref_tracker_alloc(struct ref_tracker_dir *dir, gfp_t gfp_mask = gfp; unsigned long flags; + WARN_ON_ONCE(dir->dead); + if (gfp & __GFP_DIRECT_RECLAIM) gfp_mask |= __GFP_NOFAIL; *trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask); @@ -100,6 +103,8 @@ int ref_tracker_free(struct ref_tracker_dir *dir, unsigned int nr_entries; unsigned long flags; + WARN_ON_ONCE(dir->dead); + if (!tracker) { refcount_dec(&dir->untracked); return -EEXIST; From patchwork Fri Feb 4 18:36:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12735452 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DBDCC433F5 for ; Fri, 4 Feb 2022 18:36:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377584AbiBDSgn (ORCPT ); Fri, 4 Feb 2022 13:36:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1377627AbiBDSgk (ORCPT ); Fri, 4 Feb 2022 13:36:40 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10200C061401 for ; Fri, 4 Feb 2022 10:36:40 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id y17so5864527plg.7 for ; Fri, 04 Feb 2022 10:36:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BiDjGDalIJZgno57zKrydIx7zYwfZotf0yffC1XSl/o=; b=DviQ3myROXk73iicb5/Hhhudaifzd2FOi0zrjDpnw8s8Xc8xbatcTltrmZNd9sd/zL c9Aca9Pz965q2CsKPGtZ9fh2t/XfBMnYP0WGQWtvlo0kk3VCpSovnoI+wYrvuP5LvegH K9vR3XBLrBWAzvaP03u4C6jaqZZvTRsFKft1kOImHr2IQ3bSV91ympYfBbA1AKhnVdsz Iwx9ckTCytASoq6S+JrHufhvJZK+FWkwKwumJdIkr8VuS+p5UiXLPDIMjJxJdEaea4yy ELy5R7QG4qhdFO66rSbtVFrX+pHMKBMJtaWrBy6/ayDEl7wJhdFh0eaYbdtUyrp49JFD SC+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BiDjGDalIJZgno57zKrydIx7zYwfZotf0yffC1XSl/o=; b=yFJ29U8MwNB3GhDx59TKuQ3dd673wyg06fkPzAnhUoS77whCYQcNAv5CVosVfp2PS4 uw9+lN+vNcIf7qx4sakXrisKfw8Na2Eq98iXh07c9RIhF3W5UZHi871vobz9gtgL9tXa 1xIkC+FXdcYQux7HaaOB+DgLqab6SF+e5RXoZZZYJjBk9m5lFDHEgubnntrawjzTpiOM PinSpfgJF0lmVnf0yUXWSSTAnVT41QDy831ZXtKObha9cKtPB2Y2Tmv+mYlsCV6ICoIn CbMyWeNJnGg6jhHc5vJREezd2o+zyyKIDMrDEs0E6HJ2ihpv3mpCBGhTcBUb8exxR84A W6Zw== X-Gm-Message-State: AOAM533AtvZrI3g/6csgJNsP+JW9EmkaBxNrPORzfUDt3PeZ3RIlSwzU JiESIMAZL3aM01GGkViEphw= X-Google-Smtp-Source: ABdhPJz58kY7zY/TmKQgTz/Tmi62Y+WgAzeUekgnq9gcOR+A8Wm0T+60OL6mn9bQfM0lDecwSDQOAQ== X-Received: by 2002:a17:90b:3144:: with SMTP id ip4mr4651034pjb.23.1643999799630; Fri, 04 Feb 2022 10:36:39 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:e0d3:6ec9:bd06:3e67]) by smtp.gmail.com with ESMTPSA id f15sm3483142pfv.189.2022.02.04.10.36.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Feb 2022 10:36:39 -0800 (PST) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet Subject: [PATCH net-next 2/3] ref_tracker: add a count of untracked references Date: Fri, 4 Feb 2022 10:36:29 -0800 Message-Id: <20220204183630.2376998-3-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220204183630.2376998-1-eric.dumazet@gmail.com> References: <20220204183630.2376998-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet We are still chasing a netdev refcount imbalance, and we suspect we have one rogue dev_put() that is consuming a reference taken from a dev_hold_track() To detect this case, allow ref_tracker_alloc() and ref_tracker_free() to be called with a NULL @trackerp parameter, and use a dedicated refcount_t just for them. Signed-off-by: Eric Dumazet --- include/linux/ref_tracker.h | 2 ++ lib/ref_tracker.c | 12 +++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/include/linux/ref_tracker.h b/include/linux/ref_tracker.h index a443abda937d86ff534225bf16b958a9da295a7d..9ca353ab712b5e897d9b3e5cfcd7117b610dd01a 100644 --- a/include/linux/ref_tracker.h +++ b/include/linux/ref_tracker.h @@ -13,6 +13,7 @@ struct ref_tracker_dir { spinlock_t lock; unsigned int quarantine_avail; refcount_t untracked; + refcount_t no_tracker; bool dead; struct list_head list; /* List of active trackers */ struct list_head quarantine; /* List of dead trackers */ @@ -29,6 +30,7 @@ static inline void ref_tracker_dir_init(struct ref_tracker_dir *dir, dir->quarantine_avail = quarantine_count; dir->dead = false; refcount_set(&dir->untracked, 1); + refcount_set(&dir->no_tracker, 1); stack_depot_init(); } diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c index 32ff6bd497f8e464eeb51a3628cb24bded0547da..9c0c2e09df666d19aba441f568762afbd1cad4d0 100644 --- a/lib/ref_tracker.c +++ b/lib/ref_tracker.c @@ -38,6 +38,7 @@ void ref_tracker_dir_exit(struct ref_tracker_dir *dir) spin_unlock_irqrestore(&dir->lock, flags); WARN_ON_ONCE(leak); WARN_ON_ONCE(refcount_read(&dir->untracked) != 1); + WARN_ON_ONCE(refcount_read(&dir->no_tracker) != 1); } EXPORT_SYMBOL(ref_tracker_dir_exit); @@ -75,6 +76,10 @@ int ref_tracker_alloc(struct ref_tracker_dir *dir, WARN_ON_ONCE(dir->dead); + if (!trackerp) { + refcount_inc(&dir->no_tracker); + return 0; + } if (gfp & __GFP_DIRECT_RECLAIM) gfp_mask |= __GFP_NOFAIL; *trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask); @@ -98,13 +103,18 @@ int ref_tracker_free(struct ref_tracker_dir *dir, struct ref_tracker **trackerp) { unsigned long entries[REF_TRACKER_STACK_ENTRIES]; - struct ref_tracker *tracker = *trackerp; depot_stack_handle_t stack_handle; + struct ref_tracker *tracker; unsigned int nr_entries; unsigned long flags; WARN_ON_ONCE(dir->dead); + if (!trackerp) { + refcount_dec(&dir->no_tracker); + return 0; + } + tracker = *trackerp; if (!tracker) { refcount_dec(&dir->untracked); return -EEXIST; From patchwork Fri Feb 4 18:36:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 12735453 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 159C3C433FE for ; Fri, 4 Feb 2022 18:36:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377588AbiBDSgp (ORCPT ); Fri, 4 Feb 2022 13:36:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243054AbiBDSgm (ORCPT ); Fri, 4 Feb 2022 13:36:42 -0500 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F8ADC061714 for ; Fri, 4 Feb 2022 10:36:42 -0800 (PST) Received: by mail-pl1-x62c.google.com with SMTP id z5so5865062plg.8 for ; Fri, 04 Feb 2022 10:36:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CHdSO/DuryrBkYeH5Fb7CmRIJsy0lIkMMra7/bWLPjM=; b=FdrRbRBFAG2cbnyjeGXFYSrDWEUNcXQPtJ9hS/YyAUgRisoazf8Q4c1ift9+vYJqOv IELEdpu/ZR6B2PRuuMIUniIBUb7/NwTKKqZcGoAcu43FQ+PfuZx/CoVE2idxf/jEbfxj +Gwt3Qm80UMUn8Wl1bciaruQ+fIpVoVp/sGPuBwaW4Zm/upe5/m9ZrxkJoVbfnVSfK+v 5f5tp3mQbbP8b7IWYJpqnoS374PSMShu+IqJaaztjZnWPKDzhb8rbWMH6yorgyPapgaH uNvcO/JWpI2BaoLTx72d/tJiXAo7wd6b8IldPtcwjjC8U7nWuOE+GKP1s2p7x98dVfgB /zFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CHdSO/DuryrBkYeH5Fb7CmRIJsy0lIkMMra7/bWLPjM=; b=XWnzCG+ks3eequ6r9LGD7VQDExrSCHbOAhKbpuHkIYuHAB17NpagM3ZNJ73HJyAA0a fyHTGjEgfsbyNy1xX0Ew4SWk273mWW9dRl7j0Xkle7mmTLuBt3jEjSbt/TGF0x15Vcv9 yHLnaMPR0sGacxvtqsGgUVVh9/26u/B0pukLojivC5iypXMyJE8u6HzQQJDYeVcJyHbu f901hA03oCdugbbmvGg8rDTBNACGpVZ1sSKpivkfa4PnITXIIe8LJQgN9PImRVvC06i2 SKnqx7UOjc79lL0Cu8jQcUK8sxISPDOHKETFq7tinpFU7BAsFMAOq/Yy0goY2m7YRxAJ 0IeQ== X-Gm-Message-State: AOAM530xRrKRyaOUk2au/E3U/ez+eSLVXvIiq+EN/s0RGYbrlm0uI2kf JmkZNX5deS7nQMkkfVbPDdE= X-Google-Smtp-Source: ABdhPJxNxIcGeQqU0CRetX/aLWIDUXhpfOjEjkpbFOsoBNMMR3z7OoTRLNkrSuapxqcdAeaNwDbM1Q== X-Received: by 2002:a17:90b:19d5:: with SMTP id nm21mr1814043pjb.56.1643999801628; Fri, 04 Feb 2022 10:36:41 -0800 (PST) Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:e0d3:6ec9:bd06:3e67]) by smtp.gmail.com with ESMTPSA id f15sm3483142pfv.189.2022.02.04.10.36.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Feb 2022 10:36:41 -0800 (PST) From: Eric Dumazet To: "David S . Miller" , Jakub Kicinski Cc: netdev , Eric Dumazet , Eric Dumazet Subject: [PATCH net-next 3/3] net: refine dev_put()/dev_hold() debugging Date: Fri, 4 Feb 2022 10:36:30 -0800 Message-Id: <20220204183630.2376998-4-eric.dumazet@gmail.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog In-Reply-To: <20220204183630.2376998-1-eric.dumazet@gmail.com> References: <20220204183630.2376998-1-eric.dumazet@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Eric Dumazet We are still chasing some syzbot reports where we think a rogue dev_put() is called with no corresponding prior dev_hold(). Unfortunately it eats a reference on dev->dev_refcnt taken by innocent dev_hold_track(), meaning that the refcount saturation splat comes too late to be useful. Make sure that 'not tracked' dev_put() and dev_hold() better use CONFIG_NET_DEV_REFCNT_TRACKER=y debug infrastructure: Prior patch in the series allowed ref_tracker_alloc() and ref_tracker_free() to be called with a NULL @trackerp parameter, and to use a separate refcount only to detect too many put() even in the following case: dev_hold_track(dev, tracker_1, GFP_ATOMIC); dev_hold(dev); dev_put(dev); dev_put(dev); // Should complain loudly here. dev_put_track(dev, tracker_1); // instead of here Add clarification about netdev_tracker_alloc() role. Signed-off-by: Eric Dumazet --- include/linux/netdevice.h | 69 ++++++++++++++++++++++++++------------- net/core/dev.c | 2 +- 2 files changed, 47 insertions(+), 24 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index e490b84732d1654bf067b30f2bb0b0825f88dea9..3fb6fb67ed77e70314a699c9bdf8f4b26acfcc19 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3817,14 +3817,7 @@ extern unsigned int netdev_budget_usecs; /* Called by rtnetlink.c:rtnl_unlock() */ void netdev_run_todo(void); -/** - * dev_put - release reference to device - * @dev: network device - * - * Release reference to device to allow it to be freed. - * Try using dev_put_track() instead. - */ -static inline void dev_put(struct net_device *dev) +static inline void __dev_put(struct net_device *dev) { if (dev) { #ifdef CONFIG_PCPU_DEV_REFCNT @@ -3835,14 +3828,7 @@ static inline void dev_put(struct net_device *dev) } } -/** - * dev_hold - get reference to device - * @dev: network device - * - * Hold reference to device to keep it from being freed. - * Try using dev_hold_track() instead. - */ -static inline void dev_hold(struct net_device *dev) +static inline void __dev_hold(struct net_device *dev) { if (dev) { #ifdef CONFIG_PCPU_DEV_REFCNT @@ -3853,11 +3839,24 @@ static inline void dev_hold(struct net_device *dev) } } +static inline void __netdev_tracker_alloc(struct net_device *dev, + netdevice_tracker *tracker, + gfp_t gfp) +{ +#ifdef CONFIG_NET_DEV_REFCNT_TRACKER + ref_tracker_alloc(&dev->refcnt_tracker, tracker, gfp); +#endif +} + +/* netdev_tracker_alloc() can upgrade a prior untracked reference + * taken by dev_get_by_name()/dev_get_by_index() to a tracked one. + */ static inline void netdev_tracker_alloc(struct net_device *dev, netdevice_tracker *tracker, gfp_t gfp) { #ifdef CONFIG_NET_DEV_REFCNT_TRACKER - ref_tracker_alloc(&dev->refcnt_tracker, tracker, gfp); + refcount_dec(&dev->refcnt_tracker.no_tracker); + __netdev_tracker_alloc(dev, tracker, gfp); #endif } @@ -3873,8 +3872,8 @@ static inline void dev_hold_track(struct net_device *dev, netdevice_tracker *tracker, gfp_t gfp) { if (dev) { - dev_hold(dev); - netdev_tracker_alloc(dev, tracker, gfp); + __dev_hold(dev); + __netdev_tracker_alloc(dev, tracker, gfp); } } @@ -3883,10 +3882,34 @@ static inline void dev_put_track(struct net_device *dev, { if (dev) { netdev_tracker_free(dev, tracker); - dev_put(dev); + __dev_put(dev); } } +/** + * dev_hold - get reference to device + * @dev: network device + * + * Hold reference to device to keep it from being freed. + * Try using dev_hold_track() instead. + */ +static inline void dev_hold(struct net_device *dev) +{ + dev_hold_track(dev, NULL, GFP_ATOMIC); +} + +/** + * dev_put - release reference to device + * @dev: network device + * + * Release reference to device to allow it to be freed. + * Try using dev_put_track() instead. + */ +static inline void dev_put(struct net_device *dev) +{ + dev_put_track(dev, NULL); +} + static inline void dev_replace_track(struct net_device *odev, struct net_device *ndev, netdevice_tracker *tracker, @@ -3895,11 +3918,11 @@ static inline void dev_replace_track(struct net_device *odev, if (odev) netdev_tracker_free(odev, tracker); - dev_hold(ndev); - dev_put(odev); + __dev_hold(ndev); + __dev_put(odev); if (ndev) - netdev_tracker_alloc(ndev, tracker, gfp); + __netdev_tracker_alloc(ndev, tracker, gfp); } /* Carrier loss detection, dial on demand. The functions netif_carrier_on diff --git a/net/core/dev.c b/net/core/dev.c index f79744d99413434ad28b26dee9aeeb2893a0e3ae..1eaa0b88e3ba5d800484656f2c3420af57050294 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -10172,7 +10172,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, dev->pcpu_refcnt = alloc_percpu(int); if (!dev->pcpu_refcnt) goto free_dev; - dev_hold(dev); + __dev_hold(dev); #else refcount_set(&dev->dev_refcnt, 1); #endif