From patchwork Fri Apr 4 04:18:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shawn Anastasio X-Patchwork-Id: 14038126 Received: from raptorengineering.com (mail.raptorengineering.com [23.155.224.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92F2513AA5D; Fri, 4 Apr 2025 04:26:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=23.155.224.40 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743740800; cv=none; b=Osk2VbFt6tGVmVxTdyC6AfbWDMl/m7KrMozWl0BACvQp57ZpS/7hibudBVSchkULv9YJKLXUJG8slzzVU878Rda8rYsudz/cbCx/3AMFuyKuEHgz74zHZiv6ARbgET/23x/Zb1Rg8nj1feq80frulMcsj8guYwrPtd54i0k397s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743740800; c=relaxed/simple; bh=4jafmw91MKHslv+kyVYi4PVqlnUFBsKPYLPjn0/ZJgg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eVlqSFG8KkaQPRJFfn4vpUhKXqwF/1a32F1eK/NdH/g65YgG3I7NsMBWdePAjZb1SfCY/ukDL+rfNkH5+VrB5IN1CCwfQGVIr+NLC1PaK88I1T2goLzHS8WmUnRATNNefiAMeiMztqDaVNj70fuge+cLK4OLrvbacP1Zij98C4w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=raptorengineering.com; spf=pass smtp.mailfrom=raptorengineering.com; dkim=pass (1024-bit key) header.d=raptorengineering.com header.i=@raptorengineering.com header.b=C7cDfb7Z; arc=none smtp.client-ip=23.155.224.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=raptorengineering.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=raptorengineering.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=raptorengineering.com header.i=@raptorengineering.com header.b="C7cDfb7Z" Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 1CA4A8287D28; Thu, 3 Apr 2025 23:18:15 -0500 (CDT) Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id R4hZ5MRpKMEH; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 39B3282888D4; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 39B3282888D4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD; t=1743740294; bh=fAe4Tpi/1CPIcV/MWl6tn64VBoEe+zjbtMnLCtQAsSM=; h=From:To:Date:Message-Id:MIME-Version; b=C7cDfb7ZWARbd4e0IfE7mFEDuZlh28NN29CT0E/NFCq33WYLpOQNX5FhFsO9mh1nn y+dsRpzh+7OU6Hnb5cLUOmyHKAM3udhyuwGFOcJD20EpZCzx+cTr67zaXM5w/mwQsh pF5M94SxPLECbSdw3MOJeVSL66J6BSKL3SEjtodY= X-Virus-Scanned: amavisd-new at rptsys.com Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Ax0Jm6ctHBZu; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) Received: from raptor-ewks-026.lan (5.edge.rptsys.com [23.155.224.38]) by mail.rptsys.com (Postfix) with ESMTPSA id AC3CD82878A5; Thu, 3 Apr 2025 23:18:13 -0500 (CDT) From: Shawn Anastasio To: linuxppc-dev@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, tpearson@raptorengineering.com, Madhavan Srinivasan , Michael Ellerman , Christophe Leroy , Naveen N Rao , Bjorn Helgaas , Shawn Anastasio Subject: [PATCH 1/3] pci/hotplug/pnv_php: Properly clean up allocated IRQs on unplug Date: Thu, 3 Apr 2025 23:18:08 -0500 Message-Id: <20250404041810.245984-2-sanastasio@raptorengineering.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404041810.245984-1-sanastasio@raptorengineering.com> References: <20250404041810.245984-1-sanastasio@raptorengineering.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In cases where the root of a nested PCIe bridge configuration is unplugged, the pnv_php driver would leak the allocated IRQ resources for the child bridges' hotplug event notifications, resulting in a panic. Fix this by walking all child buses and deallocating all it's IRQ resources before calling pci_hp_remove_devices. Also modify the lifetime of the workqueue at struct pnv_php_slot::wq so that it is only destroyed in pnv_php_free_slot, instead of pnv_php_disable_irq. This is required since pnv_php_disable_irq will now be called by workers triggered by hot unplug interrupts, so the workqueue needs to stay allocated. The abridged kernel panic that occurs without this patch is as follows: WARNING: CPU: 0 PID: 687 at kernel/irq/msi.c:292 msi_device_data_release+0x6c/0x9c CPU: 0 UID: 0 PID: 687 Comm: bash Not tainted 6.14.0-rc5+ #2 Call Trace: msi_device_data_release+0x34/0x9c (unreliable) release_nodes+0x64/0x13c devres_release_all+0xc0/0x140 device_del+0x2d4/0x46c pci_destroy_dev+0x5c/0x194 pci_hp_remove_devices+0x90/0x128 pci_hp_remove_devices+0x44/0x128 pnv_php_disable_slot+0x54/0xd4 power_write_file+0xf8/0x18c pci_slot_attr_store+0x40/0x5c sysfs_kf_write+0x64/0x78 kernfs_fop_write_iter+0x1b0/0x290 vfs_write+0x3bc/0x50c ksys_write+0x84/0x140 system_call_exception+0x124/0x230 system_call_vectored_common+0x15c/0x2ec Signed-off-by: Shawn Anastasio --- drivers/pci/hotplug/pnv_php.c | 76 ++++++++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 19 deletions(-) diff --git a/drivers/pci/hotplug/pnv_php.c b/drivers/pci/hotplug/pnv_php.c index 573a41869c15..2c07544216fb 100644 --- a/drivers/pci/hotplug/pnv_php.c +++ b/drivers/pci/hotplug/pnv_php.c @@ -36,8 +36,10 @@ static void pnv_php_register(struct device_node *dn); static void pnv_php_unregister_one(struct device_node *dn); static void pnv_php_unregister(struct device_node *dn); +static void pnv_php_enable_irq(struct pnv_php_slot *php_slot); + static void pnv_php_disable_irq(struct pnv_php_slot *php_slot, - bool disable_device) + bool disable_device, bool disable_msi) { struct pci_dev *pdev = php_slot->pdev; u16 ctrl; @@ -53,19 +55,15 @@ static void pnv_php_disable_irq(struct pnv_php_slot *php_slot, php_slot->irq = 0; } - if (php_slot->wq) { - destroy_workqueue(php_slot->wq); - php_slot->wq = NULL; - } - - if (disable_device) { + if (disable_device || disable_msi) { if (pdev->msix_enabled) pci_disable_msix(pdev); else if (pdev->msi_enabled) pci_disable_msi(pdev); + } + if (disable_device) pci_disable_device(pdev); - } } static void pnv_php_free_slot(struct kref *kref) @@ -74,7 +72,8 @@ static void pnv_php_free_slot(struct kref *kref) struct pnv_php_slot, kref); WARN_ON(!list_empty(&php_slot->children)); - pnv_php_disable_irq(php_slot, false); + pnv_php_disable_irq(php_slot, false, false); + destroy_workqueue(php_slot->wq); kfree(php_slot->name); kfree(php_slot); } @@ -561,8 +560,42 @@ static int pnv_php_reset_slot(struct hotplug_slot *slot, bool probe) static int pnv_php_enable_slot(struct hotplug_slot *slot) { struct pnv_php_slot *php_slot = to_pnv_php_slot(slot); + u32 prop32; + int ret; + + ret = pnv_php_enable(php_slot, true); + if (ret) + return ret; - return pnv_php_enable(php_slot, true); + /* (Re-)enable interrupt if the slot supports surprise hotplug */ + ret = of_property_read_u32(php_slot->dn, "ibm,slot-surprise-pluggable", &prop32); + if (!ret && prop32) + pnv_php_enable_irq(php_slot); + + return 0; +} + +/** + * Disable any hotplug interrupts for all slots on the provided bus, as well as + * all downstream slots in preparation for a hot unplug. + */ +static int pnv_php_disable_all_irqs(struct pci_bus *bus) +{ + struct pci_bus *child_bus; + struct pci_slot *cur_slot; + + /* First go down child busses */ + list_for_each_entry(child_bus, &bus->children, node) + pnv_php_disable_all_irqs(child_bus); + + /* Disable IRQs for all pnv_php slots on this bus */ + list_for_each_entry(cur_slot, &bus->slots, list) { + struct pnv_php_slot *php_slot = to_pnv_php_slot(cur_slot->hotplug); + + pnv_php_disable_irq(php_slot, false, true); + } + + return 0; } static int pnv_php_disable_slot(struct hotplug_slot *slot) @@ -579,6 +612,10 @@ static int pnv_php_disable_slot(struct hotplug_slot *slot) php_slot->state != PNV_PHP_STATE_REGISTERED) return 0; + + /* Free all irq resources from slot and all child slots before remove */ + pnv_php_disable_all_irqs(php_slot->bus); + /* Remove all devices behind the slot */ pci_lock_rescan_remove(); pci_hp_remove_devices(php_slot->bus); @@ -647,6 +684,15 @@ static struct pnv_php_slot *pnv_php_alloc_slot(struct device_node *dn) return NULL; } + /* Allocate workqueue for this slot's interrupt handling */ + php_slot->wq = alloc_workqueue("pciehp-%s", 0, 0, php_slot->name); + if (!php_slot->wq) { + SLOT_WARN(php_slot, "Cannot alloc workqueue\n"); + kfree(php_slot->name); + kfree(php_slot); + return NULL; + } + if (dn->child && PCI_DN(dn->child)) php_slot->slot_no = PCI_SLOT(PCI_DN(dn->child)->devfn); else @@ -843,14 +889,6 @@ static void pnv_php_init_irq(struct pnv_php_slot *php_slot, int irq) u16 sts, ctrl; int ret; - /* Allocate workqueue */ - php_slot->wq = alloc_workqueue("pciehp-%s", 0, 0, php_slot->name); - if (!php_slot->wq) { - SLOT_WARN(php_slot, "Cannot alloc workqueue\n"); - pnv_php_disable_irq(php_slot, true); - return; - } - /* Check PDC (Presence Detection Change) is broken or not */ ret = of_property_read_u32(php_slot->dn, "ibm,slot-broken-pdc", &broken_pdc); @@ -869,7 +907,7 @@ static void pnv_php_init_irq(struct pnv_php_slot *php_slot, int irq) ret = request_irq(irq, pnv_php_interrupt, IRQF_SHARED, php_slot->name, php_slot); if (ret) { - pnv_php_disable_irq(php_slot, true); + pnv_php_disable_irq(php_slot, true, true); SLOT_WARN(php_slot, "Error %d enabling IRQ %d\n", ret, irq); return; } From patchwork Fri Apr 4 04:18:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shawn Anastasio X-Patchwork-Id: 14038127 Received: from raptorengineering.com (mail.raptorengineering.com [23.155.224.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 643BB2E62B8; Fri, 4 Apr 2025 04:26:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=23.155.224.40 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743740801; cv=none; b=P6R2naH1ecFRajWXZOaB/jGMJ7IjF1MVKqZIgnsOphY1mmCMZuaXKXQClY0UP0UUuKFE1lS1p6NE/PCg5kvPwoalnLLNlADhBjGUTBCym8s1TDIrINYlOrL+nUDNcpcTNGBXsr7KV+YkBOw+RFO12U0N917VkjN4HSE4U4P7Gmk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743740801; c=relaxed/simple; bh=107+hfmFzFDDXOmCW6raL4he+f4ybLXUIsRpVGyruwg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WDdtEORD1Xry4cJgwQbeC9/vMMAUoJLUQLwab2MH368s3/HHJafflTtXvkXWUEpgaFyxc98vDrF5Tx47I6BHIyLnblMEg7GcPQK3KYdumPqOftX1hTfVvoIulxvX0Tk3KJQmWNntCQ8RKAnzEgUpd9iIDAa5MhrPJ77Cf0ZmGvA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=raptorengineering.com; spf=pass smtp.mailfrom=raptorengineering.com; dkim=pass (1024-bit key) header.d=raptorengineering.com header.i=@raptorengineering.com header.b=ZGJXyuPw; arc=none smtp.client-ip=23.155.224.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=raptorengineering.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=raptorengineering.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=raptorengineering.com header.i=@raptorengineering.com header.b="ZGJXyuPw" Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 8BD958288948; Thu, 3 Apr 2025 23:18:15 -0500 (CDT) Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id F3ugog3yIHEK; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id C832C8287DC7; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com C832C8287DC7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD; t=1743740294; bh=R3TEJwuAJZ0klnHkNfJo7ybCDbPTaShTCK/BgFmdm60=; h=From:To:Date:Message-Id:MIME-Version; b=ZGJXyuPw/vyrls4htxAk3M/fQBla3YmcPvyqrIkmVqacNhP6uZOtU5anDmg+h2i2E 57mze6ubVXAd39dkvw0BwAcwssPd5yZZ+xTGd7Ye98O2wdxmSB+8FPkunVK0CbZ+s5 9hbXD6nrfuXMSKMwsN4u2FhbJXexWmbv+zbPXQAk= X-Virus-Scanned: amavisd-new at rptsys.com Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7J193i_wQp1T; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) Received: from raptor-ewks-026.lan (5.edge.rptsys.com [23.155.224.38]) by mail.rptsys.com (Postfix) with ESMTPSA id 393F38287182; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) From: Shawn Anastasio To: linuxppc-dev@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, tpearson@raptorengineering.com, Madhavan Srinivasan , Michael Ellerman , Christophe Leroy , Naveen N Rao , Bjorn Helgaas , Shawn Anastasio Subject: [PATCH 2/3] pci/hotplug/pnv_php: Work around switches with broken presence detection Date: Thu, 3 Apr 2025 23:18:09 -0500 Message-Id: <20250404041810.245984-3-sanastasio@raptorengineering.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404041810.245984-1-sanastasio@raptorengineering.com> References: <20250404041810.245984-1-sanastasio@raptorengineering.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The Microsemi Switchtec PM8533 PFX 48xG3 [11f8:8533] PCIe switch system was observed to incorrectly assert the Presence Detect Set bit in its capabilities when tested on a Raptor Computing Systems Blackbird system, resulting in the hot insert path never attempting a rescan of the bus and any downstream devices not being re-detected. Work around this by additionally checking whether the PCIe data link is active or not when performing presence detection on downstream switches' ports, similar to the pciehp_hpc.c driver. Signed-off-by: Shawn Anastasio --- drivers/pci/hotplug/pnv_php.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/drivers/pci/hotplug/pnv_php.c b/drivers/pci/hotplug/pnv_php.c index 2c07544216fb..1a734adb5b10 100644 --- a/drivers/pci/hotplug/pnv_php.c +++ b/drivers/pci/hotplug/pnv_php.c @@ -390,6 +390,20 @@ static int pnv_php_get_power_state(struct hotplug_slot *slot, u8 *state) return 0; } +static int pcie_check_link_active(struct pci_dev *pdev) +{ + u16 lnk_status; + int ret; + + ret = pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnk_status); + if (ret == PCIBIOS_DEVICE_NOT_FOUND || PCI_POSSIBLE_ERROR(lnk_status)) + return -ENODEV; + + ret = !!(lnk_status & PCI_EXP_LNKSTA_DLLLA); + + return ret; +} + static int pnv_php_get_adapter_state(struct hotplug_slot *slot, u8 *state) { struct pnv_php_slot *php_slot = to_pnv_php_slot(slot); @@ -402,6 +416,19 @@ static int pnv_php_get_adapter_state(struct hotplug_slot *slot, u8 *state) */ ret = pnv_pci_get_presence_state(php_slot->id, &presence); if (ret >= 0) { + if (pci_pcie_type(php_slot->pdev) == PCI_EXP_TYPE_DOWNSTREAM && + presence == OPAL_PCI_SLOT_EMPTY) { + /* + * Similar to pciehp_hpc, check whether the Link Active + * bit is set to account for broken downstream bridges + * that don't properly assert Presence Detect State, as + * was observed on the Microsemi Switchtec PM8533 PFX + * [11f8:8533]. + */ + if (pcie_check_link_active(php_slot->pdev) > 0) + presence = OPAL_PCI_SLOT_PRESENT; + } + *state = presence; ret = 0; } else { From patchwork Fri Apr 4 04:18:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shawn Anastasio X-Patchwork-Id: 14038128 Received: from raptorengineering.com (mail.raptorengineering.com [23.155.224.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0E0613B797; Fri, 4 Apr 2025 04:26:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=23.155.224.40 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743740801; cv=none; b=Vi3B58V0aEuPyKaJ2qOglPRO1EKPSXob/6Mu/zs+6x2fzFAviCoitqoN2UgqT3ZI7Et1G1tGWd7+W/ZPVtw8NOrOV/QeBh+2SIRVdv3f1Peu81IXDpHNAL84iiKEVojljUy7D4QpIhFpbEUoig6gdtOJAk2mfeYf2iVQfqemK/Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743740801; c=relaxed/simple; bh=GD4PbrRWARoVxmIZIO0RTWV/Z6qA8xZnFttWFtGyuwE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=M9r/7HL/brGDs9XSrp48CRVg5H3X0Dbc11Wb2JAa4DktJDF01sn8zlhMfR/bG1oP3iqTB0j8LetvGCnc6yUJFoAxzC37vQ3uuoFe6bjN4cs6oqygPzBX2+Tn+JEZpp6LJwsE7woL8L0Za0q/DwQH/r6KKk1L8ldWv3Fl7QMNNJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=raptorengineering.com; spf=pass smtp.mailfrom=raptorengineering.com; dkim=pass (1024-bit key) header.d=raptorengineering.com header.i=@raptorengineering.com header.b=NJ2WDg/F; arc=none smtp.client-ip=23.155.224.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=raptorengineering.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=raptorengineering.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=raptorengineering.com header.i=@raptorengineering.com header.b="NJ2WDg/F" Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 4A9C18287DC7; Thu, 3 Apr 2025 23:18:16 -0500 (CDT) Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id y0x95OhCHdpj; Thu, 3 Apr 2025 23:18:15 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mail.rptsys.com (Postfix) with ESMTP id 5125A82888D4; Thu, 3 Apr 2025 23:18:15 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.rptsys.com 5125A82888D4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raptorengineering.com; s=B8E824E6-0BE2-11E6-931D-288C65937AAD; t=1743740295; bh=g8RNqE7aPiU1zvwocf1geGsqM6oe9JplNEQakBf0Reg=; h=From:To:Date:Message-Id:MIME-Version; b=NJ2WDg/FTHCCC6jFgYkZlr24HYP+QOKD2ftg3UT+clr+55Mx+4dkhbO6XW1PFMw85 kD67hZYiQ0jWO3dcGWqL9fYMPsZ19bSMXZH6fABD0/bCFLyOOwJl5bkUmIFGW0Z5T9 kEarqQZJELs45FV7AKLy+5XuXmFBbNHDUgwcNFJ4= X-Virus-Scanned: amavisd-new at rptsys.com Received: from mail.rptsys.com ([127.0.0.1]) by localhost (vali.starlink.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Bl4Uh3ydGZaJ; Thu, 3 Apr 2025 23:18:15 -0500 (CDT) Received: from raptor-ewks-026.lan (5.edge.rptsys.com [23.155.224.38]) by mail.rptsys.com (Postfix) with ESMTPSA id BAF458287D01; Thu, 3 Apr 2025 23:18:14 -0500 (CDT) From: Shawn Anastasio To: linuxppc-dev@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, tpearson@raptorengineering.com, Madhavan Srinivasan , Michael Ellerman , Christophe Leroy , Naveen N Rao , Bjorn Helgaas , Shawn Anastasio Subject: [PATCH 3/3] pci/hotplug/pnv_php: Fix refcount underflow on hot unplug Date: Thu, 3 Apr 2025 23:18:10 -0500 Message-Id: <20250404041810.245984-4-sanastasio@raptorengineering.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404041810.245984-1-sanastasio@raptorengineering.com> References: <20250404041810.245984-1-sanastasio@raptorengineering.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When hot unplugging a slot containing a PCIe switch on a PowerNV system, the reference count of the device_node corresponding to the root will underflow. This is due to improper handling of the device_nodes' refcounts in pnv_php_detach_device nodes that occurs on each unplug event. When iterating through children nodes, pnv_php_detach_nodes first recursively detach each child's children, then it would decrement the child's refcount and finally call of_detach_node on it, which in turn would decrement the refcount further and result in an underflow. Fix this by dropping the explicit of_put call and by moving the final of_detach_node call after the loop. The underflow that occurs without this patch produces the following backtrace on unplug events: refcount_t: underflow; use-after-free. WARNING: CPU: 4 PID: 669 at lib/refcount.c:28 refcount_warn_saturate+0x214/0x224 Call Trace: refcount_warn_saturate+0x210/0x224 (unreliable) kobject_put+0x154/0x2d4 of_node_put+0x2c/0x40 of_get_next_child+0x74/0xd0 pnv_php_detach_device_nodes+0x2a4/0x30c pnv_php_set_slot_power_state+0x20c/0x500 pnv_php_disable_slot+0xb8/0xdc power_write_file+0xf8/0x18c pci_slot_attr_store+0x40/0x5c sysfs_kf_write+0x64/0x78 kernfs_fop_write_iter+0x1b4/0x2a4 vfs_write+0x3bc/0x50c ksys_write+0x84/0x140 system_call_exception+0x124/0x230 system_call_vectored_common+0x15c/0x2ec Signed-off-by: Shawn Anastasio --- drivers/pci/hotplug/pnv_php.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/pci/hotplug/pnv_php.c b/drivers/pci/hotplug/pnv_php.c index 1a734adb5b10..a3fa44f7bf1a 100644 --- a/drivers/pci/hotplug/pnv_php.c +++ b/drivers/pci/hotplug/pnv_php.c @@ -156,11 +156,12 @@ static void pnv_php_detach_device_nodes(struct device_node *parent) struct device_node *dn; for_each_child_of_node(parent, dn) { + /* Detach any children of the parent node first */ pnv_php_detach_device_nodes(dn); - - of_node_put(dn); - of_detach_node(dn); } + + /* Finally, detach the parent */ + of_detach_node(parent); } static void pnv_php_rmv_devtree(struct pnv_php_slot *php_slot)