From patchwork Thu Oct 31 06:45:07 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 3119001 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id BCBE2BF924 for ; Thu, 31 Oct 2013 06:45:51 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 5479D203DF for ; Thu, 31 Oct 2013 06:45:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C3423203E6 for ; Thu, 31 Oct 2013 06:45:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752836Ab3JaGpi (ORCPT ); Thu, 31 Oct 2013 02:45:38 -0400 Received: from mail-ie0-f170.google.com ([209.85.223.170]:62525 "EHLO mail-ie0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751668Ab3JaGpI (ORCPT ); Thu, 31 Oct 2013 02:45:08 -0400 Received: by mail-ie0-f170.google.com with SMTP id at1so4314247iec.15 for ; Wed, 30 Oct 2013 23:45:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=bs0XP4dMYjfdlHeJeFWqzsgnyuFbwiW8xOgxNbECevk=; b=kbrzPKBAwUlyyL3hEHNsZ34pStriRkBw/wxnCbyo1QdDe0D8oVeMu7dX8FdJtIMLO3 PIQa+kfO9w/YJzrQcjTTUkz3KhJvIEWOjv0IR930AbguvFM2/X/8/gcBaRPhVINrVpou w6Gaby8+vjbYI/zHAPI8N/wMo3eG5v+e1wA9xRq6O89dBectAjBpwdvJzVLCwGYtwO2u usAfYZ1uEPXHf3+iL/IerXyHrKGSDZ9ks6jCj797YbiB6cnem1V9fHFH3tmd3ayX4601 CDkF5lYsHMrFVEWyHrW9rkmQ+FF4uBX0S+Dy6JL1815MRjCpzBSdXxOY/hqb3ZKFci+e wkWw== MIME-Version: 1.0 X-Received: by 10.50.29.4 with SMTP id f4mr4710791igh.11.1383201907294; Wed, 30 Oct 2013 23:45:07 -0700 (PDT) Received: by 10.64.235.70 with HTTP; Wed, 30 Oct 2013 23:45:07 -0700 (PDT) In-Reply-To: <5270DBFC.3050705@cn.fujitsu.com> References: <5270DBFC.3050705@cn.fujitsu.com> Date: Wed, 30 Oct 2013 23:45:07 -0700 X-Google-Sender-Auth: 7dAn9TROn9WbPH2y53BNdRuS5b4 Message-ID: Subject: Re: [PATCH] pci: fix invalid list entry warning for double pci device removing via sysfs From: Yinghai Lu To: Gu Zheng Cc: Bjorn Helgaas , "linux-pci@vger.kernel.org" , linux-kernel Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,T_TVD_MIME_EPI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Wed, Oct 30, 2013 at 3:14 AM, Gu Zheng wrote: > When concurent removing pci devices which are in the same pci subtree > via sysfs, such as: > echo -n 1 > /sys/bus/pci/devices/0000\:10\:00.0/remove ; echo -n 1 > > /sys/bus/pci/devices/0000\:1a\:01.0/remove > (1a:01.0 device is downstream from the 10:00.0 bridge) > > the following warning will show: > [ 1799.280918] ------------[ cut here ]------------ > [ 1799.336199] WARNING: CPU: 7 PID: 126 at lib/list_debug.c:53 __list_del_entry+0x63/0xd0() > [ 1799.433093] list_del corruption, ffff8807b4a7c000->next is LIST_POISON1 (dead000000100100) > [ 1799.532110] Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6table_mangle ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sg dm_mirror dm_region_hash dm_log dm_mod vfat fat e1000e igb iTCO_wdt iTCO_vendor_support ioatdma ptp i7core_edac ipmi_si edac_core lpc_ich pps_core i2c_i801 pcspkr mfd_core dca ipmi_msghandler acpi_cpufreq xfs libcrc32c sd_mod crc_t10dif crct10dif_common i2c_algo_bit drm_kms_helper ttm drm mptsas scsi_transport_sas mptscsih i2c_core megaraid_sas mptbase > [ 1800.276623] CPU: 7 PID: 126 Comm: kworker/u512:1 Tainted: G W 3.12.0-rc5+ #196 > [ 1800.508918] Workqueue: sysfsd sysfs_schedule_callback_work > [ 1800.574703] 0000000000000009 ffff8807adbadbd8 ffffffff8168b26c ffff8807c27d08a8 > [ 1800.663860] ffff8807adbadc28 ffff8807adbadc18 ffffffff810711dc ffff8807adbadc68 > [ 1800.753130] ffff8807b4a7c000 ffff8807b4a7c000 ffff8807ad089c00 0000000000000000 > [ 1800.842282] Call Trace: > [ 1800.871651] [] dump_stack+0x55/0x76 > [ 1800.933301] [] warn_slowpath_common+0x8c/0xc0 > [ 1801.005283] [] warn_slowpath_fmt+0x46/0x50 > [ 1801.074081] [] __list_del_entry+0x63/0xd0 > [ 1801.141839] [] list_del+0x11/0x40 > [ 1801.201320] [] pci_remove_bus_device+0x6a/0xe0 > [ 1801.274279] [] pci_stop_and_remove_bus_device+0x1e/0x30 > [ 1801.356606] [] remove_callback+0x2b/0x40 > [ 1801.423412] [] sysfs_schedule_callback_work+0x18/0x60 > [ 1801.503744] [] process_one_work+0x1f5/0x540 > [ 1801.573640] [] ? process_one_work+0x193/0x540 > [ 1801.645616] [] worker_thread+0x11c/0x370 > [ 1801.712337] [] ? rescuer_thread+0x350/0x350 > [ 1801.782178] [] kthread+0xed/0x100 > [ 1801.841661] [] ? kthread_create_on_node+0x160/0x160 > [ 1801.919919] [] ret_from_fork+0x7c/0xb0 > [ 1801.984608] [] ? kthread_create_on_node+0x160/0x160 > [ 1802.062825] ---[ end trace d77f2054de000fb7 ]--- > > This issue is related to the bug 54411: > https://bugzilla.kernel.org/show_bug.cgi?id=54411 > > Since we added the pci_bus reference management, the bug becomes a > invalid list entry warning as descripted above. Beacuse it still > trys to delete an deleted pci device from device list. > > So here we make stop device actually detach driver only, and remove > device will do device_del instead, and move bus_list change and pci device > resource free into pci_release_dev, so that it'll consistent with > bus reference managment, and the device only can be deleted from device > list in pci_release_dev just for one time. > > Besides, it also makes hostbridge to use device_unregister to be pair > with device_register before. > > This patch is based on Yinghai's privious similar patch: > http://lkml.org/lkml/2013/5/13/658 I have updated version, please check if attached patches fix the problem. virtfn_release.patch fix_cx3_hotplug_2.patch fix_racing_removing_6_1.patch fix_racing_removing_6_2.patch fix_racing_removing_6_3.patch Thanks Yinghai Subject: [PATCH 3/7] PCI: Destroy pci dev only once From: Yinghai Lu Mutliple removing via /sys will call pci_destroy_dev two times. Add is_removed to record if pci_destroy_dev is called already. During second calling, still have extra dev ref hold via device_schedule_call, so we are safe to check dev->is_removed. Signed-off-by: Yinghai Lu --- drivers/pci/remove.c | 5 ++++- include/linux/pci.h | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) Index: linux-2.6/drivers/pci/remove.c =================================================================== --- linux-2.6.orig/drivers/pci/remove.c +++ linux-2.6/drivers/pci/remove.c @@ -22,7 +22,10 @@ static void pci_stop_dev(struct pci_dev static void pci_destroy_dev(struct pci_dev *dev) { - put_device(&dev->dev); + if (!dev->is_removed) { + dev->is_removed = 1; + put_device(&dev->dev); + } } void pci_remove_bus(struct pci_bus *bus) Index: linux-2.6/include/linux/pci.h =================================================================== --- linux-2.6.orig/include/linux/pci.h +++ linux-2.6/include/linux/pci.h @@ -308,6 +308,7 @@ struct pci_dev { unsigned int multifunction:1;/* Part of multi-function device */ /* keep track of device state */ unsigned int is_added:1; + unsigned int is_removed:1; /* pci_destroy_dev is called */ unsigned int is_busmaster:1; /* device is busmaster */ unsigned int no_msi:1; /* device may not use msi */ unsigned int block_cfg_access:1; /* config space access is blocked */