From patchwork Tue Nov 28 00:24:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cong Wang X-Patchwork-Id: 10078459 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8B0026028E for ; Tue, 28 Nov 2017 00:25:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C836289F9 for ; Tue, 28 Nov 2017 00:25:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6CCAA289E8; Tue, 28 Nov 2017 00:25:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA00C289E8 for ; Tue, 28 Nov 2017 00:25:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753635AbdK1AZC (ORCPT ); Mon, 27 Nov 2017 19:25:02 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:40932 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753627AbdK1AY7 (ORCPT ); Mon, 27 Nov 2017 19:24:59 -0500 Received: by mail-pl0-f65.google.com with SMTP id 62so9529222plc.7; Mon, 27 Nov 2017 16:24:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=5Bo1oLxXH6/e++8a3LHBwTEP5VLeUuduBD4LBrb0LcU=; b=R6AO2DFxjkGUDVyGIf7avZmxNw4vEUu4wL3OdGSEXwUeKrlb3yvs5ysvTrvlzIZLYK pedDON3ltV9r3fSmeyANPzs7RPUS9nynwqo1WVmw5aERC+Baq4nrHqBTgsMh695oNt7o teZEWHE/wljOwpwk4O5PZF7+ZkIIZpJAxbW4X/oAzm6SHcCCrPj2tD2/SVs7MaHa/4e0 KAAG7qKiiDDjuVKzX9k3aWXgFn5w28Ab7V+Tf5EOapPQAdrZuY1cCRE12QMPvkk1YjKh UruPDdXOMIdmA90sNqVOvpQq9D3dS/59FJj27S4Z8WHG6UoHJHqU8oDS3royygjZgNAP 7+5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=5Bo1oLxXH6/e++8a3LHBwTEP5VLeUuduBD4LBrb0LcU=; b=cNb1ZkNCf7S2p9BdGLxvX/0klUSxNV/0bYwnwa1m4ri+NIc7zBR/XmosWPZ5RDuaMA MUiGRZrOSeIsSpgTkJax2BJJ+JyYlFlYvWiYyYEKxmnaRXQ2GGTWFAFjZCP0xBbIhoFp eqfwKDiYeh/horEFPNBNS0NvMuAQ/ZvkNUpyjHbThZhBwNf9XNa1SIRW0GfhwN2ISj7u T+5hPQvy2pXW174Vt+luoZ9DgyT7+w09pW2eBNtMI/lbMShxTzTZ0a1r2a2esGlRv2gx OHQlkcFP5Om/0B8oRDvqx5AIA0Eyj5UWN9L3EgeQlN+gST3UDAuyOUhG9MvCyHoF7sAS 3euQ== X-Gm-Message-State: AJaThX6Yo1bYkD0VhIo8qoXREWd5Ml6digpDLTV2Y2AmgFZagg5Pmkz+ f/KFUsLjFWV1vbwaDZOaVGXy0cHl X-Google-Smtp-Source: AGs4zMZ57i6zbTtSflP6m/jpZABXVn6oFvPqG55Kw/1jGWVmFsRRveVstZfg4j/c2bKnmySTQdYQTQ== X-Received: by 10.84.242.76 with SMTP id c12mr40973836pll.445.1511828698771; Mon, 27 Nov 2017 16:24:58 -0800 (PST) Received: from tw-172-25-30-113.office.twttr.net ([8.25.197.25]) by smtp.gmail.com with ESMTPSA id 73sm56826786pfr.145.2017.11.27.16.24.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 27 Nov 2017 16:24:58 -0800 (PST) From: Cong Wang To: linux-kernel@vger.kernel.org Cc: Cong Wang , Dan Williams , Johannes Thumshirn , Praveen Murali , "James E.J. Bottomley" , "Martin K. Petersen" , linux-scsi@vger.kernel.org Subject: [PATCH] libsas: flush pending destruct work in sas_unregister_domain_devices() Date: Mon, 27 Nov 2017 16:24:45 -0800 Message-Id: <20171128002445.16594-1-xiyou.wangcong@gmail.com> X-Mailer: git-send-email 2.9.4 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We saw dozens of the following kernel waring: WARNING: CPU: 0 PID: 705 at fs/sysfs/group.c:224 sysfs_remove_group+0x54/0x88() sysfs group ffffffff81ab7670 not found for kobject '6:0:3:0' Modules linked in: cpufreq_ondemand x86_pkg_temp_thermal coretemp kvm_intel kvm microcode raid0 iTCO_wdt iTCO_vendor_support sb_edac edac_core lpc_ich mfd_core ioatdma i2c_i801 shpchp wmi hed acpi_cpufreq lp parport tcp_diag inet_diag ipmi_si ipmi_devintf ipmi_msghandler sch_fq_codel igb ptp pps_core i2c_algo_bit i2c_core crc32c_intel isci libsas scsi_transport_sas dca ipv6 CPU: 0 PID: 705 Comm: kworker/u240:0 Not tainted 4.1.35.el7.x86_64 #1 Hardware name: WIWYNN Lyra/JD/S2600GZ, BIOS SE5C600.86B.02.03.2004.030620151456 03/06/2015 Workqueue: scsi_wq_6 sas_destruct_devices [libsas] 0000000000000000 ffff88056c393ba8 ffffffff81544a6d ffff88056c393bf8 0000000000000009 ffff88056c393be8 ffffffff81069b4c ffff88081790d078 ffffffff811dad37 0000000000000000 ffffffff81ab7670 ffff88081b29dc10 Call Trace: [] dump_stack+0x4d/0x63 [] warn_slowpath_common+0xa1/0xbb [] ? sysfs_remove_group+0x54/0x88 [] warn_slowpath_fmt+0x46/0x48 [] ? kernfs_find_and_get_ns+0x4d/0x58 [] sysfs_remove_group+0x54/0x88 [] dpm_sysfs_remove+0x50/0x55 [] device_del+0x47/0x1ec [] ? mutex_unlock+0x16/0x18 [] device_unregister+0x48/0x54 [] bsg_unregister_queue+0x5f/0x86 [] __scsi_remove_device+0x3a/0xc3 [] scsi_remove_device+0x26/0x33 [] scsi_remove_target+0x134/0x19b [] sas_rphy_remove+0x2c/0x72 [scsi_transport_sas] [] sas_rphy_delete+0x13/0x1f [scsi_transport_sas] [] sas_destruct_devices+0x58/0x79 [libsas] [] process_one_work+0x19b/0x2d1 [] worker_thread+0x1dd/0x2bb [] ? cancel_delayed_work+0x72/0x72 [] kthread+0xa5/0xad [] ? task_work_add+0xd/0x53 [] ? __kthread_parkme+0x61/0x61 [] ret_from_fork+0x42/0x70 [] ? __kthread_parkme+0x61/0x61 It looks like we don't wait for sas destruct work properly on tear down path, at least sas_deform_port() calls sas_unregister_domain_devices() to schedule destruct work to a workqueue and then calls sas_port_delete() to remove the related sysfs files concurrently. Dan tried to fix this with a different way: https://patchwork.kernel.org/patch/6450921/ but that patch is never applied. I take a better approach as suggested by Johannes, that is waiting for pending destruct work to remove child sysfs files and then removing the parent sysfs files. Cc: Dan Williams Cc: Johannes Thumshirn Cc: Praveen Murali Cc: "James E.J. Bottomley" Cc: "Martin K. Petersen" Cc: linux-scsi@vger.kernel.org Signed-off-by: Cong Wang --- drivers/scsi/libsas/sas_discover.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/libsas/sas_discover.c b/drivers/scsi/libsas/sas_discover.c index 60de66252fa2..27c11fc7aa2b 100644 --- a/drivers/scsi/libsas/sas_discover.c +++ b/drivers/scsi/libsas/sas_discover.c @@ -388,6 +388,11 @@ void sas_unregister_dev(struct asd_sas_port *port, struct domain_device *dev) } } +static void sas_flush_work(struct asd_sas_port *port) +{ + scsi_flush_work(port->ha->core.shost); +} + void sas_unregister_domain_devices(struct asd_sas_port *port, int gone) { struct domain_device *dev, *n; @@ -401,8 +406,8 @@ void sas_unregister_domain_devices(struct asd_sas_port *port, int gone) list_for_each_entry_safe(dev, n, &port->disco_list, disco_list_node) sas_unregister_dev(port, dev); + sas_flush_work(port); port->port->rphy = NULL; - } void sas_device_set_phy(struct domain_device *dev, struct sas_port *port)