3.9-rc1: pciehp and eSATA card SiI 3132, no XHCI

[re-sending without attachment]
[again, but only to linux-pci, as it did not make it into it (shortened a bit the leading old text)]

Yijing Wang wrote:
> On 2013/3/14 8:05, Martin Mokrejs wrote:
>> Hi Yjing,
>>

>> +11:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev ff) (prog-if ff)
>> +       !!! Unknown header type 7f
>> +00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +10: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +20: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +30: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +40: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +50: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +60: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +70: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +80: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +90: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +a0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +b0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +c0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>> +f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>  
> 
> This is so strange, seems like 11:00.0 Mass storage controller device is still in OS, but the real hardware is removedï¼?OS
> can not access device configure space, so all config register return ff.
> 
> But why the device become stale after ejectï¼? Mass storage controller driver should unregister and release itself.

I think the answer is in the PME# messages. In this particular situation
they are flipped from enabled to disabled and back in a never ending loop.
I do not want to try what happens hen I drop pcie_aspm=off from the comandline
yet ;) but somewhat fear things could start working and PME# messages would
disappear. ;-)

> 
>>
>> Please note the funny broken:
>>
>> +11:00.0 Mass storage controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev ff) (prog-if ff)
>> +       !!! Unknown header type 7f
>> +00: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>
>> Who is still thinking the card is in the system? Didn't we agree above that PresDet
>> worked fine in this particular setup?
>>
>> The rmmod sata_sil24 causes that in dmesg is every second repeatedly this, until forever:
>>
>> --- dmesg_ejected.txt   2013-03-14 00:25:59.000000000 +0100
>> +++ dmesg_ejected_and_reinserted_rmmod_sata_sil24.txt   2013-03-14 00:29:47.000000000 +0100
>> @@ -819,3 +819,277 @@
>>  [   37.426390] r8169 0000:05:00.0 eth0: link up
>>  [   38.686210] r8169 0000:05:00.0 eth0: link down
>>  [   42.551461] r8169 0000:05:00.0 eth0: link up
>> +[  432.686232] sata_sil24: IRQ status == 0xffffffff, PCI fault or device removal?
>> +[  432.695763] pcieport 0000:00:1c.7: PME# enabled
>> +[  432.956104] pcieport 0000:00:1c.7: PME# disabled
>> +[  432.965896] pcieport 0000:00:1c.7: PME# enabled
>> +[  433.006119] pcieport 0000:00:1c.7: PME# disabled
>> +[  433.016047] pcieport 0000:00:1c.7: PME# enabled
>> +[  434.037451] pcieport 0000:00:1c.7: PME# disabled
>> +[  434.047152] pcieport 0000:00:1c.7: PME# enabled
>> +[  434.087516] pcieport 0000:00:1c.7: PME# disabled
>> [cut]
>>
>>
>>
>> I should have tried below to remove 11.0 instead of 1.c7 but it demonstrates
>> that one can get rid of the partial eSATA card entry from lspci output:
>>
>> # echo 1 > /sys/bus/pci/devices/0000\:00\:1c.7/remove
>>
>> That stops the PME# storm:
>>
>> +[  611.843285] pcieport 0000:00:1c.7: PME# disabled
>> +[  611.853086] pcieport 0000:00:1c.7: PME# enabled
>> +[  611.893364] pcieport 0000:00:1c.7: PME# disabled
>> +[  611.903283] pcieport 0000:00:1c.7: PME# enabled
>> +[  612.183893] pci 0000:11:00.0: PME# disabled
>> +[  612.184789] pcieport 0000:00:1c.7: PME# disabled
>> +[  612.203753] pcieport 0000:00:1c.7: PME# disabled
>> +[  612.205521] pci_bus 0000:11: busn_res: [bus 11-16] is released
>>
>> and releases the iomem's:
>>
>> # diff -u -w iomem_ejected_and_reinserted_rmmod_sata_sil24.txt iomem_ejected_and_reinserted_rmmod_sata_sil24_remove_1c.7.txt 
>> --- iomem_ejected_and_reinserted_rmmod_sata_sil24.txt   2013-03-14 00:29:02.000000000 +0100
>> +++ iomem_ejected_and_reinserted_rmmod_sata_sil24_remove_1c.7.txt       2013-03-14 00:32:48.000000000 +0100
>> @@ -34,17 +34,12 @@
>>  dfa00000-feafffff : PCI Bus 0000:00
>>    dfa00000-dfa00fff : pnp 00:0a
>>    e0000000-efffffff : 0000:00:02.0
>> -  f0000000-f10fffff : PCI Bus 0000:11
>>    f1100000-f11fffff : PCI Bus 0000:05
>>      f1100000-f1103fff : 0000:05:00.0
>>        f1100000-f1103fff : r8169
>>      f1104000-f1104fff : 0000:05:00.0
>>        f1104000-f1104fff : r8169
>>    f6800000-f6bfffff : 0000:00:02.0
>> -  f6c00000-f7cfffff : PCI Bus 0000:11
>> -    f6c00000-f6c7ffff : 0000:11:00.0
>> -    f6c80000-f6c83fff : 0000:11:00.0
>> -    f6c84000-f6c8407f : 0000:11:00.0
>>    f7d00000-f7dfffff : PCI Bus 0000:0b
>>      f7d00000-f7d0ffff : 0000:0b:00.0
>>      f7d10000-f7d11fff : 0000:0b:00.0
>> #
>>
>> The accompanying diff in lspci:
>>
>> # diff -u -w lspci_vvvxxx_ejected_and_reinserted_rmmod_sata_sil24.txt lspci_vvvxxx_ejected_and_reinserted_rmmod_sata_sil24_remove_1c.7.txt 
>> --- lspci_vvvxxx_ejected_and_reinserted_rmmod_sata_sil24.txt    2013-03-14 00:30:06.000000000 +0100
>> +++ lspci_vvvxxx_ejected_and_reinserted_rmmod_sata_sil24_remove_1c.7.txt        2013-03-14 00:32:20.000000000 +0100
>> @@ -429,69 +429,6 @@
>>  e0: 00 3f 00 00 00 00 00 00 01 00 00 00 00 00 00 00
>>  f0: 00 00 00 00 00 00 00 00 87 0f 05 08 00 00 00 00
>>  
> 
> After you remove 00:1c.7 PCI bridge device. The 11:00.0 Mass storage controller device  still shown in lspci?

No. I thought I showed that in the diffs clearly. Please check again. ;-)
A note: It was in the diff output immediately below this place where you
wrote the question. ;))

> If 11:00.0 Mass storage controller device was removed now. you can try to echo 1 > /sys/bus/pci/rescan, rescan the 00:1c.7 PCI bridge
> device and its child devices.
> Then try to insert the card.

Will do below.

> 
> I use pciehp module in my machine and add pciehp_debug=1ï¼?debug info will print while I doing eject or insert.
> I don't know why pciehp cannot print debug info in your system, maybe you can try to build it as module.
> We can only trace eject action now, if eject action cannot complete normally, hot insert won't success.

Hmm, I did the following to my kernel commandline to be sure:

-pciehp.pciehp_debug=1 slub_debug=AFPZ pcie_aspm=off
+pciehp.pciehp_debug=1 pciehp_debug=1 slub_debug=AFPZ pcie_aspm=off

So now the test what happens if I remove 1c.7 and do a bus rescan. I can tell you
in advance the hotplugged card was not picked up until I did a second pci rescan.
But please check the diffs below. I tried to say it short but omit many details
which are out of my knowledge. ;)

The eject of a coldplugged card. Note that PresDet on the Changed: line is not set
to PresDet+ or is it because it was overwritten too early (as you said elsewhere)?

#

The slot status changes were from coldboot with card loaded:

11:18:26 SlotStatus 0040
11:19:01 SlotStatus 0040

eject 

11:19:02 SlotStatus 0100
11:20:46 SlotStatus 0100

removed 1.c7, rescan

11:21:19 SlotStatus 0100

hot insert

11:23:22 SlotStatus 0100
11:23:23 SlotStatus 0140

Martin

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Message ID	51420EBA.20505@fold.natur.cuni.cz (mailing list archive)
State	New, archived
Delegated to:	Bjorn Helgaas
Headers	show Return-Path: <linux-pci-owner@vger.kernel.org> X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 7258BDFB79 for <patchwork-linux-pci@patchwork.kernel.org>; Thu, 14 Mar 2013 17:54:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965017Ab3CNRyK (ORCPT <rfc822;patchwork-linux-pci@patchwork.kernel.org>); Thu, 14 Mar 2013 13:54:10 -0400 Received: from fold.natur.cuni.cz ([195.113.57.32]:47921 "HELO fold.natur.cuni.cz" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S964889Ab3CNRyG (ORCPT <rfc822; linux-pci@vger.kernel.org>); Thu, 14 Mar 2013 13:54:06 -0400 Received: (qmail 8592 invoked from network); 14 Mar 2013 17:54:02 -0000 Received: from unknown (HELO ?192.168.251.6?) (192.168.251.6) by 192.168.251.1 with SMTP; 14 Mar 2013 17:54:02 -0000 Message-ID: <51420EBA.20505@fold.natur.cuni.cz> Date: Thu, 14 Mar 2013 18:54:02 +0100 From: Martin Mokrejs <mmokrejs@fold.natur.cuni.cz> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20100101 Firefox/19.0 SeaMonkey/2.16 MIME-Version: 1.0 To: "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org> Subject: Re: 3.9-rc1: pciehp and eSATA card SiI 3132, no XHCI References: <513E7E1E.80508@fold.natur.cuni.cz> <513FE7AD.2020408@huawei.com> <5141145E.4020300@fold.natur.cuni.cz> <51417C28.40402@huawei.com> In-Reply-To: <51417C28.40402@huawei.com> X-Enigmail-Version: 1.5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: <linux-pci.vger.kernel.org> X-Mailing-List: linux-pci@vger.kernel.org

3.9-rc1: pciehp and eSATA card SiI 3132, no XHCI

Commit Message

Patch