| Message ID | 20231126025612.12522-1-wahrenst@gmx.net (mailing list archive) |
|---|---|
| Headers | show |
| Series | ARM: dts: bcm2711-rpi-cm4-io: Enable xHCI host | expand |
Hi Stefan, Stefan Wahren <wahrenst@gmx.net> (2023-11-26): > In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board > does not have a VL805 USB 3.0 host controller, which is connected via > PCIe. Instead, the BCM2711 on the Compute Module provides the built-in > xHCI. > > Stefan Wahren (3): > dt-bindings: usb: xhci: Add optional power-domains > ARM: dts: bcm2711: Add generic xHCI > ARM: dts: bcm2711-rpi-cm4-io: Enable xHCI host > > Documentation/devicetree/bindings/usb/generic-xhci.yaml | 3 +++ > arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts | 9 ++++++++- > arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 5 +++++ > arch/arm/boot/dts/broadcom/bcm2711.dtsi | 9 +++++++++ > 4 files changed, 25 insertions(+), 1 deletion(-) I've tried applying this series on top of v6.6-15365-g305230142ae0 (the base commit for Jim's v8 patch series regarding PCIe/clkreq[1]), since I know the unpatched kernel is working fine with a CM4 Lite if there's no PCIe hardware plugged in. 1. https://lore.kernel.org/all/20231113185607.1756-1-james.quinlan@broadcom.com/ With those patches applied, on the following hardware setup: - CM4 Lite (and SD card) - CM4 IO Board - Ethernet - HDMI - Serial console (But neither USB storage or USB keyboard.) I'm seeing various failure modes, but basically the system no longer boots. I'm a little short on time right now, but if the following excerpts aren't sufficient I can adjust logging to capture a full trace for one or more of those. This seems like a showstopper on its own, but if you'd like me to try with an eMMC-equipped CM4, I can do that as well. Loading, please wait... Starting systemd-udevd version 252.17-1~deb12u1 [ 3.009941] usb_phy_generic phy: dummy supplies not allowed for exclusive requests [ 3.019538] usbcore: registered new interface driver usbfs [ 3.025266] usbcore: registered new interface driver hub [ 3.025695] sdhci: Secure Digital Host Controller Interface driver [ 3.030748] usbcore: registered new device driver usb [ 3.036984] sdhci: Copyright(c) Pierre Ossman [ 3.057911] sdhci-pltfm: SDHCI platform and OF driver helper [ 3.067167] sdhci-iproc fe300000.mmc: allocated mmc-pwrseq [ 3.082634] xhci-hcd fe9c0000.usb: xHCI Host Controller [ 3.088051] xhci-hcd fe9c0000.usb: new USB bus registered, assigned bus number 1 [ 3.112118] mmc1: SDHCI controller on fe300000.mmc [fe300000.mmc] using PIO [ 3.119262] mmc0: SDHCI controller on fe340000.mmc [fe340000.mmc] using ADMA [ 24.084102] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [ 24.090107] rcu: 2-...0: (1 ticks this GP) idle=125c/1/0x4000000000000000 softirq=389/393 fqs=2454 [ 24.099283] rcu: (detected by 3, t=5252 jiffies, g=-607, q=619 ncpus=4) [ 24.106077] Sending NMI from CPU 3 to CPUs 2: [ 24.220101] watchdog: Watchdog detected hard LOCKUP on cpu 1 [ 24.225837] Modules linked in: mdio_bcm_unimac sdhci_iproc of_mdio crct10dif_ce sdhci_pltfm crct10dif_common fixed_phy xhci_plat_hcd(+) reset_raspberrypi fwnode_mdio xhci_hcd libphy i2c_bcm2835 usbcore gpio_regulator phy_generic usb_common sdhci fixed [ 34.111290] rcu: rcu_sched kthread timer wakeup didn't happen for 2504 jiffies! g-607 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 34.122579] rcu: Possible timer handling issue on cpu=3 timer-softirq=2120 [ 34.129634] rcu: rcu_sched kthread starved for 2510 jiffies! g-607 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=3 [ 34.139952] rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior. [ 34.149034] rcu: RCU grace-period kthread stack dump: [ 34.154150] task:rcu_sched state:I stack:0 pid:16 tgid:16 ppid:2 flags:0x00000008 [ 34.163591] Call trace: [ 34.166063] __switch_to+0xe8/0x130 [ 34.169607] __schedule+0x398/0xd48 [ 34.173140] schedule+0x30/0xf0 [ 34.176321] schedule_timeout+0xa4/0x188 [ 34.180294] rcu_gp_fqs_loop+0x128/0x488 [ 34.184268] rcu_gp_kthread+0x134/0x188 [ 34.188151] kthread+0xec/0xf8 [ 34.191250] ret_from_fork+0x10/0x20 [ 34.194874] rcu: Stack dump where RCU GP kthread last ran: [ 34.200431] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.6.0+ #1 [ 34.206430] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT) [ 34.212956] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 34.220013] pc : default_idle_call+0x54/0x100 [ 34.224427] lr : default_idle_call+0x40/0x100 [ 34.228841] sp : ffff8000800c3df0 [ 34.232193] x29: ffff8000800c3df0 x28: 0607f6804a328bb4 x27: 4f98c8ce00000000 [ 34.239431] x26: f01df0b7242a8fc6 x25: ffff0e8882a55140 x24: 0000000000000000 [ 34.246668] x23: 0000000000000000 x22: ffff0e8882a55140 x21: ffffcb7db4cdeb38 [ 34.253904] x20: 0000000000000003 x19: ffffcb7db477b008 x18: 0000000000000000 [ 34.261140] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 34.268376] x14: ffffffffffffffff x13: 0000000000000444 x12: ffffcb7db4cdeb40 [ 34.275613] x11: 0000000000000001 x10: 0000000000000bb0 x9 : ffffcb7db2f88cf0 [ 34.282848] x8 : ffff0e8882a55d50 x7 : 0000000000000000 x6 : 000000004f24a986 [ 34.290085] x5 : 4000000000000000 x4 : ffff430b02a2a000 x3 : ffff430b02a2a000 [ 34.297321] x2 : 0000000000000001 x1 : ffff430b02a2a000 x0 : ffffcb7db477b008 [ 34.304557] Call trace: [ 34.307029] default_idle_call+0x54/0x100 [ 34.311089] do_idle+0x218/0x278 [ 34.314359] cpu_startup_entry+0x3c/0x50 [ 34.318330] secondary_start_kernel+0x130/0x158 [ 34.322926] __secondary_switched+0xb8/0xc0 or: Loading, please wait... Starting systemd-udevd version 252.17-1~deb12u1 [ 2.994822] usb_phy_generic phy: dummy supplies not allowed for exclusive requests [ 3.003002] sdhci: Secure Digital Host Controller Interface driver [ 3.009319] sdhci: Copyright(c) Pierre Ossman [ 3.016194] usbcore: registered new interface driver usbfs [ 3.021950] usbcore: registered new interface driver hub [ 3.027475] usbcore: registered new device driver usb [ 3.028573] sdhci-pltfm: SDHCI platform and OF driver helper [ 3.048856] sdhci-iproc fe300000.mmc: allocated mmc-pwrseq [ 3.071825] xhci-hcd fe9c0000.usb: xHCI Host Controller [ 3.077211] xhci-hcd fe9c0000.usb: new USB bus registered, assigned bus number 1 [ 3.107780] mmc0: SDHCI controller on fe300000.mmc [fe300000.mmc] using PIO [ 3.112158] mmc1: SDHCI controller on fe340000.mmc [fe340000.mmc] using ADMA [ 13.192071] mmc0: Timeout waiting for hardware cmd interrupt. [ 13.197899] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 13.204426] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00009902 [ 13.210953] mmc0: sdhci: Blk size: 0x00000000 | Blk cnt: 0x00000000 [ 13.217479] mmc0: sdhci: Argument: 0x00000c00 | Trn mode: 0x00000000 [ 13.224007] mmc0: sdhci: Present: 0x01ff0001 | Host ctl: 0x00000001 [ 13.230532] mmc0: sdhci: Power: 0x0000000f | Blk gap: 0x00000000 [ 13.237059] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00003947 [ 13.243585] mmc0: sdhci: Timeout: 0x00000000 | Int stat: 0x00018000 [ 13.250112] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00ff0003 [ 13.256638] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 13.263163] mmc0: sdhci: Caps: 0x00000000 | Caps_1: 0x00000000 [ 13.269690] mmc0: sdhci: Cmd: 0x0000341a | Max curr: 0x00000001 [ 13.276215] mmc0: sdhci: Resp[0]: 0x00000000 | Resp[1]: 0x00000000 [ 13.282741] mmc0: sdhci: Resp[2]: 0x00000000 | Resp[3]: 0x00000000 [ 13.289267] mmc0: sdhci: Host ctl2: 0x00000000 [ 13.293766] mmc0: sdhci: ============================================ [ 23.436070] mmc0: Timeout waiting for hardware cmd interrupt. [ 23.441896] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 23.448423] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00009902 [ 23.454949] mmc0: sdhci: Blk size: 0x00000000 | Blk cnt: 0x00000000 [ 23.461476] mmc0: sdhci: Argument: 0x80000c08 | Trn mode: 0x00000000 [ 23.468002] mmc0: sdhci: Present: 0x01ff0001 | Host ctl: 0x00000001 [ 23.474528] mmc0: sdhci: Power: 0x0000000f | Blk gap: 0x00000000 [ 23.481054] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00003947 [ 23.487579] mmc0: sdhci: Timeout: 0x00000000 | Int stat: 0x00018000 [ 23.494106] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00ff0003 [ 23.500632] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 23.507158] mmc0: sdhci: Caps: 0x00000000 | Caps_1: 0x00000000 [ 23.513684] mmc0: sdhci: Cmd: 0x0000341a | Max curr: 0x00000001 [ 23.520209] mmc0: sdhci: Resp[0]: 0x00000000 | Resp[1]: 0x00000000 [ 23.526736] mmc0: sdhci: Resp[2]: 0x00000000 | Resp[3]: 0x00000000 [ 23.533262] mmc0: sdhci: Host ctl2: 0x00000000 [ 23.537760] mmc0: sdhci: ============================================ [ 24.100068] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [ 24.106072] rcu: 1-...!: (0 ticks this GP) idle=177c/1/0x4000000000000000 softirq=266/266 fqs=0 [ 24.114984] rcu: 3-...!: (0 ticks this GP) idle=1094/1/0x4000000000000000 softirq=163/163 fqs=0 [ 24.123894] rcu: (detected by 2, t=5254 jiffies, g=-579, q=440 ncpus=4) [ 24.130689] Sending NMI from CPU 2 to CPUs 1: [ 34.135902] Sending NMI from CPU 2 to CPUs 3: [ 44.141138] rcu: rcu_sched kthread timer wakeup didn't happen for 5253 jiffies! g-579 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 44.152427] rcu: Possible timer handling issue on cpu=3 timer-softirq=87 [ 44.159306] rcu: rcu_sched kthread starved for 5254 jiffies! g-579 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=3 [ 44.169624] rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior. [ 44.178705] rcu: RCU grace-period kthread stack dump: [ 44.183820] task:rcu_sched state:I stack:0 pid:16 tgid:16 ppid:2 flags:0x00000008 [ 44.193264] Call trace: [ 44.195737] __switch_to+0xe8/0x130 [ 44.199280] __schedule+0x398/0xd48 [ 44.202813] schedule+0x30/0xf0 [ 44.205994] schedule_timeout+0xa4/0x188 [ 44.209967] rcu_gp_fqs_loop+0x128/0x488 [ 44.213941] rcu_gp_kthread+0x134/0x188 [ 44.217824] kthread+0xec/0xf8 [ 44.220921] ret_from_fork+0x10/0x20 [ 44.224559] watchdog: Watchdog detected hard LOCKUP on cpu 3 [ 44.230292] Modules linked in: crct10dif_ce reset_raspberrypi crct10dif_common fixed_phy sdhci_iproc xhci_plat_hcd(+) xhci_hcd fwnode_mdio i2c_bcm2835 sdhci_pltfm usbcore libphy sdhci usb_common phy_generic fixed gpio_regulator [ 44.250781] Sending NMI from CPU 2 to CPUs 3: [ 54.264077] mmc0: Timeout waiting for hardware cmd interrupt. [ 54.269900] mmc0: sdhci: ============ SDHCI REGISTER DUMP =========== [ 54.276426] mmc0: sdhci: Sys addr: 0x00000000 | Version: 0x00009902 [ 54.282953] mmc0: sdhci: Blk size: 0x00000000 | Blk cnt: 0x00000000 [ 54.289480] mmc0: sdhci: Argument: 0x00000000 | Trn mode: 0x00000000 [ 54.296007] mmc0: sdhci: Present: 0x01ff0000 | Host ctl: 0x00000001 [ 54.302532] mmc0: sdhci: Power: 0x0000000f | Blk gap: 0x00000000 [ 54.309059] mmc0: sdhci: Wake-up: 0x00000000 | Clock: 0x00003947 [ 54.315585] mmc0: sdhci: Timeout: 0x00000000 | Int stat: 0x00018001 [ 54.322111] mmc0: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00ff0003 [ 54.328637] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 54.335162] mmc0: sdhci: Caps: 0x00000000 | Caps_1: 0x00000000 [ 54.341689] mmc0: sdhci: Cmd: 0x00000000 | Max curr: 0x00000001 [ 54.348215] mmc0: sdhci: Resp[0]: 0x00000000 | Resp[1]: 0x00000000 [ 54.354741] mmc0: sdhci: Resp[2]: 0x00000000 | Resp[3]: 0x00000000 [ 54.361267] mmc0: sdhci: Host ctl2: 0x00000000 [ 54.365766] mmc0: sdhci: ============================================ or: Loading, please wait... Starting systemd-udevd version 252.17-1~deb12u1 [ 2.973041] usb_phy_generic phy: dummy supplies not allowed for exclusive requests [ 2.981351] sdhci: Secure Digital Host Controller Interface driver [ 2.987775] sdhci: Copyright(c) Pierre Ossman [ 3.000568] sdhci-pltfm: SDHCI platform and OF driver helper [ 3.021527] sdhci-iproc fe300000.mmc: allocated mmc-pwrseq [ 3.030308] usbcore: registered new interface driver usbfs [ 3.036004] usbcore: registered new interface driver hub [ 3.041518] usbcore: registered new device driver usb [ 3.059120] mmc0: SDHCI controller on fe340000.mmc [fe340000.mmc] using ADMA [ 3.066296] mmc1: SDHCI controller on fe300000.mmc [fe300000.mmc] using PIO [ 3.081846] bcmgenet fd580000.ethernet: GENET 5.0 EPHY: 0x0000 [ 3.118407] xhci-hcd fe9c0000.usb: xHCI Host Controller [ 3.123745] xhci-hcd fe9c0000.usb: new USB bus registered, assigned bus number 1 [ 4.172083] ------------[ cut here ]------------ [ 4.176768] Firmware transaction timeout [ 4.176825] WARNING: CPU: 2 PID: 143 at drivers/firmware/raspberrypi.c:64 rpi_firmware_property_list+0x250/0x270 [ 4.191121] Modules linked in: xhci_plat_hcd(+) xhci_hcd genet(+) mdio_bcm_unimac crct10dif_ce reset_raspberrypi crct10dif_common of_mdio usbcore i2c_bcm2835 sdhci_iproc fixed_phy fwnode_mdio usb_common sdhci_pltfm libphy fixed gpio_regulator sdhci phy_generic [ 4.214527] CPU: 2 PID: 143 Comm: kworker/2:2 Not tainted 6.6.0+ #1 [ 4.220880] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT) [ 4.227407] Workqueue: events_freezable mmc_rescan [ 4.232266] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 4.239323] pc : rpi_firmware_property_list+0x250/0x270 [ 4.244618] lr : rpi_firmware_property_list+0x250/0x270 [ 4.249913] sp : ffff80008024ba70 [ 4.253265] x29: ffff80008024ba70 x28: ffff4307837b9428 x27: ffff430782b56440 [ 4.260503] x26: 0000000000000009 x25: ffff80008135d008 x24: 0000000000001000 [ 4.267739] x23: ffff430784142860 x22: ffffd0cee050ba68 x21: 0000000000000014 [ 4.274976] x20: ffff430782b56400 x19: ffff80008135d000 x18: 0000000000000006 [ 4.282212] x17: 0000000000000020 x16: 0000000000000002 x15: ffff80008024b450 [ 4.289448] x14: 0000000000000000 x13: 74756f656d697420 x12: 6e6f69746361736e [ 4.296684] x11: 00000000ffffefff x10: ffffd0cee035ee20 x9 : ffffd0cede52fc78 [ 4.303921] x8 : 0000000000017fe8 x7 : c0000000ffffefff x6 : 0000000000057fa8 [ 4.311157] x5 : 0000000000000fff x4 : 0000000000000000 x3 : 0000000000000000 [ 4.318393] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff430784188000 [ 4.325629] Call trace: [ 4.328102] rpi_firmware_property_list+0x250/0x270 [ 4.333044] rpi_firmware_property+0x8c/0xe0 [ 4.337369] rpi_exp_gpio_set+0x60/0xc0 [ 4.341260] gpio_chip_set_multiple+0x5c/0xa8 [ 4.345675] gpiod_set_array_value_complex+0x388/0x4b8 [ 4.350884] gpiod_set_array_value_cansleep+0x34/0x50 [ 4.356003] mmc_pwrseq_simple_set_gpios_value.isra.0+0x74/0xa8 [ 4.362006] mmc_pwrseq_simple_post_power_on+0x28/0x60 [ 4.367213] mmc_pwrseq_post_power_on+0x2c/0x48 [ 4.371802] mmc_power_up.part.0+0x8c/0x170 [ 4.376038] mmc_rescan+0x19c/0x360 [ 4.379570] process_one_work+0x174/0x3b0 [ 4.383632] worker_thread+0x230/0x458 [ 4.387427] kthread+0xec/0xf8 [ 4.390523] ret_from_fork+0x10/0x20 [ 4.394146] ---[ end trace 0000000000000000 ]--- [ 4.398838] raspberrypi-exp-gpio soc:firmware:gpio: Failed to set GPIO 1 state (-110 0) [ 4.398838] raspberrypi-exp-gpio soc:firmware:gpio: Failed to set GPIO 1 state (-110 0) [ 14.476073] mmc1: Timeout waiting for hardware cmd interrupt. [ 14.481897] mmc1: sdhci: ============ SDHCI REGISTER DUMP =========== [ 14.488424] mmc1: sdhci: Sys addr: 0x00000000 | Version: 0x00009902 [ 14.494952] mmc1: sdhci: Blk size: 0x00000000 | Blk cnt: 0x00000000 [ 14.501478] mmc1: sdhci: Argument: 0x00000c00 | Trn mode: 0x00000000 [ 14.508005] mmc1: sdhci: Present: 0x01ff0001 | Host ctl: 0x00000001 [ 14.514532] mmc1: sdhci: Power: 0x0000000f | Blk gap: 0x00000000 [ 14.521058] mmc1: sdhci: Wake-up: 0x00000000 | Clock: 0x0000a147 [ 14.527585] mmc1: sdhci: Timeout: 0x00000000 | Int stat: 0x00018000 [ 14.534111] mmc1: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00ff0003 [ 14.540636] mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 14.547162] mmc1: sdhci: Caps: 0x00000000 | Caps_1: 0x00000000 [ 14.553689] mmc1: sdhci: Cmd: 0x0000341a | Max curr: 0x00000001 [ 14.560215] mmc1: sdhci: Resp[0]: 0x00000000 | Resp[1]: 0x00000000 [ 14.566741] mmc1: sdhci: Resp[2]: 0x00000000 | Resp[3]: 0x00000000 [ 14.573268] mmc1: sdhci: Host ctl2: 0x00000000 [ 14.577767] mmc1: sdhci: ============================================ [ 24.132072] rcu: INFO: rcu_sched detected stalls on CPUs/tasks: [ 24.138075] rcu: 0-...0: (1 ticks this GP) idle=1574/1/0x4000000000000000 softirq=245/245 fqs=1050 [ 24.147252] rcu: 3-...0: (0 ticks this GP) idle=0e04/1/0x4000000000000000 softirq=167/170 fqs=1050 [ 24.156426] rcu: (detected by 1, t=5259 jiffies, g=-587, q=243 ncpus=4) [ 24.163220] Sending NMI from CPU 1 to CPUs 0: [ 24.224071] watchdog: Watchdog detected hard LOCKUP on cpu 3 [ 24.229806] Modules linked in: xhci_plat_hcd(+) xhci_hcd genet(+) mdio_bcm_unimac crct10dif_ce reset_raspberrypi crct10dif_common of_mdio usbcore i2c_bcm2835 sdhci_iproc fixed_phy fwnode_mdio usb_common sdhci_pltfm libphy fixed gpio_regulator sdhci phy_generic [ 24.704075] mmc1: Timeout waiting for hardware cmd interrupt. [ 24.709897] mmc1: sdhci: ============ SDHCI REGISTER DUMP =========== [ 24.716423] mmc1: sdhci: Sys addr: 0x00000000 | Version: 0x00009902 [ 24.722950] mmc1: sdhci: Blk size: 0x00000000 | Blk cnt: 0x00000000 [ 24.729476] mmc1: sdhci: Argument: 0x80000c08 | Trn mode: 0x00000000 [ 24.736002] mmc1: sdhci: Present: 0x01ff0001 | Host ctl: 0x00000001 [ 24.742528] mmc1: sdhci: Power: 0x0000000f | Blk gap: 0x00000000 [ 24.749053] mmc1: sdhci: Wake-up: 0x00000000 | Clock: 0x0000a147 [ 24.755579] mmc1: sdhci: Timeout: 0x00000000 | Int stat: 0x00018000 [ 24.762105] mmc1: sdhci: Int enab: 0x00ff0003 | Sig enab: 0x00ff0003 [ 24.768631] mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001 [ 24.775157] mmc1: sdhci: Caps: 0x00000000 | Caps_1: 0x00000000 [ 24.781682] mmc1: sdhci: Cmd: 0x0000341a | Max curr: 0x00000001 [ 24.788208] mmc1: sdhci: Resp[0]: 0x00000000 | Resp[1]: 0x00000000 [ 24.794734] mmc1: sdhci: Resp[2]: 0x00000000 | Resp[3]: 0x00000000 [ 24.801259] mmc1: sdhci: Host ctl2: 0x00000000 [ 24.805757] mmc1: sdhci: ============================================ [ 34.168434] Sending NMI from CPU 1 to CPUs 3: [ 44.173983] rcu: rcu_sched kthread timer wakeup didn't happen for 5013 jiffies! g-587 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 44.185271] rcu: Possible timer handling issue on cpu=1 timer-softirq=1166 [ 44.192326] rcu: rcu_sched kthread starved for 5019 jiffies! g-587 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1 [ 44.202645] rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior. [ 44.211726] rcu: RCU grace-period kthread stack dump: [ 44.216842] task:rcu_sched state:I stack:0 pid:16 tgid:16 ppid:2 flags:0x00000008 [ 44.226283] Call trace: [ 44.228755] __switch_to+0xe8/0x130 [ 44.232296] __schedule+0x398/0xd48 [ 44.235829] schedule+0x30/0xf0 [ 44.239010] schedule_timeout+0xa4/0x188 [ 44.242983] rcu_gp_fqs_loop+0x128/0x488 [ 44.246957] rcu_gp_kthread+0x134/0x188 [ 44.250841] kthread+0xec/0xf8 [ 44.253934] ret_from_fork+0x10/0x20 [ 44.257555] rcu: Stack dump where RCU GP kthread last ran: [ 44.263111] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 6.6.0+ #1 [ 44.270610] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT) [ 44.277135] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 44.284192] pc : default_idle_call+0x54/0x100 [ 44.288606] lr : default_idle_call+0x40/0x100 [ 44.293019] sp : ffff8000800b3df0 [ 44.296373] x29: ffff8000800b3df0 x28: 1c7bdfbf00040000 x27: 2ffd2ffd00000002 [ 44.303610] x26: bd7e9f7e00400000 x25: ffff430782a50000 x24: 0000000000000000 [ 44.310847] x23: 0000000000000000 x22: ffff430782a50000 x21: ffffd0cee02deb38 [ 44.318084] x20: 0000000000000001 x19: ffffd0cedfd7b008 x18: ffff80008007bb58 [ 44.325320] x17: ffff80008007bc20 x16: 00000000c3947ad1 x15: 000000004d0489eb [ 44.332557] x14: 000000001bca148c x13: 26357425424728cd x12: ffffd0cee02deb40 [ 44.339795] x11: 0000000000000001 x10: 0000000000000bb0 x9 : ffffd0cede588cf0 [ 44.347031] x8 : ffff430782a50c10 x7 : 0000000000000000 x6 : 000000004f43c695 [ 44.354268] x5 : 4000000000000000 x4 : ffff7238d73ec000 x3 : ffff7238d73ec000 [ 44.361505] x2 : 0000000000000001 x1 : ffff7238d73ec000 x0 : ffffd0cedfd7b008 [ 44.368741] Call trace: [ 44.371212] default_idle_call+0x54/0x100 [ 44.375273] do_idle+0x218/0x278 [ 44.378541] cpu_startup_entry+0x40/0x50 [ 44.382513] secondary_start_kernel+0x130/0x158 [ 44.387110] __secondary_switched+0xb8/0xc0 Cheers,
On 11/25/23 6:56 PM, Stefan Wahren wrote: > In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board > does not have a VL805 USB 3.0 host controller, which is connected via > PCIe. Instead, the BCM2711 on the Compute Module provides the built-in > xHCI. > Does this work? I maintain this built-in xHCI controller internally. I wasn't aware the Compute Module uses this block. This block is held in reset and needs a bit toggled to get things going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block correct? Justin > Stefan Wahren (3): > dt-bindings: usb: xhci: Add optional power-domains > ARM: dts: bcm2711: Add generic xHCI > ARM: dts: bcm2711-rpi-cm4-io: Enable xHCI host > > Documentation/devicetree/bindings/usb/generic-xhci.yaml | 3 +++ > arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts | 9 ++++++++- > arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 5 +++++ > arch/arm/boot/dts/broadcom/bcm2711.dtsi | 9 +++++++++ > 4 files changed, 25 insertions(+), 1 deletion(-) > > -- > 2.34.1 >
Hi Cyril, Am 27.11.23 um 01:34 schrieb Cyril Brulebois: > Hi Stefan, > > Stefan Wahren <wahrenst@gmx.net> (2023-11-26): >> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board >> does not have a VL805 USB 3.0 host controller, which is connected via >> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in >> xHCI. >> >> Stefan Wahren (3): >> dt-bindings: usb: xhci: Add optional power-domains >> ARM: dts: bcm2711: Add generic xHCI >> ARM: dts: bcm2711-rpi-cm4-io: Enable xHCI host >> >> Documentation/devicetree/bindings/usb/generic-xhci.yaml | 3 +++ >> arch/arm/boot/dts/broadcom/bcm2711-rpi-cm4-io.dts | 9 ++++++++- >> arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi | 5 +++++ >> arch/arm/boot/dts/broadcom/bcm2711.dtsi | 9 +++++++++ >> 4 files changed, 25 insertions(+), 1 deletion(-) > I've tried applying this series on top of v6.6-15365-g305230142ae0 (the > base commit for Jim's v8 patch series regarding PCIe/clkreq[1]), since I > know the unpatched kernel is working fine with a CM4 Lite if there's no > PCIe hardware plugged in. > > 1. https://lore.kernel.org/all/20231113185607.1756-1-james.quinlan@broadcom.com/ > > With those patches applied, on the following hardware setup: > - CM4 Lite (and SD card) > - CM4 IO Board > - Ethernet > - HDMI > - Serial console > > (But neither USB storage or USB keyboard.) > > I'm seeing various failure modes, but basically the system no longer > boots. I'm a little short on time right now, but if the following > excerpts aren't sufficient I can adjust logging to capture a full > trace for one or more of those. > > This seems like a showstopper on its own, but if you'd like me to try > with an eMMC-equipped CM4, I can do that as well. ... > > [ 3.118407] xhci-hcd fe9c0000.usb: xHCI Host Controller > [ 3.123745] xhci-hcd fe9c0000.usb: new USB bus registered, assigned bus number 1 > [ 4.172083] ------------[ cut here ]------------ > [ 4.176768] Firmware transaction timeout > [ 4.176825] WARNING: CPU: 2 PID: 143 at drivers/firmware/raspberrypi.c:64 rpi_firmware_property_list+0x250/0x270 > [ 4.191121] Modules linked in: xhci_plat_hcd(+) xhci_hcd genet(+) mdio_bcm_unimac crct10dif_ce reset_raspberrypi crct10dif_common of_mdio usbcore i2c_bcm2835 sdhci_iproc fixed_phy fwnode_mdio usb_common sdhci_pltfm libphy fixed gpio_regulator sdhci phy_generic > [ 4.214527] CPU: 2 PID: 143 Comm: kworker/2:2 Not tainted 6.6.0+ #1 > [ 4.220880] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT) thanks for testing. Are you absolutely sure that you are using bcm2711-rpi-cm4-io.dtb from the mainline tree? I would expect the following hardware name: Raspberry Pi Compute Module 4 IO Board Be aware the arm files has been moved into a broadcom subdirectory.
Hi Justin, [add Phil] Am 27.11.23 um 07:02 schrieb Justin Chen: > > > On 11/25/23 6:56 PM, Stefan Wahren wrote: >> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board >> does not have a VL805 USB 3.0 host controller, which is connected via >> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in >> xHCI. >> > > Does this work? I maintain this built-in xHCI controller internally. I > wasn't aware the Compute Module uses this block. i successful tested this with a CM4 (arm 32 bit, multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices (mouse, keyboard) connected to the host interface didn't work. After comparing vendor DTS with mainline i noticed the missing xHCI block [1]. Unfortunately i wasn't able to get further information from the public datasheets. I don't know if the VideoCore does some magic tricks on the xHCI or i missed some downstream xHCI changes. > This block is held in reset and needs a bit toggled to get things > going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block > correct? > > Justin [1] - https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119
On Mon, 27 Nov 2023 at 11:08, Stefan Wahren <wahrenst@gmx.net> wrote: > > Hi Justin, > > [add Phil] > > Am 27.11.23 um 07:02 schrieb Justin Chen: > > > > > > On 11/25/23 6:56 PM, Stefan Wahren wrote: > >> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board > >> does not have a VL805 USB 3.0 host controller, which is connected via > >> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in > >> xHCI. > >> > > > > Does this work? I maintain this built-in xHCI controller internally. I > > wasn't aware the Compute Module uses this block. > i successful tested this with a CM4 (arm 32 bit, > multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices > (mouse, keyboard) connected to the host interface didn't work. After > comparing vendor DTS with mainline i noticed the missing xHCI block [1]. > Unfortunately i wasn't able to get further information from the public > datasheets. I don't know if the VideoCore does some magic tricks on the > xHCI or i missed some downstream xHCI changes. > > > This block is held in reset and needs a bit toggled to get things > > going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block > > correct? > > > > Justin > > [1] - > https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 What's the question here? Does the XHCI block present in the raspberrypi/linux dtsi file really exist? Yes it does. Phil
Hi Phil, Am 27.11.23 um 12:22 schrieb Phil Elwell: > On Mon, 27 Nov 2023 at 11:08, Stefan Wahren <wahrenst@gmx.net> wrote: >> Hi Justin, >> >> [add Phil] >> >> Am 27.11.23 um 07:02 schrieb Justin Chen: >>> >>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board >>>> does not have a VL805 USB 3.0 host controller, which is connected via >>>> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in >>>> xHCI. >>>> >>> Does this work? I maintain this built-in xHCI controller internally. I >>> wasn't aware the Compute Module uses this block. >> i successful tested this with a CM4 (arm 32 bit, >> multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices >> (mouse, keyboard) connected to the host interface didn't work. After >> comparing vendor DTS with mainline i noticed the missing xHCI block [1]. >> Unfortunately i wasn't able to get further information from the public >> datasheets. I don't know if the VideoCore does some magic tricks on the >> xHCI or i missed some downstream xHCI changes. >> >>> This block is held in reset and needs a bit toggled to get things >>> going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block >>> correct? >>> >>> Justin >> [1] - >> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 > What's the question here? Does the XHCI block present in the > raspberrypi/linux dtsi file really exist? Yes it does. since i don't have any documentation about the xHCI block, i assumed the compatible generic-xhci is correct. But Justin seems to suggest that the xHCI block needs some special treatment and we need a specific compatible. Did i missed some xHCI driver changes? Does the VC firmware something to the xHCI especially on CM4? Thanks > > Phil
Hi Stefan, Stefan Wahren <wahrenst@gmx.net> (2023-11-27): > thanks for testing. Are you absolutely sure that you are using > bcm2711-rpi-cm4-io.dtb from the mainline tree? I'm pretty sure, yes. Starting from the unpatched kernel: root@rpi4-20231108:~# md5sum /boot/firmware/bcm2711-rpi-cm4-io.dtb /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb 5cbe07e9f85ddfefd21ffe98bf92f5ea /boot/firmware/bcm2711-rpi-cm4-io.dtb 5cbe07e9f85ddfefd21ffe98bf92f5ea /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb The second file is shipped by the linux-image package built via `make bindeb-pkg`, and sync'd into /boot/firmware as the first one. After deploying the patched kernel, I'm seeing both files getting updated: root@rpi4-20231108:~# md5sum /boot/firmware/bcm2711-rpi-cm4-io.dtb /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb c6ea63f43dcdf8ecd66dda6c494f52e2 /boot/firmware/bcm2711-rpi-cm4-io.dtb c6ea63f43dcdf8ecd66dda6c494f52e2 /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb Comparing a copy of the first set of files against the refreshed DTB, I'm seeing the attached (dt)diff. > I would expect the following hardware name: Raspberry Pi Compute > Module 4 IO Board I suppose this is an arm(32) vs. arm64 difference? - setup_arch() in arch/arm/kernel/setup.c does this: machine_desc = mdesc; machine_name = mdesc->name; dump_stack_set_arch_desc("%s", mdesc->name); - setup_machine_fdt() in arch/arm64/kernel/setup.c does that: name = of_flat_dt_get_machine_name(); if (!name) return; pr_info("Machine model: %s\n", name); dump_stack_set_arch_desc("%s (DT)", name); So I'd guess you're testing on arm(32) and seeing the name embedded in the DTB while I'm testing on arm64 and seeing the name as filled by the bootloader? > Be aware the arm files has been moved into a broadcom subdirectory. Thanks for mentioning that, but I've been working on arm64 exclusively, and those files have always been shipped in that broadcom subdirectory anyway. With 64-bit capable hardware, I didn't think of mentioning I was testing 64-bit kernel and user space (Debian 12, arm64), sorry about that. Cheers,
On Mon, 27 Nov 2023 at 11:44, Stefan Wahren <wahrenst@gmx.net> wrote: > > Hi Phil, > > Am 27.11.23 um 12:22 schrieb Phil Elwell: > > On Mon, 27 Nov 2023 at 11:08, Stefan Wahren <wahrenst@gmx.net> wrote: > >> Hi Justin, > >> > >> [add Phil] > >> > >> Am 27.11.23 um 07:02 schrieb Justin Chen: > >>> > >>> On 11/25/23 6:56 PM, Stefan Wahren wrote: > >>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board > >>>> does not have a VL805 USB 3.0 host controller, which is connected via > >>>> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in > >>>> xHCI. > >>>> > >>> Does this work? I maintain this built-in xHCI controller internally. I > >>> wasn't aware the Compute Module uses this block. > >> i successful tested this with a CM4 (arm 32 bit, > >> multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices > >> (mouse, keyboard) connected to the host interface didn't work. After > >> comparing vendor DTS with mainline i noticed the missing xHCI block [1]. > >> Unfortunately i wasn't able to get further information from the public > >> datasheets. I don't know if the VideoCore does some magic tricks on the > >> xHCI or i missed some downstream xHCI changes. > >> > >>> This block is held in reset and needs a bit toggled to get things > >>> going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block > >>> correct? > >>> > >>> Justin > >> [1] - > >> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 > > What's the question here? Does the XHCI block present in the > > raspberrypi/linux dtsi file really exist? Yes it does. > since i don't have any documentation about the xHCI block, i assumed the > compatible generic-xhci is correct. But Justin seems to suggest that the > xHCI block needs some special treatment and we need a specific compatible. > > Did i missed some xHCI driver changes? > Does the VC firmware something to the xHCI especially on CM4? The firmware switches the on-board USB pins from DWC-OTG to XHCI if otg_mode=1 is set in config.txt, or if booting over USB MSD. This also triggers the enabling of the node with the label or alias "xhci". CM4 is not handled any differently to the other 2711-based devices in this regard. Phil
Hi Phil, >>>> Hi Justin, >>>> >>>> [add Phil] >>>> >>>> Am 27.11.23 um 07:02 schrieb Justin Chen: >>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board >>>>>> does not have a VL805 USB 3.0 host controller, which is connected via >>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in >>>>>> xHCI. >>>>>> >>>>> Does this work? I maintain this built-in xHCI controller internally. I >>>>> wasn't aware the Compute Module uses this block. >>>> i successful tested this with a CM4 (arm 32 bit, >>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices >>>> (mouse, keyboard) connected to the host interface didn't work. After >>>> comparing vendor DTS with mainline i noticed the missing xHCI block [1]. >>>> Unfortunately i wasn't able to get further information from the public >>>> datasheets. I don't know if the VideoCore does some magic tricks on the >>>> xHCI or i missed some downstream xHCI changes. >>>> >>>>> This block is held in reset and needs a bit toggled to get things >>>>> going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block >>>>> correct? >>>>> >>>>> Justin >>>> [1] - >>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 >>> What's the question here? Does the XHCI block present in the >>> raspberrypi/linux dtsi file really exist? Yes it does. >> since i don't have any documentation about the xHCI block, i assumed the >> compatible generic-xhci is correct. But Justin seems to suggest that the >> xHCI block needs some special treatment and we need a specific compatible. >> >> Did i missed some xHCI driver changes? >> Does the VC firmware something to the xHCI especially on CM4? > The firmware switches the on-board USB pins from DWC-OTG to XHCI if > otg_mode=1 is set in config.txt, or if booting over USB MSD. is this pinctrl/pinmux available from ARM via 0x7e200000 or a different IO address? Thanks > This also > triggers the enabling of the node with the label or alias "xhci". > > CM4 is not handled any differently to the other 2711-based devices in > this regard. > > Phil
Hi Cyril, Am 27.11.23 um 12:55 schrieb Cyril Brulebois: > Hi Stefan, > > Stefan Wahren <wahrenst@gmx.net> (2023-11-27): >> thanks for testing. Are you absolutely sure that you are using >> bcm2711-rpi-cm4-io.dtb from the mainline tree? > I'm pretty sure, yes. > > Starting from the unpatched kernel: > > root@rpi4-20231108:~# md5sum /boot/firmware/bcm2711-rpi-cm4-io.dtb /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb > 5cbe07e9f85ddfefd21ffe98bf92f5ea /boot/firmware/bcm2711-rpi-cm4-io.dtb > 5cbe07e9f85ddfefd21ffe98bf92f5ea /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb > > The second file is shipped by the linux-image package built via `make > bindeb-pkg`, and sync'd into /boot/firmware as the first one. > > After deploying the patched kernel, I'm seeing both files getting > updated: > > root@rpi4-20231108:~# md5sum /boot/firmware/bcm2711-rpi-cm4-io.dtb /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb > c6ea63f43dcdf8ecd66dda6c494f52e2 /boot/firmware/bcm2711-rpi-cm4-io.dtb > c6ea63f43dcdf8ecd66dda6c494f52e2 /usr/lib/linux-image-6.6.0+/broadcom/bcm2711-rpi-cm4-io.dtb > > Comparing a copy of the first set of files against the refreshed DTB, > I'm seeing the attached (dt)diff. > >> I would expect the following hardware name: Raspberry Pi Compute >> Module 4 IO Board > I suppose this is an arm(32) vs. arm64 difference? > > - setup_arch() in arch/arm/kernel/setup.c does this: > > machine_desc = mdesc; > machine_name = mdesc->name; > dump_stack_set_arch_desc("%s", mdesc->name); > > - setup_machine_fdt() in arch/arm64/kernel/setup.c does that: > > name = of_flat_dt_get_machine_name(); > if (!name) > return; > > pr_info("Machine model: %s\n", name); > dump_stack_set_arch_desc("%s (DT)", name); > > So I'd guess you're testing on arm(32) and seeing the name embedded in > the DTB while I'm testing on arm64 and seeing the name as filled by the > bootloader? thanks for your fast feedback. Shame on me, i didn't test arm64 yet. Could you please provide the following information: - settings of config.txt - VC firmware version - did you use arm64/defconfig or something special? > >> Be aware the arm files has been moved into a broadcom subdirectory. > Thanks for mentioning that, but I've been working on arm64 exclusively, > and those files have always been shipped in that broadcom subdirectory > anyway. > > With 64-bit capable hardware, I didn't think of mentioning I was testing > 64-bit kernel and user space (Debian 12, arm64), sorry about that. No problem Thanks > > > Cheers,
Stefan Wahren <wahrenst@gmx.net> (2023-11-27): > Could you please provide the following information: > > - settings of config.txt Here you go: arm_64bit=1 enable_uart=1 upstream_kernel=1 kernel=vmlinuz-6.6.0+ initramfs initrd.img-6.6.0+ enable_jtag_gpio=1 force_turbo=1 > - VC firmware version This is pieeprom-2023-01-11.bin I know there's pieeprom-2023-05-11.bin as well, but I've been keeping the former as a constant throughout the PCIe patch series testing once I realized how critical it was (old beta EEPROM versions found in some CM4s made everything much harder initially). In case the config matters: [all] BOOT_UART=0 WAKE_ON_GPIO=1 POWER_OFF_ON_HALT=0 BOOT_ORDER=0xf25641 ENABLE_SELF_UPDATE=1 > - did you use arm64/defconfig or something special? I'm using a “distribution config”, starting from the latest arm64 config found in Debian unstable (6.5.10-1), and applying `make oldconfig`. You'll find it attached. Cheers,
Hi Cyril, Am 27.11.23 um 14:02 schrieb Cyril Brulebois: > Stefan Wahren <wahrenst@gmx.net> (2023-11-27): >> Could you please provide the following information: >> >> - settings of config.txt > Here you go: > > arm_64bit=1 > enable_uart=1 > upstream_kernel=1 > > kernel=vmlinuz-6.6.0+ > initramfs initrd.img-6.6.0+ > > enable_jtag_gpio=1 > force_turbo=1 > >> - VC firmware version > This is pieeprom-2023-01-11.bin > > I know there's pieeprom-2023-05-11.bin as well, but I've been keeping > the former as a constant throughout the PCIe patch series testing once > I realized how critical it was (old beta EEPROM versions found in some > CM4s made everything much harder initially). Sorry, i was a little bit unspecific. This the first level bootloader stored in the EEPROM. I meant the VC firmware version from the SD card which is logged in dmesg: raspberrypi-firmware soc:firmware: Attached to firmware from ... > In case the config matters: > > [all] > BOOT_UART=0 > WAKE_ON_GPIO=1 > POWER_OFF_ON_HALT=0 > BOOT_ORDER=0xf25641 > ENABLE_SELF_UPDATE=1 > >> - did you use arm64/defconfig or something special? > I'm using a “distribution config”, starting from the latest arm64 config > found in Debian unstable (6.5.10-1), and applying `make oldconfig`. > > You'll find it attached. Thanks > > > Cheers, > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Stefan Wahren <wahrenst@gmx.net> (2023-11-27): > Sorry, i was a little bit unspecific. This the first level bootloader > stored in the EEPROM. I meant the VC firmware version from the SD card > which is logged in dmesg: > > raspberrypi-firmware soc:firmware: Attached to firmware from ... [ 2.677987] raspberrypi-firmware soc:firmware: Attached to firmware from 2022-08-26T14:03:16 Cheers,
On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: > > Hi Phil, > > >>>> Hi Justin, > >>>> > >>>> [add Phil] > >>>> > >>>> Am 27.11.23 um 07:02 schrieb Justin Chen: > >>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: > >>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board > >>>>>> does not have a VL805 USB 3.0 host controller, which is connected via > >>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in > >>>>>> xHCI. > >>>>>> > >>>>> Does this work? I maintain this built-in xHCI controller internally. I > >>>>> wasn't aware the Compute Module uses this block. > >>>> i successful tested this with a CM4 (arm 32 bit, > >>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices > >>>> (mouse, keyboard) connected to the host interface didn't work. After > >>>> comparing vendor DTS with mainline i noticed the missing xHCI block [1]. > >>>> Unfortunately i wasn't able to get further information from the public > >>>> datasheets. I don't know if the VideoCore does some magic tricks on the > >>>> xHCI or i missed some downstream xHCI changes. > >>>> > >>>>> This block is held in reset and needs a bit toggled to get things > >>>>> going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block > >>>>> correct? > >>>>> > >>>>> Justin > >>>> [1] - > >>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 > >>> What's the question here? Does the XHCI block present in the > >>> raspberrypi/linux dtsi file really exist? Yes it does. > >> since i don't have any documentation about the xHCI block, i assumed the > >> compatible generic-xhci is correct. But Justin seems to suggest that the > >> xHCI block needs some special treatment and we need a specific compatible. > >> > >> Did i missed some xHCI driver changes? > >> Does the VC firmware something to the xHCI especially on CM4? > > The firmware switches the on-board USB pins from DWC-OTG to XHCI if > > otg_mode=1 is set in config.txt, or if booting over USB MSD. > is this pinctrl/pinmux available from ARM via 0x7e200000 or a different > IO address? It's in a different, undocumented block. Phil
Hi Cyril, Am 27.11.23 um 14:02 schrieb Cyril Brulebois: > Stefan Wahren <wahrenst@gmx.net> (2023-11-27): >> Could you please provide the following information: >> >> - settings of config.txt > Here you go: > > arm_64bit=1 > enable_uart=1 > upstream_kernel=1 > > kernel=vmlinuz-6.6.0+ > initramfs initrd.img-6.6.0+ > > enable_jtag_gpio=1 > force_turbo=1 can you please check what happens if you add the following to config.txt: [cm4] otg_mode=1 > >> - VC firmware version > This is pieeprom-2023-01-11.bin > > I know there's pieeprom-2023-05-11.bin as well, but I've been keeping > the former as a constant throughout the PCIe patch series testing once > I realized how critical it was (old beta EEPROM versions found in some > CM4s made everything much harder initially). > > In case the config matters: > > [all] > BOOT_UART=0 > WAKE_ON_GPIO=1 > POWER_OFF_ON_HALT=0 > BOOT_ORDER=0xf25641 > ENABLE_SELF_UPDATE=1 > >> - did you use arm64/defconfig or something special? > I'm using a “distribution config”, starting from the latest arm64 config > found in Debian unstable (6.5.10-1), and applying `make oldconfig`. > > You'll find it attached. > > > Cheers,
On 11/27/23 8:28 AM, Phil Elwell wrote: > On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: >> >> Hi Phil, >> >>>>>> Hi Justin, >>>>>> >>>>>> [add Phil] >>>>>> >>>>>> Am 27.11.23 um 07:02 schrieb Justin Chen: >>>>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or the IO board >>>>>>>> does not have a VL805 USB 3.0 host controller, which is connected via >>>>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the built-in >>>>>>>> xHCI. >>>>>>>> >>>>>>> Does this work? I maintain this built-in xHCI controller internally. I >>>>>>> wasn't aware the Compute Module uses this block. >>>>>> i successful tested this with a CM4 (arm 32 bit, >>>>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB devices >>>>>> (mouse, keyboard) connected to the host interface didn't work. After >>>>>> comparing vendor DTS with mainline i noticed the missing xHCI block [1]. >>>>>> Unfortunately i wasn't able to get further information from the public >>>>>> datasheets. I don't know if the VideoCore does some magic tricks on the >>>>>> xHCI or i missed some downstream xHCI changes. >>>>>> >>>>>>> This block is held in reset and needs a bit toggled to get things >>>>>>> going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" block >>>>>>> correct? >>>>>>> >>>>>>> Justin >>>>>> [1] - >>>>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 >>>>> What's the question here? Does the XHCI block present in the >>>>> raspberrypi/linux dtsi file really exist? Yes it does. >>>> since i don't have any documentation about the xHCI block, i assumed the >>>> compatible generic-xhci is correct. But Justin seems to suggest that the >>>> xHCI block needs some special treatment and we need a specific compatible. >>>> >>>> Did i missed some xHCI driver changes? >>>> Does the VC firmware something to the xHCI especially on CM4? >>> The firmware switches the on-board USB pins from DWC-OTG to XHCI if >>> otg_mode=1 is set in config.txt, or if booting over USB MSD. >> is this pinctrl/pinmux available from ARM via 0x7e200000 or a different >> IO address? > > It's in a different, undocumented block. > > Phil Well if it works, then maybe I am misunderstanding something here. Maybe its time for me to pick up a CM4 board. Justin
Hi Stefan, Stefan Wahren <wahrenst@gmx.net> (2023-11-27): > can you please check what happens if you add the following to > config.txt: > > [cm4] > otg_mode=1 I've only rebooted a few times but the system seems to be booting fine once this setting is added to config.txt. Writing stuff to a USB stick with or without the patches seems to give similar performances: ~ 18 MB/s after 3 minutes of copying /dev/null to /dev/sda with a 32M block size; similar CPU usage for dd, usb-storage, and the relevant kworker thread. Cheers,
On 11/27/23 09:44, Justin Chen wrote: > > > On 11/27/23 8:28 AM, Phil Elwell wrote: >> On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: >>> >>> Hi Phil, >>> >>>>>>> Hi Justin, >>>>>>> >>>>>>> [add Phil] >>>>>>> >>>>>>> Am 27.11.23 um 07:02 schrieb Justin Chen: >>>>>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>>>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or the >>>>>>>>> IO board >>>>>>>>> does not have a VL805 USB 3.0 host controller, which is >>>>>>>>> connected via >>>>>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the >>>>>>>>> built-in >>>>>>>>> xHCI. >>>>>>>>> >>>>>>>> Does this work? I maintain this built-in xHCI controller >>>>>>>> internally. I >>>>>>>> wasn't aware the Compute Module uses this block. >>>>>>> i successful tested this with a CM4 (arm 32 bit, >>>>>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB >>>>>>> devices >>>>>>> (mouse, keyboard) connected to the host interface didn't work. After >>>>>>> comparing vendor DTS with mainline i noticed the missing xHCI >>>>>>> block [1]. >>>>>>> Unfortunately i wasn't able to get further information from the >>>>>>> public >>>>>>> datasheets. I don't know if the VideoCore does some magic tricks >>>>>>> on the >>>>>>> xHCI or i missed some downstream xHCI changes. >>>>>>> >>>>>>>> This block is held in reset and needs a bit toggled to get things >>>>>>>> going. Florian, just to confirm, this is our "brcm,xhci-brcm-v2" >>>>>>>> block >>>>>>>> correct? >>>>>>>> >>>>>>>> Justin >>>>>>> [1] - >>>>>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 >>>>>> What's the question here? Does the XHCI block present in the >>>>>> raspberrypi/linux dtsi file really exist? Yes it does. >>>>> since i don't have any documentation about the xHCI block, i >>>>> assumed the >>>>> compatible generic-xhci is correct. But Justin seems to suggest >>>>> that the >>>>> xHCI block needs some special treatment and we need a specific >>>>> compatible. >>>>> >>>>> Did i missed some xHCI driver changes? >>>>> Does the VC firmware something to the xHCI especially on CM4? >>>> The firmware switches the on-board USB pins from DWC-OTG to XHCI if >>>> otg_mode=1 is set in config.txt, or if booting over USB MSD. >>> is this pinctrl/pinmux available from ARM via 0x7e200000 or a different >>> IO address? >> >> It's in a different, undocumented block. >> >> Phil > > Well if it works, then maybe I am misunderstanding something here. Maybe > its time for me to pick up a CM4 board. There is one on my desk that you are welcome to use, or remote into if you prefer. To answer your earlier question, yes this is the same block as the one present in 72112 for which we use the "brcm,xhci-brcm-v2" compatible string, it would be preferable to have it backed by that compatible string in case we happen to support suspend/resume on the Pi 4B one day, if nothing else. I did confirm that after applying Stefan's patches plus changing my config.txt to have otg_mode=1, USB continues to be fully functional. This is the case with using both "generic-xhci" or "brcm,xhci-brcm-v2" so with the minor request to update the compatible to "brcm,xhci-brcm-v2", this is: Tested-by: Florian Fainelli <florian.fainelli@broadcom.com> Stefan, I am getting a deadlock on boot if I leave your changes in and uncomment dwc_otg=1 in config.txt, is there an alias or something that the boot loader should be patching?
Hi, even after Cyril get managed to get xHCI running. I noticed a issue. If i compile xhci as a kernel module for arm64: # # USB Host Controller Drivers # # CONFIG_USB_C67X00_HCD is not set CONFIG_USB_XHCI_HCD=m # CONFIG_USB_XHCI_DBGCAP is not set CONFIG_USB_XHCI_PCI=m CONFIG_USB_XHCI_PCI_RENESAS=m CONFIG_USB_XHCI_PLATFORM=m The driver isn't autoloaded, i need to run "modprobe xhci-plat-hcd" manually. pi@raspberrypi:~$ modinfo xhci-plat-hcd filename: /lib/modules/6.6.0-00004-g0cdd4337c529/kernel/drivers/usb/host/xhci-plat-hcd.ko license: GPL description: xHCI Platform Host Controller Driver alias: platform:xhci-hcd alias: of:N*T*Cbrcm,bcm7445-xhciC* alias: of:N*T*Cbrcm,bcm7445-xhci alias: of:N*T*Cbrcm,xhci-brcm-v2C* alias: of:N*T*Cbrcm,xhci-brcm-v2 alias: of:N*T*Cmarvell,armada3700-xhciC* alias: of:N*T*Cmarvell,armada3700-xhci alias: of:N*T*Cmarvell,armada-380-xhciC* alias: of:N*T*Cmarvell,armada-380-xhci alias: of:N*T*Cmarvell,armada-375-xhciC* alias: of:N*T*Cmarvell,armada-375-xhci alias: of:N*T*Cxhci-platformC* alias: of:N*T*Cxhci-platform alias: of:N*T*Cgeneric-xhciC* alias: of:N*T*Cgeneric-xhci alias: acpi*:PNP0D10:* depends: xhci-hcd intree: Y name: xhci_plat_hcd vermagic: 6.6.0-00004-g0cdd4337c529 SMP preempt mod_unload aarch64 pi@raspberrypi:~$ modinfo xhci-hcd filename: /lib/modules/6.6.0-00004-g0cdd4337c529/kernel/drivers/usb/host/xhci-hcd.ko license: GPL author: Sarah Sharp description: 'eXtensible' Host Controller (xHC) Driver depends: intree: Y name: xhci_hcd vermagic: 6.6.0-00004-g0cdd4337c529 SMP preempt mod_unload aarch64 parm: link_quirk:Don't clear the chain bit on a link TRB (int) parm: quirks:Bit flags for quirks to be enabled as default (ullong) Has anyone an idea, what's wrong here?
Hi, Am 27.11.23 um 19:41 schrieb Florian Fainelli: > On 11/27/23 09:44, Justin Chen wrote: >> >> >> On 11/27/23 8:28 AM, Phil Elwell wrote: >>> On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: >>>> >>>> Hi Phil, >>>> >>>>>>>> Hi Justin, >>>>>>>> >>>>>>>> [add Phil] >>>>>>>> >>>>>>>> Am 27.11.23 um 07:02 schrieb Justin Chen: >>>>>>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>>>>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or >>>>>>>>>> the IO board >>>>>>>>>> does not have a VL805 USB 3.0 host controller, which is >>>>>>>>>> connected via >>>>>>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the >>>>>>>>>> built-in >>>>>>>>>> xHCI. >>>>>>>>>> >>>>>>>>> Does this work? I maintain this built-in xHCI controller >>>>>>>>> internally. I >>>>>>>>> wasn't aware the Compute Module uses this block. >>>>>>>> i successful tested this with a CM4 (arm 32 bit, >>>>>>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB >>>>>>>> devices >>>>>>>> (mouse, keyboard) connected to the host interface didn't work. >>>>>>>> After >>>>>>>> comparing vendor DTS with mainline i noticed the missing xHCI >>>>>>>> block [1]. >>>>>>>> Unfortunately i wasn't able to get further information from the >>>>>>>> public >>>>>>>> datasheets. I don't know if the VideoCore does some magic >>>>>>>> tricks on the >>>>>>>> xHCI or i missed some downstream xHCI changes. >>>>>>>> >>>>>>>>> This block is held in reset and needs a bit toggled to get things >>>>>>>>> going. Florian, just to confirm, this is our >>>>>>>>> "brcm,xhci-brcm-v2" block >>>>>>>>> correct? >>>>>>>>> >>>>>>>>> Justin >>>>>>>> [1] - >>>>>>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 >>>>>>>> >>>>>>> What's the question here? Does the XHCI block present in the >>>>>>> raspberrypi/linux dtsi file really exist? Yes it does. >>>>>> since i don't have any documentation about the xHCI block, i >>>>>> assumed the >>>>>> compatible generic-xhci is correct. But Justin seems to suggest >>>>>> that the >>>>>> xHCI block needs some special treatment and we need a specific >>>>>> compatible. >>>>>> >>>>>> Did i missed some xHCI driver changes? >>>>>> Does the VC firmware something to the xHCI especially on CM4? >>>>> The firmware switches the on-board USB pins from DWC-OTG to XHCI if >>>>> otg_mode=1 is set in config.txt, or if booting over USB MSD. >>>> is this pinctrl/pinmux available from ARM via 0x7e200000 or a >>>> different >>>> IO address? >>> >>> It's in a different, undocumented block. >>> >>> Phil >> >> Well if it works, then maybe I am misunderstanding something here. >> Maybe its time for me to pick up a CM4 board. > There is one on my desk that you are welcome to use, or remote into if > you prefer. > > To answer your earlier question, yes this is the same block as the one > present in 72112 for which we use the "brcm,xhci-brcm-v2" compatible > string, it would be preferable to have it backed by that compatible > string in case we happen to support suspend/resume on the Pi 4B one > day, if nothing else. > > I did confirm that after applying Stefan's patches plus changing my > config.txt to have otg_mode=1, USB continues to be fully functional. > This is the case with using both "generic-xhci" or "brcm,xhci-brcm-v2" > so with the minor request to update the compatible to > "brcm,xhci-brcm-v2", this is: > > Tested-by: Florian Fainelli <florian.fainelli@broadcom.com> > > Stefan, I am getting a deadlock on boot if I leave your changes in and > uncomment dwc_otg=1 in config.txt, is there an alias or something that > the boot loader should be patching? sorry but i'm unable reproduce those deadlocks, neither in arm or arm64, with eMMC or without eMMC, xhci builtin or module. If i uncomment this in config.txt, USB host is just disabled. I'm using the following firmware: raspberrypi-firmware soc:firmware: Attached to firmware from 2023-03-17T10:50:39 Is this DTS difference a problem? upstream -> xhci: usb@7e9c0000 downstream -> xhci: xhci@7e9c0000
On 11/27/23 11:22, Stefan Wahren wrote: > Hi, > > Am 27.11.23 um 19:41 schrieb Florian Fainelli: >> On 11/27/23 09:44, Justin Chen wrote: >>> >>> >>> On 11/27/23 8:28 AM, Phil Elwell wrote: >>>> On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: >>>>> >>>>> Hi Phil, >>>>> >>>>>>>>> Hi Justin, >>>>>>>>> >>>>>>>>> [add Phil] >>>>>>>>> >>>>>>>>> Am 27.11.23 um 07:02 schrieb Justin Chen: >>>>>>>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>>>>>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or >>>>>>>>>>> the IO board >>>>>>>>>>> does not have a VL805 USB 3.0 host controller, which is >>>>>>>>>>> connected via >>>>>>>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the >>>>>>>>>>> built-in >>>>>>>>>>> xHCI. >>>>>>>>>>> >>>>>>>>>> Does this work? I maintain this built-in xHCI controller >>>>>>>>>> internally. I >>>>>>>>>> wasn't aware the Compute Module uses this block. >>>>>>>>> i successful tested this with a CM4 (arm 32 bit, >>>>>>>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB >>>>>>>>> devices >>>>>>>>> (mouse, keyboard) connected to the host interface didn't work. >>>>>>>>> After >>>>>>>>> comparing vendor DTS with mainline i noticed the missing xHCI >>>>>>>>> block [1]. >>>>>>>>> Unfortunately i wasn't able to get further information from the >>>>>>>>> public >>>>>>>>> datasheets. I don't know if the VideoCore does some magic >>>>>>>>> tricks on the >>>>>>>>> xHCI or i missed some downstream xHCI changes. >>>>>>>>> >>>>>>>>>> This block is held in reset and needs a bit toggled to get things >>>>>>>>>> going. Florian, just to confirm, this is our >>>>>>>>>> "brcm,xhci-brcm-v2" block >>>>>>>>>> correct? >>>>>>>>>> >>>>>>>>>> Justin >>>>>>>>> [1] - >>>>>>>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 >>>>>>>>> >>>>>>>> What's the question here? Does the XHCI block present in the >>>>>>>> raspberrypi/linux dtsi file really exist? Yes it does. >>>>>>> since i don't have any documentation about the xHCI block, i >>>>>>> assumed the >>>>>>> compatible generic-xhci is correct. But Justin seems to suggest >>>>>>> that the >>>>>>> xHCI block needs some special treatment and we need a specific >>>>>>> compatible. >>>>>>> >>>>>>> Did i missed some xHCI driver changes? >>>>>>> Does the VC firmware something to the xHCI especially on CM4? >>>>>> The firmware switches the on-board USB pins from DWC-OTG to XHCI if >>>>>> otg_mode=1 is set in config.txt, or if booting over USB MSD. >>>>> is this pinctrl/pinmux available from ARM via 0x7e200000 or a >>>>> different >>>>> IO address? >>>> >>>> It's in a different, undocumented block. >>>> >>>> Phil >>> >>> Well if it works, then maybe I am misunderstanding something here. >>> Maybe its time for me to pick up a CM4 board. >> There is one on my desk that you are welcome to use, or remote into if >> you prefer. >> >> To answer your earlier question, yes this is the same block as the one >> present in 72112 for which we use the "brcm,xhci-brcm-v2" compatible >> string, it would be preferable to have it backed by that compatible >> string in case we happen to support suspend/resume on the Pi 4B one >> day, if nothing else. >> >> I did confirm that after applying Stefan's patches plus changing my >> config.txt to have otg_mode=1, USB continues to be fully functional. >> This is the case with using both "generic-xhci" or "brcm,xhci-brcm-v2" >> so with the minor request to update the compatible to >> "brcm,xhci-brcm-v2", this is: >> >> Tested-by: Florian Fainelli <florian.fainelli@broadcom.com> >> >> Stefan, I am getting a deadlock on boot if I leave your changes in and >> uncomment dwc_otg=1 in config.txt, is there an alias or something that >> the boot loader should be patching? > > sorry but i'm unable reproduce those deadlocks, neither in arm or arm64, > with eMMC or without eMMC, xhci builtin or module. If i uncomment this > in config.txt, USB host is just disabled. Here is my config.txt FWIW: # A bit too verbose uart_2ndstage=1 enable_uart=1 arm_64bit=1 # Custom kernel images kernel=kernel8-upstream.img #kernel=kernel7l.img #device_tree=bcm2711-rpi-4-b-UPSTREAM.dtb device_tree=bcm2711-rpi-cm4-io-UPSTREAM.dtb force_turbo=1 # DWC-OTG <=> XHCI #otg_mode=1 > > I'm using the following firmware: > > raspberrypi-firmware soc:firmware: Attached to firmware from > 2023-03-17T10:50:39 OK, my CM4 is at 2022-07-25T15:10:17, updating to 2023-10-17T15:39:16 does not really show any difference. > > Is this DTS difference a problem? It does not appear so, changing the node unit-name does not affect the results. > > upstream -> xhci: usb@7e9c0000 > downstream -> xhci: xhci@7e9c0000 Side question: does the VPU boot ROM or firmware take care of configuring the USB PHY somehow? Should not we also have a Device Tree node for it eventually?
Am 27.11.23 um 22:49 schrieb Florian Fainelli: > On 11/27/23 11:22, Stefan Wahren wrote: >> Hi, >> >> Am 27.11.23 um 19:41 schrieb Florian Fainelli: >>> On 11/27/23 09:44, Justin Chen wrote: >>>> >>>> >>>> On 11/27/23 8:28 AM, Phil Elwell wrote: >>>>> On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: >>>>>> >>>>>> Hi Phil, >>>>>> >>>>>>>>>> Hi Justin, >>>>>>>>>> >>>>>>>>>> [add Phil] >>>>>>>>>> >>>>>>>>>> Am 27.11.23 um 07:02 schrieb Justin Chen: >>>>>>>>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: >>>>>>>>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or >>>>>>>>>>>> the IO board >>>>>>>>>>>> does not have a VL805 USB 3.0 host controller, which is >>>>>>>>>>>> connected via >>>>>>>>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the >>>>>>>>>>>> built-in >>>>>>>>>>>> xHCI. >>>>>>>>>>>> >>>>>>>>>>> Does this work? I maintain this built-in xHCI controller >>>>>>>>>>> internally. I >>>>>>>>>>> wasn't aware the Compute Module uses this block. >>>>>>>>>> i successful tested this with a CM4 (arm 32 bit, >>>>>>>>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB >>>>>>>>>> devices >>>>>>>>>> (mouse, keyboard) connected to the host interface didn't work. >>>>>>>>>> After >>>>>>>>>> comparing vendor DTS with mainline i noticed the missing xHCI >>>>>>>>>> block [1]. >>>>>>>>>> Unfortunately i wasn't able to get further information from the >>>>>>>>>> public >>>>>>>>>> datasheets. I don't know if the VideoCore does some magic >>>>>>>>>> tricks on the >>>>>>>>>> xHCI or i missed some downstream xHCI changes. >>>>>>>>>> >>>>>>>>>>> This block is held in reset and needs a bit toggled to get >>>>>>>>>>> things >>>>>>>>>>> going. Florian, just to confirm, this is our >>>>>>>>>>> "brcm,xhci-brcm-v2" block >>>>>>>>>>> correct? >>>>>>>>>>> >>>>>>>>>>> Justin >>>>>>>>>> [1] - >>>>>>>>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 >>>>>>>>>> >>>>>>>>>> >>>>>>>>> What's the question here? Does the XHCI block present in the >>>>>>>>> raspberrypi/linux dtsi file really exist? Yes it does. >>>>>>>> since i don't have any documentation about the xHCI block, i >>>>>>>> assumed the >>>>>>>> compatible generic-xhci is correct. But Justin seems to suggest >>>>>>>> that the >>>>>>>> xHCI block needs some special treatment and we need a specific >>>>>>>> compatible. >>>>>>>> >>>>>>>> Did i missed some xHCI driver changes? >>>>>>>> Does the VC firmware something to the xHCI especially on CM4? >>>>>>> The firmware switches the on-board USB pins from DWC-OTG to XHCI if >>>>>>> otg_mode=1 is set in config.txt, or if booting over USB MSD. >>>>>> is this pinctrl/pinmux available from ARM via 0x7e200000 or a >>>>>> different >>>>>> IO address? >>>>> >>>>> It's in a different, undocumented block. >>>>> >>>>> Phil >>>> >>>> Well if it works, then maybe I am misunderstanding something here. >>>> Maybe its time for me to pick up a CM4 board. >>> There is one on my desk that you are welcome to use, or remote into if >>> you prefer. >>> >>> To answer your earlier question, yes this is the same block as the one >>> present in 72112 for which we use the "brcm,xhci-brcm-v2" compatible >>> string, it would be preferable to have it backed by that compatible >>> string in case we happen to support suspend/resume on the Pi 4B one >>> day, if nothing else. >>> >>> I did confirm that after applying Stefan's patches plus changing my >>> config.txt to have otg_mode=1, USB continues to be fully functional. >>> This is the case with using both "generic-xhci" or "brcm,xhci-brcm-v2" >>> so with the minor request to update the compatible to >>> "brcm,xhci-brcm-v2", this is: >>> >>> Tested-by: Florian Fainelli <florian.fainelli@broadcom.com> >>> >>> Stefan, I am getting a deadlock on boot if I leave your changes in and >>> uncomment dwc_otg=1 in config.txt, is there an alias or something that >>> the boot loader should be patching? >> >> sorry but i'm unable reproduce those deadlocks, neither in arm or arm64, >> with eMMC or without eMMC, xhci builtin or module. If i uncomment this >> in config.txt, USB host is just disabled. > > Here is my config.txt FWIW: > > # A bit too verbose > uart_2ndstage=1 > enable_uart=1 > arm_64bit=1 > # Custom kernel images > kernel=kernel8-upstream.img > #kernel=kernel7l.img > #device_tree=bcm2711-rpi-4-b-UPSTREAM.dtb > device_tree=bcm2711-rpi-cm4-io-UPSTREAM.dtb > force_turbo=1 > # DWC-OTG <=> XHCI > #otg_mode=1 > >> >> I'm using the following firmware: >> >> raspberrypi-firmware soc:firmware: Attached to firmware from >> 2023-03-17T10:50:39 > > OK, my CM4 is at 2022-07-25T15:10:17, updating to 2023-10-17T15:39:16 > does not really show any difference. > >> >> Is this DTS difference a problem? > > It does not appear so, changing the node unit-name does not affect the > results. > >> >> upstream -> xhci: usb@7e9c0000 >> downstream -> xhci: xhci@7e9c0000 > > Side question: does the VPU boot ROM or firmware take care of > configuring the USB PHY somehow? Should not we also have a Device Tree > node for it eventually? Sorry, as the person with the least knowledge about the hardware i cannot answer this question. But we should avoid those nop-PHY because they have source of regressions in the past.
On Tue, 28 Nov 2023 at 06:44, Stefan Wahren <wahrenst@gmx.net> wrote: > > > Am 27.11.23 um 22:49 schrieb Florian Fainelli: > > On 11/27/23 11:22, Stefan Wahren wrote: > >> Hi, > >> > >> Am 27.11.23 um 19:41 schrieb Florian Fainelli: > >>> On 11/27/23 09:44, Justin Chen wrote: > >>>> > >>>> > >>>> On 11/27/23 8:28 AM, Phil Elwell wrote: > >>>>> On Mon, 27 Nov 2023 at 12:39, Stefan Wahren <wahrenst@gmx.net> wrote: > >>>>>> > >>>>>> Hi Phil, > >>>>>> > >>>>>>>>>> Hi Justin, > >>>>>>>>>> > >>>>>>>>>> [add Phil] > >>>>>>>>>> > >>>>>>>>>> Am 27.11.23 um 07:02 schrieb Justin Chen: > >>>>>>>>>>> On 11/25/23 6:56 PM, Stefan Wahren wrote: > >>>>>>>>>>>> In contrast to the Raspberry Pi 4, the Compute Module 4 or > >>>>>>>>>>>> the IO board > >>>>>>>>>>>> does not have a VL805 USB 3.0 host controller, which is > >>>>>>>>>>>> connected via > >>>>>>>>>>>> PCIe. Instead, the BCM2711 on the Compute Module provides the > >>>>>>>>>>>> built-in > >>>>>>>>>>>> xHCI. > >>>>>>>>>>>> > >>>>>>>>>>> Does this work? I maintain this built-in xHCI controller > >>>>>>>>>>> internally. I > >>>>>>>>>>> wasn't aware the Compute Module uses this block. > >>>>>>>>>> i successful tested this with a CM4 (arm 32 bit, > >>>>>>>>>> multi_v7_lpae_defconfig) with eMMC. Before this series the USB > >>>>>>>>>> devices > >>>>>>>>>> (mouse, keyboard) connected to the host interface didn't work. > >>>>>>>>>> After > >>>>>>>>>> comparing vendor DTS with mainline i noticed the missing xHCI > >>>>>>>>>> block [1]. > >>>>>>>>>> Unfortunately i wasn't able to get further information from the > >>>>>>>>>> public > >>>>>>>>>> datasheets. I don't know if the VideoCore does some magic > >>>>>>>>>> tricks on the > >>>>>>>>>> xHCI or i missed some downstream xHCI changes. > >>>>>>>>>> > >>>>>>>>>>> This block is held in reset and needs a bit toggled to get > >>>>>>>>>>> things > >>>>>>>>>>> going. Florian, just to confirm, this is our > >>>>>>>>>>> "brcm,xhci-brcm-v2" block > >>>>>>>>>>> correct? > >>>>>>>>>>> > >>>>>>>>>>> Justin > >>>>>>>>>> [1] - > >>>>>>>>>> https://github.com/raspberrypi/linux/blob/rpi-6.1.y/arch/arm/boot/dts/bcm2711-rpi-ds.dtsi#L119 > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> What's the question here? Does the XHCI block present in the > >>>>>>>>> raspberrypi/linux dtsi file really exist? Yes it does. > >>>>>>>> since i don't have any documentation about the xHCI block, i > >>>>>>>> assumed the > >>>>>>>> compatible generic-xhci is correct. But Justin seems to suggest > >>>>>>>> that the > >>>>>>>> xHCI block needs some special treatment and we need a specific > >>>>>>>> compatible. > >>>>>>>> > >>>>>>>> Did i missed some xHCI driver changes? > >>>>>>>> Does the VC firmware something to the xHCI especially on CM4? > >>>>>>> The firmware switches the on-board USB pins from DWC-OTG to XHCI if > >>>>>>> otg_mode=1 is set in config.txt, or if booting over USB MSD. > >>>>>> is this pinctrl/pinmux available from ARM via 0x7e200000 or a > >>>>>> different > >>>>>> IO address? > >>>>> > >>>>> It's in a different, undocumented block. > >>>>> > >>>>> Phil > >>>> > >>>> Well if it works, then maybe I am misunderstanding something here. > >>>> Maybe its time for me to pick up a CM4 board. > >>> There is one on my desk that you are welcome to use, or remote into if > >>> you prefer. > >>> > >>> To answer your earlier question, yes this is the same block as the one > >>> present in 72112 for which we use the "brcm,xhci-brcm-v2" compatible > >>> string, it would be preferable to have it backed by that compatible > >>> string in case we happen to support suspend/resume on the Pi 4B one > >>> day, if nothing else. > >>> > >>> I did confirm that after applying Stefan's patches plus changing my > >>> config.txt to have otg_mode=1, USB continues to be fully functional. > >>> This is the case with using both "generic-xhci" or "brcm,xhci-brcm-v2" > >>> so with the minor request to update the compatible to > >>> "brcm,xhci-brcm-v2", this is: > >>> > >>> Tested-by: Florian Fainelli <florian.fainelli@broadcom.com> > >>> > >>> Stefan, I am getting a deadlock on boot if I leave your changes in and > >>> uncomment dwc_otg=1 in config.txt, is there an alias or something that > >>> the boot loader should be patching? > >> > >> sorry but i'm unable reproduce those deadlocks, neither in arm or arm64, > >> with eMMC or without eMMC, xhci builtin or module. If i uncomment this > >> in config.txt, USB host is just disabled. > > > > Here is my config.txt FWIW: > > > > # A bit too verbose > > uart_2ndstage=1 > > enable_uart=1 > > arm_64bit=1 > > # Custom kernel images > > kernel=kernel8-upstream.img > > #kernel=kernel7l.img > > #device_tree=bcm2711-rpi-4-b-UPSTREAM.dtb > > device_tree=bcm2711-rpi-cm4-io-UPSTREAM.dtb > > force_turbo=1 > > # DWC-OTG <=> XHCI > > #otg_mode=1 > > > >> > >> I'm using the following firmware: > >> > >> raspberrypi-firmware soc:firmware: Attached to firmware from > >> 2023-03-17T10:50:39 > > > > OK, my CM4 is at 2022-07-25T15:10:17, updating to 2023-10-17T15:39:16 > > does not really show any difference. > > > >> > >> Is this DTS difference a problem? > > > > It does not appear so, changing the node unit-name does not affect the > > results. > > > >> > >> upstream -> xhci: usb@7e9c0000 > >> downstream -> xhci: xhci@7e9c0000 > > > > Side question: does the VPU boot ROM or firmware take care of > > configuring the USB PHY somehow? Should not we also have a Device Tree > > node for it eventually? > > Sorry, as the person with the least knowledge about the hardware i > cannot answer this question. But we should avoid those nop-PHY because > they have source of regressions in the past. The EEPROM firmware initialises the USB PHY. Phil