mbox series

[v8,00/16] Enable CXL PCIe port protocol error handling and logging

Message ID 20250327014717.2988633-1-terry.bowman@amd.com
Headers show
Series Enable CXL PCIe port protocol error handling and logging | expand

Message

Bowman, Terry March 27, 2025, 1:47 a.m. UTC
This patchset updates CXL Protocol Error handling for CXL Ports and CXL
Endpoints (EP). The reach of this patchset grew from CXL Ports to include
EPs as well because updating the handling for all devices is preferable
over supporting multiple handling paths.

This patchset is a continuation of v7 and can be found here:
https://lore.kernel.org/linux-cxl/20250211192444.2292833-1-terry.bowman@amd.com/

The difference between v7 and v8 includes a significant refactor of the CXL
error handling. v8 further defines the difference between CXL errors and
PCIe errors by moving the CXL error handling to the CXL driver. The intent
is to isolate the CXL error handling from the AER driver as much as
possible.  The AER driver will continue to handle AER interrupts but will
now forward an error to the CXL driver if it's a CXL error. If an error is
not a CXL error then the existing PCIe flow is used to handle the error.

Another change from v7->v8 is the error handlers themselves. v8 introduces
the error handlers as 'struct cxl_driver::err_handler' instead of as
'struct pci_dev::cxl_err_handlers' done in v7. 

Most of the review acks and reviewed-by's had to be taken down because of
changes.

= Patch Descriptions =
The first 2 patches introduce pci_dev::is_cxl, aer_info::is_cxl, and add
bus string to AER log tracing. aer_info::is_cxl will be used to indicate a
CXL or PCI error and will affect the error handling flow in later patches.

The next 6 patches add a kfifo for forwarding CXL errors to the CXL driver.
These patches also move the CXL handling from the AER service driver into
the CXL driver and add the necessary plumbing. This subset of patches also
introduces CXL UCE handling never present in the AER service driver. 

The next 3 patches add the CXL Port RAS mapping and interface updates to
support addition of CXL error handlers.

The final 5 patches add the CXL error handlers for CXL EPs and CXL Ports.
CXL EPs keep the PCIe error handler for cases the EP error is interpreted as
a PCIe error (please see USP and EP UCE testing below). These patches also
add logic to assign the CXL error handlers to a CXL device, unmask CXL
Protocol Errors during port probing, and mask CXL Protocol Errors during
port device cleanup. 

= Testing =
 Below are test results for this patchset using QEMU with CXL Root
 Port(RP, 0C:00.0), CXL Upstream Switch Port(USP, 0D:00.0), and CXL
 Downstream Switch Port(DSP, 0E:00.0), and Endpoint (EP, 0F:00.0).

 The sub-topology for testing is:
                    ---------------------
                    | CXL RP - 0C:00.0  |
                    ---------------------
                              |
                    ---------------------
                    | CXL USP - 0D:00.0 |
                    ---------------------
                              |
                    ---------------------
                    | CXL DSP - 0E:00.0 |
                    ---------------------
                              |
                    ---------------------
                    | CXL EP - 0F:00.0  |
                    ---------------------

 root@tbowman-cxl:~# lspci -t
 -+-[0000:00]-+-00.0
  |           +-01.0
  |           +-02.0
  |           +-03.0
  |           +-1f.0
  |           +-1f.2
  |           \-1f.3
  \-[0000:0c]---00.0-[0d-0f]----00.0-[0e-0f]----00.0-[0f]----00.0

 The topology was created with:
  ${qemu} -boot menu=on \
             -cpu host \
             -nographic \
             -monitor telnet:127.0.0.1:1234,server,nowait \
             -M virt,cxl=on \
             -chardev stdio,id=s1,signal=off,mux=on -serial none \
             -device isa-serial,chardev=s1 -mon chardev=s1,mode=readline \
             -machine q35,cxl=on \
             -m 16G,maxmem=24G,slots=8 \
             -cpu EPYC-v3 \
             -smp 16 \
             -accel kvm \
             -drive file=${img},format=raw,index=0,media=disk \
             -device e1000,netdev=user.0 \
             -netdev user,id=user.0,hostfwd=tcp::5555-:22 \
             -object memory-backend-file,id=cxl-mem0,share=on,mem-path=/tmp/cxltest.raw,size=256M \
             -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa0.raw,size=256M \
             -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
             -device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \
             -device cxl-upstream,bus=root_port0,id=us0 \
             -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
             -device cxl-type3,bus=swport0,volatile-memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-vmem0 \
             -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k

 NOTE: The USP and EP below include a UCE injection that is handled as a
 PCIe error instead of a CXL error. The stack trace shows handling from
 pcie_do_recovery() instead of cxl_do_redcovery(). This is because the AER
 info is not read from EP's or USP's due to the UCE error is AER_FATAL. As
 a result, the CXL RAS is not logged. But, panic is called when the PCIe
 report_error_detected() calls CXL EP PCIe error handler, containing the panic().
 
== CXL Root Port ==
root@tbowman-cxl:~/aer-inject# ./root-ce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0c:00.0
pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0c:00.0
pcieport 0000:0c:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
pcieport 0000:0c:00.0:   device [8086:7075] error status/mask=00004000/0000a000
pcieport 0000:0c:00.0:    [14] CorrIntErr            
aer_event: 0000:0c:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available
cxl_aer_correctable_error: device=port1 (0000:0c:00.0) parent=root0 (pci0000:0c) serieal=0 status='CRC Threshold Hit'

root@tbowman-cxl:~/aer-inject# ./root-uce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0c:00.0
pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0c:00.0
pcieport 0000:0c:00.0: CXL Bus Error: severity=Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
pcieport 0000:0c:00.0:   device [8086:7075] error status/mask=00400000/02000000
pcieport 0000:0c:00.0:    [22] UncorrIntErr      
aer_event: 0000:0c:00.0 CXL Bus Error: severity=Fatal, Uncorrectable Internal Error, TLP Header=Not available
cxl_aer_uncorrectable_error: device=port1 (0000:0c:00.0) parent=root0 (pci0000:0c) serial: 0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enable Parity Error'
Kernel panic - not syncing: CXL cachemem error.
CPU: 10 UID: 0 PID: 287 Comm: kworker/10:1 Tainted: G            E      6.14.0-rc1-hp-debug+ #199
Tainted: [E]=UNSIGNED_MODULE
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
Workqueue: events cxl_prot_err_work_fn [cxl_core]
Call Trace:
 <TASK>
 dump_stack_lvl+0x27/0x90
 dump_stack+0x10/0x20
 panic+0x33e/0x380
 ? preempt_count_add+0x4b/0xc0
 cxl_do_recovery+0xc9/0xd0 [cxl_core]
 cxl_prot_err_work_fn+0x74/0x190 [cxl_core]
 process_scheduled_works+0xa6/0x420
 worker_thread+0x121/0x260
 kthread+0x10b/0x220
 ? __pfx_worker_thread+0x10/0x10
 ? _raw_spin_unlock_irq+0x1f/0x40
 ? __pfx_kthread+0x10/0x10
 ret_from_fork+0x3c/0x60
 ? __pfx_kthread+0x10/0x10
 ret_from_fork_asm+0x1a/0x30
 </TASK>
Kernel Offset: 0x4a00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
---[ end Kernel panic - not syncing: CXL cachemem error. ]---

== CXL Upstream Switch Port ==
root@tbowman-cxl:~/aer-inject# ./us-ce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0d:00.0
pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0d:00.0
pcieport 0000:0d:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
pcieport 0000:0d:00.0:   device [19e5:a128] error status/mask=00004000/0000a000
pcieport 0000:0d:00.0:    [14] CorrIntErr            
aer_event: 0000:0d:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available
cxl_aer_correctable_error: device=port2 (0000:0d:00.0) parent=port1 (0000:0c:00.0) serieal=0 status='CRC Threshold Hit'

root@tbowman-cxl:~/aer-inject# ./us-uce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0d:00.0
pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0d:00.0
pcieport 0000:0d:00.0: AER: CXL Bus Error: severity=Uncorrectable (Fatal), type=Inaccessible, (Unregistered Agent ID)
aer_event: 0000:0d:00.0 CXL Bus Error: severity=Fatal, , TLP Header=Not available
Kernel panic - not syncing: CXL cachemem error.
CPU: 10 UID: 0 PID: 147 Comm: irq/24-aerdrv Tainted: G            E      6.14.0-rc1-hp-debug+ #200
Tainted: [E]=UNSIGNED_MODULE
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
Call Trace:
 <TASK>
 dump_stack_lvl+0x27/0x90
 dump_stack+0x10/0x20
 panic+0x33e/0x380
 ? __pfx_report_frozen_detected+0x10/0x10
 pci_error_detected+0x6d/0x70 [cxl_core]
 report_error_detected+0xc2/0x180
 ? __pm_runtime_resume+0x60/0x90
 ? __pfx_report_frozen_detected+0x10/0x10
 report_frozen_detected+0x16/0x20
 __pci_walk_bus+0x50/0x70
 ? __pfx_report_frozen_detected+0x10/0x10
 __pci_walk_bus+0x39/0x70
 ? __pfx_report_frozen_detected+0x10/0x10
 __pci_walk_bus+0x39/0x70
 ? __pfx_report_frozen_detected+0x10/0x10
 pci_walk_bus+0x32/0x50
 pci_walk_bridge+0x1d/0x40
 pcie_do_recovery+0x175/0x2b0
 ? __pfx_aer_root_reset+0x10/0x10
 aer_isr_one_error.isra.0+0x656/0x720
 ? srso_return_thunk+0x5/0x5f
 ? _raw_spin_unlock+0x19/0x40
 ? srso_return_thunk+0x5/0x5f
 ? __switch_to+0x115/0x420
 ? srso_return_thunk+0x5/0x5f
 ? __schedule+0x4d1/0x1190
 aer_isr+0x4d/0x80
 irq_thread_fn+0x28/0x70
 irq_thread+0x179/0x240
 ? srso_return_thunk+0x5/0x5f
 ? __pfx_irq_thread_fn+0x10/0x10
 ? __pfx_irq_thread_dtor+0x10/0x10
 kthread+0x10b/0x220
 ? __pfx_irq_thread+0x10/0x10
 ? _raw_spin_unlock_irq+0x1f/0x40
 ? __pfx_kthread+0x10/0x10
 ret_from_fork+0x3c/0x60
 ? __pfx_kthread+0x10/0x10
 ret_from_fork_asm+0x1a/0x30
 </TASK>
Kernel Offset: 0x6e00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
---[ end Kernel panic - not syncing: CXL cachemem error. ]---

== CXL Downstream Port == 
root@tbowman-cxl:~/aer-inject# ./ds-ce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0e:00.0
pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0e:00.0
pcieport 0000:0e:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
pcieport 0000:0e:00.0:   device [19e5:a129] error status/mask=00004000/0000a000
pcieport 0000:0e:00.0:    [14] CorrIntErr            
aer_event: 0000:0e:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available
cxl_aer_correctable_error: device=port2 (0000:0e:00.0) parent=port1 (0000:0d:00.0) serieal=0 status='CRC Threshold Hit'

root@tbowman-cxl:~/aer-inject# ./ds-uce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0e:00.0
pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0e:00.0
pcieport 0000:0e:00.0: CXL Bus Error: severity=Uncorrectable (Fatal), type=Transaction Layer, (Receiver ID)
pcieport 0000:0e:00.0:   device [19e5:a129] error status/mask=00400000/02000000
pcieport 0000:0e:00.0:    [22] UncorrIntErr          
aer_event: 0000:0e:00.0 CXL Bus Error: severity=Fatal, Uncorrectable Internal Error, TLP Header=Not available
cxl_aer_uncorrectable_error: device=port2 (0000:0e:00.0) parent=port1 (0000:0d:00.0) serial: 0 status: 'Cache Byte Enable Parity Error' first_error: 'Cache Byte Enabl'
Kernel panic - not syncing: CXL cachemem error.
CPU: 10 UID: 0 PID: 81 Comm: kworker/10:0 Tainted: G            E      6.14.0-rc1-hp-debug+ #201
Tainted: [E]=UNSIGNED_MODULE
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
Workqueue: events cxl_prot_err_work_fn [cxl_core]
Call Trace:
 <TASK>
 dump_stack_lvl+0x27/0x90
 dump_stack+0x10/0x20
 panic+0x33e/0x380
 ? preempt_count_add+0x4b/0xc0
 cxl_do_recovery+0xc9/0xd0 [cxl_core]
 cxl_prot_err_work_fn+0x74/0x190 [cxl_core]
 process_scheduled_works+0xa6/0x420
 worker_thread+0x121/0x260
 kthread+0x10b/0x220
 ? __pfx_worker_thread+0x10/0x10
 ? _raw_spin_unlock_irq+0x1f/0x40
 ? __pfx_kthread+0x10/0x10
 ret_from_fork+0x3c/0x60
 ? __pfx_kthread+0x10/0x10
 ret_from_fork_asm+0x1a/0x30
 </TASK>
Kernel Offset: 0x29c00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
---[ end Kernel panic - not syncing: CXL cachemem error. ]---

== CXL Endpoint ==
root@tbowman-cxl:~/aer-inject# ./ep-ce-inject.sh
 pcieport 0000:0c:00.0: aer_inject: Injecting errors 00004000/00000000 into device 0000:0f:00.0
 pcieport 0000:0c:00.0: AER: Correctable error message received from 0000:0f:00.0
 cxl_pci 0000:0f:00.0: CXL Bus Error: severity=Correctable, type=Transaction Layer, (Receiver ID)
 cxl_pci 0000:0f:00.0:   device [8086:0d93] error status/mask=00004000/00000000
 cxl_pci 0000:0f:00.0:    [14] CorrIntErr            
 aer_event: 0000:0f:00.0 CXL Bus Error: severity=Corrected, Corrected Internal Error, TLP Header=Not available 
 cxl_aer_correctable_error: device=mem3 (0000:0f:00.0) parent=0000:0f:00.0 (0000:0e:00.0) serieal=0 status='CRC Threshold Hit'

root@tbowman-cxl:~/aer-inject# ./ep-uce-inject.sh
pcieport 0000:0c:00.0: aer_inject: Injecting errors 00000000/00400000 into device 0000:0f:00.0
pcieport 0000:0c:00.0: AER: Uncorrectable (Fatal) error message received from 0000:0f:00.0
cxl_pci 0000:0f:00.0: AER: CXL Bus Error: severity=Uncorrectable (Fatal), type=Inaccessible, (Unregistered Agent ID)
aer_event: 0000:0f:00.0 CXL Bus Error: severity=Fatal, , TLP Header=Not available
cxl_aer_uncorrectable_error: device=mem3 (0000:0f:00.0) parent=0000:0f:00.0 (0000:0e:00.0) serial: 0 status: 'Cache Byte Enable Parity '
Kernel panic - not syncing: CXL cachemem error.
CPU: 10 UID: 0 PID: 146 Comm: irq/24-aerdrv Tainted: G            E      6.14.0-rc1-hp-debug+ #203
Tainted: [E]=UNSIGNED_MODULE
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
Call Trace:
 <TASK>
 dump_stack_lvl+0x27/0x90
 dump_stack+0x10/0x20
 panic+0x33e/0x380
 pci_error_detected+0x6d/0x70 [cxl_core]
 report_error_detected+0xc2/0x180
 ? __pm_runtime_resume+0x60/0x90
 ? __pfx_report_frozen_detected+0x10/0x10
 report_frozen_detected+0x16/0x20
 __pci_walk_bus+0x50/0x70
 ? __pfx_report_frozen_detected+0x10/0x10
 pci_walk_bus+0x32/0x50
 pci_walk_bridge+0x1d/0x40
 pcie_do_recovery+0x175/0x2b0
 ? __pfx_aer_root_reset+0x10/0x10
 aer_isr_one_error.isra.0+0x656/0x720
 ? srso_return_thunk+0x5/0x5f
 ? _raw_spin_unlock+0x19/0x40
 ? srso_return_thunk+0x5/0x5f
 ? __switch_to+0x115/0x420
 ? srso_return_thunk+0x5/0x5f
 ? __schedule+0x4d1/0x1190
 aer_isr+0x4d/0x80
 irq_thread_fn+0x28/0x70
 irq_thread+0x179/0x240
 ? srso_return_thunk+0x5/0x5f
 ? __pfx_irq_thread_fn+0x10/0x10
 ? __pfx_irq_thread_dtor+0x10/0x10
 kthread+0x10b/0x220
 ? __pfx_irq_thread+0x10/0x10
 ? _raw_spin_unlock_irq+0x1f/0x40
 ? __pfx_kthread+0x10/0x10
 ret_from_fork+0x3c/0x60
 ? __pfx_kthread+0x10/0x10
 ret_from_fork_asm+0x1a/0x30
 </TASK>
Kernel Offset: 0x32200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
---[ end Kernel panic - not syncing: CXL cachemem error. ]---

Changes
=======
 Changes in v7 -> v8:
 [Dan] Use kfifo. Move handling to CXL driver. AER forwards error to CXL
 driver
 [Dan] Add device reference incrementors where needed throughout
 [Dan] Initiate CXL Port RAS init from Switch Port and Endpoint Port init 
 [Dan] Combine CXL Port and CXL Endpoint trace routine
 [Dan] Introduce aer_info::is_cxl. Use to indicate CXL or PCI errors
 [Jonathan] Add serial number for all devices in trace
 [DaveJ] Move find_cxl_port() change into patch using it
 [Terry] Move CXL Port RAS init into cxl/port.c
 [Terry] Moved kfifo functions into cxl/core/ras.c 
 
 Changes in v6 -> v7:
 [Terry] Move updated trace routine call to later patch. Was causing build
 error.
 
 Changes in v5 -> v6:
 [Ira] Move pcie_is_cxl(dev) define to a inline function
 [Ira] Update returning value from pcie_is_cxl_port() to bool w/o cast
 [Ira] Change cxl_report_error_detected() cleanup to return correct bool
 [Ira] Introduce and use PCI_ERS_RESULT_PANIC
 [Ira] Reuse comment for PCIe and CXL recovery paths
 [Jonathan] Add type check in for cxl_handle_cor_ras() and cxl_handle_ras()
 [Jonathan] cxl_uport/dport_init_ras_reporting(), added a mutex.
 [Jonathan] Add logging example to patches updating trace output
 [Jonathan] Make parameter 'const' to eliminate for cast in match_uport()
 [Jonathan] Use __free() in cxl_pci_port_ras()
 [Terry] Add patch to log the PCIe SBDF along with CXL device name
 [Terry] Add patch to handle CXL endpoint and RCH DP errors as CXL errors
 [Terry] Remove patch w USP UCE fatal support @ aer_get_device_error_info()
 [Terry] Rebase to cxl/next commit 5585e342e8d3 ("cxl/memdev: Remove unused partition values")
 [Gregory] Pre-initialize pointer to NULL in cxl_pci_port_ras()
 [Gregory] Move AER driver bus name detection to a static function

 Changes in v4 -> v5:
 [Alejandro] Refactor cxl_walk_bridge to simplify 'status' variable usage
 [Alejandro] Add WARN_ONCE() in __cxl_handle_ras() and cxl_handle_cor_ras()
 [Ming] Remove unnecessary NULL check in cxl_pci_port_ras()
 [Terry] Add failure check for call to to_cxl_port() in cxl_pci_port_ras()
 [Ming] Use port->dev for call to devm_add_action_or_reset() in
 cxl_dport_init_ras_reporting() and cxl_uport_init_ras_reporting()
 [Jonathan] Use get_device()/put_device() to prevent race condition in
 cxl_clear_port_error_handlers() and cxl_clear_port_error_handlers()
 [Terry] Commit message cleanup. Capitalize keywords from CXL and PCI
 specifications

 Changes in v3 -> v4:
 [Lukas] Capitalize PCIe and CXL device names as in specifications
 [Lukas] Move call to pcie_is_cxl() into cxl_port_devsec()
 [Lukas] Correct namespace spelling
 [Lukas] Removed export from pcie_is_cxl_port()
 [Lukas] Simplify 'if' blocks in cxl_handle_error()
 [Lukas] Change panic message to remove redundant 'panic' text
 [Ming] Update to call cxl_dport_init_ras_reporting() in RCH case
 [lkp@intel] 'host' parameter is already removed. Remove parameter description too.
 [Terry] Added field description for cxl_err_handlers in pci.h comment block

 Changes in v1 -> v2:
 [Jonathan] Remove extra NULL check and cleanup in cxl_pci_port_ras()
 [Jonathan] Update description to DSP map patch description
 [Jonathan] Update cxl_pci_port_ras() to check for NULL port
 [Jonathan] Dont call handler before handler port changes are present (patch order)
 [Bjorn] Fix linebreak in cover sheet URL
 [Bjorn] Remove timestamps from test logs in cover sheet
 [Bjorn] Retitle AER commits to use "PCI/AER:"
 [Bjorn] Retitle patch#3 to use renaming instead of refactoring
 [Bjorn] Fix base commit-id on cover sheet
 [Bjorn] Add VH spec reference/citation
 [Terry] Removed last 2 patches to enable internal errors. Is not needed
 because internal errors are enabled in AER driver.
 [Dan] Create cxl_do_recovery() and pci_driver::cxl_err_handlers.
 [Dan] Use kernel panic in CXL recovery
 [Dan] cxl_port_hndlrs -> cxl_port_error_handlers

Terry Bowman (16):
  PCI/CXL: Introduce PCIe helper function pcie_is_cxl()
  PCI/AER: Modify AER driver logging to report CXL or PCIe bus error
    type
  CXL/AER: Introduce Kfifo for forwarding CXL errors
  cxl/aer: AER service driver forwards CXL error to CXL driver
  PCI/AER: CXL driver dequeues CXL error forwarded from AER service
    driver
  CXL/PCI: Introduce CXL uncorrectable protocol error 'recovery'
  cxl/pci: Move existing CXL RAS initialization to CXL's cxl_port driver
  cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers
  cxl/pci: Update RAS handler interfaces to also support CXL PCIe Ports
  cxl/pci: Add log message if RAS registers are not mapped
  cxl/pci: Unifi CXL trace logging for CXL Endpoints and CXL Ports
  cxl/pci: Assign CXL Port protocol error handlers
  cxl/pci: Assign CXL Endpoint protocol error handlers
  cxl/pci: Remove unnecessary CXL Endpoint handling helper functions
  CXL/PCI: Enable CXL protocol errors during CXL Port probe
  CXL/PCI: Disable CXL protocol errors during CXL Port cleanup

 drivers/cxl/core/core.h       |   2 +
 drivers/cxl/core/pci.c        | 195 +++++++++-------------
 drivers/cxl/core/port.c       |   4 +-
 drivers/cxl/core/ras.c        | 304 +++++++++++++++++++++++++++++++++-
 drivers/cxl/core/regs.c       |   2 +
 drivers/cxl/core/trace.h      | 100 ++++-------
 drivers/cxl/cxl.h             |  37 +++++
 drivers/cxl/cxlpci.h          |   7 +-
 drivers/cxl/mem.c             |   3 +-
 drivers/cxl/pci.c             |   8 +-
 drivers/cxl/port.c            | 202 ++++++++++++++++++++++
 drivers/pci/pci.c             |   6 +
 drivers/pci/pci.h             |  14 +-
 drivers/pci/pcie/aer.c        | 180 ++++++++++++++------
 drivers/pci/pcie/rcec.c       |   1 +
 drivers/pci/probe.c           |  10 ++
 include/linux/aer.h           |  41 +++++
 include/linux/pci.h           |  18 ++
 include/ras/ras_event.h       |   9 +-
 include/uapi/linux/pci_regs.h |   8 +-
 20 files changed, 886 insertions(+), 265 deletions(-)


base-commit: aae0594a7053c60b82621136257c8b648c67b512

Comments

Bjorn Helgaas March 27, 2025, 5:16 p.m. UTC | #1
On Wed, Mar 26, 2025 at 08:47:01PM -0500, Terry Bowman wrote:
> ...

> Terry Bowman (16):
>   PCI/CXL: Introduce PCIe helper function pcie_is_cxl()

Something like "Add pcie_is_cxl()" is probably enough.

>   PCI/AER: Modify AER driver logging to report CXL or PCIe bus error
>     type

No need to repeat "AER" in the subject.  Could start with "Report" or
"Distinguish" since "modify AER driver logging" is kind of low-value
information.

>   CXL/AER: Introduce Kfifo for forwarding CXL errors

>   cxl/aer: AER service driver forwards CXL error to CXL driver
>   PCI/AER: CXL driver dequeues CXL error forwarded from AER service
>     driver

Both should say what the patch changes.  "AER service driver forwards"
and "CXL driver dequeues" could be descriptions of existing behavior
or something else.  Starting with a verb will help make this clearer.

Maybe don't need to repeat "AER" in "CXL/AER: AER ..."

>   CXL/PCI: Introduce CXL uncorrectable protocol error 'recovery'
>   cxl/pci: Move existing CXL RAS initialization to CXL's cxl_port driver

Drop "existing" and at least one "CXL" to increase information density
in subject.

>   cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers
>   cxl/pci: Update RAS handler interfaces to also support CXL PCIe Ports
>   cxl/pci: Add log message if RAS registers are not mapped
>   cxl/pci: Unifi CXL trace logging for CXL Endpoints and CXL Ports

s/Unifi/Unify/

>   cxl/pci: Assign CXL Port protocol error handlers
>   cxl/pci: Assign CXL Endpoint protocol error handlers
>   cxl/pci: Remove unnecessary CXL Endpoint handling helper functions
>   CXL/PCI: Enable CXL protocol errors during CXL Port probe
>   CXL/PCI: Disable CXL protocol errors during CXL Port cleanup

Don't repost just for any of this, but it looks like there are some
kernel test robot warnings that need to be addressed.  When you do,
tidy up these subject lines so they are capitalized consistently.
Bowman, Terry March 27, 2025, 10:04 p.m. UTC | #2
On 3/27/2025 12:16 PM, Bjorn Helgaas wrote:
> On Wed, Mar 26, 2025 at 08:47:01PM -0500, Terry Bowman wrote:
>> ...
>> Terry Bowman (16):
>>   PCI/CXL: Introduce PCIe helper function pcie_is_cxl()
> Something like "Add pcie_is_cxl()" is probably enough.
>
>>   PCI/AER: Modify AER driver logging to report CXL or PCIe bus error
>>     type
> No need to repeat "AER" in the subject.  Could start with "Report" or
> "Distinguish" since "modify AER driver logging" is kind of low-value
> information.
>
>>   CXL/AER: Introduce Kfifo for forwarding CXL errors
>>   cxl/aer: AER service driver forwards CXL error to CXL driver
>>   PCI/AER: CXL driver dequeues CXL error forwarded from AER service
>>     driver
> Both should say what the patch changes.  "AER service driver forwards"
> and "CXL driver dequeues" could be descriptions of existing behavior
> or something else.  Starting with a verb will help make this clearer.
>
> Maybe don't need to repeat "AER" in "CXL/AER: AER ..."
>
>>   CXL/PCI: Introduce CXL uncorrectable protocol error 'recovery'
>>   cxl/pci: Move existing CXL RAS initialization to CXL's cxl_port driver
> Drop "existing" and at least one "CXL" to increase information density
> in subject.
>
>>   cxl/pci: Map CXL Endpoint Port and CXL Switch Port RAS registers
>>   cxl/pci: Update RAS handler interfaces to also support CXL PCIe Ports
>>   cxl/pci: Add log message if RAS registers are not mapped
>>   cxl/pci: Unifi CXL trace logging for CXL Endpoints and CXL Ports
> s/Unifi/Unify/
>
>>   cxl/pci: Assign CXL Port protocol error handlers
>>   cxl/pci: Assign CXL Endpoint protocol error handlers
>>   cxl/pci: Remove unnecessary CXL Endpoint handling helper functions
>>   CXL/PCI: Enable CXL protocol errors during CXL Port probe
>>   CXL/PCI: Disable CXL protocol errors during CXL Port cleanup
> Don't repost just for any of this, but it looks like there are some
> kernel test robot warnings that need to be addressed.  When you do,
> tidy up these subject lines so they are capitalized consistently.
Hi Bjorn,

I added all the changes. The commit titles read much better.

Terry