| Message ID | 20250131182949.465530-9-cassel@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | PCI: endpoint: Allow EPF drivers to configure the size of Resizable BARs | expand |
On Fri, Jan 31, 2025 at 07:29:49PM +0100, Niklas Cassel wrote: > The PCI endpoint framework currently does not support resizable BARs. > > Add a new BAR type BAR_RESIZABLE, so that EPC drivers can support resizable > BARs properly. > > For a resizable BAR, we will only allow a single supported size. > This is by design, as we do not need/want the complexity of the host side > resizing our resizable BAR. > > In the DWC driver specifically, the DWC driver currently handles resizable > BARs using an ugly hack where a resizable BAR is force set to a fixed size > BAR with 1 MB size if detected. This is bogus, as a resizable BAR can be > configured to sizes other than 1 MB. > > With these changes, an EPF driver will be able to call pci_epc_set_bar() > to configure a resizable BAR to an arbitrary size, just like for > BAR_PROGRAMMABLE. Thus, DWC based EPF drivers will no longer be forced to > a bogus 1 MB forced size for resizable BARs. > > > Tested/verified on a Radxa Rock 5b (rk3588) by: > -Modifying pci-epf-test.c to request BAR sizes that are larger than 1 MB: > -static size_t bar_size[] = { 512, 512, 1024, 16384, 131072, 1048576 }; > +static size_t bar_size[] = { SZ_1M, SZ_1M, SZ_2M, SZ_2M, SZ_4M, SZ_4M }; > (Make sure to set CONFIG_CMA_ALIGNMENT=10 such that dma_alloc_coherent() > calls are aligned even for allocations larger than 1 MB.) > -Rebooting the host to make sure that the DWC EP driver configures the BARs > correctly after receiving a link down event. > -Modifying EPC features to configure a BAR as 64-bit, to make sure that we > handle 64-bit BARs correctly. > -Modifying the DWC EP driver to set a size larger than 2 GB, to make sure > we handle BAR sizes larger than 2 GB (for 64-bit BARs) correctly. > -Running the consecutive BAR test in pci_endpoint_test.c to make sure that > the address translation works correctly. > > > Changes since V3: > -Picked up tags. > -Addressed comments from Mani. > > > Kind regards, > Niklas > > Niklas Cassel (7): > PCI: endpoint: Allow EPF drivers to configure the size of Resizable > BARs > PCI: endpoint: Add pci_epc_bar_size_to_rebar_cap() > PCI: dwc: ep: Move dw_pcie_ep_find_ext_capability() > PCI: dwc: endpoint: Allow EPF drivers to configure the size of > Resizable BARs > PCI: keystone: Describe Resizable BARs as Resizable BARs > PCI: keystone: Specify correct alignment requirement > PCI: dw-rockchip: Describe Resizable BARs as Resizable BARs > > drivers/pci/controller/dwc/pci-keystone.c | 6 +- > .../pci/controller/dwc/pcie-designware-ep.c | 218 +++++++++++++++--- > drivers/pci/controller/dwc/pcie-dw-rockchip.c | 22 +- > drivers/pci/endpoint/pci-epc-core.c | 31 +++ > drivers/pci/endpoint/pci-epf-core.c | 4 + > include/linux/pci-epc.h | 5 + > 6 files changed, 239 insertions(+), 47 deletions(-) > > -- > 2.48.1 > Considering that all patches are have at least one R-b tag, any chance this series could get picked up? Kind regards, Niklas
On Fri, Jan 31, 2025 at 07:29:49PM +0100, Niklas Cassel wrote: > The PCI endpoint framework currently does not support resizable BARs. > > Add a new BAR type BAR_RESIZABLE, so that EPC drivers can support resizable > BARs properly. > > For a resizable BAR, we will only allow a single supported size. > This is by design, as we do not need/want the complexity of the host side > resizing our resizable BAR. > > In the DWC driver specifically, the DWC driver currently handles resizable > BARs using an ugly hack where a resizable BAR is force set to a fixed size > BAR with 1 MB size if detected. This is bogus, as a resizable BAR can be > configured to sizes other than 1 MB. > > With these changes, an EPF driver will be able to call pci_epc_set_bar() > to configure a resizable BAR to an arbitrary size, just like for > BAR_PROGRAMMABLE. Thus, DWC based EPF drivers will no longer be forced to > a bogus 1 MB forced size for resizable BARs. > > > Tested/verified on a Radxa Rock 5b (rk3588) by: > -Modifying pci-epf-test.c to request BAR sizes that are larger than 1 MB: > -static size_t bar_size[] = { 512, 512, 1024, 16384, 131072, 1048576 }; > +static size_t bar_size[] = { SZ_1M, SZ_1M, SZ_2M, SZ_2M, SZ_4M, SZ_4M }; > (Make sure to set CONFIG_CMA_ALIGNMENT=10 such that dma_alloc_coherent() > calls are aligned even for allocations larger than 1 MB.) > -Rebooting the host to make sure that the DWC EP driver configures the BARs > correctly after receiving a link down event. > -Modifying EPC features to configure a BAR as 64-bit, to make sure that we > handle 64-bit BARs correctly. > -Modifying the DWC EP driver to set a size larger than 2 GB, to make sure > we handle BAR sizes larger than 2 GB (for 64-bit BARs) correctly. > -Running the consecutive BAR test in pci_endpoint_test.c to make sure that > the address translation works correctly. > I took the liberty since this series is mostly related to endpoint and applied to pci/endpoint! - Mani > > Changes since V3: > -Picked up tags. > -Addressed comments from Mani. > > > Kind regards, > Niklas > > Niklas Cassel (7): > PCI: endpoint: Allow EPF drivers to configure the size of Resizable > BARs > PCI: endpoint: Add pci_epc_bar_size_to_rebar_cap() > PCI: dwc: ep: Move dw_pcie_ep_find_ext_capability() > PCI: dwc: endpoint: Allow EPF drivers to configure the size of > Resizable BARs > PCI: keystone: Describe Resizable BARs as Resizable BARs > PCI: keystone: Specify correct alignment requirement > PCI: dw-rockchip: Describe Resizable BARs as Resizable BARs > > drivers/pci/controller/dwc/pci-keystone.c | 6 +- > .../pci/controller/dwc/pcie-designware-ep.c | 218 +++++++++++++++--- > drivers/pci/controller/dwc/pcie-dw-rockchip.c | 22 +- > drivers/pci/endpoint/pci-epc-core.c | 31 +++ > drivers/pci/endpoint/pci-epf-core.c | 4 + > include/linux/pci-epc.h | 5 + > 6 files changed, 239 insertions(+), 47 deletions(-) > > -- > 2.48.1 >
The PCI endpoint framework currently does not support resizable BARs. Add a new BAR type BAR_RESIZABLE, so that EPC drivers can support resizable BARs properly. For a resizable BAR, we will only allow a single supported size. This is by design, as we do not need/want the complexity of the host side resizing our resizable BAR. In the DWC driver specifically, the DWC driver currently handles resizable BARs using an ugly hack where a resizable BAR is force set to a fixed size BAR with 1 MB size if detected. This is bogus, as a resizable BAR can be configured to sizes other than 1 MB. With these changes, an EPF driver will be able to call pci_epc_set_bar() to configure a resizable BAR to an arbitrary size, just like for BAR_PROGRAMMABLE. Thus, DWC based EPF drivers will no longer be forced to a bogus 1 MB forced size for resizable BARs. Tested/verified on a Radxa Rock 5b (rk3588) by: -Modifying pci-epf-test.c to request BAR sizes that are larger than 1 MB: -static size_t bar_size[] = { 512, 512, 1024, 16384, 131072, 1048576 }; +static size_t bar_size[] = { SZ_1M, SZ_1M, SZ_2M, SZ_2M, SZ_4M, SZ_4M }; (Make sure to set CONFIG_CMA_ALIGNMENT=10 such that dma_alloc_coherent() calls are aligned even for allocations larger than 1 MB.) -Rebooting the host to make sure that the DWC EP driver configures the BARs correctly after receiving a link down event. -Modifying EPC features to configure a BAR as 64-bit, to make sure that we handle 64-bit BARs correctly. -Modifying the DWC EP driver to set a size larger than 2 GB, to make sure we handle BAR sizes larger than 2 GB (for 64-bit BARs) correctly. -Running the consecutive BAR test in pci_endpoint_test.c to make sure that the address translation works correctly. Changes since V3: -Picked up tags. -Addressed comments from Mani. Kind regards, Niklas Niklas Cassel (7): PCI: endpoint: Allow EPF drivers to configure the size of Resizable BARs PCI: endpoint: Add pci_epc_bar_size_to_rebar_cap() PCI: dwc: ep: Move dw_pcie_ep_find_ext_capability() PCI: dwc: endpoint: Allow EPF drivers to configure the size of Resizable BARs PCI: keystone: Describe Resizable BARs as Resizable BARs PCI: keystone: Specify correct alignment requirement PCI: dw-rockchip: Describe Resizable BARs as Resizable BARs drivers/pci/controller/dwc/pci-keystone.c | 6 +- .../pci/controller/dwc/pcie-designware-ep.c | 218 +++++++++++++++--- drivers/pci/controller/dwc/pcie-dw-rockchip.c | 22 +- drivers/pci/endpoint/pci-epc-core.c | 31 +++ drivers/pci/endpoint/pci-epf-core.c | 4 + include/linux/pci-epc.h | 5 + 6 files changed, 239 insertions(+), 47 deletions(-)