mbox series

[v6,0/6] Improve PCI memory mapping API

Message ID 20241012113246.95634-1-dlemoal@kernel.org (mailing list archive)
Headers show
Series Improve PCI memory mapping API | expand

Message

Damien Le Moal Oct. 12, 2024, 11:32 a.m. UTC
This series introduces the new functions pci_epc_mem_map() and
pci_epc_mem_unmap() to improve handling of the PCI address mapping
alignment constraints of endpoint controllers in a controller
independent manner.

The issue fixed is that the fixed alignment defined by the "align" field
of struct pci_epc_features is defined for inbound ATU entries (e.g.
BARs) and is a fixed value, whereas some controllers need a PCI address
alignment that depends on the PCI address itself. For instance, the
rk3399 SoC controller in endpoint mode uses the lower bits of the local
endpoint memory address as the lower bits for the PCI addresses for data
transfers. That is, when mapping local memory, one must take into
account the number of bits of the RC PCI address that change from the
start address of the mapping.

To fix this, the new endpoint controller method .align_addr is
introduced and called from the new pci_epc_mem_map() function. This
method is optional and for controllers that do not define it, it is
assumed that the controller has no PCI address constraint.

The functions pci_epc_mem_map() is a helper function which obtains
the mapping information, allocates endpoint controller memory according
to the mapping size obtained and maps the memory. pci_epc_mem_unmap()
unmaps and frees the endpoint memory.

This series is organized as follows:
 - Patch 1 introduces a small helper to clean up the epc core code
 - Patch 2 improves pci_epc_mem_alloc_addr()
 - Patch 3 introduce the new align_addr() endpoint controller method
   and the epc functions pci_epc_mem_map() and pci_epc_mem_unmap().
 - Patch 4 documents these new functions.
 - Patch 5 modifies the test endpoint function driver to use 
   pci_epc_mem_map() and pci_epc_mem_unmap() to illustrate the use of
   these functions.
 - Finally, patch 6 implements the RK3588 endpoint controller driver
   .align_addr() operation to satisfy that controller PCI address
   alignment constraint.

This patch series was tested using the pci endpoint test driver (as-is
and a modified version removing memory allocation alignment on the host
side) as well as with a prototype NVMe endpoint function driver (where
data transfers use addresses that are never aligned to any specific
boundary).

Changes from v5:
 - Changed patch 3 to rename the new controller operation to align_addr
   and change its interface. Patch 6 is changed accordingly.

Changes from v4:
 - Removed the patch adding the pci_epc_map_align() function. The former
   .map_align controller operation is now introduced in patch 3 as
   "get_mem_map()" and used directly in the new pci_epf_mem_map()
   function.
 - Modified the documentation patch 4 to reflect the previous change.
 - Changed patch 6 title and modified it to rename map_align to
   get_mem_map in accordance with the new patch 3.
 - Added review tags

Changes from v3:
 - Addressed Niklas comments (improved patch 2 commit message, added
   comments in the pci_epc_map_align() function in patch 3, typos and
   improvements in patch 5, patch 7 commit message).
 - Added review tags

Changes from v2:
 - Dropped all patches for the rockchip-ep. These patches will be sent
   later as a separate series.
 - Added patch 2 and 5
 - Added review tags to patch 1

Changes from v1:
 - Changed pci_epc_check_func() to pci_epc_function_is_valid() in patch
   1.
 - Removed patch "PCI: endpoint: Improve pci_epc_mem_alloc_addr()"
   (former patch 2 of v1)
 - Various typos cleanups all over. Also fixed some blank space
   indentation.
 - Added review tags

Damien Le Moal (6):
  PCI: endpoint: Introduce pci_epc_function_is_valid()
  PCI: endpoint: Improve pci_epc_mem_alloc_addr()
  PCI: endpoint: Introduce pci_epc_mem_map()/unmap()
  PCI: endpoint: Update documentation
  PCI: endpoint: test: Use pci_epc_mem_map/unmap()
  PCI: dwc: endpoint: Implement the pci_epc_ops::align_addr() operation

 Documentation/PCI/endpoint/pci-endpoint.rst   |  29 ++
 .../pci/controller/dwc/pcie-designware-ep.c   |  15 +
 drivers/pci/endpoint/functions/pci-epf-test.c | 372 +++++++++---------
 drivers/pci/endpoint/pci-epc-core.c           | 182 ++++++---
 drivers/pci/endpoint/pci-epc-mem.c            |   9 +-
 include/linux/pci-epc.h                       |  38 ++
 6 files changed, 415 insertions(+), 230 deletions(-)

Comments

Manivannan Sadhasivam Oct. 12, 2024, 11:57 a.m. UTC | #1
On Sat, Oct 12, 2024 at 08:32:40PM +0900, Damien Le Moal wrote:
> This series introduces the new functions pci_epc_mem_map() and
> pci_epc_mem_unmap() to improve handling of the PCI address mapping
> alignment constraints of endpoint controllers in a controller
> independent manner.
> 
> The issue fixed is that the fixed alignment defined by the "align" field
> of struct pci_epc_features is defined for inbound ATU entries (e.g.
> BARs) and is a fixed value, whereas some controllers need a PCI address
> alignment that depends on the PCI address itself. For instance, the
> rk3399 SoC controller in endpoint mode uses the lower bits of the local
> endpoint memory address as the lower bits for the PCI addresses for data
> transfers. That is, when mapping local memory, one must take into
> account the number of bits of the RC PCI address that change from the
> start address of the mapping.
> 
> To fix this, the new endpoint controller method .align_addr is
> introduced and called from the new pci_epc_mem_map() function. This
> method is optional and for controllers that do not define it, it is
> assumed that the controller has no PCI address constraint.
> 
> The functions pci_epc_mem_map() is a helper function which obtains
> the mapping information, allocates endpoint controller memory according
> to the mapping size obtained and maps the memory. pci_epc_mem_unmap()
> unmaps and frees the endpoint memory.
> 
> This series is organized as follows:
>  - Patch 1 introduces a small helper to clean up the epc core code
>  - Patch 2 improves pci_epc_mem_alloc_addr()
>  - Patch 3 introduce the new align_addr() endpoint controller method
>    and the epc functions pci_epc_mem_map() and pci_epc_mem_unmap().
>  - Patch 4 documents these new functions.
>  - Patch 5 modifies the test endpoint function driver to use 
>    pci_epc_mem_map() and pci_epc_mem_unmap() to illustrate the use of
>    these functions.
>  - Finally, patch 6 implements the RK3588 endpoint controller driver
>    .align_addr() operation to satisfy that controller PCI address
>    alignment constraint.
> 
> This patch series was tested using the pci endpoint test driver (as-is
> and a modified version removing memory allocation alignment on the host
> side) as well as with a prototype NVMe endpoint function driver (where
> data transfers use addresses that are never aligned to any specific
> boundary).
> 

Applied to pci/endpoint!

- Mani

> Changes from v5:
>  - Changed patch 3 to rename the new controller operation to align_addr
>    and change its interface. Patch 6 is changed accordingly.
> 
> Changes from v4:
>  - Removed the patch adding the pci_epc_map_align() function. The former
>    .map_align controller operation is now introduced in patch 3 as
>    "get_mem_map()" and used directly in the new pci_epf_mem_map()
>    function.
>  - Modified the documentation patch 4 to reflect the previous change.
>  - Changed patch 6 title and modified it to rename map_align to
>    get_mem_map in accordance with the new patch 3.
>  - Added review tags
> 
> Changes from v3:
>  - Addressed Niklas comments (improved patch 2 commit message, added
>    comments in the pci_epc_map_align() function in patch 3, typos and
>    improvements in patch 5, patch 7 commit message).
>  - Added review tags
> 
> Changes from v2:
>  - Dropped all patches for the rockchip-ep. These patches will be sent
>    later as a separate series.
>  - Added patch 2 and 5
>  - Added review tags to patch 1
> 
> Changes from v1:
>  - Changed pci_epc_check_func() to pci_epc_function_is_valid() in patch
>    1.
>  - Removed patch "PCI: endpoint: Improve pci_epc_mem_alloc_addr()"
>    (former patch 2 of v1)
>  - Various typos cleanups all over. Also fixed some blank space
>    indentation.
>  - Added review tags
> 
> Damien Le Moal (6):
>   PCI: endpoint: Introduce pci_epc_function_is_valid()
>   PCI: endpoint: Improve pci_epc_mem_alloc_addr()
>   PCI: endpoint: Introduce pci_epc_mem_map()/unmap()
>   PCI: endpoint: Update documentation
>   PCI: endpoint: test: Use pci_epc_mem_map/unmap()
>   PCI: dwc: endpoint: Implement the pci_epc_ops::align_addr() operation
> 
>  Documentation/PCI/endpoint/pci-endpoint.rst   |  29 ++
>  .../pci/controller/dwc/pcie-designware-ep.c   |  15 +
>  drivers/pci/endpoint/functions/pci-epf-test.c | 372 +++++++++---------
>  drivers/pci/endpoint/pci-epc-core.c           | 182 ++++++---
>  drivers/pci/endpoint/pci-epc-mem.c            |   9 +-
>  include/linux/pci-epc.h                       |  38 ++
>  6 files changed, 415 insertions(+), 230 deletions(-)
> 
> -- 
> 2.47.0
>
Damien Le Moal Oct. 12, 2024, 12:03 p.m. UTC | #2
On 10/12/24 20:57, Manivannan Sadhasivam wrote:
> On Sat, Oct 12, 2024 at 08:32:40PM +0900, Damien Le Moal wrote:
>> This series introduces the new functions pci_epc_mem_map() and
>> pci_epc_mem_unmap() to improve handling of the PCI address mapping
>> alignment constraints of endpoint controllers in a controller
>> independent manner.
>>
>> The issue fixed is that the fixed alignment defined by the "align" field
>> of struct pci_epc_features is defined for inbound ATU entries (e.g.
>> BARs) and is a fixed value, whereas some controllers need a PCI address
>> alignment that depends on the PCI address itself. For instance, the
>> rk3399 SoC controller in endpoint mode uses the lower bits of the local
>> endpoint memory address as the lower bits for the PCI addresses for data
>> transfers. That is, when mapping local memory, one must take into
>> account the number of bits of the RC PCI address that change from the
>> start address of the mapping.
>>
>> To fix this, the new endpoint controller method .align_addr is
>> introduced and called from the new pci_epc_mem_map() function. This
>> method is optional and for controllers that do not define it, it is
>> assumed that the controller has no PCI address constraint.
>>
>> The functions pci_epc_mem_map() is a helper function which obtains
>> the mapping information, allocates endpoint controller memory according
>> to the mapping size obtained and maps the memory. pci_epc_mem_unmap()
>> unmaps and frees the endpoint memory.
>>
>> This series is organized as follows:
>>  - Patch 1 introduces a small helper to clean up the epc core code
>>  - Patch 2 improves pci_epc_mem_alloc_addr()
>>  - Patch 3 introduce the new align_addr() endpoint controller method
>>    and the epc functions pci_epc_mem_map() and pci_epc_mem_unmap().
>>  - Patch 4 documents these new functions.
>>  - Patch 5 modifies the test endpoint function driver to use 
>>    pci_epc_mem_map() and pci_epc_mem_unmap() to illustrate the use of
>>    these functions.
>>  - Finally, patch 6 implements the RK3588 endpoint controller driver
>>    .align_addr() operation to satisfy that controller PCI address
>>    alignment constraint.
>>
>> This patch series was tested using the pci endpoint test driver (as-is
>> and a modified version removing memory allocation alignment on the host
>> side) as well as with a prototype NVMe endpoint function driver (where
>> data transfers use addresses that are never aligned to any specific
>> boundary).
>>
> 
> Applied to pci/endpoint!

Awesome ! Thanks a lot !