diff mbox series

[v3,17/17] Documentation: PCI: Add userguide for PCI endpoint NTB function

Message ID 20200904075052.8911-18-kishon@ti.com
State Superseded
Delegated to: Lorenzo Pieralisi
Headers show
Series Implement NTB Controller using multiple PCI EP | expand

Commit Message

Kishon Vijay Abraham I Sept. 4, 2020, 7:50 a.m. UTC
Add documentation to help users use pci-epf-ntb function driver and
existing host side NTB infrastructure for NTB functionality.

Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
---
 Documentation/PCI/endpoint/index.rst         |   1 +
 Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 +++++++++++++++++++
 2 files changed, 161 insertions(+)
 create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst

Comments

Randy Dunlap Sept. 5, 2020, 10:08 p.m. UTC | #1
On 9/4/20 12:50 AM, Kishon Vijay Abraham I wrote:
> Add documentation to help users use pci-epf-ntb function driver and
> existing host side NTB infrastructure for NTB functionality.
> 
> Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
> ---
>  Documentation/PCI/endpoint/index.rst         |   1 +
>  Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 +++++++++++++++++++
>  2 files changed, 161 insertions(+)
>  create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst

Hi,
There are a few edits below:


> diff --git a/Documentation/PCI/endpoint/pci-ntb-howto.rst b/Documentation/PCI/endpoint/pci-ntb-howto.rst
> new file mode 100644
> index 000000000000..2fbb0a051c3b
> --- /dev/null
> +++ b/Documentation/PCI/endpoint/pci-ntb-howto.rst
> @@ -0,0 +1,160 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======================
> +PCI NTB EPF User Guide
> +======================
> +
> +:Author: Kishon Vijay Abraham I <kishon@ti.com>
> +
> +This document is a guide to help users use pci-epf-ntb function driver
> +and ntb_hw_epf host driver for NTB functionality. The list of steps to
> +be followed in the host side and EP side is given below. For the hardware
> +configuration and internals of NTB using configurable endpoints see
> +Documentation/PCI/endpoint/pci-ntb-function.rst
> +
> +Endpoint Device
> +===============
> +
> +Endpoint Controller Devices
> +---------------------------
> +
> +For implementing NTB functionality atleast two endpoint controller devices

                                      at least

> +are required.
> +To find the list of endpoint controller devices in the system::
> +
> +        # ls /sys/class/pci_epc/
> +          2900000.pcie-ep  2910000.pcie-ep
> +
> +If PCI_ENDPOINT_CONFIGFS is enabled::
> +
> +	# ls /sys/kernel/config/pci_ep/controllers
> +	  2900000.pcie-ep  2910000.pcie-ep
> +
> +
> +Endpoint Function Drivers
> +-------------------------
> +
> +To find the list of endpoint function drivers in the system::
> +
> +	# ls /sys/bus/pci-epf/drivers
> +	  pci_epf_ntb   pci_epf_ntb
> +
> +If PCI_ENDPOINT_CONFIGFS is enabled::
> +
> +	# ls /sys/kernel/config/pci_ep/functions
> +	  pci_epf_ntb   pci_epf_ntb
> +
> +
> +Creating pci-epf-ntb Device
> +----------------------------
> +
> +PCI endpoint function device can be created using the configfs. To create
> +pci-epf-ntb device, the following commands can be used::
> +
> +	# mount -t configfs none /sys/kernel/config
> +	# cd /sys/kernel/config/pci_ep/
> +	# mkdir functions/pci_epf_ntb/func1
> +
> +The "mkdir func1" above creates the pci-epf-ntb function device that will
> +be probed by pci_epf_ntb driver.
> +
> +The PCI endpoint framework populates the directory with the following
> +configurable fields::
> +
> +	# ls functions/pci_epf_ntb/func1
> +          baseclass_code    deviceid          msi_interrupts    pci-epf-ntb.0
> +          progif_code       secondary         subsys_id         vendorid
> +          cache_line_size   interrupt_pin     msix_interrupts   primary
> +          revid             subclass_code     subsys_vendor_id
> +
> +The PCI endpoint function driver populates these entries with default values
> +when the device is bound to the driver. The pci-epf-ntb driver populates
> +vendorid with 0xffff and interrupt_pin with 0x0001::
> +
> +	# cat functions/pci_epf_ntb/func1/vendorid
> +	  0xffff
> +	# cat functions/pci_epf_ntb/func1/interrupt_pin
> +	  0x0001
> +
> +
> +Configuring pci-epf-ntb Device
> +-------------------------------
> +
> +The user can configure the pci-epf-ntb device using configfs entry. In order

                                                 using its configfs entry.

> +to change the vendorid and the deviceid, the following
> +commands can be used::
> +
> +	# echo 0x104c > functions/pci_epf_ntb/func1/vendorid
> +	# echo 0xb00d > functions/pci_epf_ntb/func1/deviceid
> +
> +In-order to configure NTB specific attributes, a new sub-directory to func1

   In order

> +should be created::
> +
> +	# mkdir functions/pci_epf_ntb/func1/pci_epf_ntb.0/
> +
> +The NTB function driver will populate this directory with various attributes
> +that can be configured by the user::
> +
> +	# ls functions/pci_epf_ntb/func1/pci_epf_ntb.0/
> +          db_count    mw1         mw2         mw3         mw4         num_mws
> +          spad_count
> +
> +A sample configuration for NTB function is given below::
> +
> +	# echo 4 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/db_count
> +	# echo 128 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/spad_count
> +	# echo 2 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/num_mws
> +	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw1
> +	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw2
> +
> +Binding pci-epf-ntb Device to EP Controller
> +--------------------------------------------
> +
> +NTB function device should be attached to two PCIe endpoint controllers
> +connected to the two hosts. Use the 'primary' and 'secondary' entries
> +inside NTB function device to attach one PCIe endpoint controller to
> +primary interface and the other PCIe endpoint controller to the secondary
> +interface. ::
> +
> +        # ln -s controllers/2900000.pcie-ep/ functions/pci-epf-ntb/func1/primary
> +        # ln -s controllers/2910000.pcie-ep/ functions/pci-epf-ntb/func1/secondary
> +
> +Once the above step is completed, both the PCI endpoint controllers is ready to

                                                                       are ready

> +establish a link with the host.
> +
> +
> +Start the Link
> +--------------
> +
> +In order for the endpoint device to establish a link with the host, the _start_
> +field should be populated with '1'. For NTB, both the PCIe endpoint controllers
> +should establish link with the host::
> +
> +        #echo 1 > controllers/2900000.pcie-ep/start
> +        #echo 1 > controllers/2910000.pcie-ep/start
> +
> +
> +RootComplex Device
> +==================
> +
> +lspci Output
> +------------
> +
> +Note that the devices listed here correspond to the value populated in 1.4

Can you use a section name (or reference) here instead of "1.4"?  I can't see 1.4
when reading with an editor.

> +above::
> +
> +        # lspci
> +        0000:00:00.0 PCI bridge: Texas Instruments Device b00d
> +        0000:01:00.0 RAM memory: Texas Instruments Device b00d
> +
> +
> +Using ntb_hw_epf Device
> +-----------------------
> +
> +The host side software follows the standard NTB software architecture in Linux.
> +All the existing client side NTB utilities like NTB Transport Client and NTB
> +Netdev, NTB Ping Pong Test Client and NTB Tool Test Clientcan be used with NTB

                                                       Client can be

> +function device.
> +
> +For more information on NTB see
> +Documentation/driver-api/ntb.rst


thanks.
Kishon Vijay Abraham I Sept. 14, 2020, 3:36 p.m. UTC | #2
Hi Randy,

On 06/09/20 3:38 am, Randy Dunlap wrote:
> On 9/4/20 12:50 AM, Kishon Vijay Abraham I wrote:
>> Add documentation to help users use pci-epf-ntb function driver and
>> existing host side NTB infrastructure for NTB functionality.
>>
>> Signed-off-by: Kishon Vijay Abraham I <kishon@ti.com>
>> ---
>>  Documentation/PCI/endpoint/index.rst         |   1 +
>>  Documentation/PCI/endpoint/pci-ntb-howto.rst | 160 +++++++++++++++++++
>>  2 files changed, 161 insertions(+)
>>  create mode 100644 Documentation/PCI/endpoint/pci-ntb-howto.rst
> 
> Hi,
> There are a few edits below:

Thanks for reviewing. I'll fix and post a new revision.

Thanks
Kishon

> 
> 
>> diff --git a/Documentation/PCI/endpoint/pci-ntb-howto.rst b/Documentation/PCI/endpoint/pci-ntb-howto.rst
>> new file mode 100644
>> index 000000000000..2fbb0a051c3b
>> --- /dev/null
>> +++ b/Documentation/PCI/endpoint/pci-ntb-howto.rst
>> @@ -0,0 +1,160 @@
>> +.. SPDX-License-Identifier: GPL-2.0
>> +
>> +======================
>> +PCI NTB EPF User Guide
>> +======================
>> +
>> +:Author: Kishon Vijay Abraham I <kishon@ti.com>
>> +
>> +This document is a guide to help users use pci-epf-ntb function driver
>> +and ntb_hw_epf host driver for NTB functionality. The list of steps to
>> +be followed in the host side and EP side is given below. For the hardware
>> +configuration and internals of NTB using configurable endpoints see
>> +Documentation/PCI/endpoint/pci-ntb-function.rst
>> +
>> +Endpoint Device
>> +===============
>> +
>> +Endpoint Controller Devices
>> +---------------------------
>> +
>> +For implementing NTB functionality atleast two endpoint controller devices
> 
>                                       at least
> 
>> +are required.
>> +To find the list of endpoint controller devices in the system::
>> +
>> +        # ls /sys/class/pci_epc/
>> +          2900000.pcie-ep  2910000.pcie-ep
>> +
>> +If PCI_ENDPOINT_CONFIGFS is enabled::
>> +
>> +	# ls /sys/kernel/config/pci_ep/controllers
>> +	  2900000.pcie-ep  2910000.pcie-ep
>> +
>> +
>> +Endpoint Function Drivers
>> +-------------------------
>> +
>> +To find the list of endpoint function drivers in the system::
>> +
>> +	# ls /sys/bus/pci-epf/drivers
>> +	  pci_epf_ntb   pci_epf_ntb
>> +
>> +If PCI_ENDPOINT_CONFIGFS is enabled::
>> +
>> +	# ls /sys/kernel/config/pci_ep/functions
>> +	  pci_epf_ntb   pci_epf_ntb
>> +
>> +
>> +Creating pci-epf-ntb Device
>> +----------------------------
>> +
>> +PCI endpoint function device can be created using the configfs. To create
>> +pci-epf-ntb device, the following commands can be used::
>> +
>> +	# mount -t configfs none /sys/kernel/config
>> +	# cd /sys/kernel/config/pci_ep/
>> +	# mkdir functions/pci_epf_ntb/func1
>> +
>> +The "mkdir func1" above creates the pci-epf-ntb function device that will
>> +be probed by pci_epf_ntb driver.
>> +
>> +The PCI endpoint framework populates the directory with the following
>> +configurable fields::
>> +
>> +	# ls functions/pci_epf_ntb/func1
>> +          baseclass_code    deviceid          msi_interrupts    pci-epf-ntb.0
>> +          progif_code       secondary         subsys_id         vendorid
>> +          cache_line_size   interrupt_pin     msix_interrupts   primary
>> +          revid             subclass_code     subsys_vendor_id
>> +
>> +The PCI endpoint function driver populates these entries with default values
>> +when the device is bound to the driver. The pci-epf-ntb driver populates
>> +vendorid with 0xffff and interrupt_pin with 0x0001::
>> +
>> +	# cat functions/pci_epf_ntb/func1/vendorid
>> +	  0xffff
>> +	# cat functions/pci_epf_ntb/func1/interrupt_pin
>> +	  0x0001
>> +
>> +
>> +Configuring pci-epf-ntb Device
>> +-------------------------------
>> +
>> +The user can configure the pci-epf-ntb device using configfs entry. In order
> 
>                                                  using its configfs entry.
> 
>> +to change the vendorid and the deviceid, the following
>> +commands can be used::
>> +
>> +	# echo 0x104c > functions/pci_epf_ntb/func1/vendorid
>> +	# echo 0xb00d > functions/pci_epf_ntb/func1/deviceid
>> +
>> +In-order to configure NTB specific attributes, a new sub-directory to func1
> 
>    In order
> 
>> +should be created::
>> +
>> +	# mkdir functions/pci_epf_ntb/func1/pci_epf_ntb.0/
>> +
>> +The NTB function driver will populate this directory with various attributes
>> +that can be configured by the user::
>> +
>> +	# ls functions/pci_epf_ntb/func1/pci_epf_ntb.0/
>> +          db_count    mw1         mw2         mw3         mw4         num_mws
>> +          spad_count
>> +
>> +A sample configuration for NTB function is given below::
>> +
>> +	# echo 4 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/db_count
>> +	# echo 128 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/spad_count
>> +	# echo 2 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/num_mws
>> +	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw1
>> +	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw2
>> +
>> +Binding pci-epf-ntb Device to EP Controller
>> +--------------------------------------------
>> +
>> +NTB function device should be attached to two PCIe endpoint controllers
>> +connected to the two hosts. Use the 'primary' and 'secondary' entries
>> +inside NTB function device to attach one PCIe endpoint controller to
>> +primary interface and the other PCIe endpoint controller to the secondary
>> +interface. ::
>> +
>> +        # ln -s controllers/2900000.pcie-ep/ functions/pci-epf-ntb/func1/primary
>> +        # ln -s controllers/2910000.pcie-ep/ functions/pci-epf-ntb/func1/secondary
>> +
>> +Once the above step is completed, both the PCI endpoint controllers is ready to
> 
>                                                                        are ready
> 
>> +establish a link with the host.
>> +
>> +
>> +Start the Link
>> +--------------
>> +
>> +In order for the endpoint device to establish a link with the host, the _start_
>> +field should be populated with '1'. For NTB, both the PCIe endpoint controllers
>> +should establish link with the host::
>> +
>> +        #echo 1 > controllers/2900000.pcie-ep/start
>> +        #echo 1 > controllers/2910000.pcie-ep/start
>> +
>> +
>> +RootComplex Device
>> +==================
>> +
>> +lspci Output
>> +------------
>> +
>> +Note that the devices listed here correspond to the value populated in 1.4
> 
> Can you use a section name (or reference) here instead of "1.4"?  I can't see 1.4
> when reading with an editor.
> 
>> +above::
>> +
>> +        # lspci
>> +        0000:00:00.0 PCI bridge: Texas Instruments Device b00d
>> +        0000:01:00.0 RAM memory: Texas Instruments Device b00d
>> +
>> +
>> +Using ntb_hw_epf Device
>> +-----------------------
>> +
>> +The host side software follows the standard NTB software architecture in Linux.
>> +All the existing client side NTB utilities like NTB Transport Client and NTB
>> +Netdev, NTB Ping Pong Test Client and NTB Tool Test Clientcan be used with NTB
> 
>                                                        Client can be
> 
>> +function device.
>> +
>> +For more information on NTB see
>> +Documentation/driver-api/ntb.rst
> 
> 
> thanks.
>
diff mbox series

Patch

diff --git a/Documentation/PCI/endpoint/index.rst b/Documentation/PCI/endpoint/index.rst
index 9cb6e5f3c4d5..38ea1f604b6d 100644
--- a/Documentation/PCI/endpoint/index.rst
+++ b/Documentation/PCI/endpoint/index.rst
@@ -12,6 +12,7 @@  PCI Endpoint Framework
    pci-test-function
    pci-test-howto
    pci-ntb-function
+   pci-ntb-howto
 
    function/binding/pci-test
    function/binding/pci-ntb
diff --git a/Documentation/PCI/endpoint/pci-ntb-howto.rst b/Documentation/PCI/endpoint/pci-ntb-howto.rst
new file mode 100644
index 000000000000..2fbb0a051c3b
--- /dev/null
+++ b/Documentation/PCI/endpoint/pci-ntb-howto.rst
@@ -0,0 +1,160 @@ 
+.. SPDX-License-Identifier: GPL-2.0
+
+======================
+PCI NTB EPF User Guide
+======================
+
+:Author: Kishon Vijay Abraham I <kishon@ti.com>
+
+This document is a guide to help users use pci-epf-ntb function driver
+and ntb_hw_epf host driver for NTB functionality. The list of steps to
+be followed in the host side and EP side is given below. For the hardware
+configuration and internals of NTB using configurable endpoints see
+Documentation/PCI/endpoint/pci-ntb-function.rst
+
+Endpoint Device
+===============
+
+Endpoint Controller Devices
+---------------------------
+
+For implementing NTB functionality atleast two endpoint controller devices
+are required.
+To find the list of endpoint controller devices in the system::
+
+        # ls /sys/class/pci_epc/
+          2900000.pcie-ep  2910000.pcie-ep
+
+If PCI_ENDPOINT_CONFIGFS is enabled::
+
+	# ls /sys/kernel/config/pci_ep/controllers
+	  2900000.pcie-ep  2910000.pcie-ep
+
+
+Endpoint Function Drivers
+-------------------------
+
+To find the list of endpoint function drivers in the system::
+
+	# ls /sys/bus/pci-epf/drivers
+	  pci_epf_ntb   pci_epf_ntb
+
+If PCI_ENDPOINT_CONFIGFS is enabled::
+
+	# ls /sys/kernel/config/pci_ep/functions
+	  pci_epf_ntb   pci_epf_ntb
+
+
+Creating pci-epf-ntb Device
+----------------------------
+
+PCI endpoint function device can be created using the configfs. To create
+pci-epf-ntb device, the following commands can be used::
+
+	# mount -t configfs none /sys/kernel/config
+	# cd /sys/kernel/config/pci_ep/
+	# mkdir functions/pci_epf_ntb/func1
+
+The "mkdir func1" above creates the pci-epf-ntb function device that will
+be probed by pci_epf_ntb driver.
+
+The PCI endpoint framework populates the directory with the following
+configurable fields::
+
+	# ls functions/pci_epf_ntb/func1
+          baseclass_code    deviceid          msi_interrupts    pci-epf-ntb.0
+          progif_code       secondary         subsys_id         vendorid
+          cache_line_size   interrupt_pin     msix_interrupts   primary
+          revid             subclass_code     subsys_vendor_id
+
+The PCI endpoint function driver populates these entries with default values
+when the device is bound to the driver. The pci-epf-ntb driver populates
+vendorid with 0xffff and interrupt_pin with 0x0001::
+
+	# cat functions/pci_epf_ntb/func1/vendorid
+	  0xffff
+	# cat functions/pci_epf_ntb/func1/interrupt_pin
+	  0x0001
+
+
+Configuring pci-epf-ntb Device
+-------------------------------
+
+The user can configure the pci-epf-ntb device using configfs entry. In order
+to change the vendorid and the deviceid, the following
+commands can be used::
+
+	# echo 0x104c > functions/pci_epf_ntb/func1/vendorid
+	# echo 0xb00d > functions/pci_epf_ntb/func1/deviceid
+
+In-order to configure NTB specific attributes, a new sub-directory to func1
+should be created::
+
+	# mkdir functions/pci_epf_ntb/func1/pci_epf_ntb.0/
+
+The NTB function driver will populate this directory with various attributes
+that can be configured by the user::
+
+	# ls functions/pci_epf_ntb/func1/pci_epf_ntb.0/
+          db_count    mw1         mw2         mw3         mw4         num_mws
+          spad_count
+
+A sample configuration for NTB function is given below::
+
+	# echo 4 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/db_count
+	# echo 128 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/spad_count
+	# echo 2 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/num_mws
+	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw1
+	# echo 0x100000 > functions/pci_epf_ntb/func1/pci_epf_ntb.0/mw2
+
+Binding pci-epf-ntb Device to EP Controller
+--------------------------------------------
+
+NTB function device should be attached to two PCIe endpoint controllers
+connected to the two hosts. Use the 'primary' and 'secondary' entries
+inside NTB function device to attach one PCIe endpoint controller to
+primary interface and the other PCIe endpoint controller to the secondary
+interface. ::
+
+        # ln -s controllers/2900000.pcie-ep/ functions/pci-epf-ntb/func1/primary
+        # ln -s controllers/2910000.pcie-ep/ functions/pci-epf-ntb/func1/secondary
+
+Once the above step is completed, both the PCI endpoint controllers is ready to
+establish a link with the host.
+
+
+Start the Link
+--------------
+
+In order for the endpoint device to establish a link with the host, the _start_
+field should be populated with '1'. For NTB, both the PCIe endpoint controllers
+should establish link with the host::
+
+        #echo 1 > controllers/2900000.pcie-ep/start
+        #echo 1 > controllers/2910000.pcie-ep/start
+
+
+RootComplex Device
+==================
+
+lspci Output
+------------
+
+Note that the devices listed here correspond to the value populated in 1.4
+above::
+
+        # lspci
+        0000:00:00.0 PCI bridge: Texas Instruments Device b00d
+        0000:01:00.0 RAM memory: Texas Instruments Device b00d
+
+
+Using ntb_hw_epf Device
+-----------------------
+
+The host side software follows the standard NTB software architecture in Linux.
+All the existing client side NTB utilities like NTB Transport Client and NTB
+Netdev, NTB Ping Pong Test Client and NTB Tool Test Clientcan be used with NTB
+function device.
+
+For more information on NTB see
+Documentation/driver-api/ntb.rst