diff mbox

[RFC,v3,01/11] IB/hfi-vnic: Virtual Network Interface Controller (VNIC) documentation

Message ID 1486498990-76562-2-git-send-email-niranjana.vishwanathapura@intel.com (mailing list archive)
State Superseded
Headers show

Commit Message

Niranjana Vishwanathapura Feb. 7, 2017, 8:23 p.m. UTC 
Add HFI VNIC design document explaining the VNIC architecture and the
driver design.

Reviewed-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
---
 Documentation/infiniband/hfi_vnic.txt | 102 ++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 Documentation/infiniband/hfi_vnic.txt

Comments

Bart Van Assche Feb. 8, 2017, 5 p.m. UTC | #1 
On Tue, 2017-02-07 at 12:23 -0800, Vishwanathapura, Niranjana wrote:
> diff --git a/Documentation/infiniband/hfi_vnic.txt b/Documentation/infiniband/hfi_vnic.txt
> new file mode 100644
> index 0000000..c6c801e
> --- /dev/null
> +++ b/Documentation/infiniband/hfi_vnic.txt
> @@ -0,0 +1,102 @@
> +Intel Omni-Path Host Fabric Interface (HFI) Virtual Network Interface
> +Controller (VNIC) feature supports Ethernet functionality over Omni-Path
> +fabric by encapsulating the Ethernet packets between HFI nodes.

Please add a reference to the document that defines the Ethernet encapsulation
wire protocol.

> +The patterns of exchanges of Omni-Path encapsulated Ethernet packets
> +involves one or more virtual Ethernet switches overlaid on the Omni-Path
> +fabric topology. A subset of HFI nodes on the Omni-Path fabric are
> +permitted to exchange encapsulated Ethernet packets across a particular
> +virtual Ethernet switch. The virtual Ethernet switches are logical
> +abstractions achieved by configuring the HFI nodes on the fabric for
> +header generation and processing. In the simplest configuration all HFI
> +nodes across the fabric exchange encapsulated Ethernet packets over a
> +single virtual Ethernet switch. A virtual Ethernet switch, is effectively
> +an independent Ethernet network. The configuration is performed by an
> +Ethernet Manager (EM) which is part of the trusted Fabric Manager (FM)
> +application. HFI nodes can have multiple VNICs each connected to a
> +different virtual Ethernet switch. The below diagram presents a case
> +of two virtual Ethernet switches with two HFI nodes.

Please elaborate this section. What is a virtual Ethernet switch? Is it a
software entity or something that is implemented in hardware? Also, how are
these independent Ethernet networks identified on the wire? The Linux kernel
already supports IB partitions and Ethernet VLANs. How do these independent
Ethernet networks compare to IB partitions and Ethernet VLANs? Which wire-
level header contains the identity of these Ethernet networks? Is it
possible to query from user space which Ethernet network a VNIC belongs to?
If so, with which API and which tools?

Thanks,

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Niranjana Vishwanathapura Feb. 23, 2017, 7:24 a.m. UTC | #2 
On Wed, Feb 08, 2017 at 05:00:45PM +0000, Bart Van Assche wrote:
>On Tue, 2017-02-07 at 12:23 -0800, Vishwanathapura, Niranjana wrote:
>
>Please elaborate this section. What is a virtual Ethernet switch? Is it a
>software entity or something that is implemented in hardware? Also, how are
>these independent Ethernet networks identified on the wire? The Linux kernel
>already supports IB partitions and Ethernet VLANs. How do these independent
>Ethernet networks compare to IB partitions and Ethernet VLANs? Which wire-
>level header contains the identity of these Ethernet networks? Is it
>possible to query from user space which Ethernet network a VNIC belongs to?
>If so, with which API and which tools?
>

I have added the VNIC packet format and some related information to the 
documentation in the PATCH series I just sent out.

>Thanks,
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/Documentation/infiniband/hfi_vnic.txt b/Documentation/infiniband/hfi_vnic.txt
new file mode 100644
index 0000000..c6c801e
--- /dev/null
+++ b/Documentation/infiniband/hfi_vnic.txt
@@ -0,0 +1,102 @@ 
+Intel Omni-Path Host Fabric Interface (HFI) Virtual Network Interface
+Controller (VNIC) feature supports Ethernet functionality over Omni-Path
+fabric by encapsulating the Ethernet packets between HFI nodes.
+
+The patterns of exchanges of Omni-Path encapsulated Ethernet packets
+involves one or more virtual Ethernet switches overlaid on the Omni-Path
+fabric topology. A subset of HFI nodes on the Omni-Path fabric are
+permitted to exchange encapsulated Ethernet packets across a particular
+virtual Ethernet switch. The virtual Ethernet switches are logical
+abstractions achieved by configuring the HFI nodes on the fabric for
+header generation and processing. In the simplest configuration all HFI
+nodes across the fabric exchange encapsulated Ethernet packets over a
+single virtual Ethernet switch. A virtual Ethernet switch, is effectively
+an independent Ethernet network. The configuration is performed by an
+Ethernet Manager (EM) which is part of the trusted Fabric Manager (FM)
+application. HFI nodes can have multiple VNICs each connected to a
+different virtual Ethernet switch. The below diagram presents a case
+of two virtual Ethernet switches with two HFI nodes.
+
+                             +-------------------+
+                             |      Subnet/      |
+                             |     Ethernet      |
+                             |      Manager      |
+                             +-------------------+
+                                /          /
+                              /           /
+                            /            /
+                          /             /
++-----------------------------+  +------------------------------+
+|  Virtual Ethernet Switch    |  |  Virtual Ethernet Switch     |
+|  +---------+    +---------+ |  | +---------+    +---------+   |
+|  | VPORT   |    |  VPORT  | |  | |  VPORT  |    |  VPORT  |   |
++--+---------+----+---------+-+  +-+---------+----+---------+---+
+         |                 \        /                 |
+         |                   \    /                   |
+         |                     \/                     |
+         |                    /  \                    |
+         |                  /      \                  |
+     +-----------+------------+  +-----------+------------+
+     |   VNIC    |    VNIC    |  |    VNIC   |    VNIC    |
+     +-----------+------------+  +-----------+------------+
+     |          HFI           |  |          HFI           |
+     +------------------------+  +------------------------+
+
+
+Intel HFI VNIC software design is presented in the below diagram.
+HFI VNIC functionality has a HW dependent component and a HW
+independent component.
+
+The support has been added for IB device to allocate and free the RDMA
+netdev devices. The RDMA netdev supports interfacing with the network
+stack thus creating standard network interfaces. HFI_VNIC is an RDMA
+netdev device type.
+
+The HW dependent VNIC functionality is part of the HFI1 driver. It
+implements the verbs to allocate and free the HFI_VNIC RDMA netdev.
+It involves HW resource allocation/management for VNIC functionality.
+It interfaces with the network stack and implements the required
+net_device_ops functions. It expects Omni-Path encapsulated Ethernet
+packets in the transmit path and provides HW access to them. It strips
+the Omni-Path header from the received packets before passing them up
+the network stack. It also implements the RDMA netdev control operations.
+
+The HFI VNIC module implements the HW independent VNIC functionality.
+It consists of two parts. The VNIC Ethernet Management Agent (VEMA)
+registers itself with IB core as an IB client and interfaces with the
+IB MAD stack. It exchanges the management information with the Ethernet
+Manager (EM) and the VNIC netdev. The VNIC netdev part allocates and frees
+the HFI_VNIC RDMA netdev devices. It overrides the net_device_ops functions
+set by HW dependent VNIC driver where required to accommodate any control
+operation. It also handles the encapsulation of Ethernet packets with an
+Omni-Path header in the transmit path. For each VNIC interface, the
+information required for encapsulation is configured by the EM via VEMA MAD
+interface. It also passes any control information to the HW dependent driver
+by invoking the RDMA netdev control operations.
+
+        +-------------------+ +----------------------+
+        |                   | |       Linux          |
+        |     IB MAD        | |      Network         |
+        |                   | |       Stack          |
+        +-------------------+ +----------------------+
+                 |               |          |
+                 |               |          |
+        +----------------------------+      |
+        |                            |      |
+        |      HFI VNIC Module       |      |
+        |  (HFI VNIC RDMA Netdev     |      |
+        |     & EMA functions)       |      |
+        |                            |      |
+        +----------------------------+      |
+                    |                       |
+                    |                       |
+           +------------------+             |
+           |     IB core      |             |
+           +------------------+             |
+                    |                       |
+                    |                       |
+        +--------------------------------------------+
+        |                                            |
+        |      HFI1 Driver with VNIC support         |
+        |                                            |
+        +--------------------------------------------+