mbox series

[net-next,v2,00/15] Introduce Intel IDPF driver

Message ID 20230411011354.2619359-1-pavan.kumar.linga@intel.com (mailing list archive)
Headers show
Series Introduce Intel IDPF driver | expand

Message

Pavan Kumar Linga April 11, 2023, 1:13 a.m. UTC
This patch series introduces the Intel Infrastructure Data Path Function
(IDPF) driver. It is used for both physical and virtual functions. Except
for some of the device operations the rest of the functionality is the
same for both PF and VF. IDPF uses virtchnl version2 opcodes and
structures defined in the virtchnl2 header file which helps the driver
to learn the capabilities and register offsets from the device
Control Plane (CP) instead of assuming the default values.

The format of the series follows the driver init flow to interface open.
To start with, probe gets called and kicks off the driver initialization
by spawning the 'vc_event_task' work queue which in turn calls the
'hard reset' function. As part of that, the mailbox is initialized which
is used to send/receive the virtchnl messages to/from the CP. Once that is
done, 'core init' kicks in which requests all the required global resources
from the CP and spawns the 'init_task' work queue to create the vports.

Based on the capability information received, the driver creates the said
number of vports (one or many) where each vport is associated to a netdev.
Also, each vport has its own resources such as queues, vectors etc.
From there, rest of the netdev_ops and data path are added.

IDPF implements both single queue which is traditional queueing model
as well as split queue model. In split queue model, it uses separate queue
for both completion descriptors and buffers which helps to implement
out-of-order completions. It also helps to implement asymmetric queues,
for example multiple RX completion queues can be processed by a single
RX buffer queue and multiple TX buffer queues can be processed by a
single TX completion queue. In single queue model, same queue is used
for both descriptor completions as well as buffer completions. It also
supports features such as generic checksum offload, generic receive
offload (hardware GRO) etc.

v1 --> v2: link [1]
 * removed the OASIS reference in the commit message to make it clear
   that this is an Intel vendor specific driver
 * fixed misspells
 * used comment starter "/**" for struct and definition headers in
   virtchnl header files
 * removed AVF reference
 * renamed APF reference to IDPF
 * added a comment to explain the reason for 'no flex field' at the end of
   virtchnl2_get_ptype_info struct
 * removed 'key[1]' in virtchnl2_rss_key struct as it is not used
 * set VIRTCHNL2_RXDID_2_FLEX_SQ_NIC to VIRTCHNL2_RXDID_2_FLEX_SPLITQ
   instead of assigning the same value
 * cleanup unnecessary NULL assignment to the rx_buf skb pointer since
   it is not used in splitq model
 * added comments to clarify the generation bit usage in splitq model
 * introduced 'reuse_bias' in the page_info structure and make use of it
   in the hot path
 * fixed RCT format in idpf_rx_construct_skb
 * report SPEED_UNKNOWN and DUPLEX_UNKNOWN when the link is down
 * fixed -Wframe-larger-than warning reported by lkp bot in
   idpf_vport_queue_ids_init
 * updated the documentation in idpf.rst to fix LKP bot warning

[1] https://lore.kernel.org/netdev/20230329140404.1647925-1-pavan.kumar.linga@intel.com/

Pavan Kumar Linga (15):
  virtchnl: add virtchnl version 2 ops
  idpf: add module register and probe functionality
  idpf: add controlq init and reset checks
  idpf: add core init and interrupt request
  idpf: add create vport and netdev configuration
  idpf: continue expanding init task
  idpf: configure resources for TX queues
  idpf: configure resources for RX queues
  idpf: initialize interrupts and enable vport
  idpf: add splitq start_xmit
  idpf: add TX splitq napi poll support
  idpf: add RX splitq napi poll support
  idpf: add singleq start_xmit and napi poll
  idpf: add ethtool callbacks
  idpf: configure SRIOV and add other ndo_ops

 .../device_drivers/ethernet/intel/idpf.rst    |  162 +
 drivers/net/ethernet/intel/Kconfig            |   11 +
 drivers/net/ethernet/intel/Makefile           |    1 +
 drivers/net/ethernet/intel/idpf/Makefile      |   18 +
 drivers/net/ethernet/intel/idpf/idpf.h        |  734 +++
 .../net/ethernet/intel/idpf/idpf_controlq.c   |  644 +++
 .../net/ethernet/intel/idpf/idpf_controlq.h   |  131 +
 .../ethernet/intel/idpf/idpf_controlq_api.h   |  188 +
 .../ethernet/intel/idpf/idpf_controlq_setup.c |  175 +
 drivers/net/ethernet/intel/idpf/idpf_dev.c    |  179 +
 drivers/net/ethernet/intel/idpf/idpf_devids.h |   10 +
 .../net/ethernet/intel/idpf/idpf_ethtool.c    | 1330 +++++
 .../ethernet/intel/idpf/idpf_lan_pf_regs.h    |  124 +
 .../net/ethernet/intel/idpf/idpf_lan_txrx.h   |  293 +
 .../ethernet/intel/idpf/idpf_lan_vf_regs.h    |  128 +
 drivers/net/ethernet/intel/idpf/idpf_lib.c    | 2551 +++++++++
 drivers/net/ethernet/intel/idpf/idpf_main.c   |   85 +
 drivers/net/ethernet/intel/idpf/idpf_mem.h    |   20 +
 .../ethernet/intel/idpf/idpf_singleq_txrx.c   | 1262 +++++
 drivers/net/ethernet/intel/idpf/idpf_txrx.c   | 4861 +++++++++++++++++
 drivers/net/ethernet/intel/idpf/idpf_txrx.h   |  854 +++
 drivers/net/ethernet/intel/idpf/idpf_vf_dev.c |  180 +
 .../net/ethernet/intel/idpf/idpf_virtchnl.c   | 3821 +++++++++++++
 drivers/net/ethernet/intel/idpf/virtchnl2.h   | 1201 ++++
 .../ethernet/intel/idpf/virtchnl2_lan_desc.h  |  666 +++
 25 files changed, 19629 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/ethernet/intel/idpf.rst
 create mode 100644 drivers/net/ethernet/intel/idpf/Makefile
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_controlq.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_controlq.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_controlq_api.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_controlq_setup.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_dev.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_devids.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_ethtool.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_lan_pf_regs.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_lan_txrx.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_lan_vf_regs.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_lib.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_main.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_mem.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_singleq_txrx.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_txrx.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_txrx.h
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_vf_dev.c
 create mode 100644 drivers/net/ethernet/intel/idpf/idpf_virtchnl.c
 create mode 100644 drivers/net/ethernet/intel/idpf/virtchnl2.h
 create mode 100644 drivers/net/ethernet/intel/idpf/virtchnl2_lan_desc.h

Comments

Sasha Levin April 12, 2023, 6:25 p.m. UTC | #1
On Mon, Apr 10, 2023 at 06:13:39PM -0700, Pavan Kumar Linga wrote:
>v1 --> v2: link [1]
> * removed the OASIS reference in the commit message to make it clear
>   that this is an Intel vendor specific driver

How will this work when the OASIS driver is ready down the road?

We'll end up with two "idpf" drivers, where one will work with hardware
that is not fully spec compliant using this Intel driver, and everything
else will use the OASIS driver?

Does Intel plan to remove this driver when the OASIS one lands?

At the very least, having two "idpf" drivers will be very confusing.
Willem de Bruijn April 12, 2023, 7:16 p.m. UTC | #2
Sasha Levin wrote:
> On Mon, Apr 10, 2023 at 06:13:39PM -0700, Pavan Kumar Linga wrote:
> >v1 --> v2: link [1]
> > * removed the OASIS reference in the commit message to make it clear
> >   that this is an Intel vendor specific driver
> 
> How will this work when the OASIS driver is ready down the road?
> 
> We'll end up with two "idpf" drivers, where one will work with hardware
> that is not fully spec compliant using this Intel driver, and everything
> else will use the OASIS driver?
> 
> Does Intel plan to remove this driver when the OASIS one lands?
> 
> At the very least, having two "idpf" drivers will be very confusing.

One approach is that when the OASIS v1 spec is published, this driver
is updated to match that and moved out of the intel directory.

This IPU/DPU/SmartNIC/.. device highly programmable. I assume a goal
is to make sure that the device meets the new v1 spec. That quite
likely requires a firmware update, but that is fine.
Samudrala, Sridhar April 13, 2023, 12:03 a.m. UTC | #3
On 4/12/2023 2:16 PM, Willem de Bruijn wrote:
> Sasha Levin wrote:
>> On Mon, Apr 10, 2023 at 06:13:39PM -0700, Pavan Kumar Linga wrote:
>>> v1 --> v2: link [1]
>>> * removed the OASIS reference in the commit message to make it clear
>>>    that this is an Intel vendor specific driver
>>
>> How will this work when the OASIS driver is ready down the road?
>>
>> We'll end up with two "idpf" drivers, where one will work with hardware
>> that is not fully spec compliant using this Intel driver, and everything
>> else will use the OASIS driver?
>>
>> Does Intel plan to remove this driver when the OASIS one lands?
>>
>> At the very least, having two "idpf" drivers will be very confusing.
> 
> One approach is that when the OASIS v1 spec is published, this driver
> is updated to match that and moved out of the intel directory.

Yes. We don't want to have 2 idpf drivers in the upstream kernel.
It will be an Intel vendor driver until it becomes a standard.
Hope it will be OK to move the driver out of the intel directory when 
that happens.

> 
> This IPU/DPU/SmartNIC/.. device highly programmable. I assume a goal
> is to make sure that the device meets the new v1 spec. That quite
> likely requires a firmware update, but that is fine.
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan@osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
Jakub Kicinski April 13, 2023, 2:24 a.m. UTC | #4
On Wed, 12 Apr 2023 19:03:22 -0500 Samudrala, Sridhar wrote:
> On 4/12/2023 2:16 PM, Willem de Bruijn wrote:
> > Sasha Levin wrote:  
> >> On Mon, Apr 10, 2023 at 06:13:39PM -0700, Pavan Kumar Linga wrote:  
> >> How will this work when the OASIS driver is ready down the road?
> >>
> >> We'll end up with two "idpf" drivers, where one will work with hardware
> >> that is not fully spec compliant using this Intel driver, and everything
> >> else will use the OASIS driver?
> >>
> >> Does Intel plan to remove this driver when the OASIS one lands?
> >>
> >> At the very least, having two "idpf" drivers will be very confusing.  
> > 
> > One approach is that when the OASIS v1 spec is published, this driver
> > is updated to match that and moved out of the intel directory.  
> 
> Yes. We don't want to have 2 idpf drivers in the upstream kernel.
> It will be an Intel vendor driver until it becomes a standard.
> Hope it will be OK to move the driver out of the intel directory when 
> that happens.

As I said previously in [0] until there is a compatible, widely
available implementation from a second vendor - this is an Intel
driver and nothing more. It's not moving anywhere.

I think that's a reasonable position which should allow Intel to ship
your code and me to remain professional.

[0] https://lore.kernel.org/all/20230403163025.5f40a87c@kernel.org/
Leon Romanovsky April 13, 2023, 7:15 a.m. UTC | #5
On Wed, Apr 12, 2023 at 07:24:34PM -0700, Jakub Kicinski wrote:
> On Wed, 12 Apr 2023 19:03:22 -0500 Samudrala, Sridhar wrote:
> > On 4/12/2023 2:16 PM, Willem de Bruijn wrote:
> > > Sasha Levin wrote:  
> > >> On Mon, Apr 10, 2023 at 06:13:39PM -0700, Pavan Kumar Linga wrote:  
> > >> How will this work when the OASIS driver is ready down the road?
> > >>
> > >> We'll end up with two "idpf" drivers, where one will work with hardware
> > >> that is not fully spec compliant using this Intel driver, and everything
> > >> else will use the OASIS driver?
> > >>
> > >> Does Intel plan to remove this driver when the OASIS one lands?
> > >>
> > >> At the very least, having two "idpf" drivers will be very confusing.  
> > > 
> > > One approach is that when the OASIS v1 spec is published, this driver
> > > is updated to match that and moved out of the intel directory.  
> > 
> > Yes. We don't want to have 2 idpf drivers in the upstream kernel.
> > It will be an Intel vendor driver until it becomes a standard.
> > Hope it will be OK to move the driver out of the intel directory when 
> > that happens.
> 
> As I said previously in [0] until there is a compatible, widely
> available implementation from a second vendor - this is an Intel
> driver and nothing more. It's not moving anywhere.

Even if second implementation arrives, it is unlikely that this
idpf driver will be moved. Mainly because of different level of
review between vendor driver vs. standard one, and expected pushback
to any incompatible changes in existing driver as it is already deployed.

Thanks

> 
> I think that's a reasonable position which should allow Intel to ship
> your code and me to remain professional.
> 
> [0] https://lore.kernel.org/all/20230403163025.5f40a87c@kernel.org/
Sasha Levin April 14, 2023, 10:01 p.m. UTC | #6
On Wed, Apr 12, 2023 at 07:24:34PM -0700, Jakub Kicinski wrote:
>On Wed, 12 Apr 2023 19:03:22 -0500 Samudrala, Sridhar wrote:
>> On 4/12/2023 2:16 PM, Willem de Bruijn wrote:
>> > Sasha Levin wrote:
>> >> On Mon, Apr 10, 2023 at 06:13:39PM -0700, Pavan Kumar Linga wrote:
>> >> How will this work when the OASIS driver is ready down the road?
>> >>
>> >> We'll end up with two "idpf" drivers, where one will work with hardware
>> >> that is not fully spec compliant using this Intel driver, and everything
>> >> else will use the OASIS driver?
>> >>
>> >> Does Intel plan to remove this driver when the OASIS one lands?
>> >>
>> >> At the very least, having two "idpf" drivers will be very confusing.
>> >
>> > One approach is that when the OASIS v1 spec is published, this driver
>> > is updated to match that and moved out of the intel directory.
>>
>> Yes. We don't want to have 2 idpf drivers in the upstream kernel.
>> It will be an Intel vendor driver until it becomes a standard.
>> Hope it will be OK to move the driver out of the intel directory when
>> that happens.
>
>As I said previously in [0] until there is a compatible, widely
>available implementation from a second vendor - this is an Intel
>driver and nothing more. It's not moving anywhere.

My concern isn't around the OASIS legal stuff, it's about the users who
end up using this and will end up getting confused when we have two (or
more) in-kernel "IDPF" drivers.

I don't think that moving is an option - it'll brake distros and upset
users: we can't rename drivers as we go because it has a good chance of
breaking users.

>I think that's a reasonable position which should allow Intel to ship
>your code and me to remain professional.

No concerns about OASIS or the current code, I just don't want to make
this a mess for the users.
Jakub Kicinski April 14, 2023, 10:27 p.m. UTC | #7
On Fri, 14 Apr 2023 18:01:42 -0400 Sasha Levin wrote:
>> As I said previously in [0] until there is a compatible, widely
>> available implementation from a second vendor - this is an Intel
>> driver and nothing more. It's not moving anywhere.  
> 
> My concern isn't around the OASIS legal stuff, it's about the users who
> end up using this and will end up getting confused when we have two (or
> more) in-kernel "IDPF" drivers.
> 
> I don't think that moving is an option - it'll brake distros and upset
> users: we can't rename drivers as we go because it has a good chance of
> breaking users.

Minor pain for backports but I don't think we need to rename anything,
just move.

Or we can just leave it be under intel/, since there are not other
participant now. Unless perhaps under google/ is a better option?

Drivers are organized by the vendor for better or worse. We have a
number of drivers under the "wrong directly" already. Companies merge /
buy each others product lines, there's also some confusion about common
IP drivers. It's all fine, whatever. 

Users are very, very unlikely to care.

>> I think that's a reasonable position which should allow Intel to ship
>> your code and me to remain professional.  
> 
> No concerns about OASIS or the current code, I just don't want to make
> this a mess for the users.

It's not a standard until someone else actually adopts it. What stops
all the other vendors from declaring that their driver interface is a
standard now, too?

We have a long standing rule in netdev against using marketing language.
Sasha Levin April 15, 2023, 5:16 p.m. UTC | #8
On Fri, Apr 14, 2023 at 03:27:44PM -0700, Jakub Kicinski wrote:
>On Fri, 14 Apr 2023 18:01:42 -0400 Sasha Levin wrote:
>>> As I said previously in [0] until there is a compatible, widely
>>> available implementation from a second vendor - this is an Intel
>>> driver and nothing more. It's not moving anywhere.
>>
>> My concern isn't around the OASIS legal stuff, it's about the users who
>> end up using this and will end up getting confused when we have two (or
>> more) in-kernel "IDPF" drivers.
>>
>> I don't think that moving is an option - it'll brake distros and upset
>> users: we can't rename drivers as we go because it has a good chance of
>> breaking users.
>
>Minor pain for backports but I don't think we need to rename anything,
>just move.
>
>Or we can just leave it be under intel/, since there are not other
>participant now. Unless perhaps under google/ is a better option?
>
>Drivers are organized by the vendor for better or worse. We have a
>number of drivers under the "wrong directly" already. Companies merge /
>buy each others product lines, there's also some confusion about common
>IP drivers. It's all fine, whatever.
>
>Users are very, very unlikely to care.
>
>>> I think that's a reasonable position which should allow Intel to ship
>>> your code and me to remain professional.
>>
>> No concerns about OASIS or the current code, I just don't want to make
>> this a mess for the users.
>
>It's not a standard until someone else actually adopts it. What stops
>all the other vendors from declaring that their driver interface is a
>standard now, too?
>
>We have a long standing rule in netdev against using marketing language.

Sorry, I may not have explained myself well. My concern is not around
what's standard and what's not, nor around where in the kernel tree
these drivers live.

I'm concerned that down the road we may end up with two drivers that
have the same name, and are working with hardware so similar that it
might be confusing to understand which driver a user should be using.

Yes, it's not something too big, but we have an opportunity to think
about this before committing to anything that might be a pain down the
road.
Jakub Kicinski April 17, 2023, 4:38 p.m. UTC | #9
On Sat, 15 Apr 2023 13:16:23 -0400 Sasha Levin wrote:
> Sorry, I may not have explained myself well. My concern is not around
> what's standard and what's not, nor around where in the kernel tree
> these drivers live.

My bad, I thought you were looking at this from the stable tree's angle.

> I'm concerned that down the road we may end up with two drivers that
> have the same name, and are working with hardware so similar that it
> might be confusing to understand which driver a user should be using.
> 
> Yes, it's not something too big, but we have an opportunity to think
> about this before committing to anything that might be a pain down the
> road.

Indeed, the "update" Willem mentioned should be at most a quirk or
capability exchange with the device within this driver. Two drivers
would be unacceptable.