| Message ID | 1655345240-26411-4-git-send-email-longli@linuxonhyperv.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | Introduce Microsoft Azure Network Adapter (MANA) RDMA driver | expand |
| Context | Check | Description |
|---|---|---|
| netdev/tree_selection | success | Guessed tree name to be net-next |
| netdev/fixes_present | success | Fixes tag not required for -next series |
| netdev/subject_prefix | success | Link |
| netdev/cover_letter | success | Series has a cover letter |
| netdev/patch_count | success | Link |
| netdev/header_inline | success | No static functions without inline keyword in header files |
| netdev/build_32bit | success | Errors and warnings before: 0 this patch: 0 |
| netdev/cc_maintainers | success | CCed 12 of 12 maintainers |
| netdev/build_clang | success | Errors and warnings before: 0 this patch: 0 |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Signed-off-by tag matches author and committer |
| netdev/check_selftest | success | No net selftest shell script |
| netdev/verify_fixes | success | No Fixes tag |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 0 this patch: 0 |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 95 lines checked |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/source_inline | success | Was 0 now: 0 |
> From: longli@linuxonhyperv.com <longli@linuxonhyperv.com> > Sent: Wednesday, June 15, 2022 7:07 PM > +void mana_uncfg_vport(struct mana_port_context *apc) > +{ > + mutex_lock(&apc->vport_mutex); > + apc->vport_use_count--; > + WARN_ON(apc->vport_use_count < 0); > + mutex_unlock(&apc->vport_mutex); > +} > +EXPORT_SYMBOL_GPL(mana_uncfg_vport); > + > +int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, > + u32 doorbell_pg_id) > { > struct mana_config_vport_resp resp = {}; > struct mana_config_vport_req req = {}; > int err; > > + /* Ethernet driver and IB driver can't take the port at the same time */ > + mutex_lock(&apc->vport_mutex); > + if (apc->vport_use_count > 0) { > + mutex_unlock(&apc->vport_mutex); > + return -ENODEV; Maybe -EBUSY is better? > @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct mana_port_context > *apc, u32 protection_dom_id, > > apc->tx_shortform_allowed = resp.short_form_allowed; > apc->tx_vp_offset = resp.tx_vport_offset; > + > + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", > + apc->port_handle, protection_dom_id, doorbell_pg_id); Should this be netdev_dbg()? The log buffer can be flooded if there are many vPorts per VF PCI device and there are a lot of VFs. > out: > + if (err) { > + mutex_lock(&apc->vport_mutex); > + apc->vport_use_count--; > + mutex_unlock(&apc->vport_mutex); > + } Change this to the blelow? if (err) mana_uncfg_vport(apc); > @@ -626,6 +654,9 @@ static int mana_cfg_vport_steering(struct > mana_port_context *apc, > resp.hdr.status); > err = -EPROTO; > } > + > + netdev_info(ndev, "Configured steering vPort %llu entries %u\n", > + apc->port_handle, num_entries); netdev_dbg()? In general, the patch looks good to me. Reviewed-by: Dexuan Cui <decui@microsoft.com>
> Subject: RE: [Patch v4 03/12] net: mana: Handle vport sharing between devices > > > From: longli@linuxonhyperv.com <longli@linuxonhyperv.com> > > Sent: Wednesday, June 15, 2022 7:07 PM > > +void mana_uncfg_vport(struct mana_port_context *apc) { > > + mutex_lock(&apc->vport_mutex); > > + apc->vport_use_count--; > > + WARN_ON(apc->vport_use_count < 0); > > + mutex_unlock(&apc->vport_mutex); > > +} > > +EXPORT_SYMBOL_GPL(mana_uncfg_vport); > > + > > +int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, > > + u32 doorbell_pg_id) > > { > > struct mana_config_vport_resp resp = {}; > > struct mana_config_vport_req req = {}; > > int err; > > > > + /* Ethernet driver and IB driver can't take the port at the same time */ > > + mutex_lock(&apc->vport_mutex); > > + if (apc->vport_use_count > 0) { > > + mutex_unlock(&apc->vport_mutex); > > + return -ENODEV; > Maybe -EBUSY is better? I agree with you, EBUSY is a better value. Will change this in v5. > > > @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct > > mana_port_context *apc, u32 protection_dom_id, > > > > apc->tx_shortform_allowed = resp.short_form_allowed; > > apc->tx_vp_offset = resp.tx_vport_offset; > > + > > + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", > > + apc->port_handle, protection_dom_id, doorbell_pg_id); > Should this be netdev_dbg()? > The log buffer can be flooded if there are many vPorts per VF PCI device and > there are a lot of VFs. The reason netdev_info () is used is that this message is important for troubleshooting initial setup issues with Ethernet driver. We rely on user to get this configured right to share the same hardware port between Ethernet and RDMA driver. As far as I know, there is no easy way for a driver to "take over" an exclusive hardware resource from another driver. If it is acceptable that we have one such message for each opened Ethernet port on the system, I suggest we keep it this way. > > > out: > > + if (err) { > > + mutex_lock(&apc->vport_mutex); > > + apc->vport_use_count--; > > + mutex_unlock(&apc->vport_mutex); > > + } > > Change this to the blelow? > if (err) > mana_uncfg_vport(apc); > > > @@ -626,6 +654,9 @@ static int mana_cfg_vport_steering(struct > > mana_port_context *apc, > > resp.hdr.status); > > err = -EPROTO; > > } > > + > > + netdev_info(ndev, "Configured steering vPort %llu entries %u\n", > > + apc->port_handle, num_entries); > > netdev_dbg()? > > In general, the patch looks good to me. > Reviewed-by: Dexuan Cui <decui@microsoft.com>
On Tue, Jul 12, 2022 at 06:48:09PM +0000, Long Li wrote: > > > @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct > > > mana_port_context *apc, u32 protection_dom_id, > > > > > > apc->tx_shortform_allowed = resp.short_form_allowed; > > > apc->tx_vp_offset = resp.tx_vport_offset; > > > + > > > + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", > > > + apc->port_handle, protection_dom_id, doorbell_pg_id); > > Should this be netdev_dbg()? > > The log buffer can be flooded if there are many vPorts per VF PCI device and > > there are a lot of VFs. > > The reason netdev_info () is used is that this message is important > for troubleshooting initial setup issues with Ethernet driver. We > rely on user to get this configured right to share the same hardware > port between Ethernet and RDMA driver. As far as I know, there is no > easy way for a driver to "take over" an exclusive hardware resource > from another driver. This seems like a really strange statement. Exactly how does all of this work? Jason
> Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing between > devices > > On Tue, Jul 12, 2022 at 06:48:09PM +0000, Long Li wrote: > > > > @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct > > > > mana_port_context *apc, u32 protection_dom_id, > > > > > > > > apc->tx_shortform_allowed = resp.short_form_allowed; > > > > apc->tx_vp_offset = resp.tx_vport_offset; > > > > + > > > > + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", > > > > + apc->port_handle, protection_dom_id, doorbell_pg_id); > > > Should this be netdev_dbg()? > > > The log buffer can be flooded if there are many vPorts per VF PCI > > > device and there are a lot of VFs. > > > > The reason netdev_info () is used is that this message is important > > for troubleshooting initial setup issues with Ethernet driver. We rely > > on user to get this configured right to share the same hardware port > > between Ethernet and RDMA driver. As far as I know, there is no easy > > way for a driver to "take over" an exclusive hardware resource from > > another driver. > > This seems like a really strange statement. > > Exactly how does all of this work? > > Jason "vport" is a hardware resource that can either be used by an Ethernet device, or an RDMA device. But it can't be used by both at the same time. The "vport" is associated with a protection domain and doorbell, it's programmed in the hardware. Outgoing traffic is enforced on this vport based on how it is programmed. Hardware is not responsible for tracking which one is using this "vport", it's up to the software to make sure it's correctly configured for that device. Long
On Thu, Jul 21, 2022 at 12:06:12AM +0000, Long Li wrote: > > Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing between > > devices > > > > On Tue, Jul 12, 2022 at 06:48:09PM +0000, Long Li wrote: > > > > > @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct > > > > > mana_port_context *apc, u32 protection_dom_id, > > > > > > > > > > apc->tx_shortform_allowed = resp.short_form_allowed; > > > > > apc->tx_vp_offset = resp.tx_vport_offset; > > > > > + > > > > > + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", > > > > > + apc->port_handle, protection_dom_id, doorbell_pg_id); > > > > Should this be netdev_dbg()? > > > > The log buffer can be flooded if there are many vPorts per VF PCI > > > > device and there are a lot of VFs. > > > > > > The reason netdev_info () is used is that this message is important > > > for troubleshooting initial setup issues with Ethernet driver. We rely > > > on user to get this configured right to share the same hardware port > > > between Ethernet and RDMA driver. As far as I know, there is no easy > > > way for a driver to "take over" an exclusive hardware resource from > > > another driver. > > > > This seems like a really strange statement. > > > > Exactly how does all of this work? > > > > Jason > > "vport" is a hardware resource that can either be used by an > Ethernet device, or an RDMA device. But it can't be used by both at > the same time. The "vport" is associated with a protection domain > and doorbell, it's programmed in the hardware. Outgoing traffic is > enforced on this vport based on how it is programmed. Sure, but how is the users problem to "get this configured right" and what exactly is the user supposed to do? I would expect the allocation of HW resources to be completely transparent to the user. Why is it not? Jason
> Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing between > devices > > On Thu, Jul 21, 2022 at 12:06:12AM +0000, Long Li wrote: > > > Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing > > > between devices > > > > > > On Tue, Jul 12, 2022 at 06:48:09PM +0000, Long Li wrote: > > > > > > @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct > > > > > > mana_port_context *apc, u32 protection_dom_id, > > > > > > > > > > > > apc->tx_shortform_allowed = resp.short_form_allowed; > > > > > > apc->tx_vp_offset = resp.tx_vport_offset; > > > > > > + > > > > > > + netdev_info(apc->ndev, "Configured vPort %llu PD %u > DB %u\n", > > > > > > + apc->port_handle, protection_dom_id, > doorbell_pg_id); > > > > > Should this be netdev_dbg()? > > > > > The log buffer can be flooded if there are many vPorts per VF > > > > > PCI device and there are a lot of VFs. > > > > > > > > The reason netdev_info () is used is that this message is > > > > important for troubleshooting initial setup issues with Ethernet > > > > driver. We rely on user to get this configured right to share the > > > > same hardware port between Ethernet and RDMA driver. As far as I > > > > know, there is no easy way for a driver to "take over" an > > > > exclusive hardware resource from another driver. > > > > > > This seems like a really strange statement. > > > > > > Exactly how does all of this work? > > > > > > Jason > > > > "vport" is a hardware resource that can either be used by an Ethernet > > device, or an RDMA device. But it can't be used by both at the same > > time. The "vport" is associated with a protection domain and doorbell, > > it's programmed in the hardware. Outgoing traffic is enforced on this > > vport based on how it is programmed. > > Sure, but how is the users problem to "get this configured right" and what > exactly is the user supposed to do? > > I would expect the allocation of HW resources to be completely transparent > to the user. Why is it not? > > Jason In the hardware, RDMA RAW_QP shares the same hardware resource (in this case, the vPort in hardware table) with the ethernet NIC. When an RDMA user creates a RAW_QP, we can't just shut down the ethernet. The user is required to make sure the ethernet is not in used when he creates this QP type.
On Thu, Jul 21, 2022 at 05:58:39PM +0000, Long Li wrote: > > > "vport" is a hardware resource that can either be used by an Ethernet > > > device, or an RDMA device. But it can't be used by both at the same > > > time. The "vport" is associated with a protection domain and doorbell, > > > it's programmed in the hardware. Outgoing traffic is enforced on this > > > vport based on how it is programmed. > > > > Sure, but how is the users problem to "get this configured right" and what > > exactly is the user supposed to do? > > > > I would expect the allocation of HW resources to be completely transparent > > to the user. Why is it not? > > > > In the hardware, RDMA RAW_QP shares the same hardware resource (in > this case, the vPort in hardware table) with the ethernet NIC. When > an RDMA user creates a RAW_QP, we can't just shut down the > ethernet. The user is required to make sure the ethernet is not in > used when he creates this QP type. You haven't answered my question - how is the user supposed to achieve this? And now I also want to know why the ethernet device and rdma device can even be loaded together if they cannot share the physical port? Exclusivity is not a sharing model that any driver today implements. Jason
> Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing between devices > > On Thu, Jul 21, 2022 at 05:58:39PM +0000, Long Li wrote: > > > > "vport" is a hardware resource that can either be used by an > > > > Ethernet device, or an RDMA device. But it can't be used by both > > > > at the same time. The "vport" is associated with a protection > > > > domain and doorbell, it's programmed in the hardware. Outgoing > > > > traffic is enforced on this vport based on how it is programmed. > > > > > > Sure, but how is the users problem to "get this configured right" > > > and what exactly is the user supposed to do? > > > > > > I would expect the allocation of HW resources to be completely > > > transparent to the user. Why is it not? > > > > > > > In the hardware, RDMA RAW_QP shares the same hardware resource (in > > this case, the vPort in hardware table) with the ethernet NIC. When an > > RDMA user creates a RAW_QP, we can't just shut down the ethernet. The > > user is required to make sure the ethernet is not in used when he > > creates this QP type. > > You haven't answered my question - how is the user supposed to achieve this? The user needs to configure the network interface so the kernel will not use it when the user creates a RAW QP on this port. This can be done via system configuration to not bring this interface online on system boot, or equivalently doing "ifconfig xxx down" to make the interface down when creating a RAW QP on this port. > > And now I also want to know why the ethernet device and rdma device can even > be loaded together if they cannot share the physical port? > Exclusivity is not a sharing model that any driver today implements. > This physical port limitation only applies to the RAW QP. For RC QP, the hardware doesn't have this limitation. The user can create RC QPs on a physical port up to the hardware limits independent of the Ethernet usage on the same port. For Ethernet usage, the hardware supports only one active user on a physical port. The driver checks on the port usage before programming the hardware when creating the RAW QP. Because the RDMA driver doesn't know in advance which QP type the user will create, it exposes the device with all its ports. The user may not be able to create RAW QP on a port if this port is already in used by the kernel. As a comparison, Mellanox NICs can expose both Ethernet and RDMA RAW_QP on the same physical port to software. They can work at the same time, but with some "quirks". The RDMA RAW_QP can preempt/interfere Ethernet traffic under certain conditions commonly used by DPDK (a heavy user of RAW_QP). Here are two scenarios that a Mellanox NIC port works on both Ethernet and RAW_QP. Scenario 1: The Ethernet loses TCP connection. 1. User A runs a program listing on a TCP port, accepts an incoming TCP connection and is communicating with the remote peer over this TCP connection. 2. User B creates an RDMA RAW_QP on the same port on the device. 3. As soon as the RAW_QP is created, the program in 1 can't send/receive data over this TCP connection. After some period of inactivity, the TCP connection terminates. Please note that this may also pose a security risk. User B with RAW_QP can potentially hijack this TCP connection from the kernel by framing the correct Ethernet packets and send over this QP to trick the remote peer, making it believe it's User A. Scenario 2: The Ethernet port state changes after RDMA RAW_QP is used on the port. 1. User uses "ifconfig ethx down" on the NIC, intending to make it offline 2. User creates a RDMA RAW_QP on the same port on the device. 3. User destroys this RAW_QP. 4. The ethx device in 1 reports carrier state in step 2, in many Linux distributions this makes it online without user interaction. "ifconfig ethx" shows its state changes to "up". The two activities on Ethernet and on RDMA RAW_QP should not happen concurrently and the user either gets unexpected behavior (Scenario 1) or the user needs to explicitly serialize the use (Scenario 2). In this sense, I think MANA is not materially different to how the Mellanox NICs implement the RAW_QP. IMHO, it's better to have the user explicitly decide whether to use Ethernet or RDMA RAW_QP on a specific port. Long
On Fri, Jul 29, 2022 at 06:44:22PM +0000, Long Li wrote: > > Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing between devices > > > > On Thu, Jul 21, 2022 at 05:58:39PM +0000, Long Li wrote: > > > > > "vport" is a hardware resource that can either be used by an > > > > > Ethernet device, or an RDMA device. But it can't be used by both > > > > > at the same time. The "vport" is associated with a protection > > > > > domain and doorbell, it's programmed in the hardware. Outgoing > > > > > traffic is enforced on this vport based on how it is programmed. > > > > > > > > Sure, but how is the users problem to "get this configured right" > > > > and what exactly is the user supposed to do? > > > > > > > > I would expect the allocation of HW resources to be completely > > > > transparent to the user. Why is it not? > > > > > > > > > > In the hardware, RDMA RAW_QP shares the same hardware resource (in > > > this case, the vPort in hardware table) with the ethernet NIC. When an > > > RDMA user creates a RAW_QP, we can't just shut down the ethernet. The > > > user is required to make sure the ethernet is not in used when he > > > creates this QP type. > > > > You haven't answered my question - how is the user supposed to achieve this? > > The user needs to configure the network interface so the kernel will not use it when the user creates a RAW QP on this port. > > This can be done via system configuration to not bring this > interface online on system boot, or equivalently doing "ifconfig xxx > down" to make the interface down when creating a RAW QP on this > port. That sounds horrible, why allow the user to even bind two drivers if the two drivers can't be used together? > > And now I also want to know why the ethernet device and rdma device can even > > be loaded together if they cannot share the physical port? > > Exclusivity is not a sharing model that any driver today implements. > > This physical port limitation only applies to the RAW QP. For RC QP, > the hardware doesn't have this limitation. The user can create RC > QPs on a physical port up to the hardware limits independent of the > Ethernet usage on the same port. .. and it is because you support sharing models in other cases :\ > Scenario 1: The Ethernet loses TCP connection. > 1. User A runs a program listing on a TCP port, accepts an incoming > TCP connection and is communicating with the remote peer over this > TCP connection. > 2. User B creates an RDMA RAW_QP on the same port on the device. > 3. As soon as the RAW_QP is created, the program in 1 can't > send/receive data over this TCP connection. After some period of > inactivity, the TCP connection terminates. It is a little more complicated than that, but yes, that could possibly happen if the userspace captures the right traffic. > Please note that this may also pose a security risk. User B with > RAW_QP can potentially hijack this TCP connection from the kernel by > framing the correct Ethernet packets and send over this QP to trick > the remote peer, making it believe it's User A. Any root user can do this with the netstack using eg tcpdump, bpf, XDP, raw sockets, etc. This is why the capability is guarded by CAP_NET_RAW. It is nothing unusual. > Scenario 2: The Ethernet port state changes after RDMA RAW_QP is used on the port. > 1. User uses "ifconfig ethx down" on the NIC, intending to make it offline > 2. User creates a RDMA RAW_QP on the same port on the device. > 3. User destroys this RAW_QP. > 4. The ethx device in 1 reports carrier state in step 2, in many > Linux distributions this makes it online without user > interaction. "ifconfig ethx" shows its state changes to "up". This I'm not familiar with, it actually sounds like a bug that the RAW_QP's interfere with the netdev carrier state. > the Mellanox NICs implement the RAW_QP. IMHO, it's better to have > the user explicitly decide whether to use Ethernet or RDMA RAW_QP on > a specific port. It should all be carefully documented someplace. Jason
> > the Mellanox NICs implement the RAW_QP. IMHO, it's better to have the > > user explicitly decide whether to use Ethernet or RDMA RAW_QP on a > > specific port. > > It should all be carefully documented someplace. The use case for RAW_QP is from user-mode. Is it acceptable that we document the detailed usage in rdma-core? Long
On Fri, Jul 29, 2022 at 09:20:05PM +0000, Long Li wrote: > > > the Mellanox NICs implement the RAW_QP. IMHO, it's better to have the > > > user explicitly decide whether to use Ethernet or RDMA RAW_QP on a > > > specific port. > > > > It should all be carefully documented someplace. > > The use case for RAW_QP is from user-mode. Is it acceptable that we > document the detailed usage in rdma-core? Yes, but add a suitable comment someplace in the kernel too Jason
> Subject: Re: [Patch v4 03/12] net: mana: Handle vport sharing between devices > > On Fri, Jul 29, 2022 at 09:20:05PM +0000, Long Li wrote: > > > > the Mellanox NICs implement the RAW_QP. IMHO, it's better to have > > > > the user explicitly decide whether to use Ethernet or RDMA RAW_QP > > > > on a specific port. > > > > > > It should all be carefully documented someplace. > > > > The use case for RAW_QP is from user-mode. Is it acceptable that we > > document the detailed usage in rdma-core? > > Yes, but add a suitable comment someplace in the kernel too Thanks. I will add detailed comments. Long
diff --git a/drivers/net/ethernet/microsoft/mana/mana.h b/drivers/net/ethernet/microsoft/mana/mana.h index 51bff91b63ee..8e58abdce906 100644 --- a/drivers/net/ethernet/microsoft/mana/mana.h +++ b/drivers/net/ethernet/microsoft/mana/mana.h @@ -376,6 +376,10 @@ struct mana_port_context { mana_handle_t port_handle; + /* Mutex for sharing access to vport_use_count */ + struct mutex vport_mutex; + int vport_use_count; + u16 port_idx; bool port_is_up; @@ -567,4 +571,7 @@ struct mana_adev { struct gdma_dev *mdev; }; +int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, + u32 doorbell_pg_id); +void mana_uncfg_vport(struct mana_port_context *apc); #endif /* _MANA_H */ diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index 745a9783dd70..23e7e423a544 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -530,13 +530,31 @@ static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index, return 0; } -static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, - u32 doorbell_pg_id) +void mana_uncfg_vport(struct mana_port_context *apc) +{ + mutex_lock(&apc->vport_mutex); + apc->vport_use_count--; + WARN_ON(apc->vport_use_count < 0); + mutex_unlock(&apc->vport_mutex); +} +EXPORT_SYMBOL_GPL(mana_uncfg_vport); + +int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, + u32 doorbell_pg_id) { struct mana_config_vport_resp resp = {}; struct mana_config_vport_req req = {}; int err; + /* Ethernet driver and IB driver can't take the port at the same time */ + mutex_lock(&apc->vport_mutex); + if (apc->vport_use_count > 0) { + mutex_unlock(&apc->vport_mutex); + return -ENODEV; + } + apc->vport_use_count++; + mutex_unlock(&apc->vport_mutex); + mana_gd_init_req_hdr(&req.hdr, MANA_CONFIG_VPORT_TX, sizeof(req), sizeof(resp)); req.vport = apc->port_handle; @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, apc->tx_shortform_allowed = resp.short_form_allowed; apc->tx_vp_offset = resp.tx_vport_offset; + + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", + apc->port_handle, protection_dom_id, doorbell_pg_id); out: + if (err) { + mutex_lock(&apc->vport_mutex); + apc->vport_use_count--; + mutex_unlock(&apc->vport_mutex); + } + return err; } +EXPORT_SYMBOL_GPL(mana_cfg_vport); static int mana_cfg_vport_steering(struct mana_port_context *apc, enum TRI_STATE rx, @@ -626,6 +654,9 @@ static int mana_cfg_vport_steering(struct mana_port_context *apc, resp.hdr.status); err = -EPROTO; } + + netdev_info(ndev, "Configured steering vPort %llu entries %u\n", + apc->port_handle, num_entries); out: kfree(req); return err; @@ -1678,6 +1709,8 @@ static void mana_destroy_vport(struct mana_port_context *apc) } mana_destroy_txq(apc); + + mana_uncfg_vport(apc); } static int mana_create_vport(struct mana_port_context *apc, @@ -1929,6 +1962,9 @@ static int mana_probe_port(struct mana_context *ac, int port_idx, apc->port_handle = INVALID_MANA_HANDLE; apc->port_idx = port_idx; + mutex_init(&apc->vport_mutex); + apc->vport_use_count = 0; + ndev->netdev_ops = &mana_devops; ndev->ethtool_ops = &mana_ethtool_ops; ndev->mtu = ETH_DATA_LEN;