[net-next,RESEND,v2,0/9] ibmvnic: Reuse ltb, rx, tx pools

Message

Sukadev Bhattiprolu Sept. 15, 2021, 3:52 a.m. UTC

It can take a long time to free and reallocate rx and tx pools and long
term buffer (LTB) during each reset of the VNIC. This is specially true
when the partition (LPAR) is heavily loaded and going through a Logical
Partition Migration (LPM). The long drawn reset causes the LPAR to lose
connectivity for extended periods of time and results in "RMC connection"
errors and the LPM failing.

What is worse is that during the LPM we could get a failover because
of the lost connectivity. At that point, the vnic driver releases
even the resources it has already allocated and starts over.

As long as the resources we have already allocated are valid/applicable,
we might as well hold on to them while trying to allocate the remaining
resources. This patch set attempts to reuse the resources previously
allocated as long as they are valid. It seems to vastly improve the
time taken for the vnic reset and signficantly reduces the chances of
getting the RMC connection errors. We do get still them occasionally,
but appears to be for reasons other than memory allocation delays and
those are still being investigated.

If the backing devices for a vnic adapter are not "matched" (see "pool
parameters" in patches 8 and 9) it is possible that we will still free
all the resources and allocate them. If that becomes a common problem,
we have to address it separately.

Thanks to input and extensive testing from Brian King, Cris Forno,
Dany Madden, Rick Lindsley.

Changelog[v2]
	[Jakub Kicinski] Fix kdoc issues

Sukadev Bhattiprolu (9):
  ibmvnic: consolidate related code in replenish_rx_pool()
  ibmvnic: Fix up some comments and messages
  ibmvnic: Use/rename local vars in init_rx_pools
  ibmvnic: Use/rename local vars in init_tx_pools
  ibmvnic: init_tx_pools move loop-invariant code out
  ibmvnic: use bitmap for LTB map_ids
  ibmvnic: Reuse LTB when possible
  ibmvnic: Reuse rx pools when possible
  ibmvnic: Reuse tx pools when possible

 drivers/net/ethernet/ibm/ibmvnic.c | 640 +++++++++++++++++++----------
 drivers/net/ethernet/ibm/ibmvnic.h |  10 +-
 2 files changed, 427 insertions(+), 223 deletions(-)

Comments

[e-mail] Sept. 15, 2021, 10:30 a.m. UTC | #1

Hello:

This series was applied to netdev/net-next.git (refs/heads/master):

On Tue, 14 Sep 2021 20:52:50 -0700 you wrote:
> It can take a long time to free and reallocate rx and tx pools and long
> term buffer (LTB) during each reset of the VNIC. This is specially true
> when the partition (LPAR) is heavily loaded and going through a Logical
> Partition Migration (LPM). The long drawn reset causes the LPAR to lose
> connectivity for extended periods of time and results in "RMC connection"
> errors and the LPM failing.
> 
> [...]

Here is the summary with links:
  - [net-next,RESEND,v2,1/9] ibmvnic: Consolidate code in replenish_rx_pool()
    https://git.kernel.org/netdev/net-next/c/38106b2c433e
  - [net-next,RESEND,v2,2/9] ibmvnic: Fix up some comments and messages
    https://git.kernel.org/netdev/net-next/c/0f2bf3188c43
  - [net-next,RESEND,v2,3/9] ibmvnic: Use/rename local vars in init_rx_pools
    https://git.kernel.org/netdev/net-next/c/0df7b9ad8f84
  - [net-next,RESEND,v2,4/9] ibmvnic: Use/rename local vars in init_tx_pools
    https://git.kernel.org/netdev/net-next/c/8243c7ed6d08
  - [net-next,RESEND,v2,5/9] ibmvnic: init_tx_pools move loop-invariant code
    https://git.kernel.org/netdev/net-next/c/0d1af4fa7124
  - [net-next,RESEND,v2,6/9] ibmvnic: Use bitmap for LTB map_ids
    https://git.kernel.org/netdev/net-next/c/129854f061d8
  - [net-next,RESEND,v2,7/9] ibmvnic: Reuse LTB when possible
    https://git.kernel.org/netdev/net-next/c/f8ac0bfa7d7a
  - [net-next,RESEND,v2,8/9] ibmvnic: Reuse rx pools when possible
    https://git.kernel.org/netdev/net-next/c/489de956e7a2
  - [net-next,RESEND,v2,9/9] ibmvnic: Reuse tx pools when possible
    https://git.kernel.org/netdev/net-next/c/bbd809305bc7

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html