diff mbox series

[v2,net-next,3/9] net: enetc: only commit preemptible TCs to hardware when MM TX is active

Message ID 20230418111459.811553-4-vladimir.oltean@nxp.com (mailing list archive)
State Accepted
Commit 827145392a4aad635b93e5235b7d7fecc2fa31c7
Headers show
Series ethtool mm API consolidation | expand

Commit Message

Vladimir Oltean April 18, 2023, 11:14 a.m. UTC 
This was left as TODO in commit 01e23b2b3bad ("net: enetc: add support
for preemptible traffic classes") since it's relatively complicated.

Where this makes a difference is with a configuration as follows:

ethtool --set-mm eno0 pmac-enabled on tx-enabled on verify-enabled on

Preemptible packets should only be sent when the MAC Merge TX direction
becomes active (i.o.w. when the verification process succeeds, aka when
the link partner confirms it can process preemptible traffic). But the
tc qdisc with the preemptible traffic classes is offloaded completely
asynchronously w.r.t. the MM becoming active.

The ENETC manual does suggest that this should be handled in the driver:
"On startup, software should wait for the verification process to
complete (MMCSR[VSTS]=011) before initiating traffic".

Adding the necessary logic allows future selftests to uphold the claim
that an inactive or disabled MAC Merge layer should never send data
packets through the pMAC.

This change moves enetc_set_ptcfpr() from enetc.c to enetc_ethtool.c,
where its only caller is now - enetc_mm_commit_preemptible_tcs().

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
v1->v2: patch is new

 drivers/net/ethernet/freescale/enetc/enetc.c  | 23 ++-----
 drivers/net/ethernet/freescale/enetc/enetc.h  |  5 +-
 .../ethernet/freescale/enetc/enetc_ethtool.c  | 62 +++++++++++++++++++
 .../net/ethernet/freescale/enetc/enetc_hw.h   |  3 +
 4 files changed, 75 insertions(+), 18 deletions(-)

Comments

Simon Horman April 20, 2023, 2:42 p.m. UTC | #1 
On Tue, Apr 18, 2023 at 02:14:53PM +0300, Vladimir Oltean wrote:
> This was left as TODO in commit 01e23b2b3bad ("net: enetc: add support
> for preemptible traffic classes") since it's relatively complicated.
> 
> Where this makes a difference is with a configuration as follows:
> 
> ethtool --set-mm eno0 pmac-enabled on tx-enabled on verify-enabled on
> 
> Preemptible packets should only be sent when the MAC Merge TX direction
> becomes active (i.o.w. when the verification process succeeds, aka when
> the link partner confirms it can process preemptible traffic). But the
> tc qdisc with the preemptible traffic classes is offloaded completely
> asynchronously w.r.t. the MM becoming active.
> 
> The ENETC manual does suggest that this should be handled in the driver:
> "On startup, software should wait for the verification process to
> complete (MMCSR[VSTS]=011) before initiating traffic".
> 
> Adding the necessary logic allows future selftests to uphold the claim
> that an inactive or disabled MAC Merge layer should never send data
> packets through the pMAC.
> 
> This change moves enetc_set_ptcfpr() from enetc.c to enetc_ethtool.c,
> where its only caller is now - enetc_mm_commit_preemptible_tcs().
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Reviewed-by: Simon Horman <simon.horman@corigine.com>

>  int enetc_qos_query_caps(struct net_device *ndev, void *type_data);
> diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
> index deb674752851..838a92131963 100644
> --- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
> +++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
> @@ -991,6 +991,64 @@ static int enetc_get_mm(struct net_device *ndev, struct ethtool_mm_state *state)
>  	return 0;
>  }
>  
> +static int enetc_mm_wait_tx_active(struct enetc_hw *hw, int verify_time)
> +{
> +	int timeout = verify_time * USEC_PER_MSEC * ENETC_MM_VERIFY_RETRIES;
> +	u32 val;
> +
> +	/* This will time out after the standard value of 3 verification
> +	 * attempts. To not sleep forever, it relies on a non-zero verify_time,
> +	 * guarantee which is provided by the ethtool nlattr policy.
> +	 */
> +	return read_poll_timeout(enetc_port_rd, val,
> +				 ENETC_MMCSR_GET_VSTS(val) == 3,

nit: 3 is doing a lot of work here.
     As a follow-up, perhaps it could become part of an enum?

> +				 ENETC_MM_VERIFY_SLEEP_US, timeout,
> +				 true, hw, ENETC_MMCSR);
> +}

...
Vladimir Oltean April 20, 2023, 4:34 p.m. UTC | #2 
On Thu, Apr 20, 2023 at 04:42:52PM +0200, Simon Horman wrote:
> > +	/* This will time out after the standard value of 3 verification
> > +	 * attempts. To not sleep forever, it relies on a non-zero verify_time,
> > +	 * guarantee which is provided by the ethtool nlattr policy.
> > +	 */
> > +	return read_poll_timeout(enetc_port_rd, val,
> > +				 ENETC_MMCSR_GET_VSTS(val) == 3,
> 
> nit: 3 is doing a lot of work here.
>      As a follow-up, perhaps it could become part of an enum?

IMHO it's easy to abuse enums, when numbers could do just fine. I think
that in context (seeing the entire enetc_ethtool.c), this is not as bad
as just this patch makes it to be, because the other occurrence of
ENETC_MMCSR_GET_VSTS() is:

	switch (ENETC_MMCSR_GET_VSTS(val)) {
	case 0:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED;
		break;
	case 2:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING;
		break;
	case 3:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED;
		break;
	case 4:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED;
		break;
	case 5:
	default:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN;
		break;
	}

so it's immediately clear what the 3 represents (in vim I just press '*'
to see the other occurrences of ENETC_MMCSR_GET_VSTS).

I considered it, but I don't feel an urgent necessity to add an enum here.
Doing that would essentially transform the code into:

	return read_poll_timeout(enetc_port_rd, val,
				 ENETC_MMCSR_GET_VSTS(val) == ENETC_MM_VSTS_SUCCEEDED,

	switch (ENETC_MMCSR_GET_VSTS(val)) {
	case ENETC_MMCSR_VSTS_DISABLED:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED;
		break;
	case ENETC_MMCSR_VSTS_VERIFYING:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING;
		break;
	case ENETC_MMCSR_VSTS_SUCCEEDED:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED;
		break;
	case ENETC_MMCSR_VSTS_FAILED:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED;
		break;
	case ENETC_MMCSR_VSTS_UNKNOWN:
	default:
		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN;
		break;
	}

which to my eye is more bloated.
Simon Horman April 20, 2023, 4:49 p.m. UTC | #3 
On Thu, Apr 20, 2023 at 07:34:53PM +0300, Vladimir Oltean wrote:
> On Thu, Apr 20, 2023 at 04:42:52PM +0200, Simon Horman wrote:
> > > +	/* This will time out after the standard value of 3 verification
> > > +	 * attempts. To not sleep forever, it relies on a non-zero verify_time,
> > > +	 * guarantee which is provided by the ethtool nlattr policy.
> > > +	 */
> > > +	return read_poll_timeout(enetc_port_rd, val,
> > > +				 ENETC_MMCSR_GET_VSTS(val) == 3,
> > 
> > nit: 3 is doing a lot of work here.
> >      As a follow-up, perhaps it could become part of an enum?
> 
> IMHO it's easy to abuse enums, when numbers could do just fine. I think
> that in context (seeing the entire enetc_ethtool.c), this is not as bad
> as just this patch makes it to be, because the other occurrence of
> ENETC_MMCSR_GET_VSTS() is:
> 
> 	switch (ENETC_MMCSR_GET_VSTS(val)) {
> 	case 0:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED;
> 		break;
> 	case 2:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING;
> 		break;
> 	case 3:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED;
> 		break;
> 	case 4:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED;
> 		break;
> 	case 5:
> 	default:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN;
> 		break;
> 	}
> 
> so it's immediately clear what the 3 represents (in vim I just press '*'
> to see the other occurrences of ENETC_MMCSR_GET_VSTS).

Thanks.

I did see the code above, and I do agree it is informational
wrt the meaning of the values.

> I considered it, but I don't feel an urgent necessity to add an enum here.
> Doing that would essentially transform the code into:
> 
> 	return read_poll_timeout(enetc_port_rd, val,
> 				 ENETC_MMCSR_GET_VSTS(val) == ENETC_MM_VSTS_SUCCEEDED,
> 
> 	switch (ENETC_MMCSR_GET_VSTS(val)) {
> 	case ENETC_MMCSR_VSTS_DISABLED:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_DISABLED;
> 		break;
> 	case ENETC_MMCSR_VSTS_VERIFYING:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_VERIFYING;
> 		break;
> 	case ENETC_MMCSR_VSTS_SUCCEEDED:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED;
> 		break;
> 	case ENETC_MMCSR_VSTS_FAILED:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_FAILED;
> 		break;
> 	case ENETC_MMCSR_VSTS_UNKNOWN:
> 	default:
> 		state->verify_status = ETHTOOL_MM_VERIFY_STATUS_UNKNOWN;
> 		break;
> 	}
> 
> which to my eye is more bloated.

I guess it's subjective.
I certainly don't feel strongly about this.
And I appreciate you taking the time to respond to my idea.

I have no objections to leaving this patch as is (with '3').
diff mbox series

Patch

diff --git a/drivers/net/ethernet/freescale/enetc/enetc.c b/drivers/net/ethernet/freescale/enetc/enetc.c
index 41c194c1672d..3c4fa26f0f9b 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc.c
@@ -25,23 +25,12 @@  void enetc_port_mac_wr(struct enetc_si *si, u32 reg, u32 val)
 }
 EXPORT_SYMBOL_GPL(enetc_port_mac_wr);
 
-void enetc_set_ptcfpr(struct enetc_hw *hw, unsigned long preemptible_tcs)
+static void enetc_change_preemptible_tcs(struct enetc_ndev_priv *priv,
+					 u8 preemptible_tcs)
 {
-	u32 val;
-	int tc;
-
-	for (tc = 0; tc < 8; tc++) {
-		val = enetc_port_rd(hw, ENETC_PTCFPR(tc));
-
-		if (preemptible_tcs & BIT(tc))
-			val |= ENETC_PTCFPR_FPE;
-		else
-			val &= ~ENETC_PTCFPR_FPE;
-
-		enetc_port_wr(hw, ENETC_PTCFPR(tc), val);
-	}
+	priv->preemptible_tcs = preemptible_tcs;
+	enetc_mm_commit_preemptible_tcs(priv);
 }
-EXPORT_SYMBOL_GPL(enetc_set_ptcfpr);
 
 static int enetc_num_stack_tx_queues(struct enetc_ndev_priv *priv)
 {
@@ -2659,7 +2648,7 @@  static void enetc_reset_tc_mqprio(struct net_device *ndev)
 
 	enetc_debug_tx_ring_prios(priv);
 
-	enetc_set_ptcfpr(hw, 0);
+	enetc_change_preemptible_tcs(priv, 0);
 }
 
 int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
@@ -2714,7 +2703,7 @@  int enetc_setup_tc_mqprio(struct net_device *ndev, void *type_data)
 
 	enetc_debug_tx_ring_prios(priv);
 
-	enetc_set_ptcfpr(hw, mqprio->preemptible_tcs);
+	enetc_change_preemptible_tcs(priv, mqprio->preemptible_tcs);
 
 	return 0;
 
diff --git a/drivers/net/ethernet/freescale/enetc/enetc.h b/drivers/net/ethernet/freescale/enetc/enetc.h
index 143078a9ef16..c97a8e3d7a7f 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc.h
@@ -355,6 +355,9 @@  struct enetc_ndev_priv {
 	u16 rx_bd_count, tx_bd_count;
 
 	u16 msg_enable;
+
+	u8 preemptible_tcs;
+
 	enum enetc_active_offloads active_offloads;
 
 	u32 speed; /* store speed for compare update pspeed */
@@ -433,6 +436,7 @@  int enetc_xdp_xmit(struct net_device *ndev, int num_frames,
 /* ethtool */
 void enetc_set_ethtool_ops(struct net_device *ndev);
 void enetc_mm_link_state_update(struct enetc_ndev_priv *priv, bool link);
+void enetc_mm_commit_preemptible_tcs(struct enetc_ndev_priv *priv);
 
 /* control buffer descriptor ring (CBDR) */
 int enetc_setup_cbdr(struct device *dev, struct enetc_hw *hw, int bd_count,
@@ -486,7 +490,6 @@  static inline void enetc_cbd_free_data_mem(struct enetc_si *si, int size,
 
 void enetc_reset_ptcmsdur(struct enetc_hw *hw);
 void enetc_set_ptcmsdur(struct enetc_hw *hw, u32 *queue_max_sdu);
-void enetc_set_ptcfpr(struct enetc_hw *hw, unsigned long preemptible_tcs);
 
 #ifdef CONFIG_FSL_ENETC_QOS
 int enetc_qos_query_caps(struct net_device *ndev, void *type_data);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
index deb674752851..838a92131963 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_ethtool.c
@@ -991,6 +991,64 @@  static int enetc_get_mm(struct net_device *ndev, struct ethtool_mm_state *state)
 	return 0;
 }
 
+static int enetc_mm_wait_tx_active(struct enetc_hw *hw, int verify_time)
+{
+	int timeout = verify_time * USEC_PER_MSEC * ENETC_MM_VERIFY_RETRIES;
+	u32 val;
+
+	/* This will time out after the standard value of 3 verification
+	 * attempts. To not sleep forever, it relies on a non-zero verify_time,
+	 * guarantee which is provided by the ethtool nlattr policy.
+	 */
+	return read_poll_timeout(enetc_port_rd, val,
+				 ENETC_MMCSR_GET_VSTS(val) == 3,
+				 ENETC_MM_VERIFY_SLEEP_US, timeout,
+				 true, hw, ENETC_MMCSR);
+}
+
+static void enetc_set_ptcfpr(struct enetc_hw *hw, u8 preemptible_tcs)
+{
+	u32 val;
+	int tc;
+
+	for (tc = 0; tc < 8; tc++) {
+		val = enetc_port_rd(hw, ENETC_PTCFPR(tc));
+
+		if (preemptible_tcs & BIT(tc))
+			val |= ENETC_PTCFPR_FPE;
+		else
+			val &= ~ENETC_PTCFPR_FPE;
+
+		enetc_port_wr(hw, ENETC_PTCFPR(tc), val);
+	}
+}
+
+/* ENETC does not have an IRQ to notify changes to the MAC Merge TX status
+ * (active/inactive), but the preemptible traffic classes should only be
+ * committed to hardware once TX is active. Resort to polling.
+ */
+void enetc_mm_commit_preemptible_tcs(struct enetc_ndev_priv *priv)
+{
+	struct enetc_hw *hw = &priv->si->hw;
+	u8 preemptible_tcs = 0;
+	u32 val;
+	int err;
+
+	val = enetc_port_rd(hw, ENETC_MMCSR);
+	if (!(val & ENETC_MMCSR_ME))
+		goto out;
+
+	if (!(val & ENETC_MMCSR_VDIS)) {
+		err = enetc_mm_wait_tx_active(hw, ENETC_MMCSR_GET_VT(val));
+		if (err)
+			goto out;
+	}
+
+	preemptible_tcs = priv->preemptible_tcs;
+out:
+	enetc_set_ptcfpr(hw, preemptible_tcs);
+}
+
 /* FIXME: Workaround for the link partner's verification failing if ENETC
  * priorly received too much express traffic. The documentation doesn't
  * suggest this is needed.
@@ -1061,6 +1119,8 @@  static int enetc_set_mm(struct net_device *ndev, struct ethtool_mm_cfg *cfg,
 
 	enetc_restart_emac_rx(priv->si);
 
+	enetc_mm_commit_preemptible_tcs(priv);
+
 	mutex_unlock(&priv->mm_lock);
 
 	return 0;
@@ -1094,6 +1154,8 @@  void enetc_mm_link_state_update(struct enetc_ndev_priv *priv, bool link)
 
 	enetc_port_wr(hw, ENETC_MMCSR, val);
 
+	enetc_mm_commit_preemptible_tcs(priv);
+
 	mutex_unlock(&priv->mm_lock);
 }
 EXPORT_SYMBOL_GPL(enetc_mm_link_state_update);
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_hw.h b/drivers/net/ethernet/freescale/enetc/enetc_hw.h
index 36bb2d6d5658..1619943fb263 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_hw.h
+++ b/drivers/net/ethernet/freescale/enetc/enetc_hw.h
@@ -3,6 +3,9 @@ 
 
 #include <linux/bitops.h>
 
+#define ENETC_MM_VERIFY_SLEEP_US	USEC_PER_MSEC
+#define ENETC_MM_VERIFY_RETRIES		3
+
 /* ENETC device IDs */
 #define ENETC_DEV_ID_PF		0xe100
 #define ENETC_DEV_ID_VF		0xef00