| Message ID | 20231212112521.3774610-2-quic_prashk@quicinc.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Set XHCI_SG_TRB_CACHE_SIZE_QUIRK for DWC3 devices | expand |
On Tue, Dec 12, 2023 at 04:55:20PM +0530, Prashanth K wrote: > Upstream commit bac1ec551434 ("usb: xhci: Set quirk for > XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI > which fixes XHC timeout, which was seen on synopsys XHCs while > using SG buffers. But the support for this quirk isn't present > in the DWC3 layer. > > We will encounter this XHCI timeout/hung issue if we run iperf > loopback tests using RTL8156 ethernet adaptor on DWC3 targets > with scatter-gather enabled. This gets resolved after enabling > the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using > the xhci device property since its needed for DWC3 controller. > > In Synopsys DWC3 databook, > Table 9-3: xHCI Debug Capability Limitations > Chained TRBs greater than TRB cache size: The debug capability > driver must not create a multi-TRB TD that describes smaller > than a 1K packet that spreads across 8 or more TRBs on either > the IN TR or the OUT TR. > > Cc: <stable@vger.kernel.org> > Signed-off-by: Prashanth K <quic_prashk@quicinc.com> What commit id does this fix? > --- > drivers/usb/dwc3/host.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c > index 61f57fe5bb78..31a496233d87 100644 > --- a/drivers/usb/dwc3/host.c > +++ b/drivers/usb/dwc3/host.c > @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) > > memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); > > + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); And this is ok if the entry is not present? thanks, greg k-h
On 15-12-23 06:12 pm, Greg Kroah-Hartman wrote: > On Tue, Dec 12, 2023 at 04:55:20PM +0530, Prashanth K wrote: >> Upstream commit bac1ec551434 ("usb: xhci: Set quirk for >> XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI >> which fixes XHC timeout, which was seen on synopsys XHCs while >> using SG buffers. But the support for this quirk isn't present >> in the DWC3 layer. >> >> We will encounter this XHCI timeout/hung issue if we run iperf >> loopback tests using RTL8156 ethernet adaptor on DWC3 targets >> with scatter-gather enabled. This gets resolved after enabling >> the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using >> the xhci device property since its needed for DWC3 controller. >> >> In Synopsys DWC3 databook, >> Table 9-3: xHCI Debug Capability Limitations >> Chained TRBs greater than TRB cache size: The debug capability >> driver must not create a multi-TRB TD that describes smaller >> than a 1K packet that spreads across 8 or more TRBs on either >> the IN TR or the OUT TR. >> >> Cc: <stable@vger.kernel.org> >> Signed-off-by: Prashanth K <quic_prashk@quicinc.com> > > What commit id does this fix? > This doesn't fix any commit as such, but adds the support for XHCI_SG_TRB_CACHE_SIZE_QUIRK (which is present in XHCI layer) to DWC3 layer. I have CC'ed stable kernel for this to be back-ported to older kernels (#5.11). > >> --- >> drivers/usb/dwc3/host.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c >> index 61f57fe5bb78..31a496233d87 100644 >> --- a/drivers/usb/dwc3/host.c >> +++ b/drivers/usb/dwc3/host.c >> @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) >> >> memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); >> >> + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); > > And this is ok if the entry is not present? > We are intending to use this quirk for all the dwc3 based devices since the DWC3 XHC needs it. If the entry is not present then we will hit stall if certain conditions aren't met (have mentioned the condition in commit text). Thanks, Prashanth K
On Fri, Dec 22, 2023 at 11:29:01AM +0530, Prashanth K wrote: > On 15-12-23 06:12 pm, Greg Kroah-Hartman wrote: > > On Tue, Dec 12, 2023 at 04:55:20PM +0530, Prashanth K wrote: > > > Upstream commit bac1ec551434 ("usb: xhci: Set quirk for > > > XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI > > > which fixes XHC timeout, which was seen on synopsys XHCs while > > > using SG buffers. But the support for this quirk isn't present > > > in the DWC3 layer. > > > > > > We will encounter this XHCI timeout/hung issue if we run iperf > > > loopback tests using RTL8156 ethernet adaptor on DWC3 targets > > > with scatter-gather enabled. This gets resolved after enabling > > > the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using > > > the xhci device property since its needed for DWC3 controller. > > > > > > In Synopsys DWC3 databook, > > > Table 9-3: xHCI Debug Capability Limitations > > > Chained TRBs greater than TRB cache size: The debug capability > > > driver must not create a multi-TRB TD that describes smaller > > > than a 1K packet that spreads across 8 or more TRBs on either > > > the IN TR or the OUT TR. > > > > > > Cc: <stable@vger.kernel.org> > > > Signed-off-by: Prashanth K <quic_prashk@quicinc.com> > > > > What commit id does this fix? > > > This doesn't fix any commit as such, but adds the support for > XHCI_SG_TRB_CACHE_SIZE_QUIRK (which is present in XHCI layer) to DWC3 layer. So this is a new feature? How does this fit into the stable kernel rules? > I have CC'ed stable kernel for this to be back-ported to older kernels > (#5.11). Why that specific kernel version and newer? Why not list it as documented? > > > > > --- > > > drivers/usb/dwc3/host.c | 2 ++ > > > 1 file changed, 2 insertions(+) > > > > > > diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c > > > index 61f57fe5bb78..31a496233d87 100644 > > > --- a/drivers/usb/dwc3/host.c > > > +++ b/drivers/usb/dwc3/host.c > > > @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) > > > memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); > > > + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); > > > > And this is ok if the entry is not present? > > > We are intending to use this quirk for all the dwc3 based devices since the > DWC3 XHC needs it. So you do not have this quirk yet in the kernel tree? We can't take code without any in-tree users. > If the entry is not present then we will hit stall if > certain conditions aren't met (have mentioned the condition in commit text). When will the quirk be added? To what platforms? thanks, greg k-h
On 22-12-23 11:40 am, Greg Kroah-Hartman wrote: > On Fri, Dec 22, 2023 at 11:29:01AM +0530, Prashanth K wrote: >> On 15-12-23 06:12 pm, Greg Kroah-Hartman wrote: >>> On Tue, Dec 12, 2023 at 04:55:20PM +0530, Prashanth K wrote: >>>> Upstream commit bac1ec551434 ("usb: xhci: Set quirk for >>>> XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI >>>> which fixes XHC timeout, which was seen on synopsys XHCs while >>>> using SG buffers. But the support for this quirk isn't present >>>> in the DWC3 layer. >>>> >>>> We will encounter this XHCI timeout/hung issue if we run iperf >>>> loopback tests using RTL8156 ethernet adaptor on DWC3 targets >>>> with scatter-gather enabled. This gets resolved after enabling >>>> the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using >>>> the xhci device property since its needed for DWC3 controller. >>>> >>>> In Synopsys DWC3 databook, >>>> Table 9-3: xHCI Debug Capability Limitations >>>> Chained TRBs greater than TRB cache size: The debug capability >>>> driver must not create a multi-TRB TD that describes smaller >>>> than a 1K packet that spreads across 8 or more TRBs on either >>>> the IN TR or the OUT TR. >>>> >>>> Cc: <stable@vger.kernel.org> >>>> Signed-off-by: Prashanth K <quic_prashk@quicinc.com> >>> >>> What commit id does this fix? >>> >> This doesn't fix any commit as such, but adds the support for >> XHCI_SG_TRB_CACHE_SIZE_QUIRK (which is present in XHCI layer) to DWC3 layer. > > So this is a new feature? > > How does this fit into the stable kernel rules? This isn't a new feature. To give some background, upstream commit bac1ec551434 ("usb: xhci: Set quirk for XHCI_SG_TRB_CACHE_SIZE_QUIRK") added a XHCI quirk which converts SG lists to CMA buffers/URBS if certain conditions aren't met. But they never enabled this xhci quirk since no issues were hit at that time. So, the support for the above mentioned quirk is added from 5.11 kernel onwards, but was never enabled anywhere. From commit bac1ec551434 : "We discovered this issue with devices on other platforms but have not yet come across any device that triggers this on Linux. But it could be a real problem now or in the future. All it takes is N number of small chained TRBs. And other instances of the Synopsys IP may have smaller values for the TRB_CACHE_SIZE which would exacerbate the problem." For more info: https://lore.kernel.org/all/20201208092912.1773650-3-mathias.nyman@linux.intel.com/ > >> I have CC'ed stable kernel for this to be back-ported to older kernels >> (#5.11). > > Why that specific kernel version and newer? Why not list it as > documented? I mentioned 5.11 because commit bac1ec551434 ("usb: xhci: Set quirk for XHCI_SG_TRB_CACHE_SIZE_QUIRK") is present from 5.11. > >>> >>>> --- >>>> drivers/usb/dwc3/host.c | 2 ++ >>>> 1 file changed, 2 insertions(+) >>>> >>>> diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c >>>> index 61f57fe5bb78..31a496233d87 100644 >>>> --- a/drivers/usb/dwc3/host.c >>>> +++ b/drivers/usb/dwc3/host.c >>>> @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) >>>> memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); >>>> + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); >>> >>> And this is ok if the entry is not present? >>> >> We are intending to use this quirk for all the dwc3 based devices since the >> DWC3 XHC needs it. > > So you do not have this quirk yet in the kernel tree? We can't take > code without any in-tree users. This is a 2 patch series, patch 1/2 sets a property from dwc3 layer. And patch 2 enables XHCI quirk based on the property set from DWC3. > >> If the entry is not present then we will hit stall if >> certain conditions aren't met (have mentioned the condition in commit text). > > When will the quirk be added? To what platforms? I guess there is some sort of confusion here, sorry for that. Earlier Tejas Joglekar from synopsys pushed a patch in XHCI layer which converts certain SG lists to CMA buffers if some pre-requisites aren't met. And this operation is done if an xhci->quirk is set (XHCI_SG_TRB_CACHE_SIZE_QUIRK - BIT39) - https://lore.kernel.org/all/20201208092912.1773650-2-mathias.nyman@linux.intel.com/ - https://lore.kernel.org/all/20201208092912.1773650-3-mathias.nyman@linux.intel.com/ But here the option to enable this quirk was done using XHCI priv data diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c index aa2d35f98200..4d34f6005381 100644 --- a/drivers/usb/host/xhci-plat.c +++ b/drivers/usb/host/xhci-plat.c @@ -333,6 +333,9 @@ static int xhci_plat_probe(struct platform_device *pdev) if (priv && (priv->quirks & XHCI_SKIP_PHY_INIT)) hcd->skip_phy_initialization = 1; + if (priv && (priv->quirks & XHCI_SG_TRB_CACHE_SIZE_QUIRK)) + xhci->quirks |= XHCI_SG_TRB_CACHE_SIZE_QUIRK; + ret = usb_add_hcd(hcd, irq, IRQF_SHARED); if (ret) goto disable_usb_phy; And this XHCI quirk (XHCI_SG_TRB_CACHE_SIZE_QUIRK) needs to be enabled for DWC3 controllers. There are 2 ways to do it. One way is by directly accessing XHCI private data from DWC3 layer (dwc3/host.c) which is not cleaner approach. So I'm reusing the device_create_managed_software_node() which is present in dwc3/host.c to add a quirk to XHCI node, and enable XHCI_SG_TRB_CACHE_SIZE_QUIRK based on property set from DWC3 layer. Thanks, Prashanth K
On 26.12.2023 7.24, Prashanth K wrote: > > > On 22-12-23 11:40 am, Greg Kroah-Hartman wrote: >> On Fri, Dec 22, 2023 at 11:29:01AM +0530, Prashanth K wrote: >>> On 15-12-23 06:12 pm, Greg Kroah-Hartman wrote: >>>> On Tue, Dec 12, 2023 at 04:55:20PM +0530, Prashanth K wrote: >>>>> Upstream commit bac1ec551434 ("usb: xhci: Set quirk for >>>>> XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI >>>>> which fixes XHC timeout, which was seen on synopsys XHCs while >>>>> using SG buffers. But the support for this quirk isn't present >>>>> in the DWC3 layer. >>>>> >>>>> We will encounter this XHCI timeout/hung issue if we run iperf >>>>> loopback tests using RTL8156 ethernet adaptor on DWC3 targets >>>>> with scatter-gather enabled. This gets resolved after enabling >>>>> the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using >>>>> the xhci device property since its needed for DWC3 controller. >>>>> >>>>> In Synopsys DWC3 databook, >>>>> Table 9-3: xHCI Debug Capability Limitations >>>>> Chained TRBs greater than TRB cache size: The debug capability >>>>> driver must not create a multi-TRB TD that describes smaller >>>>> than a 1K packet that spreads across 8 or more TRBs on either >>>>> the IN TR or the OUT TR. >>>>> >>>>> Cc: <stable@vger.kernel.org> >>>>> Signed-off-by: Prashanth K <quic_prashk@quicinc.com> >>>> >>>> What commit id does this fix? >>>> >>> This doesn't fix any commit as such, but adds the support for >>> XHCI_SG_TRB_CACHE_SIZE_QUIRK (which is present in XHCI layer) to DWC3 layer. >> >> So this is a new feature? >> >> How does this fit into the stable kernel rules? > > This isn't a new feature. To give some background, upstream commit bac1ec551434 ("usb: xhci: Set quirk for XHCI_SG_TRB_CACHE_SIZE_QUIRK") > added a XHCI quirk which converts SG lists to CMA buffers/URBS if certain conditions aren't met. But they never enabled this xhci quirk > since no issues were hit at that time. So, the support for the above mentioned quirk is added from 5.11 kernel onwards, but was never enabled anywhere. I remember this now. Original series had three patches, two adding the feature to xhci, and one for dwc3 enabling it. The xhci patches were fine and got in. https://marc.info/?l=linux-usb&m=160570849625065&w=2 The last dwc3 patch had issues and never apparently got in https://marc.info/?l=linux-usb&m=161008968009766&w=2 As this feature hasn't been enabled and code not widely run I think it would be better to skip stable for now. Stable can be added later once this has been successfully running in upstream for a while. Thanks Mathias
On 02-01-24 06:02 pm, Mathias Nyman wrote: > > I remember this now. > > Original series had three patches, two adding the feature to xhci, and > one for dwc3 enabling it. > The xhci patches were fine and got in. > > https://marc.info/?l=linux-usb&m=160570849625065&w=2 > > The last dwc3 patch had issues and never apparently got in > > https://marc.info/?l=linux-usb&m=161008968009766&w=2 > > As this feature hasn't been enabled and code not widely run I think it > would be better > to skip stable for now. Stable can be added later once this has been > successfully running > in upstream for a while. > > Thanks > Mathias > Thanks Mathias, do you want me to send the patch again without the stable tag? Regards, Prashanth K
Hi, On 12/12/23 12:25, Prashanth K wrote: > Upstream commit bac1ec551434 ("usb: xhci: Set quirk for > XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI > which fixes XHC timeout, which was seen on synopsys XHCs while > using SG buffers. But the support for this quirk isn't present > in the DWC3 layer. > > We will encounter this XHCI timeout/hung issue if we run iperf > loopback tests using RTL8156 ethernet adaptor on DWC3 targets > with scatter-gather enabled. This gets resolved after enabling > the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using > the xhci device property since its needed for DWC3 controller. > > In Synopsys DWC3 databook, > Table 9-3: xHCI Debug Capability Limitations > Chained TRBs greater than TRB cache size: The debug capability > driver must not create a multi-TRB TD that describes smaller > than a 1K packet that spreads across 8 or more TRBs on either > the IN TR or the OUT TR. > > Cc: <stable@vger.kernel.org> > Signed-off-by: Prashanth K <quic_prashk@quicinc.com> > --- > drivers/usb/dwc3/host.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c > index 61f57fe5bb78..31a496233d87 100644 > --- a/drivers/usb/dwc3/host.c > +++ b/drivers/usb/dwc3/host.c > @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) > > memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); > > + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); > + If you do this you also need to make the props array 1 entry bigger (increase it from 4 to 5 entries). Before this patch there are max 3 properties added and there needs to be an empty terminating property at the end (which is what the memset is for). So before this patch props[] needs to have 4 entries (which it does) and thus after this patch props[] needs to have 5 entire.s Regards, Hans
On 11-01-24 03:46 pm, Hans de Goede wrote: >> --- >> drivers/usb/dwc3/host.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c >> index 61f57fe5bb78..31a496233d87 100644 >> --- a/drivers/usb/dwc3/host.c >> +++ b/drivers/usb/dwc3/host.c >> @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) >> >> memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); >> >> + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); >> + > > If you do this you also need to make the props array 1 entry bigger (increase > it from 4 to 5 entries). Before this patch there are max 3 properties added > and there needs to be an empty terminating property at the end (which > is what the memset is for). So before this patch props[] needs to have > 4 entries (which it does) and thus after this patch props[] needs to > have 5 entire.s > > Regards, > > Hans > Thanks for pointing this out Hans! Didn't change it from 4->5 since we had only 4 properties, wasn't aware of the terminating property. Will update the patch. Thanks again Prashanth K
diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c index 61f57fe5bb78..31a496233d87 100644 --- a/drivers/usb/dwc3/host.c +++ b/drivers/usb/dwc3/host.c @@ -89,6 +89,8 @@ int dwc3_host_init(struct dwc3 *dwc) memset(props, 0, sizeof(struct property_entry) * ARRAY_SIZE(props)); + props[prop_idx++] = PROPERTY_ENTRY_BOOL("xhci-sg-trb-cache-size-quirk"); + if (dwc->usb3_lpm_capable) props[prop_idx++] = PROPERTY_ENTRY_BOOL("usb3-lpm-capable");
Upstream commit bac1ec551434 ("usb: xhci: Set quirk for XHCI_SG_TRB_CACHE_SIZE_QUIRK") introduced a new quirk in XHCI which fixes XHC timeout, which was seen on synopsys XHCs while using SG buffers. But the support for this quirk isn't present in the DWC3 layer. We will encounter this XHCI timeout/hung issue if we run iperf loopback tests using RTL8156 ethernet adaptor on DWC3 targets with scatter-gather enabled. This gets resolved after enabling the XHCI_SG_TRB_CACHE_SIZE_QUIRK. This patch enables it using the xhci device property since its needed for DWC3 controller. In Synopsys DWC3 databook, Table 9-3: xHCI Debug Capability Limitations Chained TRBs greater than TRB cache size: The debug capability driver must not create a multi-TRB TD that describes smaller than a 1K packet that spreads across 8 or more TRBs on either the IN TR or the OUT TR. Cc: <stable@vger.kernel.org> Signed-off-by: Prashanth K <quic_prashk@quicinc.com> --- drivers/usb/dwc3/host.c | 2 ++ 1 file changed, 2 insertions(+)