From patchwork Fri Feb 21 20:26:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Cassel X-Patchwork-Id: 13986250 X-Patchwork-Delegate: kw@linux.com Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 725DC253359 for ; Fri, 21 Feb 2025 20:26:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740169617; cv=none; b=fMDVUva67FGf+0glJWr5FqspEYZWNbYGL2mZ3dHYe6FORC017hPkOmOi2niL1J7Ss2Nq06D8daTaNcNrS5O34RyjNnA11xMPgbcTfdHnmNfJ1NCMTB4c4uV7dELj041EJSbVLltq8mwzkkbvZz2yuZJpOa3cczlEsRZcJ2rAsy4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740169617; c=relaxed/simple; bh=1LyqPB8VGK1Jxgl+h1/rXaUt9zwo9HtL8AJzA2oi8JA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=kmBliIm0S5vPFPYz+hq5eXRnqoshTeCVAgaoEpWF8jzVL5Y41ImHL3H9ai/y4+/xG+4I3xegaEP7Gt2TLHSxchHcE6hjlMZ+yToFAFrvcQtHrcOB4NtaV35Mm02Okhpoo+EyadgFd2a8eIFVhi1dY1SUQ5SUPPjqHsYL4xNIm9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jgFaDA1M; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jgFaDA1M" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F1BEC4CED6; Fri, 21 Feb 2025 20:26:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740169617; bh=1LyqPB8VGK1Jxgl+h1/rXaUt9zwo9HtL8AJzA2oi8JA=; h=From:To:Cc:Subject:Date:From; b=jgFaDA1MYtGjyogJS2kMagQaCe3qvjZ60m+cp1NgxdS8PEuVGK3GFlwY1pWVzG6mE bgPlJiRfTFvuIEJvCX5HtANn6+gWxn8x26HAyHqFVLIf6Qo4dbgEJ2ZW3Uhwm3vJgj xa5yg/Z3MKC1imRlxE0mM0n/Tlt2+RRisa9CK8+KurtloWXJ2aS9LzrMOeRs8JDDgS yZnajhaMsPs1q5PIOCsVjlSuWBtVgKUHqIzGwPsL8P8kQUbeUnV/csY09hNeu057Rw bCqsgw+JIPMvuO4T5JycCXYHM9QVfvyRqsmXkTAqxGrwzHHq6olPLfSson85jhRtbQ SGuBi+5lqX3AQ== From: Niklas Cassel To: Jingoo Han , Manivannan Sadhasivam , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , Bjorn Helgaas Cc: Damien Le Moal , Shawn Lin , Niklas Cassel , linux-pci@vger.kernel.org Subject: [PATCH 1/2] PCI: dwc: ep: Remove superfluous function dw_pcie_ep_find_ext_capability() Date: Fri, 21 Feb 2025 21:26:47 +0100 Message-ID: <20250221202646.395252-3-cassel@kernel.org> X-Mailer: git-send-email 2.48.1 Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2208; i=cassel@kernel.org; h=from:subject; bh=1LyqPB8VGK1Jxgl+h1/rXaUt9zwo9HtL8AJzA2oi8JA=; b=owGbwMvMwCV2MsVw8cxjvkWMp9WSGNJ3PGw7/LbLhzco5FfuVI55rgyqx89dShWNuKUuc9jhN iPXlt2nO0pZGMS4GGTFFFl8f7jsL+52n3Jc8Y4NzBxWJpAhDFycAjCR3DhGhmaOGdw//9mGWLjG 7sqbMlE+eXWi7nvBz03uhzJqTj2R92X4X2h2fNIMjsYz/xdWZv7iW1q9fUXVlcao+qvZ5ZqOVWm PmAE= X-Developer-Key: i=cassel@kernel.org; a=openpgp; fpr=5ADE635C0E631CBBD5BE065A352FE6582ED9B5DA Remove the superfluous function dw_pcie_ep_find_ext_capability(), as it is virtually identical to dw_pcie_find_ext_capability(). Signed-off-by: Niklas Cassel Reviewed-by: Manivannan Sadhasivam --- .../pci/controller/dwc/pcie-designware-ep.c | 24 +++---------------- 1 file changed, 3 insertions(+), 21 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index f9d7f3f989ad..20f2436c7091 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -102,24 +102,6 @@ static u8 dw_pcie_ep_find_capability(struct dw_pcie_ep *ep, u8 func_no, u8 cap) return __dw_pcie_ep_find_next_cap(ep, func_no, next_cap_ptr, cap); } -static unsigned int dw_pcie_ep_find_ext_capability(struct dw_pcie *pci, int cap) -{ - u32 header; - int pos = PCI_CFG_SPACE_SIZE; - - while (pos) { - header = dw_pcie_readl_dbi(pci, pos); - if (PCI_EXT_CAP_ID(header) == cap) - return pos; - - pos = PCI_EXT_CAP_NEXT(header); - if (!pos) - break; - } - - return 0; -} - static int dw_pcie_ep_write_header(struct pci_epc *epc, u8 func_no, u8 vfunc_no, struct pci_epf_header *hdr) { @@ -230,7 +212,7 @@ static unsigned int dw_pcie_ep_get_rebar_offset(struct dw_pcie *pci, unsigned int offset, nbars; int i; - offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); + offset = dw_pcie_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); if (!offset) return offset; @@ -847,7 +829,7 @@ static void dw_pcie_ep_init_non_sticky_registers(struct dw_pcie *pci) enum pci_barno bar; u32 reg, i, val; - offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); + offset = dw_pcie_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); dw_pcie_dbi_ro_wr_en(pci); @@ -970,7 +952,7 @@ int dw_pcie_ep_init_registers(struct dw_pcie_ep *ep) if (ep->ops->init) ep->ops->init(ep); - ptm_cap_base = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_PTM); + ptm_cap_base = dw_pcie_find_ext_capability(pci, PCI_EXT_CAP_ID_PTM); /* * PTM responder capability can be disabled only after disabling From patchwork Fri Feb 21 20:26:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Cassel X-Patchwork-Id: 13986251 X-Patchwork-Delegate: kw@linux.com Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E28ED253B44 for ; Fri, 21 Feb 2025 20:27:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740169622; cv=none; b=DRdhqhgJ8zIduTOGaF7flD56o4O0sb+4lpaSEXait3mm+GnkbPz1a5ef5SNYU2okSX6FCGShxe+8i0AhRqNiJ8TGzv+rho3GEGEJc9qon++lWP0LMghyo5qbi6IiQ7pl60r5vrJ4cWxmWAleZvC92cGRURy+P92UI1m6ilBTSFM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740169622; c=relaxed/simple; bh=omw9Xk2GUaGYZHJp+OWs2mQdBQfQtllVg5cCAlRAdds=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=USCCWLXrmRj/ecm91ep26ZL2hh5B9ZxlNZLObuc+u9/oIaOAXhNhp3lxsegvTOHwLgVimvz4wO627ux3aQYjj9eGUT4PYmC5Gt6svTAcIXrs/06holWHZCIINyLvLPFy4f5+8QTmW0mlNme0G/H1YvrSrmEUc/WlRFcT6y/xawg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=M5hM/b07; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="M5hM/b07" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D844EC4CED6; Fri, 21 Feb 2025 20:26:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740169621; bh=omw9Xk2GUaGYZHJp+OWs2mQdBQfQtllVg5cCAlRAdds=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M5hM/b07D9l80ndsPjYbtELZE2g50mcX+dAdsf/sfl3e0kFXuvm9uC613fwKV4iM/ BnCJrBS869sxmwm8jsClTDKTZPiV28auojHMmQY7LEgrd2v5z5JBbGSnbzgKvoOWEB OepChQ01pzcRGEASdBlbh/Vt60oEwZqA6ZgGy9kOieIgzzl7jEO2BtNsfqTXvA/SFv YgSoBNEy8L0lf2gzdvjY3HWM7cCnDURSDIAuWEGOlF9BcVYvdjfHw3ge4nKKqulS6w vXAUN46JDUWzsKhPW6ldDS4GnUJ3bJ/KEdJJlwB5UNkZuYfilJazrStPKC78sefkfR yEzRa2nWk021w== From: Niklas Cassel To: Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy?= =?utf-8?q?=C5=84ski?= , Manivannan Sadhasivam , Rob Herring , Bjorn Helgaas , Heiko Stuebner Cc: Damien Le Moal , Shawn Lin , Niklas Cassel , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org Subject: [PATCH 2/2] PCI: dw-rockchip: hide broken ATS capability Date: Fri, 21 Feb 2025 21:26:48 +0100 Message-ID: <20250221202646.395252-4-cassel@kernel.org> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250221202646.395252-3-cassel@kernel.org> References: <20250221202646.395252-3-cassel@kernel.org> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3476; i=cassel@kernel.org; h=from:subject; bh=omw9Xk2GUaGYZHJp+OWs2mQdBQfQtllVg5cCAlRAdds=; b=owGbwMvMwCV2MsVw8cxjvkWMp9WSGNJ3POyKVZ34w/sUT7mY3k2T0PXW5QGfb+XFipRJexolF 0ZFTjfrKGVhEONikBVTZPH94bK/uNt9ynHFOzYwc1iZQIYwcHEKwET6TzH84bgu1Ni7NPzq2/8z 1uzOEnpYo68jGuUkEBWedmVvQiPHd0aGb2ucr/3+Nc1/wc9nfVv3XnkQe3X39YdaVp3b5s/5Zb9 9HRcA X-Developer-Key: i=cassel@kernel.org; a=openpgp; fpr=5ADE635C0E631CBBD5BE065A352FE6582ED9B5DA When running the rk3588 in endpoint mode, with an Intel host with IOMMU enabled, the host side prints: DMAR: VT-d detected Invalidation Time-out Error: SID 0 When running the rk3588 in endpoint mode, with an AMD host with IOMMU enabled, the host side prints: iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=63:00.0 address=0x42b5b01a0] Usually, to handle these issues, we add a quirk for the PCI vendor and device ID in drivers/pci/quirks.c with quirk_no_ats(). That is because we cannot usually modify the capabilities on the EP side. In this case, we can modify the capabilties on the EP side. Thus, hide the broken ATS capability on rk3588 when running in EP mode. That way, we don't need any quirk on the host side, and we see no errors on the host side, and we can run pci_endpoint_test successfully, with the IOMMU enabled on the host side. Signed-off-by: Niklas Cassel --- drivers/pci/controller/dwc/pcie-dw-rockchip.c | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c b/drivers/pci/controller/dwc/pcie-dw-rockchip.c index 836ea10eafbb..2be005c1a161 100644 --- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c +++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c @@ -242,6 +242,51 @@ static const struct dw_pcie_host_ops rockchip_pcie_host_ops = { .init = rockchip_pcie_host_init, }; +/* + * ATS does not work on rk3588 when running in EP mode. + * After a host has enabled ATS on the EP side, it will send an IOTLB + * invalidation request to the EP side. The rk3588 will never send a completion + * back and eventually the host will print an IOTLB_INV_TIMEOUT error, and the + * EP will not be operational. If we hide the ATS cap, things work as expected. + */ +static void rockchip_pcie_ep_hide_broken_ats_cap_rk3588(struct dw_pcie_ep *ep) +{ + struct dw_pcie *pci = to_dw_pcie_from_ep(ep); + struct device *dev = pci->dev; + unsigned int spcie_cap_offset, next_cap_offset; + u32 spcie_cap_header, next_cap_header; + + /* only hide the ATS cap for rk3588 running in EP mode */ + if (!of_device_is_compatible(dev->of_node, "rockchip,rk3588-pcie-ep")) + return; + + spcie_cap_offset = dw_pcie_find_ext_capability(pci, PCI_EXT_CAP_ID_SECPCI); + if (!spcie_cap_offset) + return; + + spcie_cap_header = dw_pcie_readl_dbi(pci, spcie_cap_offset); + next_cap_offset = PCI_EXT_CAP_NEXT(spcie_cap_header); + + next_cap_header = dw_pcie_readl_dbi(pci, next_cap_offset); + if (PCI_EXT_CAP_ID(next_cap_header) != PCI_EXT_CAP_ID_ATS) + return; + + /* clear next ptr */ + spcie_cap_header &= ~GENMASK(31, 20); + + /* set next ptr to next ptr of ATS_CAP */ + spcie_cap_header |= next_cap_header & GENMASK(31, 20); + + dw_pcie_dbi_ro_wr_en(pci); + dw_pcie_writel_dbi(pci, spcie_cap_offset, spcie_cap_header); + dw_pcie_dbi_ro_wr_dis(pci); +} + +static void rockchip_pcie_ep_pre_init(struct dw_pcie_ep *ep) +{ + rockchip_pcie_ep_hide_broken_ats_cap_rk3588(ep); +} + static void rockchip_pcie_ep_init(struct dw_pcie_ep *ep) { struct dw_pcie *pci = to_dw_pcie_from_ep(ep); @@ -314,6 +359,7 @@ rockchip_pcie_get_features(struct dw_pcie_ep *ep) static const struct dw_pcie_ep_ops rockchip_pcie_ep_ops = { .init = rockchip_pcie_ep_init, + .pre_init = rockchip_pcie_ep_pre_init, .raise_irq = rockchip_pcie_raise_irq, .get_features = rockchip_pcie_get_features, };