diff mbox series

[v2,8/9] PCI: microchip: Partition inbound address translation

Message ID 20221221162630.3632486-9-daire.mcnamara@microchip.com (mailing list archive)
State Superseded
Delegated to: Conor Dooley
Headers show
Series PCI: microchip: Partition address translations | expand

Checks

Context Check Description
conchuod/patch_count success Link
conchuod/cover_letter success Series has a cover letter
conchuod/tree_selection success Guessed tree name to be fixes
conchuod/fixes_present success Fixes tag present in non-next series
conchuod/maintainers_pattern success MAINTAINERS pattern errors before the patch: 12 and now 12
conchuod/verify_signedoff success Signed-off-by tag matches author and committer
conchuod/kdoc success Errors and warnings before: 0 this patch: 0
conchuod/module_param success Was 0 now: 0
conchuod/alphanumeric_selects success Out of order selects before the patch: 57 and now 57
conchuod/build_rv32_defconfig success Build OK
conchuod/build_warn_rv64 success Errors and warnings before: 0 this patch: 0
conchuod/dtb_warn_rv64 success Errors and warnings before: 0 this patch: 0
conchuod/header_inline success No static functions without inline keyword in header files
conchuod/checkpatch success total: 0 errors, 0 warnings, 0 checks, 235 lines checked
conchuod/source_inline success Was 0 now: 0
conchuod/build_rv64_nommu_k210_defconfig success Build OK
conchuod/verify_fixes success No Fixes tag
conchuod/build_rv64_nommu_virt_defconfig success Build OK

Commit Message

Daire McNamara Dec. 21, 2022, 4:26 p.m. UTC 
From: Daire McNamara <daire.mcnamara@microchip.com>

On Microchip PolarFire SoC the PCIe Root Port is behind a set of Fabric
Interface Controller (FIC) buses that encapsulate buses like ABP/AHP,
AXI-S, and AXI-M. Depending on which FIC(s) the Root Port is wired
through to cpu space, the Root Port driver needs to take account of the
address translation done by a parent (e.g. fabric) node before setting
up its own inbound address translation tables from attached devices.

Parse the dma-range properties to determine how much address translation
to perform in the Root Port and how much is being provided by the
fabric.

Signed-off-by: Daire McNamara <daire.mcnamara@microchip.com>
Reviewed-by: Conor Dooley <conor.dooley@microchip.com>
---
 drivers/pci/controller/pcie-microchip-host.c | 178 ++++++++++++++++++-
 1 file changed, 172 insertions(+), 6 deletions(-)

Comments

kernel test robot Dec. 22, 2022, 5:30 a.m. UTC | #1 
Hi,

I love your patch! Perhaps something to improve:

[auto build test WARNING on 3c1f24109dfc4fb1a3730ed237e50183c6bb26b3]

url:    https://github.com/intel-lab-lkp/linux/commits/daire-mcnamara-microchip-com/PCI-microchip-Partition-address-translations/20221222-002812
base:   3c1f24109dfc4fb1a3730ed237e50183c6bb26b3
patch link:    https://lore.kernel.org/r/20221221162630.3632486-9-daire.mcnamara%40microchip.com
patch subject: [PATCH v2 8/9] PCI: microchip: Partition inbound address translation
config: arm-randconfig-r046-20221218
compiler: clang version 16.0.0 (https://github.com/llvm/llvm-project 98b13979fb05f3ed288a900deb843e7b27589e58)
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # install arm cross compiling tool for clang build
        # apt-get install binutils-arm-linux-gnueabi
        # https://github.com/intel-lab-lkp/linux/commit/c128777fa7543f8ce931e855b7a69663509047c8
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review daire-mcnamara-microchip-com/PCI-microchip-Partition-address-translations/20221222-002812
        git checkout c128777fa7543f8ce931e855b7a69663509047c8
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm olddefconfig
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm SHELL=/bin/bash drivers/pci/controller/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/pci/controller/pcie-microchip-host.c:896:32: warning: cast from 'void (*)(struct clk *)' to 'void (*)(void *)' converts to incompatible function type [-Wcast-function-type-strict]
           devm_add_action_or_reset(dev, (void (*) (void *))clk_disable_unprepare,
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> drivers/pci/controller/pcie-microchip-host.c:1216:18: warning: shift count is negative [-Wshift-count-negative]
           u64 start_axi = GENMASK(63, 0);
                           ^~~~~~~~~~~~~~
   include/linux/bits.h:38:31: note: expanded from macro 'GENMASK'
           (GENMASK_INPUT_CHECK(h, l) + __GENMASK(h, l))
                                        ^~~~~~~~~~~~~~~
   include/linux/bits.h:36:11: note: expanded from macro '__GENMASK'
            (~UL(0) >> (BITS_PER_LONG - 1 - (h))))
                    ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/pci/controller/pcie-microchip-host.c:1218:18: warning: shift count is negative [-Wshift-count-negative]
           u64 start_pci = GENMASK(63, 0);
                           ^~~~~~~~~~~~~~
   include/linux/bits.h:38:31: note: expanded from macro 'GENMASK'
           (GENMASK_INPUT_CHECK(h, l) + __GENMASK(h, l))
                                        ^~~~~~~~~~~~~~~
   include/linux/bits.h:36:11: note: expanded from macro '__GENMASK'
            (~UL(0) >> (BITS_PER_LONG - 1 - (h))))
                    ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/pci/controller/pcie-microchip-host.c:1282:15: warning: shift count is negative [-Wshift-count-negative]
           start_axi &= MC_ATT_MASK;
                        ^~~~~~~~~~~
   drivers/pci/controller/pcie-microchip-host.c:26:24: note: expanded from macro 'MC_ATT_MASK'
   #define MC_ATT_MASK                             GENMASK(63, 31)
                                                   ^~~~~~~~~~~~~~~
   include/linux/bits.h:38:31: note: expanded from macro 'GENMASK'
           (GENMASK_INPUT_CHECK(h, l) + __GENMASK(h, l))
                                        ^~~~~~~~~~~~~~~
   include/linux/bits.h:36:11: note: expanded from macro '__GENMASK'
            (~UL(0) >> (BITS_PER_LONG - 1 - (h))))
                    ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
   drivers/pci/controller/pcie-microchip-host.c:1283:15: warning: shift count is negative [-Wshift-count-negative]
           start_pci &= MC_ATT_MASK;
                        ^~~~~~~~~~~
   drivers/pci/controller/pcie-microchip-host.c:26:24: note: expanded from macro 'MC_ATT_MASK'
   #define MC_ATT_MASK                             GENMASK(63, 31)
                                                   ^~~~~~~~~~~~~~~
   include/linux/bits.h:38:31: note: expanded from macro 'GENMASK'
           (GENMASK_INPUT_CHECK(h, l) + __GENMASK(h, l))
                                        ^~~~~~~~~~~~~~~
   include/linux/bits.h:36:11: note: expanded from macro '__GENMASK'
            (~UL(0) >> (BITS_PER_LONG - 1 - (h))))
                    ^  ~~~~~~~~~~~~~~~~~~~~~~~~~
   5 warnings generated.


vim +1216 drivers/pci/controller/pcie-microchip-host.c

  1205	
  1206	static int mc_check_for_parent_dma_range_handling(struct platform_device *pdev,
  1207							  struct mc_pcie *port)
  1208	{
  1209		struct device *dev = &pdev->dev;
  1210		struct device_node *dn = dev->of_node;
  1211		struct of_range_parser parser;
  1212		struct of_range range;
  1213		int num_parent_ranges = 0;
  1214		int num_ranges = 0;
  1215		struct inbound_windows ranges[MC_MAX_NUM_INBOUND_WINDOWS] = { 0 };
> 1216		u64 start_axi = GENMASK(63, 0);
  1217		u64 end_axi = 0;
  1218		u64 start_pci = GENMASK(63, 0);
  1219		s64 size;
  1220		u64 window_size;
  1221		int i;
  1222	
  1223		/* Find all dma-ranges */
  1224		if (of_pci_dma_range_parser_init(&parser, dn)) {
  1225			dev_err(dev, "missing dma-ranges property\n");
  1226			return -EINVAL;
  1227		}
  1228	
  1229		for_each_of_range(&parser, &range) {
  1230			if (num_ranges > MC_MAX_NUM_INBOUND_WINDOWS) {
  1231				dev_err(dev, "too many inbound ranges; %d available tables\n",
  1232					MC_MAX_NUM_INBOUND_WINDOWS);
  1233				return -EINVAL;
  1234			}
  1235			ranges[num_ranges].axi_addr = range.cpu_addr;
  1236			ranges[num_ranges].pci_addr = range.pci_addr;
  1237			ranges[num_ranges].size = range.size;
  1238	
  1239			num_ranges++;
  1240		}
  1241	
  1242		/*
  1243		 * Check for one level up; will need to adjust address translation
  1244		 * tables for these
  1245		 */
  1246		dn = of_get_parent(dn);
  1247		if (dn) {
  1248			of_pci_dma_range_parser_init(&parser, dn);
  1249	
  1250			for_each_of_range(&parser, &range) {
  1251				if (num_parent_ranges > MC_MAX_NUM_INBOUND_WINDOWS) {
  1252					dev_err(dev, "too many parent inbound ranges; %d available tables\n",
  1253						MC_MAX_NUM_INBOUND_WINDOWS);
  1254					return -EINVAL;
  1255				}
  1256				ranges[num_parent_ranges].axi_addr = range.pci_addr;
  1257				num_parent_ranges++;
  1258			}
  1259		}
  1260	
  1261		if (num_parent_ranges) {
  1262			if (num_ranges != num_parent_ranges) {
  1263				dev_err(dev, "num parent inbound ranges must be 0 or match num inbound ranges\n");
  1264				return -EINVAL;
  1265			}
  1266		}
  1267	
  1268		/* Merge ranges */
  1269		for (i = 0; i < num_ranges; i++) {
  1270			struct inbound_windows *range = &ranges[i];
  1271	
  1272			if (range->axi_addr < start_axi) {
  1273				start_axi = range->axi_addr;
  1274				start_pci = range->pci_addr;
  1275			}
  1276	
  1277			if (range->axi_addr + range->size > end_axi)
  1278				end_axi = range->axi_addr + range->size;
  1279		}
  1280	
  1281		/* Move starts back as far as possible */
  1282		start_axi &= MC_ATT_MASK;
  1283		start_pci &= MC_ATT_MASK;
  1284	
  1285		/* Adjust size to take account of that change */
  1286		size = end_axi - start_axi;
  1287	
  1288		/* May need to adjust size up to the next largest power of 2 */
  1289		if (size < 1ull << ilog2(size))
  1290			size = 1ull << (ilog2(size) + 1);
  1291	
  1292		window_size = 1ull << (ilog2(size) - 1);
  1293	
  1294		/* Divide merged range into windows */
  1295		i = 0;
  1296		while (size > 0 && i < MC_MAX_NUM_INBOUND_WINDOWS) {
  1297			port->inbound_windows[i].axi_addr = start_axi;
  1298			port->inbound_windows[i].pci_addr = start_pci;
  1299			port->inbound_windows[i].size = window_size;
  1300	
  1301			size -= window_size;
  1302			start_axi += window_size;
  1303			start_pci += window_size;
  1304			i++;
  1305			port->num_inbound_windows = i;
  1306		}
  1307	
  1308		if (size < 0) {
  1309			dev_err(dev, "insufficient windows to map inbound ranges\n");
  1310			return -EINVAL;
  1311		}
  1312	
  1313		return 0;
  1314	}
  1315
diff mbox series

Patch

diff --git a/drivers/pci/controller/pcie-microchip-host.c b/drivers/pci/controller/pcie-microchip-host.c
index ee8796db461e..ea95a482e506 100644
--- a/drivers/pci/controller/pcie-microchip-host.c
+++ b/drivers/pci/controller/pcie-microchip-host.c
@@ -22,6 +22,9 @@ 
 /* Number of MSI IRQs */
 #define MC_MAX_NUM_MSI_IRQS			32
 
+#define MC_MAX_NUM_INBOUND_WINDOWS		8
+#define MC_ATT_MASK				GENMASK(63, 31)
+
 /* PCIe Bridge Phy and Controller Phy offsets */
 #define MC_PCIE1_BRIDGE_ADDR			0x00008000u
 #define MC_PCIE1_CTRL_ADDR			0x0000a000u
@@ -86,10 +89,13 @@ 
 #define ISTATUS_MSI				0x194
 
 #define ATR_WINDOW_DESC_SIZE			32
-#define ATR_PCIE_ATR_SIZE			0x25
 #define ATR_SIZE_SHIFT				1
 #define ATR_IMPL_ENABLE				1
 
+#define ATR_PCIE_WIN0_SRCADDR			0x80000000
+#define ATR_PCIE_ATR_SIZE			(512 * 1024 * 1024ul)
+#define ATR_PCIE_NUM_WINDOWS			8
+
 /* PCIe Master table init defines */
 #define ATR0_PCIE_WIN0_SRCADDR_PARAM		0x600u
 #define ATR0_PCIE_WIN0_SRC_ADDR			0x604u
@@ -278,6 +284,12 @@  struct mc_msi {
 	DECLARE_BITMAP(used, MC_MAX_NUM_MSI_IRQS);
 };
 
+struct inbound_windows {
+	u64 axi_addr;
+	u64 pci_addr;
+	u64 size;
+};
+
 struct mc_pcie {
 	void __iomem *axi_base_addr;
 	struct device *dev;
@@ -286,6 +298,8 @@  struct mc_pcie {
 	raw_spinlock_t lock;
 	struct mc_msi msi;
 	u64 outbound_range_offset;
+	u32 num_inbound_windows;
+	struct inbound_windows inbound_windows[MC_MAX_NUM_INBOUND_WINDOWS];
 };
 
 struct cause {
@@ -942,6 +956,43 @@  static int mc_pcie_init_irq_domains(struct mc_pcie *port)
 	return mc_allocate_msi_domains(port);
 }
 
+static int mc_pcie_setup_inbound_ranges(struct platform_device *pdev, struct mc_pcie *port)
+{
+	void __iomem *bridge_base_addr = port->axi_base_addr + MC_PCIE_BRIDGE_ADDR;
+	phys_addr_t pcie_addr;
+	phys_addr_t axi_addr;
+	u32 atr_size;
+	u32 val;
+	int i;
+
+	for (i = 0; i < port->num_inbound_windows; i++) {
+		atr_size = ilog2(port->inbound_windows[i].size) - 1;
+		atr_size &= GENMASK(5, 0);
+
+		pcie_addr = port->inbound_windows[i].pci_addr;
+
+		val = lower_32_bits(pcie_addr) & GENMASK(31, 12);
+		val |= (atr_size << ATR_SIZE_SHIFT);
+		val |= ATR_IMPL_ENABLE;
+		writel(val, bridge_base_addr +
+		       ATR0_PCIE_WIN0_SRCADDR_PARAM + (i * ATR_WINDOW_DESC_SIZE));
+		writel(upper_32_bits(pcie_addr), bridge_base_addr +
+		       ATR0_PCIE_WIN0_SRC_ADDR + (i * ATR_WINDOW_DESC_SIZE));
+
+		axi_addr = port->inbound_windows[i].axi_addr;
+
+		writel(lower_32_bits(axi_addr), bridge_base_addr +
+		       ATR0_PCIE_WIN0_TRSL_ADDR_LSB + (i * ATR_WINDOW_DESC_SIZE));
+		writel(upper_32_bits(axi_addr), bridge_base_addr +
+		       ATR0_PCIE_WIN0_TRSL_ADDR_UDW + (i * ATR_WINDOW_DESC_SIZE));
+
+		writel(TRSL_ID_AXI4_MASTER_0, bridge_base_addr +
+		       ATR0_PCIE_WIN0_TRSL_PARAM + (i * ATR_WINDOW_DESC_SIZE));
+	}
+
+	return 0;
+}
+
 static void mc_pcie_setup_window(void __iomem *bridge_base_addr, u32 index,
 				 phys_addr_t axi_addr, phys_addr_t pci_addr,
 				 size_t size)
@@ -973,11 +1024,6 @@  static void mc_pcie_setup_window(void __iomem *bridge_base_addr, u32 index,
 	val = upper_32_bits(pci_addr);
 	writel(val, bridge_base_addr + (index * ATR_WINDOW_DESC_SIZE) +
 	       ATR0_AXI4_SLV0_TRSL_ADDR_UDW);
-
-	val = readl(bridge_base_addr + ATR0_PCIE_WIN0_SRCADDR_PARAM);
-	val |= (ATR_PCIE_ATR_SIZE << ATR_SIZE_SHIFT);
-	writel(val, bridge_base_addr + ATR0_PCIE_WIN0_SRCADDR_PARAM);
-	writel(0, bridge_base_addr + ATR0_PCIE_WIN0_SRC_ADDR);
 }
 
 static int mc_pcie_setup_windows(struct platform_device *pdev,
@@ -1157,6 +1203,116 @@  static int mc_check_for_parent_range_handling(struct platform_device *pdev, stru
 	return 0;
 }
 
+static int mc_check_for_parent_dma_range_handling(struct platform_device *pdev,
+						  struct mc_pcie *port)
+{
+	struct device *dev = &pdev->dev;
+	struct device_node *dn = dev->of_node;
+	struct of_range_parser parser;
+	struct of_range range;
+	int num_parent_ranges = 0;
+	int num_ranges = 0;
+	struct inbound_windows ranges[MC_MAX_NUM_INBOUND_WINDOWS] = { 0 };
+	u64 start_axi = GENMASK(63, 0);
+	u64 end_axi = 0;
+	u64 start_pci = GENMASK(63, 0);
+	s64 size;
+	u64 window_size;
+	int i;
+
+	/* Find all dma-ranges */
+	if (of_pci_dma_range_parser_init(&parser, dn)) {
+		dev_err(dev, "missing dma-ranges property\n");
+		return -EINVAL;
+	}
+
+	for_each_of_range(&parser, &range) {
+		if (num_ranges > MC_MAX_NUM_INBOUND_WINDOWS) {
+			dev_err(dev, "too many inbound ranges; %d available tables\n",
+				MC_MAX_NUM_INBOUND_WINDOWS);
+			return -EINVAL;
+		}
+		ranges[num_ranges].axi_addr = range.cpu_addr;
+		ranges[num_ranges].pci_addr = range.pci_addr;
+		ranges[num_ranges].size = range.size;
+
+		num_ranges++;
+	}
+
+	/*
+	 * Check for one level up; will need to adjust address translation
+	 * tables for these
+	 */
+	dn = of_get_parent(dn);
+	if (dn) {
+		of_pci_dma_range_parser_init(&parser, dn);
+
+		for_each_of_range(&parser, &range) {
+			if (num_parent_ranges > MC_MAX_NUM_INBOUND_WINDOWS) {
+				dev_err(dev, "too many parent inbound ranges; %d available tables\n",
+					MC_MAX_NUM_INBOUND_WINDOWS);
+				return -EINVAL;
+			}
+			ranges[num_parent_ranges].axi_addr = range.pci_addr;
+			num_parent_ranges++;
+		}
+	}
+
+	if (num_parent_ranges) {
+		if (num_ranges != num_parent_ranges) {
+			dev_err(dev, "num parent inbound ranges must be 0 or match num inbound ranges\n");
+			return -EINVAL;
+		}
+	}
+
+	/* Merge ranges */
+	for (i = 0; i < num_ranges; i++) {
+		struct inbound_windows *range = &ranges[i];
+
+		if (range->axi_addr < start_axi) {
+			start_axi = range->axi_addr;
+			start_pci = range->pci_addr;
+		}
+
+		if (range->axi_addr + range->size > end_axi)
+			end_axi = range->axi_addr + range->size;
+	}
+
+	/* Move starts back as far as possible */
+	start_axi &= MC_ATT_MASK;
+	start_pci &= MC_ATT_MASK;
+
+	/* Adjust size to take account of that change */
+	size = end_axi - start_axi;
+
+	/* May need to adjust size up to the next largest power of 2 */
+	if (size < 1ull << ilog2(size))
+		size = 1ull << (ilog2(size) + 1);
+
+	window_size = 1ull << (ilog2(size) - 1);
+
+	/* Divide merged range into windows */
+	i = 0;
+	while (size > 0 && i < MC_MAX_NUM_INBOUND_WINDOWS) {
+		port->inbound_windows[i].axi_addr = start_axi;
+		port->inbound_windows[i].pci_addr = start_pci;
+		port->inbound_windows[i].size = window_size;
+
+		size -= window_size;
+		start_axi += window_size;
+		start_pci += window_size;
+		i++;
+		port->num_inbound_windows = i;
+	}
+
+	if (size < 0) {
+		dev_err(dev, "insufficient windows to map inbound ranges\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int mc_platform_init(struct pci_config_window *cfg)
 {
 	struct device *dev = cfg->parent;
@@ -1174,6 +1330,11 @@  static int mc_platform_init(struct pci_config_window *cfg)
 	if (ret)
 		return ret;
 
+	/* And similarly, check for inbound address translation */
+	ret = mc_check_for_parent_dma_range_handling(pdev, port);
+	if (ret)
+		return ret;
+
 	/* Configure address translation table 0 for PCIe config space */
 	mc_pcie_setup_window(bridge_base_addr, 0, cfg->res.start - port->outbound_range_offset,
 			     cfg->res.start - port->outbound_range_offset,
@@ -1187,6 +1348,11 @@  static int mc_platform_init(struct pci_config_window *cfg)
 	if (ret)
 		return ret;
 
+	/* Configure inbound translation tables */
+	ret = mc_pcie_setup_inbound_ranges(pdev, port);
+	if (ret)
+		return ret;
+
 	/* Address translation is up; safe to enable interrupts */
 	ret = mc_init_interrupts(pdev, port);
 	if (ret)