From patchwork Wed Apr 7 22:26:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F9D5C43462 for ; Wed, 7 Apr 2021 22:26:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB27461369 for ; Wed, 7 Apr 2021 22:26:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229540AbhDGW1E (ORCPT ); Wed, 7 Apr 2021 18:27:04 -0400 Received: from mga09.intel.com ([134.134.136.24]:14309 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229648AbhDGW0w (ORCPT ); Wed, 7 Apr 2021 18:26:52 -0400 IronPort-SDR: azuVK6m0rKXbjpIHRM4/Mnv32GW0NX/i7AQ7x6cQwHyDwsOXeKFZ+FAdowF7iwT3ptwSdkFt9S TjNKXgM1DNMQ== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524912" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524912" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:38 -0700 IronPort-SDR: nXmuEZ4eA97r1e4GYdIvBECXcH+Z12PfklUt1M92z0WmmU8TPLt8o4DG+XQUlFHzCaF+bciM3Z vf8vixY4sZ9g== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548529" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:38 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 1/7] cxl/mem: Use dev instead of pdev->dev Date: Wed, 7 Apr 2021 15:26:19 -0700 Message-Id: <20210407222625.320177-2-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Trivial cleanup. Signed-off-by: Ben Widawsky Acked-by: Jonathan Cameron --- drivers/cxl/mem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index b6fe4e81d38a..99534260034e 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -935,7 +935,7 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, u8 bar; int rc; - cxlm = devm_kzalloc(&pdev->dev, sizeof(*cxlm), GFP_KERNEL); + cxlm = devm_kzalloc(dev, sizeof(*cxlm), GFP_KERNEL); if (!cxlm) { dev_err(dev, "No memory available\n"); return NULL; From patchwork Wed Apr 7 22:26:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ADE1C433ED for ; Wed, 7 Apr 2021 22:26:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 358736128A for ; Wed, 7 Apr 2021 22:26:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229712AbhDGW1G (ORCPT ); Wed, 7 Apr 2021 18:27:06 -0400 Received: from mga09.intel.com ([134.134.136.24]:14310 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229598AbhDGW0w (ORCPT ); Wed, 7 Apr 2021 18:26:52 -0400 IronPort-SDR: eTJBf6Jmz2/s8bY+l5JbH4iCGBzRNnAiZXwO7C4/jLAHswc0YvfCHsfKV96uMfZXBMJKcMOwOt YSsosqA7YXFg== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524917" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524917" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 IronPort-SDR: TcfA77NmEoYkSd3GGFafs/dTMmDQ1a3NKWYAZoXuKOITlN0qRLpYjzzaghOz8o3cBWwOXV51iy q0I05IKpm7CA== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548533" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:38 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 2/7] cxl/mem: Split creation from mapping in probe Date: Wed, 7 Apr 2021 15:26:20 -0700 Message-Id: <20210407222625.320177-3-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Add a new function specifically for mapping the register blocks and offsets within. The new function can be used more generically for other register block identifiers. No functional change is meant to be introduced in this patch with the exception of a dev_err printed when the device register block isn't found. Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron --- drivers/cxl/mem.c | 64 +++++++++++++++++++++++++++++------------------ 1 file changed, 40 insertions(+), 24 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 99534260034e..520edaf233d4 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -925,22 +925,40 @@ static int cxl_mem_setup_mailbox(struct cxl_mem *cxlm) return 0; } -static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, - u32 reg_hi) +static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev) { struct device *dev = &pdev->dev; struct cxl_mem *cxlm; - void __iomem *regs; - u64 offset; - u8 bar; - int rc; cxlm = devm_kzalloc(dev, sizeof(*cxlm), GFP_KERNEL); if (!cxlm) { dev_err(dev, "No memory available\n"); - return NULL; + return ERR_PTR(-ENOMEM); + } + + mutex_init(&cxlm->mbox_mutex); + cxlm->pdev = pdev; + cxlm->enabled_cmds = + devm_kmalloc_array(dev, BITS_TO_LONGS(cxl_cmd_count), + sizeof(unsigned long), + GFP_KERNEL | __GFP_ZERO); + if (!cxlm->enabled_cmds) { + dev_err(dev, "No memory available for bitmap\n"); + return ERR_PTR(-ENOMEM); } + return cxlm; +} + +static int cxl_mem_map_regblock(struct cxl_mem *cxlm, u32 reg_lo, u32 reg_hi) +{ + struct pci_dev *pdev = cxlm->pdev; + struct device *dev = &pdev->dev; + void __iomem *regs; + u64 offset; + u8 bar; + int rc; + offset = ((u64)reg_hi << 32) | FIELD_GET(CXL_REGLOC_ADDR_MASK, reg_lo); bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo); @@ -948,30 +966,20 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, if (pci_resource_len(pdev, bar) < offset) { dev_err(dev, "BAR%d: %pr: too small (offset: %#llx)\n", bar, &pdev->resource[bar], (unsigned long long)offset); - return NULL; + return -ENXIO; } rc = pcim_iomap_regions(pdev, BIT(bar), pci_name(pdev)); if (rc) { dev_err(dev, "failed to map registers\n"); - return NULL; + return rc; } regs = pcim_iomap_table(pdev)[bar]; - mutex_init(&cxlm->mbox_mutex); - cxlm->pdev = pdev; cxlm->base = regs + offset; - cxlm->enabled_cmds = - devm_kmalloc_array(dev, BITS_TO_LONGS(cxl_cmd_count), - sizeof(unsigned long), - GFP_KERNEL | __GFP_ZERO); - if (!cxlm->enabled_cmds) { - dev_err(dev, "No memory available for bitmap\n"); - return NULL; - } dev_dbg(dev, "Mapped CXL Memory Device resource\n"); - return cxlm; + return 0; } static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) @@ -1403,14 +1411,18 @@ static int cxl_mem_identify(struct cxl_mem *cxlm) static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct device *dev = &pdev->dev; - struct cxl_mem *cxlm = NULL; u32 regloc_size, regblocks; + struct cxl_mem *cxlm; int rc, regloc, i; rc = pcim_enable_device(pdev); if (rc) return rc; + cxlm = cxl_mem_create(pdev); + if (IS_ERR(cxlm)) + return PTR_ERR(cxlm); + regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); if (!regloc) { dev_err(dev, "register location dvsec not found\n"); @@ -1435,13 +1447,17 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo); if (reg_type == CXL_REGLOC_RBI_MEMDEV) { - cxlm = cxl_mem_create(pdev, reg_lo, reg_hi); + rc = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); + if (rc) + return rc; break; } } - if (!cxlm) - return -ENODEV; + if (i == regblocks) { + dev_err(dev, "Missing register locator for device registers\n"); + return -ENXIO; + } rc = cxl_mem_setup_regs(cxlm); if (rc) From patchwork Wed Apr 7 22:26:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9E94C43617 for ; Wed, 7 Apr 2021 22:26:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F35361262 for ; Wed, 7 Apr 2021 22:26:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229734AbhDGW1G (ORCPT ); Wed, 7 Apr 2021 18:27:06 -0400 Received: from mga09.intel.com ([134.134.136.24]:14309 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229657AbhDGW06 (ORCPT ); Wed, 7 Apr 2021 18:26:58 -0400 IronPort-SDR: fHHpqEZiczKjV2oBkqp2/ZbL/oqW7ZTdjhdGd8UH2U5NBUytZoB1AkfRPwJXvV8cUNRoM39Xvi 0IaL3QnizFaw== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524928" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524928" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 IronPort-SDR: pClVTSlqwwacSc9z/awTDGKWaVz1hLuOS38qdfm6o+pAZ1fUWlleqahXU628TVhmTUVKmm66Ja cY9IRoQRsOIQ== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548538" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 3/7] cxl/mem: Move register locator logic into reg setup Date: Wed, 7 Apr 2021 15:26:21 -0700 Message-Id: <20210407222625.320177-4-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Start moving code around to ultimately get rid of @cxlm.base. The @cxlm.base member serves no purpose other than intermediate storage of the offset found in cxl_mem_map_regblock() later used by cxl_mem_setup_regs(). Aside from wanting to get rid of this useless member, it will help later when adding new register block identifiers. While @cxlm.base still exists, it will become trivial to remove it in a future patch. No functional change is meant to be introduced in this patch. Signed-off-by: Ben Widawsky Reviewed-by: Jonathan Cameron --- drivers/cxl/mem.c | 135 +++++++++++++++++++++++----------------------- 1 file changed, 68 insertions(+), 67 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 520edaf233d4..04b4f7445083 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -870,34 +870,6 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm, u16 opcode, return 0; } -/** - * cxl_mem_setup_regs() - Setup necessary MMIO. - * @cxlm: The CXL memory device to communicate with. - * - * Return: 0 if all necessary registers mapped. - * - * A memory device is required by spec to implement a certain set of MMIO - * regions. The purpose of this function is to enumerate and map those - * registers. - */ -static int cxl_mem_setup_regs(struct cxl_mem *cxlm) -{ - struct device *dev = &cxlm->pdev->dev; - struct cxl_regs *regs = &cxlm->regs; - - cxl_setup_device_regs(dev, cxlm->base, ®s->device_regs); - - if (!regs->status || !regs->mbox || !regs->memdev) { - dev_err(dev, "registers not found: %s%s%s\n", - !regs->status ? "status " : "", - !regs->mbox ? "mbox " : "", - !regs->memdev ? "memdev" : ""); - return -ENXIO; - } - - return 0; -} - static int cxl_mem_setup_mailbox(struct cxl_mem *cxlm) { const int cap = readl(cxlm->regs.mbox + CXLDEV_MBOX_CAPS_OFFSET); @@ -1005,6 +977,73 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) return 0; } +/** + * cxl_mem_setup_regs() - Setup necessary MMIO. + * @cxlm: The CXL memory device to communicate with. + * + * Return: 0 if all necessary registers mapped. + * + * A memory device is required by spec to implement a certain set of MMIO + * regions. The purpose of this function is to enumerate and map those + * registers. + */ +static int cxl_mem_setup_regs(struct cxl_mem *cxlm) +{ + struct cxl_regs *regs = &cxlm->regs; + struct pci_dev *pdev = cxlm->pdev; + struct device *dev = &pdev->dev; + u32 regloc_size, regblocks; + int rc, regloc, i; + + regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); + if (!regloc) { + dev_err(dev, "register location dvsec not found\n"); + return -ENXIO; + } + + /* Get the size of the Register Locator DVSEC */ + pci_read_config_dword(pdev, regloc + PCI_DVSEC_HEADER1, ®loc_size); + regloc_size = FIELD_GET(PCI_DVSEC_HEADER1_LENGTH_MASK, regloc_size); + + regloc += PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET; + regblocks = (regloc_size - PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET) / 8; + + for (i = 0; i < regblocks; i++, regloc += 8) { + u32 reg_lo, reg_hi; + u8 reg_type; + + /* "register low and high" contain other bits */ + pci_read_config_dword(pdev, regloc, ®_lo); + pci_read_config_dword(pdev, regloc + 4, ®_hi); + + reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo); + + if (reg_type == CXL_REGLOC_RBI_MEMDEV) { + rc = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); + if (rc) + return rc; + break; + } + } + + if (i == regblocks) { + dev_err(dev, "Missing register locator for device registers\n"); + return -ENXIO; + } + + cxl_setup_device_regs(dev, cxlm->base, ®s->device_regs); + + if (!regs->status || !regs->mbox || !regs->memdev) { + dev_err(dev, "registers not found: %s%s%s\n", + !regs->status ? "status " : "", + !regs->mbox ? "mbox " : "", + !regs->memdev ? "memdev" : ""); + return -ENXIO; + } + + return 0; +} + static struct cxl_memdev *to_cxl_memdev(struct device *dev) { return container_of(dev, struct cxl_memdev, dev); @@ -1410,10 +1449,8 @@ static int cxl_mem_identify(struct cxl_mem *cxlm) static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) { - struct device *dev = &pdev->dev; - u32 regloc_size, regblocks; struct cxl_mem *cxlm; - int rc, regloc, i; + int rc; rc = pcim_enable_device(pdev); if (rc) @@ -1423,42 +1460,6 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (IS_ERR(cxlm)) return PTR_ERR(cxlm); - regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); - if (!regloc) { - dev_err(dev, "register location dvsec not found\n"); - return -ENXIO; - } - - /* Get the size of the Register Locator DVSEC */ - pci_read_config_dword(pdev, regloc + PCI_DVSEC_HEADER1, ®loc_size); - regloc_size = FIELD_GET(PCI_DVSEC_HEADER1_LENGTH_MASK, regloc_size); - - regloc += PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET; - regblocks = (regloc_size - PCI_DVSEC_ID_CXL_REGLOC_BLOCK1_OFFSET) / 8; - - for (i = 0; i < regblocks; i++, regloc += 8) { - u32 reg_lo, reg_hi; - u8 reg_type; - - /* "register low and high" contain other bits */ - pci_read_config_dword(pdev, regloc, ®_lo); - pci_read_config_dword(pdev, regloc + 4, ®_hi); - - reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo); - - if (reg_type == CXL_REGLOC_RBI_MEMDEV) { - rc = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); - if (rc) - return rc; - break; - } - } - - if (i == regblocks) { - dev_err(dev, "Missing register locator for device registers\n"); - return -ENXIO; - } - rc = cxl_mem_setup_regs(cxlm); if (rc) return rc; From patchwork Wed Apr 7 22:26:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 451A5C43460 for ; Wed, 7 Apr 2021 22:27:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1755761284 for ; Wed, 7 Apr 2021 22:27:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229813AbhDGW1O (ORCPT ); Wed, 7 Apr 2021 18:27:14 -0400 Received: from mga09.intel.com ([134.134.136.24]:14322 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229751AbhDGW1J (ORCPT ); Wed, 7 Apr 2021 18:27:09 -0400 IronPort-SDR: 3f8t/ToPf5qo75uizm0+DPfYsBG8tGexY6lDb29dHzgfDq6duvuscpA4d8GARY9z2ljnoQvVxS Qu2uKGendfnA== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524929" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524929" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 IronPort-SDR: KFu7CyY6/zJ+KBMtr+wAANDK+6kSKw9/27JY/eRm14ynO44VTmO1N3CC/Wllhfa4HEqtdq55EC riQD0VljYM7w== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548542" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 4/7] cxl/mem: Get rid of @cxlm.base Date: Wed, 7 Apr 2021 15:26:22 -0700 Message-Id: <20210407222625.320177-5-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org @cxlm.base only existed to support holding the base found in the register block mapping code, and pass it along to the register setup code. Now that the register setup function has all logic around managing the registers, from DVSEC to iomapping up to populating our CXL specific information, it is easy to turn the @base values into local variables and remove them from our device driver state. Signed-off-by: Ben Widawsky Acked-by: Jonathan Cameron --- drivers/cxl/mem.c | 24 +++++++++++------------- drivers/cxl/mem.h | 2 -- 2 files changed, 11 insertions(+), 15 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 04b4f7445083..60b95c524c3e 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -922,11 +922,10 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev) return cxlm; } -static int cxl_mem_map_regblock(struct cxl_mem *cxlm, u32 reg_lo, u32 reg_hi) +static void __iomem *cxl_mem_map_regblock(struct cxl_mem *cxlm, u32 reg_lo, u32 reg_hi) { struct pci_dev *pdev = cxlm->pdev; struct device *dev = &pdev->dev; - void __iomem *regs; u64 offset; u8 bar; int rc; @@ -938,20 +937,18 @@ static int cxl_mem_map_regblock(struct cxl_mem *cxlm, u32 reg_lo, u32 reg_hi) if (pci_resource_len(pdev, bar) < offset) { dev_err(dev, "BAR%d: %pr: too small (offset: %#llx)\n", bar, &pdev->resource[bar], (unsigned long long)offset); - return -ENXIO; + return (void __iomem *)ERR_PTR(-ENXIO); } rc = pcim_iomap_regions(pdev, BIT(bar), pci_name(pdev)); if (rc) { dev_err(dev, "failed to map registers\n"); - return rc; + return (void __iomem *)ERR_PTR(rc); } - regs = pcim_iomap_table(pdev)[bar]; - - cxlm->base = regs + offset; dev_dbg(dev, "Mapped CXL Memory Device resource\n"); - return 0; + + return pcim_iomap_table(pdev)[bar] + offset; } static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) @@ -993,7 +990,8 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) struct pci_dev *pdev = cxlm->pdev; struct device *dev = &pdev->dev; u32 regloc_size, regblocks; - int rc, regloc, i; + void __iomem *base; + int regloc, i; regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); if (!regloc) { @@ -1019,9 +1017,9 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo); if (reg_type == CXL_REGLOC_RBI_MEMDEV) { - rc = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); - if (rc) - return rc; + base = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); + if (IS_ERR(base)) + return PTR_ERR(base); break; } } @@ -1031,7 +1029,7 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) return -ENXIO; } - cxl_setup_device_regs(dev, cxlm->base, ®s->device_regs); + cxl_setup_device_regs(dev, base, ®s->device_regs); if (!regs->status || !regs->mbox || !regs->memdev) { dev_err(dev, "registers not found: %s%s%s\n", diff --git a/drivers/cxl/mem.h b/drivers/cxl/mem.h index 8bad7166adba..bfcfef461b16 100644 --- a/drivers/cxl/mem.h +++ b/drivers/cxl/mem.h @@ -49,7 +49,6 @@ struct cxl_memdev { /** * struct cxl_mem - A CXL memory device * @pdev: The PCI device associated with this CXL device. - * @base: IO mappings to the device's MMIO * @cxlmd: Logical memory device chardev / interface * @regs: Parsed register blocks * @payload_size: Size of space for payload @@ -62,7 +61,6 @@ struct cxl_memdev { */ struct cxl_mem { struct pci_dev *pdev; - void __iomem *base; struct cxl_memdev *cxlmd; struct cxl_regs regs; From patchwork Wed Apr 7 22:26:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 298AEC43460 for ; Wed, 7 Apr 2021 22:27:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 035AA61262 for ; Wed, 7 Apr 2021 22:27:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230322AbhDGW1X (ORCPT ); Wed, 7 Apr 2021 18:27:23 -0400 Received: from mga09.intel.com ([134.134.136.24]:14322 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229808AbhDGW1P (ORCPT ); Wed, 7 Apr 2021 18:27:15 -0400 IronPort-SDR: dW4IYUcxDExgcLO0ZQNl/7FZkmS37KpxyyG7A/nRh3pcETDti0r4lsKoYv/xVeAT/Tgss4tOaE I10ITMTRJZbw== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524931" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524931" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 IronPort-SDR: rhY2/D6RpXcG5mGIq1pg/6BWHHDCa1u27cFeIKVeUy4w9M1YDbKppyPJ/2I6lSlIDQJ5V2QSmF NfMpvZoWCtRQ== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548547" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 5/7] cxl/mem: Move device register setup Date: Wed, 7 Apr 2021 15:26:23 -0700 Message-Id: <20210407222625.320177-6-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Support expansion of register block types that the driver will attempt to recognize by pulling the code up into the register block scanning loop. Subsequent code can easily add in new register block types with this. Signed-off-by: Ben Widawsky Acked-by: Jonathan Cameron > --- drivers/cxl/mem.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 60b95c524c3e..49f651694cb0 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -1020,6 +1020,15 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) base = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); if (IS_ERR(base)) return PTR_ERR(base); + + cxl_setup_device_regs(dev, base, ®s->device_regs); + if (!regs->status || !regs->mbox || !regs->memdev) { + dev_err(dev, "registers not found: %s%s%s\n", + !regs->status ? "status " : "", + !regs->mbox ? "mbox " : "", + !regs->memdev ? "memdev" : ""); + return -ENXIO; + } break; } } @@ -1029,16 +1038,6 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) return -ENXIO; } - cxl_setup_device_regs(dev, base, ®s->device_regs); - - if (!regs->status || !regs->mbox || !regs->memdev) { - dev_err(dev, "registers not found: %s%s%s\n", - !regs->status ? "status " : "", - !regs->mbox ? "mbox " : "", - !regs->memdev ? "memdev" : ""); - return -ENXIO; - } - return 0; } From patchwork Wed Apr 7 22:26:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72D76C43600 for ; Wed, 7 Apr 2021 22:27:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D0C461262 for ; Wed, 7 Apr 2021 22:27:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230350AbhDGW1Y (ORCPT ); Wed, 7 Apr 2021 18:27:24 -0400 Received: from mga09.intel.com ([134.134.136.24]:14331 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229830AbhDGW1P (ORCPT ); Wed, 7 Apr 2021 18:27:15 -0400 IronPort-SDR: uUTE5S0JR5LavUcoC3Z3dKFerghBmsoSSXFHhxLyaD7pQGtaxdojyy8oHuTyr7TMGTMfLzkhNZ BvQ+DH7Q6xgQ== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524933" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524933" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:40 -0700 IronPort-SDR: dNHa0MOT1sReN41KYCAdZKPnlccO+W8MxQB87GzVASQfR56+bXSuo5naYNAqHVKte7lW8BJjfJ SE+CHQgOxdJQ== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548554" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:39 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 6/7] cxl/mem: Create a helper to setup device regs Date: Wed, 7 Apr 2021 15:26:24 -0700 Message-Id: <20210407222625.320177-7-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org Memory devices have a list of required register regions within the register block, but this isn't required of all CXL components or devices. To make things more tidy, and allow for easily setting up other block types in this loop, the helper is introduced. Signed-off-by: Ben Widawsky --- drivers/cxl/mem.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 49f651694cb0..b7342aaf38c4 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -974,6 +974,24 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) return 0; } +static int __cxl_setup_device_regs(struct cxl_mem *cxlm, void __iomem *base) +{ + struct cxl_regs *regs = &cxlm->regs; + struct pci_dev *pdev = cxlm->pdev; + struct device *dev = &pdev->dev; + + cxl_setup_device_regs(dev, base, ®s->device_regs); + if (!regs->status || !regs->mbox || !regs->memdev) { + dev_err(dev, "registers not found: %s%s%s\n", + !regs->status ? "status " : "", + !regs->mbox ? "mbox " : "", + !regs->memdev ? "memdev" : ""); + return -ENXIO; + } + + return 0; +} + /** * cxl_mem_setup_regs() - Setup necessary MMIO. * @cxlm: The CXL memory device to communicate with. @@ -986,12 +1004,11 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) */ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) { - struct cxl_regs *regs = &cxlm->regs; struct pci_dev *pdev = cxlm->pdev; struct device *dev = &pdev->dev; u32 regloc_size, regblocks; void __iomem *base; - int regloc, i; + int regloc, i, rc; regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC_OFFSET); if (!regloc) { @@ -1021,23 +1038,14 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) if (IS_ERR(base)) return PTR_ERR(base); - cxl_setup_device_regs(dev, base, ®s->device_regs); - if (!regs->status || !regs->mbox || !regs->memdev) { - dev_err(dev, "registers not found: %s%s%s\n", - !regs->status ? "status " : "", - !regs->mbox ? "mbox " : "", - !regs->memdev ? "memdev" : ""); - return -ENXIO; - } + rc = __cxl_setup_device_regs(cxlm, base); + if (rc) + return rc; + break; } } - if (i == regblocks) { - dev_err(dev, "Missing register locator for device registers\n"); - return -ENXIO; - } - return 0; } From patchwork Wed Apr 7 22:26:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Widawsky X-Patchwork-Id: 12189507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ABC1C433ED for ; Wed, 7 Apr 2021 22:28:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CF8C61176 for ; Wed, 7 Apr 2021 22:28:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229702AbhDGW2o (ORCPT ); Wed, 7 Apr 2021 18:28:44 -0400 Received: from mga09.intel.com ([134.134.136.24]:14322 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230019AbhDGW1S (ORCPT ); Wed, 7 Apr 2021 18:27:18 -0400 IronPort-SDR: lPaykhO6VkLQW/HdKprY7nxn9boQ/Jiaj0ue62D1GsaCNYo8nMg8IBYDejnnxminzpNJZ6oN02 Es+gBZRZbNaw== X-IronPort-AV: E=McAfee;i="6000,8403,9947"; a="193524935" X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="193524935" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:40 -0700 IronPort-SDR: R9ujXl52iR8YMDKrsOUx8qsQTner51AqLlnqDBiirEAx2zaj3ODQ3vmT0nFZ44ZJXw0Yc8192x XVtG3rxIOSsg== X-IronPort-AV: E=Sophos;i="5.82,204,1613462400"; d="scan'208";a="458548559" Received: from hmfaraby-mobl.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.128.243]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2021 15:26:40 -0700 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: Ben Widawsky , linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, dan.j.williams@intel.com, linux-kernel@vger.kernel.org Subject: [PATCH 7/7] cxl: Add HDM decoder capbilities Date: Wed, 7 Apr 2021 15:26:25 -0700 Message-Id: <20210407222625.320177-8-ben.widawsky@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210407222625.320177-1-ben.widawsky@intel.com> References: <20210407222625.320177-1-ben.widawsky@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org An HDM decoder is defined in the CXL 2.0 specification as a mechanism that allow devices and upstream ports to claim memory address ranges and participate in interleave sets. HDM decoder registers are within the component register block defined in CXL 2.0 8.2.3 CXL 2.0 Component Registers as part of the CXL.cache and CXL.mem subregion. The Component Register Block is found via the Register Locator DVSEC in a similar fashion to how the CXL Device Register Block is found. The primary difference is the capability id size of the Component Register Block is a single DWORD instead of 4 DWORDS. It's now possible to configure a CXL type 3 device's HDM decoder. Such programming is expected for CXL devices with persistent memory, and hot plugged CXL devices that participate in CXL.mem with volatile memory. Signed-off-by: Ben Widawsky --- drivers/cxl/core.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 48 ++++++++++++++++++++++++++++++ drivers/cxl/mem.c | 37 ++++++++++++++++++++--- drivers/cxl/pci.h | 1 + 4 files changed, 155 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core.c b/drivers/cxl/core.c index 65cd704581bc..db6a83eed0a2 100644 --- a/drivers/cxl/core.c +++ b/drivers/cxl/core.c @@ -479,6 +479,79 @@ struct cxl_port *devm_cxl_add_port(struct device *host, } EXPORT_SYMBOL_GPL(devm_cxl_add_port); +void cxl_setup_component_regs(struct device *dev, void __iomem *base, + struct cxl_component_regs *regs) +{ + int cap, cap_count; + u64 cap_array; + + *regs = (struct cxl_component_regs) { 0 }; + + /* + * CXL.cache and CXL.mem registers are at offset 0x1000 as defined in + * CXL 2.0 8.2.4 Table 141. + * + * TODO: Map other registers as needed. + */ + base += CXL_CM_OFFSET; + + cap_array = readq(base + CXL_CM_CAP_HDR_OFFSET); + + if (FIELD_GET(CXL_CM_CAP_HDR_ID_MASK, cap_array) != CM_CAP_HDR_CAP_ID) { + dev_err(dev, + "Couldn't locate the CXL.cache and CXL.mem capability array header./n"); + return; + } + + /* It's assumed that future versions will be backward compatible */ +#define CAPID_VERSION_CHECK(data, mask, expected, capability_msg) \ + do { \ + if (FIELD_GET(mask, data) < expected) { \ + dev_err(dev, \ + capability_msg \ + " version %ld is below expected %d", \ + FIELD_GET(mask, data), expected); \ + return; \ + } \ + } while (0) + + CAPID_VERSION_CHECK(cap_array, CXL_CM_CAP_HDR_VERSION_MASK, + CM_CAP_HDR_CAP_VERSION, "Capability array header"); + CAPID_VERSION_CHECK(cap_array, CXL_CM_CAP_HDR_CACHE_MEM_VERSION_MASK, + CM_CAP_HDR_CACHE_MEM_VERSION, + "Capability array header CXL.cache CXL.mem"); + + cap_count = FIELD_GET(CXL_CM_CAP_HDR_ARRAY_SIZE_MASK, cap_array); + + for (cap = 1; cap <= cap_count; cap++) { + void __iomem *register_block; + u32 hdr; + u16 cap_id, offset; + + hdr = readl(base + cap * 0x4); + + cap_id = FIELD_GET(CXL_CM_CAP_HDR_ID_MASK, hdr); + offset = FIELD_GET(CXL_CM_CAP_PTR_MASK, hdr); + register_block = base + offset; + + switch (cap_id) { + case CXL_CM_CAP_CAP_ID_HDM: + CAPID_VERSION_CHECK(hdr, CXL_CM_CAP_HDR_VERSION_MASK, + CXL_CM_CAP_CAP_HDM_VERSION, + "HDM decoder capability"); + dev_dbg(dev, "found HDM decoder capability (0x%x)\n", + offset); + regs->hdm_decoder = register_block; + break; + default: + dev_dbg(dev, "Unknown CM cap ID: %d (0x%x)\n", cap_id, + offset); + break; + } + } +} +EXPORT_SYMBOL_GPL(cxl_setup_component_regs); + /* * cxl_setup_device_regs() - Detect CXL Device register blocks * @dev: Host device of the @base mapping diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0211f44c95a2..a4ad1176dc5a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -8,6 +8,31 @@ #include #include +/* CXL 2.0 8.2.5 CXL.cache and CXL.mem Registers*/ +#define CXL_CM_OFFSET 0x1000 +#define CXL_CM_CAP_HDR_OFFSET 0x0 +#define CXL_CM_CAP_HDR_ID_MASK GENMASK(15, 0) +#define CM_CAP_HDR_CAP_ID 1 +#define CXL_CM_CAP_HDR_VERSION_MASK GENMASK(19, 16) +#define CM_CAP_HDR_CAP_VERSION 1 +#define CXL_CM_CAP_HDR_CACHE_MEM_VERSION_MASK GENMASK(23, 20) +#define CM_CAP_HDR_CACHE_MEM_VERSION 1 +#define CXL_CM_CAP_HDR_ARRAY_SIZE_MASK GENMASK(31, 24) +#define CXL_CM_CAP_PTR_MASK GENMASK(31, 20) + +#define CXL_CM_CAP_CAP_ID_HDM 0x5 +#define CXL_CM_CAP_CAP_HDM_VERSION 1 + +/* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */ +#define CXL_HDM_DECODER_CAP_OFFSET 0x0 +#define CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0) +#define CXL_HDM_DECODER_TARGET_COUNT_MASK GENMASK(7, 4) +#define CXL_HDM_DECODER0_BASE_LOW_OFFSET 0x10 +#define CXL_HDM_DECODER0_BASE_HIGH_OFFSET 0x14 +#define CXL_HDM_DECODER0_SIZE_LOW_OFFSET 0x18 +#define CXL_HDM_DECODER0_SIZE_HIGH_OFFSET 0x1c +#define CXL_HDM_DECODER0_CTRL_OFFSET 0x20 + /* CXL 2.0 8.2.8.1 Device Capabilities Array Register */ #define CXLDEV_CAP_ARRAY_OFFSET 0x0 #define CXLDEV_CAP_ARRAY_CAP_ID 0 @@ -35,11 +60,26 @@ #define CXLDEV_MBOX_PAYLOAD_OFFSET 0x20 /* See note for 'struct cxl_regs' for the rationale of this organization */ +#define CXL_COMPONENT_REGS() \ + void __iomem *hdm_decoder + #define CXL_DEVICE_REGS() \ void __iomem *status; \ void __iomem *mbox; \ void __iomem *memdev +/** + * struct cxl_component_regs - Common container of CXL component register block + * base pointers. + * + * The only component registers that we care about are the CXL.cache and CXL.mem + * registers which are at offset 0x1000 from the component register base (CXL + * 2.0 8.2.4) + */ +struct cxl_component_regs { + CXL_COMPONENT_REGS(); +}; + /** * struct cxl_device_regs - Common container of CXL Device register * block base pointers @@ -59,6 +99,12 @@ struct cxl_device_regs { * The specificity reads naturally from left-to-right. */ struct cxl_regs { + union { + struct { + CXL_COMPONENT_REGS(); + }; + struct cxl_component_regs component; + }; union { struct { CXL_DEVICE_REGS(); @@ -67,6 +113,8 @@ struct cxl_regs { }; }; +void cxl_setup_component_regs(struct device *dev, void __iomem *base, + struct cxl_component_regs *regs); void cxl_setup_device_regs(struct device *dev, void __iomem *base, struct cxl_device_regs *regs); diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index b7342aaf38c4..e915e3743b76 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -974,6 +974,21 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) return 0; } +static int __cxl_setup_component_regs(struct cxl_mem *cxlm, void __iomem *base) +{ + struct cxl_regs *regs = &cxlm->regs; + struct pci_dev *pdev = cxlm->pdev; + struct device *dev = &pdev->dev; + + cxl_setup_component_regs(dev, base, ®s->component); + if (!regs->hdm_decoder) { + dev_err(dev, "HDM decoder registers not found\n"); + return -ENXIO; + } + + return 0; +} + static int __cxl_setup_device_regs(struct cxl_mem *cxlm, void __iomem *base) { struct cxl_regs *regs = &cxlm->regs; @@ -1032,16 +1047,30 @@ static int cxl_mem_setup_regs(struct cxl_mem *cxlm) pci_read_config_dword(pdev, regloc + 4, ®_hi); reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo); + if (reg_type == CXL_REGLOC_RBI_EMPTY || + reg_type > CXL_REGLOC_RBI_MAX) + continue; + + base = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); + if (IS_ERR(base)) + return PTR_ERR(base); - if (reg_type == CXL_REGLOC_RBI_MEMDEV) { - base = cxl_mem_map_regblock(cxlm, reg_lo, reg_hi); - if (IS_ERR(base)) - return PTR_ERR(base); + switch (reg_type) { + case CXL_REGLOC_RBI_COMPONENT: + rc = __cxl_setup_component_regs(cxlm, base); + if (rc) + return rc; + dev_dbg(dev, "Set up component registers\n"); + break; + case CXL_REGLOC_RBI_MEMDEV: rc = __cxl_setup_device_regs(cxlm, base); if (rc) return rc; + dev_dbg(dev, "Set up device registers\n"); + break; + default: break; } } diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h index af3ec078cf6c..8b8c6afbe605 100644 --- a/drivers/cxl/pci.h +++ b/drivers/cxl/pci.h @@ -25,6 +25,7 @@ #define CXL_REGLOC_RBI_COMPONENT 1 #define CXL_REGLOC_RBI_VIRT 2 #define CXL_REGLOC_RBI_MEMDEV 3 +#define CXL_REGLOC_RBI_MAX CXL_REGLOC_RBI_MEMDEV #define CXL_REGLOC_ADDR_MASK GENMASK(31, 16)