From patchwork Thu Jun 16 15:12:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dandamudi, Priyanka" X-Patchwork-Id: 12884360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C17F7C433EF for ; Thu, 16 Jun 2022 15:23:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 241B810F006; Thu, 16 Jun 2022 15:23:08 +0000 (UTC) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6596210E2DA for ; Thu, 16 Jun 2022 15:23:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655392986; x=1686928986; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=cL/wjsAPqeqkGUL0G1pRSal339HEruSd0uUvTgal0mA=; b=bb1pcAF74xaVK83mmy5VXY3T8jrI2KFIOhJANxh1GdFyOogE4tTO945U m3LovSlCOLMOihiGtTk+O7LbIA+Zec7W7ah7swPSG8SfVn7VZvMhfWPAq a8cYPrpe9HomDvqWEfVQzybuTPotuq7rpgNWpXhpM891P3cAwwgDOKNvj tufNnRBXI2xwTsws+jof99IJ7xLgvODiO+3w6dItu3vUkUBNeQfleACLw H9kG1g1F9BP1SOp7aPNYGWYeZyGGth482FYEyYOLGBVJ9BKRIi4JLwYkT X8T6JE2Gk+bGA+G50norYYoIgMPZdhR7PiUFpFUbvskNkyJ3jBeW5H6pf w==; X-IronPort-AV: E=McAfee;i="6400,9594,10379"; a="259734230" X-IronPort-AV: E=Sophos;i="5.92,305,1650956400"; d="scan'208";a="259734230" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2022 08:23:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,305,1650956400"; d="scan'208";a="687862210" Received: from priyanka-nuc7i7bnh.iind.intel.com ([10.145.162.209]) by fmsmga002.fm.intel.com with ESMTP; 16 Jun 2022 08:23:04 -0700 From: priyanka.dandamudi@intel.com To: priyanka.dandamudi@intel.com, matthew.auld@intel.com, intel-gfx@lists.freedesktop.org Date: Thu, 16 Jun 2022 20:42:46 +0530 Message-Id: <20220616151247.1192063-2-priyanka.dandamudi@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220616151247.1192063-1-priyanka.dandamudi@intel.com> References: <20220616151247.1192063-1-priyanka.dandamudi@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/2] drm/i915: Add support for LMEM PCIe resizable bar X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Akeem G Abodunrin Add support for the local memory PICe resizable bar, so that local memory can be resized to the maximum size supported by the device, and mapped correctly to the PCIe memory bar. It is usual that GPU devices expose only 256MB BARs primarily to be compatible with 32-bit systems. So, those devices cannot claim larger memory BAR windows size due to the system BIOS limitation. With this change, it would be possible to reprogram the windows of the bridge directly above the requesting device on the same BAR type. Signed-off-by: Akeem G Abodunrin Signed-off-by: Michał Winiarski Cc: Stuart Summers Cc: Michael J Ruhl Cc: Prathap Kumar Valsan Signed-off-by: Priyanka Dandamudi Reviewed-by: Matthew Auld --- drivers/gpu/drm/i915/i915_driver.c | 92 ++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index d26dcca7e654..4bdb471cb2e2 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -303,6 +303,95 @@ static void sanitize_gpu(struct drm_i915_private *i915) __intel_gt_reset(to_gt(i915), ALL_ENGINES); } +static void __release_bars(struct pci_dev *pdev) +{ + int resno; + + for (resno = PCI_STD_RESOURCES; resno < PCI_STD_RESOURCE_END; resno++) { + if (pci_resource_len(pdev, resno)) + pci_release_resource(pdev, resno); + } +} + +static void +__resize_bar(struct drm_i915_private *i915, int resno, resource_size_t size) +{ + struct pci_dev *pdev = to_pci_dev(i915->drm.dev); + int bar_size = pci_rebar_bytes_to_size(size); + int ret; + + __release_bars(pdev); + + ret = pci_resize_resource(pdev, resno, bar_size); + if (ret) { + drm_info(&i915->drm, "Failed to resize BAR%d to %dM (%pe)\n", + resno, 1 << bar_size, ERR_PTR(ret)); + return; + } + + drm_info(&i915->drm, "BAR%d resized to %dM\n", resno, 1 << bar_size); +} + +/* BAR size starts from 1MB - 2^20 */ +#define BAR_SIZE_SHIFT 20 +static resource_size_t +__lmem_rebar_size(struct drm_i915_private *i915, int resno) +{ + struct pci_dev *pdev = to_pci_dev(i915->drm.dev); + u32 rebar = pci_rebar_get_possible_sizes(pdev, resno); + resource_size_t size; + + if (!rebar) + return 0; + + size = 1ULL << (__fls(rebar) + BAR_SIZE_SHIFT); + + if (size <= pci_resource_len(pdev, resno)) + return 0; + + return size; +} + +#define LMEM_BAR_NUM 2 +static void i915_resize_lmem_bar(struct drm_i915_private *i915) +{ + struct pci_dev *pdev = to_pci_dev(i915->drm.dev); + struct pci_bus *root = pdev->bus; + struct resource *root_res; + resource_size_t rebar_size = __lmem_rebar_size(i915, LMEM_BAR_NUM); + u32 pci_cmd; + int i; + + if (!rebar_size) + return; + + /* Find out if root bus contains 64bit memory addressing */ + while (root->parent) + root = root->parent; + + pci_bus_for_each_resource(root, root_res, i) { + if (root_res && root_res->flags & (IORESOURCE_MEM | + IORESOURCE_MEM_64) && root_res->start > 0x100000000ull) + break; + } + + /* pci_resize_resource will fail anyways */ + if (!root_res) { + drm_info(&i915->drm, "Can't resize LMEM BAR - platform support is missing\n"); + return; + } + + /* First disable PCI memory decoding references */ + pci_read_config_dword(pdev, PCI_COMMAND, &pci_cmd); + pci_write_config_dword(pdev, PCI_COMMAND, + pci_cmd & ~PCI_COMMAND_MEMORY); + + __resize_bar(i915, LMEM_BAR_NUM, rebar_size); + + pci_assign_unassigned_bus_resources(pdev->bus); + pci_write_config_dword(pdev, PCI_COMMAND, pci_cmd); +} + /** * i915_driver_early_probe - setup state not requiring device access * @dev_priv: device private @@ -852,6 +941,9 @@ int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent) disable_rpm_wakeref_asserts(&i915->runtime_pm); + if (HAS_LMEM(i915)) + i915_resize_lmem_bar(i915); + intel_vgpu_detect(i915); ret = intel_gt_probe_all(i915); From patchwork Thu Jun 16 15:12:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dandamudi, Priyanka" X-Patchwork-Id: 12884361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84690C433EF for ; Thu, 16 Jun 2022 15:23:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DC8B610E2DA; Thu, 16 Jun 2022 15:23:09 +0000 (UTC) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by gabe.freedesktop.org (Postfix) with ESMTPS id E1B9310E784 for ; Thu, 16 Jun 2022 15:23:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655392988; x=1686928988; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=py8l564JVZtoL3FVjILa+yhh0wAa2MLCAS56BoDOMzc=; b=WZq/Y4xZp1gl1AFDqPBsPnL4uial41pNWnghP22ogrGxzBYhcSjS3nBL h0SLOkAmRGosiSf05jGgDCkdM51SatGeNrlOLGS8MSf7satPJ8Dd+BnTo GEaeu8Hpu1xVNW2crhHAdOg4lDt5dLKywqZ5LMoDBt0zwAS7+KfPGxvE4 68IDuGrGRLhApdJnKwe6PsSpVnZ1jDwk5eHgDR4svLHxyFvE6kf04Co1I c3VJKm6Le4FwS7+/A0DwkaKxcoyic3omQPHuYJ+TADPlviHc6uBs2u6Me PENasYBvOYaU4wpyOi+rhVhPMukZqUL4lJUsETblSU62/zkLaM5cdxQCX w==; X-IronPort-AV: E=McAfee;i="6400,9594,10379"; a="259734241" X-IronPort-AV: E=Sophos;i="5.92,305,1650956400"; d="scan'208";a="259734241" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2022 08:23:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,305,1650956400"; d="scan'208";a="687862249" Received: from priyanka-nuc7i7bnh.iind.intel.com ([10.145.162.209]) by fmsmga002.fm.intel.com with ESMTP; 16 Jun 2022 08:23:05 -0700 From: priyanka.dandamudi@intel.com To: priyanka.dandamudi@intel.com, matthew.auld@intel.com, intel-gfx@lists.freedesktop.org Date: Thu, 16 Jun 2022 20:42:47 +0530 Message-Id: <20220616151247.1192063-3-priyanka.dandamudi@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220616151247.1192063-1-priyanka.dandamudi@intel.com> References: <20220616151247.1192063-1-priyanka.dandamudi@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/2] drm/i915: Add lmem_bar_size modparam X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" From: Priyanka Dandamudi For testing purposes, support forcing the lmem_bar_size through a new modparam. In CI we only have a limited number of configurations for DG2, but we still need to be reasonably sure we get a usable device (also verifying we report the correct values for things like probed_cpu_visible_size etc) with all the potential lmem_bar sizes that we might expect see in the wild. v2: Update commit message and a minor modification.(Matt) Cc: Matthew Auld Signed-off-by: Priyanka Dandamudi --- drivers/gpu/drm/i915/gt/intel_region_lmem.c | 4 +++ drivers/gpu/drm/i915/i915_driver.c | 28 ++++++++++++++++++++- drivers/gpu/drm/i915/i915_params.c | 2 ++ drivers/gpu/drm/i915/i915_params.h | 1 + 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gt/intel_region_lmem.c b/drivers/gpu/drm/i915/gt/intel_region_lmem.c index e9c12e0d6f59..4614c30f878f 100644 --- a/drivers/gpu/drm/i915/gt/intel_region_lmem.c +++ b/drivers/gpu/drm/i915/gt/intel_region_lmem.c @@ -111,6 +111,10 @@ static struct intel_memory_region *setup_lmem(struct intel_gt *gt) flat_ccs_base = intel_gt_read_register(gt, XEHPSDV_FLAT_CCS_BASE_ADDR); flat_ccs_base = (flat_ccs_base >> XEHPSDV_CCS_BASE_SHIFT) * SZ_64K; + /* XXX: Remove this once we have small-bar uapi bits */ + if (i915->params.lmem_bar_size > 0) + lmem_size = pci_resource_len(pdev, 2); + /* FIXME: Remove this when we have small-bar enabled */ if (pci_resource_len(pdev, 2) < lmem_size) { drm_err(&i915->drm, "System requires small-BAR support, which is currently unsupported on this kernel\n"); diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index 4bdb471cb2e2..b2763b032012 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -362,8 +362,34 @@ static void i915_resize_lmem_bar(struct drm_i915_private *i915) u32 pci_cmd; int i; - if (!rebar_size) + if (i915->params.lmem_bar_size > 0) { + u32 lmem_bar_size; + u32 set_bit; + u32 rebar; + u32 msb; + int k; + + lmem_bar_size = i915->params.lmem_bar_size; + rebar = pci_rebar_get_possible_sizes(pdev, LMEM_BAR_NUM); + msb = __fls(rebar); + + for (k = msb; k >= 0; k--) { + set_bit = (1 << k); + + if (set_bit & rebar) { + if (set_bit == lmem_bar_size) { + rebar_size = 1ULL << (__fls(lmem_bar_size) + + BAR_SIZE_SHIFT); + + if (rebar_size == pci_resource_len(pdev, LMEM_BAR_NUM)) + return; + break; + } + } + } + } else if (!rebar_size) { return; + } /* Find out if root bus contains 64bit memory addressing */ while (root->parent) diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c index 701fbc98afa0..6fc475a5db61 100644 --- a/drivers/gpu/drm/i915/i915_params.c +++ b/drivers/gpu/drm/i915/i915_params.c @@ -204,6 +204,8 @@ i915_param_named_unsafe(request_timeout_ms, uint, 0600, i915_param_named_unsafe(lmem_size, uint, 0400, "Set the lmem size(in MiB) for each region. (default: 0, all memory)"); +i915_param_named_unsafe(lmem_bar_size, uint, 0400, + "Set the lmem bar size(in MiB)."); static __always_inline void _print_param(struct drm_printer *p, const char *name, diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h index b5e7ea45d191..2733cb6cfe09 100644 --- a/drivers/gpu/drm/i915/i915_params.h +++ b/drivers/gpu/drm/i915/i915_params.h @@ -74,6 +74,7 @@ struct drm_printer; param(char *, force_probe, CONFIG_DRM_I915_FORCE_PROBE, 0400) \ param(unsigned int, request_timeout_ms, CONFIG_DRM_I915_REQUEST_TIMEOUT, CONFIG_DRM_I915_REQUEST_TIMEOUT ? 0600 : 0) \ param(unsigned int, lmem_size, 0, 0400) \ + param(unsigned int, lmem_bar_size, 0, 0400) \ /* leave bools at the end to not create holes */ \ param(bool, enable_hangcheck, true, 0600) \ param(bool, load_detect_test, false, 0600) \