From patchwork Mon Nov 26 06:05:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhenyu Wang X-Patchwork-Id: 10697603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1918E15A7 for ; Mon, 26 Nov 2018 06:06:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0627C29800 for ; Mon, 26 Nov 2018 06:06:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ECC9E2980B; Mon, 26 Nov 2018 06:06:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3F6F029800 for ; Mon, 26 Nov 2018 06:06:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 864866E2C4; Mon, 26 Nov 2018 06:06:07 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by gabe.freedesktop.org (Postfix) with ESMTPS id D09276E2C4; Mon, 26 Nov 2018 06:06:06 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Nov 2018 22:06:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,280,1539673200"; d="scan'208";a="89170731" Received: from debian-nuc.sh.intel.com ([10.239.159.41]) by fmsmga008.fm.intel.com with ESMTP; 25 Nov 2018 22:06:05 -0800 From: Zhenyu Wang To: joonas.lahtinen@linux.intel.com Date: Mon, 26 Nov 2018 14:05:53 +0800 Message-Id: <20181126060554.16481-2-zhenyuw@linux.intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181126060554.16481-1-zhenyuw@linux.intel.com> References: <20181126060554.16481-1-zhenyuw@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC PATCH v2 1/2] drm/i915: Export current required functions for GVT X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP This trys to export all required i915 functions for GVT. Signed-off-by: Zhenyu Wang --- drivers/gpu/drm/i915/i915_gem.c | 11 +++++++++++ drivers/gpu/drm/i915/i915_gem_context.c | 2 ++ drivers/gpu/drm/i915/i915_gem_dmabuf.c | 1 + drivers/gpu/drm/i915/i915_gem_fence_reg.c | 2 ++ drivers/gpu/drm/i915/i915_gem_gtt.c | 1 + drivers/gpu/drm/i915/i915_request.c | 3 +++ drivers/gpu/drm/i915/i915_vma.c | 2 ++ drivers/gpu/drm/i915/intel_ringbuffer.c | 1 + drivers/gpu/drm/i915/intel_runtime_pm.c | 2 ++ drivers/gpu/drm/i915/intel_uncore.c | 3 +++ 10 files changed, 28 insertions(+) diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index c55b1f75c980..9af6e9810f85 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -701,6 +701,7 @@ void *i915_gem_object_alloc(struct drm_i915_private *dev_priv) { return kmem_cache_zalloc(dev_priv->objects, GFP_KERNEL); } +EXPORT_SYMBOL_GPL(i915_gem_object_alloc); void i915_gem_object_free(struct drm_i915_gem_object *obj) { @@ -1029,6 +1030,7 @@ int i915_gem_obj_prepare_shmem_write(struct drm_i915_gem_object *obj, i915_gem_object_unpin_pages(obj); return ret; } +EXPORT_SYMBOL_GPL(i915_gem_obj_prepare_shmem_write); static void shmem_clflush_swizzled_range(char *addr, unsigned long length, @@ -2764,6 +2766,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj, list_add(&obj->mm.link, &i915->mm.unbound_list); spin_unlock(&i915->mm.obj_lock); } +EXPORT_SYMBOL_GPL(__i915_gem_object_set_pages); static int ____i915_gem_object_get_pages(struct drm_i915_gem_object *obj) { @@ -2930,6 +2933,7 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj, ptr = ERR_PTR(ret); goto out_unlock; } +EXPORT_SYMBOL_GPL(i915_gem_object_pin_map); static int i915_gem_object_pwrite_gtt(struct drm_i915_gem_object *obj, @@ -4041,6 +4045,7 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write) i915_gem_object_unpin_pages(obj); return 0; } +EXPORT_SYMBOL_GPL(i915_gem_object_set_to_gtt_domain); /** * Changes the cache-level of an object across all VMA. @@ -4406,6 +4411,7 @@ i915_gem_object_set_to_cpu_domain(struct drm_i915_gem_object *obj, bool write) return 0; } +EXPORT_SYMBOL_GPL(i915_gem_object_set_to_cpu_domain); /* Throttle our rendering by waiting until the ring has completed our requests * emitted over 20 msec ago. @@ -4535,6 +4541,7 @@ i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj, return vma; } +EXPORT_SYMBOL_GPL(i915_gem_object_ggtt_pin); static __always_inline unsigned int __busy_read_flag(unsigned int id) { @@ -4758,6 +4765,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj, i915_gem_info_add_obj(to_i915(obj->base.dev), obj->base.size); } +EXPORT_SYMBOL_GPL(i915_gem_object_init); static const struct drm_i915_gem_object_ops i915_gem_object_ops = { .flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE | @@ -4864,6 +4872,7 @@ i915_gem_object_create(struct drm_i915_private *dev_priv, u64 size) i915_gem_object_free(obj); return ERR_PTR(ret); } +EXPORT_SYMBOL_GPL(i915_gem_object_create); static bool discard_backing_storage(struct drm_i915_gem_object *obj) { @@ -5061,6 +5070,7 @@ void __i915_gem_object_release_unless_active(struct drm_i915_gem_object *obj) else i915_gem_object_put(obj); } +EXPORT_SYMBOL_GPL(__i915_gem_object_release_unless_active); void i915_gem_sanitize(struct drm_i915_private *i915) { @@ -6168,6 +6178,7 @@ i915_gem_object_get_page(struct drm_i915_gem_object *obj, unsigned int n) sg = i915_gem_object_get_sg(obj, n, &offset); return nth_page(sg_page(sg), offset); } +EXPORT_SYMBOL_GPL(i915_gem_object_get_page); /* Like i915_gem_object_get_page(), but mark the returned page dirty */ struct page * diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c index b97963db0287..212b8d176d25 100644 --- a/drivers/gpu/drm/i915/i915_gem_context.c +++ b/drivers/gpu/drm/i915/i915_gem_context.c @@ -274,6 +274,7 @@ void i915_gem_context_release(struct kref *ref) if (llist_add(&ctx->free_link, &i915->contexts.free_list)) queue_work(i915->wq, &i915->contexts.free_work); } +EXPORT_SYMBOL_GPL(i915_gem_context_release); static void context_close(struct i915_gem_context *ctx) { @@ -473,6 +474,7 @@ i915_gem_context_create_gvt(struct drm_device *dev) mutex_unlock(&dev->struct_mutex); return ctx; } +EXPORT_SYMBOL_GPL(i915_gem_context_create_gvt); static void destroy_kernel_context(struct i915_gem_context **ctxp) diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c index 82e2ca17a441..ac98b094220c 100644 --- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c +++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c @@ -244,6 +244,7 @@ struct dma_buf *i915_gem_prime_export(struct drm_device *dev, return drm_gem_dmabuf_export(dev, &exp_info); } +EXPORT_SYMBOL_GPL(i915_gem_prime_export); static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj) { diff --git a/drivers/gpu/drm/i915/i915_gem_fence_reg.c b/drivers/gpu/drm/i915/i915_gem_fence_reg.c index d548ac05ccd7..21b79c1f61d0 100644 --- a/drivers/gpu/drm/i915/i915_gem_fence_reg.c +++ b/drivers/gpu/drm/i915/i915_gem_fence_reg.c @@ -420,6 +420,7 @@ i915_reserve_fence(struct drm_i915_private *dev_priv) list_del(&fence->link); return fence; } +EXPORT_SYMBOL_GPL(i915_reserve_fence); /** * i915_unreserve_fence - Reclaim a reserved fence @@ -433,6 +434,7 @@ void i915_unreserve_fence(struct drm_i915_fence_reg *fence) list_add(&fence->link, &fence->i915->mm.fence_list); } +EXPORT_SYMBOL_GPL(i915_unreserve_fence); /** * i915_gem_revoke_fences - revoke fence state diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c index add1fe7aeb93..75dd6ebf517e 100644 --- a/drivers/gpu/drm/i915/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/i915_gem_gtt.c @@ -4040,6 +4040,7 @@ int i915_gem_gtt_insert(struct i915_address_space *vm, size, alignment, color, start, end, DRM_MM_INSERT_EVICT); } +EXPORT_SYMBOL_GPL(i915_gem_gtt_insert); #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) #include "selftests/mock_gtt.c" diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c index 71107540581d..0048fd9bb899 100644 --- a/drivers/gpu/drm/i915/i915_request.c +++ b/drivers/gpu/drm/i915/i915_request.c @@ -749,6 +749,7 @@ i915_request_alloc(struct intel_engine_cs *engine, struct i915_gem_context *ctx) intel_context_unpin(ce); return ERR_PTR(ret); } +EXPORT_SYMBOL_GPL(i915_request_alloc); static int i915_request_await_request(struct i915_request *to, struct i915_request *from) @@ -1073,6 +1074,7 @@ void i915_request_add(struct i915_request *request) if (prev && i915_request_completed(prev)) i915_request_retire_upto(prev); } +EXPORT_SYMBOL_GPL(i915_request_add); static unsigned long local_clock_us(unsigned int *cpu) { @@ -1339,6 +1341,7 @@ long i915_request_wait(struct i915_request *rq, return timeout; } +EXPORT_SYMBOL_GPL(i915_request_wait); static void ring_retire_requests(struct intel_ring *ring) { diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c index 5b4d78cdb4ca..704f18a8ecd3 100644 --- a/drivers/gpu/drm/i915/i915_vma.c +++ b/drivers/gpu/drm/i915/i915_vma.c @@ -783,6 +783,7 @@ void i915_vma_close(struct i915_vma *vma) */ list_add_tail(&vma->closed_link, &vma->vm->i915->gt.closed_vma); } +EXPORT_SYMBOL_GPL(i915_vma_close); void i915_vma_reopen(struct i915_vma *vma) { @@ -1028,6 +1029,7 @@ int i915_vma_move_to_active(struct i915_vma *vma, export_fence(vma, rq, flags); return 0; } +EXPORT_SYMBOL_GPL(i915_vma_move_to_active); int i915_vma_unbind(struct i915_vma *vma) { diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c index 87eebc13c0d8..c0d8693f1110 100644 --- a/drivers/gpu/drm/i915/intel_ringbuffer.c +++ b/drivers/gpu/drm/i915/intel_ringbuffer.c @@ -1957,6 +1957,7 @@ u32 *intel_ring_begin(struct i915_request *rq, unsigned int num_dwords) return cs; } +EXPORT_SYMBOL_GPL(intel_ring_begin); /* Align the ring tail to a cacheline boundary */ int intel_ring_cacheline_align(struct i915_request *rq) diff --git a/drivers/gpu/drm/i915/intel_runtime_pm.c b/drivers/gpu/drm/i915/intel_runtime_pm.c index 1c2de9b69a19..90a15e8d2afa 100644 --- a/drivers/gpu/drm/i915/intel_runtime_pm.c +++ b/drivers/gpu/drm/i915/intel_runtime_pm.c @@ -4004,6 +4004,7 @@ void intel_runtime_pm_get(struct drm_i915_private *dev_priv) atomic_inc(&dev_priv->runtime_pm.wakeref_count); assert_rpm_wakelock_held(dev_priv); } +EXPORT_SYMBOL_GPL(intel_runtime_pm_get); /** * intel_runtime_pm_get_if_in_use - grab a runtime pm reference if device in use @@ -4087,6 +4088,7 @@ void intel_runtime_pm_put(struct drm_i915_private *dev_priv) pm_runtime_mark_last_busy(kdev); pm_runtime_put_autosuspend(kdev); } +EXPORT_SYMBOL_GPL(intel_runtime_pm_put); /** * intel_runtime_pm_enable - enable runtime pm diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c index 9289515108c3..35293ac7dff9 100644 --- a/drivers/gpu/drm/i915/intel_uncore.c +++ b/drivers/gpu/drm/i915/intel_uncore.c @@ -644,6 +644,7 @@ void intel_uncore_forcewake_get(struct drm_i915_private *dev_priv, __intel_uncore_forcewake_get(dev_priv, fw_domains); spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags); } +EXPORT_SYMBOL_GPL(intel_uncore_forcewake_get); /** * intel_uncore_forcewake_user_get - claim forcewake on behalf of userspace @@ -756,6 +757,7 @@ void intel_uncore_forcewake_put(struct drm_i915_private *dev_priv, __intel_uncore_forcewake_put(dev_priv, fw_domains); spin_unlock_irqrestore(&dev_priv->uncore.lock, irqflags); } +EXPORT_SYMBOL_GPL(intel_uncore_forcewake_put); /** * intel_uncore_forcewake_put__locked - grab forcewake domain references @@ -2388,6 +2390,7 @@ intel_uncore_forcewake_for_reg(struct drm_i915_private *dev_priv, return fw_domains; } +EXPORT_SYMBOL_GPL(intel_uncore_forcewake_for_reg); #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST) #include "selftests/mock_uncore.c" From patchwork Mon Nov 26 06:05:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhenyu Wang X-Patchwork-Id: 10697607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DA1113AD for ; Mon, 26 Nov 2018 06:06:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68FD429800 for ; Mon, 26 Nov 2018 06:06:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5C9B32980B; Mon, 26 Nov 2018 06:06:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UPPERCASE_50_75 autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8D08B29800 for ; Mon, 26 Nov 2018 06:06:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 105176E2F1; Mon, 26 Nov 2018 06:06:14 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by gabe.freedesktop.org (Postfix) with ESMTPS id 70F5F6E2C7; Mon, 26 Nov 2018 06:06:08 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Nov 2018 22:06:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,280,1539673200"; d="scan'208";a="89170741" Received: from debian-nuc.sh.intel.com ([10.239.159.41]) by fmsmga008.fm.intel.com with ESMTP; 25 Nov 2018 22:06:06 -0800 From: Zhenyu Wang To: joonas.lahtinen@linux.intel.com Date: Mon, 26 Nov 2018 14:05:54 +0800 Message-Id: <20181126060554.16481-3-zhenyuw@linux.intel.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181126060554.16481-1-zhenyuw@linux.intel.com> References: <20181126060554.16481-1-zhenyuw@linux.intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [RFC PATCH v2 2/2] drm/i915: Move GVT device model into separate module X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP This makes GVT device model into a stand alone module named as "i915_gvt", which depends on i915 but load after i915 that do all left GVT host initialization setup. Currently i915_gvt module would still load "kvmgt" module to initialize for VFIO/mdev device, but this might be changed in future. As will make hypervisor module load to initialize GVT host instead. One problem by this split is that GVT requires HW initial MMIO state as base initial state for each vGPU after create, which should be taken in very early stage of i915 load to reflect HW state. Just reading all MMIO won't work, as it could cause unknown side effect e.g HW hang. So current choice is to read back MMIO that GVT actually track. But instead of reference any GVT files, this one just copied register list. So GVT will use initial dump instead when load. Cc: Joonas Lahtinen Signed-off-by: Zhenyu Wang --- drivers/gpu/drm/i915/Kconfig | 2 +- drivers/gpu/drm/i915/Makefile | 4 +- drivers/gpu/drm/i915/gvt/Makefile | 8 +- drivers/gpu/drm/i915/gvt/firmware.c | 2 +- drivers/gpu/drm/i915/gvt/gvt.c | 52 +- drivers/gpu/drm/i915/gvt/gvt.h | 3 + drivers/gpu/drm/i915/i915_drv.c | 43 +- drivers/gpu/drm/i915/i915_drv.h | 10 +- drivers/gpu/drm/i915/i915_params.c | 5 - drivers/gpu/drm/i915/i915_params.h | 3 +- drivers/gpu/drm/i915/intel_gvt.c | 1519 +++++++++++++++++++++++++-- drivers/gpu/drm/i915/intel_gvt.h | 29 +- 12 files changed, 1547 insertions(+), 133 deletions(-) diff --git a/drivers/gpu/drm/i915/Kconfig b/drivers/gpu/drm/i915/Kconfig index 33a458b7f1fc..a05261cabf53 100644 --- a/drivers/gpu/drm/i915/Kconfig +++ b/drivers/gpu/drm/i915/Kconfig @@ -96,7 +96,7 @@ config DRM_I915_USERPTR If in doubt, say "Y". config DRM_I915_GVT - bool "Enable Intel GVT-g graphics virtualization host support" + tristate "Enable Intel GVT-g graphics virtualization host support" depends on DRM_I915 depends on 64BIT default n diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile index 0ff878c994e2..3c23d0a5552a 100644 --- a/drivers/gpu/drm/i915/Makefile +++ b/drivers/gpu/drm/i915/Makefile @@ -186,12 +186,12 @@ i915-y += i915_perf.o \ i915_oa_cnl.o \ i915_oa_icl.o -ifeq ($(CONFIG_DRM_I915_GVT),y) +ifneq ($(CONFIG_DRM_I915_GVT),n) i915-y += intel_gvt.o -include $(src)/gvt/Makefile endif # LPE Audio for VLV and CHT i915-y += intel_lpe_audio.o obj-$(CONFIG_DRM_I915) += i915.o +obj-$(CONFIG_DRM_I915_GVT) += gvt/ diff --git a/drivers/gpu/drm/i915/gvt/Makefile b/drivers/gpu/drm/i915/gvt/Makefile index b016dc753db9..f41cfc7625b4 100644 --- a/drivers/gpu/drm/i915/gvt/Makefile +++ b/drivers/gpu/drm/i915/gvt/Makefile @@ -1,10 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 -GVT_DIR := gvt GVT_SOURCE := gvt.o aperture_gm.o handlers.o vgpu.o trace_points.o firmware.o \ interrupt.o gtt.o cfg_space.o opregion.o mmio.o display.o edid.o \ execlist.o scheduler.o sched_policy.o mmio_context.o cmd_parser.o debugfs.o \ fb_decoder.o dmabuf.o page_track.o -ccflags-y += -I$(src) -I$(src)/$(GVT_DIR) -i915-y += $(addprefix $(GVT_DIR)/, $(GVT_SOURCE)) -obj-$(CONFIG_DRM_I915_GVT_KVMGT) += $(GVT_DIR)/kvmgt.o +ccflags-y += -I$(src) -I$(src)/../ +i915_gvt-y := $(GVT_SOURCE) +obj-$(CONFIG_DRM_I915_GVT_KVMGT) += kvmgt.o +obj-$(CONFIG_DRM_I915_GVT) += i915_gvt.o diff --git a/drivers/gpu/drm/i915/gvt/firmware.c b/drivers/gpu/drm/i915/gvt/firmware.c index 4ac18b447247..7a3441fb9f70 100644 --- a/drivers/gpu/drm/i915/gvt/firmware.c +++ b/drivers/gpu/drm/i915/gvt/firmware.c @@ -70,7 +70,7 @@ static int mmio_snapshot_handler(struct intel_gvt *gvt, u32 offset, void *data) { struct drm_i915_private *dev_priv = gvt->dev_priv; - *(u32 *)(data + offset) = I915_READ_NOTRACE(_MMIO(offset)); + *(u32 *)(data + offset) = dev_priv->hw_init_mmio[offset >> 2]; return 0; } diff --git a/drivers/gpu/drm/i915/gvt/gvt.c b/drivers/gpu/drm/i915/gvt/gvt.c index 733a2a0d0c30..bde20d86acf5 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.c +++ b/drivers/gpu/drm/i915/gvt/gvt.c @@ -30,10 +30,11 @@ * */ +#include #include #include #include - +#include #include "i915_drv.h" #include "gvt.h" #include @@ -216,8 +217,8 @@ int intel_gvt_init_host(void) } else { #if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT) /* not in Xen. Try KVMGT */ - intel_gvt_host.mpt = try_then_request_module( - symbol_get(kvmgt_mpt), "kvmgt"); + request_module("kvmgt"); + intel_gvt_host.mpt = &kvmgt_mpt; intel_gvt_host.hypervisor_type = INTEL_GVT_HYPERVISOR_KVM; #endif } @@ -467,6 +468,51 @@ int intel_gvt_init_device(struct drm_i915_private *dev_priv) return ret; } +static struct drm_i915_private *i915_priv; + +static int __init i915_gvt_init(void) +{ + struct drm_i915_private *priv; + + if (intel_gvt_init_host()) { + gvt_err("init host fail\n"); + goto err1; + } + + priv = i915_private_get(); + if (!priv) { + gvt_err("get i915 private instance fail\n"); + goto err1; + } + + if (intel_gvt_init_device(priv)) { + gvt_err("init device fail\n"); + goto err2; + } + i915_priv = priv; + + printk("i915_gvt_init success\n"); + return 0; +err2: + i915_private_put(priv); +err1: + return -1; +} + +static void __exit i915_gvt_exit(void) +{ + if (!intel_gvt_active(i915_priv)) + return; + intel_gvt_clean_device(i915_priv); + i915_private_put(i915_priv); +} + +module_init(i915_gvt_init); +module_exit(i915_gvt_exit); + #if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT) MODULE_SOFTDEP("pre: kvmgt"); #endif + +MODULE_LICENSE("GPL and additional rights"); +MODULE_AUTHOR("Intel Corporation"); diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h index 31f6cdbe5c42..f5e9869ae91a 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.h +++ b/drivers/gpu/drm/i915/gvt/gvt.h @@ -480,6 +480,9 @@ static inline void intel_vgpu_write_pci_bar(struct intel_vgpu *vgpu, int intel_gvt_init_vgpu_types(struct intel_gvt *gvt); void intel_gvt_clean_vgpu_types(struct intel_gvt *gvt); +extern int intel_gvt_init_device(struct drm_i915_private *dev_priv); +extern void intel_gvt_clean_device(struct drm_i915_private *dev_priv); +extern int intel_gvt_init_host(void); struct intel_vgpu *intel_gvt_create_idle_vgpu(struct intel_gvt *gvt); void intel_gvt_destroy_idle_vgpu(struct intel_vgpu *vgpu); diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c index b1d23c73c147..c9387345180d 100644 --- a/drivers/gpu/drm/i915/i915_drv.c +++ b/drivers/gpu/drm/i915/i915_drv.c @@ -53,6 +53,7 @@ #include "i915_vgpu.h" #include "intel_drv.h" #include "intel_uc.h" +#include "intel_gvt.h" static struct drm_driver driver; @@ -1023,6 +1024,8 @@ static int i915_driver_init_mmio(struct drm_i915_private *dev_priv) intel_uncore_init(dev_priv); + intel_gvt_init_mmio(dev_priv); + intel_device_info_init_mmio(dev_priv); intel_uncore_prune(dev_priv); @@ -1057,11 +1060,6 @@ static void i915_driver_cleanup_mmio(struct drm_i915_private *dev_priv) pci_dev_put(dev_priv->bridge_dev); } -static void intel_sanitize_options(struct drm_i915_private *dev_priv) -{ - intel_gvt_sanitize_options(dev_priv); -} - static enum dram_rank skl_get_dimm_rank(u8 size, u32 rank) { if (size == 0) @@ -1383,8 +1381,6 @@ static int i915_driver_init_hw(struct drm_i915_private *dev_priv) } } - intel_sanitize_options(dev_priv); - i915_perf_init(dev_priv); ret = i915_ggtt_probe_hw(dev_priv); @@ -1478,10 +1474,6 @@ static int i915_driver_init_hw(struct drm_i915_private *dev_priv) DRM_DEBUG_DRIVER("can't enable MSI"); } - ret = intel_gvt_init(dev_priv); - if (ret) - goto err_msi; - intel_opregion_setup(dev_priv); /* * Fill the dram structure to get the system raw bandwidth and @@ -1492,10 +1484,6 @@ static int i915_driver_init_hw(struct drm_i915_private *dev_priv) return 0; -err_msi: - if (pdev->msi_enabled) - pci_disable_msi(pdev); - pm_qos_remove_request(&dev_priv->pm_qos); err_ggtt: i915_ggtt_cleanup_hw(dev_priv); err_perf: @@ -1677,6 +1665,25 @@ static void i915_driver_destroy(struct drm_i915_private *i915) pci_set_drvdata(pdev, NULL); } +static struct drm_i915_private *i915_priv_prime; + +struct drm_i915_private *i915_private_get(void) +{ + if (!i915_priv_prime) + return NULL; + + try_module_get(i915_priv_prime->drm.dev->driver->owner); + return i915_priv_prime; +} +EXPORT_SYMBOL_GPL(i915_private_get); + +void i915_private_put(struct drm_i915_private *priv) +{ + if (priv) + module_put(priv->drm.dev->driver->owner); +} +EXPORT_SYMBOL_GPL(i915_private_put); + /** * i915_driver_load - setup chip and create an initial config * @pdev: PCI device @@ -1731,6 +1738,8 @@ int i915_driver_load(struct pci_dev *pdev, const struct pci_device_id *ent) i915_welcome_messages(dev_priv); + i915_priv_prime = dev_priv; + return 0; out_cleanup_hw: @@ -1755,6 +1764,8 @@ void i915_driver_unload(struct drm_device *dev) disable_rpm_wakeref_asserts(dev_priv); + intel_gvt_clean_mmio(dev_priv); + i915_driver_unregister(dev_priv); if (i915_gem_suspend(dev_priv)) @@ -1762,8 +1773,6 @@ void i915_driver_unload(struct drm_device *dev) drm_atomic_helper_shutdown(dev); - intel_gvt_cleanup(dev_priv); - intel_modeset_cleanup(dev); intel_bios_cleanup(dev_priv); diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h index 21e4405e2168..28495b558b05 100644 --- a/drivers/gpu/drm/i915/i915_drv.h +++ b/drivers/gpu/drm/i915/i915_drv.h @@ -81,8 +81,6 @@ #include "i915_timeline.h" #include "i915_vma.h" -#include "intel_gvt.h" - /* General customization: */ @@ -1432,6 +1430,8 @@ struct intel_cdclk_state { u8 voltage_level; }; +struct intel_gvt; + struct drm_i915_private { struct drm_device drm; @@ -2013,6 +2013,9 @@ struct drm_i915_private { struct i915_pmu pmu; +#if defined(CONFIG_DRM_I915_GVT) || defined(CONFIG_DRM_I915_GVT_MODULE) + u32 *hw_init_mmio; +#endif /* * NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch * will be rejected. Instead look for a better place. @@ -3720,4 +3723,7 @@ static inline int intel_hws_csb_write_index(struct drm_i915_private *i915) return I915_HWS_CSB_WRITE_INDEX; } +struct drm_i915_private *i915_private_get(void); +void i915_private_put(struct drm_i915_private *); + #endif diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c index 2e0356561839..3928959aa65e 100644 --- a/drivers/gpu/drm/i915/i915_params.c +++ b/drivers/gpu/drm/i915/i915_params.c @@ -167,11 +167,6 @@ i915_param_named_unsafe(inject_load_failure, uint, 0400, i915_param_named(enable_dpcd_backlight, bool, 0600, "Enable support for DPCD backlight control (default:false)"); -#if IS_ENABLED(CONFIG_DRM_I915_GVT) -i915_param_named(enable_gvt, bool, 0400, - "Enable support for Intel GVT-g graphics virtualization host support(default:false)"); -#endif - static __always_inline void _print_param(struct drm_printer *p, const char *name, const char *type, diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h index 7e56c516c815..f27078556c50 100644 --- a/drivers/gpu/drm/i915/i915_params.h +++ b/drivers/gpu/drm/i915/i915_params.h @@ -66,8 +66,7 @@ struct drm_printer; param(bool, verbose_state_checks, true) \ param(bool, nuclear_pageflip, false) \ param(bool, enable_dp_mst, true) \ - param(bool, enable_dpcd_backlight, false) \ - param(bool, enable_gvt, false) + param(bool, enable_dpcd_backlight, false) #define MEMBER(T, member, ...) T member; struct i915_params { diff --git a/drivers/gpu/drm/i915/intel_gvt.c b/drivers/gpu/drm/i915/intel_gvt.c index c22b3e18a0f5..20d276458c45 100644 --- a/drivers/gpu/drm/i915/intel_gvt.c +++ b/drivers/gpu/drm/i915/intel_gvt.c @@ -19,7 +19,7 @@ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - */ +*/ #include "i915_drv.h" #include "intel_gvt.h" @@ -39,6 +39,1425 @@ * doc is available on https://01.org/group/2230/documentation-list. */ +/* + * GVT-g needs initial MMIO/cfg state for each vGPU, which should be got + * very early to reflect initial HW state. So this trys to read early during + * i915 load, and provide for GVT later when vGPU init. But we can't just + * blindly read all MMIO which might simply cause side effect e.g hang. So + * this trys to only read GVT-g tracked MMIO. + * + * This file is aligned with gvt/handler.c's definition for all tracked MMIO. + * Any change in gvt should also add here. + */ + +#define MMIO_R(reg, size) do { \ + int i; \ + for (i = 0; i < size; i+=4) {\ + unsigned int off = i915_mmio_reg_offset((reg)); \ + dev_priv->hw_init_mmio[(off + i) >> 2] = I915_READ_FW(_MMIO(off + i)); \ + }\ +} while (0) + +#define MMIO_F(reg) do {\ + unsigned int off = i915_mmio_reg_offset((reg));\ + dev_priv->hw_init_mmio[off>>2] = I915_READ_FW((reg)); \ + } while(0) +#define MMIO_D(reg) MMIO_F(reg) +#define MMIO_DH(reg) MMIO_F(reg) +#define MMIO_DFH(reg) MMIO_F(reg) +#define MMIO_GM(reg) MMIO_F(reg) +#define MMIO_GM_RDR(reg) MMIO_F(reg) +#define MMIO_RO(reg) MMIO_F(reg) + +#define MMIO_RING_F(prefix) \ + MMIO_F(prefix(RENDER_RING_BASE)); \ + MMIO_F(prefix(BLT_RING_BASE)); \ + MMIO_F(prefix(GEN6_BSD_RING_BASE)); \ + MMIO_F(prefix(VEBOX_RING_BASE)); \ + MMIO_F(prefix(GEN8_BSD2_RING_BASE)) + +#define MMIO_RING_D(prefix) MMIO_RING_F(prefix) +#define MMIO_RING_DFH(prefix) MMIO_RING_F(prefix) +#define MMIO_RING_GM(prefix) MMIO_RING_F(prefix) +#define MMIO_RING_GM_RDR(prefix) MMIO_RING_F(prefix) +#define MMIO_RING_RO(prefix) MMIO_RING_F(prefix) + +/* GVT add-on regs */ +#define PCH_GPIO_BASE _MMIO(0xc5010) + +#define PCH_GMBUS0 _MMIO(0xc5100) +#define PCH_GMBUS1 _MMIO(0xc5104) +#define PCH_GMBUS2 _MMIO(0xc5108) +#define PCH_GMBUS3 _MMIO(0xc510c) +#define PCH_GMBUS4 _MMIO(0xc5110) +#define PCH_GMBUS5 _MMIO(0xc5120) +#define TRVATTL3PTRDW(i) _MMIO(0x4de0 + (i) * 4) +#define TRNULLDETCT _MMIO(0x4de8) +#define TRINVTILEDETCT _MMIO(0x4dec) +#define TRVADR _MMIO(0x4df0) +#define TRTTE _MMIO(0x4df4) +#define RING_EXCC(base) _MMIO((base) + 0x28) +#define RING_GFX_MODE(base) _MMIO((base) + 0x29c) +#define VF_GUARDBAND _MMIO(0x83a4) + +#define PCH_PP_STATUS _MMIO(0xc7200) +#define PCH_PP_CONTROL _MMIO(0xc7204) +#define PCH_PP_ON_DELAYS _MMIO(0xc7208) +#define PCH_PP_OFF_DELAYS _MMIO(0xc720c) +#define PCH_PP_DIVISOR _MMIO(0xc7210) + +#define _REG_701C0(pipe, plane) (0x701c0 + pipe * 0x1000 + (plane - 1) * 0x100) +#define _REG_701C4(pipe, plane) (0x701c4 + pipe * 0x1000 + (plane - 1) * 0x100) + +static void +init_general_range(struct drm_i915_private *dev_priv) +{ + MMIO_R(_MMIO(MCHBAR_MIRROR_BASE_SNB), 0x40000); + MMIO_R(LGC_PALETTE(PIPE_A, 0), 1024); + MMIO_R(LGC_PALETTE(PIPE_B, 0), 1024); + MMIO_R(LGC_PALETTE(PIPE_C, 0), 1024); + + MMIO_R(_MMIO(0x60220), 0x20); + MMIO_R(PCH_GMBUS0, 16); + MMIO_R(PCH_GPIO_BASE, 24); + MMIO_R(_MMIO(0xe4f00), 0x28); + MMIO_R(_MMIO(_PCH_DPB_AUX_CH_CTL), 6 * 4); + MMIO_R(_MMIO(_PCH_DPC_AUX_CH_CTL), 6 * 4); + MMIO_R(_MMIO(_PCH_DPD_AUX_CH_CTL), 6 * 4); + MMIO_R(PREC_PAL_GC_MAX(PIPE_A, 0), 4 * 3); + MMIO_R(PREC_PAL_GC_MAX(PIPE_B, 0), 4 * 3); + MMIO_R(PREC_PAL_GC_MAX(PIPE_C, 0), 4 * 3); + MMIO_R(_MMIO(0x70400), 0x40); + MMIO_R(_MMIO(0x71400), 0x40); + MMIO_R(_MMIO(0x72400), 0x40); + MMIO_R(_MMIO(0x70440), 0xc); + MMIO_R(_MMIO(0x71440), 0xc); + MMIO_R(_MMIO(0x72440), 0xc); + MMIO_R(_MMIO(0x7044c), 0xc); + MMIO_R(_MMIO(0x7144c), 0xc); + MMIO_R(_MMIO(0x7244c), 0xc); + + MMIO_R(_MMIO(0x49090), 0x14); + MMIO_R(_MMIO(0x49190), 0x14); + MMIO_R(_MMIO(0x49290), 0x14); + + MMIO_R(_MMIO(_DPA_AUX_CH_CTL), 6 * 4); + + MMIO_R(_MMIO(_DDI_BUF_TRANS_A), 0x50); + MMIO_R(_MMIO(0x64e60), 0x50); + MMIO_R(_MMIO(0x64eC0), 0x50); + MMIO_R(_MMIO(0x64f20), 0x50); + MMIO_R(_MMIO(0x64f80), 0x50); + + MMIO_R(FENCE_REG_GEN6_LO(0), 0x80); + MMIO_R(_MMIO(0x4f000), 0x90); + MMIO_R(_MMIO(0x2290), 8); + MMIO_R(_MMIO(0x5200), 32); + MMIO_R(_MMIO(0x5240), 32); + MMIO_R(_MMIO(0x5280), 16); + MMIO_R(HS_INVOCATION_COUNT, 8); + MMIO_R(DS_INVOCATION_COUNT, 8); + MMIO_R(IA_VERTICES_COUNT, 8); + MMIO_R(IA_PRIMITIVES_COUNT, 8); + MMIO_R(VS_INVOCATION_COUNT, 8); + MMIO_R(GS_INVOCATION_COUNT, 8); + MMIO_R(GS_PRIMITIVES_COUNT, 8); + MMIO_R(CL_INVOCATION_COUNT, 8); + MMIO_R(CL_PRIMITIVES_COUNT, 8); + MMIO_R(PS_INVOCATION_COUNT, 8); + MMIO_R(PS_DEPTH_COUNT, 8); +} + +static void init_bdw_range(struct drm_i915_private *dev_priv) +{ + MMIO_R(_MMIO(RENDER_RING_BASE + 0x234), 8); + MMIO_R(_MMIO(BLT_RING_BASE + 0x234), 8); + MMIO_R(_MMIO(GEN6_BSD_RING_BASE + 0x234), 8); + MMIO_R(_MMIO(VEBOX_RING_BASE + 0x234), 8); + MMIO_R(_MMIO(GEN8_BSD2_RING_BASE + 0x234), 8); + + MMIO_R( _MMIO(RENDER_RING_BASE + 0x370), 48); + MMIO_R(_MMIO(BLT_RING_BASE + 0x370), 48); + MMIO_R(_MMIO(GEN6_BSD_RING_BASE + 0x370), 48); + MMIO_R(_MMIO(VEBOX_RING_BASE + 0x370), 48); + MMIO_R(_MMIO(GEN8_BSD2_RING_BASE + 0x370), 48); + + MMIO_R(_MMIO(RENDER_RING_BASE + 0x270), 32); + MMIO_R(_MMIO(BLT_RING_BASE + 0x270), 32); + MMIO_R(_MMIO(GEN6_BSD_RING_BASE + 0x270), 32); + MMIO_R(_MMIO(VEBOX_RING_BASE + 0x270), 32); + MMIO_R(_MMIO(GEN8_BSD2_RING_BASE + 0x270), 32); + + MMIO_R(_MMIO(0x24d0), 48); +} + +static void init_skl_range(struct drm_i915_private *dev_priv) +{ + MMIO_R(_MMIO(CSR_MMIO_START_RANGE), 0x3000); + + MMIO_R(_MMIO(_DPB_AUX_CH_CTL), 6 * 4); + MMIO_R(_MMIO(_DPC_AUX_CH_CTL), 6 * 4); + MMIO_R(_MMIO(_DPD_AUX_CH_CTL), 6 * 4); + + MMIO_R(PLANE_WM(PIPE_A, 0, 0), 4 * 8); + MMIO_R(PLANE_WM(PIPE_A, 1, 0), 4 * 8); + MMIO_R(PLANE_WM(PIPE_A, 2, 0), 4 * 8); + + MMIO_R(PLANE_WM(PIPE_B, 0, 0), 4 * 8); + MMIO_R(PLANE_WM(PIPE_B, 1, 0), 4 * 8); + MMIO_R(PLANE_WM(PIPE_B, 2, 0), 4 * 8); + + MMIO_R(PLANE_WM(PIPE_C, 0, 0), 4 * 8); + MMIO_R(PLANE_WM(PIPE_C, 1, 0), 4 * 8); + MMIO_R(PLANE_WM(PIPE_C, 2, 0), 4 * 8); + + MMIO_R(CUR_WM(PIPE_A, 0), 4 * 8); + MMIO_R(CUR_WM(PIPE_B, 0), 4 * 8); + MMIO_R(CUR_WM(PIPE_C, 0), 4 * 8); + + MMIO_R(_MMIO(0xc800), 0x7f8); + MMIO_R(_MMIO(0xb020), 0x80); +} + +static void init_bxt_range(struct drm_i915_private *dev_priv) +{ + MMIO_R(_MMIO(0x80000), 0x3000); + MMIO_R(SOFT_SCRATCH(0), 16 * 4); +} + +static void init_general_mmio(struct drm_i915_private *dev_priv) +{ + MMIO_RING_DFH(RING_IMR); + + MMIO_DFH(SDEIMR); + MMIO_DFH(SDEIER); + MMIO_DFH(SDEIIR); + MMIO_D(SDEISR); + + MMIO_RING_DFH(RING_HWSTAM); + + MMIO_DH(GEN8_GAMW_ECO_DEV_RW_IA); + MMIO_GM_RDR(BSD_HWS_PGA_GEN7); + MMIO_GM_RDR(BLT_HWS_PGA_GEN7); + MMIO_GM_RDR(VEBOX_HWS_PGA_GEN7); + +#define RING_REG(base) _MMIO((base) + 0x28) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + +#define RING_REG(base) _MMIO((base) + 0x134) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + +#define RING_REG(base) _MMIO((base) + 0x6c) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + MMIO_DH(GEN7_SC_INSTDONE); + + MMIO_GM_RDR(_MMIO(0x2148)); + MMIO_GM_RDR(CCID); + MMIO_GM_RDR(_MMIO(0x12198)); + MMIO_D(GEN7_CXT_SIZE); + + MMIO_RING_DFH(RING_TAIL); + MMIO_RING_DFH(RING_HEAD); + MMIO_RING_DFH(RING_CTL); + MMIO_RING_DFH(RING_ACTHD); + MMIO_RING_GM_RDR(RING_START); + + /* RING MODE */ +#define RING_REG(base) _MMIO((base) + 0x29c) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + + MMIO_RING_DFH(RING_MI_MODE); + MMIO_RING_DFH(RING_INSTPM); + MMIO_RING_DFH(RING_TIMESTAMP); + MMIO_RING_DFH(RING_TIMESTAMP_UDW); + MMIO_DFH(GEN7_GT_MODE); + MMIO_DFH(CACHE_MODE_0_GEN7); + MMIO_DFH(CACHE_MODE_1); + MMIO_DFH(CACHE_MODE_0); + MMIO_DFH(_MMIO(0x2124)); + + MMIO_DFH(_MMIO(0x20dc)); + MMIO_DFH(_3D_CHICKEN3); + MMIO_DFH(_MMIO(0x2088)); + MMIO_DFH(_MMIO(0x20e4)); + MMIO_DFH(_MMIO(0x2470)); + MMIO_DFH(GAM_ECOCHK); + MMIO_DFH(GEN7_COMMON_SLICE_CHICKEN1); + MMIO_DFH(COMMON_SLICE_CHICKEN2); + MMIO_DFH(_MMIO(0x9030)); + MMIO_DFH(_MMIO(0x20a0)); + MMIO_DFH(_MMIO(0x2420)); + MMIO_DFH(_MMIO(0x2430)); + MMIO_DFH(_MMIO(0x2434)); + MMIO_DFH(_MMIO(0x2438)); + MMIO_DFH(_MMIO(0x243c)); + MMIO_DFH(_MMIO(0x7018)); + MMIO_DFH(HALF_SLICE_CHICKEN3); + MMIO_DFH(GEN7_HALF_SLICE_CHICKEN1); + + /* display */ + MMIO_D(_MMIO(0x602a0)); + + MMIO_D(_MMIO(0x65050)); + MMIO_D(_MMIO(0x650b4)); + + MMIO_D(_MMIO(0xc4040)); + MMIO_D(DERRMR); + + MMIO_D(PIPEDSL(PIPE_A)); + MMIO_D(PIPEDSL(PIPE_B)); + MMIO_D(PIPEDSL(PIPE_C)); + MMIO_D(PIPEDSL(_PIPE_EDP)); + + MMIO_DH(PIPECONF(PIPE_A)); + MMIO_DH(PIPECONF(PIPE_B)); + MMIO_DH(PIPECONF(PIPE_C)); + MMIO_DH(PIPECONF(_PIPE_EDP)); + + MMIO_D(PIPESTAT(PIPE_A)); + MMIO_D(PIPESTAT(PIPE_B)); + MMIO_D(PIPESTAT(PIPE_C)); + MMIO_D(PIPESTAT(_PIPE_EDP)); + + MMIO_D(PIPE_FLIPCOUNT_G4X(PIPE_A)); + MMIO_D(PIPE_FLIPCOUNT_G4X(PIPE_B)); + MMIO_D(PIPE_FLIPCOUNT_G4X(PIPE_C)); + MMIO_D(PIPE_FLIPCOUNT_G4X(_PIPE_EDP)); + + MMIO_D(PIPE_FRMCOUNT_G4X(PIPE_A)); + MMIO_D(PIPE_FRMCOUNT_G4X(PIPE_B)); + MMIO_D(PIPE_FRMCOUNT_G4X(PIPE_C)); + MMIO_D(PIPE_FRMCOUNT_G4X(_PIPE_EDP)); + + MMIO_D(CURCNTR(PIPE_A)); + MMIO_D(CURCNTR(PIPE_B)); + MMIO_D(CURCNTR(PIPE_C)); + + MMIO_D(CURPOS(PIPE_A)); + MMIO_D(CURPOS(PIPE_B)); + MMIO_D(CURPOS(PIPE_C)); + + MMIO_D(CURBASE(PIPE_A)); + MMIO_D(CURBASE(PIPE_B)); + MMIO_D(CURBASE(PIPE_C)); + + MMIO_D(CUR_FBC_CTL(PIPE_A)); + MMIO_D(CUR_FBC_CTL(PIPE_B)); + MMIO_D(CUR_FBC_CTL(PIPE_C)); + + MMIO_D(_MMIO(0x700ac)); + MMIO_D(_MMIO(0x710ac)); + MMIO_D(_MMIO(0x720ac)); + + MMIO_D(_MMIO(0x70090)); + MMIO_D(_MMIO(0x70094)); + MMIO_D(_MMIO(0x70098)); + MMIO_D(_MMIO(0x7009c)); + + MMIO_D(DSPCNTR(PIPE_A)); + MMIO_D(DSPADDR(PIPE_A)); + MMIO_D(DSPSTRIDE(PIPE_A)); + MMIO_D(DSPPOS(PIPE_A)); + MMIO_D(DSPSIZE(PIPE_A)); + MMIO_DH(DSPSURF(PIPE_A)); + MMIO_D(DSPOFFSET(PIPE_A)); + MMIO_D(DSPSURFLIVE(PIPE_A)); + + MMIO_D(DSPCNTR(PIPE_B)); + MMIO_D(DSPADDR(PIPE_B)); + MMIO_D(DSPSTRIDE(PIPE_B)); + MMIO_D(DSPPOS(PIPE_B)); + MMIO_D(DSPSIZE(PIPE_B)); + MMIO_DH(DSPSURF(PIPE_B)); + MMIO_D(DSPOFFSET(PIPE_B)); + MMIO_D(DSPSURFLIVE(PIPE_B)); + + MMIO_D(DSPCNTR(PIPE_C)); + MMIO_D(DSPADDR(PIPE_C)); + MMIO_D(DSPSTRIDE(PIPE_C)); + MMIO_D(DSPPOS(PIPE_C)); + MMIO_D(DSPSIZE(PIPE_C)); + MMIO_DH(DSPSURF(PIPE_C)); + MMIO_D(DSPOFFSET(PIPE_C)); + MMIO_D(DSPSURFLIVE(PIPE_C)); + + MMIO_D(SPRCTL(PIPE_A)); + MMIO_D(SPRLINOFF(PIPE_A)); + MMIO_D(SPRSTRIDE(PIPE_A)); + MMIO_D(SPRPOS(PIPE_A)); + MMIO_D(SPRSIZE(PIPE_A)); + MMIO_D(SPRKEYVAL(PIPE_A)); + MMIO_D(SPRKEYMSK(PIPE_A)); + MMIO_DH(SPRSURF(PIPE_A)); + MMIO_D(SPRKEYMAX(PIPE_A)); + MMIO_D(SPROFFSET(PIPE_A)); + MMIO_D(SPRSCALE(PIPE_A)); + MMIO_D(SPRSURFLIVE(PIPE_A)); + + MMIO_D(SPRCTL(PIPE_B)); + MMIO_D(SPRLINOFF(PIPE_B)); + MMIO_D(SPRSTRIDE(PIPE_B)); + MMIO_D(SPRPOS(PIPE_B)); + MMIO_D(SPRSIZE(PIPE_B)); + MMIO_D(SPRKEYVAL(PIPE_B)); + MMIO_D(SPRKEYMSK(PIPE_B)); + MMIO_DH(SPRSURF(PIPE_B)); + MMIO_D(SPRKEYMAX(PIPE_B)); + MMIO_D(SPROFFSET(PIPE_B)); + MMIO_D(SPRSCALE(PIPE_B)); + MMIO_D(SPRSURFLIVE(PIPE_B)); + + MMIO_D(SPRCTL(PIPE_C)); + MMIO_D(SPRLINOFF(PIPE_C)); + MMIO_D(SPRSTRIDE(PIPE_C)); + MMIO_D(SPRPOS(PIPE_C)); + MMIO_D(SPRSIZE(PIPE_C)); + MMIO_D(SPRKEYVAL(PIPE_C)); + MMIO_D(SPRKEYMSK(PIPE_C)); + MMIO_DH(SPRSURF(PIPE_C)); + MMIO_D(SPRKEYMAX(PIPE_C)); + MMIO_D(SPROFFSET(PIPE_C)); + MMIO_D(SPRSCALE(PIPE_C)); + MMIO_D(SPRSURFLIVE(PIPE_C)); + + MMIO_D(HTOTAL(TRANSCODER_A)); + MMIO_D(HBLANK(TRANSCODER_A)); + MMIO_D(HSYNC(TRANSCODER_A)); + MMIO_D(VTOTAL(TRANSCODER_A)); + MMIO_D(VBLANK(TRANSCODER_A)); + MMIO_D(VSYNC(TRANSCODER_A)); + MMIO_D(BCLRPAT(TRANSCODER_A)); + MMIO_D(VSYNCSHIFT(TRANSCODER_A)); + MMIO_D(PIPESRC(TRANSCODER_A)); + + MMIO_D(HTOTAL(TRANSCODER_B)); + MMIO_D(HBLANK(TRANSCODER_B)); + MMIO_D(HSYNC(TRANSCODER_B)); + MMIO_D(VTOTAL(TRANSCODER_B)); + MMIO_D(VBLANK(TRANSCODER_B)); + MMIO_D(VSYNC(TRANSCODER_B)); + MMIO_D(BCLRPAT(TRANSCODER_B)); + MMIO_D(VSYNCSHIFT(TRANSCODER_B)); + MMIO_D(PIPESRC(TRANSCODER_B)); + + MMIO_D(HTOTAL(TRANSCODER_C)); + MMIO_D(HBLANK(TRANSCODER_C)); + MMIO_D(HSYNC(TRANSCODER_C)); + MMIO_D(VTOTAL(TRANSCODER_C)); + MMIO_D(VBLANK(TRANSCODER_C)); + MMIO_D(VSYNC(TRANSCODER_C)); + MMIO_D(BCLRPAT(TRANSCODER_C)); + MMIO_D(VSYNCSHIFT(TRANSCODER_C)); + MMIO_D(PIPESRC(TRANSCODER_C)); + + MMIO_D(HTOTAL(TRANSCODER_EDP)); + MMIO_D(HBLANK(TRANSCODER_EDP)); + MMIO_D(HSYNC(TRANSCODER_EDP)); + MMIO_D(VTOTAL(TRANSCODER_EDP)); + MMIO_D(VBLANK(TRANSCODER_EDP)); + MMIO_D(VSYNC(TRANSCODER_EDP)); + MMIO_D(BCLRPAT(TRANSCODER_EDP)); + MMIO_D(VSYNCSHIFT(TRANSCODER_EDP)); + + MMIO_D(PIPE_DATA_M1(TRANSCODER_A)); + MMIO_D(PIPE_DATA_N1(TRANSCODER_A)); + MMIO_D(PIPE_DATA_M2(TRANSCODER_A)); + MMIO_D(PIPE_DATA_N2(TRANSCODER_A)); + MMIO_D(PIPE_LINK_M1(TRANSCODER_A)); + MMIO_D(PIPE_LINK_N1(TRANSCODER_A)); + MMIO_D(PIPE_LINK_M2(TRANSCODER_A)); + MMIO_D(PIPE_LINK_N2(TRANSCODER_A)); + + MMIO_D(PIPE_DATA_M1(TRANSCODER_B)); + MMIO_D(PIPE_DATA_N1(TRANSCODER_B)); + MMIO_D(PIPE_DATA_M2(TRANSCODER_B)); + MMIO_D(PIPE_DATA_N2(TRANSCODER_B)); + MMIO_D(PIPE_LINK_M1(TRANSCODER_B)); + MMIO_D(PIPE_LINK_N1(TRANSCODER_B)); + MMIO_D(PIPE_LINK_M2(TRANSCODER_B)); + MMIO_D(PIPE_LINK_N2(TRANSCODER_B)); + + MMIO_D(PIPE_DATA_M1(TRANSCODER_C)); + MMIO_D(PIPE_DATA_N1(TRANSCODER_C)); + MMIO_D(PIPE_DATA_M2(TRANSCODER_C)); + MMIO_D(PIPE_DATA_N2(TRANSCODER_C)); + MMIO_D(PIPE_LINK_M1(TRANSCODER_C)); + MMIO_D(PIPE_LINK_N1(TRANSCODER_C)); + MMIO_D(PIPE_LINK_M2(TRANSCODER_C)); + MMIO_D(PIPE_LINK_N2(TRANSCODER_C)); + + MMIO_D(PIPE_DATA_M1(TRANSCODER_EDP)); + MMIO_D(PIPE_DATA_N1(TRANSCODER_EDP)); + MMIO_D(PIPE_DATA_M2(TRANSCODER_EDP)); + MMIO_D(PIPE_DATA_N2(TRANSCODER_EDP)); + MMIO_D(PIPE_LINK_M1(TRANSCODER_EDP)); + MMIO_D(PIPE_LINK_N1(TRANSCODER_EDP)); + MMIO_D(PIPE_LINK_M2(TRANSCODER_EDP)); + MMIO_D(PIPE_LINK_N2(TRANSCODER_EDP)); + + MMIO_D(PF_CTL(PIPE_A)); + MMIO_D(PF_WIN_SZ(PIPE_A)); + MMIO_D(PF_WIN_POS(PIPE_A)); + MMIO_D(PF_VSCALE(PIPE_A)); + MMIO_D(PF_HSCALE(PIPE_A)); + + MMIO_D(PF_CTL(PIPE_B)); + MMIO_D(PF_WIN_SZ(PIPE_B)); + MMIO_D(PF_WIN_POS(PIPE_B)); + MMIO_D(PF_VSCALE(PIPE_B)); + MMIO_D(PF_HSCALE(PIPE_B)); + + MMIO_D(PF_CTL(PIPE_C)); + MMIO_D(PF_WIN_SZ(PIPE_C)); + MMIO_D(PF_WIN_POS(PIPE_C)); + MMIO_D(PF_VSCALE(PIPE_C)); + MMIO_D(PF_HSCALE(PIPE_C)); + + MMIO_D(WM0_PIPEA_ILK); + MMIO_D(WM0_PIPEB_ILK); + MMIO_D(WM0_PIPEC_IVB); + MMIO_D(WM1_LP_ILK); + MMIO_D(WM2_LP_ILK); + MMIO_D(WM3_LP_ILK); + MMIO_D(WM1S_LP_ILK); + MMIO_D(WM2S_LP_IVB); + MMIO_D(WM3S_LP_IVB); + + MMIO_D(BLC_PWM_CPU_CTL2); + MMIO_D(BLC_PWM_CPU_CTL); + MMIO_D(BLC_PWM_PCH_CTL1); + MMIO_D(BLC_PWM_PCH_CTL2); + + MMIO_D(_MMIO(0x48268)); + + MMIO_DH(PCH_ADPA); + + MMIO_DH(_MMIO(_PCH_TRANSACONF)); + MMIO_DH(_MMIO(_PCH_TRANSBCONF)); + + MMIO_DH(FDI_RX_IIR(PIPE_A)); + MMIO_DH(FDI_RX_IIR(PIPE_B)); + MMIO_DH(FDI_RX_IIR(PIPE_C)); + MMIO_DH(FDI_RX_IMR(PIPE_A)); + MMIO_DH(FDI_RX_IMR(PIPE_B)); + MMIO_DH(FDI_RX_IMR(PIPE_C)); + MMIO_DH(FDI_RX_CTL(PIPE_A)); + MMIO_DH(FDI_RX_CTL(PIPE_B)); + MMIO_DH(FDI_RX_CTL(PIPE_C)); + + MMIO_D(_MMIO(_PCH_TRANS_HTOTAL_A)); + MMIO_D(_MMIO(_PCH_TRANS_HBLANK_A)); + MMIO_D(_MMIO(_PCH_TRANS_HSYNC_A)); + MMIO_D(_MMIO(_PCH_TRANS_VTOTAL_A)); + MMIO_D(_MMIO(_PCH_TRANS_VBLANK_A)); + MMIO_D(_MMIO(_PCH_TRANS_VSYNC_A)); + MMIO_D(_MMIO(_PCH_TRANS_VSYNCSHIFT_A)); + + MMIO_D(_MMIO(_PCH_TRANS_HTOTAL_B)); + MMIO_D(_MMIO(_PCH_TRANS_HBLANK_B)); + MMIO_D(_MMIO(_PCH_TRANS_HSYNC_B)); + MMIO_D(_MMIO(_PCH_TRANS_VTOTAL_B)); + MMIO_D(_MMIO(_PCH_TRANS_VBLANK_B)); + MMIO_D(_MMIO(_PCH_TRANS_VSYNC_B)); + MMIO_D(_MMIO(_PCH_TRANS_VSYNCSHIFT_B)); + + MMIO_D(_MMIO(_PCH_TRANSA_DATA_M1)); + MMIO_D(_MMIO(_PCH_TRANSA_DATA_N1)); + MMIO_D(_MMIO(_PCH_TRANSA_DATA_M2)); + MMIO_D(_MMIO(_PCH_TRANSA_DATA_N2)); + MMIO_D(_MMIO(_PCH_TRANSA_LINK_M1)); + MMIO_D(_MMIO(_PCH_TRANSA_LINK_N1)); + MMIO_D(_MMIO(_PCH_TRANSA_LINK_M2)); + MMIO_D(_MMIO(_PCH_TRANSA_LINK_N2)); + + MMIO_D(TRANS_DP_CTL(PIPE_A)); + MMIO_D(TRANS_DP_CTL(PIPE_B)); + MMIO_D(TRANS_DP_CTL(PIPE_C)); + + MMIO_D(TVIDEO_DIP_CTL(PIPE_A)); + MMIO_D(TVIDEO_DIP_DATA(PIPE_A)); + MMIO_D(TVIDEO_DIP_GCP(PIPE_A)); + + MMIO_D(TVIDEO_DIP_CTL(PIPE_B)); + MMIO_D(TVIDEO_DIP_DATA(PIPE_B)); + MMIO_D(TVIDEO_DIP_GCP(PIPE_B)); + + MMIO_D(TVIDEO_DIP_CTL(PIPE_C)); + MMIO_D(TVIDEO_DIP_DATA(PIPE_C)); + MMIO_D(TVIDEO_DIP_GCP(PIPE_C)); + + MMIO_D(_MMIO(_FDI_RXA_MISC)); + MMIO_D(_MMIO(_FDI_RXB_MISC)); + MMIO_D(_MMIO(_FDI_RXA_TUSIZE1)); + MMIO_D(_MMIO(_FDI_RXA_TUSIZE2)); + MMIO_D(_MMIO(_FDI_RXB_TUSIZE1)); + MMIO_D(_MMIO(_FDI_RXB_TUSIZE2)); + + MMIO_DH(PCH_PP_CONTROL); + MMIO_D(PCH_PP_DIVISOR); + MMIO_D(PCH_PP_STATUS); + MMIO_D(PCH_LVDS); + MMIO_D(_MMIO(_PCH_DPLL_A)); + MMIO_D(_MMIO(_PCH_DPLL_B)); + MMIO_D(_MMIO(_PCH_FPA0)); + MMIO_D(_MMIO(_PCH_FPA1)); + MMIO_D(_MMIO(_PCH_FPB0)); + MMIO_D(_MMIO(_PCH_FPB1)); + MMIO_D(PCH_DREF_CONTROL); + MMIO_D(PCH_RAWCLK_FREQ); + MMIO_D(PCH_DPLL_SEL); + + MMIO_D(_MMIO(0x61208)); + MMIO_D(_MMIO(0x6120c)); + MMIO_D(PCH_PP_ON_DELAYS); + MMIO_D(PCH_PP_OFF_DELAYS); + + MMIO_DH(_MMIO(0xe651c)); + MMIO_DH(_MMIO(0xe661c)); + MMIO_DH(_MMIO(0xe671c)); + MMIO_DH(_MMIO(0xe681c)); + MMIO_DH(_MMIO(0xe6c04)); + MMIO_DH(_MMIO(0xe6e1c)); + + MMIO_RO(PCH_PORT_HOTPLUG); + MMIO_DH(LCPLL_CTL); + MMIO_D(FUSE_STRAP); + MMIO_D(DIGITAL_PORT_HOTPLUG_CNTRL); + + MMIO_D(DISP_ARB_CTL); + MMIO_D(DISP_ARB_CTL2); + + MMIO_D(ILK_DISPLAY_CHICKEN1); + MMIO_D(ILK_DISPLAY_CHICKEN2); + MMIO_D(ILK_DSPCLK_GATE_D); + + MMIO_D(SOUTH_CHICKEN1); + MMIO_DH(SOUTH_CHICKEN2); + MMIO_D(_MMIO(_TRANSA_CHICKEN1)); + MMIO_D(_MMIO(_TRANSB_CHICKEN1)); + MMIO_D(SOUTH_DSPCLK_GATE_D); + MMIO_D(_MMIO(_TRANSA_CHICKEN2)); + MMIO_D(_MMIO(_TRANSB_CHICKEN2)); + + MMIO_D(ILK_DPFC_CB_BASE); + MMIO_D(ILK_DPFC_CONTROL); + MMIO_D(ILK_DPFC_RECOMP_CTL); + MMIO_D(ILK_DPFC_STATUS); + MMIO_D(ILK_DPFC_FENCE_YOFF); + MMIO_D(ILK_DPFC_CHICKEN); + MMIO_D(ILK_FBC_RT_BASE); + + MMIO_D(IPS_CTL); + + MMIO_D(PIPE_CSC_COEFF_RY_GY(PIPE_A)); + MMIO_D(PIPE_CSC_COEFF_BY(PIPE_A)); + MMIO_D(PIPE_CSC_COEFF_RU_GU(PIPE_A)); + MMIO_D(PIPE_CSC_COEFF_BU(PIPE_A)); + MMIO_D(PIPE_CSC_COEFF_RV_GV(PIPE_A)); + MMIO_D(PIPE_CSC_COEFF_BV(PIPE_A)); + MMIO_D(PIPE_CSC_MODE(PIPE_A)); + MMIO_D(PIPE_CSC_PREOFF_HI(PIPE_A)); + MMIO_D(PIPE_CSC_PREOFF_ME(PIPE_A)); + MMIO_D(PIPE_CSC_PREOFF_LO(PIPE_A)); + MMIO_D(PIPE_CSC_POSTOFF_HI(PIPE_A)); + MMIO_D(PIPE_CSC_POSTOFF_ME(PIPE_A)); + MMIO_D(PIPE_CSC_POSTOFF_LO(PIPE_A)); + + MMIO_D(PIPE_CSC_COEFF_RY_GY(PIPE_B)); + MMIO_D(PIPE_CSC_COEFF_BY(PIPE_B)); + MMIO_D(PIPE_CSC_COEFF_RU_GU(PIPE_B)); + MMIO_D(PIPE_CSC_COEFF_BU(PIPE_B)); + MMIO_D(PIPE_CSC_COEFF_RV_GV(PIPE_B)); + MMIO_D(PIPE_CSC_COEFF_BV(PIPE_B)); + MMIO_D(PIPE_CSC_MODE(PIPE_B)); + MMIO_D(PIPE_CSC_PREOFF_HI(PIPE_B)); + MMIO_D(PIPE_CSC_PREOFF_ME(PIPE_B)); + MMIO_D(PIPE_CSC_PREOFF_LO(PIPE_B)); + MMIO_D(PIPE_CSC_POSTOFF_HI(PIPE_B)); + MMIO_D(PIPE_CSC_POSTOFF_ME(PIPE_B)); + MMIO_D(PIPE_CSC_POSTOFF_LO(PIPE_B)); + + MMIO_D(PIPE_CSC_COEFF_RY_GY(PIPE_C)); + MMIO_D(PIPE_CSC_COEFF_BY(PIPE_C)); + MMIO_D(PIPE_CSC_COEFF_RU_GU(PIPE_C)); + MMIO_D(PIPE_CSC_COEFF_BU(PIPE_C)); + MMIO_D(PIPE_CSC_COEFF_RV_GV(PIPE_C)); + MMIO_D(PIPE_CSC_COEFF_BV(PIPE_C)); + MMIO_D(PIPE_CSC_MODE(PIPE_C)); + MMIO_D(PIPE_CSC_PREOFF_HI(PIPE_C)); + MMIO_D(PIPE_CSC_PREOFF_ME(PIPE_C)); + MMIO_D(PIPE_CSC_PREOFF_LO(PIPE_C)); + MMIO_D(PIPE_CSC_POSTOFF_HI(PIPE_C)); + MMIO_D(PIPE_CSC_POSTOFF_ME(PIPE_C)); + MMIO_D(PIPE_CSC_POSTOFF_LO(PIPE_C)); + + MMIO_D(PREC_PAL_INDEX(PIPE_A)); + MMIO_D(PREC_PAL_DATA(PIPE_A)); + + MMIO_D(PREC_PAL_INDEX(PIPE_B)); + MMIO_D(PREC_PAL_DATA(PIPE_B)); + + MMIO_D(PREC_PAL_INDEX(PIPE_C)); + MMIO_D(PREC_PAL_DATA(PIPE_C)); + + MMIO_D(_MMIO(0x60110)); + MMIO_D(_MMIO(0x61110)); + + MMIO_D(PIPE_WM_LINETIME(PIPE_A)); + MMIO_D(PIPE_WM_LINETIME(PIPE_B)); + MMIO_D(PIPE_WM_LINETIME(PIPE_C)); + MMIO_D(SPLL_CTL); + MMIO_D(_MMIO(_WRPLL_CTL1)); + MMIO_D(_MMIO(_WRPLL_CTL2)); + MMIO_D(PORT_CLK_SEL(PORT_A)); + MMIO_D(PORT_CLK_SEL(PORT_B)); + MMIO_D(PORT_CLK_SEL(PORT_C)); + MMIO_D(PORT_CLK_SEL(PORT_D)); + MMIO_D(PORT_CLK_SEL(PORT_E)); + MMIO_D(TRANS_CLK_SEL(TRANSCODER_A)); + MMIO_D(TRANS_CLK_SEL(TRANSCODER_B)); + MMIO_D(TRANS_CLK_SEL(TRANSCODER_C)); + + MMIO_D(HSW_NDE_RSTWRN_OPT); + MMIO_D(_MMIO(0x46508)); + + MMIO_D(_MMIO(0x49080)); + MMIO_D(_MMIO(0x49180)); + MMIO_D(_MMIO(0x49280)); + + MMIO_D(GAMMA_MODE(PIPE_A)); + MMIO_D(GAMMA_MODE(PIPE_B)); + MMIO_D(GAMMA_MODE(PIPE_C)); + + MMIO_D(PIPE_MULT(PIPE_A)); + MMIO_D(PIPE_MULT(PIPE_B)); + MMIO_D(PIPE_MULT(PIPE_C)); + + MMIO_D(HSW_TVIDEO_DIP_CTL(TRANSCODER_A)); + MMIO_D(HSW_TVIDEO_DIP_CTL(TRANSCODER_B)); + MMIO_D(HSW_TVIDEO_DIP_CTL(TRANSCODER_C)); + + MMIO_DH(SFUSE_STRAP); + MMIO_D(SBI_ADDR); + MMIO_DH(SBI_DATA); + MMIO_DH(SBI_CTL_STAT); + MMIO_D(PIXCLK_GATE); + + MMIO_DH(DDI_BUF_CTL(PORT_A)); + MMIO_DH(DDI_BUF_CTL(PORT_B)); + MMIO_DH(DDI_BUF_CTL(PORT_C)); + MMIO_DH(DDI_BUF_CTL(PORT_D)); + MMIO_DH(DDI_BUF_CTL(PORT_E)); + + MMIO_DH(DP_TP_CTL(PORT_A)); + MMIO_DH(DP_TP_CTL(PORT_B)); + MMIO_DH(DP_TP_CTL(PORT_C)); + MMIO_DH(DP_TP_CTL(PORT_D)); + MMIO_DH(DP_TP_CTL(PORT_E)); + + MMIO_DH(DP_TP_STATUS(PORT_A)); + MMIO_DH(DP_TP_STATUS(PORT_B)); + MMIO_DH(DP_TP_STATUS(PORT_C)); + MMIO_DH(DP_TP_STATUS(PORT_D)); + MMIO_DH(DP_TP_STATUS(PORT_E)); + + MMIO_D(HSW_AUD_CFG(PIPE_A)); + MMIO_D(HSW_AUD_PIN_ELD_CP_VLD); + MMIO_D(HSW_AUD_MISC_CTRL(PIPE_A)); + + MMIO_DH(_MMIO(_TRANS_DDI_FUNC_CTL_A)); + MMIO_DH(_MMIO(_TRANS_DDI_FUNC_CTL_B)); + MMIO_DH(_MMIO(_TRANS_DDI_FUNC_CTL_C)); + MMIO_DH(_MMIO(_TRANS_DDI_FUNC_CTL_EDP)); + + MMIO_D(_MMIO(_TRANSA_MSA_MISC)); + MMIO_D(_MMIO(_TRANSB_MSA_MISC)); + MMIO_D(_MMIO(_TRANSC_MSA_MISC)); + MMIO_D(_MMIO(_TRANS_EDP_MSA_MISC)); + + MMIO_DH(FORCEWAKE); + MMIO_D(FORCEWAKE_ACK); + MMIO_D(GEN6_GT_CORE_STATUS); + MMIO_D(GEN6_GT_THREAD_STATUS_REG); + MMIO_DFH(GTFIFODBG); + MMIO_DFH(GTFIFOCTL); + MMIO_DH(FORCEWAKE_MT); + MMIO_DH(FORCEWAKE_ACK_HSW); + MMIO_D(ECOBUS); + MMIO_DH(GEN6_RC_CONTROL); + MMIO_DH(GEN6_RC_STATE); + MMIO_D(GEN6_RPNSWREQ); + MMIO_D(GEN6_RC_VIDEO_FREQ); + MMIO_D(GEN6_RP_DOWN_TIMEOUT); + MMIO_D(GEN6_RP_INTERRUPT_LIMITS); + MMIO_D(GEN6_RPSTAT1); + MMIO_D(GEN6_RP_CONTROL); + MMIO_D(GEN6_RP_UP_THRESHOLD); + MMIO_D(GEN6_RP_DOWN_THRESHOLD); + MMIO_D(GEN6_RP_CUR_UP_EI); + MMIO_D(GEN6_RP_CUR_UP); + MMIO_D(GEN6_RP_PREV_UP); + MMIO_D(GEN6_RP_CUR_DOWN_EI); + MMIO_D(GEN6_RP_CUR_DOWN); + MMIO_D(GEN6_RP_PREV_DOWN); + MMIO_D(GEN6_RP_UP_EI); + MMIO_D(GEN6_RP_DOWN_EI); + MMIO_D(GEN6_RP_IDLE_HYSTERSIS); + MMIO_D(GEN6_RC1_WAKE_RATE_LIMIT); + MMIO_D(GEN6_RC6_WAKE_RATE_LIMIT); + MMIO_D(GEN6_RC6pp_WAKE_RATE_LIMIT); + MMIO_D(GEN6_RC_EVALUATION_INTERVAL); + MMIO_D(GEN6_RC_IDLE_HYSTERSIS); + MMIO_D(GEN6_RC_SLEEP); + MMIO_D(GEN6_RC1e_THRESHOLD); + MMIO_D(GEN6_RC6_THRESHOLD); + MMIO_D(GEN6_RC6p_THRESHOLD); + MMIO_D(GEN6_RC6pp_THRESHOLD); + MMIO_D(GEN6_PMINTRMSK); + MMIO_DH(HSW_PWR_WELL_CTL1); + MMIO_DH(HSW_PWR_WELL_CTL2); + MMIO_DH(HSW_PWR_WELL_CTL3); + MMIO_DH(HSW_PWR_WELL_CTL4); + MMIO_DH(HSW_PWR_WELL_CTL5); + MMIO_DH(HSW_PWR_WELL_CTL6); + + MMIO_D(RSTDBYCTL); + + MMIO_DH(GEN6_GDRST); + MMIO_DH(CPU_VGACNTRL); + + MMIO_D(TILECTL); + + MMIO_D(GEN6_UCGCTL1); + MMIO_D(GEN6_UCGCTL2); + + MMIO_D(GEN6_PCODE_DATA); + MMIO_D(_MMIO(0x13812c)); + MMIO_DH(GEN7_ERR_INT); + MMIO_D(HSW_EDRAM_CAP); + MMIO_D(HSW_IDICR); + MMIO_DH(GFX_FLSH_CNTL_GEN6); + + MMIO_D(_MMIO(0x3c)); + MMIO_D(_MMIO(0x860)); + MMIO_D(ECOSKPD); + MMIO_D(_MMIO(0x121d0)); + MMIO_D(GEN6_BLITTER_ECOSKPD); + MMIO_D(_MMIO(0x41d0)); + MMIO_D(GAC_ECO_BITS); + MMIO_D(_MMIO(0x6200)); + MMIO_D(_MMIO(0x6204)); + MMIO_D(_MMIO(0x6208)); + MMIO_D(_MMIO(0x7118)); + MMIO_D(_MMIO(0x7180)); + MMIO_D(_MMIO(0x7408)); + MMIO_D(_MMIO(0x7c00)); + MMIO_DH(GEN6_MBCTL); + MMIO_D(_MMIO(0x911c)); + MMIO_D(_MMIO(0x9120)); + MMIO_DFH(GEN7_UCGCTL4); + + MMIO_D(GAB_CTL); + MMIO_D(_MMIO(0x48800)); + MMIO_D(_MMIO(0xce044)); + MMIO_D(_MMIO(0xe6500)); + MMIO_D(_MMIO(0xe6504)); + MMIO_D(_MMIO(0xe6600)); + MMIO_D(_MMIO(0xe6604)); + MMIO_D(_MMIO(0xe6700)); + MMIO_D(_MMIO(0xe6704)); + MMIO_D(_MMIO(0xe6800)); + MMIO_D(_MMIO(0xe6804)); + MMIO_D(PCH_GMBUS4); + MMIO_D(PCH_GMBUS5); + + MMIO_D(_MMIO(0x902c)); + MMIO_D(_MMIO(0xec008)); + MMIO_D(_MMIO(0xec00c)); + MMIO_D(_MMIO(0xec008 + 0x18)); + MMIO_D(_MMIO(0xec00c + 0x18)); + MMIO_D(_MMIO(0xec008 + 0x18 * 2)); + MMIO_D(_MMIO(0xec00c + 0x18 * 2)); + MMIO_D(_MMIO(0xec008 + 0x18 * 3)); + MMIO_D(_MMIO(0xec00c + 0x18 * 3)); + MMIO_D(_MMIO(0xec408)); + MMIO_D(_MMIO(0xec40c)); + MMIO_D(_MMIO(0xec408 + 0x18)); + MMIO_D(_MMIO(0xec40c + 0x18)); + MMIO_D(_MMIO(0xec408 + 0x18 * 2)); + MMIO_D(_MMIO(0xec40c + 0x18 * 2)); + MMIO_D(_MMIO(0xec408 + 0x18 * 3)); + MMIO_D(_MMIO(0xec40c + 0x18 * 3)); + MMIO_D(_MMIO(0xfc810)); + MMIO_D(_MMIO(0xfc81c)); + MMIO_D(_MMIO(0xfc828)); + MMIO_D(_MMIO(0xfc834)); + MMIO_D(_MMIO(0xfcc00)); + MMIO_D(_MMIO(0xfcc0c)); + MMIO_D(_MMIO(0xfcc18)); + MMIO_D(_MMIO(0xfcc24)); + MMIO_D(_MMIO(0xfd000)); + MMIO_D(_MMIO(0xfd00c)); + MMIO_D(_MMIO(0xfd018)); + MMIO_D(_MMIO(0xfd024)); + MMIO_D(_MMIO(0xfd034)); + + MMIO_DH(FPGA_DBG); + MMIO_D(_MMIO(0x2054)); + MMIO_D(_MMIO(0x12054)); + MMIO_D(_MMIO(0x22054)); + MMIO_D(_MMIO(0x1a054)); + + MMIO_D(_MMIO(0x44070)); + MMIO_DFH(_MMIO(0x215c)); + MMIO_DFH(_MMIO(0x2178)); + MMIO_DFH(_MMIO(0x217c)); + MMIO_DFH(_MMIO(0x12178)); + MMIO_DFH(_MMIO(0x1217c)); + + MMIO_D(_MMIO(0x2b00)); + MMIO_D(_MMIO(0x2360)); + + MMIO_DFH(_MMIO(0x1c17c)); + MMIO_DFH(_MMIO(0x1c178)); + MMIO_DFH(BCS_SWCTRL); + + MMIO_DH(_MMIO(0x4260)); + MMIO_DH(_MMIO(0x4264)); + MMIO_DH(_MMIO(0x4268)); + MMIO_DH(_MMIO(0x426c)); + MMIO_DH(_MMIO(0x4270)); + MMIO_DFH(_MMIO(0x4094)); + + MMIO_DFH(ARB_MODE); + MMIO_RING_GM_RDR(RING_BBADDR); + MMIO_DFH(_MMIO(0x2220)); + MMIO_DFH(_MMIO(0x12220)); + MMIO_DFH(_MMIO(0x22220)); + MMIO_RING_DFH(RING_SYNC_1); + MMIO_RING_DFH(RING_SYNC_0); + MMIO_DFH(_MMIO(0x22178)); + MMIO_DFH(_MMIO(0x1a178)); + MMIO_DFH(_MMIO(0x1a17c)); + MMIO_DFH(_MMIO(0x2217c)); + + MMIO_DH(EDP_PSR_IMR); + MMIO_DH(EDP_PSR_IIR); +} + +static void init_bdw_mmio(struct drm_i915_private *dev_priv) +{ + MMIO_DH(GEN8_GT_IMR(0)); + MMIO_DH(GEN8_GT_IER(0)); + MMIO_DH(GEN8_GT_IIR(0)); + MMIO_D(GEN8_GT_ISR(0)); + + MMIO_DH(GEN8_GT_IMR(1)); + MMIO_DH(GEN8_GT_IER(1)); + MMIO_DH(GEN8_GT_IIR(1)); + MMIO_D(GEN8_GT_ISR(1)); + + MMIO_DH(GEN8_GT_IMR(2)); + MMIO_DH(GEN8_GT_IER(2)); + MMIO_DH(GEN8_GT_IIR(2)); + MMIO_D(GEN8_GT_ISR(2)); + + MMIO_DH(GEN8_GT_IMR(3)); + MMIO_DH(GEN8_GT_IER(3)); + MMIO_DH(GEN8_GT_IIR(3)); + MMIO_D(GEN8_GT_ISR(3)); + + MMIO_DH(GEN8_DE_PIPE_IMR(PIPE_A)); + MMIO_DH(GEN8_DE_PIPE_IER(PIPE_A)); + MMIO_DH(GEN8_DE_PIPE_IIR(PIPE_A)); + MMIO_D(GEN8_DE_PIPE_ISR(PIPE_A)); + + MMIO_DH(GEN8_DE_PIPE_IMR(PIPE_B)); + MMIO_DH(GEN8_DE_PIPE_IER(PIPE_B)); + MMIO_DH(GEN8_DE_PIPE_IIR(PIPE_B)); + MMIO_D(GEN8_DE_PIPE_ISR(PIPE_B)); + + MMIO_DH(GEN8_DE_PIPE_IMR(PIPE_C)); + MMIO_DH(GEN8_DE_PIPE_IER(PIPE_C)); + MMIO_DH(GEN8_DE_PIPE_IIR(PIPE_C)); + MMIO_D(GEN8_DE_PIPE_ISR(PIPE_C)); + + MMIO_DH(GEN8_DE_PORT_IMR); + MMIO_DH(GEN8_DE_PORT_IER); + MMIO_DH(GEN8_DE_PORT_IIR); + MMIO_D(GEN8_DE_PORT_ISR); + + MMIO_DH(GEN8_DE_MISC_IMR); + MMIO_DH(GEN8_DE_MISC_IER); + MMIO_DH(GEN8_DE_MISC_IIR); + MMIO_D(GEN8_DE_MISC_ISR); + + MMIO_DH(GEN8_PCU_IMR); + MMIO_DH(GEN8_PCU_IER); + MMIO_DH(GEN8_PCU_IIR); + MMIO_D(GEN8_PCU_ISR); + MMIO_DH(GEN8_MASTER_IRQ); + + MMIO_RING_DFH(RING_ACTHD_UDW); + +#define RING_REG(base) _MMIO((base) + 0xd0) + MMIO_RING_F(RING_REG); +#undef RING_REG + +#define RING_REG(base) _MMIO((base) + 0x230) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + +#define RING_REG(base) _MMIO((base) + 0x244) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + +#define RING_REG(base) _MMIO((base) + 0x3a0) + MMIO_RING_DFH(RING_REG); +#undef RING_REG + + MMIO_D(PIPEMISC(PIPE_A)); + MMIO_D(PIPEMISC(PIPE_B)); + MMIO_D(PIPEMISC(PIPE_C)); + MMIO_D(_MMIO(0x1c1d0)); + MMIO_D(GEN6_MBCUNIT_SNPCR); + MMIO_D(GEN7_MISCCPCTL); + MMIO_D(_MMIO(0x1c054)); + + MMIO_DH(GEN6_PCODE_MAILBOX); + + MMIO_D(GEN8_PRIVATE_PAT_LO); + MMIO_D(GEN8_PRIVATE_PAT_HI); + + MMIO_D(GAMTARBMODE); + + MMIO_RING_GM_RDR(RING_HWS_PGA); + + MMIO_DFH(HDC_CHICKEN0); + + MMIO_D(CHICKEN_PIPESL_1(PIPE_A)); + MMIO_D(CHICKEN_PIPESL_1(PIPE_B)); + MMIO_D(CHICKEN_PIPESL_1(PIPE_C)); + + MMIO_D(WM_MISC); + MMIO_D(_MMIO(BDW_EDP_PSR_BASE)); + + MMIO_D(_MMIO(0x6671c)); + MMIO_D(_MMIO(0x66c00)); + MMIO_D(_MMIO(0x66c04)); + + MMIO_D(HSW_GTT_CACHE_EN); + + MMIO_D(GEN8_EU_DISABLE0); + MMIO_D(GEN8_EU_DISABLE1); + MMIO_D(GEN8_EU_DISABLE2); + + MMIO_D(_MMIO(0xfdc)); + MMIO_DFH(GEN8_ROW_CHICKEN); + MMIO_DFH(GEN7_ROW_CHICKEN2); + MMIO_DFH(GEN8_UCGCTL6); + + MMIO_DFH(_MMIO(0xb1f0)); + MMIO_DFH(_MMIO(0xb1c0)); + MMIO_DFH(GEN8_L3SQCREG4); + MMIO_DFH(_MMIO(0xb100)); + MMIO_DFH(_MMIO(0xb10c)); + MMIO_D(_MMIO(0xb110)); + MMIO_D(_MMIO(0x44484)); + MMIO_D(_MMIO(0x4448c)); + + MMIO_DFH(_MMIO(0x83a4)); + MMIO_D(GEN8_L3_LRA_1_GPGPU); + + MMIO_DFH(_MMIO(0x8430)); + + MMIO_D(_MMIO(0x110000)); + + MMIO_D(_MMIO(0x48400)); + MMIO_D(_MMIO(0x6e570)); + MMIO_D(_MMIO(0x65f10)); + + MMIO_DFH(_MMIO(0xe194)); + MMIO_DFH(_MMIO(0xe188)); + MMIO_DFH(HALF_SLICE_CHICKEN2); + MMIO_DFH(_MMIO(0x2580)); + + MMIO_DFH(_MMIO(0x2248)); + + MMIO_DFH(_MMIO(0xe220)); + MMIO_DFH(_MMIO(0xe230)); + MMIO_DFH(_MMIO(0xe240)); + MMIO_DFH(_MMIO(0xe260)); + MMIO_DFH(_MMIO(0xe270)); + MMIO_DFH(_MMIO(0xe280)); + MMIO_DFH(_MMIO(0xe2a0)); + MMIO_DFH(_MMIO(0xe2b0)); + MMIO_DFH(_MMIO(0xe2c0)); +} + +static void init_skl_mmio(struct drm_i915_private *dev_priv) +{ + MMIO_DH(FORCEWAKE_RENDER_GEN9); + MMIO_DH(FORCEWAKE_ACK_RENDER_GEN9); + MMIO_DH(FORCEWAKE_BLITTER_GEN9); + MMIO_DH(FORCEWAKE_ACK_BLITTER_GEN9); + MMIO_DH(FORCEWAKE_MEDIA_GEN9); + MMIO_DH(FORCEWAKE_ACK_MEDIA_GEN9); + + MMIO_D(HSW_PWR_WELL_CTL1); + MMIO_DH(HSW_PWR_WELL_CTL2); + + MMIO_DH(DBUF_CTL); + + MMIO_D(_MMIO(0xa210)); + MMIO_D(GEN9_MEDIA_PG_IDLE_HYSTERESIS); + MMIO_D(GEN9_RENDER_PG_IDLE_HYSTERESIS); + MMIO_DFH(GEN9_GAMT_ECO_REG_RW_IA); + MMIO_DH(_MMIO(0x4ddc)); + MMIO_DH(_MMIO(0x42080)); + MMIO_D(_MMIO(0x45504)); + MMIO_D(_MMIO(0x45520)); + MMIO_D(_MMIO(0x46000)); + MMIO_DH(_MMIO(0x46010)); + MMIO_DH(_MMIO(0x46014)); + MMIO_D(_MMIO(0x6C040)); + MMIO_D(_MMIO(0x6C048)); + MMIO_D(_MMIO(0x6C050)); + MMIO_D(_MMIO(0x6C044)); + MMIO_D(_MMIO(0x6C04C)); + MMIO_D(_MMIO(0x6C054)); + MMIO_D(_MMIO(0x6c058)); + MMIO_D(_MMIO(0x6c05c)); + MMIO_DH(_MMIO(0x6c060)); + + MMIO_DH(SKL_PS_WIN_POS(PIPE_A, 0)); + MMIO_DH(SKL_PS_WIN_POS(PIPE_A, 1)); + MMIO_DH(SKL_PS_WIN_POS(PIPE_B, 0)); + MMIO_DH(SKL_PS_WIN_POS(PIPE_B, 1)); + MMIO_DH(SKL_PS_WIN_POS(PIPE_C, 0)); + MMIO_DH(SKL_PS_WIN_POS(PIPE_C, 1)); + + MMIO_DH(SKL_PS_WIN_SZ(PIPE_A, 0)); + MMIO_DH(SKL_PS_WIN_SZ(PIPE_A, 1)); + MMIO_DH(SKL_PS_WIN_SZ(PIPE_B, 0)); + MMIO_DH(SKL_PS_WIN_SZ(PIPE_B, 1)); + MMIO_DH(SKL_PS_WIN_SZ(PIPE_C, 0)); + MMIO_DH(SKL_PS_WIN_SZ(PIPE_C, 1)); + + MMIO_DH(SKL_PS_CTRL(PIPE_A, 0)); + MMIO_DH(SKL_PS_CTRL(PIPE_A, 1)); + MMIO_DH(SKL_PS_CTRL(PIPE_B, 0)); + MMIO_DH(SKL_PS_CTRL(PIPE_B, 1)); + MMIO_DH(SKL_PS_CTRL(PIPE_C, 0)); + MMIO_DH(SKL_PS_CTRL(PIPE_C, 1)); + + MMIO_DH(PLANE_BUF_CFG(PIPE_A, 0)); + MMIO_DH(PLANE_BUF_CFG(PIPE_A, 1)); + MMIO_DH(PLANE_BUF_CFG(PIPE_A, 2)); + MMIO_DH(PLANE_BUF_CFG(PIPE_A, 3)); + + MMIO_DH(PLANE_BUF_CFG(PIPE_B, 0)); + MMIO_DH(PLANE_BUF_CFG(PIPE_B, 1)); + MMIO_DH(PLANE_BUF_CFG(PIPE_B, 2)); + MMIO_DH(PLANE_BUF_CFG(PIPE_B, 3)); + + MMIO_DH(PLANE_BUF_CFG(PIPE_C, 0)); + MMIO_DH(PLANE_BUF_CFG(PIPE_C, 1)); + MMIO_DH(PLANE_BUF_CFG(PIPE_C, 2)); + MMIO_DH(PLANE_BUF_CFG(PIPE_C, 3)); + + MMIO_DH(CUR_BUF_CFG(PIPE_A)); + MMIO_DH(CUR_BUF_CFG(PIPE_B)); + MMIO_DH(CUR_BUF_CFG(PIPE_C)); + + MMIO_DH(PLANE_WM_TRANS(PIPE_A, 0)); + MMIO_DH(PLANE_WM_TRANS(PIPE_A, 1)); + MMIO_DH(PLANE_WM_TRANS(PIPE_A, 2)); + + MMIO_DH(PLANE_WM_TRANS(PIPE_B, 0)); + MMIO_DH(PLANE_WM_TRANS(PIPE_B, 1)); + MMIO_DH(PLANE_WM_TRANS(PIPE_B, 2)); + + MMIO_DH(PLANE_WM_TRANS(PIPE_C, 0)); + MMIO_DH(PLANE_WM_TRANS(PIPE_C, 1)); + MMIO_DH(PLANE_WM_TRANS(PIPE_C, 2)); + + MMIO_DH(CUR_WM_TRANS(PIPE_A)); + MMIO_DH(CUR_WM_TRANS(PIPE_B)); + MMIO_DH(CUR_WM_TRANS(PIPE_C)); + + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_A, 0)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_A, 1)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_A, 2)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_A, 3)); + + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_B, 0)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_B, 1)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_B, 2)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_B, 3)); + + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_C, 0)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_C, 1)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_C, 2)); + MMIO_DH(PLANE_NV12_BUF_CFG(PIPE_C, 3)); + + MMIO_DH(_MMIO(_REG_701C0(PIPE_A, 1))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_A, 2))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_A, 3))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_A, 4))); + + MMIO_DH(_MMIO(_REG_701C0(PIPE_B, 1))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_B, 2))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_B, 3))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_B, 4))); + + MMIO_DH(_MMIO(_REG_701C0(PIPE_C, 1))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_C, 2))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_C, 3))); + MMIO_DH(_MMIO(_REG_701C0(PIPE_C, 4))); + + MMIO_DH(_MMIO(_REG_701C4(PIPE_A, 1))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_A, 2))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_A, 3))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_A, 4))); + + MMIO_DH(_MMIO(_REG_701C4(PIPE_B, 1))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_B, 2))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_B, 3))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_B, 4))); + + MMIO_DH(_MMIO(_REG_701C4(PIPE_C, 1))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_C, 2))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_C, 3))); + MMIO_DH(_MMIO(_REG_701C4(PIPE_C, 4))); + + MMIO_D(_MMIO(0x70380)); + MMIO_D(_MMIO(0x71380)); + MMIO_D(_MMIO(0x72380)); + MMIO_D(_MMIO(0x7239c)); + MMIO_D(_MMIO(0x7039c)); + + MMIO_D(_MMIO(0x8f074)); + MMIO_D(_MMIO(0x8f004)); + MMIO_D(_MMIO(0x8f034)); + + MMIO_D(_MMIO(0xb11c)); + + MMIO_D(_MMIO(0x51000)); + MMIO_D(_MMIO(0x6c00c)); + + MMIO_D(RPM_CONFIG0); + MMIO_D(_MMIO(0xd08)); + MMIO_D(RC6_LOCATION); + MMIO_DFH(_MMIO(0x20e0)); + MMIO_DFH(_MMIO(0x20ec)); + + MMIO_DFH(_MMIO(0x4de0)); + MMIO_DFH(_MMIO(0x4de4)); + MMIO_DFH(_MMIO(0x4de8)); + MMIO_DFH(_MMIO(0x4dec)); + MMIO_DFH(_MMIO(0x4df0)); + MMIO_DFH(_MMIO(0x4df4)); + MMIO_DH(_MMIO(0x4dfc)); + MMIO_D(_MMIO(0x46430)); + MMIO_D(_MMIO(0x46520)); + MMIO_D(_MMIO(0xc403c)); + MMIO_D(_MMIO(0xb004)); + MMIO_DH(DMA_CTRL); + + MMIO_D(_MMIO(0x65900)); + MMIO_D(_MMIO(0x1082c0)); + MMIO_D(_MMIO(0x4068)); + MMIO_D(_MMIO(0x67054)); + MMIO_D(_MMIO(0x6e560)); + MMIO_D(_MMIO(0x6e554)); + MMIO_D(_MMIO(0x2b20)); + MMIO_D(_MMIO(0x65f00)); + MMIO_D(_MMIO(0x65f08)); + MMIO_D(_MMIO(0x320f0)); + + MMIO_D(_MMIO(0x70034)); + MMIO_D(_MMIO(0x71034)); + MMIO_D(_MMIO(0x72034)); + + MMIO_D(_MMIO(_PLANE_KEYVAL_1(PIPE_A))); + MMIO_D(_MMIO(_PLANE_KEYVAL_1(PIPE_B))); + MMIO_D(_MMIO(_PLANE_KEYVAL_1(PIPE_C))); + MMIO_D(_MMIO(_PLANE_KEYMAX_1(PIPE_A))); + MMIO_D(_MMIO(_PLANE_KEYMAX_1(PIPE_B))); + MMIO_D(_MMIO(_PLANE_KEYMAX_1(PIPE_C))); + MMIO_D(_MMIO(_PLANE_KEYMSK_1(PIPE_A))); + MMIO_D(_MMIO(_PLANE_KEYMSK_1(PIPE_B))); + MMIO_D(_MMIO(_PLANE_KEYMSK_1(PIPE_C))); + + MMIO_D(_MMIO(0x44500)); + MMIO_DFH(GEN9_CSFE_CHICKEN1_RCS); + MMIO_DFH(GEN8_HDC_CHICKEN1); + MMIO_DFH(GEN9_WM_CHICKEN3); + + MMIO_D(_MMIO(0x4ab8)); + MMIO_D(_MMIO(0x2248)); +} + +static void init_bxt_mmio(struct drm_i915_private *dev_priv) +{ + MMIO_D(GEN7_SAMPLER_INSTDONE); + MMIO_D(GEN7_ROW_INSTDONE); + MMIO_D(GEN8_FAULT_TLB_DATA0); + MMIO_D(GEN8_FAULT_TLB_DATA1); + MMIO_D(ERROR_GEN6); + MMIO_D(DONE_REG); + MMIO_D(EIR); + MMIO_D(PGTBL_ER); + MMIO_D(_MMIO(0x4194)); + MMIO_D(_MMIO(0x4294)); + MMIO_D(_MMIO(0x4494)); + + MMIO_RING_D(RING_PSMI_CTL); + MMIO_RING_D(RING_DMA_FADD); + MMIO_RING_D(RING_DMA_FADD_UDW); + MMIO_RING_D(RING_IPEHR); + MMIO_RING_D(RING_INSTPS); + MMIO_RING_D(RING_BBADDR_UDW); + MMIO_RING_D(RING_BBSTATE); + MMIO_RING_D(RING_IPEIR); + + MMIO_DH(BXT_P_CR_GT_DISP_PWRON); + MMIO_D(BXT_RP_STATE_CAP); + MMIO_DH(BXT_PHY_CTL_FAMILY(DPIO_PHY0)); + MMIO_DH(BXT_PHY_CTL_FAMILY(DPIO_PHY1)); + MMIO_D(BXT_PHY_CTL(PORT_A)); + MMIO_D(BXT_PHY_CTL(PORT_B)); + MMIO_D(BXT_PHY_CTL(PORT_C)); + MMIO_DH(BXT_PORT_PLL_ENABLE(PORT_A)); + MMIO_DH(BXT_PORT_PLL_ENABLE(PORT_B)); + MMIO_DH(BXT_PORT_PLL_ENABLE(PORT_C)); + + MMIO_D(BXT_PORT_CL1CM_DW0(DPIO_PHY0)); + MMIO_D(BXT_PORT_CL1CM_DW9(DPIO_PHY0)); + MMIO_D(BXT_PORT_CL1CM_DW10(DPIO_PHY0)); + MMIO_D(BXT_PORT_CL1CM_DW28(DPIO_PHY0)); + MMIO_D(BXT_PORT_CL1CM_DW30(DPIO_PHY0)); + MMIO_D(BXT_PORT_CL2CM_DW6(DPIO_PHY0)); + MMIO_D(BXT_PORT_REF_DW3(DPIO_PHY0)); + MMIO_D(BXT_PORT_REF_DW6(DPIO_PHY0)); + MMIO_D(BXT_PORT_REF_DW8(DPIO_PHY0)); + + MMIO_D(BXT_PORT_CL1CM_DW0(DPIO_PHY1)); + MMIO_D(BXT_PORT_CL1CM_DW9(DPIO_PHY1)); + MMIO_D(BXT_PORT_CL1CM_DW10(DPIO_PHY1)); + MMIO_D(BXT_PORT_CL1CM_DW28(DPIO_PHY1)); + MMIO_D(BXT_PORT_CL1CM_DW30(DPIO_PHY1)); + MMIO_D(BXT_PORT_CL2CM_DW6(DPIO_PHY1)); + MMIO_D(BXT_PORT_REF_DW3(DPIO_PHY1)); + MMIO_D(BXT_PORT_REF_DW6(DPIO_PHY1)); + MMIO_D(BXT_PORT_REF_DW8(DPIO_PHY1)); + + MMIO_D(BXT_PORT_PLL_EBB_0(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_PLL_EBB_4(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW10_LN01(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW10_GRP(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW12_LN01(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW12_LN23(DPIO_PHY0, DPIO_CH0)); + MMIO_DH(BXT_PORT_PCS_DW12_GRP(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW2_LN0(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW2_GRP(DPIO_PHY0, DPIO_CH0)); + MMIO_DH(BXT_PORT_TX_DW3_LN0(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW3_GRP(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW4_LN0(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW4_GRP(DPIO_PHY0, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH0, 0)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH0, 1)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH0, 2)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH0, 3)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 0)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 1)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 2)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 3)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 6)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 8)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 9)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH0, 10)); + + MMIO_D(BXT_PORT_PLL_EBB_0(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_PLL_EBB_4(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_PCS_DW10_LN01(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_PCS_DW10_GRP(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_PCS_DW12_LN01(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_PCS_DW12_LN23(DPIO_PHY0, DPIO_CH1)); + MMIO_DH(BXT_PORT_PCS_DW12_GRP(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_TX_DW2_LN0(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_TX_DW2_GRP(DPIO_PHY0, DPIO_CH1)); + MMIO_DH(BXT_PORT_TX_DW3_LN0(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_TX_DW3_GRP(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_TX_DW4_LN0(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_TX_DW4_GRP(DPIO_PHY0, DPIO_CH1)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH1, 0)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH1, 1)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH1, 2)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY0, DPIO_CH1, 3)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 0)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 1)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 2)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 3)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 6)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 8)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 9)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY0, DPIO_CH1, 10)); + + MMIO_D(BXT_PORT_PLL_EBB_0(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_PLL_EBB_4(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW10_LN01(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW10_GRP(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW12_LN01(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_PCS_DW12_LN23(DPIO_PHY1, DPIO_CH0)); + MMIO_DH(BXT_PORT_PCS_DW12_GRP(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW2_LN0(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW2_GRP(DPIO_PHY1, DPIO_CH0)); + MMIO_DH(BXT_PORT_TX_DW3_LN0(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW3_GRP(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW4_LN0(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW4_GRP(DPIO_PHY1, DPIO_CH0)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY1, DPIO_CH0, 0)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY1, DPIO_CH0, 1)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY1, DPIO_CH0, 2)); + MMIO_D(BXT_PORT_TX_DW14_LN(DPIO_PHY1, DPIO_CH0, 3)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 0)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 1)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 2)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 3)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 6)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 8)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 9)); + MMIO_D(BXT_PORT_PLL(DPIO_PHY1, DPIO_CH0, 10)); + + MMIO_D(BXT_DE_PLL_CTL); + MMIO_DH(BXT_DE_PLL_ENABLE); + MMIO_D(BXT_DSI_PLL_CTL); + MMIO_D(BXT_DSI_PLL_ENABLE); + + MMIO_D(GEN9_CLKGATE_DIS_0); + MMIO_D(GEN9_CLKGATE_DIS_4); + + MMIO_D(HSW_TVIDEO_DIP_GCP(TRANSCODER_A)); + MMIO_D(HSW_TVIDEO_DIP_GCP(TRANSCODER_B)); + MMIO_D(HSW_TVIDEO_DIP_GCP(TRANSCODER_C)); + + MMIO_D(RC6_CTX_BASE); + + MMIO_D(GEN8_PUSHBUS_CONTROL); + MMIO_D(GEN8_PUSHBUS_ENABLE); + MMIO_D(GEN8_PUSHBUS_SHIFT); + MMIO_D(GEN6_GFXPAUSE); + MMIO_D(GEN8_L3SQCREG1); + + MMIO_DFH(GEN9_CTX_PREEMPT_REG); +} + static bool is_supported_device(struct drm_i915_private *dev_priv) { if (IS_BROADWELL(dev_priv)) @@ -52,92 +1471,44 @@ static bool is_supported_device(struct drm_i915_private *dev_priv) return false; } -/** - * intel_gvt_sanitize_options - sanitize GVT related options - * @dev_priv: drm i915 private data - * - * This function is called at the i915 options sanitize stage. - */ -void intel_gvt_sanitize_options(struct drm_i915_private *dev_priv) +/* Very early HW snapshot for vGPU's initial state */ +void intel_gvt_init_mmio(struct drm_i915_private *dev_priv) { - if (!i915_modparams.enable_gvt) - return; - if (intel_vgpu_active(dev_priv)) { DRM_INFO("GVT-g is disabled for guest\n"); - goto bail; + return; } if (!is_supported_device(dev_priv)) { DRM_INFO("Unsupported device. GVT-g is disabled\n"); - goto bail; - } - - return; -bail: - i915_modparams.enable_gvt = 0; -} - -/** - * intel_gvt_init - initialize GVT components - * @dev_priv: drm i915 private data - * - * This function is called at the initialization stage to create a GVT device. - * - * Returns: - * Zero on success, negative error code if failed. - * - */ -int intel_gvt_init(struct drm_i915_private *dev_priv) -{ - int ret; - - if (i915_inject_load_failure()) - return -ENODEV; - - if (!i915_modparams.enable_gvt) { - DRM_DEBUG_DRIVER("GVT-g is disabled by kernel params\n"); - return 0; + return; } + + dev_priv->hw_init_mmio = kvzalloc(2 * 1024 * 1024, + GFP_KERNEL); + if (!dev_priv->hw_init_mmio) + return; - if (USES_GUC_SUBMISSION(dev_priv)) { - DRM_ERROR("i915 GVT-g loading failed due to Graphics virtualization is not yet supported with GuC submission\n"); - return -EIO; - } + intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL); - /* - * We're not in host or fail to find a MPT module, disable GVT-g - */ - ret = intel_gvt_init_host(); - if (ret) { - DRM_DEBUG_DRIVER("Not in host or MPT modules not found\n"); - goto bail; + init_general_range(dev_priv); + init_bdw_range(dev_priv); + init_general_mmio(dev_priv); + init_bdw_mmio(dev_priv); + if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) || + IS_BROXTON(dev_priv)) { + init_skl_range(dev_priv); + init_skl_mmio(dev_priv); } - - ret = intel_gvt_init_device(dev_priv); - if (ret) { - DRM_DEBUG_DRIVER("Fail to init GVT device\n"); - goto bail; + if (IS_BROXTON(dev_priv)) { + init_bxt_range(dev_priv); + init_bxt_mmio(dev_priv); } - - return 0; - -bail: - i915_modparams.enable_gvt = 0; - return 0; + + intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL); } -/** - * intel_gvt_cleanup - cleanup GVT components when i915 driver is unloading - * @dev_priv: drm i915 private * - * - * This function is called at the i915 driver unloading stage, to shutdown - * GVT components and release the related resources. - */ -void intel_gvt_cleanup(struct drm_i915_private *dev_priv) +void intel_gvt_clean_mmio(struct drm_i915_private *dev_priv) { - if (!intel_gvt_active(dev_priv)) - return; - - intel_gvt_clean_device(dev_priv); + kvfree(dev_priv->hw_init_mmio); } diff --git a/drivers/gpu/drm/i915/intel_gvt.h b/drivers/gpu/drm/i915/intel_gvt.h index 61b246470282..503ca48ecbce 100644 --- a/drivers/gpu/drm/i915/intel_gvt.h +++ b/drivers/gpu/drm/i915/intel_gvt.h @@ -19,32 +19,17 @@ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - */ +*/ #ifndef _INTEL_GVT_H_ #define _INTEL_GVT_H_ -struct intel_gvt; - -#ifdef CONFIG_DRM_I915_GVT -int intel_gvt_init(struct drm_i915_private *dev_priv); -void intel_gvt_cleanup(struct drm_i915_private *dev_priv); -int intel_gvt_init_device(struct drm_i915_private *dev_priv); -void intel_gvt_clean_device(struct drm_i915_private *dev_priv); -int intel_gvt_init_host(void); -void intel_gvt_sanitize_options(struct drm_i915_private *dev_priv); +#if defined(CONFIG_DRM_I915_GVT) || defined(CONFIG_DRM_I915_GVT_MODULE) +void intel_gvt_init_mmio(struct drm_i915_private *dev_priv); +void intel_gvt_clean_mmio(struct drm_i915_private *dev_priv); #else -static inline int intel_gvt_init(struct drm_i915_private *dev_priv) -{ - return 0; -} -static inline void intel_gvt_cleanup(struct drm_i915_private *dev_priv) -{ -} - -static inline void intel_gvt_sanitize_options(struct drm_i915_private *dev_priv) -{ -} +static void intel_gvt_init_mmio(struct drm_i915_private *dev_priv) {} +static void intel_gvt_clean_mmio(struct drm_i915_private *dev_priv) {} #endif -#endif /* _INTEL_GVT_H_ */ +#endif