[v5] drm/i915 : Added Programming of the MOCS
diff mbox

Message ID 1434630585-6613-1-git-send-email-peter.antoine@intel.com
State New
Headers show

Commit Message

Peter Antoine June 18, 2015, 12:29 p.m. UTC
This change adds the programming of the MOCS registers to the gen 9+
platforms. This change set programs the MOCS register values to a set
of values that are defined to be optimal.

It creates a fixed register set that is programmed across the different
engines so that all engines have the same table. This is done as the
main RCS context only holds the registers for itself and the shared
L3 values. By trying to keep the registers consistent across the
different engines it should make the programming for the registers
consistent.

v2:
-'static const' for private data structures and style changes.(Matt Turner)
v3:
- Make the tables "slightly" more readable. (Damien Lespiau)
- Updated tables fix performance regression.
v4:
- Code formatting. (Chris Wilson)
- re-privatised mocs code. (Daniel Vetter)
v5:
- Changed the name of a function. (Chris Wilson)

Signed-off-by: Peter Antoine <peter.antoine@intel.com>
---
 drivers/gpu/drm/i915/Makefile     |   1 +
 drivers/gpu/drm/i915/i915_reg.h   |   9 +
 drivers/gpu/drm/i915/intel_lrc.c  |  10 +-
 drivers/gpu/drm/i915/intel_lrc.h  |   4 +
 drivers/gpu/drm/i915/intel_mocs.c | 370 ++++++++++++++++++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_mocs.h |  61 +++++++
 6 files changed, 454 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/i915/intel_mocs.c
 create mode 100644 drivers/gpu/drm/i915/intel_mocs.h

Comments

Chris Wilson June 18, 2015, 12:59 p.m. UTC | #1
On Thu, Jun 18, 2015 at 01:29:45PM +0100, Peter Antoine wrote:
> @@ -1379,6 +1380,13 @@ static int gen8_init_rcs_context(struct intel_engine_cs *ring,
>  	if (ret)
>  		return ret;
>  
> +	/*
> +	 * Failing to program the MOCS is non-fatal.The system will not
> +	 * run at peak performance. So generate a warning and carry on.
> +	 */
> +	if (intel_rcs_context_init_mocs(ring, ctx) != 0)
> +		DRM_ERROR("MOCS failed to program: expect performance issues.");

You said to expect display corruption as well if this failed.
Fortunately, if this fails, we have severe driver issues...

> +/**
> + * emit_mocs_l3cc_table() - emit the mocs control table
> + * @ringbuf:	DRM device.
> + * @table:	The values to program into the control regs.
> + *
> + * This function simply emits a MI_LOAD_REGISTER_IMM command for the
> + * given table starting at the given address. This register set is  programmed
> + * in pairs.
> + *
> + * Return: Nothing.
> + */
> +static void emit_mocs_l3cc_table(struct intel_ringbuffer *ringbuf,
> +			 struct drm_i915_mocs_table *table) {
> +	unsigned int count;
> +	unsigned int i;
> +	u32 value;
> +	u32 filler = (table->table[0].l3cc_value & 0xffff) |
> +			((table->table[0].l3cc_value & 0xffff) << 16);

l3cc_value is only u16, & 0xffff is just noise, without & you don't need
the parantheses.

> +int intel_rcs_context_init_mocs(struct intel_engine_cs *ring,
> +				struct intel_context *ctx)
> +{
> +	int ret = 0;
> +
> +	struct drm_i915_mocs_table t;
> +	struct drm_device *dev = ring->dev;
> +	struct intel_ringbuffer *ringbuf = ctx->engine[ring->id].ringbuf;
> +
> +	if (get_mocs_settings(dev, &t)) {
> +		u32 table_size;
> +
> +		/*
> +		 * OK. For each supported ring:
> +		 *  number of mocs entries * 2 dwords for each control_value
> +		 *  plus number of mocs entries /2 dwords for l3cc values.
> +		 *
> +		 *  Plus 1 for the load command and 1 for the NOOP per ring
> +		 *  and the l3cc programming.
		 *
		 * With 5 rings and 63 mocs entries, this gives 715
		 * dwords.
> +		 */

> +		table_size = GEN9_NUM_MOCS_RINGS *
> +				((2 * GEN9_NUM_MOCS_ENTRIES) + 2) +
> +				GEN9_NUM_MOCS_ENTRIES + 2;

If you pushed the ring_begin into each function, not only would it be
easier to verify, you then don't need an explanation that starts with
"This looks like a mistake". Validation of ring_begin/ring_advance is by
review, so it has to be easy to review.

> +		ret = intel_logical_ring_begin(ringbuf, ctx, table_size);
> +		if (ret) {
> +			DRM_DEBUG("intel_logical_ring_begin failed %d\n", ret);
> +			return ret;
> +		}
Lespiau, Damien June 18, 2015, 1:50 p.m. UTC | #2
On Thu, Jun 18, 2015 at 01:29:45PM +0100, Peter Antoine wrote:
> @@ -1379,6 +1380,13 @@ static int gen8_init_rcs_context(struct intel_engine_cs *ring,
>  	if (ret)
>  		return ret;
>  
> +	/*
> +	 * Failing to program the MOCS is non-fatal.The system will not
> +	 * run at peak performance. So generate a warning and carry on.
> +	 */
> +	if (intel_rcs_context_init_mocs(ring, ctx) != 0)
> +		DRM_ERROR("MOCS failed to program: expect performance issues.");
> +

Missing a '\n'.

> +static const struct drm_i915_mocs_entry skylake_mocs_table[] = {
> +	 /* {0x00000009, 0x0010} */
> +	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC_ELLC) |
> +		MOCS_LRUM(0) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
> +		MOC_PFM(0) | MOCS_SCF(0)),
> +		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
> +	 /* {0x0000003b, 0x0030} */

We're still missing the usage hints for those configuration entries
That'd help user space a lot, which means make this patch land quicker
as well.

> +int intel_rcs_context_init_mocs(struct intel_engine_cs *ring,
> +				struct intel_context *ctx)
> +{
> +	int ret = 0;
> +
> +	struct drm_i915_mocs_table t;
> +	struct drm_device *dev = ring->dev;
> +	struct intel_ringbuffer *ringbuf = ctx->engine[ring->id].ringbuf;
> +
> +	if (get_mocs_settings(dev, &t)) {
> +		u32 table_size;
> +
> +		/*
> +		 * OK. For each supported ring:
> +		 *  number of mocs entries * 2 dwords for each control_value
> +		 *  plus number of mocs entries /2 dwords for l3cc values.
> +		 *
> +		 *  Plus 1 for the load command and 1 for the NOOP per ring
> +		 *  and the l3cc programming.
> +		 */
> +		table_size = GEN9_NUM_MOCS_RINGS *
> +				((2 * GEN9_NUM_MOCS_ENTRIES) + 2) +
> +				GEN9_NUM_MOCS_ENTRIES + 2;
> +		ret = intel_logical_ring_begin(ringbuf, ctx, table_size);
> +		if (ret) {
> +			DRM_DEBUG("intel_logical_ring_begin failed %d\n", ret);
> +			return ret;
> +		}
> +
> +		/* program the control registers */
> +		emit_mocs_control_table(ringbuf, &t, GEN9_GFX_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_MFX0_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_MFX1_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_VEBOX_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_BLT_MOCS_0);

So, if I'm not mistaken, I think this only works because we fully
initialize the default context at start/reset time through:

  + i915_gem_init_hw()
    + i915_gem_context_enable()
      + cycle through all the rings and call ring->init_context()
        + gen8_init_rcs_context()
	  + intel_rcs_context_init_mocs()
	    (initalize ALL the MOCS!)

So, intializing the other (non-render) MOCS in gen8_init_rcs_context()
isn't the most logical thing to do I'm afraid. What happens if we
suddenly decide that we don't want to fully initialize the default
context at startup but initialize each ring on-demand for that context
as well? We can end up in a situation where we use the blitter first and
we wouldn't have the blitter MOCS initialized.

In that sense, that code makes an assumption about how we do things in a
completely different part of the driver and that's always a potential
source of bugs.

Chris, how far am I ? :p

One way to "solve" this (if that's indeed the issue pointed at by Chris)
would be to decouple the render MOCS from the others, still keep the
render ones in there as they need to be emitted from the ring but put
the other writes (which could be done through MMIO as well) higher in
the chain, could probably make sense in i915_gem_context_enable()?
(which, by the way is awfully namedm should have an _init somewhere?).
It could also be a per-ring vfunc I suppose.

For similar reasons, I think the GuC MOCS should be part of the GuC
init as well so we don't couple too hard different part of the code.

Now, is that really a blocker? I'd say no if we had userspace ready and
could commit that today, because we really want it. Still something to
look at, I could be totally wrong.

The separate header for a single function isn't something we usually do
either, but that can always be folded in later.
Peter Antoine June 18, 2015, 2:45 p.m. UTC | #3
-----Original Message-----
From: Lespiau, Damien 
Sent: Thursday, June 18, 2015 2:51 PM
To: Antoine, Peter
Cc: intel-gfx@lists.freedesktop.org; daniel.vetter.intel.com@irsmsx102.ger.corp.intel.com; chris@chris-wilson.co.uk; mattst88@gmail.com
Subject: Re: [PATCH v5] drm/i915 : Added Programming of the MOCS

On Thu, Jun 18, 2015 at 01:29:45PM +0100, Peter Antoine wrote:
> @@ -1379,6 +1380,13 @@ static int gen8_init_rcs_context(struct intel_engine_cs *ring,
>  	if (ret)
>  		return ret;
>  
> +	/*
> +	 * Failing to program the MOCS is non-fatal.The system will not
> +	 * run at peak performance. So generate a warning and carry on.
> +	 */
> +	if (intel_rcs_context_init_mocs(ring, ctx) != 0)
> +		DRM_ERROR("MOCS failed to program: expect performance issues.");
> +

Missing a '\n'.

Will fix.

> +static const struct drm_i915_mocs_entry skylake_mocs_table[] = {
> +	 /* {0x00000009, 0x0010} */
> +	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC_ELLC) |
> +		MOCS_LRUM(0) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
> +		MOC_PFM(0) | MOCS_SCF(0)),
> +		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
> +	 /* {0x0000003b, 0x0030} */

We're still missing the usage hints for those configuration entries That'd help user space a lot, which means make this patch land quicker as well.

These are boiled down from 250+ requirements from different usecases (opencl, Media, etc...), I can't really generate anymore usage hints.

> +int intel_rcs_context_init_mocs(struct intel_engine_cs *ring,
> +				struct intel_context *ctx)
> +{
> +	int ret = 0;
> +
> +	struct drm_i915_mocs_table t;
> +	struct drm_device *dev = ring->dev;
> +	struct intel_ringbuffer *ringbuf = ctx->engine[ring->id].ringbuf;
> +
> +	if (get_mocs_settings(dev, &t)) {
> +		u32 table_size;
> +
> +		/*
> +		 * OK. For each supported ring:
> +		 *  number of mocs entries * 2 dwords for each control_value
> +		 *  plus number of mocs entries /2 dwords for l3cc values.
> +		 *
> +		 *  Plus 1 for the load command and 1 for the NOOP per ring
> +		 *  and the l3cc programming.
> +		 */
> +		table_size = GEN9_NUM_MOCS_RINGS *
> +				((2 * GEN9_NUM_MOCS_ENTRIES) + 2) +
> +				GEN9_NUM_MOCS_ENTRIES + 2;
> +		ret = intel_logical_ring_begin(ringbuf, ctx, table_size);
> +		if (ret) {
> +			DRM_DEBUG("intel_logical_ring_begin failed %d\n", ret);
> +			return ret;
> +		}
> +
> +		/* program the control registers */
> +		emit_mocs_control_table(ringbuf, &t, GEN9_GFX_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_MFX0_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_MFX1_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_VEBOX_MOCS_0);
> +		emit_mocs_control_table(ringbuf, &t, GEN9_BLT_MOCS_0);

So, if I'm not mistaken, I think this only works because we fully initialize the default context at start/reset time through:

  + i915_gem_init_hw()
    + i915_gem_context_enable()
      + cycle through all the rings and call ring->init_context()
        + gen8_init_rcs_context()
	  + intel_rcs_context_init_mocs()
	    (initalize ALL the MOCS!)

Yes.

So, intializing the other (non-render) MOCS in gen8_init_rcs_context() isn't the most logical thing to do I'm afraid. What happens if we suddenly decide that we don't want to fully initialize the default context at startup but initialize each ring on-demand for that context as well? We can end up in a situation where we use the blitter first and we wouldn't have the blitter MOCS initialized.

In that sense, that code makes an assumption about how we do things in a completely different part of the driver and that's always a potential source of bugs.

Yes, but this is the same with the golden context and the workarounds (as I understand it) so all this code would have to be moved. 

Chris, how far am I ? :p

One way to "solve" this (if that's indeed the issue pointed at by Chris) would be to decouple the render MOCS from the others, still keep the render ones in there as they need to be emitted from the ring but put the other writes (which could be done through MMIO as well) higher in the chain, could probably make sense in i915_gem_context_enable()?
(which, by the way is awfully namedm should have an _init somewhere?).
It could also be a per-ring vfunc I suppose.

For similar reasons, I think the GuC MOCS should be part of the GuC init as well so we don't couple too hard different part of the code.

Now, is that really a blocker? I'd say no if we had userspace ready and could commit that today, because we really want it. Still something to look at, I could be totally wrong.

Not a blocker. It gets a little more interesting, as the L3CC registers are shared across all engines, but is only saved in the RCS context. But, it is reset on the context switch when ELSP is set. So we would have to program it (i.e. MMIO) and also set it in the batch start for the RCS. Each ring would have to have a proper init_context() and these registers programmed there.

The separate header for a single function isn't something we usually do either, but that can always be folded in later.

Yep, I agree that is overkill.

--
Damien
Lespiau, Damien June 18, 2015, 3:25 p.m. UTC | #4
>>On Thu, Jun 18, 2015 at 03:45:44PM +0100, Antoine, Peter wrote:
>> So, intializing the other (non-render) MOCS in gen8_init_rcs_context()
>> isn't the most logical thing to do I'm afraid. What happens if we
>> suddenly decide that we don't want to fully initialize the default
>> context at startup but initialize each ring on-demand for that context
>> as well? We can end up in a situation where we use the blitter first
>> and we wouldn't have the blitter MOCS initialized.
>> 
>> In that sense, that code makes an assumption about how we do things in
>> a completely different part of the driver and that's always a
>> potential source of bugs.
>> 
>
>Yes, but this is the same with the golden context and the workarounds
>(as I understand it) so all this code would have to be moved. 

Ah, but the workarounds in that function are only for registers in the
render context, not other rings/engine.
Chris Wilson June 18, 2015, 3:35 p.m. UTC | #5
On Thu, Jun 18, 2015 at 04:25:47PM +0100, Damien Lespiau wrote:
> >>On Thu, Jun 18, 2015 at 03:45:44PM +0100, Antoine, Peter wrote:
> >> So, intializing the other (non-render) MOCS in gen8_init_rcs_context()
> >> isn't the most logical thing to do I'm afraid. What happens if we
> >> suddenly decide that we don't want to fully initialize the default
> >> context at startup but initialize each ring on-demand for that context
> >> as well? We can end up in a situation where we use the blitter first
> >> and we wouldn't have the blitter MOCS initialized.
> >> 
> >> In that sense, that code makes an assumption about how we do things in
> >> a completely different part of the driver and that's always a
> >> potential source of bugs.
> >> 
> >
> >Yes, but this is the same with the golden context and the workarounds
> >(as I understand it) so all this code would have to be moved. 
> 
> Ah, but the workarounds in that function are only for registers in the
> render context, not other rings/engine.

Yes, but it just so happens that we initialise the default context
before userspace so that we know that context is pristine before sending
batches to the GPU.

This is the reason why I think it is important to mark this function as
being executed at that stage, so that all parties can be sure that the
execution is before real use of the GPU and so we can use the RCS to
initialise the other rings. At the moment, I am happy with baking that
assumption into the code, we can readdress it later if there are non-RCS
operations that must be performed at context init and conflict with the
RCS programming.

If you can think of a suitable comment to forewarn us in future about
potential conflicts in adding xcs->init_context(), be my guest.
-Chris
Lespiau, Damien June 18, 2015, 3:50 p.m. UTC | #6
On Thu, Jun 18, 2015 at 03:45:44PM +0100, Antoine, Peter wrote:
> Not a blocker. It gets a little more interesting, as the L3CC
> registers are shared across all engines, but is only saved in the RCS
> context. But, it is reset on the context switch when ELSP is set. So
> we would have to program it (i.e. MMIO) and also set it in the batch
> start for the RCS. Each ring would have to have a proper
> init_context() and these registers programmed there.

Hum, so yes, it's like you say. I think leaving a comment somewhere in
the init path telling us we rely on the RCS init_context() for all the
rings would be nice, but that's extra topping that can be done any time.
Peter Antoine June 19, 2015, 6:34 a.m. UTC | #7
On Thu, 2015-06-18 at 16:50 +0100, Damien Lespiau wrote:
> On Thu, Jun 18, 2015 at 03:45:44PM +0100, Antoine, Peter wrote:

> > Not a blocker. It gets a little more interesting, as the L3CC

> > registers are shared across all engines, but is only saved in the RCS

> > context. But, it is reset on the context switch when ELSP is set. So

> > we would have to program it (i.e. MMIO) and also set it in the batch

> > start for the RCS. Each ring would have to have a proper

> > init_context() and these registers programmed there.

> 

> Hum, so yes, it's like you say. I think leaving a comment somewhere in

> the init path telling us we rely on the RCS init_context() for all the

> rings would be nice, but that's extra topping that can be done any time.

> 

It is in the file header for the mocs.h in the comment before the table
is defines and in the function header for the init. I can add another
one before it's called, but that might be a little overkill. :)
Daniel Vetter June 22, 2015, 1:50 p.m. UTC | #8
On Fri, Jun 19, 2015 at 06:34:22AM +0000, Antoine, Peter wrote:
> On Thu, 2015-06-18 at 16:50 +0100, Damien Lespiau wrote:
> > On Thu, Jun 18, 2015 at 03:45:44PM +0100, Antoine, Peter wrote:
> > > Not a blocker. It gets a little more interesting, as the L3CC
> > > registers are shared across all engines, but is only saved in the RCS
> > > context. But, it is reset on the context switch when ELSP is set. So
> > > we would have to program it (i.e. MMIO) and also set it in the batch
> > > start for the RCS. Each ring would have to have a proper
> > > init_context() and these registers programmed there.
> > 
> > Hum, so yes, it's like you say. I think leaving a comment somewhere in
> > the init path telling us we rely on the RCS init_context() for all the
> > rings would be nice, but that's extra topping that can be done any time.
> > 
> It is in the file header for the mocs.h in the comment before the table
> is defines and in the function header for the init. I can add another
> one before it's called, but that might be a little overkill. :)

Or can we demote the init_context vfunc to a device-global one? We're
already piling up gt vfuncs in dev_priv->gt, I think it'd fit in there
nicely. Bonus points for pulling the actual setup calls out of per-ring
code too.

Doing setup across all rings in an RCS callback is indeed really
surprising and pretty much guarantees tears and screaming down the road.
Our init paths are littered with little confusions like this causing
really annoying bugs all over.
-Daniel

Patch
diff mbox

diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index b7ddf48..c781e19 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -35,6 +35,7 @@  i915-y += i915_cmd_parser.o \
 	  i915_irq.o \
 	  i915_trace_points.o \
 	  intel_lrc.o \
+	  intel_mocs.o \
 	  intel_ringbuffer.o \
 	  intel_uncore.o
 
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index 7213224..3a435b5 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -7829,4 +7829,13 @@  enum skl_disp_power_wells {
 #define _PALETTE_A (dev_priv->info.display_mmio_offset + 0xa000)
 #define _PALETTE_B (dev_priv->info.display_mmio_offset + 0xa800)
 
+/* MOCS (Memory Object Control State) registers */
+#define GEN9_LNCFCMOCS0		(0xB020)	/* L3 Cache Control base */
+
+#define GEN9_GFX_MOCS_0		(0xc800)	/* Graphics MOCS base register*/
+#define GEN9_MFX0_MOCS_0	(0xc900)	/* Media 0 MOCS base register*/
+#define GEN9_MFX1_MOCS_0	(0xcA00)	/* Media 1 MOCS base register*/
+#define GEN9_VEBOX_MOCS_0	(0xcB00)	/* Video MOCS base register*/
+#define GEN9_BLT_MOCS_0		(0xcc00)	/* Blitter MOCS base register*/
+
 #endif /* _I915_REG_H_ */
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 9f5485d..dd01caf 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -135,6 +135,7 @@ 
 #include <drm/drmP.h>
 #include <drm/i915_drm.h>
 #include "i915_drv.h"
+#include "intel_mocs.h"
 
 #define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
 #define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -796,7 +797,7 @@  static int logical_ring_prepare(struct intel_ringbuffer *ringbuf,
  *
  * Return: non-zero if the ringbuffer is not ready to be written to.
  */
-static int intel_logical_ring_begin(struct intel_ringbuffer *ringbuf,
+int intel_logical_ring_begin(struct intel_ringbuffer *ringbuf,
 				    struct intel_context *ctx, int num_dwords)
 {
 	struct intel_engine_cs *ring = ringbuf->ring;
@@ -1379,6 +1380,13 @@  static int gen8_init_rcs_context(struct intel_engine_cs *ring,
 	if (ret)
 		return ret;
 
+	/*
+	 * Failing to program the MOCS is non-fatal.The system will not
+	 * run at peak performance. So generate a warning and carry on.
+	 */
+	if (intel_rcs_context_init_mocs(ring, ctx) != 0)
+		DRM_ERROR("MOCS failed to program: expect performance issues.");
+
 	return intel_lr_context_render_state_init(ring, ctx);
 }
 
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index 04d3a6d..dbbd6af 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -44,6 +44,10 @@  int intel_logical_rings_init(struct drm_device *dev);
 
 int logical_ring_flush_all_caches(struct intel_ringbuffer *ringbuf,
 				  struct intel_context *ctx);
+
+int intel_logical_ring_begin(struct intel_ringbuffer *ringbuf,
+				    struct intel_context *ctx, int num_dwords);
+
 /**
  * intel_logical_ring_advance() - advance the ringbuffer tail
  * @ringbuf: Ringbuffer to advance.
diff --git a/drivers/gpu/drm/i915/intel_mocs.c b/drivers/gpu/drm/i915/intel_mocs.c
new file mode 100644
index 0000000..1651379e
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_mocs.c
@@ -0,0 +1,370 @@ 
+/*
+ * Copyright (c) 2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions: *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "intel_mocs.h"
+#include "intel_lrc.h"
+#include "intel_ringbuffer.h"
+
+/* structures required */
+struct drm_i915_mocs_entry {
+	u32	control_value;
+	u16	l3cc_value;
+};
+
+struct drm_i915_mocs_table {
+	u32					size;
+	const struct drm_i915_mocs_entry	*table;
+};
+
+/* Defines for the tables (XXX_MOCS_0 - XXX_MOCS_63) */
+#define	MOCS_CACHEABILITY(value)	(value << 0)
+#define	MOCS_TGT_CACHE(value)		(value << 2)
+#define	MOCS_LRUM(value)		(value << 4)
+#define	MOCS_AOM(value)			(value << 6)
+#define	MOCS_LECC_ESC(value)		(value << 7)
+#define	MOCS_LECC_SCC(value)		(value << 8)
+#define	MOC_PFM(value)			(value << 11)
+#define	MOCS_SCF(value)			(value << 14)
+
+/* Defines for the tables (LNCFMOCS0 - LNCFMOCS31) - two entries per word */
+#define	MOCS_ESC(value)			(value << 0)
+#define	MOCS_SCC(value)			(value << 1)
+#define	MOCS_L3_CACHEABILITY(value)	(value << 4)
+
+/* Helper defines */
+#define GEN9_NUM_MOCS_RINGS	(5)	/* Number of mocs engines to program */
+#define GEN9_NUM_MOCS_ENTRIES	(63)	/* 63 out of 64 - 64 is rsvrd */
+
+/* EDRAM Caching options */
+#define EDRAM_PAGETABLE		(0)
+#define EDRAM_UC		(1)
+#define EDRAM_RESERVED		(2)
+#define EDRAM_WB		(3)
+
+/* L3 Caching options */
+#define L3_DIRECT		(0)
+#define L3_UC			(1)
+#define L3_RESERVED		(2)
+#define L3_WB			(3)
+
+/* target cache */
+#define ELLC			(0)
+#define LLC			(1)
+#define LLC_ELLC		(2)
+
+/*
+ * MOCS tables
+ *
+ * These are the MOCS tables that are programmed across all the rings.
+ * The control value is programmed to all the rings that support the
+ * MOCS registers. While the l3cc_values are only programmed to the
+ * LNCFCMOCS0 - LNCFCMOCS32 registers.
+ *
+ * NOTE: These tables MUST start with being uncached and the length MUST be
+ *       less than 63 as the last two registers are reserved by the hardware.
+ */
+static const struct drm_i915_mocs_entry skylake_mocs_table[] = {
+	 /* {0x00000009, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(0) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x0000003b, 0x0030} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_WB))},
+	 /* {0x00000039, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000017, 0x0030} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(1) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_WB))},
+	 /* {0x00000017, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(1) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000019, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(1) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000037, 0x0030} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_WB))},
+	 /* {0x00000037, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x0000003b, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+};
+
+static const struct drm_i915_mocs_entry broxton_mocs_table[] = {
+	 /* {0x00000001, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(ELLC) |
+		MOCS_LRUM(0) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000005, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(0) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000005, 0x0030} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(0) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_WB))},
+	 /* {0x00000017, 0x0030} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(1) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_WB))},
+	 /* {0x00000017, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(1) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000019, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_UC) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(1) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x00000037, 0x0030} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_WB))},
+	 /* {0x00000037, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+	 /* {0x0000003b, 0x0010} */
+	{(MOCS_CACHEABILITY(EDRAM_WB) | MOCS_TGT_CACHE(LLC_ELLC) |
+		MOCS_LRUM(3) | MOCS_AOM(0) | MOCS_LECC_ESC(0) | MOCS_SCC(0) |
+		MOC_PFM(0) | MOCS_SCF(0)),
+		(MOCS_ESC(0) | MOCS_SCC(0) | MOCS_L3_CACHEABILITY(L3_UC))},
+};
+
+/**
+ * get_mocs_settings
+ *
+ * This function will return the values of the MOCS table that needs to
+ * be programmed for the platform. It will return the values that need
+ * to be programmed and if they need to be programmed.
+ *
+ * If the return values is false then the registers do not need programming.
+ */
+static bool get_mocs_settings(struct drm_device *dev,
+				struct drm_i915_mocs_table *table) {
+	bool	result = false;
+
+	if (IS_SKYLAKE(dev)) {
+		table->size  = ARRAY_SIZE(skylake_mocs_table);
+		table->table = skylake_mocs_table;
+		result = true;
+	} else if (IS_BROXTON(dev)) {
+		table->size  = ARRAY_SIZE(broxton_mocs_table);
+		table->table = broxton_mocs_table;
+		result = true;
+	} else {
+		/* Platform that should have a MOCS table does not */
+		WARN_ON(INTEL_INFO(dev)->gen >= 9);
+	}
+
+	return result;
+}
+
+/**
+ * emit_mocs_control_table() - emit the mocs control table
+ * @ringbuf:	DRM device.
+ * @table:	The values to program into the control regs.
+ * @reg_base:	The base for the Engine that needs to be programmed.
+ *
+ * This function simply emits a MI_LOAD_REGISTER_IMM command for the
+ * given table starting at the given address.
+ *
+ * Return: Nothing.
+ */
+static void emit_mocs_control_table(struct intel_ringbuffer *ringbuf,
+				    struct drm_i915_mocs_table *table,
+				    u32 reg_base)
+{
+	unsigned int index;
+
+	intel_logical_ring_emit(ringbuf,
+			MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
+
+	for (index = 0; index < table->size; index++) {
+		intel_logical_ring_emit(ringbuf, reg_base + (index * 4));
+		intel_logical_ring_emit(ringbuf,
+					table->table[index].control_value);
+	}
+
+	/*
+	 * Ok, now set the unused entries to uncached. These entries are
+	 * officially undefined and no contact is given for the contents and
+	 * settings is given for these entries.
+	 *
+	 * Entry 0 in the table is uncached - so we are just written that
+	 * value to all the used entries.
+	 */
+	for (; index < GEN9_NUM_MOCS_ENTRIES; index++) {
+		intel_logical_ring_emit(ringbuf, reg_base + (index * 4));
+		intel_logical_ring_emit(ringbuf, table->table[0].control_value);
+	}
+
+	intel_logical_ring_emit(ringbuf, MI_NOOP);
+}
+
+/**
+ * emit_mocs_l3cc_table() - emit the mocs control table
+ * @ringbuf:	DRM device.
+ * @table:	The values to program into the control regs.
+ *
+ * This function simply emits a MI_LOAD_REGISTER_IMM command for the
+ * given table starting at the given address. This register set is  programmed
+ * in pairs.
+ *
+ * Return: Nothing.
+ */
+static void emit_mocs_l3cc_table(struct intel_ringbuffer *ringbuf,
+			 struct drm_i915_mocs_table *table) {
+	unsigned int count;
+	unsigned int i;
+	u32 value;
+	u32 filler = (table->table[0].l3cc_value & 0xffff) |
+			((table->table[0].l3cc_value & 0xffff) << 16);
+
+	intel_logical_ring_emit(ringbuf,
+			MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES / 2));
+
+	for (i = 0, count = 0; i < table->size / 2; i++, count += 2) {
+		value = (table->table[count].l3cc_value & 0xffff) |
+			((table->table[count + 1].l3cc_value & 0xffff) << 16);
+
+		intel_logical_ring_emit(ringbuf, GEN9_LNCFCMOCS0 + (i * 4));
+		intel_logical_ring_emit(ringbuf, value);
+	}
+
+	if (table->size & 0x01) {
+		/* Odd table size - 1 left over */
+		value = (table->table[count].l3cc_value & 0xffff) |
+			((table->table[0].l3cc_value & 0xffff) << 16);
+	} else
+		value = filler;
+
+	/*
+	 * Now set the rest of the table to uncached - use entry 0 as this
+	 * will be uncached. Leave the last pair as initialised as they are
+	 * reserved by the hardware.
+	 */
+	for (; i < (GEN9_NUM_MOCS_ENTRIES / 2) - 1; i++) {
+		intel_logical_ring_emit(ringbuf, GEN9_LNCFCMOCS0 + (i * 4));
+		intel_logical_ring_emit(ringbuf, value);
+
+		value = filler;
+	}
+
+	intel_logical_ring_emit(ringbuf, MI_NOOP);
+}
+
+/*
+ * intel_rcs_context_init_mocs() - program the MOCS register.
+ *
+ * ring:	The ring that the programming batch will be run in.
+ * ctx:		The intel_context to be used.
+ *
+ * This function will emit a batch buffer with the values required for
+ * programming the MOCS register values for all the currently supported
+ * rings.
+ *
+ * These registers are partially stored in the RCS context, so they are
+ * emitted at the same time so that when a context is created these registers
+ * are set up. These registers have to be emitted into the start of the
+ * context as setting the ELSP will re-init some of these registers back
+ * to the hw values.
+ *
+ * Return: 0 on success, otherwise the error status.
+ */
+int intel_rcs_context_init_mocs(struct intel_engine_cs *ring,
+				struct intel_context *ctx)
+{
+	int ret = 0;
+
+	struct drm_i915_mocs_table t;
+	struct drm_device *dev = ring->dev;
+	struct intel_ringbuffer *ringbuf = ctx->engine[ring->id].ringbuf;
+
+	if (get_mocs_settings(dev, &t)) {
+		u32 table_size;
+
+		/*
+		 * OK. For each supported ring:
+		 *  number of mocs entries * 2 dwords for each control_value
+		 *  plus number of mocs entries /2 dwords for l3cc values.
+		 *
+		 *  Plus 1 for the load command and 1 for the NOOP per ring
+		 *  and the l3cc programming.
+		 */
+		table_size = GEN9_NUM_MOCS_RINGS *
+				((2 * GEN9_NUM_MOCS_ENTRIES) + 2) +
+				GEN9_NUM_MOCS_ENTRIES + 2;
+		ret = intel_logical_ring_begin(ringbuf, ctx, table_size);
+		if (ret) {
+			DRM_DEBUG("intel_logical_ring_begin failed %d\n", ret);
+			return ret;
+		}
+
+		/* program the control registers */
+		emit_mocs_control_table(ringbuf, &t, GEN9_GFX_MOCS_0);
+		emit_mocs_control_table(ringbuf, &t, GEN9_MFX0_MOCS_0);
+		emit_mocs_control_table(ringbuf, &t, GEN9_MFX1_MOCS_0);
+		emit_mocs_control_table(ringbuf, &t, GEN9_VEBOX_MOCS_0);
+		emit_mocs_control_table(ringbuf, &t, GEN9_BLT_MOCS_0);
+
+		/* now program the l3cc registers */
+		emit_mocs_l3cc_table(ringbuf, &t);
+
+		intel_logical_ring_advance(ringbuf);
+
+		DRM_DEBUG("MOCS: Table set in Context\n");
+	} else {
+		DRM_DEBUG("MOCS: Table Not supported on platform\n");
+	}
+
+	return ret;
+}
+
diff --git a/drivers/gpu/drm/i915/intel_mocs.h b/drivers/gpu/drm/i915/intel_mocs.h
new file mode 100644
index 0000000..67c2deb
--- /dev/null
+++ b/drivers/gpu/drm/i915/intel_mocs.h
@@ -0,0 +1,61 @@ 
+/*
+ * Copyright (c) 2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef INTEL_MOCS_H
+#define INTEL_MOCS_H
+
+/**
+ * DOC: Memory Objects Control State (MOCS)
+ *
+ * Motivation:
+ * In previous Gens the MOCS settings was a value that was set by user land as
+ * part of the batch. In Gen9 this has changed to be a single table (per ring)
+ * that all batches now reference by index instead of programming the MOCS
+ * directly.
+ *
+ * The one wrinkle in this is that only PART of the MOCS tables are included
+ * in context (The GFX_MOCS_0 - GFX_MOCS_64 and the LNCFCMOCS0 - LNCFCMOCS32
+ * registers). The rest are not (the settings for the other rings).
+ *
+ * This table needs to be set at system start-up because the way the table
+ * interacts with the contexts and the GmmLib interface.
+ *
+ *
+ * Implementation:
+ *
+ * The table is programmed on a platform basis from a table that is generated
+ * from the one that has been agreed by the different responsible parties. This
+ * tables (one per supported platform) is defined in intel_mocs.c and is
+ * programmed in the first batch after the context is loaded (with the hardware
+ * workarounds). This will then let the usual context handling keep the MOCS in
+ * step.
+ */
+
+#include <drm/drmP.h>
+#include "i915_drv.h"
+
+int intel_rcs_context_init_mocs(struct intel_engine_cs *ring,
+			struct intel_context *ctx);
+
+#endif
+