| Message ID | 169824893473.1403938.16110924262989774582.stgit@bgt-140510-bm03.eng.stellus.in |
|---|---|
| State | Accepted |
| Commit | e5adb87f2021419add052220b0ceceec734bc3c2 |
| Headers | show |
| Series | [v2] cxl/region: use region (not root decoder) granularity for calculations | expand |
Jim Harris wrote: > Root decoder granularity must match value from CFWMS, which may not > be the region's granularity for non-interleaved root decoders. > > So when calculating granularities for host bridge decoders, use the > region's granularity instead of the root decoder's granularity to ensure > the correct granularities are set for the host bridge decoders and any > downstream switch decoders. > > Test configuration is 1 host bridge * 2 switches * 2 endpoints per switch. > > Region created with 2048 granularity using following command line: > > cxl create-region -m -d decoder0.0 -w 4 mem0 mem2 mem1 mem3 \ > -g 2048 -s 2048M > > Use "cxl list -PDE | grep granularity" to get a view of the granularity > set at each level of the topology. > > Before this patch: > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":512, > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":512, > "interleave_granularity":256, > > After: > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":4096, > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":4096, > "interleave_granularity":2048, > > Signed-off-by: Jim Harris <jim.harris@samsung.com> Applied, I did end up changing the subject to: "cxl/region: Fix x1 root-decoder granularity calculations" ...and added: Fixes: 27b3f8d13830 ("cxl/region: Program target lists") Cc: <stable@vger.kernel.org> Thanks, Jim!
Jim Harris wrote: > Root decoder granularity must match value from CFWMS, which may not > be the region's granularity for non-interleaved root decoders. > > So when calculating granularities for host bridge decoders, use the > region's granularity instead of the root decoder's granularity to ensure > the correct granularities are set for the host bridge decoders and any > downstream switch decoders. > > Test configuration is 1 host bridge * 2 switches * 2 endpoints per switch. > > Region created with 2048 granularity using following command line: > > cxl create-region -m -d decoder0.0 -w 4 mem0 mem2 mem1 mem3 \ > -g 2048 -s 2048M > > Use "cxl list -PDE | grep granularity" to get a view of the granularity > set at each level of the topology. > > Before this patch: > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":512, > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":512, > "interleave_granularity":256, > > After: > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":4096, > "interleave_granularity":2048, > "interleave_granularity":2048, > "interleave_granularity":4096, > "interleave_granularity":2048, > > Signed-off-by: Jim Harris <jim.harris@samsung.com> > --- > drivers/cxl/core/region.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c > index 6d63b8798c29..c25be26d0bdf 100644 > --- a/drivers/cxl/core/region.c > +++ b/drivers/cxl/core/region.c > @@ -1133,7 +1133,14 @@ static int cxl_port_setup_targets(struct cxl_port *port, > } > > if (is_cxl_root(parent_port)) { > - parent_ig = cxlrd->cxlsd.cxld.interleave_granularity; > + /* > + * Root decoder IG is always set to value in CFMWS which > + * may be different than this region's IG. We can use the > + * region's IG here since interleave_granularity_store() > + * does not allow interleaved host-bridges with > + * root IG != region IG. > + */ > + parent_ig = p->interleave_granularity; > parent_iw = cxlrd->cxlsd.cxld.interleave_ways; This causes the kernel to correctly detect that cxl_test was programming an incorrect host-bridge level granularity value. Fixup the test. diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index fb6ab9cef84f..b88546299902 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -831,7 +831,7 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld) cxld->interleave_ways = 2; else cxld->interleave_ways = 1; - cxld->interleave_granularity = 256; + cxld->interleave_granularity = 4096; cxld->hpa_range = (struct range) { .start = base, .end = base + size - 1,
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 6d63b8798c29..c25be26d0bdf 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1133,7 +1133,14 @@ static int cxl_port_setup_targets(struct cxl_port *port, } if (is_cxl_root(parent_port)) { - parent_ig = cxlrd->cxlsd.cxld.interleave_granularity; + /* + * Root decoder IG is always set to value in CFMWS which + * may be different than this region's IG. We can use the + * region's IG here since interleave_granularity_store() + * does not allow interleaved host-bridges with + * root IG != region IG. + */ + parent_ig = p->interleave_granularity; parent_iw = cxlrd->cxlsd.cxld.interleave_ways; /* * For purposes of address bit routing, use power-of-2 math for
Root decoder granularity must match value from CFWMS, which may not be the region's granularity for non-interleaved root decoders. So when calculating granularities for host bridge decoders, use the region's granularity instead of the root decoder's granularity to ensure the correct granularities are set for the host bridge decoders and any downstream switch decoders. Test configuration is 1 host bridge * 2 switches * 2 endpoints per switch. Region created with 2048 granularity using following command line: cxl create-region -m -d decoder0.0 -w 4 mem0 mem2 mem1 mem3 \ -g 2048 -s 2048M Use "cxl list -PDE | grep granularity" to get a view of the granularity set at each level of the topology. Before this patch: "interleave_granularity":2048, "interleave_granularity":2048, "interleave_granularity":512, "interleave_granularity":2048, "interleave_granularity":2048, "interleave_granularity":512, "interleave_granularity":256, After: "interleave_granularity":2048, "interleave_granularity":2048, "interleave_granularity":4096, "interleave_granularity":2048, "interleave_granularity":2048, "interleave_granularity":4096, "interleave_granularity":2048, Signed-off-by: Jim Harris <jim.harris@samsung.com> --- drivers/cxl/core/region.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)