From patchwork Wed Sep 13 13:25:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13383015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF060EDEC51 for ; Wed, 13 Sep 2023 13:25:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240893AbjIMN0B (ORCPT ); Wed, 13 Sep 2023 09:26:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240900AbjIMN0B (ORCPT ); Wed, 13 Sep 2023 09:26:01 -0400 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA1AD19BD for ; Wed, 13 Sep 2023 06:25:56 -0700 (PDT) Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Rm1R57042z6K6TF; Wed, 13 Sep 2023 21:25:21 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 13 Sep 2023 14:25:54 +0100 From: Jonathan Cameron To: , Michael Tsirkin , Fan Ni , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , CC: Subject: [PATCH v4 1/4] hw/cxl: Push cxl_decoder_count_enc() and cxl_decode_ig() into .c Date: Wed, 13 Sep 2023 14:25:20 +0100 Message-ID: <20230913132523.29780-2-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> References: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhrpeml100004.china.huawei.com (7.191.162.219) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org There is no strong justification for keeping these in the header so push them down into the associated cxl-component-utils.c file. Suggested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Fan Ni Signed-off-by: Jonathan Cameron --- include/hw/cxl/cxl_component.h | 18 ++---------------- hw/cxl/cxl-component-utils.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+), 16 deletions(-) diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h index 42c7e581a7..bdb3881a6b 100644 --- a/include/hw/cxl/cxl_component.h +++ b/include/hw/cxl/cxl_component.h @@ -225,26 +225,12 @@ void cxl_component_create_dvsec(CXLComponentState *cxl_cstate, enum reg_type cxl_dev_type, uint16_t length, uint16_t type, uint8_t rev, uint8_t *body); -static inline int cxl_decoder_count_enc(int count) -{ - switch (count) { - case 1: return 0; - case 2: return 1; - case 4: return 2; - case 6: return 3; - case 8: return 4; - case 10: return 5; - } - return 0; -} +int cxl_decoder_count_enc(int count); uint8_t cxl_interleave_ways_enc(int iw, Error **errp); uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp); -static inline hwaddr cxl_decode_ig(int ig) -{ - return 1ULL << (ig + 8); -} +hwaddr cxl_decode_ig(int ig); CXLComponentState *cxl_get_hb_cstate(PCIHostState *hb); bool cxl_get_hb_passthrough(PCIHostState *hb); diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index 378f1082ce..ea2d4770ec 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -13,6 +13,24 @@ #include "hw/pci/pci.h" #include "hw/cxl/cxl.h" +int cxl_decoder_count_enc(int count) +{ + switch (count) { + case 1: return 0; + case 2: return 1; + case 4: return 2; + case 6: return 3; + case 8: return 4; + case 10: return 5; + } + return 0; +} + +hwaddr cxl_decode_ig(int ig) +{ + return 1ULL << (ig + 8); +} + static uint64_t cxl_cache_mem_read_reg(void *opaque, hwaddr offset, unsigned size) { From patchwork Wed Sep 13 13:25:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13383016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8135AEDEC50 for ; Wed, 13 Sep 2023 13:26:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240877AbjIMN0f (ORCPT ); Wed, 13 Sep 2023 09:26:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240915AbjIMN0b (ORCPT ); Wed, 13 Sep 2023 09:26:31 -0400 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FD4B19B6 for ; Wed, 13 Sep 2023 06:26:27 -0700 (PDT) Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.201]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Rm1Ly2SPPz6D9BV; Wed, 13 Sep 2023 21:21:46 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 13 Sep 2023 14:26:25 +0100 From: Jonathan Cameron To: , Michael Tsirkin , Fan Ni , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , CC: Subject: [PATCH v4 2/4] hw/cxl: Add utility functions decoder interleave ways and target count. Date: Wed, 13 Sep 2023 14:25:21 +0100 Message-ID: <20230913132523.29780-3-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> References: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhrpeml100004.china.huawei.com (7.191.162.219) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org As an encoded version of these key configuration parameters is available in a register, provide functions to extract it again so as to avoid the need for duplicating the storage. Whilst here update the _enc() function to include additional values as defined in the CXL 3.0 specification. Whilst they are not currently used in the emulation, they may be in future and it is easier to compare with the specification if all values are covered. Add a spec reference for cxl_interleave_ways_enc() for consistency with the target count equivalent (and because it's nice to know where the magic numbers come from). Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Fan Ni Signed-off-by: Jonathan Cameron --- v4: Add specification reference for cxl_interleave_ways_enc() as suggested by Fan. --- include/hw/cxl/cxl_component.h | 2 ++ hw/cxl/cxl-component-utils.c | 60 ++++++++++++++++++++++++++++++---- 2 files changed, 56 insertions(+), 6 deletions(-) diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h index bdb3881a6b..ef9e033919 100644 --- a/include/hw/cxl/cxl_component.h +++ b/include/hw/cxl/cxl_component.h @@ -226,8 +226,10 @@ void cxl_component_create_dvsec(CXLComponentState *cxl_cstate, uint16_t type, uint8_t rev, uint8_t *body); int cxl_decoder_count_enc(int count); +int cxl_decoder_count_dec(int enc_cnt); uint8_t cxl_interleave_ways_enc(int iw, Error **errp); +int cxl_interleave_ways_dec(uint8_t iw_enc, Error **errp); uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp); hwaddr cxl_decode_ig(int ig); diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index ea2d4770ec..5f38f2016f 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -13,15 +13,45 @@ #include "hw/pci/pci.h" #include "hw/cxl/cxl.h" +/* CXL r3.0 Section 8.2.4.19.1 CXL HDM Decoder Capability Register */ int cxl_decoder_count_enc(int count) { switch (count) { - case 1: return 0; - case 2: return 1; - case 4: return 2; - case 6: return 3; - case 8: return 4; - case 10: return 5; + case 1: return 0x0; + case 2: return 0x1; + case 4: return 0x2; + case 6: return 0x3; + case 8: return 0x4; + case 10: return 0x5; + /* Switches and Host Bridges may have more than 10 decoders */ + case 12: return 0x6; + case 14: return 0x7; + case 16: return 0x8; + case 20: return 0x9; + case 24: return 0xa; + case 28: return 0xb; + case 32: return 0xc; + } + return 0; +} + +int cxl_decoder_count_dec(int enc_cnt) +{ + switch (enc_cnt) { + case 0x0: return 1; + case 0x1: return 2; + case 0x2: return 4; + case 0x3: return 6; + case 0x4: return 8; + case 0x5: return 10; + /* Switches and Host Bridges may have more than 10 decoders */ + case 0x6: return 12; + case 0x7: return 14; + case 0x8: return 16; + case 0x9: return 20; + case 0xa: return 24; + case 0xb: return 28; + case 0xc: return 32; } return 0; } @@ -393,6 +423,7 @@ void cxl_component_create_dvsec(CXLComponentState *cxl, cxl->dvsec_offset += length; } +/* CXL r3.0 Section 8.2.4.19.7 CXL HDM Decoder n Control Register */ uint8_t cxl_interleave_ways_enc(int iw, Error **errp) { switch (iw) { @@ -410,6 +441,23 @@ uint8_t cxl_interleave_ways_enc(int iw, Error **errp) } } +int cxl_interleave_ways_dec(uint8_t iw_enc, Error **errp) +{ + switch (iw_enc) { + case 0x0: return 1; + case 0x1: return 2; + case 0x2: return 4; + case 0x3: return 8; + case 0x4: return 16; + case 0x8: return 3; + case 0x9: return 6; + case 0xa: return 12; + default: + error_setg(errp, "Encoded interleave ways: %d not supported", iw_enc); + return 0; + } +} + uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp) { switch (gran) { From patchwork Wed Sep 13 13:25:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13383017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6151EDEC4D for ; Wed, 13 Sep 2023 13:27:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240894AbjIMN1D (ORCPT ); Wed, 13 Sep 2023 09:27:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240923AbjIMN1C (ORCPT ); Wed, 13 Sep 2023 09:27:02 -0400 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5A5F19B1 for ; Wed, 13 Sep 2023 06:26:58 -0700 (PDT) Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.226]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Rm1SH1j7Mz6K5sp; Wed, 13 Sep 2023 21:26:23 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 13 Sep 2023 14:26:56 +0100 From: Jonathan Cameron To: , Michael Tsirkin , Fan Ni , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , CC: Subject: [PATCH v4 3/4] hw/cxl: Fix and use same calculation for HDM decoder block size everywhere Date: Wed, 13 Sep 2023 14:25:22 +0100 Message-ID: <20230913132523.29780-4-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> References: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhrpeml100004.china.huawei.com (7.191.162.219) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org In order to avoid having the size of the per HDM decoder register block repeated in lots of places, create the register definitions for HDM decoder 1 and use the offset between the first registers in HDM decoder 0 and HDM decoder 1 to establish the offset. Calculate in each function as this is more obvious and leads to shorter line lengths than a single #define which would need a long name to be specific enough. Note that the code currently only supports one decoder, so the bugs this fixes don't actually affect anything. Signed-off-by: Jonathan Cameron Reviewed-by: Fan Ni Reviewed-by: Philippe Mathieu-Daudé --- v3: New patch to separate this out from the addition of HDM decoders. --- include/hw/cxl/cxl_component.h | 2 ++ hw/cxl/cxl-component-utils.c | 19 +++++++++++-------- hw/cxl/cxl-host.c | 4 +++- hw/mem/cxl_type3.c | 24 +++++++++++++++--------- 4 files changed, 31 insertions(+), 18 deletions(-) diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h index ef9e033919..7c864d2044 100644 --- a/include/hw/cxl/cxl_component.h +++ b/include/hw/cxl/cxl_component.h @@ -148,6 +148,8 @@ REG32(CXL_HDM_DECODER_GLOBAL_CONTROL, CXL_HDM_REGISTERS_OFFSET + 4) FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, HDM_DECODER_ENABLE, 1, 1) HDM_DECODER_INIT(0); +/* Only used for HDM decoder registers block address increment */ +HDM_DECODER_INIT(1); /* 8.2.5.13 - CXL Extended Security Capability Structure (Root complex only) */ #define EXTSEC_ENTRY_MAX 256 diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index 5f38f2016f..c0630ba5c1 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -210,6 +210,7 @@ static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk, enum reg_type type) { int decoder_count = 1; + int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; int i; ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, DECODER_COUNT, @@ -222,19 +223,21 @@ static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk, HDM_DECODER_ENABLE, 0); write_msk[R_CXL_HDM_DECODER_GLOBAL_CONTROL] = 0x3; for (i = 0; i < decoder_count; i++) { - write_msk[R_CXL_HDM_DECODER0_BASE_LO + i * 0x20] = 0xf0000000; - write_msk[R_CXL_HDM_DECODER0_BASE_HI + i * 0x20] = 0xffffffff; - write_msk[R_CXL_HDM_DECODER0_SIZE_LO + i * 0x20] = 0xf0000000; - write_msk[R_CXL_HDM_DECODER0_SIZE_HI + i * 0x20] = 0xffffffff; - write_msk[R_CXL_HDM_DECODER0_CTRL + i * 0x20] = 0x13ff; + write_msk[R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc] = 0xf0000000; + write_msk[R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc] = 0xffffffff; + write_msk[R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc] = 0xf0000000; + write_msk[R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc] = 0xffffffff; + write_msk[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc] = 0x13ff; if (type == CXL2_DEVICE || type == CXL2_TYPE3_DEVICE || type == CXL2_LOGICAL_DEVICE) { - write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * 0x20] = 0xf0000000; + write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * hdm_inc] = + 0xf0000000; } else { - write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * 0x20] = 0xffffffff; + write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * hdm_inc] = + 0xffffffff; } - write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_HI + i * 0x20] = 0xffffffff; + write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_HI + i * hdm_inc] = 0xffffffff; } } diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index f0920da956..73c5426476 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -101,12 +101,14 @@ void cxl_fmws_link_targets(CXLState *cxl_state, Error **errp) static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr, uint8_t *target) { + int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; uint32_t ctrl; uint32_t ig_enc; uint32_t iw_enc; uint32_t target_idx; + int i = 0; - ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL]; + ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc]; if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { return false; } diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 4e314748d3..cd92813436 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -377,34 +377,36 @@ static void build_dvsecs(CXLType3Dev *ct3d) static void hdm_decoder_commit(CXLType3Dev *ct3d, int which) { + int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; ComponentRegisters *cregs = &ct3d->cxl_cstate.crb; uint32_t *cache_mem = cregs->cache_mem_registers; uint32_t ctrl; assert(which == 0); - ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL); + ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc); /* TODO: Sanity checks that the decoder is possible */ ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0); ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1); - stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL, ctrl); + stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl); } static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which) { + int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; ComponentRegisters *cregs = &ct3d->cxl_cstate.crb; uint32_t *cache_mem = cregs->cache_mem_registers; uint32_t ctrl; assert(which == 0); - ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL); + ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc); ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0); ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 0); - stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL, ctrl); + stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl); } static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err) @@ -761,26 +763,30 @@ static void ct3_exit(PCIDevice *pci_dev) /* TODO: Support multiple HDM decoders and DPA skip */ static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr, uint64_t *dpa) { + int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; uint32_t *cache_mem = ct3d->cxl_cstate.crb.cache_mem_registers; uint64_t decoder_base, decoder_size, hpa_offset; uint32_t hdm0_ctrl; int ig, iw; + int i = 0; - decoder_base = (((uint64_t)cache_mem[R_CXL_HDM_DECODER0_BASE_HI] << 32) | - cache_mem[R_CXL_HDM_DECODER0_BASE_LO]); + decoder_base = + (((uint64_t)cache_mem[R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc] << 32) | + cache_mem[R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc]); if ((uint64_t)host_addr < decoder_base) { return false; } hpa_offset = (uint64_t)host_addr - decoder_base; - decoder_size = ((uint64_t)cache_mem[R_CXL_HDM_DECODER0_SIZE_HI] << 32) | - cache_mem[R_CXL_HDM_DECODER0_SIZE_LO]; + decoder_size = + ((uint64_t)cache_mem[R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc] << 32) | + cache_mem[R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc]; if (hpa_offset >= decoder_size) { return false; } - hdm0_ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL]; + hdm0_ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc]; iw = FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IW); ig = FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IG); From patchwork Wed Sep 13 13:25:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 13383018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE66FEDEC4D for ; Wed, 13 Sep 2023 13:27:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240886AbjIMN1h (ORCPT ); Wed, 13 Sep 2023 09:27:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240948AbjIMN1e (ORCPT ); Wed, 13 Sep 2023 09:27:34 -0400 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4BEBF19B4 for ; Wed, 13 Sep 2023 06:27:30 -0700 (PDT) Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Rm1Ss4fP7z6K6T2; Wed, 13 Sep 2023 21:26:53 +0800 (CST) Received: from SecurePC-101-06.china.huawei.com (10.122.247.231) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Wed, 13 Sep 2023 14:27:26 +0100 From: Jonathan Cameron To: , Michael Tsirkin , Fan Ni , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , CC: Subject: [PATCH v4 4/4] hw/cxl: Support 4 HDM decoders at all levels of topology Date: Wed, 13 Sep 2023 14:25:23 +0100 Message-ID: <20230913132523.29780-5-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> References: <20230913132523.29780-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.122.247.231] X-ClientProxiedBy: lhrpeml100004.china.huawei.com (7.191.162.219) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Support these decoders in CXL host bridges (pxb-cxl), CXL Switch USP and CXL Type 3 end points. Signed-off-by: Jonathan Cameron --- v3: Factor out the hdm_inc changes to previous patch. Fix use of encoded hdm count as if it were decoded in cxl-host. Minor refactoring to make that path and one in cxl_type3.c look more similar. --- include/hw/cxl/cxl_component.h | 10 +++- hw/cxl/cxl-component-utils.c | 7 ++- hw/cxl/cxl-host.c | 67 ++++++++++++++++-------- hw/mem/cxl_type3.c | 96 +++++++++++++++++++++++----------- 4 files changed, 124 insertions(+), 56 deletions(-) diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h index 7c864d2044..3c795a6278 100644 --- a/include/hw/cxl/cxl_component.h +++ b/include/hw/cxl/cxl_component.h @@ -135,6 +135,10 @@ REG32(CXL_RAS_ERR_HEADER0, CXL_RAS_REGISTERS_OFFSET + 0x18) REG32(CXL_HDM_DECODER##n##_TARGET_LIST_LO, \ CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24) \ REG32(CXL_HDM_DECODER##n##_TARGET_LIST_HI, \ + CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28) \ + REG32(CXL_HDM_DECODER##n##_DPA_SKIP_LO, \ + CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24) \ + REG32(CXL_HDM_DECODER##n##_DPA_SKIP_HI, \ CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28) REG32(CXL_HDM_DECODER_CAPABILITY, CXL_HDM_REGISTERS_OFFSET) @@ -147,9 +151,13 @@ REG32(CXL_HDM_DECODER_GLOBAL_CONTROL, CXL_HDM_REGISTERS_OFFSET + 4) FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, POISON_ON_ERR_EN, 0, 1) FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, HDM_DECODER_ENABLE, 1, 1) +/* Support 4 decoders at all levels of topology */ +#define CXL_HDM_DECODER_COUNT 4 + HDM_DECODER_INIT(0); -/* Only used for HDM decoder registers block address increment */ HDM_DECODER_INIT(1); +HDM_DECODER_INIT(2); +HDM_DECODER_INIT(3); /* 8.2.5.13 - CXL Extended Security Capability Structure (Root complex only) */ #define EXTSEC_ENTRY_MAX 256 diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c index c0630ba5c1..f3bbf0fd13 100644 --- a/hw/cxl/cxl-component-utils.c +++ b/hw/cxl/cxl-component-utils.c @@ -90,6 +90,9 @@ static void dumb_hdm_handler(CXLComponentState *cxl_cstate, hwaddr offset, switch (offset) { case A_CXL_HDM_DECODER0_CTRL: + case A_CXL_HDM_DECODER1_CTRL: + case A_CXL_HDM_DECODER2_CTRL: + case A_CXL_HDM_DECODER3_CTRL: should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); should_uncommit = !should_commit; break; @@ -129,7 +132,7 @@ static void cxl_cache_mem_write_reg(void *opaque, hwaddr offset, uint64_t value, } if (offset >= A_CXL_HDM_DECODER_CAPABILITY && - offset <= A_CXL_HDM_DECODER0_TARGET_LIST_HI) { + offset <= A_CXL_HDM_DECODER3_TARGET_LIST_HI) { dumb_hdm_handler(cxl_cstate, offset, value); } else { cregs->cache_mem_registers[offset / sizeof(*cregs->cache_mem_registers)] = value; @@ -209,7 +212,7 @@ static void ras_init_common(uint32_t *reg_state, uint32_t *write_msk) static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk, enum reg_type type) { - int decoder_count = 1; + int decoder_count = CXL_HDM_DECODER_COUNT; int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; int i; diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c index 73c5426476..2aa776c79c 100644 --- a/hw/cxl/cxl-host.c +++ b/hw/cxl/cxl-host.c @@ -97,35 +97,58 @@ void cxl_fmws_link_targets(CXLState *cxl_state, Error **errp) } } -/* TODO: support, multiple hdm decoders */ static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr, uint8_t *target) { int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; - uint32_t ctrl; - uint32_t ig_enc; - uint32_t iw_enc; - uint32_t target_idx; - int i = 0; - - ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc]; - if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { - return false; - } - - ig_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG); - iw_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW); - target_idx = (addr / cxl_decode_ig(ig_enc)) % (1 << iw_enc); + unsigned int hdm_count; + bool found = false; + int i; + uint32_t cap; + + cap = ldl_le_p(cache_mem + R_CXL_HDM_DECODER_CAPABILITY); + hdm_count = cxl_decoder_count_dec(FIELD_EX32(cap, + CXL_HDM_DECODER_CAPABILITY, + DECODER_COUNT)); + for (i = 0; i < hdm_count; i++) { + uint32_t ctrl, ig_enc, iw_enc, target_idx; + uint32_t low, high; + uint64_t base, size; + + low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc); + high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc); + base = (low & 0xf0000000) | ((uint64_t)high << 32); + low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc); + high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc); + size = (low & 0xf0000000) | ((uint64_t)high << 32); + if (addr < base || addr >= base + size) { + continue; + } - if (target_idx < 4) { - *target = extract32(cache_mem[R_CXL_HDM_DECODER0_TARGET_LIST_LO], - target_idx * 8, 8); - } else { - *target = extract32(cache_mem[R_CXL_HDM_DECODER0_TARGET_LIST_HI], - (target_idx - 4) * 8, 8); + ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + i * hdm_inc); + if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { + return false; + } + found = true; + ig_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG); + iw_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW); + target_idx = (addr / cxl_decode_ig(ig_enc)) % (1 << iw_enc); + + if (target_idx < 4) { + uint32_t val = ldl_le_p(cache_mem + + R_CXL_HDM_DECODER0_TARGET_LIST_LO + + i * hdm_inc); + *target = extract32(val, target_idx * 8, 8); + } else { + uint32_t val = ldl_le_p(cache_mem + + R_CXL_HDM_DECODER0_TARGET_LIST_HI + + i * hdm_inc); + *target = extract32(val, (target_idx - 4) * 8, 8); + } + break; } - return true; + return found; } static PCIDevice *cxl_cfmws_find_device(CXLFixedWindow *fw, hwaddr addr) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index cd92813436..1658e0cc59 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -382,8 +382,6 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which) uint32_t *cache_mem = cregs->cache_mem_registers; uint32_t ctrl; - assert(which == 0); - ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc); /* TODO: Sanity checks that the decoder is possible */ ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0); @@ -399,8 +397,6 @@ static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which) uint32_t *cache_mem = cregs->cache_mem_registers; uint32_t ctrl; - assert(which == 0); - ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc); ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0); @@ -489,6 +485,21 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, uint64_t value, should_uncommit = !should_commit; which_hdm = 0; break; + case A_CXL_HDM_DECODER1_CTRL: + should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); + should_uncommit = !should_commit; + which_hdm = 1; + break; + case A_CXL_HDM_DECODER2_CTRL: + should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); + should_uncommit = !should_commit; + which_hdm = 2; + break; + case A_CXL_HDM_DECODER3_CTRL: + should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT); + should_uncommit = !should_commit; + which_hdm = 3; + break; case A_CXL_RAS_UNC_ERR_STATUS: { uint32_t capctrl = ldl_le_p(cache_mem + R_CXL_RAS_ERR_CAP_CTRL); @@ -760,40 +771,63 @@ static void ct3_exit(PCIDevice *pci_dev) } } -/* TODO: Support multiple HDM decoders and DPA skip */ static bool cxl_type3_dpa(CXLType3Dev *ct3d, hwaddr host_addr, uint64_t *dpa) { int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO; uint32_t *cache_mem = ct3d->cxl_cstate.crb.cache_mem_registers; - uint64_t decoder_base, decoder_size, hpa_offset; - uint32_t hdm0_ctrl; - int ig, iw; - int i = 0; - - decoder_base = - (((uint64_t)cache_mem[R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc] << 32) | - cache_mem[R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc]); - if ((uint64_t)host_addr < decoder_base) { - return false; - } - - hpa_offset = (uint64_t)host_addr - decoder_base; - - decoder_size = - ((uint64_t)cache_mem[R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc] << 32) | - cache_mem[R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc]; - if (hpa_offset >= decoder_size) { - return false; - } + unsigned int hdm_count; + uint32_t cap; + uint64_t dpa_base = 0; + int i; - hdm0_ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc]; - iw = FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IW); - ig = FIELD_EX32(hdm0_ctrl, CXL_HDM_DECODER0_CTRL, IG); + cap = ldl_le_p(cache_mem + R_CXL_HDM_DECODER_CAPABILITY); + hdm_count = cxl_decoder_count_dec(FIELD_EX32(cap, + CXL_HDM_DECODER_CAPABILITY, + DECODER_COUNT)); + + for (i = 0; i < hdm_count; i++) { + uint64_t decoder_base, decoder_size, hpa_offset, skip; + uint32_t hdm_ctrl, low, high; + int ig, iw; + + low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc); + high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc); + decoder_base = ((uint64_t)high << 32) | (low & 0xf0000000); + + low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc); + high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc); + decoder_size = ((uint64_t)high << 32) | (low & 0xf0000000); + + low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_DPA_SKIP_LO + + i * hdm_inc); + high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_DPA_SKIP_HI + + i * hdm_inc); + skip = ((uint64_t)high << 32) | (low & 0xf0000000); + dpa_base += skip; + + hpa_offset = (uint64_t)host_addr - decoder_base; + + hdm_ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + i * hdm_inc); + iw = FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, IW); + ig = FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, IG); + if (!FIELD_EX32(hdm_ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) { + return false; + } + if (((uint64_t)host_addr < decoder_base) || + (hpa_offset >= decoder_size)) { + dpa_base += decoder_size / + cxl_interleave_ways_dec(iw, &error_fatal); + continue; + } - *dpa = (MAKE_64BIT_MASK(0, 8 + ig) & hpa_offset) | - ((MAKE_64BIT_MASK(8 + ig + iw, 64 - 8 - ig - iw) & hpa_offset) >> iw); + *dpa = dpa_base + + ((MAKE_64BIT_MASK(0, 8 + ig) & hpa_offset) | + ((MAKE_64BIT_MASK(8 + ig + iw, 64 - 8 - ig - iw) & hpa_offset) + >> iw)); - return true; + return true; + } + return false; } static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3d,