From patchwork Thu Jul 20 08:49:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9854315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0C0B260388 for ; Thu, 20 Jul 2017 09:06:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00D8428782 for ; Thu, 20 Jul 2017 09:06:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E9A4828787; Thu, 20 Jul 2017 09:06:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2D3EF28782 for ; Thu, 20 Jul 2017 09:06:25 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dY7NQ-0007SB-0k; Thu, 20 Jul 2017 09:04:08 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dY7NO-0007RN-M2 for xen-devel@lists.xenproject.org; Thu, 20 Jul 2017 09:04:06 +0000 Received: from [193.109.254.147] by server-6.bemta-6.messagelabs.com id 3A/DC-03937-60270795; Thu, 20 Jul 2017 09:04:06 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrFLMWRWlGSWpSXmKPExsXS1tYhr8tSVBB p8OejjsX3LZOZHBg9Dn+4whLAGMWamZeUX5HAmnFw+memguuRFXMf3GNsYJzn2MXIwSEkUCGx /ldxFyMnh4QAr8SRZTNYIewAiVMXl7J3MXIBlTQwSux4sR4swSagLvH4aw8TiC0ioCRxb9VkJ pAiZoFtTBLvL79gB0kIC9hJfNozB8xmEVCVePe8gQXE5hXwkOj/f5YdYoOcxMljk8GGcgp4Sj zv3gwWFwKqubfyL/MERt4FjAyrGDWKU4vKUot0DY31kooy0zNKchMzc3QNDcz0clOLixPTU3M Sk4r1kvNzNzECw4EBCHYwflkWcIhRkoNJSZSXJbAgUogvKT+lMiOxOCO+qDQntfgQowwHh5IE 78MCoJxgUWp6akVaZg4wMGHSEhw8SiIQad7igsTc4sx0iNQpRl2OVxP+f2MSYsnLz0uVEuddC lIkAFKUUZoHNwIWJZcYZaWEeRmBjhLiKUgtys0sQZV/xSjOwagkzPscZApPZl4J3KZXQEcwAR 3xyA3siJJEhJRUA+O2NMsJ2y68XRxt9mWheGjRwvL7E+fnLo87v8ng1YKlj1s5559gSHfYa5E TpJQ9NZnLqCwqVXee7ezdW8TLLnwxio+Zeu/zct2QEJZjn548C8/U7/ZgCc7apC+iuerT82/R gp/+Bb14acnJ7H5mi8iJK5amohl3uOr9OQTVtjx/t7Yr0OZ682UlluKMREMt5qLiRAD8ySAxj QIAAA== X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-9.tower-27.messagelabs.com!1500541426!108131258!7 X-Originating-IP: [134.134.136.31] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 62225 invoked from network); 20 Jul 2017 09:04:04 -0000 Received: from mga06.intel.com (HELO mga06.intel.com) (134.134.136.31) by server-9.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Jul 2017 09:04:04 -0000 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga104.jf.intel.com with ESMTP; 20 Jul 2017 02:04:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.40,383,1496127600"; d="scan'208"; a="1153387065" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.80]) by orsmga001.jf.intel.com with ESMTP; 20 Jul 2017 02:04:01 -0700 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Thu, 20 Jul 2017 16:49:08 +0800 Message-Id: <1500540553-29199-8-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1500540553-29199-1-git-send-email-yi.y.sun@linux.intel.com> References: <1500540553-29199-1-git-send-email-yi.y.sun@linux.intel.com> Cc: kevin.tian@intel.com, wei.liu2@citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, ian.jackson@eu.citrix.com, Yi Sun , julien.grall@arm.com, mengxu@cis.upenn.edu, jbeulich@suse.com, chao.p.peng@linux.intel.com, roger.pau@citrix.com Subject: [Xen-devel] [RFC v2 07/12] x86: implement set value flow for MBA. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP This patch implements set value flow for MBA including its callback function and domctl interface. It also changes the memebers in 'cos_write_info' to transfer the feature array, feature properties array and value array. Then, we can write all features values on the cos id into MSRs. Signed-off-by: Yi Sun --- xen/arch/x86/domctl.c | 6 ++ xen/arch/x86/psr.c | 184 ++++++++++++++++++++++++++++++++------------ xen/include/public/domctl.h | 1 + 3 files changed, 143 insertions(+), 48 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index fa5e6d4..0aa9f34 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1468,6 +1468,12 @@ long arch_do_domctl( PSR_VAL_TYPE_L2); break; + case XEN_DOMCTL_PSR_MBA_OP_SET_THRTL: + ret = psr_set_val(d, domctl->u.psr_alloc_op.target, + domctl->u.psr_alloc_op.data, + PSR_VAL_TYPE_MBA); + break; + case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM: ret = psr_get_val(d, domctl->u.psr_alloc_op.target, &val32, PSR_VAL_TYPE_L3); diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index c75755b..dafbbb0 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -136,6 +136,12 @@ static const struct feat_props { /* write_msr is used to write out feature MSR register. */ void (*write_msr)(unsigned int cos, uint32_t val, enum psr_val_type type); + + /* + * check_change_val is used to check if input val fulfills SDM requirement. + * Change it to valid value if SDM allows. + */ + bool (*check_change_val)(const struct feat_node *feat, unsigned long *val); } *feat_props[FEAT_TYPE_NUM]; /* @@ -273,29 +279,6 @@ static enum psr_feat_type psr_val_type_to_feat_type(enum psr_val_type type) return feat_type; } -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm) -{ - unsigned int first_bit, zero_bit; - - /* Set bits should only in the range of [0, cbm_len]. */ - if ( cbm & (~0ul << cbm_len) ) - return false; - - /* At least one bit need to be set. */ - if ( cbm == 0 ) - return false; - - first_bit = find_first_bit(&cbm, cbm_len); - zero_bit = find_next_zero_bit(&cbm, cbm_len, first_bit); - - /* Set bits should be contiguous. */ - if ( zero_bit < cbm_len && - find_next_bit(&cbm, cbm_len, zero_bit) < cbm_len ) - return false; - - return true; -} - /* CAT common functions implementation. */ static int init_alloc_features(const struct cpuid_leaf *regs, struct feat_node *feat, @@ -410,6 +393,31 @@ static bool cat_get_feat_info(const struct feat_node *feat, return true; } +static bool cat_check_change_val(const struct feat_node *feat, + unsigned long *cbm) +{ + unsigned int first_bit, zero_bit; + unsigned int cbm_len = feat->cat_info.cbm_len; + + /* Set bits should only in the range of [0, cbm_len]. */ + if ( *cbm & (~0ul << cbm_len) ) + return false; + + /* At least one bit need to be set. */ + if ( *cbm == 0 ) + return false; + + first_bit = find_first_bit(cbm, cbm_len); + zero_bit = find_next_zero_bit(cbm, cbm_len, first_bit); + + /* Set bits should be contiguous. */ + if ( zero_bit < cbm_len && + find_next_bit(cbm, cbm_len, zero_bit) < cbm_len ) + return false; + + return true; +} + /* L3 CAT props */ static void l3_cat_write_msr(unsigned int cos, uint32_t val, enum psr_val_type type) @@ -423,6 +431,7 @@ static const struct feat_props l3_cat_props = { .alt_type = PSR_VAL_TYPE_UNKNOWN, .get_feat_info = cat_get_feat_info, .write_msr = l3_cat_write_msr, + .check_change_val = cat_check_change_val, }; /* L3 CDP props */ @@ -453,6 +462,7 @@ static const struct feat_props l3_cdp_props = { .alt_type = PSR_VAL_TYPE_L3, .get_feat_info = l3_cdp_get_feat_info, .write_msr = l3_cdp_write_msr, + .check_change_val = cat_check_change_val, }; /* L2 CAT props */ @@ -468,6 +478,7 @@ static const struct feat_props l2_cat_props = { .alt_type = PSR_VAL_TYPE_UNKNOWN, .get_feat_info = cat_get_feat_info, .write_msr = l2_cat_write_msr, + .check_change_val = cat_check_change_val, }; /* MBA props */ @@ -487,6 +498,44 @@ static bool mba_get_feat_info(const struct feat_node *feat, static void mba_write_msr(unsigned int cos, uint32_t val, enum psr_val_type type) { + wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val); +} + +static bool mba_check_change_val(const struct feat_node *feat, + unsigned long *thrtl) +{ + if ( *thrtl > feat->mba_info.thrtl_max ) + return false; + + /* + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"): + * 1. Linear mode: In the linear mode the input precision is defined + * as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the + * input precision is 10%. Values not an even multiple of the + * precision (e.g., 12%) will be rounded down (e.g., to 10% delay + * applied). + * 2. Non-linear mode: Input delay values are powers-of-two from zero + * to the MBA_MAX value from CPUID. In this case any values not a + * power of two will be rounded down the next nearest power of two. + */ + if ( feat->mba_info.linear ) + { + unsigned int mod; + + if ( feat->mba_info.thrtl_max >= 100 ) + return false; + + mod = *thrtl % (100 - feat->mba_info.thrtl_max); + *thrtl -= mod; + } + else + { + /* Not power of 2. */ + if ( *thrtl & (*thrtl - 1) ) + *thrtl = *thrtl & (1 << (flsl(*thrtl) - 1)); + } + + return true; } static const struct feat_props mba_props = { @@ -495,6 +544,7 @@ static const struct feat_props mba_props = { .alt_type = PSR_VAL_TYPE_UNKNOWN, .get_feat_info = mba_get_feat_info, .write_msr = mba_write_msr, + .check_change_val = mba_check_change_val, }; static void __init parse_psr_bool(char *s, char *value, char *feature, @@ -915,6 +965,7 @@ static int insert_val_into_array(uint32_t val[], const struct feat_node *feat; const struct feat_props *props; unsigned int i; + unsigned long check_val = new_val; int ret; ASSERT(feat_type < FEAT_TYPE_NUM); @@ -939,9 +990,11 @@ static int insert_val_into_array(uint32_t val[], if ( array_len < props->cos_num ) return -ENOSPC; - if ( !psr_check_cbm(feat->cat_info.cbm_len, new_val) ) + if ( !props->check_change_val(feat, &check_val) ) return -EINVAL; + new_val = check_val; + /* * Value setting position is same as feature array. * For CDP, user may set both DATA and CODE to same value. For such case, @@ -1167,28 +1220,74 @@ static unsigned int get_socket_cpu(unsigned int socket) return nr_cpu_ids; } +/* + * Because multiple features may co-exist, we need handle all features to write + * values of them into a COS register with new COS ID. E.g: + * 1. L3 CAT and MBA co-exist. + * 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff, + * the MBA Thrtle of Dom1 is 0xa. + * 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is + * used by Dom2 too, we have to pick a new COS ID 3. The original values of + * Dom1 on COS ID 3 may be below: + * --------- + * | COS 3 | + * --------- + * L3 CAT | 0x7ff | + * --------- + * MBA | 0x0 | + * --------- + * 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA + * Thrtl is set. So, the values on COS ID 3 should be below. + * --------- + * | COS 3 | + * --------- + * L3 CAT | 0x1ff | + * --------- + * MBA | 0x14 | + * --------- + * + * So, we should write all features values into their MSRs. That requires the + * feature array, feature properties array and value array are input. + */ struct cos_write_info { unsigned int cos; - struct feat_node *feature; + struct feat_node **features; const uint32_t *val; - const struct feat_props *props; + unsigned int array_len; + const struct feat_props **props; }; static void do_write_psr_msrs(void *data) { const struct cos_write_info *info = data; - struct feat_node *feat = info->feature; - const struct feat_props *props = info->props; - unsigned int i, cos = info->cos, cos_num = props->cos_num; + unsigned int i, j, index = 0, array_len = info->array_len, cos = info->cos; + const uint32_t *val_array = info->val; - for ( i = 0; i < cos_num; i++ ) + for ( i = 0; i < ARRAY_SIZE(feat_props); i++ ) { - if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] ) + struct feat_node *feat = info->features[i]; + const struct feat_props *props = info->props[i]; + unsigned int cos_num; + + if ( !feat || !props ) + continue; + + cos_num = props->cos_num; + if ( array_len < cos_num ) + return; + + for ( j = 0; j < cos_num; j++ ) { - feat->cos_reg_val[cos * cos_num + i] = info->val[i]; - props->write_msr(cos, info->val[i], props->type[i]); + if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] ) + { + feat->cos_reg_val[cos * cos_num + j] = val_array[index + j]; + props->write_msr(cos, val_array[index + j], props->type[j]); + } } + + array_len -= cos_num; + index += cos_num; } } @@ -1196,30 +1295,19 @@ static int write_psr_msrs(unsigned int socket, unsigned int cos, const uint32_t val[], unsigned int array_len, enum psr_feat_type feat_type) { - int ret; struct psr_socket_info *info = get_socket_info(socket); struct cos_write_info data = { .cos = cos, - .feature = info->features[feat_type], - .props = feat_props[feat_type], + .features = info->features, + .val = val, + .array_len = array_len, + .props = feat_props, }; if ( cos > info->features[feat_type]->cos_max ) return -EINVAL; - /* Skip to the feature's value head. */ - ret = skip_prior_features(&array_len, feat_type); - if ( ret < 0 ) - return ret; - else - val += ret; - - if ( array_len < feat_props[feat_type]->cos_num ) - return -ENOSPC; - - data.val = val; - if ( socket == cpu_to_socket(smp_processor_id()) ) do_write_psr_msrs(&data); else diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index 319666e..33dc909 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1144,6 +1144,7 @@ struct xen_domctl_psr_alloc_op { #define XEN_DOMCTL_PSR_CAT_OP_GET_L3_DATA 5 #define XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM 6 #define XEN_DOMCTL_PSR_CAT_OP_GET_L2_CBM 7 +#define XEN_DOMCTL_PSR_MBA_OP_SET_THRTL 8 #define XEN_DOMCTL_PSR_MBA_OP_GET_THRTL 9 uint32_t cmd; /* IN: XEN_DOMCTL_PSR_CAT_OP_* */ uint32_t target; /* IN */