From patchwork Wed Feb 15 08:49:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Sun X-Patchwork-Id: 9573597 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DC9BE60493 for ; Wed, 15 Feb 2017 08:55:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE3C028448 for ; Wed, 15 Feb 2017 08:55:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C31012844E; Wed, 15 Feb 2017 08:55:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 13B9128448 for ; Wed, 15 Feb 2017 08:55:47 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cdvKx-0007NZ-4n; Wed, 15 Feb 2017 08:53:19 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cdvKv-0007MA-J6 for xen-devel@lists.xenproject.org; Wed, 15 Feb 2017 08:53:17 +0000 Received: from [193.109.254.147] by server-6.bemta-6.messagelabs.com id 23/C7-15112-CF614A85; Wed, 15 Feb 2017 08:53:16 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHLMWRWlGSWpSXmKPExsXS1tYhoftbbEm Ewb9LnBbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8blTf/YCtbFVhxs28vewPjPoYuRi0NIYBqj RNfui+xdjJwcEgK8EkeWzWCFsAMkJjz8zAxR1MAo0b6kixkkwSagLvH4aw8TiC0ioCRxb9VkJ pAiZoFFTBIrJn1n62Lk4BAW8JE49ZkHpIZFQFVi8dTbYL28Au4SU+Y+YIZYICdx8thksGWcAh 4Sxy7MAbOFgGqO9kxjmsDIu4CRYRWjenFqUVlqka6ZXlJRZnpGSW5iZo6uoYGZXm5qcXFiemp OYlKxXnJ+7iZGYDgwAMEOxnkn/A8xSnIwKYnysh1bHCHEl5SfUpmRWJwRX1Sak1p8iFGGg0NJ gpcJGF5CgkWp6akVaZk5wMCESUtw8CiJ8P4UBUrzFhck5hZnpkOkTjEqSonzfgRJCIAkMkrz4 Npg0XCJUVZKmJcR6BAhnoLUotzMElT5V4ziHIxKwrz/QabwZOaVwE1/BbSYCWgxa9xCkMUliQ gpqQbGjfIrn/GVs15XOtaSktl1I+PU89W+sVe39PIlb3rKIyT8epKHu0akZpk309OfumHvklc 83fGLY8qZMH6597M2JN8MPWiudzhw2XOrD2GPWgpyN9y+5/R/b3dCrfIW3ie7LB0Wzbj7sbXC nPuKdb5QZ29KqBbLFgbjlYvMxD4J1E+9GNnL5yOlxFKckWioxVxUnAgAhapU/oECAAA= X-Env-Sender: yi.y.sun@linux.intel.com X-Msg-Ref: server-8.tower-27.messagelabs.com!1487148772!76852937!9 X-Originating-IP: [134.134.136.24] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjQgPT4gMzkwOTcx\n X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 26177 invoked from network); 15 Feb 2017 08:53:14 -0000 Received: from mga09.intel.com (HELO mga09.intel.com) (134.134.136.24) by server-8.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 15 Feb 2017 08:53:14 -0000 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Feb 2017 00:53:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.35,165,1484035200"; d="scan'208"; a="1094969692" Received: from vmmmba-s2600wft.bj.intel.com ([10.240.193.63]) by orsmga001.jf.intel.com with ESMTP; 15 Feb 2017 00:53:11 -0800 From: Yi Sun To: xen-devel@lists.xenproject.org Date: Wed, 15 Feb 2017 16:49:23 +0800 Message-Id: <1487148579-7243-9-git-send-email-yi.y.sun@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1487148579-7243-1-git-send-email-yi.y.sun@linux.intel.com> References: <1487148579-7243-1-git-send-email-yi.y.sun@linux.intel.com> Cc: kevin.tian@intel.com, wei.liu2@citrix.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, he.chen@linux.intel.com, ian.jackson@eu.citrix.com, Yi Sun , mengxu@cis.upenn.edu, jbeulich@suse.com, chao.p.peng@linux.intel.com Subject: [Xen-devel] [PATCH v8 08/24] x86: refactor psr: set value: implement framework. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP As set value flow is the most complicated one in psr, it will be divided to some patches to make things clearer. This patch implements the set value framework to show a whole picture firstly. It also changes domctl interface to make it more general. To make the set value flow be general and can support multiple features at same time, it includes below steps: 1. Get COS ID of current domain using. 2. Assemble a value array to store all features current value in it and replace the current value of the feature which is being set to the new input value. 3. Find if there is already a COS ID on which all features' values are same as the array. Then, we can reuse this COS ID. 4. If fail to find, we need pick an available COS ID. Only COS ID which ref is 0 or 1 can be picked. 5. Write all features MSRs according to the COS ID. 6. Update ref according to COS ID. 7. Save the COS ID into current domain's psr_cos_ids[socket] so that we can know which COS the domain is using on the socket. So, some functions are abstracted and the callback functions will be implemented in next patches. Here is an example to understand the process. The CPU supports two featuers, e.g. L3 CAT and L2 CAT. user wants to set L3 CAT of Dom1 to 0x1ff. 1. Get the old_cos of Dom1 which is 0. L3 CAT is the first element of feature list. The COS registers values are below at this time. ------------------------------- | COS 0 | COS 1 | COS 2 | ... | ------------------------------- L3 CAT | 0x7ff | ... | ... | ... | ------------------------------- L2 CAT | 0xff | ... | ... | ... | ------------------------------- 2. Assemble The value array to be: val[0]: 0x1ff val[1]: 0xff 3. It cannot find a matching COS. 4. Allocate COS 1 to store the value set. 5. Write the COS 1 registers. The COS registers values are changed to below now. ------------------------------- | COS 0 | COS 1 | COS 2 | ... | ------------------------------- L3 CAT | 0x7ff | 0x1ff | ... | ... | ------------------------------- L2 CAT | 0xff | 0xff | ... | ... | ------------------------------- 6. The ref[1] is increased to 1 because Dom1 is using it now. 7. Save 1 to Dom1's psr_cos_ids[socket]. Then, user wants to set L3 CAT of Dom2 to 0x1ff too. The old_cos of Dom2 is 0 too. Repeat above flow. The val array assembled is: val[0]: 0x1ff val[1]: 0xff So, it can find a matching COS, COS 1. Then, it can reuse COS 1 for Dom2. The ref[1] is increased to 2 now because both Dom1 and Dom2 are using this COS ID. Set 1 to Dom2's psr_cos_ids[socket]. Signed-off-by: Yi Sun --- xen/arch/x86/domctl.c | 18 ++--- xen/arch/x86/psr.c | 202 +++++++++++++++++++++++++++++++++++++++++++++- xen/include/asm-x86/psr.h | 4 +- 3 files changed, 210 insertions(+), 14 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 098e399..f8f5539 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -1422,21 +1422,21 @@ long arch_do_domctl( switch ( domctl->u.psr_cat_op.cmd ) { case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM: - ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target, - domctl->u.psr_cat_op.data, - PSR_CBM_TYPE_L3); + ret = psr_set_val(d, domctl->u.psr_cat_op.target, + domctl->u.psr_cat_op.data, + PSR_CBM_TYPE_L3); break; case XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE: - ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target, - domctl->u.psr_cat_op.data, - PSR_CBM_TYPE_L3_CODE); + ret = psr_set_val(d, domctl->u.psr_cat_op.target, + domctl->u.psr_cat_op.data, + PSR_CBM_TYPE_L3_CODE); break; case XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA: - ret = psr_set_l3_cbm(d, domctl->u.psr_cat_op.target, - domctl->u.psr_cat_op.data, - PSR_CBM_TYPE_L3_DATA); + ret = psr_set_val(d, domctl->u.psr_cat_op.target, + domctl->u.psr_cat_op.data, + PSR_CBM_TYPE_L3_DATA); break; case XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM: diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index c1afd36..d414b5e 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -546,18 +546,214 @@ int psr_get_val(struct domain *d, unsigned int socket, return psr_get(socket, type, NULL, 0, d, val); } -int psr_set_l3_cbm(struct domain *d, unsigned int socket, - uint64_t cbm, enum cbm_type type) +/* Set value functions */ +static unsigned int get_cos_num(const struct psr_socket_info *info) { return 0; } +static int assemble_val_array(uint64_t *val, + uint32_t array_len, + const struct psr_socket_info *info, + unsigned int old_cos) +{ + return -EINVAL; +} + +static int set_new_val_to_array(uint64_t *val, + uint32_t array_len, + const struct psr_socket_info *info, + enum psr_feat_type feat_type, + enum cbm_type type, + uint64_t m) +{ + return -EINVAL; +} + +static int find_cos(const uint64_t *val, uint32_t array_len, + enum psr_feat_type feat_type, + const struct psr_socket_info *info) +{ + return -ENOENT; +} + +static int pick_avail_cos(const struct psr_socket_info *info, + const uint64_t *val, uint32_t array_len, + unsigned int old_cos, + enum psr_feat_type feat_type) +{ + return -ENOENT; +} + +static int write_psr_msr(unsigned int socket, unsigned int cos, + const uint64_t *val) +{ + return -ENOENT; +} + +int psr_set_val(struct domain *d, unsigned int socket, + uint64_t val, enum cbm_type type) +{ + unsigned int old_cos; + int cos, ret; + unsigned int *ref; + uint64_t *val_array; + struct psr_socket_info *info = get_socket_info(socket); + uint32_t array_len; + enum psr_feat_type feat_type; + + if ( IS_ERR(info) ) + return PTR_ERR(info); + + feat_type = psr_cbm_type_to_feat_type(type); + if ( !test_bit(feat_type, &info->feat_mask) ) + return -ENOENT; + + /* + * Step 0: + * old_cos means the COS ID current domain is using. By default, it is 0. + * + * For every COS ID, there is a reference count to record how many domains + * are using the COS register corresponding to this COS ID. + * - If ref[old_cos] is 0, that means this COS is not used by any domain. + * - If ref[old_cos] is 1, that means this COS is only used by current + * domain. + * - If ref[old_cos] is more than 1, that mean multiple domains are using + * this COS. + */ + old_cos = d->arch.psr_cos_ids[socket]; + if ( old_cos > MAX_COS_REG_CNT ) + return -EOVERFLOW; + + ref = info->cos_ref; + + /* + * Step 1: + * Assemle a value array to store all featues cos_reg_val[old_cos]. + * And, set the input val into array according to the feature's + * position in array. + */ + array_len = get_cos_num(info); + val_array = xzalloc_array(uint64_t, array_len); + if ( !val_array ) + return -ENOMEM; + + if ( (ret = assemble_val_array(val_array, array_len, info, old_cos)) != 0 ) + { + xfree(val_array); + return ret; + } + + if ( (ret = set_new_val_to_array(val_array, array_len, info, + feat_type, type, val)) != 0 ) + { + xfree(val_array); + return ret; + } + + /* + * Lock here to make sure the ref is not changed during find and + * write process. + */ + spin_lock(&info->ref_lock); + + /* + * Step 2: + * Try to find if there is already a COS ID on which all features' values + * are same as the array. Then, we can reuse this COS ID. + */ + cos = find_cos(val_array, array_len, feat_type, info); + if ( cos >= 0 ) + { + if ( cos == old_cos ) + { + spin_unlock(&info->ref_lock); + xfree(val_array); + return 0; + } + } + else + { + /* + * Step 3: + * If fail to find, we need allocate a new COS ID. + * If multiple domains are using same COS ID, its ref is more + * than 1. That means we cannot free this COS to make current domain + * use it. Because other domains are using the value saved in the COS. + * Unless the ref is changed to 1 (mean only current domain is using + * it), we cannot allocate the COS ID to current domain. + * So, only the COS ID which ref is 1 or 0 can be allocated. + */ + cos = pick_avail_cos(info, val_array, array_len, old_cos, feat_type); + if ( cos < 0 ) + { + spin_unlock(&info->ref_lock); + xfree(val_array); + return cos; + } + + /* + * Step 4: + * Write all features MSRs according to the COS ID. + */ + ret = write_psr_msr(socket, cos, val_array); + if ( ret ) + { + spin_unlock(&info->ref_lock); + xfree(val_array); + return ret; + } + } + + /* + * Step 5: + * Update ref according to COS ID. + */ + ref[cos]++; + ASSERT(ref[cos] || cos == 0); + ref[old_cos]--; + spin_unlock(&info->ref_lock); + + /* + * Step 6: + * Save the COS ID into current domain's psr_cos_ids[] so that we can know + * which COS the domain is using on the socket. One domain can only use + * one COS ID at same time on each socket. + */ + d->arch.psr_cos_ids[socket] = cos; + xfree(val_array); + + return 0; +} + /* Called with domain lock held, no extra lock needed for 'psr_cos_ids' */ static void psr_free_cos(struct domain *d) { - if( !d->arch.psr_cos_ids ) + unsigned int socket, cos; + + if ( !d->arch.psr_cos_ids ) return; + /* Domain is free so its cos_ref should be decreased. */ + for ( socket = 0; socket < nr_sockets; socket++ ) + { + struct psr_socket_info *info; + + /* cos 0 is default one which does not need be handled. */ + if ( (cos = d->arch.psr_cos_ids[socket]) == 0 ) + continue; + + /* + * If domain uses other cos ids, all corresponding refs must have been + * increased 1 for this domain. So, we need decrease them. + */ + info = socket_info + socket; + ASSERT(info->cos_ref[cos] || cos == 0); + spin_lock(&info->ref_lock); + info->cos_ref[cos]--; + spin_unlock(&info->ref_lock); + } + xfree(d->arch.psr_cos_ids); d->arch.psr_cos_ids = NULL; } diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h index 569e7df..fde7882 100644 --- a/xen/include/asm-x86/psr.h +++ b/xen/include/asm-x86/psr.h @@ -72,8 +72,8 @@ int psr_get_info(unsigned int socket, enum cbm_type type, uint32_t data[], unsigned int array_len); int psr_get_val(struct domain *d, unsigned int socket, uint64_t *val, enum cbm_type type); -int psr_set_l3_cbm(struct domain *d, unsigned int socket, - uint64_t cbm, enum cbm_type type); +int psr_set_val(struct domain *d, unsigned int socket, + uint64_t val, enum cbm_type type); int psr_domain_init(struct domain *d); void psr_domain_free(struct domain *d);