From patchwork Thu Feb 4 22:50:41 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chong Li X-Patchwork-Id: 8228841 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 9910CBEEE5 for ; Thu, 4 Feb 2016 22:54:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 45D6320390 for ; Thu, 4 Feb 2016 22:54:05 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D7D70201CD for ; Thu, 4 Feb 2016 22:54:03 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aRSjq-0000Io-UI; Thu, 04 Feb 2016 22:50:58 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aRSjp-0000Ia-NK for xen-devel@lists.xen.org; Thu, 04 Feb 2016 22:50:57 +0000 Received: from [85.158.137.68] by server-3.bemta-3.messagelabs.com id 25/D1-02499-0D5D3B65; Thu, 04 Feb 2016 22:50:56 +0000 X-Env-Sender: lichong659@gmail.com X-Msg-Ref: server-15.tower-31.messagelabs.com!1454626254!20146558!1 X-Originating-IP: [209.85.213.193] X-SpamReason: No, hits=0.3 required=7.0 tests=MAILTO_TO_SPAM_ADDR X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 33367 invoked from network); 4 Feb 2016 22:50:55 -0000 Received: from mail-ig0-f193.google.com (HELO mail-ig0-f193.google.com) (209.85.213.193) by server-15.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 4 Feb 2016 22:50:55 -0000 Received: by mail-ig0-f193.google.com with SMTP id o2so138570iga.3 for ; Thu, 04 Feb 2016 14:50:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ePCkg2twBtBcsHRqcaCQ/Rk5MDZJ8Ls9fC1JOwB+GU0=; b=sV4l7N2k37XbvAu3DSz6QAVti8w2YgY/0Xh7JUI72J10v0sBOdDI21ch8UhAzXRPLs XMOXCzXyXHc8Nj6+8KwI7wi9Cqd2xSBxmTvncP9QUp57uHhJcaL4uj/vKNmCzTT7chL4 jRlUwpCnSKbFyxwydnecosyMzKtULOapEFsXDLeOirYux2lZeJhitAIMwvK4JZPz5oze rMxqUEmxgzH+HZKMaGtpClblG6TOMDPE9WGgrv/caQ5KUHfOCJben+HX0R9USo0e+UCe nCsGC04auN1lHNT2NtrokQ4SLWfpRVHNdQtnHEQ4iRP0gLUQCKJSg30xTTyo4bG6z02f /wqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ePCkg2twBtBcsHRqcaCQ/Rk5MDZJ8Ls9fC1JOwB+GU0=; b=GCW1fL0/Kklctq9JwM9vDFapeShnyrx9Jw3yzUvSFCqyPSl8ULHDIQ50W/fhh5fiec 5ZSbV1Dg+TSG9xt7qm7rGFfAonTodG0CouLDp1AszSY4vWMcy6HuFPTIYaXDDJ+J9iq5 a1gzDqtmr5FiiAwNo1n3G61H1th9k9VSUjbLDiSS3Qx/iFw7c9lDlQPLgtR2EGibY7KT 52vcAhLOf3yME/14ly+mG/o+cINdGwiKhVsPycTumllfTJw5RuzDXBb9Gyzi7EJnvnEI QDzsy/I8HK9Ru5B8eZHe6MzFchAep0O1nNxmbN4QPQuhbIutsw8FHLCo9M4RQzMCrHWx iW2Q== X-Gm-Message-State: AG10YOS0B1GNUonRa6hzujbQ1fQ0R0k8Ia0xF69MOAg5Eg24ly0UwUlbRZMpF7MuCwy7MQ== X-Received: by 10.50.88.74 with SMTP id be10mr6706187igb.93.1454626254610; Thu, 04 Feb 2016 14:50:54 -0800 (PST) Received: from chong-OptiPlex-960.seas.wustl.edu (admin998.cec.wustl.edu. [128.252.20.193]) by smtp.googlemail.com with ESMTPSA id fk5sm5605439igb.6.2016.02.04.14.50.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Feb 2016 14:50:53 -0800 (PST) From: Chong Li To: xen-devel@lists.xen.org Date: Thu, 4 Feb 2016 16:50:41 -0600 Message-Id: <1454626244-5511-2-git-send-email-lichong659@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1454626244-5511-1-git-send-email-lichong659@gmail.com> References: <1454626244-5511-1-git-send-email-lichong659@gmail.com> Cc: Chong Li , Sisu Xi , george.dunlap@eu.citrix.com, dario.faggioli@citrix.com, Meng Xu , jbeulich@suse.com, lichong659@gmail.com, dgolomb@seas.upenn.edu Subject: [Xen-devel] [PATCH v5 for Xen 4.7 1/4] xen: enable per-VCPU parameter settings for RTDS scheduler X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add XEN_DOMCTL_SCHEDOP_getvcpuinfo and _putvcpuinfo hypercalls to independently get and set the scheduling parameters of each vCPU of a domain Signed-off-by: Chong Li Signed-off-by: Meng Xu Signed-off-by: Sisu Xi --- Changes on PATCH v4: 1) Add uint32_t vcpu_index to struct xen_domctl_scheduler_op. When processing XEN_DOMCTL_SCHEDOP_get/putvcpuinfo, we call hypercall_preemption_check in case the current hypercall lasts too long. If we decide to preempt the current hypercall, we record the index of the most-recent finished vcpu into the vcpu_index of struct xen_domctl_scheduler_op. So when we resume the hypercall after preemption, we start processing from the posion specified by vcpu_index, and don't need to repeat the work that has already been done in the hypercall before the preemption. (This design is based on the do_grant_table_op() in grant_table.c) 2) Coding style changes Changes on PATCH v3: 1) Remove struct xen_domctl_schedparam_t. 2) Change struct xen_domctl_scheduler_op. 3) Check if period/budget is within a validated range Changes on PATCH v2: 1) Change struct xen_domctl_scheduler_op, for transferring per-vcpu parameters between libxc and hypervisor. 2) Handler of XEN_DOMCTL_SCHEDOP_getinfo now just returns the default budget and period values of RTDS scheduler. 3) Handler of XEN_DOMCTL_SCHEDOP_getvcpuinfo now can return a random subset of the parameters of the VCPUs of a specific domain CC: CC: CC: CC: CC: CC: --- xen/common/domctl.c | 5 ++ xen/common/sched_credit.c | 4 ++ xen/common/sched_credit2.c | 4 ++ xen/common/sched_rt.c | 127 ++++++++++++++++++++++++++++++++++++++------ xen/common/schedule.c | 15 ++++-- xen/include/public/domctl.h | 57 +++++++++++++++----- 6 files changed, 180 insertions(+), 32 deletions(-) diff --git a/xen/common/domctl.c b/xen/common/domctl.c index 46b967e..b294221 100644 --- a/xen/common/domctl.c +++ b/xen/common/domctl.c @@ -847,9 +847,14 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl) } case XEN_DOMCTL_scheduler_op: + { ret = sched_adjust(d, &op->u.scheduler_op); + if ( ret == -ERESTART ) + ret = hypercall_create_continuation( + __HYPERVISOR_domctl, "h", u_domctl); copyback = 1; break; + } case XEN_DOMCTL_getdomaininfo: { diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 0dce790..455c684 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -1054,6 +1054,10 @@ csched_dom_cntl( * lock. Runq lock not needed anywhere in here. */ spin_lock_irqsave(&prv->lock, flags); + if ( op->cmd == XEN_DOMCTL_SCHEDOP_putvcpuinfo || + op->cmd == XEN_DOMCTL_SCHEDOP_getvcpuinfo ) + return -EINVAL; + if ( op->cmd == XEN_DOMCTL_SCHEDOP_getinfo ) { op->u.credit.weight = sdom->weight; diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 3c49ffa..c3049a0 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1421,6 +1421,10 @@ csched2_dom_cntl( * runq lock to update csvcs. */ spin_lock_irqsave(&prv->lock, flags); + if ( op->cmd == XEN_DOMCTL_SCHEDOP_putvcpuinfo || + op->cmd == XEN_DOMCTL_SCHEDOP_getvcpuinfo ) + return -EINVAL; + if ( op->cmd == XEN_DOMCTL_SCHEDOP_getinfo ) { op->u.credit2.weight = sdom->weight; diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c index 3f1d047..34ae48d 100644 --- a/xen/common/sched_rt.c +++ b/xen/common/sched_rt.c @@ -86,8 +86,21 @@ #define RTDS_DEFAULT_PERIOD (MICROSECS(10000)) #define RTDS_DEFAULT_BUDGET (MICROSECS(4000)) +/* + * Max period: max delta of time type + * Min period: 100 us, considering the scheduling overhead + */ +#define RTDS_MAX_PERIOD (STIME_DELTA_MAX) +#define RTDS_MIN_PERIOD (MICROSECS(10)) + +/* + * Min budget: 100 us + */ +#define RTDS_MIN_BUDGET (MICROSECS(10)) + #define UPDATE_LIMIT_SHIFT 10 #define MAX_SCHEDULE (MILLISECS(1)) + /* * Flags */ @@ -1129,26 +1142,18 @@ rt_dom_cntl( struct vcpu *v; unsigned long flags; int rc = 0; - + xen_domctl_schedparam_vcpu_t local_sched; + s_time_t period, budget; + uint32_t index; switch ( op->cmd ) { - case XEN_DOMCTL_SCHEDOP_getinfo: - if ( d->max_vcpus > 0 ) - { - spin_lock_irqsave(&prv->lock, flags); - svc = rt_vcpu(d->vcpu[0]); - op->u.rtds.period = svc->period / MICROSECS(1); - op->u.rtds.budget = svc->budget / MICROSECS(1); - spin_unlock_irqrestore(&prv->lock, flags); - } - else - { - /* If we don't have vcpus yet, let's just return the defaults. */ - op->u.rtds.period = RTDS_DEFAULT_PERIOD; - op->u.rtds.budget = RTDS_DEFAULT_BUDGET; - } + case XEN_DOMCTL_SCHEDOP_getinfo: /* return the default parameters */ + spin_lock_irqsave(&prv->lock, flags); + op->u.rtds.period = RTDS_DEFAULT_PERIOD / MICROSECS(1); /* transfer to us */ + op->u.rtds.budget = RTDS_DEFAULT_BUDGET / MICROSECS(1); + spin_unlock_irqrestore(&prv->lock, flags); break; - case XEN_DOMCTL_SCHEDOP_putinfo: + case XEN_DOMCTL_SCHEDOP_putinfo: /* set parameters for all vcpus */ if ( op->u.rtds.period == 0 || op->u.rtds.budget == 0 ) { rc = -EINVAL; @@ -1163,6 +1168,94 @@ rt_dom_cntl( } spin_unlock_irqrestore(&prv->lock, flags); break; + case XEN_DOMCTL_SCHEDOP_getvcpuinfo: + for ( index = op->u.v.vcpu_index; index < op->u.v.nr_vcpus; index++ ) + { + spin_lock_irqsave(&prv->lock, flags); + if ( copy_from_guest_offset(&local_sched, + op->u.v.vcpus, index, 1) ) + { + rc = -EFAULT; + spin_unlock_irqrestore(&prv->lock, flags); + break; + } + if ( local_sched.vcpuid >= d->max_vcpus || + d->vcpu[local_sched.vcpuid] == NULL ) + { + rc = -EINVAL; + spin_unlock_irqrestore(&prv->lock, flags); + break; + } + svc = rt_vcpu(d->vcpu[local_sched.vcpuid]); + + local_sched.s.rtds.budget = svc->budget / MICROSECS(1); + local_sched.s.rtds.period = svc->period / MICROSECS(1); + + if ( __copy_to_guest_offset(op->u.v.vcpus, index, + &local_sched, 1) ) + { + rc = -EFAULT; + spin_unlock_irqrestore(&prv->lock, flags); + break; + } + spin_unlock_irqrestore(&prv->lock, flags); + if ( hypercall_preempt_check() ) + { + op->u.v.vcpu_index = index + 1; + /* hypercall (after preemption) will continue at vcpu_index */ + rc = -ERESTART; + break; + } + } + break; + case XEN_DOMCTL_SCHEDOP_putvcpuinfo: + for ( index = op->u.v.vcpu_index; index < op->u.v.nr_vcpus; index++ ) + { + spin_lock_irqsave(&prv->lock, flags); + if ( copy_from_guest_offset(&local_sched, + op->u.v.vcpus, index, 1) ) + { + rc = -EFAULT; + spin_unlock_irqrestore(&prv->lock, flags); + break; + } + if ( local_sched.vcpuid >= d->max_vcpus || + d->vcpu[local_sched.vcpuid] == NULL ) + { + rc = -EINVAL; + spin_unlock_irqrestore(&prv->lock, flags); + break; + } + svc = rt_vcpu(d->vcpu[local_sched.vcpuid]); + period = MICROSECS(local_sched.s.rtds.period); + budget = MICROSECS(local_sched.s.rtds.budget); + if ( period > RTDS_MAX_PERIOD || budget < RTDS_MIN_BUDGET || + budget > period ) + { + rc = -EINVAL; + spin_unlock_irqrestore(&prv->lock, flags); + break; + } + + /* + * We accept period/budget less than 100 us, but will warn users about + * the large scheduling overhead due to it + */ + if ( period < MICROSECS(100) || budget < MICROSECS(100) ) + printk("Warning: period/budget less than 100 micro-secs " + "results in large scheduling overhead.\n"); + + svc->period = period; + svc->budget = budget; + spin_unlock_irqrestore(&prv->lock, flags); + if ( hypercall_preempt_check() ) + { + op->u.v.vcpu_index = index + 1; + rc = -ERESTART; + break; + } + } + break; } return rc; diff --git a/xen/common/schedule.c b/xen/common/schedule.c index c195129..f4a4032 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -1148,10 +1148,19 @@ long sched_adjust(struct domain *d, struct xen_domctl_scheduler_op *op) if ( ret ) return ret; - if ( (op->sched_id != DOM2OP(d)->sched_id) || - ((op->cmd != XEN_DOMCTL_SCHEDOP_putinfo) && - (op->cmd != XEN_DOMCTL_SCHEDOP_getinfo)) ) + if ( op->sched_id != DOM2OP(d)->sched_id ) return -EINVAL; + else + switch ( op->cmd ) + { + case XEN_DOMCTL_SCHEDOP_putinfo: + case XEN_DOMCTL_SCHEDOP_getinfo: + case XEN_DOMCTL_SCHEDOP_putvcpuinfo: + case XEN_DOMCTL_SCHEDOP_getvcpuinfo: + break; + default: + return -EINVAL; + } /* NB: the pluggable scheduler code needs to take care * of locking by itself. */ diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index 7a56b3f..6f429ec 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -338,24 +338,57 @@ DEFINE_XEN_GUEST_HANDLE(xen_domctl_max_vcpus_t); #define XEN_SCHEDULER_ARINC653 7 #define XEN_SCHEDULER_RTDS 8 -/* Set or get info? */ +typedef struct xen_domctl_sched_credit { + uint16_t weight; + uint16_t cap; +} xen_domctl_sched_credit_t; + +typedef struct xen_domctl_sched_credit2 { + uint16_t weight; +} xen_domctl_sched_credit2_t; + +typedef struct xen_domctl_sched_rtds { + uint32_t period; + uint32_t budget; +} xen_domctl_sched_rtds_t; + +typedef struct xen_domctl_schedparam_vcpu { + union { + xen_domctl_sched_credit_t credit; + xen_domctl_sched_credit2_t credit2; + xen_domctl_sched_rtds_t rtds; + } s; + uint16_t vcpuid; + uint16_t padding[3]; +} xen_domctl_schedparam_vcpu_t; +DEFINE_XEN_GUEST_HANDLE(xen_domctl_schedparam_vcpu_t); + +/* Set or get info? + * For schedulers supporting per-vcpu settings (e.g., RTDS): + * using XEN_DOMCTL_SCHEDOP_putinfo sets params for all vcpus; + * using XEN_DOMCTL_SCHEDOP_getinfo gets default params; + * using XEN_DOMCTL_SCHEDOP_put(get)vcpuinfo sets (gets) params of vcpus; + * For schedulers not supporting per-vcpu settings: + * using XEN_DOMCTL_SCHEDOP_putinfo sets params for all vcpus; + * using XEN_DOMCTL_SCHEDOP_getinfo gets domain-wise params; + * using XEN_DOMCTL_SCHEDOP_put(get)vcpuinfo returns error code; + */ #define XEN_DOMCTL_SCHEDOP_putinfo 0 #define XEN_DOMCTL_SCHEDOP_getinfo 1 +#define XEN_DOMCTL_SCHEDOP_putvcpuinfo 2 +#define XEN_DOMCTL_SCHEDOP_getvcpuinfo 3 struct xen_domctl_scheduler_op { uint32_t sched_id; /* XEN_SCHEDULER_* */ uint32_t cmd; /* XEN_DOMCTL_SCHEDOP_* */ union { - struct xen_domctl_sched_credit { - uint16_t weight; - uint16_t cap; - } credit; - struct xen_domctl_sched_credit2 { - uint16_t weight; - } credit2; - struct xen_domctl_sched_rtds { - uint32_t period; - uint32_t budget; - } rtds; + xen_domctl_sched_credit_t credit; + xen_domctl_sched_credit2_t credit2; + xen_domctl_sched_rtds_t rtds; + struct { + XEN_GUEST_HANDLE_64(xen_domctl_schedparam_vcpu_t) vcpus; + uint32_t nr_vcpus; + uint32_t vcpu_index; + } v; } u; }; typedef struct xen_domctl_scheduler_op xen_domctl_scheduler_op_t;