From patchwork Thu Feb 4 22:50:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chong Li X-Patchwork-Id: 8228811 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 5B48ABEEE5 for ; Thu, 4 Feb 2016 22:53:51 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0452B201CD for ; Thu, 4 Feb 2016 22:53:50 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9B8DF20390 for ; Thu, 4 Feb 2016 22:53:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aRSju-0000Jy-43; Thu, 04 Feb 2016 22:51:02 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aRSjs-0000JH-SH for xen-devel@lists.xen.org; Thu, 04 Feb 2016 22:51:01 +0000 Received: from [85.158.139.211] by server-13.bemta-5.messagelabs.com id 25/F5-09478-4D5D3B65; Thu, 04 Feb 2016 22:51:00 +0000 X-Env-Sender: lichong659@gmail.com X-Msg-Ref: server-11.tower-206.messagelabs.com!1454626258!20440314!1 X-Originating-IP: [209.85.213.194] X-SpamReason: No, hits=0.3 required=7.0 tests=MAILTO_TO_SPAM_ADDR X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 35463 invoked from network); 4 Feb 2016 22:50:58 -0000 Received: from mail-ig0-f194.google.com (HELO mail-ig0-f194.google.com) (209.85.213.194) by server-11.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 4 Feb 2016 22:50:58 -0000 Received: by mail-ig0-f194.google.com with SMTP id hb3so145152igb.0 for ; Thu, 04 Feb 2016 14:50:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bnUavhuF1qQE+FmAXVjRY9pRNcFMBIJNgtnF50gPA2c=; b=prC6XGX2/Ok8AckxboG99dVpSGczMcybCneVtGSr+n55X7cBWPFJom8/Y4UP2pVxBa Is63MV9bqHhy7rtKNPRyoAbWKIP1zpx8m5pJNzR29kWxwbXlp4Da9yrI6pkwmiZgCEAa cjVJIJioCiL5YQHGLXY0mNt/xpwixtIelACgfg4o5s9xGcZbXsxZFu14tr/eX3jWd8eK thxEloAUTSmPjo9BDeVClU55bnUXsv0q3XjpiWFJ66iVzet/G86BHDUZWUHHvoxMPyFl TTG+GvQscmG/JkDcdmLOFwUi1rdwkYJvG6yGA7SBHDw5ZadCsZFgFutM/gMK6k854MyR FZ6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bnUavhuF1qQE+FmAXVjRY9pRNcFMBIJNgtnF50gPA2c=; b=ZU9/ZP/qsN3x+8pBaQx/9Ac3wrCzrmzhQH471TXv8/FMHaNOfDhGaTVmGbEyys7UWA nPFPypfcA8AsTQrGz08Ya3JD4Qg0ROVxj1ohHY+lXjDXvgkzs8PQZ0YQZMVGmfOLnQpQ 67uJa1dQ/F4yrgLxSZotw0BSDdhWag6xI0wXOkdvL5BVNOY8tcBl2DF7mBF2eRXJGXGX 578/s/3/WQKJRUk6laGPMk4slm+cRFX6dRAY7EMn9rQGSc0J65bMAwSe7a4Cnk2BbrJQ mN2vPbIH/UWMgKXkvaU7/4PUq8THpfqUGNdrLNj5OV1s9aN/PJqpcrVuv80HAK8IKp0S kjlA== X-Gm-Message-State: AG10YOQdy1gMwHmsikgYafVJu78zHnpOCJuLbsvKQnGew6XljV4IWZURUA/UnNCxix5qSg== X-Received: by 10.50.41.68 with SMTP id d4mr6914329igl.31.1454626257854; Thu, 04 Feb 2016 14:50:57 -0800 (PST) Received: from chong-OptiPlex-960.seas.wustl.edu (admin998.cec.wustl.edu. [128.252.20.193]) by smtp.googlemail.com with ESMTPSA id fk5sm5605439igb.6.2016.02.04.14.50.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 04 Feb 2016 14:50:56 -0800 (PST) From: Chong Li To: xen-devel@lists.xen.org Date: Thu, 4 Feb 2016 16:50:43 -0600 Message-Id: <1454626244-5511-4-git-send-email-lichong659@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1454626244-5511-1-git-send-email-lichong659@gmail.com> References: <1454626244-5511-1-git-send-email-lichong659@gmail.com> Cc: Chong Li , wei.liu2@citrix.com, Sisu Xi , george.dunlap@eu.citrix.com, dario.faggioli@citrix.com, ian.jackson@eu.citrix.com, ian.campbell@eu.citrix.com, Meng Xu , lichong659@gmail.com, dgolomb@seas.upenn.edu Subject: [Xen-devel] [PATCH v5 for Xen 4.7 3/4] libxl: enable per-VCPU parameter settings for RTDS scheduler X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add libxl_vcpu_sched_params_get/set and sched_rtds_vcpu_get/set functions to support per-VCPU settings. Signed-off-by: Chong Li Signed-off-by: Meng Xu Signed-off-by: Sisu Xi --- Changes on PATCH v4: 1) Coding style changes Changes on PATCH v3: 1) Add sanity check on vcpuid 2) Add comments on per-domain and per-vcpu functions for libxl users Changes on PATCH v2: 1) New data structure (libxl_vcpu_sched_params and libxl_sched_params) to help per-VCPU settings. 2) sched_rtds_vcpu_get now can return a random subset of the parameters of the VCPUs of a specific domain. CC: CC: CC: CC: CC: CC: CC: CC: --- tools/libxl/libxl.c | 249 ++++++++++++++++++++++++++++++++++++++++---- tools/libxl/libxl.h | 19 ++++ tools/libxl/libxl_types.idl | 16 +++ 3 files changed, 262 insertions(+), 22 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index bd3aac8..ac4a103 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -5770,6 +5770,151 @@ static int sched_credit2_domain_set(libxl__gc *gc, uint32_t domid, return 0; } +static int sched_rtds_validate_params(libxl__gc *gc, int period, + int budget, uint32_t *sdom_period, + uint32_t *sdom_budget) +{ + if (period != LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT) { + if (period < 1) { + LOG(ERROR, "VCPU period is out of range, " + "valid values are larger than or equal to 1"); + return 1; /* error scheduling parameter */ + } + *sdom_period = period; + } + + if (budget != LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT) { + if (budget < 1) { + LOG(ERROR, "VCPU budget is not set or out of range, " + "valid values are larger than or equal to 1"); + return 1; + } + *sdom_budget = budget; + } + + if (budget > period) { + LOG(ERROR, "VCPU budget must be smaller than " + "or equal to VCPU period"); + return 1; + } + + return 0; /* period and budget are valid */ +} + +static int sched_rtds_vcpu_get(libxl__gc *gc, uint32_t domid, + libxl_vcpu_sched_params *scinfo) +{ + uint32_t num_vcpus; + int rc, i; + xc_dominfo_t info; + struct xen_domctl_schedparam_vcpu *vcpus; + + rc = xc_domain_getinfo(CTX->xch, domid, 1, &info); + if (rc < 0) { + LOGE(ERROR, "getting domain info"); + return ERROR_FAIL; + } + + num_vcpus = scinfo->num_vcpus ? scinfo->num_vcpus : + info.max_vcpu_id + 1; + + GCNEW_ARRAY(vcpus, num_vcpus); + + if (scinfo->num_vcpus > 0) + for (i=0; i < num_vcpus; i++) { + if (scinfo->vcpus[i].vcpuid < 0 || + scinfo->vcpus[i].vcpuid > info.max_vcpu_id) { + LOG(ERROR, "VCPU index is out of range, " + "valid values are within range from 0 to %d", + info.max_vcpu_id); + return ERROR_INVAL; + } + vcpus[i].vcpuid = scinfo->vcpus[i].vcpuid; + } else + for (i=0; i < num_vcpus; i++) + vcpus[i].vcpuid = i; + + rc = xc_sched_rtds_vcpu_get(CTX->xch, domid, vcpus, num_vcpus); + if (rc != 0) { + LOGE(ERROR, "getting vcpu sched rtds"); + return ERROR_FAIL; + } + scinfo->sched = LIBXL_SCHEDULER_RTDS; + if (scinfo->num_vcpus == 0) { + scinfo->num_vcpus = num_vcpus; + scinfo->vcpus = libxl__calloc(NOGC, num_vcpus, + sizeof(libxl_sched_params)); + } + for(i = 0; i < num_vcpus; i++) { + scinfo->vcpus[i].period = vcpus[i].s.rtds.period; + scinfo->vcpus[i].budget = vcpus[i].s.rtds.budget; + scinfo->vcpus[i].vcpuid = vcpus[i].vcpuid; + } + + return 0; +} + +static int sched_rtds_vcpu_set(libxl__gc *gc, uint32_t domid, + const libxl_vcpu_sched_params *scinfo) +{ + int rc; + int i; + uint16_t max_vcpuid; + xc_dominfo_t info; + struct xen_domctl_schedparam_vcpu *vcpus; + uint32_t num_vcpus; + + rc = xc_domain_getinfo(CTX->xch, domid, 1, &info); + if (rc < 0) { + LOGE(ERROR, "getting domain info"); + return ERROR_FAIL; + } + max_vcpuid = info.max_vcpu_id; + + if (scinfo->num_vcpus > 0) { + num_vcpus = scinfo->num_vcpus; + GCNEW_ARRAY(vcpus, num_vcpus); + for (i = 0; i < num_vcpus; i++) { + if (scinfo->vcpus[i].vcpuid < 0 || + scinfo->vcpus[i].vcpuid > max_vcpuid) { + LOG(ERROR, "VCPU index is out of range, " + "valid values are within range from 0 to %d", + max_vcpuid); + return ERROR_INVAL; + } + vcpus[i].vcpuid = scinfo->vcpus[i].vcpuid; + + rc = sched_rtds_validate_params(gc, + scinfo->vcpus[i].period, scinfo->vcpus[i].budget, + &vcpus[i].s.rtds.period, &vcpus[i].s.rtds.budget); + if (rc) + return ERROR_INVAL; + } + } else { + num_vcpus = max_vcpuid + 1; + GCNEW_ARRAY(vcpus, num_vcpus); + if (sched_rtds_validate_params(gc, scinfo->vcpus[0].period, + scinfo->vcpus[0].budget, + &vcpus[0].s.rtds.period, + &vcpus[0].s.rtds.budget)) + return ERROR_INVAL; + for (i = 0; i < num_vcpus; i++) { + vcpus[i].vcpuid = i; + vcpus[i].s.rtds.period = scinfo->vcpus[0].period; + vcpus[i].s.rtds.budget = scinfo->vcpus[0].budget; + } + } + + rc = xc_sched_rtds_vcpu_set(CTX->xch, domid, + vcpus, num_vcpus); + if (rc != 0) { + LOGE(ERROR, "setting vcpu sched rtds"); + return ERROR_FAIL; + } + + return rc; +} + static int sched_rtds_domain_get(libxl__gc *gc, uint32_t domid, libxl_domain_sched_params *scinfo) { @@ -5803,29 +5948,9 @@ static int sched_rtds_domain_set(libxl__gc *gc, uint32_t domid, return ERROR_FAIL; } - if (scinfo->period != LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT) { - if (scinfo->period < 1) { - LOG(ERROR, "VCPU period is not set or out of range, " - "valid values are larger than 1"); - return ERROR_INVAL; - } - sdom.period = scinfo->period; - } - - if (scinfo->budget != LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT) { - if (scinfo->budget < 1) { - LOG(ERROR, "VCPU budget is not set or out of range, " - "valid values are larger than 1"); - return ERROR_INVAL; - } - sdom.budget = scinfo->budget; - } - - if (sdom.budget > sdom.period) { - LOG(ERROR, "VCPU budget is larger than VCPU period, " - "VCPU budget should be no larger than VCPU period"); + if (sched_rtds_validate_params(gc, scinfo->period, scinfo->budget, + &sdom.period, &sdom.budget)) return ERROR_INVAL; - } rc = xc_sched_rtds_domain_set(CTX->xch, domid, &sdom); if (rc < 0) { @@ -5836,6 +5961,11 @@ static int sched_rtds_domain_set(libxl__gc *gc, uint32_t domid, return 0; } +/* Set the per-domain scheduling parameters. +* For schedulers that support per-vcpu settings (e.g., RTDS), +* calling *_domain_set functions will set all vcpus with the same +* scheduling parameters. +*/ int libxl_domain_sched_params_set(libxl_ctx *ctx, uint32_t domid, const libxl_domain_sched_params *scinfo) { @@ -5873,6 +6003,47 @@ int libxl_domain_sched_params_set(libxl_ctx *ctx, uint32_t domid, return ret; } +/* Set the per-vcpu scheduling parameters */ +int libxl_vcpu_sched_params_set(libxl_ctx *ctx, uint32_t domid, + const libxl_vcpu_sched_params *scinfo) +{ + GC_INIT(ctx); + libxl_scheduler sched = scinfo->sched; + int ret; + + if (sched == LIBXL_SCHEDULER_UNKNOWN) + sched = libxl__domain_scheduler(gc, domid); + + switch (sched) { + case LIBXL_SCHEDULER_SEDF: + LOG(ERROR, "SEDF scheduler no longer available"); + ret=ERROR_FEATURE_REMOVED; + break; + case LIBXL_SCHEDULER_CREDIT: + case LIBXL_SCHEDULER_CREDIT2: + case LIBXL_SCHEDULER_ARINC653: + LOG(ERROR, "per-VCPU parameter setting " + "not supported for this scheduler"); + ret = ERROR_INVAL; + break; + case LIBXL_SCHEDULER_RTDS: + ret = sched_rtds_vcpu_set(gc, domid, scinfo); + break; + default: + LOG(ERROR, "Unknown scheduler"); + ret = ERROR_INVAL; + break; + } + + GC_FREE; + return ret; +} + +/* Get the per-domain scheduling parameters. +* For schedulers that support per-vcpu settings (e.g., RTDS), +* calling *_domain_get functions will get default scheduling +* parameters. +*/ int libxl_domain_sched_params_get(libxl_ctx *ctx, uint32_t domid, libxl_domain_sched_params *scinfo) { @@ -5907,6 +6078,40 @@ int libxl_domain_sched_params_get(libxl_ctx *ctx, uint32_t domid, return ret; } +/* Get the per-vcpu scheduling parameters */ +int libxl_vcpu_sched_params_get(libxl_ctx *ctx, uint32_t domid, + libxl_vcpu_sched_params *scinfo) +{ + GC_INIT(ctx); + int ret; + + scinfo->sched = libxl__domain_scheduler(gc, domid); + + switch (scinfo->sched) { + case LIBXL_SCHEDULER_SEDF: + LOG(ERROR, "SEDF scheduler no longer available"); + ret=ERROR_FEATURE_REMOVED; + break; + case LIBXL_SCHEDULER_CREDIT: + case LIBXL_SCHEDULER_CREDIT2: + case LIBXL_SCHEDULER_ARINC653: + LOG(ERROR, "per-VCPU parameter getting " + "not supported for this scheduler"); + ret = ERROR_INVAL; + break; + case LIBXL_SCHEDULER_RTDS: + ret = sched_rtds_vcpu_get(gc, domid, scinfo); + break; + default: + LOG(ERROR, "Unknown scheduler"); + ret = ERROR_INVAL; + break; + } + + GC_FREE; + return ret; +} + static int libxl__domain_s3_resume(libxl__gc *gc, int domid) { int rc = 0; diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 6b73848..4ba30d3 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -206,6 +206,18 @@ #define LIBXL_HAVE_DEVICE_MODEL_USER 1 /* + * libxl_vcpu_sched_params is used to store per-vcpu params. + * The 'vcpuid' field specifies the vcpu to be set or read. +*/ +#define LIBXL_HAVE_VCPU_SCHED_PARAMS 1 + +/* + * LIBXL_HAVE_SCHED_RTDS_VCPU_PARAMS indicates RTDS scheduler + * now supports per-vcpu settings. +*/ +#define LIBXL_HAVE_SCHED_RTDS_VCPU_PARAMS 1 + +/* * libxl_domain_build_info has the arm.gic_version field. */ #define LIBXL_HAVE_BUILDINFO_ARM_GIC_VERSION 1 @@ -1647,10 +1659,17 @@ int libxl_sched_credit_params_set(libxl_ctx *ctx, uint32_t poolid, #define LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT -1 #define LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT -1 +/* Per-VCPU parameters*/ +#define LIBXL_SCHED_PARAM_VCPU_INDEX_DEFAULT -1 + int libxl_domain_sched_params_get(libxl_ctx *ctx, uint32_t domid, libxl_domain_sched_params *params); int libxl_domain_sched_params_set(libxl_ctx *ctx, uint32_t domid, const libxl_domain_sched_params *params); +int libxl_vcpu_sched_params_get(libxl_ctx *ctx, uint32_t domid, + libxl_vcpu_sched_params *params); +int libxl_vcpu_sched_params_set(libxl_ctx *ctx, uint32_t domid, + const libxl_vcpu_sched_params *params); int libxl_send_trigger(libxl_ctx *ctx, uint32_t domid, libxl_trigger trigger, uint32_t vcpuid); diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index cf3730f..4e7210e 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -378,6 +378,22 @@ libxl_domain_restore_params = Struct("domain_restore_params", [ ("stream_version", uint32, {'init_val': '1'}), ]) +libxl_sched_params = Struct("sched_params",[ + ("vcpuid", integer, {'init_val': 'LIBXL_SCHED_PARAM_VCPU_INDEX_DEFAULT'}), + ("weight", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT'}), + ("cap", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_CAP_DEFAULT'}), + ("period", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_PERIOD_DEFAULT'}), + ("slice", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_SLICE_DEFAULT'}), + ("latency", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_LATENCY_DEFAULT'}), + ("extratime", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_EXTRATIME_DEFAULT'}), + ("budget", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_BUDGET_DEFAULT'}), + ]) + +libxl_vcpu_sched_params = Struct("vcpu_sched_params",[ + ("sched", libxl_scheduler), + ("vcpus", Array(libxl_sched_params, "num_vcpus")), + ]) + libxl_domain_sched_params = Struct("domain_sched_params",[ ("sched", libxl_scheduler), ("weight", integer, {'init_val': 'LIBXL_DOMAIN_SCHED_PARAM_WEIGHT_DEFAULT'}),