From patchwork Fri Apr 8 01:24:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 8779591 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 6573FC0553 for ; Fri, 8 Apr 2016 01:26:37 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 16AA5201CD for ; Fri, 8 Apr 2016 01:26:36 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C17B2201ED for ; Fri, 8 Apr 2016 01:26:34 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aoLAB-0008HP-Og; Fri, 08 Apr 2016 01:24:43 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1aoLAA-0008GN-BP for xen-devel@lists.xenproject.org; Fri, 08 Apr 2016 01:24:42 +0000 Received: from [85.158.137.68] by server-15.bemta-3.messagelabs.com id B1/CD-03172-95807075; Fri, 08 Apr 2016 01:24:41 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrAIsWRWlGSWpSXmKPExsXiVRvkrBvBwR5 usOK6vMX3LZOZHBg9Dn+4whLAGMWamZeUX5HAmnHuWXjBdf+Kx/8/MzUwXrLtYuTiEBKYyihx 4+JmRhCHRWANq8SqQxuZQBwJgUusEi9nfANyOIGcGIk3G3azQdjlEpP3TwWzhQRUJG5uX8UEY S9mkjhwKAbEFhbQkzhy9Ac7hJ0k8eJbBzOIzSZgIPFmx15WEFtEQEni3qrJYL3MAtESKx82g9 WzCKhKvL26Bsjm4OAVsJNouQDWyingIDFxw1OoVfYS27pPgNmiAnISKy+3gI3kFRCUODnzCQt IK7OApsT6XfoQ0+Ultr+dwzyBUWQWkqpZCFWzkFQtYGRexahRnFpUllqka2iql1SUmZ5RkpuY maNraGCsl5taXJyYnpqTmFSsl5yfu4kRGPoMQLCDcc12z0OMkhxMSqK8V96whQvxJeWnVGYkF mfEF5XmpBYfYpTh4FCS4P3Ixh4uJFiUmp5akZaZA4xCmLQEB4+SCK81O1Cat7ggMbc4Mx0idY pRl2PL1HtrmYRY8vLzUqXEeV+DzBAAKcoozYMbAUsIlxhlpYR5GYGOEuIpSC3KzSxBlX/FKM7 BqCTMKwOyiiczrwRu0yugI5iAjrjAzwZyREkiQkqqgXFdz4Hnm9m2sS3pefBOu+iq2vE/O7f/ OyNloha3QG51hOvLT2r8h56FvhKv7Nx5+c+/1dOqT89gs9nxb9LbJf5fda7tu1DzsDp92pLvY mIX6vLD+Ve1SispFFjlqESuFI2eGvHqi1NFwGbXidEJv8VCGrnqpE4qNRSe4U/Z5r+95bzL56 fs39mVWIozEg21mIuKEwGeRK4sAwMAAA== X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-2.tower-31.messagelabs.com!1460078680!33628841!1 X-Originating-IP: [74.125.82.67] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 8.28; banners=-,-,- X-VirusChecked: Checked Received: (qmail 13734 invoked from network); 8 Apr 2016 01:24:40 -0000 Received: from mail-wm0-f67.google.com (HELO mail-wm0-f67.google.com) (74.125.82.67) by server-2.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 8 Apr 2016 01:24:40 -0000 Received: by mail-wm0-f67.google.com with SMTP id n3so762993wmn.1 for ; Thu, 07 Apr 2016 18:24:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=ChCEQwDiYicr3puJagD6PqbMgiGWsAGQ06L7Wd9BbiQ=; b=kBkncBsKSCJ8ArtanYVrjYkSpZQGrWJfz9/IbOOR+ylP7gYNmMGZwLzKdAmK38ZUpf xzmi10RTCyAITTbu6mPjwnEhqLjqw76zWKL2gcUiNiTZVSVNIvpINYbWYCiVjJrIump+ TPn0n/sU43BIpoBRX2/3YP5JyZTmPWB2ZrhnGInUF3JIZyQG1eTWo0BySRxrWCO9/oE5 HwFfWPQtx5bsvFv59wgWyEWKGSEd0sIY2SVFO2rkUmrp4MxHbb2Pbluh8nSCRhSoIrUr xD4/KCN4r+81bAjRgQtPYCXeBlfsOmDuvPv+i4OX/XRZanrjSuKRk4ZDDGxQoEPvEG6g y/2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=ChCEQwDiYicr3puJagD6PqbMgiGWsAGQ06L7Wd9BbiQ=; b=gt9xQHAeaFVU34cyiWIYKjhgYWvyCLEdZQOtM2tKdsxiOrMvzBfAf2vtFrqqJ0nxBl IEKs2B5OwgJzj1y2ztr76+BK0hohWY9ZlMwWZsVPmjAQLMdNlymQiYVvEU19sGtHEx4R 1MQgYIXMQhkTsm2LGjiQMAxsEI0cga02FN5FFsWeNdtl9ZI/MGQxs+CqNgLt1VpaZ6d2 9cp2dk3yonK8JdVCiB73rOrSohZA+mFTOtLOgw7pK7Tfroz2gW+pk9ys3By03sorIc7M eXQarZYegTdrcre1lhHx9k452NklL6y5cQPyjr/NlDKVZ2Z49DrZj0z3R/OOgCsPX8WN I7jg== X-Gm-Message-State: AD7BkJLc195Tfz8Ml8d/DNYjm8qvyt2iDbdj09n20qwd8KYZNoRxjM+4KDBSEYhnQK/WFg== X-Received: by 10.28.173.71 with SMTP id w68mr612824wme.88.1460078680188; Thu, 07 Apr 2016 18:24:40 -0700 (PDT) Received: from Solace.fritz.box (net-37-116-155-252.cust.vodafonedsl.it. [37.116.155.252]) by smtp.gmail.com with ESMTPSA id l135sm575735wmb.13.2016.04.07.18.24.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Apr 2016 18:24:39 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 08 Apr 2016 03:24:37 +0200 Message-ID: <20160408012437.10762.4282.stgit@Solace.fritz.box> In-Reply-To: <20160408011204.10762.14241.stgit@Solace.fritz.box> References: <20160408011204.10762.14241.stgit@Solace.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap , Andrew Cooper Subject: [Xen-devel] [PATCH v3 10/11] xen: sched: privde some scratch space for not putting cpumasks on stack X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP directly, from schedule.c, for any scheduler that needs it to use it. In fact, Credit1 and RTDS needs this already. Credit2 is also going to need it, for supporting hard affinity (which is, typically, what requires a lot of cpumask manipulations, inside various functions). Therefore, let's define the scratch space at a broader scope, to limit code duplication in handling it. Signed-off-by: Dario Faggioli Reviewed-by: George Dunlap --- Cc: Andrew Cooper --- Changes from v1: * scratch space for cpumask is not "global", and defined in schedule.c, as suggested during review. --- xen/common/sched_credit.c | 34 ++++++------------------- xen/common/sched_credit2.c | 1 + xen/common/sched_rt.c | 59 +++----------------------------------------- xen/common/schedule.c | 8 ++++++ xen/include/xen/sched-if.h | 4 +++ 5 files changed, 25 insertions(+), 81 deletions(-) diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 490f10b..bc36837 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -171,20 +171,9 @@ struct csched_pcpu { struct timer ticker; unsigned int tick; unsigned int idle_bias; - /* Store this here to avoid having too many cpumask_var_t-s on stack */ - cpumask_var_t balance_mask; }; /* - * Convenience macro for accessing the per-PCPU cpumask we need for - * implementing the two steps (soft and hard affinity) balancing logic. - * It is stored in csched_pcpu so that serialization is not an issue, - * as there is a csched_pcpu for each PCPU, and we always hold the - * runqueue lock for the proper PCPU when using this. - */ -#define csched_balance_mask(c) (CSCHED_PCPU(c)->balance_mask) - -/* * Virtual CPU */ struct csched_vcpu { @@ -416,10 +405,10 @@ static inline void __runq_tickle(struct csched_vcpu *new) /* Are there idlers suitable for new (for this balance step)? */ csched_balance_cpumask(new->vcpu, balance_step, - csched_balance_mask(cpu)); - cpumask_and(csched_balance_mask(cpu), - csched_balance_mask(cpu), &idle_mask); - new_idlers_empty = cpumask_empty(csched_balance_mask(cpu)); + cpumask_scratch_cpu(cpu)); + cpumask_and(cpumask_scratch_cpu(cpu), + cpumask_scratch_cpu(cpu), &idle_mask); + new_idlers_empty = cpumask_empty(cpumask_scratch_cpu(cpu)); /* * Let's not be too harsh! If there aren't idlers suitable @@ -445,8 +434,8 @@ static inline void __runq_tickle(struct csched_vcpu *new) if ( new_idlers_empty && new->pri > cur->pri ) { csched_balance_cpumask(cur->vcpu, balance_step, - csched_balance_mask(cpu)); - if ( cpumask_intersects(csched_balance_mask(cpu), + cpumask_scratch_cpu(cpu)); + if ( cpumask_intersects(cpumask_scratch_cpu(cpu), &idle_mask) ) { SCHED_VCPU_STAT_CRANK(cur, kicked_away); @@ -519,7 +508,6 @@ csched_free_pdata(const struct scheduler *ops, void *pcpu, int cpu) spin_unlock_irqrestore(&prv->lock, flags); - free_cpumask_var(spc->balance_mask); xfree(spc); } @@ -533,12 +521,6 @@ csched_alloc_pdata(const struct scheduler *ops, int cpu) if ( spc == NULL ) return ERR_PTR(-ENOMEM); - if ( !alloc_cpumask_var(&spc->balance_mask) ) - { - xfree(spc); - return ERR_PTR(-ENOMEM); - } - return spc; } @@ -1592,9 +1574,9 @@ csched_runq_steal(int peer_cpu, int cpu, int pri, int balance_step) && !__vcpu_has_soft_affinity(vc, vc->cpu_hard_affinity) ) continue; - csched_balance_cpumask(vc, balance_step, csched_balance_mask(cpu)); + csched_balance_cpumask(vc, balance_step, cpumask_scratch_cpu(cpu)); if ( __csched_vcpu_is_migrateable(vc, cpu, - csched_balance_mask(cpu)) ) + cpumask_scratch_cpu(cpu)) ) { /* We got a candidate. Grab it! */ TRACE_3D(TRC_CSCHED_STOLEN_VCPU, peer_cpu, diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index d44cc3d..8617c9b 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -2259,6 +2259,7 @@ csched2_init(struct scheduler *ops) if ( prv == NULL ) return -ENOMEM; ops->sched_data = prv; + spin_lock_init(&prv->lock); INIT_LIST_HEAD(&prv->sdom); diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c index 3bb8c71..673fc92 100644 --- a/xen/common/sched_rt.c +++ b/xen/common/sched_rt.c @@ -155,24 +155,6 @@ #define TRC_RTDS_BUDGET_REPLENISH TRC_SCHED_CLASS_EVT(RTDS, 4) #define TRC_RTDS_SCHED_TASKLET TRC_SCHED_CLASS_EVT(RTDS, 5) - /* - * Useful to avoid too many cpumask_var_t on the stack. - */ -static cpumask_var_t *_cpumask_scratch; -#define cpumask_scratch _cpumask_scratch[smp_processor_id()] - -/* - * We want to only allocate the _cpumask_scratch array the first time an - * instance of this scheduler is used, and avoid reallocating and leaking - * the old one when more instance are activated inside new cpupools. We - * also want to get rid of it when the last instance is de-inited. - * - * So we (sort of) reference count the number of initialized instances. This - * does not need to happen via atomic_t refcounters, as it only happens either - * during boot, or under the protection of the cpupool_lock spinlock. - */ -static unsigned int nr_rt_ops; - static void repl_timer_handler(void *data); /* @@ -301,12 +283,11 @@ rt_dump_vcpu(const struct scheduler *ops, const struct rt_vcpu *svc) /* * We can't just use 'cpumask_scratch' because the dumping can * happen from a pCPU outside of this scheduler's cpupool, and - * hence it's not right to use the pCPU's scratch mask (which - * may even not exist!). On the other hand, it is safe to use - * svc->vcpu->processor's own scratch space, since we hold the - * runqueue lock. + * hence it's not right to use its pCPU's scratch mask. + * On the other hand, it is safe to use svc->vcpu->processor's + * own scratch space, since we hold the runqueue lock. */ - mask = _cpumask_scratch[svc->vcpu->processor]; + mask = cpumask_scratch_cpu(svc->vcpu->processor); cpupool_mask = cpupool_domain_cpumask(svc->vcpu->domain); cpumask_and(mask, cpupool_mask, svc->vcpu->cpu_hard_affinity); @@ -609,16 +590,6 @@ rt_init(struct scheduler *ops) if ( prv == NULL ) return -ENOMEM; - ASSERT( _cpumask_scratch == NULL || nr_rt_ops > 0 ); - - if ( !_cpumask_scratch ) - { - _cpumask_scratch = xmalloc_array(cpumask_var_t, nr_cpu_ids); - if ( !_cpumask_scratch ) - goto no_mem; - } - nr_rt_ops++; - spin_lock_init(&prv->lock); INIT_LIST_HEAD(&prv->sdom); INIT_LIST_HEAD(&prv->runq); @@ -636,10 +607,6 @@ rt_init(struct scheduler *ops) prv->repl_timer = NULL; return 0; - - no_mem: - xfree(prv); - return -ENOMEM; } static void @@ -647,14 +614,6 @@ rt_deinit(struct scheduler *ops) { struct rt_private *prv = rt_priv(ops); - ASSERT( _cpumask_scratch && nr_rt_ops > 0 ); - - if ( (--nr_rt_ops) == 0 ) - { - xfree(_cpumask_scratch); - _cpumask_scratch = NULL; - } - kill_timer(prv->repl_timer); xfree(prv->repl_timer); @@ -718,9 +677,6 @@ rt_alloc_pdata(const struct scheduler *ops, int cpu) { struct rt_private *prv = rt_priv(ops); - if ( !alloc_cpumask_var(&_cpumask_scratch[cpu]) ) - return ERR_PTR(-ENOMEM); - if ( prv->repl_timer == NULL ) { /* Allocate the timer on the first cpu of this pool. */ @@ -735,12 +691,6 @@ rt_alloc_pdata(const struct scheduler *ops, int cpu) return NULL; } -static void -rt_free_pdata(const struct scheduler *ops, void *pcpu, int cpu) -{ - free_cpumask_var(_cpumask_scratch[cpu]); -} - static void * rt_alloc_domdata(const struct scheduler *ops, struct domain *dom) { @@ -1484,7 +1434,6 @@ static const struct scheduler sched_rtds_def = { .init = rt_init, .deinit = rt_deinit, .alloc_pdata = rt_alloc_pdata, - .free_pdata = rt_free_pdata, .init_pdata = rt_init_pdata, .switch_sched = rt_switch_sched, .alloc_domdata = rt_alloc_domdata, diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 5559aa1..922b035 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -65,6 +65,14 @@ static void poll_timer_fn(void *data); DEFINE_PER_CPU(struct schedule_data, schedule_data); DEFINE_PER_CPU(struct scheduler *, scheduler); +/* + * Scratch space, for avoiding having too many cpumask_var_t on the stack. + * Properly serializing access, if necessary, is responsibility of each + * scheduler (typically, one can expect this to be protected by the per pCPU + * or per runqueue lock). + */ +DEFINE_PER_CPU(cpumask_t, cpumask_scratch); + extern const struct scheduler *__start_schedulers_array[], *__end_schedulers_array[]; #define NUM_SCHEDULERS (__end_schedulers_array - __start_schedulers_array) #define schedulers __start_schedulers_array diff --git a/xen/include/xen/sched-if.h b/xen/include/xen/sched-if.h index 9cebe41..1db7c8d 100644 --- a/xen/include/xen/sched-if.h +++ b/xen/include/xen/sched-if.h @@ -47,6 +47,10 @@ DECLARE_PER_CPU(struct schedule_data, schedule_data); DECLARE_PER_CPU(struct scheduler *, scheduler); DECLARE_PER_CPU(struct cpupool *, cpupool); +DECLARE_PER_CPU(cpumask_t, cpumask_scratch); +#define cpumask_scratch (&this_cpu(cpumask_scratch)) +#define cpumask_scratch_cpu(c) (&per_cpu(cpumask_scratch, c)) + #define sched_lock(kind, param, cpu, irq, arg...) \ static inline spinlock_t *kind##_schedule_lock##irq(param EXTRA_TYPE(arg)) \ { \