From patchwork Fri Mar 17 18:19:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9631163 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B61E9602D6 for ; Fri, 17 Mar 2017 18:21:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B16FF28533 for ; Fri, 17 Mar 2017 18:21:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A4ECD28604; Fri, 17 Mar 2017 18:21:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_SPAM,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DC2FF2856B for ; Fri, 17 Mar 2017 18:21:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cowTW-0002f5-K9; Fri, 17 Mar 2017 18:19:42 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cowTV-0002ee-6t for xen-devel@lists.xenproject.org; Fri, 17 Mar 2017 18:19:41 +0000 Received: from [85.158.137.68] by server-12.bemta-3.messagelabs.com id 60/02-12861-CB82CC85; Fri, 17 Mar 2017 18:19:40 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrPIsWRWlGSWpSXmKPExsVyMbThoO5ujTM RBh0tBhbft0xmcmD0OPzhCksAYxRrZl5SfkUCa0bvmT6WgmmSFcsu32NqYDwj0sXIxSEkMItR ovN8LzuIwyKwhlXiUMtSFhBHQuASq8SESwuBHE4gJ0bi/4u/zBB2tcTT9wfYQWwhARWJm9tXM UGM+s0o8efwVUaQhLCAnsSRoz/YIewIiY5PN8HibAIGEm927GUFsUUElCTurZrMBGIzC+RLrJ rQBxZnEVCVePNjE5jNK+AncenqfLDFnAL+Eq9v7WOGWOwncWD9Y7DjRAXkJFZeboGqF5Q4OfM JUJwDaKamxPpd+hDj5SW2v53DPIFRZBaSqlkIVbOQVC1gZF7FqFGcWlSWWqRrZKaXVJSZnlGS m5iZo2toYKyXm1pcnJiempOYVKyXnJ+7iREYAfUMDIw7GBv2+h1ilORgUhLlvX7uRIQQX1J+S mVGYnFGfFFpTmrxIUYZDg4lCd4j94BygkWp6akVaZk5wFiESUtw8CiJ8B4ASfMWFyTmFmemQ6 ROMVpyPDi16w0Tx62GPUDyU//hN0xCLHn5ealS4ry7QBoEQBoySvPgxsHSxSVGWSlhXkYGBgY hnoLUotzMElT5V4ziHIxKwrz+IFN4MvNK4La+AjqICeigxJ9HQA4qSURISTUwVuVJl33XP/Hq +AuZRLkNGl/N06dy/zix7p1kY5HDp6lGh16ebe/uCv7O+y/g8lS12NK5i9ZWeph7bNRSXuO/q lK75Zmdc+/V0BPXpG7JiBcfXnYnscXuAU/ZRN+3YW3sXT6BdxwEmv8zlK2TiHqbtTXq63fHjF /zbp26FrHy/aU/P3uMV0seVWIpzkg01GIuKk4EACEMASESAwAA X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-6.tower-31.messagelabs.com!1489774779!64501375!1 X-Originating-IP: [209.85.128.193] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 3295 invoked from network); 17 Mar 2017 18:19:39 -0000 Received: from mail-wr0-f193.google.com (HELO mail-wr0-f193.google.com) (209.85.128.193) by server-6.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 17 Mar 2017 18:19:39 -0000 Received: by mail-wr0-f193.google.com with SMTP id u108so10520686wrb.2 for ; Fri, 17 Mar 2017 11:19:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=XAUvoZiGPYkGKBsUuf+AQ1N4pUzT0pTJx+Y42wVboc8=; b=dr+ALk7jWPDoLa4P0+Isrxwb9GzBq34Thrmi9GCtZTtpB4u2u/1O126U3AkP3kGjCI WWo01pa2savI72c27M7pgSoRxVGbMx5Aqq+18Q7zlBKOiEmoLg1IkAYaQ15wzcufaVVH v06Cov8qz0awsPfluWCxvn8VKWoeD7/yiidkcDKL23c5zZ16iIAZh81phq/+nogTVh/t LRPPelsjlBp4NrDhHTjTZNDLVQagLLTYIrm2BNNQHJGCSW1iR7XOxDHoZqlXtY+EdFGm LfuIjOnkuwrE8mrmKA/+bbgtHmhi5z2cCaI852u3tNYNFhc5ooVUEtfzYeIPqEYfNOJV Pctg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=XAUvoZiGPYkGKBsUuf+AQ1N4pUzT0pTJx+Y42wVboc8=; b=K+Ps+Jn2ulDFkaTonPhNPXshNlzuhPsxUPWNNIR9LQEOmHCYhZscrjUc4GWc8jj6zk 6jtD6REBkvMJXm31IJItBdc5bEqnyaPpEzx2jo9y6FyL4HnYtCOhDdOeDE+RHjkdNac0 eH0b+6+k/iHqNReDGXKpfLh8U9glRYmlrw6xmeqSS0eTXDQqClAw7PMWmdrC5uW1TWYd NwL7hIHqIN2+izB4YY6ZHK0982RtINrxZCP2GFn5N0SsoG+moUs0ODl9ahqzuq6Mhzwm I8I6ih7iJLlundgArg3mUFODOxTFVgMDjT7ox4TkHbcG0CZjOWz+ONvSB3OXOv78KGjd y57g== X-Gm-Message-State: AFeK/H3f/mZg18gy2ai/OVuM0/cJAxCKMzkRurxAeV63px+sQqDfS1Fspx/bPRdBI0P8cw== X-Received: by 10.223.163.21 with SMTP id c21mr13943122wrb.115.1489774779385; Fri, 17 Mar 2017 11:19:39 -0700 (PDT) Received: from Palanthas.fritz.box ([80.66.223.93]) by smtp.gmail.com with ESMTPSA id b82sm3506191wmh.4.2017.03.17.11.19.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2017 11:19:38 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 17 Mar 2017 19:19:37 +0100 Message-ID: <148977477739.22479.7043154591081622447.stgit@Palanthas.fritz.box> In-Reply-To: <148977465656.22479.5382577625088079334.stgit@Palanthas.fritz.box> References: <148977465656.22479.5382577625088079334.stgit@Palanthas.fritz.box> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: Juergen Gross , George Dunlap , Jan Beulich Subject: [Xen-devel] [PATCH v2 1/2] xen: sched: don't call hooks of the wrong scheduler via VCPU2OP X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Within context_saved(), we call the context_saved hook, and we use VCPU2OP() to determine from what scheduler. VCPU2OP uses DOM2OP, which uses d->cpupool, which is NULL when d is the idle domain. And in that case, DOM2OP just returns ops, the scheduler of cpupool0. Therefore, if: - cpupool0's scheduler defines context_saved (like Credit2 and RTDS do), - we are not in cpupool0 (i.e., our scheduler is not ops), - we are context switching from idle, we call VCPU2OP(idle_vcpu), which means DOM2OP(idle->cpupool), which is ops. Therefore, we both: - check if context_saved is defined in the wrong scheduler; - if yes, call the wrong one. When using Credit2 at boot, and also Credit2 in the other cpupool, this is wrong but innocuous, because it only involves the idle vcpus. When using Credit2 at boot, and Credit1 in the other cpupool, this is *totally* wrong, and it's by chance it does not explode! When using Credit2 and other schedulers I'm developping, I hit the following assert (in sched_credit2.c, on a CPU inside a cpupool that does not use Credit2): csched2_context_saved() { ... ASSERT(!vcpu_on_runq(svc)); ... } Fix this by dealing explicitly, in VCPU2OP, with idle vcpus, returning the scheduler of the pCPU they (always) run on. While there, rename VCPU2OP itself to something that makes it easier to understand what it does. Signed-off-by: Dario Faggioli Reviewed-by: Juergen Gross Reviewed-by: Juergen Gross Reviewed-by: George Dunlap --- Cc: George Dunlap Cc: Juergen Gross Cc: Jan Beulich --- Changes from v1: - refactored according to review comments. Added code comments and an ASSERT(). --- Cc-ing Jan, as this should be backported at least to 4.8, but, IMO, as back as possible. --- xen/common/schedule.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 223a120..d344b7c 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -78,7 +78,27 @@ static struct scheduler __read_mostly ops; : (typeof((opsptr)->fn(opsptr, ##__VA_ARGS__)))0 ) #define DOM2OP(_d) (((_d)->cpupool == NULL) ? &ops : ((_d)->cpupool->sched)) -#define VCPU2OP(_v) (DOM2OP((_v)->domain)) +static inline struct scheduler *VCPU2OP(const struct vcpu *v) +{ + struct domain *d = v->domain; + + if ( likely(d->cpupool != NULL) ) + return d->cpupool->sched; + + /* + * If d->cpupool is NULL, this is a vCPU of the idle domain. And this + * case is special because the idle domain does not really belong to + * a cpupool and, hence, doesn't really have a scheduler). In fact, its + * vCPUs (may) run on pCPUs which are in different pools, with different + * schedulers. + * + * What we want, in this case, is the scheduler of the pCPU where this + * particular idle vCPU is running. And, since v->processor never changes + * for idle vCPUs, it is safe to use it, with no locks, to figure that out. + */ + ASSERT(is_idle_domain(d)); + return per_cpu(scheduler, v->processor); +} #define VCPU2ONLINE(_v) cpupool_domain_cpumask((_v)->domain) static inline void trace_runstate_change(struct vcpu *v, int new_state)