From patchwork Fri Jun 23 10:55:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dario Faggioli X-Patchwork-Id: 9806257 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E4AB460329 for ; Fri, 23 Jun 2017 10:57:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D58D9285EE for ; Fri, 23 Jun 2017 10:57:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA2B228616; Fri, 23 Jun 2017 10:57:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 15EC8285EE for ; Fri, 23 Jun 2017 10:57:28 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dOMFB-0006LY-EF; Fri, 23 Jun 2017 10:55:17 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dOMFA-0006L2-Ta for xen-devel@lists.xenproject.org; Fri, 23 Jun 2017 10:55:17 +0000 Received: from [85.158.137.68] by server-8.bemta-3.messagelabs.com id A7/10-23755-493FC495; Fri, 23 Jun 2017 10:55:16 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmplleJIrShJLcpLzFFi42K5GNpwWHfyZ59 Igx2/xCy+b5nM5MDocfjDFZYAxijWzLyk/IoE1owzNyeyFay0rpi6bT9zA+N0zS5GLg4hgZmM Etdfv2YBcVgE1rBK/Pl+iBnEkRC4xCrRfWY5kMMJ5MRJPJm0hwnCrpTo2LSEFcQWElCRuLl9F RPEqB+MEotvrWYDSQgL6EkcOfqDHcL2kljZ0ALWwCZgIPFmx14wW0RASeLeqslgQ5kFQiXmvD sD1ssioCpxbdMmMJtXwF7i7MLDYHM4BZwkJqzayQSx2FGi5f8LsONEBeQkVl6GmM8rIChxcuY ToHc4gGZqSqzfpQ8xXl5i+9s5zBMYRWYhqZqFUDULSdUCRuZVjBrFqUVlqUW6hpZ6SUWZ6Rkl uYmZObqGBsZ6uanFxYnpqTmJScV6yfm5mxiBEVDPwMC4g/H3cb9DjJIcTEqivLFnfCKF+JLyU yozEosz4otKc1KLDzFqcHAITDg7dzqTFEtefl6qkgTv409AdYJFqempFWmZOcAYhSmV4OBREu Hd/gEozVtckJhbnJkOkTrFaMxx5cq6L0wcUw5s/8IkBDZJSpz3JMgkAZDSjNI8uEGw1HGJUVZ KmJeRgYFBiKcgtSg3swRV/hWjOAejkjBvMcgUnsy8Erh9r4BOYQI6ZcYasFNKEhFSUg2M7paT E50X5/nX/ZO2CRKKOf/uilq91cSw8zGB9ZJzOyVXcy1NnVL3MPDq6fIV+SozZ6QXiklEv712g GP6Uh2zIJVGP6kaD8Mn0SpXLV/9+X9X4q/ptMYa19LcLW/K9LtWC5dr6CwLfFTae4B1enT/20 0nE9YssPlxhMPvcV71NqPcVD6OY05KLMUZiYZazEXFiQAl8Gk2GAMAAA== X-Env-Sender: raistlin.df@gmail.com X-Msg-Ref: server-8.tower-31.messagelabs.com!1498215315!107141192!1 X-Originating-IP: [209.85.128.195] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.19; banners=-,-,- X-VirusChecked: Checked Received: (qmail 3542 invoked from network); 23 Jun 2017 10:55:15 -0000 Received: from mail-wr0-f195.google.com (HELO mail-wr0-f195.google.com) (209.85.128.195) by server-8.tower-31.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 23 Jun 2017 10:55:15 -0000 Received: by mail-wr0-f195.google.com with SMTP id x23so11636594wrb.0 for ; Fri, 23 Jun 2017 03:55:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Vq7CEh+GvjghgL8LbQu1jbTkc+mgkShRaGIf4vYTtG0=; b=W3wMWFCcqToWy7vGSTqNNaez9nKYD35/WAN3jOxYoMITMv+vZ8F1yeLE4vjt412gdG obgvg1MFv0OruDdCsmAquaSvTJnfb9yFGsspPFK+qATuyHxHU9jEgmKVE9i+91N9YE56 awr15cn0K25EPfVBf+hEIWXDZXjc+gyxnLZSyssOihBisFHVgDz5UMzfY6BTBHShI4Zb g3w6mXfchQSye9kxxYjEePcuItunKDK1PAlEmDsgjnIUkIq/Em9PR6q1mspPaNjyit7O siFDfFlIatc10OM7wslGOdLMSChRj5Upvh4eh0eBEjRAPGjpp5dP2EweRRfkmp5Kbj+a ZSKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Vq7CEh+GvjghgL8LbQu1jbTkc+mgkShRaGIf4vYTtG0=; b=V8PjI0Lr15jlcXJjRuLNdDdF25VCZOZ8x3wAaD9hxBEzCNtHVWlIDheIe/1i6OuQ11 9AkC4GOimrpnbvXwcVYvjfsyOSVWGyg1ePg37HqHG2IuKtTzFSR/KbL6uUhazIIHHtNu 9e9kr4rfQogAv6eKsNmh2X9L5idN8FojmAge714XAwv+wzfSU+dpV2nGwuG7yboP+M29 Mq3y8/BdxallHLfbiKqy1l4kNLwFtW/pt/FLW4w82YNfq7jkS/UJnBtySHZXNoLg7e4T CJJeaWX4MSD55EjbQcsEGOtgTaM0jbwOObv7MXg+fRXtqDZh4ph/nQvJUoEYUP99MCki CBVA== X-Gm-Message-State: AKS2vOwkVVNeOz2hHHtFAj8Z8wzuWxXCfzmr+6FCr7+ictOjOW0kmqNb AEcWDtfhW98J3qs1 X-Received: by 10.223.177.201 with SMTP id r9mr5491169wra.36.1498215314601; Fri, 23 Jun 2017 03:55:14 -0700 (PDT) Received: from [192.168.0.31] ([80.66.223.81]) by smtp.gmail.com with ESMTPSA id j196sm1975975wmd.32.2017.06.23.03.55.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jun 2017 03:55:13 -0700 (PDT) From: Dario Faggioli To: xen-devel@lists.xenproject.org Date: Fri, 23 Jun 2017 12:55:12 +0200 Message-ID: <149821531270.5914.329757532058629047.stgit@Solace> In-Reply-To: <149821475587.5914.12193327340105859241.stgit@Solace> References: <149821475587.5914.12193327340105859241.stgit@Solace> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: George Dunlap , Anshul Makkar Subject: [Xen-devel] [PATCH 4/6] xen: credit2: rearrange members of control structures X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP With the aim of improving memory size and layout, and at the same time trying to put related fields reside in the same cacheline. Here's a summary of the output of `pahole`, with and without this patch, for the affected data structures. csched2_runqueue_data: * Before: size: 216, cachelines: 4, members: 14 sum members: 208, holes: 2, sum holes: 8 last cacheline: 24 bytes * After: size: 208, cachelines: 4, members: 14 last cacheline: 16 bytes csched2_private: * Before: size: 120, cachelines: 2, members: 8 sum members: 112, holes: 1, sum holes: 4 padding: 4 last cacheline: 56 bytes * After: size: 112, cachelines: 2, members: 8 last cacheline: 48 bytes csched2_vcpu: * Before: size: 112, cachelines: 2, members: 14 sum members: 108, holes: 1, sum holes: 4 last cacheline: 48 bytes * After: size: 112, cachelines: 2, members: 14 padding: 4 last cacheline: 48 bytes While there, improve the wording, style and alignment of comments too. Signed-off-by: Dario Faggioli Acked-by: George Dunlap --- Cc: George Dunlap Cc: Anshul Makkar --- xen/common/sched_credit2.c | 102 ++++++++++++++++++++++---------------------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 15862f2..9814072 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -355,40 +355,41 @@ custom_param("credit2_runqueue", parse_credit2_runqueue); * Per-runqueue data */ struct csched2_runqueue_data { - int id; - - spinlock_t lock; /* Lock for this runqueue. */ - cpumask_t active; /* CPUs enabled for this runqueue */ - - struct list_head runq; /* Ordered list of runnable vms */ - struct list_head svc; /* List of all vcpus assigned to this runqueue */ - unsigned int max_weight; - unsigned int pick_bias;/* Last CPU we picked. Start from it next time */ - - cpumask_t idle, /* Currently idle pcpus */ - smt_idle, /* Fully idle-and-untickled cores (see below) */ - tickled; /* Have been asked to go through schedule */ - int load; /* Instantaneous load: Length of queue + num non-idle threads */ - s_time_t load_last_update; /* Last time average was updated */ - s_time_t avgload; /* Decaying queue load */ - s_time_t b_avgload; /* Decaying queue load modified by balancing */ + spinlock_t lock; /* Lock for this runqueue */ + + struct list_head runq; /* Ordered list of runnable vms */ + int id; /* ID of this runqueue (-1 if invalid) */ + + int load; /* Instantaneous load (num of non-idle vcpus) */ + s_time_t load_last_update; /* Last time average was updated */ + s_time_t avgload; /* Decaying queue load */ + s_time_t b_avgload; /* Decaying queue load modified by balancing */ + + cpumask_t active, /* CPUs enabled for this runqueue */ + smt_idle, /* Fully idle-and-untickled cores (see below) */ + tickled, /* Have been asked to go through schedule */ + idle; /* Currently idle pcpus */ + + struct list_head svc; /* List of all vcpus assigned to the runqueue */ + unsigned int max_weight; /* Max weight of the vcpus in this runqueue */ + unsigned int pick_bias; /* Last picked pcpu. Start from it next time */ }; /* * System-wide private data */ struct csched2_private { - rwlock_t lock; - cpumask_t initialized; /* CPU is initialized for this pool */ - - struct list_head sdom; /* Used mostly for dump keyhandler. */ + rwlock_t lock; /* Private scheduler lock */ - cpumask_t active_queues; /* Queues which may have active cpus */ - struct csched2_runqueue_data *rqd; + unsigned int load_precision_shift; /* Precision of load calculations */ + unsigned int load_window_shift; /* Lenght of load decaying window */ + unsigned int ratelimit_us; /* Rate limiting for this scheduler */ + + cpumask_t active_queues; /* Runqueues with (maybe) active cpus */ + struct csched2_runqueue_data *rqd; /* Data of the various runqueues */ - unsigned int load_precision_shift; - unsigned int load_window_shift; - unsigned ratelimit_us; /* each cpupool can have its own ratelimit */ + cpumask_t initialized; /* CPUs part of this scheduler */ + struct list_head sdom; /* List of domains (for debug key) */ }; /* @@ -403,37 +404,36 @@ static DEFINE_PER_CPU(int, runq_map); * Virtual CPU */ struct csched2_vcpu { - struct list_head rqd_elem; /* On the runqueue data list */ - struct list_head runq_elem; /* On the runqueue */ - struct csched2_runqueue_data *rqd; /* Up-pointer to the runqueue */ - - /* Up-pointers */ - struct csched2_dom *sdom; - struct vcpu *vcpu; - - unsigned int weight; - unsigned int residual; - - int credit; - s_time_t start_time; /* When we were scheduled (used for credit) */ - unsigned flags; /* 16 bits doesn't seem to play well with clear_bit() */ - int tickled_cpu; /* cpu tickled for picking us up (-1 if none) */ - - /* Individual contribution to load */ - s_time_t load_last_update; /* Last time average was updated */ - s_time_t avgload; /* Decaying queue load */ - - struct csched2_runqueue_data *migrate_rqd; /* Pre-determined rqd to which to migrate */ + struct list_head rqd_elem; /* On csched2_runqueue_data's svc list */ + struct csched2_runqueue_data *rqd; /* Up-pointer to the runqueue */ + + int credit; /* Current amount of credit */ + unsigned int weight; /* Weight of this vcpu */ + unsigned int residual; /* Reminder of div(max_weight/weight) */ + unsigned flags; /* Status flags (16 bits would be ok, */ + /* but clear_bit() does not like that) */ + s_time_t start_time; /* Time we were scheduled (for credit) */ + + /* Individual contribution to load */ + s_time_t load_last_update; /* Last time average was updated */ + s_time_t avgload; /* Decaying queue load */ + + struct list_head runq_elem; /* On the runqueue (rqd->runq) */ + struct csched2_dom *sdom; /* Up-pointer to domain */ + struct vcpu *vcpu; /* Up-pointer, to vcpu */ + + struct csched2_runqueue_data *migrate_rqd; /* Pre-determined migr. target */ + int tickled_cpu; /* Cpu that will pick us (-1 if none) */ }; /* * Domain */ struct csched2_dom { - struct list_head sdom_elem; - struct domain *dom; - uint16_t weight; - uint16_t nr_vcpus; + struct list_head sdom_elem; /* On csched2_runqueue_data's sdom list */ + struct domain *dom; /* Up-pointer to domain */ + uint16_t weight; /* User specified weight */ + uint16_t nr_vcpus; /* Number of vcpus of this domain */ }; /*