From patchwork Mon May 6 06:55:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 10930545 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5709A1390 for ; Mon, 6 May 2019 06:58:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45DCE28397 for ; Mon, 6 May 2019 06:58:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 397D228590; Mon, 6 May 2019 06:58:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 45FE32861E for ; Mon, 6 May 2019 06:58:47 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hNXYU-0001sG-0c; Mon, 06 May 2019 06:56:54 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hNXYT-0001s0-Do for xen-devel@lists.xenproject.org; Mon, 06 May 2019 06:56:53 +0000 X-Inumbo-ID: 1ecc4688-6fcc-11e9-843c-bc764e045a96 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 1ecc4688-6fcc-11e9-843c-bc764e045a96; Mon, 06 May 2019 06:56:50 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 83686AC8C; Mon, 6 May 2019 06:56:48 +0000 (UTC) From: Juergen Gross To: xen-devel@lists.xenproject.org Date: Mon, 6 May 2019 08:55:59 +0200 Message-Id: <20190506065644.7415-1-jgross@suse.com> X-Mailer: git-send-email 2.16.4 Subject: [Xen-devel] [PATCH RFC V2 00/45] xen: add core scheduling support X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Tim Deegan , Stefano Stabellini , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Robert VanVossen , Dario Faggioli , Julien Grall , Josh Whitehead , Meng Xu , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Add support for core- and socket-scheduling in the Xen hypervisor. Via boot parameter sched-gran=core (or sched-gran=socket) it is possible to change the scheduling granularity from cpu (the default) to either whole cores or even sockets. All logical cpus (threads) of the core or socket are always scheduled together. This means that on a core always vcpus of the same domain will be active, and those vcpus will always be scheduled at the same time. This is achieved by switching the scheduler to no longer see vcpus as the primary object to schedule, but "schedule items". Each schedule item consists of as many vcpus as each core has threads on the current system. The vcpu->item relation is fixed. I have done some very basic performance testing: on a 4 cpu system (2 cores with 2 threads each) I did a "make -j 4" for building the Xen hypervisor. With This test has been run on dom0, once with no other guest active and once with another guest with 4 vcpus running the same test. The results are (always elapsed time, system time, user time): sched-gran=cpu, no other guest: 116.10 177.65 207.84 sched-gran=core, no other guest: 114.04 175.47 207.45 sched-gran=cpu, other guest: 202.30 334.21 384.63 sched-gran=core, other guest: 207.24 293.04 371.37 All tests have been performed with credit2, the other schedulers are untested up to now. Cpupools are not yet working, as moving cpus between cpupools needs more work (this is the reason for the series still being RFC). Same applies to cpu add/remove. Changes in RFC V2: - ARM is building now - HVM domains are working now - idling will always be done with idle_vcpu active - other small changes see individual patches Juergen Gross (45): xen/sched: add inline wrappers for calling per-scheduler functions xen/sched: use new sched_item instead of vcpu in scheduler interfaces xen/sched: alloc struct sched_item for each vcpu xen/sched: move per-vcpu scheduler private data pointer to sched_item xen/sched: build a linked list of struct sched_item xen/sched: introduce struct sched_resource xen/sched: let pick_cpu return a scheduler resource xen/sched: switch schedule_data.curr to point at sched_item xen/sched: move per cpu scheduler private data into struct sched_resource xen/sched: switch vcpu_schedule_lock to item_schedule_lock xen/sched: move some per-vcpu items to struct sched_item xen/sched: add scheduler helpers hiding vcpu xen/sched: add domain pointer to struct sched_item xen/sched: add id to struct sched_item xen/sched: rename scheduler related perf counters xen/sched: switch struct task_slice from vcpu to sched_item xen/sched: add is_running indicator to struct sched_item xen/sched: make null scheduler vcpu agnostic. xen/sched: make rt scheduler vcpu agnostic. xen/sched: make credit scheduler vcpu agnostic. xen/sched: make credit2 scheduler vcpu agnostic. xen/sched: make arinc653 scheduler vcpu agnostic. xen: add sched_item_pause_nosync() and sched_item_unpause() xen: let vcpu_create() select processor xen/sched: use sched_resource cpu instead smp_processor_id in schedulers xen/sched: switch schedule() from vcpus to sched_items xen/sched: switch sched_move_irqs() to take sched_item as parameter xen: switch from for_each_vcpu() to for_each_sched_item() xen/sched: add runstate counters to struct sched_item xen/sched: rework and rename vcpu_force_reschedule() xen/sched: Change vcpu_migrate_*() to operate on schedule item xen/sched: move struct task_slice into struct sched_item xen/sched: add code to sync scheduling of all vcpus of a sched item xen/sched: introduce item_runnable_state() xen/sched: add support for multiple vcpus per sched item where missing x86: make loading of GDT at context switch more modular x86: optimize loading of GDT at context switch xen/sched: modify cpupool_domain_cpumask() to be an item mask xen/sched: support allocating multiple vcpus into one sched item xen/sched: add a scheduler_percpu_init() function xen/sched: add a percpu resource index xen/sched: add fall back to idle vcpu when scheduling item xen/sched: make vcpu_wake() and vcpu_sleep() core scheduling aware xen/sched: carve out freeing sched_item memory into dedicated function xen/sched: add scheduling granularity enum xen/arch/arm/domain.c | 2 +- xen/arch/arm/domain_build.c | 13 +- xen/arch/arm/smpboot.c | 2 + xen/arch/x86/dom0_build.c | 10 +- xen/arch/x86/domain.c | 95 ++- xen/arch/x86/hvm/dom0_build.c | 9 +- xen/arch/x86/pv/dom0_build.c | 10 +- xen/arch/x86/pv/emul-priv-op.c | 1 + xen/arch/x86/pv/shim.c | 4 +- xen/arch/x86/pv/traps.c | 5 +- xen/arch/x86/setup.c | 2 + xen/arch/x86/smpboot.c | 2 + xen/arch/x86/traps.c | 9 +- xen/common/cpupool.c | 30 +- xen/common/domain.c | 34 +- xen/common/domctl.c | 23 +- xen/common/keyhandler.c | 4 +- xen/common/sched_arinc653.c | 258 ++++---- xen/common/sched_credit.c | 743 ++++++++++----------- xen/common/sched_credit2.c | 1119 +++++++++++++++---------------- xen/common/sched_null.c | 424 ++++++------ xen/common/sched_rt.c | 544 +++++++-------- xen/common/schedule.c | 1348 +++++++++++++++++++++++++++++--------- xen/common/softirq.c | 6 +- xen/common/wait.c | 4 +- xen/include/asm-arm/current.h | 1 + xen/include/asm-x86/cpuidle.h | 2 +- xen/include/asm-x86/current.h | 7 +- xen/include/asm-x86/dom0_build.h | 3 +- xen/include/asm-x86/smp.h | 3 + xen/include/xen/domain.h | 3 +- xen/include/xen/perfc_defn.h | 32 +- xen/include/xen/sched-if.h | 418 ++++++++++-- xen/include/xen/sched.h | 95 ++- xen/include/xen/softirq.h | 1 + 35 files changed, 3198 insertions(+), 2068 deletions(-)