From patchwork Wed Sep 11 10:32:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140865 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C200014E5 for ; Wed, 11 Sep 2019 10:34:04 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9E4002082C for ; Wed, 11 Sep 2019 10:34:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="A21Wk5W0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E4002082C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvO-00006J-9a; Wed, 11 Sep 2019 10:32:34 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvN-000064-8V for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:32:33 +0000 X-Inumbo-ID: 742a4384-d47f-11e9-978d-bc764e2007e4 Received: from mail-lf1-x142.google.com (unknown [2a00:1450:4864:20::142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 742a4384-d47f-11e9-978d-bc764e2007e4; Wed, 11 Sep 2019 10:32:29 +0000 (UTC) Received: by mail-lf1-x142.google.com with SMTP id t8so15988225lfc.13 for ; Wed, 11 Sep 2019 03:32:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wjAeCZij2Xw6xoNlfZ2Z1Stlf0K5I20sXHD+nMPhkoI=; b=A21Wk5W0TFKbr1uHis+Qk7ZnFcg5HeHxrLmEOi8sRCxC9G9zToYSwqLvQKqGSM5oWN AZgweoS/81b0sSsJN+gDCbLhz06Vnd7vyrf7fS1qaCh8Z+/KcCN8erihO8PolKfDANYz 1T9sNmAsC3yJqP/CRA3XfD3K9hn1nxtq43IAbycKBMbgnwQIsZG/PTRES+/yF/Q1r9DF 9EuxQYsJQLbFdfdn3dcedfCkBvFI5HAgdhSdM5/WEE6VsSF+gaREOaWsS46s3EvsnMV2 AYmgrvpqdsjrheD5gBgvo3Ge5yj0jWgBMdaH86YEiO+t6PeoJL9qKjQ58ZzUsHvtsG00 nysQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wjAeCZij2Xw6xoNlfZ2Z1Stlf0K5I20sXHD+nMPhkoI=; b=cGyaDEMk8Rkawf0Le/WCBDnjJxMJXf6DUiBCH4+NHR2pQwaEuInvy1efB8nw5n7QHP ZJxP5Jyl2fCbZbtw2c8KIJOUpwAmpZYA4EboDNsB5PeqxtbEQMsmG0jY7unhymc+M4Zb b5bOoYUnop/Jmh3U6Ic7rdUaR8pOTPgnI+yf9HK2o0+eMMzEs3MXdyzUwxM4LqhfQ2o0 wcIt4bxpaEhye573FBq+EzzZY45LeFu0KNEKgsm0FtcfskRvRvkU3MJrLBJyqQ+49Yih HLOy9qYVA/ibGCIRGvkUNN+GfZG8UknuShk/byMgOimowZ/yRTgAHi70AeVbMMhiqb7n y3gA== X-Gm-Message-State: APjAAAWOOG6ADwYEQcmDLlaRItZ/IXdQOFFBFIw0lXFQSmYh8R391dm4 rkyQ17QiXQ48exsHklrlZ1qQpwqXkNY= X-Google-Smtp-Source: APXvYqwYktQJELQC55+1f3aWu6a8jxZrzr8GcXcq/hSCeKYctx4MxDcLYm+pOCcoIGjMxfg93iTcCA== X-Received: by 2002:ac2:5203:: with SMTP id a3mr340474lfl.151.1568197947575; Wed, 11 Sep 2019 03:32:27 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:26 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:14 +0300 Message-Id: <1568197942-15374-2-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 1/9] schedule: Introduce per-pcpu time accounting X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tim Deegan , Stefano Stabellini , Andrii Anisov , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov Introduce per-pcpu time accounting what includes the following states: TACC_HYP - the pcpu executes hypervisor code like softirq processing (including scheduling), tasklets and context switches TACC_GUEST - the pcpu executes guests code TACC_IDLE - the low-power state of the pcpu TACC_IRQ - the pcpu performs interrupts processing, without separation to guest or hypervisor interrupts TACC_GSYNC - the pcpu executes hypervisor code to process synchronous trap from the guest. E.g. hypercall processing or io emulation. Currently, the only reenterant state is TACC_IRQ. It is assumed, no changes to state other than TACC_IRQ could happen until we return from nested interrupts. IRQ time is accounted in a distinct way comparing to other states. It is acumulated between other states transition moments, and is substracted from the old state on states transion calculation. Signed-off-by: Andrii Anisov --- xen/common/schedule.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/sched.h | 27 +++++++++++++++++ 2 files changed, 108 insertions(+) diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 7b71581..6dd6603 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -1539,6 +1539,87 @@ static void schedule(void) context_switch(prev, next); } +DEFINE_PER_CPU(struct tacc, tacc); + +static void tacc_state_change(enum TACC_STATES new_state) +{ + s_time_t now, delta; + struct tacc* tacc = &this_cpu(tacc); + unsigned long flags; + + local_irq_save(flags); + + now = NOW(); + delta = now - tacc->state_entry_time; + + /* We do not expect states reenterability (at least through this function)*/ + ASSERT(new_state != tacc->state); + + tacc->state_time[tacc->state] += delta - tacc->irq_time; + tacc->state_time[TACC_IRQ] += tacc->irq_time; + tacc->irq_time = 0; + tacc->state = new_state; + tacc->state_entry_time = now; + + local_irq_restore(flags); +} + +void tacc_hyp(int place) +{ +// printk("\ttacc_hyp %u, place %d\n", smp_processor_id(), place); + tacc_state_change(TACC_HYP); +} + +void tacc_guest(int place) +{ +// printk("\ttacc_guest %u, place %d\n", smp_processor_id(), place); + tacc_state_change(TACC_GUEST); +} + +void tacc_idle(int place) +{ +// printk("\tidle cpu %u, place %d\n", smp_processor_id(), place); + tacc_state_change(TACC_IDLE); +} + +void tacc_gsync(int place) +{ +// printk("\ttacc_gsync %u, place %d\n", smp_processor_id(), place); + tacc_state_change(TACC_GSYNC); +} + +void tacc_irq_enter(int place) +{ + struct tacc* tacc = &this_cpu(tacc); + +// printk("\ttacc_irq_enter %u, place %d, cnt %d\n", smp_processor_id(), place, this_cpu(tacc).irq_cnt); + ASSERT(!local_irq_is_enabled()); + ASSERT(tacc->irq_cnt >= 0); + + if ( tacc->irq_cnt == 0 ) + { + tacc->irq_enter_time = NOW(); + } + + tacc->irq_cnt++; +} + +void tacc_irq_exit(int place) +{ + struct tacc* tacc = &this_cpu(tacc); + +// printk("\ttacc_irq_exit %u, place %d, cnt %d\n", smp_processor_id(), place, tacc->irq_cnt); + ASSERT(!local_irq_is_enabled()); + ASSERT(tacc->irq_cnt > 0); + if ( tacc->irq_cnt == 1 ) + { + tacc->irq_time = NOW() - tacc->irq_enter_time; + tacc->irq_enter_time = 0; + } + + tacc->irq_cnt--; +} + void context_saved(struct vcpu *prev) { /* Clear running flag /after/ writing context to memory. */ diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index e3601c1..04a8724 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -1002,6 +1002,33 @@ extern void dump_runq(unsigned char key); void arch_do_physinfo(struct xen_sysctl_physinfo *pi); +enum TACC_STATES { + TACC_HYP = 0, + TACC_GUEST = 1, + TACC_IDLE = 2, + TACC_IRQ = 3, + TACC_GSYNC = 4, + TACC_STATES_MAX +}; + +struct tacc +{ + s_time_t state_time[TACC_STATES_MAX]; + s_time_t state_entry_time; + int state; + + s_time_t guest_time; + + s_time_t irq_enter_time; + s_time_t irq_time; + int irq_cnt; +}; + +DECLARE_PER_CPU(struct tacc, tacc); + +void tacc_hyp(int place); +void tacc_idle(int place); + #endif /* __SCHED_H__ */ /* From patchwork Wed Sep 11 10:32:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140873 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0A4414E5 for ; Wed, 11 Sep 2019 10:34:28 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ABEDD2087E for ; Wed, 11 Sep 2019 10:34:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q/90VE4Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABEDD2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvT-00007Z-K9; Wed, 11 Sep 2019 10:32:39 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvS-00007G-AS for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:32:38 +0000 X-Inumbo-ID: 74df8a82-d47f-11e9-b299-bc764e2007e4 Received: from mail-lf1-x143.google.com (unknown [2a00:1450:4864:20::143]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 74df8a82-d47f-11e9-b299-bc764e2007e4; Wed, 11 Sep 2019 10:32:30 +0000 (UTC) Received: by mail-lf1-x143.google.com with SMTP id w67so16009003lff.4 for ; Wed, 11 Sep 2019 03:32:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aqXFQn3KxWtsIDGkNKwWIgnyKzS5VXO95uQ8ju2Spbk=; b=Q/90VE4ZsQYdXaQZj0SMpAOryuSI4oLF014698oCBwgip3cEhFWFwdgsHd2Z5hP2Af 5s0EbjXov/zOjTok7I7AMBfPe9OyiuNt+pWBnb9AXea0L92tiMW+nO9pNCcr4ZPtq2f+ Z0fdxxeLEWsPNvTQNe8xnMz1JMa/1UU+M0Lso3czkFew5EO1ajQr6RkH76tRjrnfaT+u AmW8SzEkCNyMZJtkPsJn4W4T0EUwILs+I6AeiOSzIY3DwX+npRDWD0q5/ZxcLDlMxRay 164vu7VAwvPEqImM+vus132RB7w4Fsjy+Te7LwOwWVCH+tDSM1FxoeNIKysP+WjNtic9 N7gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aqXFQn3KxWtsIDGkNKwWIgnyKzS5VXO95uQ8ju2Spbk=; b=j6c18ILiyX5zKU43/wSGRIKnQbjbzY+6nICPG+DmLvTbIxmocZ5Hp8xZhPp9nlUmWx Nf01AjSWNou7snkF3CJweKVqZhv++qSKxMKIiZMdchql75Ijw9SkiRcLLqJYda30J5bA gRzjR27AxYrqImu3PMmJrWKrNN1gYh0ouqaSOS3FEXrqMgr6e8y+Qj3kYqWQT8CELOwG kormlF5S1CjkW8TWjJmjFtJZdCo0qOJ9rhG/BNwP5eUtbOChRg0SzvPasIkOXF9WTrQY 0nf4ntuTmv75cEMTp97HvcjaD3mN9nvBfAoNFZPOUzQZmV+efSUnucMpUgZHGi+IarCm gl3A== X-Gm-Message-State: APjAAAUNWZSQaRkV/5HbhQ1aIZFWHIt2sqzpHomdzN47rMyDwsOwc2U9 J9Jjea0F6wrXwS5dx6WlCc2rlfPde38= X-Google-Smtp-Source: APXvYqxwhq4bqGgpmK+m18qIu6USIpEwLeSuAJRw7jtd6RNL/NRsYXdybm/xkxz6mT9NmS7BUwRXJA== X-Received: by 2002:a05:6512:6c:: with SMTP id i12mr23753466lfo.40.1568197948968; Wed, 11 Sep 2019 03:32:28 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:28 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:15 +0300 Message-Id: <1568197942-15374-3-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 2/9] sysctl: extend XEN_SYSCTL_getcpuinfo interface X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tim Deegan , Stefano Stabellini , Andrii Anisov , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov Extend XEN_SYSCTL_getcpuinfo interface with timing information provided by introduced time accounting infrastructure. Signed-off-by: Andrii Anisov --- xen/common/schedule.c | 33 ++++++++++++++++++++++++++++----- xen/common/sysctl.c | 4 ++++ xen/include/public/sysctl.h | 4 ++++ xen/include/xen/sched.h | 4 ++++ 4 files changed, 40 insertions(+), 5 deletions(-) diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 6dd6603..2007034 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -208,13 +208,36 @@ void vcpu_runstate_get(struct vcpu *v, struct vcpu_runstate_info *runstate) uint64_t get_cpu_idle_time(unsigned int cpu) { - struct vcpu_runstate_info state = { 0 }; - struct vcpu *v = idle_vcpu[cpu]; + struct tacc *tacc = &per_cpu(tacc, cpu); - if ( cpu_online(cpu) && v ) - vcpu_runstate_get(v, &state); + return tacc->state_time[TACC_IDLE]; +} + +uint64_t get_cpu_guest_time(unsigned int cpu) +{ + struct tacc *tacc = &per_cpu(tacc, cpu); + + return tacc->state_time[TACC_GUEST]; +} + +uint64_t get_cpu_hyp_time(unsigned int cpu) +{ + struct tacc *tacc = &per_cpu(tacc, cpu); + + return tacc->state_time[TACC_HYP]; +} + +uint64_t get_cpu_irq_time(unsigned int cpu) +{ + struct tacc *tacc = &per_cpu(tacc, cpu); + + return tacc->state_time[TACC_IRQ]; +} +uint64_t get_cpu_gsync_time(unsigned int cpu) +{ + struct tacc *tacc = &per_cpu(tacc, cpu); - return state.time[RUNSTATE_running]; + return tacc->state_time[TACC_GSYNC]; } /* diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c index 92b4ea0..b63083c 100644 --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -152,6 +152,10 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) for ( i = 0; i < nr_cpus; i++ ) { cpuinfo.idletime = get_cpu_idle_time(i); + cpuinfo.guesttime = get_cpu_guest_time(i); + cpuinfo.hyptime = get_cpu_hyp_time(i); + cpuinfo.gsynctime = get_cpu_gsync_time(i); + cpuinfo.irqtime = get_cpu_irq_time(i); if ( copy_to_guest_offset(op->u.getcpuinfo.info, i, &cpuinfo, 1) ) goto out; diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index 5401f9c..cdada1f 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -168,6 +168,10 @@ struct xen_sysctl_debug_keys { /* XEN_SYSCTL_getcpuinfo */ struct xen_sysctl_cpuinfo { uint64_aligned_t idletime; + uint64_aligned_t hyptime; + uint64_aligned_t guesttime; + uint64_aligned_t irqtime; + uint64_aligned_t gsynctime; }; typedef struct xen_sysctl_cpuinfo xen_sysctl_cpuinfo_t; DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpuinfo_t); diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 04a8724..8167608 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -876,6 +876,10 @@ void restore_vcpu_affinity(struct domain *d); void vcpu_runstate_get(struct vcpu *v, struct vcpu_runstate_info *runstate); uint64_t get_cpu_idle_time(unsigned int cpu); +uint64_t get_cpu_hyp_time(unsigned int cpu); +uint64_t get_cpu_guest_time(unsigned int cpu); +uint64_t get_cpu_gsync_time(unsigned int cpu); +uint64_t get_cpu_irq_time(unsigned int cpu); /* * Used by idle loop to decide whether there is work to do: From patchwork Wed Sep 11 10:32:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140867 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D27417E6 for ; Wed, 11 Sep 2019 10:34:10 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 688192082C for ; Wed, 11 Sep 2019 10:34:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fRMTOSwJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 688192082C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvY-00009s-VW; Wed, 11 Sep 2019 10:32:44 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvX-00009C-9Q for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:32:43 +0000 X-Inumbo-ID: 75f7655c-d47f-11e9-a337-bc764e2007e4 Received: from mail-lj1-x244.google.com (unknown [2a00:1450:4864:20::244]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 75f7655c-d47f-11e9-a337-bc764e2007e4; Wed, 11 Sep 2019 10:32:32 +0000 (UTC) Received: by mail-lj1-x244.google.com with SMTP id u14so19450462ljj.11 for ; Wed, 11 Sep 2019 03:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AC9kNkmArPC3CzG+J65Zgd+nHkAn9+JKqJkWk1/eavM=; b=fRMTOSwJNbVaQuzqiIUP2BxsXXEOPY35/+4BhqAAYesK6QvtpW7qJYOOel8W6OK9mb H6pYZiaPM/5FtLOmukohyjNEcnKN+nqlyBHvCASq2li02l9iI2a67mwd6oppFu2pyOci FnZVgIwa91FTLb3tO7xOj75GB1UV0YNliifzVhIB+0GwMRTokC4yBK7eNQ/+I/bphYjs e5fpt1bYOx9OimcQ4RXs38vpiozBLGwUQ4f4SYHDl7oL9tOGXqTEVnD7QS6m8RpaGIJT rjiiZpqYkZ3xlptInbh6paUW9BNUs/E2OQ/pYepQLdxu2iA+b8/yOhKu+wcLFplvt3xQ EiSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AC9kNkmArPC3CzG+J65Zgd+nHkAn9+JKqJkWk1/eavM=; b=g7nXkAROIwEeB/N+iAlBiJ4s6Ql0m70B6zs0vQGDbVojUN8/CXr1YM4IQFuigtVgOI CBKeNo2nElqXBdw0pMaKh1WlFIcQ0emXofcg9aC8CZGkGNBa2vdghHuZGyiJv65q3LbA Ek0LpNjbpLefT0ItTODiGnZl5XLWdmRBs/Q0rdUsYr9JO59YaqbnAjMHw1p+2z4HZ2Ll p5F0ziTwH/SwcYgxscRUtsTIA0h1MKy2xJOHiTJv/cgoNjL/kX80PynrK4BCA6xmVbPT 3+iSVz0fPnKakI20Kc8NYbfPImsBBc9UNr+wYfd074GpKDHZPL5LQjPA+8dbhR+3yq0X YX/w== X-Gm-Message-State: APjAAAXNA4RL3PaZUDPDaWvqYfSXpmk+ISVJIbV26YPxMvA7Oj4isJYW xvYFmRd1m+tZlovCiUPvgp3qKgun4mE= X-Google-Smtp-Source: APXvYqzks7WvwhsxPXamr1i/FvYqbuwkN9K8Ylqe93xKrQE6r7f1nYgJpT93k1dnj/RYpmNZdd9+Jg== X-Received: by 2002:a2e:9586:: with SMTP id w6mr21889348ljh.47.1568197950711; Wed, 11 Sep 2019 03:32:30 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:30 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:16 +0300 Message-Id: <1568197942-15374-4-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 3/9] xentop: show CPU load information X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ian Jackson , Andrii Anisov , Wei Liu MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov Let xentop request and show information about CPU load provided by new time accounting infrastructure. Signed-off-by: Andrii Anisov --- tools/xenstat/libxenstat/src/xenstat.c | 50 +++++++++++++++++++++++++++++ tools/xenstat/libxenstat/src/xenstat.h | 15 +++++++++ tools/xenstat/libxenstat/src/xenstat_priv.h | 5 +++ tools/xenstat/xentop/xentop.c | 36 +++++++++++++++++++++ 4 files changed, 106 insertions(+) diff --git a/tools/xenstat/libxenstat/src/xenstat.c b/tools/xenstat/libxenstat/src/xenstat.c index 6f93d4e..cfb6504 100644 --- a/tools/xenstat/libxenstat/src/xenstat.c +++ b/tools/xenstat/libxenstat/src/xenstat.c @@ -134,6 +134,9 @@ void xenstat_uninit(xenstat_handle * handle) xenstat_node *xenstat_get_node(xenstat_handle * handle, unsigned int flags) { #define DOMAIN_CHUNK_SIZE 256 + xc_cpuinfo_t *cpuinfo; + int max_cpus, nr_cpus; + xenstat_node *node; xc_physinfo_t physinfo = { 0 }; xc_domaininfo_t domaininfo[DOMAIN_CHUNK_SIZE]; @@ -163,6 +166,28 @@ xenstat_node *xenstat_get_node(xenstat_handle * handle, unsigned int flags) * handle->page_size; node->freeable_mb = 0; + + max_cpus = node->num_cpus; + + cpuinfo = calloc(max_cpus, sizeof(xc_cpuinfo_t)); + if (!cpuinfo) + return NULL; + + if (xc_getcpuinfo(handle->xc_handle, max_cpus, cpuinfo, &nr_cpus) < 0) { + free(cpuinfo); + return NULL; + } + + for ( i = 0; i < nr_cpus; i++) { + node->idle_time += cpuinfo[i].idletime; + node->hyp_time += cpuinfo[i].hyptime; + node->guest_time += cpuinfo[i].guesttime; + node->gsync_time += cpuinfo[i].gsynctime; + node->irq_time += cpuinfo[i].irqtime; + } + + free(cpuinfo); + /* malloc(0) is not portable, so allocate a single domain. This will * be resized below. */ node->domains = malloc(sizeof(xenstat_domain)); @@ -332,6 +357,31 @@ unsigned long long xenstat_node_cpu_hz(xenstat_node * node) return node->cpu_hz; } +unsigned long long xenstat_node_idle_time(xenstat_node * node) +{ + return node->idle_time; +} + +unsigned long long xenstat_node_guest_time(xenstat_node * node) +{ + return node->guest_time; +} + +unsigned long long xenstat_node_hyp_time(xenstat_node * node) +{ + return node->hyp_time; +} + +unsigned long long xenstat_node_gsync_time(xenstat_node * node) +{ + return node->gsync_time; +} + +unsigned long long xenstat_node_irq_time(xenstat_node * node) +{ + return node->irq_time; +} + /* Get the domain ID for this domain */ unsigned xenstat_domain_id(xenstat_domain * domain) { diff --git a/tools/xenstat/libxenstat/src/xenstat.h b/tools/xenstat/libxenstat/src/xenstat.h index 76a660f..9881ce5 100644 --- a/tools/xenstat/libxenstat/src/xenstat.h +++ b/tools/xenstat/libxenstat/src/xenstat.h @@ -80,6 +80,21 @@ unsigned int xenstat_node_num_cpus(xenstat_node * node); /* Get information about the CPU speed */ unsigned long long xenstat_node_cpu_hz(xenstat_node * node); +/* Get information about the CPU idle time */ +unsigned long long xenstat_node_idle_time(xenstat_node * node); + +/* Get information about the CPU guest execution time */ +unsigned long long xenstat_node_guest_time(xenstat_node * node); + +/* Get information about the CPU hypervisor execution time */ +unsigned long long xenstat_node_hyp_time(xenstat_node * node); + +/* Get information about the CPU guest syncronous trap execution time */ +unsigned long long xenstat_node_gsync_time(xenstat_node * node); + +/* Get information about the CPU IRQ serving time */ +unsigned long long xenstat_node_irq_time(xenstat_node * node); + /* * Domain functions - extract information from a xenstat_domain */ diff --git a/tools/xenstat/libxenstat/src/xenstat_priv.h b/tools/xenstat/libxenstat/src/xenstat_priv.h index 4eb44a8..64387cd 100644 --- a/tools/xenstat/libxenstat/src/xenstat_priv.h +++ b/tools/xenstat/libxenstat/src/xenstat_priv.h @@ -45,6 +45,11 @@ struct xenstat_node { unsigned int flags; unsigned long long cpu_hz; unsigned int num_cpus; + unsigned long long hyp_time; + unsigned long long guest_time; + unsigned long long idle_time; + unsigned long long gsync_time; + unsigned long long irq_time; unsigned long long tot_mem; unsigned long long free_mem; unsigned int num_domains; diff --git a/tools/xenstat/xentop/xentop.c b/tools/xenstat/xentop/xentop.c index af11ebf..7514559 100644 --- a/tools/xenstat/xentop/xentop.c +++ b/tools/xenstat/xentop/xentop.c @@ -930,6 +930,40 @@ void adjust_field_widths(xenstat_domain *domain) fields[FIELD_VBD_WSECT-1].default_width = length; } +void do_utilization(void) +{ + double us_elapsed = 0.0, + us_hyp = 0.0, + us_guest = 0.0, + us_idle = 0.0, + us_gsync = 0.0, + us_irq = 0.0; + + /* Can't calculate CPU percentage without a current and a previous sample.*/ + if(prev_node != NULL && cur_node != NULL) { + + /* Calculate the time elapsed in microseconds */ + us_elapsed = ((curtime.tv_sec-oldtime.tv_sec)*1000000.0 + +(curtime.tv_usec - oldtime.tv_usec)); + + /* In the following, nanoseconds must be multiplied by 1000.0 to + * convert to microseconds, then divided by 100.0 to get a percentage, + * resulting in a multiplication by 10.0 */ + us_idle = ((xenstat_node_idle_time(cur_node) - + xenstat_node_idle_time(prev_node))/10.0)/us_elapsed; + us_guest = ((xenstat_node_guest_time(cur_node) - + xenstat_node_guest_time(prev_node))/10.0)/us_elapsed; + us_hyp = ((xenstat_node_hyp_time(cur_node) - + xenstat_node_hyp_time(prev_node))/10.0)/us_elapsed; + us_gsync = ((xenstat_node_gsync_time(cur_node) - + xenstat_node_gsync_time(prev_node))/10.0)/us_elapsed; + us_irq = ((xenstat_node_irq_time(cur_node) - + xenstat_node_irq_time(prev_node))/10.0)/us_elapsed; + } + + print("%%CPU(s): %6.1f gu, %6.1f gs, %6.1f ir, %6.1f hy, %6.1f id \n", + us_guest, us_gsync, us_irq, us_hyp, us_idle); +} /* Section printing functions */ /* Prints the top summary, above the domain table */ @@ -972,6 +1006,8 @@ void do_summary(void) used = xenstat_node_tot_mem(cur_node); freeable_mb = 0; + do_utilization(); + /* Dump node memory and cpu information */ if ( freeable_mb <= 0 ) print("Mem: %lluk total, %lluk used, %lluk free ", From patchwork Wed Sep 11 10:32:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140879 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC13914E5 for ; Wed, 11 Sep 2019 10:34:32 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 981D02087E for ; Wed, 11 Sep 2019 10:34:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C9Y2hzfU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 981D02087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvd-0000Bk-B7; Wed, 11 Sep 2019 10:32:49 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvc-0000BL-A9 for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:32:48 +0000 X-Inumbo-ID: 76e7155c-d47f-11e9-b299-bc764e2007e4 Received: from mail-lj1-x243.google.com (unknown [2a00:1450:4864:20::243]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 76e7155c-d47f-11e9-b299-bc764e2007e4; Wed, 11 Sep 2019 10:32:33 +0000 (UTC) Received: by mail-lj1-x243.google.com with SMTP id q64so8938613ljb.12 for ; Wed, 11 Sep 2019 03:32:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=FeNrC6ayJRxDhm8dZguxagsFl4ax8cTGLoDLGNor5EU=; b=C9Y2hzfU687HJuQP7Y3fi8qDQl9xytXyv9Kyrme6dgV4Xs/WHWxV6XmP7oasg8VfO6 SJJvsPpb20s4iOcVESxN/NvoBjs78eXLiQ0+RzENMu9W2LZ3gcb5Mk5M5Eyxyo0AoKZk lnVlmJW40/0Lf+arcy74Gkj2/vgnabYiuCBcGVZs1akW60WVOsJfmimJHWHNr/GsW8F9 PNr25QvQAkqBGvheRa9rhIwY3EecSrT3pn2cR8yT3xdjPoE5LWhMbIgQCHWqen8Jg/dn OYUokLg9GskhtC99lkwbDYAUuA5kJmWWHI4AsJcsT5wTiehHXDmVlWH+Bgwhi2N2blie L7wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FeNrC6ayJRxDhm8dZguxagsFl4ax8cTGLoDLGNor5EU=; b=HtEQJneMhHVv+PMKfX5s+z+FnYgEFHCyTUJKmELKpdk0eppVlfpGA6jTMus8sRVR2O scbFRSE0LG+rUhq8ID58x2zkHgzowzG8NkkfzSvbk0NKa5lvKDkGkvPO0UT8WC5a7xw/ 3Pc7IiYIXG4Ib9vidrfAQy8POKA/3znxdqZjoV16Dg0X0le1nstO6l83OJXSxmAiqbAJ kwOyWcgqQD/kdH4Uo31E97hPItooAvMPHLXQ4nmwhl3bZO9V6EoTtCHD+SV/ZmySh5bD CUYumcuZMkznid6uuDFeqDkfPPFV9hMsr4D1SfmUdjPDAojdNzSQJa702XJ6xi+ZoN+e Z2wA== X-Gm-Message-State: APjAAAWrWgMp+AY95EB8ocvxcu1Mb811b+yV0xgEtCp1b7k0OjIKLtHi kIo/vA+W/Mw7myT0hXN96RbGoXSZo4A= X-Google-Smtp-Source: APXvYqz5iVXjde/moak1Zqnzh85/RxP2W4dXNPHLLSW6UjD/1QSFoiU471ZYRqoCaWuOqP1lM0sDyQ== X-Received: by 2002:a2e:9882:: with SMTP id b2mr23024769ljj.225.1568197952337; Wed, 11 Sep 2019 03:32:32 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:31 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:17 +0300 Message-Id: <1568197942-15374-5-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 4/9] arm64: utilize time accounting X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Volodymyr Babchuk , Julien Grall , Stefano Stabellini , Andrii Anisov MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov Call time accounting hooks from appropriate transition points of the ARM64 code. Signed-off-by: Andrii Anisov --- xen/arch/arm/arm64/entry.S | 39 ++++++++++++++++++++++++++++++++++++--- xen/arch/arm/domain.c | 2 ++ 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/arm64/entry.S b/xen/arch/arm/arm64/entry.S index 2d9a271..6fb2fa9 100644 --- a/xen/arch/arm/arm64/entry.S +++ b/xen/arch/arm/arm64/entry.S @@ -143,12 +143,21 @@ .endm - .macro exit, hyp, compat + .macro exit, hyp, compat, tacc=1 .if \hyp == 0 /* Guest mode */ + .if \tacc == 1 + + mov x0, #1 + bl tacc_hyp + + .endif + bl leave_hypervisor_tail /* Disables interrupts on return */ + mov x0, #1 + bl tacc_guest exit_guest \compat .endif @@ -205,9 +214,15 @@ hyp_sync: hyp_irq: entry hyp=1 + mov x0,#5 + bl tacc_irq_enter msr daifclr, #4 mov x0, sp bl do_trap_irq + + mov x0,#5 + bl tacc_irq_exit + exit hyp=1 guest_sync: @@ -291,6 +306,9 @@ guest_sync_slowpath: * to save them. */ entry hyp=0, compat=0, save_x0_x1=0 + + mov x0,#1 + bl tacc_gsync /* * The vSError will be checked while SKIP_SYNCHRONIZE_SERROR_ENTRY_EXIT * is not set. If a vSError took place, the initial exception will be @@ -307,6 +325,10 @@ guest_sync_slowpath: guest_irq: entry hyp=0, compat=0 + + mov x0,#6 + bl tacc_irq_enter + /* * The vSError will be checked while SKIP_SYNCHRONIZE_SERROR_ENTRY_EXIT * is not set. If a vSError took place, the initial exception will be @@ -319,6 +341,8 @@ guest_irq: mov x0, sp bl do_trap_irq 1: + mov x0,#6 + bl tacc_irq_exit exit hyp=0, compat=0 guest_fiq_invalid: @@ -334,6 +358,9 @@ guest_error: guest_sync_compat: entry hyp=0, compat=1 + + mov x0,#2 + bl tacc_gsync /* * The vSError will be checked while SKIP_SYNCHRONIZE_SERROR_ENTRY_EXIT * is not set. If a vSError took place, the initial exception will be @@ -350,6 +377,10 @@ guest_sync_compat: guest_irq_compat: entry hyp=0, compat=1 + + mov x0,#7 + bl tacc_irq_enter + /* * The vSError will be checked while SKIP_SYNCHRONIZE_SERROR_ENTRY_EXIT * is not set. If a vSError took place, the initial exception will be @@ -362,6 +393,8 @@ guest_irq_compat: mov x0, sp bl do_trap_irq 1: + mov x0,#7 + bl tacc_irq_exit exit hyp=0, compat=1 guest_fiq_invalid_compat: @@ -376,9 +409,9 @@ guest_error_compat: exit hyp=0, compat=1 ENTRY(return_to_new_vcpu32) - exit hyp=0, compat=1 + exit hyp=0, compat=1, tacc=0 ENTRY(return_to_new_vcpu64) - exit hyp=0, compat=0 + exit hyp=0, compat=0, tacc=0 return_from_trap: msr daifset, #2 /* Mask interrupts */ diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c index a9c4113..53ef630 100644 --- a/xen/arch/arm/domain.c +++ b/xen/arch/arm/domain.c @@ -51,11 +51,13 @@ static void do_idle(void) process_pending_softirqs(); local_irq_disable(); + tacc_idle(1); if ( cpu_is_haltable(cpu) ) { dsb(sy); wfi(); } + tacc_hyp(2); local_irq_enable(); sched_tick_resume(); From patchwork Wed Sep 11 10:32:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140869 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 221FA184E for ; Wed, 11 Sep 2019 10:34:11 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F1AAA20CC7 for ; Wed, 11 Sep 2019 10:34:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BnFZb4zk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1AAA20CC7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvi-0000FK-Sw; Wed, 11 Sep 2019 10:32:54 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvh-0000EQ-9u for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:32:53 +0000 X-Inumbo-ID: 77adbebe-d47f-11e9-a337-bc764e2007e4 Received: from mail-lj1-x244.google.com (unknown [2a00:1450:4864:20::244]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 77adbebe-d47f-11e9-a337-bc764e2007e4; Wed, 11 Sep 2019 10:32:35 +0000 (UTC) Received: by mail-lj1-x244.google.com with SMTP id a22so19511223ljd.0 for ; Wed, 11 Sep 2019 03:32:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MxYXirbw6xPemib9KGe3LPit2WCpm5IHJxLF3QhMyss=; b=BnFZb4zkh2Vi/LyxIKfqWpTOqMoIEu6BHvmnds1YYCeXR64vHew35ijMfBsIGEzmXP +94/wYp2b/jE0XhgoFUl/6yOW7M79jWSYp2wiNCMuIjtJEDcLUrsL1nJmTeCfejTNxNk U3T57S6i/IWTHn/g3ttVmcKarb3g/iVbFdN+0EdnqvqfL4aoX1zr0/LxV7MSJR1boohY ySFnEmsTwFTT+Ferk//eheLkr7IY68wLd+M3EE64UfEnMrs7M90HFtulddvJn4o8VTLd ba9So8Tnad1hokXfYS8ty7VdNF+e3hJd9wN+DTWjar/syQmk4rUUc37PKwkQE88I6oNW nyuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MxYXirbw6xPemib9KGe3LPit2WCpm5IHJxLF3QhMyss=; b=ZKjeQz4XJmBsNp4RZet1MbNg4ppfELST+z/l2N8jSqizSzmhLhquSIwEW6ANfBR88h iFZOEUeBF0NDeh9ZJyLTJUeiii8la2Ho6O6e+XPivEdaKFQiMI5SWD73CwnEIiZY5D7W LQqGzWgbpGFkp9tHaCxDnyCtVM6xpCyZ2OKCJ8fziygeGdMkLwWXpxdFmRsheAWhPAT+ 12tVOtW4tYL0Z8WC4N24j4spS8REJrYuMAxRvFfr7vl4hgAnxDmTUb7LCuEHHb+TIvEj q79znxQdj98KO+RO5wJWpgWhrA9QU6k2lAGzT3lkfQ2vpycaKQCQKNmF+jy6U0nTjpwp rspA== X-Gm-Message-State: APjAAAXf+MqjyGgoeTRBsQzSDA+f5/U6n8TajcGPwUbQKFOA7v8Ixjg3 50yT+j/4n68dz5LI8jr/1Icjk+tf04M= X-Google-Smtp-Source: APXvYqxHC/9oq3VXTPaJPxfA6HiwbnCV8O4kfZcimSKZ+5co849oMuU2HcoQhHVnUa59R9x7sdW4KQ== X-Received: by 2002:a2e:7a03:: with SMTP id v3mr9291670ljc.138.1568197953676; Wed, 11 Sep 2019 03:32:33 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:33 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:18 +0300 Message-Id: <1568197942-15374-6-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 5/9] tacc: Introduce a lockless interface for guest time X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Tim Deegan , Stefano Stabellini , Andrii Anisov , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Dario Faggioli , Julien Grall , Jan Beulich MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov The lockless interface to acquire guest time by scheduling code is introduced. It can be used by schedulers what do not require guest time from a different pcpu to take scheduling decission. Signed-off-by: Andrii Anisov --- xen/common/schedule.c | 10 ++++++++++ xen/include/xen/sched.h | 8 ++++++++ 2 files changed, 18 insertions(+) diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 2007034..62df77e 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -1643,6 +1643,16 @@ void tacc_irq_exit(int place) tacc->irq_cnt--; } +s_time_t tacc_get_guest_time(struct tacc *tacc) +{ + s_time_t guest_time; + + guest_time = tacc->state_time[TACC_GUEST]; + guest_time += tacc->state_time[TACC_GSYNC]; + + return guest_time; +} + void context_saved(struct vcpu *prev) { /* Clear running flag /after/ writing context to memory. */ diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 8167608..5b41805 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -266,6 +266,8 @@ struct vcpu struct evtchn_fifo_vcpu *evtchn_fifo; + s_time_t pcpu_guest_time; + /* vPCI per-vCPU area, used to store data for long running operations. */ struct vpci_vcpu vpci; @@ -1033,6 +1035,12 @@ DECLARE_PER_CPU(struct tacc, tacc); void tacc_hyp(int place); void tacc_idle(int place); +s_time_t tacc_get_guest_time(struct tacc *tacc); +inline s_time_t tacc_get_guest_time_delta(void) +{ + return tacc_get_guest_time(&this_cpu(tacc)) - current->pcpu_guest_time; +} + #endif /* __SCHED_H__ */ /* From patchwork Wed Sep 11 10:32:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140881 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA08717E6 for ; Wed, 11 Sep 2019 10:34:38 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A60612087E for ; Wed, 11 Sep 2019 10:34:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BqXnLW68" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A60612087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvo-0000Ib-74; Wed, 11 Sep 2019 10:33:00 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvm-0000HO-9T for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:32:58 +0000 X-Inumbo-ID: 7872d6ea-d47f-11e9-978d-bc764e2007e4 Received: from mail-lf1-x141.google.com (unknown [2a00:1450:4864:20::141]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7872d6ea-d47f-11e9-978d-bc764e2007e4; Wed, 11 Sep 2019 10:32:36 +0000 (UTC) Received: by mail-lf1-x141.google.com with SMTP id t8so15988533lfc.13 for ; Wed, 11 Sep 2019 03:32:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=08Tfqlh32NonuTtd2lAC2CC6l3pp2R7W8WbzNwGVHbA=; b=BqXnLW68Bh5QTW0Ew/mCLYKCqvWmb/YPGKeQEAPTGOZotGTFSmw4ATOHcollF/TsBB dEmF2tOGjYrMh//MMdu5usIUo8L9tqGjIba0zOCC06Ms/OkdEWdcLPHcGfE9riHEqTnF abJV9QX3hUB9bLo7HYKTMZj3m5oGTFDSCg7w5rscLcLc6atTZVm6ht5vf3n7+9mD//ui Q2+RbmY3R7syxHPVzRUfFdtogEHRYFUtf+uOeSJ/124fdQPDiRPOCxqRjqdA+pQYk1gM Bbl4Br0klvYTWCrITEvr7uTMm68chLWkiF6+4adXGk8rgxqXeIu3Y/m42iA2hpoCCShr zR8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=08Tfqlh32NonuTtd2lAC2CC6l3pp2R7W8WbzNwGVHbA=; b=pQYRWFk2FuLWOqW3WPXQF/uA56tmXV+93BcbFsjNdvjZAOhfwxh8L0nov5wOxmdS7y BHQh82JIfkIjrap1snWC2Cjx3oUhp0R0TeZF2jBHPkPHFEhsw3drgI1Jxsb46iTE7PHc e5PkB2Rr16m4jWV7AuNbJfZ7StYWFatES6QaWs0zVX3hjhNYPY5LdJ5Cs8VGV0RBQlPw p5zT6daBkOoIlLzKq1+ktwjvzBzCy7JRXQBTnn3zUl5AYhBNiWxQGET+8acPHJeDIytq tMhOXa4Rb6RFTsxngqDX4Luddia8URx3WHdHFMR32BTxIB2L6qLoHEsyY8sTI8e3MeIG 6BtA== X-Gm-Message-State: APjAAAXRKMw+lrvsOxPZvXTlvVzIkftFIZ/6Q1H1BR5fQdsS0JOAzGN0 VekeGwObxnwj001mQ67Cqro3V7C1NgU= X-Google-Smtp-Source: APXvYqx0Y2MLLMusealu4mWXiRk+PuMbEJzgqM8TyJJ/YqX67dWzFfUWbRs8DaLxD53ZYt1aCG9+1Q== X-Received: by 2002:ac2:4a69:: with SMTP id q9mr22552432lfp.86.1568197954984; Wed, 11 Sep 2019 03:32:34 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:34 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:19 +0300 Message-Id: <1568197942-15374-7-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 6/9] sched:rtds: get guest time from time accounting code X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: George Dunlap , Andrii Anisov , Meng Xu , Dario Faggioli MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov While the RTDS scheduler code does not use guest time from the other pcpu, we are free to go with lockless time accounting. Signed-off-by: Andrii Anisov --- xen/common/sched_rt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/common/sched_rt.c b/xen/common/sched_rt.c index e0e350b..2ce200b 100644 --- a/xen/common/sched_rt.c +++ b/xen/common/sched_rt.c @@ -945,8 +945,9 @@ burn_budget(const struct scheduler *ops, struct rt_vcpu *svc, s_time_t now) if ( is_idle_vcpu(svc->vcpu) ) return; + ASSERT(svc->vcpu == current); /* burn at nanoseconds level */ - delta = now - svc->last_start; + delta = tacc_get_guest_time_delta(); /* * delta < 0 only happens in nested virtualization; * TODO: how should we handle delta < 0 in a better way? @@ -960,7 +961,6 @@ burn_budget(const struct scheduler *ops, struct rt_vcpu *svc, s_time_t now) } svc->cur_budget -= delta; - svc->last_start = now; if ( svc->cur_budget <= 0 ) { From patchwork Wed Sep 11 10:32:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140875 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4541017E6 for ; Wed, 11 Sep 2019 10:34:30 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 211C12087E for ; Wed, 11 Sep 2019 10:34:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NyaNagQ/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 211C12087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvs-0000Lb-IZ; Wed, 11 Sep 2019 10:33:04 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvr-0000Ks-AR for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:33:03 +0000 X-Inumbo-ID: 793ed45c-d47f-11e9-a337-bc764e2007e4 Received: from mail-lf1-x144.google.com (unknown [2a00:1450:4864:20::144]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 793ed45c-d47f-11e9-a337-bc764e2007e4; Wed, 11 Sep 2019 10:32:37 +0000 (UTC) Received: by mail-lf1-x144.google.com with SMTP id r134so16000689lff.12 for ; Wed, 11 Sep 2019 03:32:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Pe9dvRwPhdT31/9Du9G7/kjBTUkLjLa2MmB2iXYgZwg=; b=NyaNagQ/0rkjRnn2FpInVnjY6JHRRjh3OlJ2NA6KLGo0THcH9KEsNCS3XJoEetGs3i jHeXHwXYN0mmcEAV/e8k7MQV+Bs9uJjxCR5E4KAnqFn5vIMeb007oQ/PTa9smFPDExi/ MkmA1Rqp7Dx3a4mXaaYwRr5ZoIpVXkwx9ds/4J3FsgxzPXM0afijlkND4+1wbzorl2R2 3Bs44QgGvJuFNT1yy2nEk50e/DfTPaX9jmVifXshFJQ8KsqKdyoo8r9uqFJq+qjuxdba 0CnEbXapbcmEFt5KmYl8ycUTI5UtFUWwoMHMqPHxjSUR8OWa3wjI3s1b5RS45HFW6Xp4 stsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Pe9dvRwPhdT31/9Du9G7/kjBTUkLjLa2MmB2iXYgZwg=; b=sIWJkYr30bDQ6M1ZHY2modvncPAGBOgF+SvdXuyqwRdksu9TnnSgr1tAduBXKK/3YS fPoNWPVanoK3QmEgZysDQzh50RhHXv+LNXuNcAURXKwsYW3tGdh3oBRriBptk/RQur3X 7sA2nhMP8ZbKMzkfOiXFwNYrzEhxYyTBtg712q+lTqAG6Ap/Fr5O6VCI6TRoLraNoo0P uOKLlretouJ5i/li7KEpv+G7wW1Ck01D79vU7zNzI5ePiH6uktbdkn9X64kZDLzzDa2I sUwXF2iiY1EZw8P7yWFUh71vVjXA+/2dhu/pMnZ+P6fopGYl/yOYpLjEBzT7eIRBElEP DaEA== X-Gm-Message-State: APjAAAWleSZdA6A1UFRA+bZvVJDwyfyDASiv5dF7Kng9uzAS3DyGObFN lJyIto2oJDbxP6AudOfQFdtBpI+4l0k= X-Google-Smtp-Source: APXvYqxjejrhFx/4g2irDO7w4Gg156RQ5cKIophC9kg0Yr2AE0r6OpyXiLm4oNq1tXRpJjIv/+EvCQ== X-Received: by 2002:ac2:4adb:: with SMTP id m27mr5227384lfp.1.1568197956313; Wed, 11 Sep 2019 03:32:36 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:35 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:20 +0300 Message-Id: <1568197942-15374-8-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 7/9] tacc: Introduce a locked interface for guest time X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Andrii Anisov , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Julien Grall , Jan Beulich , Dario Faggioli MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov The locked interface to acquire guest time by scheduling code is introduced. It can be used by schedulers what do require guest time from a different pcpu to take scheduling decission. Signed-off-by: Andrii Anisov --- xen/common/Kconfig | 3 +++ xen/common/schedule.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/sched.h | 11 +++++++++++ 3 files changed, 58 insertions(+) diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 16829f6..c1748dd 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -221,6 +221,9 @@ config ARGO menu "Schedulers" visible if EXPERT = "y" +config TACC_NEEDS_LOCK + bool + config SCHED_CREDIT bool "Credit scheduler support" default y diff --git a/xen/common/schedule.c b/xen/common/schedule.c index 62df77e..98b739f 100644 --- a/xen/common/schedule.c +++ b/xen/common/schedule.c @@ -1562,6 +1562,14 @@ static void schedule(void) context_switch(prev, next); } +#ifdef CONFIG_TACC_NEEDS_LOCK +#define tacc_lock(tacc) spin_lock(&tacc->tacc_lock) +#define tacc_unlock(tacc) spin_unlock(&tacc->tacc_lock) +#else +#define tacc_lock(tacc) +#define tacc_unlock(tacc) +#endif + DEFINE_PER_CPU(struct tacc, tacc); static void tacc_state_change(enum TACC_STATES new_state) @@ -1571,6 +1579,7 @@ static void tacc_state_change(enum TACC_STATES new_state) unsigned long flags; local_irq_save(flags); + tacc_lock(tacc); now = NOW(); delta = now - tacc->state_entry_time; @@ -1584,6 +1593,7 @@ static void tacc_state_change(enum TACC_STATES new_state) tacc->state = new_state; tacc->state_entry_time = now; + tacc_unlock(tacc); local_irq_restore(flags); } @@ -1621,7 +1631,9 @@ void tacc_irq_enter(int place) if ( tacc->irq_cnt == 0 ) { + tacc_lock(tacc); tacc->irq_enter_time = NOW(); + tacc_unlock(tacc); } tacc->irq_cnt++; @@ -1636,8 +1648,10 @@ void tacc_irq_exit(int place) ASSERT(tacc->irq_cnt > 0); if ( tacc->irq_cnt == 1 ) { + tacc_lock(tacc); tacc->irq_time = NOW() - tacc->irq_enter_time; tacc->irq_enter_time = 0; + tacc_unlock(tacc); } tacc->irq_cnt--; @@ -1653,6 +1667,36 @@ s_time_t tacc_get_guest_time(struct tacc *tacc) return guest_time; } +#ifdef CONFIG_TACC_NEEDS_LOCK +s_time_t tacc_get_guest_time_cpu(int cpu) +{ + struct tacc* tacc = &per_cpu(tacc, cpu); + s_time_t guest_time; + s_time_t now; + + tacc_lock(tacc); + + now = NOW(); + guest_time = tacc_get_guest_time(tacc); + if (tacc->state == TACC_GUEST || tacc->state == TACC_GSYNC) + { + guest_time += NOW() - tacc->state_entry_time; + } + + if (tacc->irq_enter_time) + { + guest_time -= NOW() - tacc->irq_enter_time; + } + + guest_time -= tacc->irq_time; + + tacc_unlock(tacc); + + return guest_time; +} +#endif + + void context_saved(struct vcpu *prev) { /* Clear running flag /after/ writing context to memory. */ diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h index 5b41805..a649d1f 100644 --- a/xen/include/xen/sched.h +++ b/xen/include/xen/sched.h @@ -1028,6 +1028,9 @@ struct tacc s_time_t irq_enter_time; s_time_t irq_time; int irq_cnt; +#ifdef CONFIG_TACC_NEEDS_LOCK + spinlock_t tacc_lock; +#endif }; DECLARE_PER_CPU(struct tacc, tacc); @@ -1041,6 +1044,14 @@ inline s_time_t tacc_get_guest_time_delta(void) return tacc_get_guest_time(&this_cpu(tacc)) - current->pcpu_guest_time; } +#ifdef CONFIG_TACC_NEEDS_LOCK +s_time_t tacc_get_guest_time_cpu(int cpu); +inline s_time_t tacc_get_guest_time_delta_vcpu(struct vcpu* vcpu) +{ + return tacc_get_guest_time_cpu(vcpu->processor) - vcpu->pcpu_guest_time; +} +#endif + #endif /* __SCHED_H__ */ /* From patchwork Wed Sep 11 10:32:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140883 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA25217E6 for ; Wed, 11 Sep 2019 10:34:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5D9A2087E for ; Wed, 11 Sep 2019 10:34:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mkSkUftQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5D9A2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvx-0000Q7-Ta; Wed, 11 Sep 2019 10:33:09 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zvw-0000P1-AV for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:33:08 +0000 X-Inumbo-ID: 7a46ec36-d47f-11e9-b299-bc764e2007e4 Received: from mail-lj1-x244.google.com (unknown [2a00:1450:4864:20::244]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7a46ec36-d47f-11e9-b299-bc764e2007e4; Wed, 11 Sep 2019 10:32:39 +0000 (UTC) Received: by mail-lj1-x244.google.com with SMTP id u14so19450744ljj.11 for ; Wed, 11 Sep 2019 03:32:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=D1tmQx4QQQTXiGm/taYp9ljWrLsYP/6Sdp2V1mTvY5Q=; b=mkSkUftQESBwTE4pIFjxK9nOfDpujozkVjuwu1pLch7xAXRF/bECdIU1TZyHk5wChg UOWQlOXQPb+UjPLj1fcfEKamv9s8PynMB/StH3U1vRP3AVaxbIt6pFCra/d5IR0oaQ7N y1dARxwpsAU8HA4wOmcyUrXnodrkPo2Z2BlzebyvDBGOSb5pOKWcckzW0t2j7WZ1hVT7 R0m8q0eaAzywezFAe9j1YCCFj/MzugvLzL2bEl1hD7r+TQXlb1Z6aydmgpSAnAFAVSFH 4ybJJfc3bCZccsg+w7QQKvZRgQdFcaChtvEEeXHp1GLQq+AB+t2h01uvp+cZrj+uefcl dLTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=D1tmQx4QQQTXiGm/taYp9ljWrLsYP/6Sdp2V1mTvY5Q=; b=Gi0uxWIXJRxiLsFeowNR8QaCk/ZcT1BtiEG0h9rrZLOUU8AtRKatA5tq85Y/B457TX GSAEGcWY6QaHXIn52YV45MESDIi6t9t589qYFQunRgHe/5lMYFuqrPiw6TGQCQUURhN4 dk84kLfOtbKFMUU6oF+JqELdjN/UP8cYBrvJT99dmF2eI47erFfP2buIWNwjR+jDV+iI 9H9yU+V2/D8ZSDA4NEDvsjbdUSKCSVbqMWNMBnBBaFW5DShDKhh8gcHiwiJClCpdnXvC vmX8mXh2R3rjThzcxUC9eYjF14RBiZfqk6dSW8YvVi+ZXCyAsWO2wX+Ut2YdNuKOSJpG Ur3w== X-Gm-Message-State: APjAAAUNA9baGP0Pw155WKcVU6WkOAm9+rc2Lp4h3KsxRqPw4WdNmUXY R1d+k5tH2c9GYhBGHP8r6zkenr/m40U= X-Google-Smtp-Source: APXvYqzHZOS70e5XAk+zkJW0cCEdc6izNMK3UieVBImeIDCwGtsLRl0Oq/AKUp5DG3TGajQ0KAUjAw== X-Received: by 2002:a2e:504f:: with SMTP id v15mr23161379ljd.67.1568197957942; Wed, 11 Sep 2019 03:32:37 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:37 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:21 +0300 Message-Id: <1568197942-15374-9-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 8/9] sched:credit: get guest time from time accounting code X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Andrii Anisov , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Julien Grall , Jan Beulich , Dario Faggioli MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov While the Credit scheduler code uses guest time from the other pcpu, we have to use locked time accounting. Signed-off-by: Andrii Anisov --- xen/common/Kconfig | 1 + xen/common/sched_credit.c | 12 +++++------- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/xen/common/Kconfig b/xen/common/Kconfig index c1748dd..d17a8b4 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -227,6 +227,7 @@ config TACC_NEEDS_LOCK config SCHED_CREDIT bool "Credit scheduler support" default y + select TACC_NEEDS_LOCK ---help--- The traditional credit scheduler is a general purpose scheduler. diff --git a/xen/common/sched_credit.c b/xen/common/sched_credit.c index 81dee5e..ac6b9e6 100644 --- a/xen/common/sched_credit.c +++ b/xen/common/sched_credit.c @@ -324,16 +324,15 @@ runq_remove(struct csched_vcpu *svc) __runq_remove(svc); } -static void burn_credits(struct csched_vcpu *svc, s_time_t now) +static void burn_credits(struct csched_vcpu *svc, s_time_t delta) { - s_time_t delta; uint64_t val; unsigned int credits; /* Assert svc is current */ ASSERT( svc == CSCHED_VCPU(curr_on_cpu(svc->vcpu->processor)) ); - if ( (delta = now - svc->start_time) <= 0 ) + if ( delta <= 0 ) return; val = delta * CSCHED_CREDITS_PER_MSEC + svc->residual; @@ -341,7 +340,6 @@ static void burn_credits(struct csched_vcpu *svc, s_time_t now) credits = val; ASSERT(credits == val); /* make sure we haven't truncated val */ atomic_sub(credits, &svc->credit); - svc->start_time += (credits * MILLISECS(1)) / CSCHED_CREDITS_PER_MSEC; } static bool_t __read_mostly opt_tickle_one_idle = 1; @@ -956,7 +954,7 @@ csched_vcpu_acct(struct csched_private *prv, unsigned int cpu) /* * Update credits */ - burn_credits(svc, NOW()); + burn_credits(svc, tacc_get_guest_time_delta_vcpu(svc->vcpu)); /* * Put this VCPU and domain back on the active list if it was @@ -1856,14 +1854,14 @@ csched_schedule( (unsigned char *)&d); } - runtime = now - current->runstate.state_entry_time; + runtime = tacc_get_guest_time_delta(); if ( runtime < 0 ) /* Does this ever happen? */ runtime = 0; if ( !is_idle_vcpu(scurr->vcpu) ) { /* Update credits of a non-idle VCPU. */ - burn_credits(scurr, now); + burn_credits(scurr, runtime); scurr->start_time -= now; scurr->last_sched_time = now; } From patchwork Wed Sep 11 10:32:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Anisov X-Patchwork-Id: 11140877 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA74617E6 for ; Wed, 11 Sep 2019 10:34:31 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C656E2087E for ; Wed, 11 Sep 2019 10:34:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="uj8hwhqH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C656E2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zw3-0000UL-8N; Wed, 11 Sep 2019 10:33:15 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i7zw1-0000T0-AK for xen-devel@lists.xenproject.org; Wed, 11 Sep 2019 10:33:13 +0000 X-Inumbo-ID: 7b29eefa-d47f-11e9-b76c-bc764e2007e4 Received: from mail-lj1-x244.google.com (unknown [2a00:1450:4864:20::244]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 7b29eefa-d47f-11e9-b76c-bc764e2007e4; Wed, 11 Sep 2019 10:32:41 +0000 (UTC) Received: by mail-lj1-x244.google.com with SMTP id y23so19132831ljn.5 for ; Wed, 11 Sep 2019 03:32:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nTxSIglmusaJTq+slsHa0UgJ/0RrGOlxtblHgFLIsZ4=; b=uj8hwhqHEt/YuJweovSp6KjRqgNBJw149U13EqsUdElkHyDRWKs2ERrR5JC3TyROTE pETd614g2kxfVzS2TX2Vf6MtN+l7zm0DRFgilUicOfsCz4rR5uOVstSqCdZqkcqgfIsr QoiOZHxg6Tr/hdRRqvateyzO2vF39//gz/tdd2nUu62ZQWMdgYCdE5sUjC8mT8LTCj8P aQ3KPkkHQtWLiV6FUWAQD6ChOFGbQTVQxtNfR64FPVy9vYc+HdGOo3JkWyTLh75FnQmD yWh1aIKOG97JEiYgnavoWAG9ZG6loIVH1mV2J2fN/7uBUD3GjHSy7aI5MtDMiy5DW9vC gYbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nTxSIglmusaJTq+slsHa0UgJ/0RrGOlxtblHgFLIsZ4=; b=WdheIRjDD0cjz2fiJxKthDRXZsrwfd+lH2TuLJmmejaOsAsiQdGY0iyFO7zJrn0AEK StmnRtXNqPNXrwI8xIvFbvK76R2HUpOvHOyRnH2sLX3TiK7jR/sgFuCKswb1EYCxZWGg dxeMCHmFcMVZru+mhM/1JoGY+SJZYdTK6YlXfj8IjILTy54i+azrxvSc7NOo7DedLEvI RMfpSURHo+Fq1wcwZFI+/PfqSAV1MVPe+SDdEHjwZ5e2f/04FGpA8vJYl4qbeGs5byiC mGfDcTbQyMi1LUM6CoQVRwRWsIEUx8TAoyGrzbtuyXXg+s58wRSeGekdeHh0MF2hQLLv 1Xug== X-Gm-Message-State: APjAAAUWnbSHnfvHo/izFNwB60MDU88vRGLJ/OaaZdlxxI7KxkboLKRW w4r/BiU7JvlA9OXt02sLa8gXxTYJKYQ= X-Google-Smtp-Source: APXvYqw7Zpe3MGMlHtgBgNuE9VQf7Mufhx7GHZJO8jeINlKDc+9rZNrgrh9XUoy2yHDqq24BiZzGIA== X-Received: by 2002:a2e:88c6:: with SMTP id a6mr23164351ljk.39.1568197959507; Wed, 11 Sep 2019 03:32:39 -0700 (PDT) Received: from aanisov-work.kyiv.epam.com (ll-74.141.223.85.sovam.net.ua. [85.223.141.74]) by smtp.gmail.com with ESMTPSA id w27sm4585275ljd.55.2019.09.11.03.32.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Sep 2019 03:32:38 -0700 (PDT) From: Andrii Anisov To: xen-devel@lists.xenproject.org Date: Wed, 11 Sep 2019 13:32:22 +0300 Message-Id: <1568197942-15374-10-git-send-email-andrii.anisov@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> References: <1568197942-15374-1-git-send-email-andrii.anisov@gmail.com> Subject: [Xen-devel] [RFC 9/9] sched:credit2: get guest time from time accounting code X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Andrii Anisov , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Julien Grall , Jan Beulich , Dario Faggioli MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Andrii Anisov While the Credit2 scheduler code uses guest time from the other pcpu, we have to use locked time accounting. Signed-off-by: Andrii Anisov --- xen/common/Kconfig | 1 + xen/common/sched_credit2.c | 17 +++++++++-------- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/xen/common/Kconfig b/xen/common/Kconfig index d17a8b4..6408c18 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -234,6 +234,7 @@ config SCHED_CREDIT config SCHED_CREDIT2 bool "Credit2 scheduler support" default y + select TACC_NEEDS_LOCK ---help--- The credit2 scheduler is a general purpose scheduler that is optimized for lower latency and higher VM density. diff --git a/xen/common/sched_credit2.c b/xen/common/sched_credit2.c index 6b77da7..3b3888b 100644 --- a/xen/common/sched_credit2.c +++ b/xen/common/sched_credit2.c @@ -1342,7 +1342,7 @@ static inline bool is_preemptable(const struct csched2_vcpu *svc, return true; ASSERT(svc->vcpu->is_running); - return now - svc->vcpu->runstate.state_entry_time > + return tacc_get_guest_time_delta_vcpu(svc->vcpu)> ratelimit - CSCHED2_RATELIMIT_TICKLE_TOLERANCE; } @@ -1722,7 +1722,7 @@ void burn_credits(struct csched2_runqueue_data *rqd, return; } - delta = now - svc->start_time; + delta = tacc_get_guest_time_delta_vcpu(svc->vcpu); if ( unlikely(delta <= 0) ) { @@ -1739,7 +1739,7 @@ void burn_credits(struct csched2_runqueue_data *rqd, if ( has_cap(svc) ) svc->budget -= delta; - svc->start_time = now; + svc->vcpu->pcpu_guest_time += delta; out: if ( unlikely(tb_init_done) ) @@ -3189,8 +3189,8 @@ csched2_runtime(const struct scheduler *ops, int cpu, { s_time_t ratelimit_min = MICROSECS(prv->ratelimit_us); if ( snext->vcpu->is_running ) - ratelimit_min = snext->vcpu->runstate.state_entry_time + - MICROSECS(prv->ratelimit_us) - now; + ratelimit_min = tacc_get_guest_time_delta_vcpu(snext->vcpu) + + MICROSECS(prv->ratelimit_us); if ( ratelimit_min > min_time ) min_time = ratelimit_min; } @@ -3265,6 +3265,7 @@ runq_candidate(struct csched2_runqueue_data *rqd, struct csched2_vcpu *snext = NULL; struct csched2_private *prv = csched2_priv(per_cpu(scheduler, cpu)); bool yield = false, soft_aff_preempt = false; + s_time_t guest_time; *skipped = 0; @@ -3286,8 +3287,8 @@ runq_candidate(struct csched2_runqueue_data *rqd, * no point forcing it to do so until rate limiting expires. */ if ( !yield && prv->ratelimit_us && vcpu_runnable(scurr->vcpu) && - (now - scurr->vcpu->runstate.state_entry_time) < - MICROSECS(prv->ratelimit_us) ) + ((guest_time = tacc_get_guest_time_delta_vcpu(scurr->vcpu)) < + MICROSECS(prv->ratelimit_us))) { if ( unlikely(tb_init_done) ) { @@ -3297,7 +3298,7 @@ runq_candidate(struct csched2_runqueue_data *rqd, } d; d.dom = scurr->vcpu->domain->domain_id; d.vcpu = scurr->vcpu->vcpu_id; - d.runtime = now - scurr->vcpu->runstate.state_entry_time; + d.runtime = guest_time; __trace_var(TRC_CSCHED2_RATELIMIT, 1, sizeof(d), (unsigned char *)&d);