From patchwork Mon Sep 9 13:13:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Kotov X-Patchwork-Id: 11137929 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4AC1E76 for ; Mon, 9 Sep 2019 13:15:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1CEA82086D for ; Mon, 9 Sep 2019 13:15:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="vVHPELjS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1CEA82086D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:56508 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7JVq-0005Ta-10 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 09 Sep 2019 09:15:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56972) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7JUP-0004Dh-D0 for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:13:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i7JUK-0002Xs-Gg for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:13:49 -0400 Received: from forwardcorp1p.mail.yandex.net ([2a02:6b8:0:1472:2741:0:8b6:217]:48772) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i7JUJ-0002Vd-V1 for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:13:48 -0400 Received: from mxbackcorp2j.mail.yandex.net (mxbackcorp2j.mail.yandex.net [IPv6:2a02:6b8:0:1619::119]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id F13322E14D9; Mon, 9 Sep 2019 16:13:42 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id jy5G6Leivf-DgBa2QBp; Mon, 09 Sep 2019 16:13:42 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1568034822; bh=COgAT6GmThVhhc3naf4NxoYI3s42g79B3gxherqLzf0=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=vVHPELjSUcz1pU3qsGPNlvhXs/lwoD30ljb6uvNSdNlxtttYUxWDHAXv3/6T4C7RN yAXJ9aHJ5PxlaTBT11r9BvrKRUFSnuG6Aur30Y72+/f1l+w3s2ONWLKmCfImgr0Qlp qvz8y3A0QMYMvwDkw0XdzY6hfgU+HBoXwdtw59Rs= Authentication-Results: mxbackcorp2j.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:408:c41:6db7:e9d4:e9e0]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id epZFdTz6L0-Df7CY604; Mon, 09 Sep 2019 16:13:42 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) From: Yury Kotov To: "Dr. David Alan Gilbert" , Eric Blake , Juan Quintela , Laurent Vivier , Paolo Bonzini , Richard Henderson , Stefan Weil , Thomas Huth Date: Mon, 9 Sep 2019 16:13:33 +0300 Message-Id: <20190909131335.16848-2-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190909131335.16848-1-yury-kotov@yandex-team.ru> References: <20190909131335.16848-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a02:6b8:0:1472:2741:0:8b6:217 Subject: [Qemu-devel] [PATCH v7 1/3] qemu-thread: Add qemu_cond_timedwait X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" The new function is needed to implement conditional sleep for CPU throttling. It's possible to reuse qemu_sem_timedwait, but it's more difficult than just add qemu_cond_timedwait. Also moved compute_abs_deadline function up the code to reuse it in qemu_cond_timedwait_impl win32. Signed-off-by: Yury Kotov --- include/qemu/thread.h | 19 +++++++++++++++++++ util/qemu-thread-posix.c | 41 ++++++++++++++++++++++++++++------------ util/qemu-thread-win32.c | 17 +++++++++++++++++ util/qsp.c | 20 ++++++++++++++++++++ 4 files changed, 85 insertions(+), 12 deletions(-) diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 55d83a907c..047db0307e 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -34,6 +34,8 @@ typedef void (*QemuRecMutexLockFunc)(QemuRecMutex *m, const char *f, int l); typedef int (*QemuRecMutexTrylockFunc)(QemuRecMutex *m, const char *f, int l); typedef void (*QemuCondWaitFunc)(QemuCond *c, QemuMutex *m, const char *f, int l); +typedef bool (*QemuCondTimedWaitFunc)(QemuCond *c, QemuMutex *m, int ms, + const char *f, int l); extern QemuMutexLockFunc qemu_bql_mutex_lock_func; extern QemuMutexLockFunc qemu_mutex_lock_func; @@ -41,6 +43,7 @@ extern QemuMutexTrylockFunc qemu_mutex_trylock_func; extern QemuRecMutexLockFunc qemu_rec_mutex_lock_func; extern QemuRecMutexTrylockFunc qemu_rec_mutex_trylock_func; extern QemuCondWaitFunc qemu_cond_wait_func; +extern QemuCondTimedWaitFunc qemu_cond_timedwait_func; /* convenience macros to bypass the profiler */ #define qemu_mutex_lock__raw(m) \ @@ -63,6 +66,8 @@ extern QemuCondWaitFunc qemu_cond_wait_func; qemu_rec_mutex_trylock_impl(m, __FILE__, __LINE__); #define qemu_cond_wait(c, m) \ qemu_cond_wait_impl(c, m, __FILE__, __LINE__); +#define qemu_cond_timedwait(c, m, ms) \ + qemu_cond_wait_impl(c, m, ms, __FILE__, __LINE__); #else #define qemu_mutex_lock(m) ({ \ QemuMutexLockFunc _f = atomic_read(&qemu_mutex_lock_func); \ @@ -89,6 +94,11 @@ extern QemuCondWaitFunc qemu_cond_wait_func; QemuCondWaitFunc _f = atomic_read(&qemu_cond_wait_func); \ _f(c, m, __FILE__, __LINE__); \ }) + +#define qemu_cond_timedwait(c, m, ms) ({ \ + QemuCondTimedWaitFunc _f = atomic_read(&qemu_cond_timedwait_func); \ + _f(c, m, ms, __FILE__, __LINE__); \ + }) #endif #define qemu_mutex_unlock(mutex) \ @@ -134,12 +144,21 @@ void qemu_cond_signal(QemuCond *cond); void qemu_cond_broadcast(QemuCond *cond); void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, const int line); +bool qemu_cond_timedwait_impl(QemuCond *cond, QemuMutex *mutex, int ms, + const char *file, const int line); static inline void (qemu_cond_wait)(QemuCond *cond, QemuMutex *mutex) { qemu_cond_wait(cond, mutex); } +/* Returns true if timeout has not expired, and false otherwise */ +static inline bool (qemu_cond_timedwait)(QemuCond *cond, QemuMutex *mutex, + int ms) +{ + return qemu_cond_timedwait(cond, mutex, ms); +} + void qemu_sem_init(QemuSemaphore *sem, int init); void qemu_sem_post(QemuSemaphore *sem); void qemu_sem_wait(QemuSemaphore *sem); diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c index 1bf5e65dea..838980aaa5 100644 --- a/util/qemu-thread-posix.c +++ b/util/qemu-thread-posix.c @@ -36,6 +36,18 @@ static void error_exit(int err, const char *msg) abort(); } +static void compute_abs_deadline(struct timespec *ts, int ms) +{ + struct timeval tv; + gettimeofday(&tv, NULL); + ts->tv_nsec = tv.tv_usec * 1000 + (ms % 1000) * 1000000; + ts->tv_sec = tv.tv_sec + ms / 1000; + if (ts->tv_nsec >= 1000000000) { + ts->tv_sec++; + ts->tv_nsec -= 1000000000; + } +} + void qemu_mutex_init(QemuMutex *mutex) { int err; @@ -164,6 +176,23 @@ void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, con error_exit(err, __func__); } +bool qemu_cond_timedwait_impl(QemuCond *cond, QemuMutex *mutex, int ms, + const char *file, const int line) +{ + int err; + struct timespec ts; + + assert(cond->initialized); + trace_qemu_mutex_unlock(mutex, file, line); + compute_abs_deadline(&ts, ms); + err = pthread_cond_timedwait(&cond->cond, &mutex->lock, &ts); + trace_qemu_mutex_locked(mutex, file, line); + if (err && err != ETIMEDOUT) { + error_exit(err, __func__); + } + return err != ETIMEDOUT; +} + void qemu_sem_init(QemuSemaphore *sem, int init) { int rc; @@ -238,18 +267,6 @@ void qemu_sem_post(QemuSemaphore *sem) #endif } -static void compute_abs_deadline(struct timespec *ts, int ms) -{ - struct timeval tv; - gettimeofday(&tv, NULL); - ts->tv_nsec = tv.tv_usec * 1000 + (ms % 1000) * 1000000; - ts->tv_sec = tv.tv_sec + ms / 1000; - if (ts->tv_nsec >= 1000000000) { - ts->tv_sec++; - ts->tv_nsec -= 1000000000; - } -} - int qemu_sem_timedwait(QemuSemaphore *sem, int ms) { int rc; diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c index 572f88535d..56a83333da 100644 --- a/util/qemu-thread-win32.c +++ b/util/qemu-thread-win32.c @@ -145,6 +145,23 @@ void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, con qemu_mutex_post_lock(mutex, file, line); } +bool qemu_cond_timedwait_impl(QemuCond *cond, QemuMutex *mutex, int ms, + const char *file, const int line) +{ + int rc = 0; + + assert(cond->initialized); + trace_qemu_mutex_unlock(mutex, file, line); + if (!SleepConditionVariableSRW(&cond->var, &mutex->lock, ms, 0)) { + rc = GetLastError(); + } + trace_qemu_mutex_locked(mutex, file, line); + if (rc && rc != ERROR_TIMEOUT) { + error_exit(rc, __func__); + } + return rc != ERROR_TIMEOUT; +} + void qemu_sem_init(QemuSemaphore *sem, int init) { /* Manual reset. */ diff --git a/util/qsp.c b/util/qsp.c index 5264c97342..62265417fd 100644 --- a/util/qsp.c +++ b/util/qsp.c @@ -131,6 +131,7 @@ QemuRecMutexLockFunc qemu_rec_mutex_lock_func = qemu_rec_mutex_lock_impl; QemuRecMutexTrylockFunc qemu_rec_mutex_trylock_func = qemu_rec_mutex_trylock_impl; QemuCondWaitFunc qemu_cond_wait_func = qemu_cond_wait_impl; +QemuCondTimedWaitFunc qemu_cond_timedwait_func = qemu_cond_timedwait_impl; /* * It pays off to _not_ hash callsite->file; hashing a string is slow, and @@ -412,6 +413,23 @@ qsp_cond_wait(QemuCond *cond, QemuMutex *mutex, const char *file, int line) qsp_entry_record(e, t1 - t0); } +static bool +qsp_cond_timedwait(QemuCond *cond, QemuMutex *mutex, int ms, + const char *file, int line) +{ + QSPEntry *e; + int64_t t0, t1; + bool ret; + + t0 = get_clock(); + ret = qemu_cond_timedwait_impl(cond, mutex, ms, file, line); + t1 = get_clock(); + + e = qsp_entry_get(cond, file, line, QSP_CONDVAR); + qsp_entry_record(e, t1 - t0); + return ret; +} + bool qsp_is_enabled(void) { return atomic_read(&qemu_mutex_lock_func) == qsp_mutex_lock; @@ -425,6 +443,7 @@ void qsp_enable(void) atomic_set(&qemu_rec_mutex_lock_func, qsp_rec_mutex_lock); atomic_set(&qemu_rec_mutex_trylock_func, qsp_rec_mutex_trylock); atomic_set(&qemu_cond_wait_func, qsp_cond_wait); + atomic_set(&qemu_cond_timedwait_func, qsp_cond_timedwait); } void qsp_disable(void) @@ -435,6 +454,7 @@ void qsp_disable(void) atomic_set(&qemu_rec_mutex_lock_func, qemu_rec_mutex_lock_impl); atomic_set(&qemu_rec_mutex_trylock_func, qemu_rec_mutex_trylock_impl); atomic_set(&qemu_cond_wait_func, qemu_cond_wait_impl); + atomic_set(&qemu_cond_timedwait_func, qemu_cond_timedwait_impl); } static gint qsp_tree_cmp(gconstpointer ap, gconstpointer bp, gpointer up) From patchwork Mon Sep 9 13:13:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Kotov X-Patchwork-Id: 11137931 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9380D17E6 for ; Mon, 9 Sep 2019 13:15:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6A6922086D for ; Mon, 9 Sep 2019 13:15:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="1cu+3DFh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A6922086D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:56506 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7JVq-0005Rv-CQ for patchwork-qemu-devel@patchwork.kernel.org; Mon, 09 Sep 2019 09:15:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56987) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7JUP-0004Dk-GQ for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:13:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i7JUO-0002ag-8F for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:13:53 -0400 Received: from forwardcorp1j.mail.yandex.net ([2a02:6b8:0:1619::183]:46168) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i7JUN-0002Yw-UL for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:13:52 -0400 Received: from mxbackcorp1g.mail.yandex.net (mxbackcorp1g.mail.yandex.net [IPv6:2a02:6b8:0:1402::301]) by forwardcorp1j.mail.yandex.net (Yandex) with ESMTP id BEFE52E1481; Mon, 9 Sep 2019 16:13:47 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp1g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id 81RcB9CVoZ-DhCuu20S; Mon, 09 Sep 2019 16:13:47 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1568034827; bh=SDYDVXTpeI0Jj9mPg05YC4j/rBpohEZTKIE7+3MKUQI=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=1cu+3DFhOWbfXjwAq1fhvouOBOIR1oWylTlQVXc8VtFMjj/TSGPVkhizwxaqF4zWM rTiHJENhir3KbD5sOGK78EFIcNgYABWHPiYXYCqw9522rpLykXW2bN40bv2SDavGHM PxvI8rMsnlqAT7dpAHmufLj9ni4ZjA4FWpT0seIk= Authentication-Results: mxbackcorp1g.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:408:c41:6db7:e9d4:e9e0]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id epZFdTz6L0-Dh7CYkbB; Mon, 09 Sep 2019 16:13:43 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) From: Yury Kotov To: "Dr. David Alan Gilbert" , Eric Blake , Juan Quintela , Laurent Vivier , Paolo Bonzini , Richard Henderson , Stefan Weil , Thomas Huth Date: Mon, 9 Sep 2019 16:13:34 +0300 Message-Id: <20190909131335.16848-3-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190909131335.16848-1-yury-kotov@yandex-team.ru> References: <20190909131335.16848-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a02:6b8:0:1619::183 Subject: [Qemu-devel] [PATCH v7 2/3] cpus: Fix throttling during vm_stop X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Throttling thread sleeps in VCPU thread. For high throttle percentage this sleep is more than 10ms. E.g. for 60% - 15ms, for 99% - 990ms. vm_stop() kicks all VCPUs and waits for them. It's called at the end of migration and because of the long sleep the migration downtime might be more than 100ms even for downtime-limit 1ms. Use qemu_cond_timedwait for high percentage to wake up during vm_stop. Signed-off-by: Yury Kotov Reviewed-by: Eric Blake --- cpus.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/cpus.c b/cpus.c index 85cd451a86..d2c61ff155 100644 --- a/cpus.c +++ b/cpus.c @@ -77,6 +77,8 @@ #endif /* CONFIG_LINUX */ +static QemuMutex qemu_global_mutex; + int64_t max_delay; int64_t max_advance; @@ -782,7 +784,7 @@ static void cpu_throttle_thread(CPUState *cpu, run_on_cpu_data opaque) { double pct; double throttle_ratio; - long sleeptime_ns; + int64_t sleeptime_ns, endtime_ns; if (!cpu_throttle_get_percentage()) { return; @@ -790,11 +792,20 @@ static void cpu_throttle_thread(CPUState *cpu, run_on_cpu_data opaque) pct = (double)cpu_throttle_get_percentage()/100; throttle_ratio = pct / (1 - pct); - sleeptime_ns = (long)(throttle_ratio * CPU_THROTTLE_TIMESLICE_NS); - - qemu_mutex_unlock_iothread(); - g_usleep(sleeptime_ns / 1000); /* Convert ns to us for usleep call */ - qemu_mutex_lock_iothread(); + /* Add 1ns to fix double's rounding error (like 0.9999999...) */ + sleeptime_ns = (int64_t)(throttle_ratio * CPU_THROTTLE_TIMESLICE_NS + 1); + endtime_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + sleeptime_ns; + while (sleeptime_ns > 0 && !cpu->stop) { + if (sleeptime_ns > SCALE_MS) { + qemu_cond_timedwait(cpu->halt_cond, &qemu_global_mutex, + sleeptime_ns / SCALE_MS); + } else { + qemu_mutex_unlock_iothread(); + g_usleep(sleeptime_ns / SCALE_US); + qemu_mutex_lock_iothread(); + } + sleeptime_ns = endtime_ns - qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + } atomic_set(&cpu->throttle_thread_scheduled, 0); } @@ -1172,8 +1183,6 @@ static void qemu_init_sigbus(void) } #endif /* !CONFIG_LINUX */ -static QemuMutex qemu_global_mutex; - static QemuThread io_thread; /* cpu creation */ From patchwork Mon Sep 9 13:13:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Kotov X-Patchwork-Id: 11137933 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 529E976 for ; Mon, 9 Sep 2019 13:15:40 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2883B2086D for ; Mon, 9 Sep 2019 13:15:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="FrXNFO0w" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2883B2086D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:56512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7JW6-0005gB-S0 for patchwork-qemu-devel@patchwork.kernel.org; Mon, 09 Sep 2019 09:15:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57034) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i7JUg-0004IO-0V for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:14:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i7JUd-0002hi-O9 for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:14:09 -0400 Received: from forwardcorp1p.mail.yandex.net ([77.88.29.217]:37192) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i7JUW-0002bn-6m for qemu-devel@nongnu.org; Mon, 09 Sep 2019 09:14:06 -0400 Received: from mxbackcorp1j.mail.yandex.net (mxbackcorp1j.mail.yandex.net [IPv6:2a02:6b8:0:1619::162]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 861192E14DB; Mon, 9 Sep 2019 16:13:50 +0300 (MSK) Received: from smtpcorp1j.mail.yandex.net (smtpcorp1j.mail.yandex.net [2a02:6b8:0:1619::137]) by mxbackcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id kWLgziSdme-DmWqBcvd; Mon, 09 Sep 2019 16:13:50 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1568034830; bh=E+K9hcGSlptWefzRbVgBrVGNWSvcs/T5tzmCLKGNSew=; h=In-Reply-To:Message-Id:References:Date:Subject:To:From:Cc; b=FrXNFO0wicqlw0aaklZbHTWyVLGb4Kq3eoxmjtwOzVuy8V3Iw9FsBzH76Tg3cHIE3 VaIzMzqEY6EKD5dZemBOK77q0E/azA4AV2eyJQAbXtEhhcNPl+dT8zegPbU1n+B05y iH5AciyRiyuA1bLK66/AOcHtZ8HJcVN5HIpI6sOQ= Authentication-Results: mxbackcorp1j.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:408:c41:6db7:e9d4:e9e0]) by smtpcorp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id epZFdTz6L0-Dl7CDY2d; Mon, 09 Sep 2019 16:13:47 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) From: Yury Kotov To: "Dr. David Alan Gilbert" , Eric Blake , Juan Quintela , Laurent Vivier , Paolo Bonzini , Richard Henderson , Stefan Weil , Thomas Huth Date: Mon, 9 Sep 2019 16:13:35 +0300 Message-Id: <20190909131335.16848-4-yury-kotov@yandex-team.ru> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190909131335.16848-1-yury-kotov@yandex-team.ru> References: <20190909131335.16848-1-yury-kotov@yandex-team.ru> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 77.88.29.217 Subject: [Qemu-devel] [PATCH v7 3/3] tests/migration: Add a test for auto converge X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, yc-core@yandex-team.ru Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Yury Kotov Reviewed-by: Dr. David Alan Gilbert --- tests/migration-test.c | 130 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 119 insertions(+), 11 deletions(-) diff --git a/tests/migration-test.c b/tests/migration-test.c index a9f81cc185..966d458d72 100644 --- a/tests/migration-test.c +++ b/tests/migration-test.c @@ -240,6 +240,17 @@ static int64_t read_ram_property_int(QTestState *who, const char *property) return result; } +static int64_t read_migrate_property_int(QTestState *who, const char *property) +{ + QDict *rsp_return; + int64_t result; + + rsp_return = migrate_query(who); + result = qdict_get_try_int(rsp_return, property, 0); + qobject_unref(rsp_return); + return result; +} + static uint64_t get_migration_pass(QTestState *who) { return read_ram_property_int(who, "dirty-sync-count"); @@ -254,20 +265,22 @@ static void read_blocktime(QTestState *who) qobject_unref(rsp_return); } +static bool check_migration_status(QTestState *who, const char *status) +{ + bool completed; + char *current_status; + + current_status = migrate_query_status(who); + completed = strcmp(current_status, status) == 0; + g_assert_cmpstr(current_status, !=, "failed"); + g_free(current_status); + return completed; +} + static void wait_for_migration_status(QTestState *who, const char *goal) { - while (true) { - bool completed; - char *status; - - status = migrate_query_status(who); - completed = strcmp(status, goal) == 0; - g_assert_cmpstr(status, !=, "failed"); - g_free(status); - if (completed) { - return; - } + while (!check_migration_status(who, goal)) { usleep(1000); } } @@ -445,6 +458,17 @@ static void migrate_pause(QTestState *who) qobject_unref(rsp); } +static void migrate_continue(QTestState *who, const char *state) +{ + QDict *rsp; + + rsp = wait_command(who, + "{ 'execute': 'migrate-continue'," + " 'arguments': { 'state': %s } }", + state); + qobject_unref(rsp); +} + static void migrate_recover(QTestState *who, const char *uri) { QDict *rsp; @@ -1125,6 +1149,89 @@ static void test_migrate_fd_proto(void) test_migrate_end(from, to, true); } +static void test_migrate_auto_converge(void) +{ + char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs); + QTestState *from, *to; + int64_t remaining, percentage; + + /* + * We want the test to be stable and as fast as possible. + * E.g., with 1Gb/s bandwith migration may pass without throttling, + * so we need to decrease a bandwidth. + */ + const int64_t init_pct = 5, inc_pct = 50, max_pct = 95; + const int64_t max_bandwidth = 400000000; /* ~400Mb/s */ + const int64_t downtime_limit = 250; /* 250ms */ + /* + * We migrate through unix-socket (> 500Mb/s). + * Thus, expected migration speed ~= bandwidth limit (< 500Mb/s). + * So, we can predict expected_threshold + */ + const int64_t expected_threshold = max_bandwidth * downtime_limit / 1000; + + if (test_migrate_start(&from, &to, uri, false, false)) { + return; + } + + migrate_set_capability(from, "auto-converge", true); + migrate_set_parameter_int(from, "cpu-throttle-initial", init_pct); + migrate_set_parameter_int(from, "cpu-throttle-increment", inc_pct); + migrate_set_parameter_int(from, "max-cpu-throttle", max_pct); + + /* + * Set the initial parameters so that the migration could not converge + * without throttling. + */ + migrate_set_parameter_int(from, "downtime-limit", 1); + migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */ + + /* To check remaining size after precopy */ + migrate_set_capability(from, "pause-before-switchover", true); + + /* Wait for the first serial output from the source */ + wait_for_serial("src_serial"); + + migrate(from, uri, "{}"); + + /* Wait for throttling begins */ + percentage = 0; + while (percentage == 0) { + percentage = read_migrate_property_int(from, "cpu-throttle-percentage"); + usleep(100); + g_assert_false(got_stop); + } + /* The first percentage of throttling should be equal to init_pct */ + g_assert_cmpint(percentage, ==, init_pct); + /* Now, when we tested that throttling works, let it converge */ + migrate_set_parameter_int(from, "downtime-limit", downtime_limit); + migrate_set_parameter_int(from, "max-bandwidth", max_bandwidth); + + /* + * Wait for pre-switchover status to check last throttle percentage + * and remaining. These values will be zeroed later + */ + wait_for_migration_status(from, "pre-switchover"); + + /* The final percentage of throttling shouldn't be greater than max_pct */ + percentage = read_migrate_property_int(from, "cpu-throttle-percentage"); + g_assert_cmpint(percentage, <=, max_pct); + + remaining = read_ram_property_int(from, "remaining"); + g_assert_cmpint(remaining, <, expected_threshold); + + migrate_continue(from, "pre-switchover"); + + qtest_qmp_eventwait(to, "RESUME"); + + wait_for_serial("dest_serial"); + wait_for_migration_complete(from); + + g_free(uri); + + test_migrate_end(from, to, true); +} + int main(int argc, char **argv) { char template[] = "/tmp/migration-test-XXXXXX"; @@ -1180,6 +1287,7 @@ int main(int argc, char **argv) /* qtest_add_func("/migration/ignore_shared", test_ignore_shared); */ qtest_add_func("/migration/xbzrle/unix", test_xbzrle_unix); qtest_add_func("/migration/fd_proto", test_migrate_fd_proto); + qtest_add_func("/migration/auto_converge", test_migrate_auto_converge); ret = g_test_run();