From patchwork Fri Apr 15 11:32:03 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 8850451 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id C56E29F54F for ; Fri, 15 Apr 2016 11:38:49 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id DF558202A1 for ; Fri, 15 Apr 2016 11:38:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0DC3E20256 for ; Fri, 15 Apr 2016 11:38:48 +0000 (UTC) Received: from localhost ([::1]:60565 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar25H-0000Ki-BQ for patchwork-qemu-devel@patchwork.kernel.org; Fri, 15 Apr 2016 07:38:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55613) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar1zC-00064x-Ce for qemu-devel@nongnu.org; Fri, 15 Apr 2016 07:32:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ar1z6-0008TK-U8 for qemu-devel@nongnu.org; Fri, 15 Apr 2016 07:32:30 -0400 Received: from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241]:36062) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar1z6-0008S5-GM for qemu-devel@nongnu.org; Fri, 15 Apr 2016 07:32:24 -0400 Received: by mail-wm0-x241.google.com with SMTP id l6so5402695wml.3 for ; Fri, 15 Apr 2016 04:32:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=oSRhgxu683t4O9/P5+Li3+CtOAyUGjLl3J04/6pwzBI=; b=Q3pNtTDjpZPG04D3YoMsgLmLfba1xxtzPpouzhRmmTPPm9/+em90jr+iSIceiUCDfd z6+JeoamFKP17B438FJ7uaQOU0fAXlDvhQZSKzoSQon4JSQGHpXGox5YK+EFocRRcQlx ZlUhKCdyjl/pFY7YS4nENJIzZQaeZz2xcdnKlfaaZaRKPX/RUeKI+5u2p3VJ/o5wxrmN cakOxmkxNUePU7xelxRijV6Hp0yWBDnt+hyM1OGRe+idb6atl36TpSrT4/48zH/xjFDa 55LvN4EP/niLkvIXx8Zv6iLAtja9uUxzbBbtL6Hq+wbz+fvQtdALvA/ByORF54GnfQEZ d6fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=oSRhgxu683t4O9/P5+Li3+CtOAyUGjLl3J04/6pwzBI=; b=WZEfWdBBGulSJR+1jLqcierFYtDTtpaxLn2qID+tVKA4qcefylL/cWV6MGxU8PQR/x wYqWRkGkj1Z0KKDga3UoUWHXNOOP+WW+9+DF5iGYDDgeV/yByX9eQYK2KO4pQhVdrvV/ QvYIuGreSWfU2DPrVXu8ApnL9jMZv0Fde8cmy6vX0+TQdba6S+dqITK7vjYYYSmKZVHH 4OyGADVQIR46SPS4MESZyV+3vVrXYJEyRawIyEsn9tXafrUNR07YHtgTotQRor97a1zY C7XxwX1YH2735YL3/yE1DrkdGWZUicWt0hIbO3QtEZ8bMs1zW1Zc3ZPqNc57a2mhF9xr q/iQ== X-Gm-Message-State: AOPr4FV2SxiLpg7fnJ3DCU0WN7zvSG54OnAEE5JgT3CT6NNMv6DD6ZjKG2pjW7yoStgKOw== X-Received: by 10.28.3.138 with SMTP id 132mr3848026wmd.20.1460719943957; Fri, 15 Apr 2016 04:32:23 -0700 (PDT) Received: from donizetti.redhat.com (94-39-141-76.adsl-ull.clienti.tiscali.it. [94.39.141.76]) by smtp.gmail.com with ESMTPSA id qt3sm15217493wjc.32.2016.04.15.04.32.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Apr 2016 04:32:23 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Fri, 15 Apr 2016 13:32:03 +0200 Message-Id: <1460719926-12950-9-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1460719926-12950-1-git-send-email-pbonzini@redhat.com> References: <1460719926-12950-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::241 Subject: [Qemu-devel] [PATCH 08/11] test-aio-multithread: add performance comparison with thread-based mutexes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, berto@igalia.com, famz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add two implementations of the same benchmark as the previous patch, but using pthreads. One uses a normal QemuMutex, the other is Linux only and implements a fair mutex based on MCS locks and futexes. This shows that the slower performance of the 5-thread case is due to the fairness of CoMutex, rather than to coroutines. If fairness does not matter, as is the case with two threads, CoMutex can actually be faster than pthreads. Signed-off-by: Paolo Bonzini --- tests/test-aio-multithread.c | 152 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 152 insertions(+) diff --git a/tests/test-aio-multithread.c b/tests/test-aio-multithread.c index 18b3548..d7bc1bf 100644 --- a/tests/test-aio-multithread.c +++ b/tests/test-aio-multithread.c @@ -279,6 +279,150 @@ static void test_multi_co_mutex_2_30(void) test_multi_co_mutex(2, 30); } +/* Same test with fair mutexes, for performance comparison. */ + +#ifdef CONFIG_LINUX +#include "qemu/futex.h" + +/* The nodes for the mutex reside in this structure (on which we try to avoid + * false sharing). The head of the mutex is in the "mutex_head" variable. + */ +static struct { + int next, locked; + int padding[14]; +} nodes[NUM_CONTEXTS] __attribute__((__aligned__(64))); + +static int mutex_head = -1; + +static void mcs_mutex_lock(void) +{ + int prev; + + nodes[id].next = -1; + nodes[id].locked = 1; + prev = atomic_xchg(&mutex_head, id); + if (prev != -1) { + atomic_set(&nodes[prev].next, id); + futex_wait(&nodes[id].locked, 1); + } +} + +static void mcs_mutex_unlock(void) +{ + int next; + if (nodes[id].next == -1) { + if (atomic_read(&mutex_head) == id && + atomic_cmpxchg(&mutex_head, id, -1) == id) { + /* Last item in the list, exit. */ + return; + } + while (atomic_read(&nodes[id].next) == -1) { + /* Spin... */ + } + } + + /* Wake up the next in line. */ + next = nodes[id].next; + nodes[next].locked = 0; + futex_wake(&nodes[next].locked, 1); +} + +static void test_multi_fair_mutex_entry(void *opaque) +{ + while (!atomic_mb_read(&now_stopping)) { + mcs_mutex_lock(); + counter++; + mcs_mutex_unlock(); + atomic_inc(&atomic_counter); + } + +} + +static void test_multi_fair_mutex(int threads, int seconds) +{ + int i; + + assert(mutex_head == -1); + counter = 0; + atomic_counter = 0; + now_stopping = false; + + create_aio_contexts(); + assert(threads <= NUM_CONTEXTS); + for (i = 0; i < threads; i++) { + Coroutine *co1 = qemu_coroutine_create(test_multi_fair_mutex_entry); + aio_co_schedule(ctx[i], co1); + } + + g_usleep(seconds * 1000000); + + atomic_mb_set(&now_stopping, true); + join_aio_contexts(); + g_test_message("%d iterations/second\n", counter / seconds); + g_assert_cmpint(counter, ==, atomic_counter); +} + +static void test_multi_fair_mutex_1(void) +{ + test_multi_fair_mutex(NUM_CONTEXTS, 1); +} + +static void test_multi_fair_mutex_10(void) +{ + test_multi_fair_mutex(NUM_CONTEXTS, 10); +} +#endif + +/* Same test with pthread mutexes, for performance comparison and + * portability. */ + +static QemuMutex mutex; + +static void test_multi_mutex_entry(void *opaque) +{ + while (!atomic_mb_read(&now_stopping)) { + qemu_mutex_lock(&mutex); + counter++; + qemu_mutex_unlock(&mutex); + atomic_inc(&atomic_counter); + } + +} + +static void test_multi_mutex(int threads, int seconds) +{ + int i; + + qemu_mutex_init(&mutex); + counter = 0; + atomic_counter = 0; + now_stopping = false; + + create_aio_contexts(); + assert(threads <= NUM_CONTEXTS); + for (i = 0; i < threads; i++) { + Coroutine *co1 = qemu_coroutine_create(test_multi_mutex_entry); + aio_co_schedule(ctx[i], co1); + } + + g_usleep(seconds * 1000000); + + atomic_mb_set(&now_stopping, true); + join_aio_contexts(); + g_test_message("%d iterations/second\n", counter / seconds); + g_assert_cmpint(counter, ==, atomic_counter); +} + +static void test_multi_mutex_1(void) +{ + test_multi_mutex(NUM_CONTEXTS, 1); +} + +static void test_multi_mutex_10(void) +{ + test_multi_mutex(NUM_CONTEXTS, 10); +} + /* End of tests. */ int main(int argc, char **argv) @@ -291,10 +435,18 @@ int main(int argc, char **argv) g_test_add_func("/aio/multi/schedule", test_multi_co_schedule_1); g_test_add_func("/aio/multi/mutex", test_multi_co_mutex_1); g_test_add_func("/aio/multi/mutex/handoff", test_multi_co_mutex_2_3); +#ifdef CONFIG_LINUX + g_test_add_func("/aio/multi/mutex/mcs", test_multi_fair_mutex_1); +#endif + g_test_add_func("/aio/multi/mutex/pthread", test_multi_mutex_1); } else { g_test_add_func("/aio/multi/schedule", test_multi_co_schedule_10); g_test_add_func("/aio/multi/mutex", test_multi_co_mutex_10); g_test_add_func("/aio/multi/mutex/handoff", test_multi_co_mutex_2_30); +#ifdef CONFIG_LINUX + g_test_add_func("/aio/multi/mutex/mcs", test_multi_fair_mutex_10); +#endif + g_test_add_func("/aio/multi/mutex/pthread", test_multi_mutex_10); } return g_test_run(); }