From patchwork Fri Apr 15 11:32:02 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 8850431 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C3F98C0553 for ; Fri, 15 Apr 2016 11:36:46 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C5A2020295 for ; Fri, 15 Apr 2016 11:36:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D55520256 for ; Fri, 15 Apr 2016 11:36:45 +0000 (UTC) Received: from localhost ([::1]:60517 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar23I-0005Fl-Dy for patchwork-qemu-devel@patchwork.kernel.org; Fri, 15 Apr 2016 07:36:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55350) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar1z6-0005rz-Nh for qemu-devel@nongnu.org; Fri, 15 Apr 2016 07:32:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ar1z5-0008RY-H5 for qemu-devel@nongnu.org; Fri, 15 Apr 2016 07:32:24 -0400 Received: from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241]:36057) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ar1z5-0008RO-7M for qemu-devel@nongnu.org; Fri, 15 Apr 2016 07:32:23 -0400 Received: by mail-wm0-x241.google.com with SMTP id l6so5402522wml.3 for ; Fri, 15 Apr 2016 04:32:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=AGiefJ/Ele7/c14HeT8IxY5ks3u7dISy+vFIx00c3YY=; b=ntM8HdwVplKyBpWr7uNsrQGAm7QYV1s7OCJAbz1IjBWJucZYAzn49uO7XIfUv72aAA AI+SZ9ALoUX3Ka/t5fLJ1AfAWH3G4icCgH1lMhr0Y2QphFl49By/q+X6r6GAeGFRyJ0k u6VTbMpHkBkEQ8fKN8SOyd7mO1FBFoAGDvzM0zMOIPm9gUfh3pzbXhB2suux+DJnLmLr ZinigC/PGRi85KOlffpOXEZoTKMpild//Qmv1Z6TKw1d5DKpRtnb2spXSqkktJkfaq0+ JD7DfPezsjz46iOjrGMI9jDco3o+4jREXU3k4zetcauJEamXEev7ayi+dEBQFyTdzwCl UEnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=AGiefJ/Ele7/c14HeT8IxY5ks3u7dISy+vFIx00c3YY=; b=MzYW67wMH6r7hoYtBS+4HhngZekHRY/R/WS58JyiSBnuSA0qWoxPx8Q6ol9z4pzUrz p47pyRpyRjh8TT9neoZfo8pilRx8fHFj1YcNURrqY8zGdwdy/XXcbsHjA31+PiNhHxOr svBZF3dbtwAEnJFPkh6bgemGz+9iQigMqoQccq87YT7UdAk422Mou7wf4aI/vz068b/C rD7jq/iy6TlqDp0WckDRK4+Kjm9TOeyW7wKCteO2iW1wxnHSNTe7Y5Veox8tlHFBv9BN cJsDVD6vt0/6oAxCLg5F9eFDMsIUITGMJv1Ve+yi/BNp0DvqiQAAFRlOFyhyVyNtcxEt wz8Q== X-Gm-Message-State: AOPr4FWcZJF9nJFabI1nGAZ/3U72SmIzf16jl5UOLvijr923WOXhxLKPt8k9kq09PJABKw== X-Received: by 10.194.41.104 with SMTP id e8mr20540501wjl.177.1460719942683; Fri, 15 Apr 2016 04:32:22 -0700 (PDT) Received: from donizetti.redhat.com (94-39-141-76.adsl-ull.clienti.tiscali.it. [94.39.141.76]) by smtp.gmail.com with ESMTPSA id qt3sm15217493wjc.32.2016.04.15.04.32.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Apr 2016 04:32:21 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Fri, 15 Apr 2016 13:32:02 +0200 Message-Id: <1460719926-12950-8-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1460719926-12950-1-git-send-email-pbonzini@redhat.com> References: <1460719926-12950-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::241 Subject: [Qemu-devel] [PATCH 07/11] coroutine-lock: add limited spinning to CoMutex X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, berto@igalia.com, famz@redhat.com, stefanha@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Running a very small critical section on pthread_mutex_t and CoMutex shows that pthread_mutex_t is much faster because it doesn't actually go to sleep. What happens is that the critical section is shorter than the latency of entering the kernel and thus FUTEX_WAIT always fails. With CoMutex there is no such latency but you still want to avoid wait and wakeup. So introduce it artificially. This only works with two waiters; because CoMutex is fair, it will always have more waits and wakeups than a pthread_mutex_t. Signed-off-by: Paolo Bonzini --- include/qemu/coroutine.h | 5 +++++ util/qemu-coroutine-lock.c | 34 ++++++++++++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h index 018a60d..d15a09a 100644 --- a/include/qemu/coroutine.h +++ b/include/qemu/coroutine.h @@ -163,6 +163,11 @@ typedef struct CoMutex { */ unsigned locked; + /* Context that is holding the lock. Useful to avoid spinning + * when two coroutines on the same AioContext try to get the lock. :) + */ + AioContext *ctx; + /* A queue of waiters. Elements are added atomically in front of * from_push. to_pop is only populated, and popped from, by whoever * is in charge of the next wakeup. This can be an unlocker or, diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c index 7ed0f37..aa59e82 100644 --- a/util/qemu-coroutine-lock.c +++ b/util/qemu-coroutine-lock.c @@ -177,18 +177,44 @@ void qemu_co_mutex_init(CoMutex *mutex) void coroutine_fn qemu_co_mutex_lock(CoMutex *mutex) { + AioContext *ctx = qemu_get_current_aio_context(); Coroutine *self = qemu_coroutine_self(); CoWaitRecord w; unsigned old_handoff; + int waiters, i; + + /* Running a very small critical section on pthread_mutex_t and CoMutex + * shows that pthread_mutex_t is much faster because it doesn't actually + * go to sleep. What happens is that the critical section is shorter + * than the latency of entering the kernel and thus FUTEX_WAIT always + * fails. With CoMutex there is no such latency but you still want to + * avoid wait and wakeup. So introduce it artificially. + */ + i = 0; +retry_fast_path: + waiters = atomic_cmpxchg(&mutex->locked, 0, 1); + if (waiters != 0) { + while (waiters == 1 && ++i < 1000) { + if (atomic_read(&mutex->ctx) == ctx) { + break; + } + if (atomic_read(&mutex->locked) == 0) { + goto retry_fast_path; + } + /* cpu_relax(); */ + } + waiters = atomic_fetch_inc(&mutex->locked); + } - if (atomic_fetch_inc(&mutex->locked) == 0) { + if (waiters == 0) { /* Uncontended. */ trace_qemu_co_mutex_lock_uncontended(mutex, self); + mutex->ctx = ctx; return; } trace_qemu_co_mutex_lock_entry(mutex, self); - self->ctx = qemu_get_current_aio_context(); + self->ctx = ctx; w.co = self; push_waiter(mutex, &w); @@ -207,9 +233,11 @@ void coroutine_fn qemu_co_mutex_lock(CoMutex *mutex) if (co == self) { /* We got the lock ourselves! */ assert(to_wake == &w); + mutex->ctx = ctx; return; } + mutex->ctx = co->ctx; qemu_coroutine_wake(co->ctx, co); } @@ -223,6 +251,7 @@ void coroutine_fn qemu_co_mutex_unlock(CoMutex *mutex) trace_qemu_co_mutex_unlock_entry(mutex, self); + mutex->ctx = NULL; assert(mutex->locked); assert(qemu_in_coroutine()); @@ -237,6 +266,7 @@ void coroutine_fn qemu_co_mutex_unlock(CoMutex *mutex) if (to_wake) { Coroutine *co = to_wake->co; + mutex->ctx = co->ctx; qemu_coroutine_wake(co->ctx, co); goto out; }