From patchwork Thu Dec 1 14:06:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Nicolai_H=C3=A4hnle?= X-Patchwork-Id: 9456243 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F310F60235 for ; Thu, 1 Dec 2016 14:07:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6C5B28469 for ; Thu, 1 Dec 2016 14:07:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB54C284DE; Thu, 1 Dec 2016 14:07:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7FE5628469 for ; Thu, 1 Dec 2016 14:07:46 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F236D6E80E; Thu, 1 Dec 2016 14:07:17 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wj0-x244.google.com (mail-wj0-x244.google.com [IPv6:2a00:1450:400c:c01::244]) by gabe.freedesktop.org (Postfix) with ESMTPS id 10B746E80B for ; Thu, 1 Dec 2016 14:07:15 +0000 (UTC) Received: by mail-wj0-x244.google.com with SMTP id xy5so26516381wjc.1 for ; Thu, 01 Dec 2016 06:07:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZKQ90nldqegiwq5WtrtqSXAn01dIwAGLpFF7qYiX28U=; b=DbGZxtP6sXJ+w+QbJNcPo2MRIYGeVMvJB2WcpwKZZm5Vfgu+ESVqVGqrjQ4lN53uIJ dNxV3mnOXQ2Qbgbv8YiIeKMtgd4Jp8WwneraOQZyi/t+3gHdFJjoNVOepJ4mOzCfg3ye G86RqTp6PeoVxNTQDCAtJRW+Kk4fdM4QNbwcGbQy0myEcdZHNfLMDarrXiVXw1DXKel5 8XtktC1FPjs0R5mDWdkI2/K7Y1HbPkX/kHlGBqkDkMamm+rmYSMnX5Ngo9ob4mZfp4tr rrAh5g8vqLI0Es86A8VIJPnFgsRN5TOZIlqPztLzA3TNtple1QLu0sDNR2kn+BJ6emkP mJeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZKQ90nldqegiwq5WtrtqSXAn01dIwAGLpFF7qYiX28U=; b=JXltdzyJZ7awHhai83gABM3NVJdX0t1lCvp/yZoZteP4fs4yXONl8VHsgoyltbg6sY adHbi6P9Fleapk3LWnmG7f9ZT2vlQHN/xoqSCkrN7ox/ROZMQAGbtHLbR6kqxyMwR8ui XY3QJ6pXftuunHRgoBy8Dv8yM9wue5wY88HNuLZOsOMJz54PZIXXoDMYoW6WF8S2glTd FHsaNmYJUxRsryblyEWwqWft2X5dbbey1hUBqsqeFqwyLoyvpQtobVB9ExmL8YzsN2OY W3HBtw+9g4u8HHhrzTOzSMt0vA0BdmquJHHfXiQ7tEULcvre0pP9xYPbpYJWcJoBhvhR p2GA== X-Gm-Message-State: AKaTC02aDFAbsUEl/p/YWk/vfX06XYmbakErSazt04oIUw7s4IndhrXQQJXYB+6WPjhvMA== X-Received: by 10.194.123.201 with SMTP id mc9mr40695242wjb.47.1480601233174; Thu, 01 Dec 2016 06:07:13 -0800 (PST) Received: from cassiopeia.fritz.box ([2001:a61:116e:7d01:dc12:8e88:ab9d:8694]) by smtp.gmail.com with ESMTPSA id vr9sm411514wjc.35.2016.12.01.06.07.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Dec 2016 06:07:12 -0800 (PST) From: =?UTF-8?q?Nicolai=20H=C3=A4hnle?= To: linux-kernel@vger.kernel.org Subject: [PATCH v2 11/11] [rfc] locking/ww_mutex: Always spin optimistically for the first waiter Date: Thu, 1 Dec 2016 15:06:54 +0100 Message-Id: <1480601214-26583-12-git-send-email-nhaehnle@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1480601214-26583-1-git-send-email-nhaehnle@gmail.com> References: <1480601214-26583-1-git-send-email-nhaehnle@gmail.com> MIME-Version: 1.0 Cc: Maarten Lankhorst , =?UTF-8?q?Nicolai=20H=C3=A4hnle?= , Peter Zijlstra , dri-devel@lists.freedesktop.org, Ingo Molnar X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Nicolai Hähnle Check the current owner's context once against our stamp. If our stamp is lower, we continue to spin optimistically instead of backing off. This is correct with respect to deadlock detection because while the (owner, ww_ctx) pair may re-appear if the owner task manages to unlock and re-acquire the lock while we're spinning, the context could only have been re-initialized with an even higher stamp. We also still detect when we have to back off for other waiters that join the list while we're spinning. But taking the wait_lock in mutex_spin_on_owner feels iffy, even if it is done only once. Median timings taken of a contention-heavy GPU workload: Before: real 0m53.086s user 0m7.360s sys 1m46.204s After: real 0m52.577s user 0m7.544s sys 1m49.200s Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Maarten Lankhorst Cc: Daniel Vetter Cc: Chris Wilson Cc: dri-devel@lists.freedesktop.org Signed-off-by: Nicolai Hähnle --- kernel/locking/mutex.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 38d173c..9216239 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -378,6 +378,28 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, struct mutex_waiter *waiter) { bool ret = true; + struct ww_acquire_ctx *owner_ww_ctx = NULL; + + if (use_ww_ctx && ww_ctx && ww_ctx->acquired > 0) { + struct ww_mutex *ww; + unsigned long flags; + + ww = container_of(lock, struct ww_mutex, base); + + /* + * Check the stamp of the current owner once. This allows us + * to spin optimistically in the case where the current owner + * has a higher stamp than us. + */ + spin_lock_mutex(&lock->wait_lock, flags); + owner_ww_ctx = ww->ctx; + if (owner_ww_ctx && + __ww_mutex_stamp_after(ww_ctx, owner_ww_ctx)) { + spin_unlock_mutex(&lock->wait_lock, flags); + return false; + } + spin_unlock_mutex(&lock->wait_lock, flags); + } rcu_read_lock(); while (__mutex_owner(lock) == owner) { @@ -414,9 +436,16 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, * Check this in every inner iteration because we may * be racing against another thread's ww_mutex_lock. */ - if (ww_ctx->acquired > 0 && READ_ONCE(ww->ctx)) { - ret = false; - break; + if (ww_ctx->acquired > 0) { + struct ww_acquire_ctx *current_ctx; + + current_ctx = READ_ONCE(ww->ctx); + + if (current_ctx && + current_ctx != owner_ww_ctx) { + ret = false; + break; + } } /*