From patchwork Wed Dec 21 18:46:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Nicolai_H=C3=A4hnle?= X-Patchwork-Id: 9483633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 72B78601D3 for ; Wed, 21 Dec 2016 18:47:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6565428419 for ; Wed, 21 Dec 2016 18:47:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 58D9228458; Wed, 21 Dec 2016 18:47:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E560C28419 for ; Wed, 21 Dec 2016 18:47:10 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0A0D36ECB7; Wed, 21 Dec 2016 18:47:02 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wj0-x242.google.com (mail-wj0-x242.google.com [IPv6:2a00:1450:400c:c01::242]) by gabe.freedesktop.org (Postfix) with ESMTPS id 437E36ECB7 for ; Wed, 21 Dec 2016 18:46:59 +0000 (UTC) Received: by mail-wj0-x242.google.com with SMTP id kp2so33294670wjc.0 for ; Wed, 21 Dec 2016 10:46:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HYrAPxKSzXiTumpbVI5momN1ojIMZ+WPX9yw7c4H5S8=; b=QNoxViC35+2LnslAU+KrVL+egYQ4vWktEsQqqP5W2xdF2JV4RiqcMdEBdV83SvRlh0 KzhDTpIFeJiEZrfpn18r62t7gSCOzuJpnwFxyJk2HuyZtaIpE+Ek8xvraNYtJcPXI/K6 Jf/KTiKsZMIi/iPP5h3of3MdQrEobysf/N8oAVh4cssOuK4s6L1KWqLdRoRRQhLA8VjE Tj1A0EHAfy2tDeBZgvSCynG2rhXrP6jq3M3ONxP1rg0d07DcKmU7D3vUP9a6qAfV96SW tzbpHdcC3SFQLh1fKgATY8xKw9+gvalNKLd6VjQ7Sx7IkzQmzYzvEJFQhi/6WeT25Jx4 0N4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HYrAPxKSzXiTumpbVI5momN1ojIMZ+WPX9yw7c4H5S8=; b=DbwxGPMJJKWIII9ZCRDFrJpqu4HF8CFz7RX5T9H/61955GcIIj2NRJGDnfuVEF5C/y Iz5GCInckF/0GDbyJLC89ifo03nm0gw5zBjwfzwa2nFlaFb09f8JY2DU9K5YewY7mtWL 65hLCQ4CjSHgYzgr3lG3j9kEN4/tCH4bFEw0GdNYQ2JSnkQaAqsh+N8ND7K0vSWjPVtV Kz2GoXXSdqkm3zSh4US+hyX/T7BztQGM6hcrhfivUvFmUWFXaITU0dl56H0xt6bZbhfb fkAj+/UQNU1AI0TZFpFSmd3YUlrUcAoWXYhTRm1FO3Qt42ErukKEQ3csMdNdKUz7TSzn N/YQ== X-Gm-Message-State: AIkVDXIucZf3lOvf3R0OOjK2+y/mwo8Wa9C36Mlvt9LLhnu+BTRDU/6ZYhJmZpjo4khV9Q== X-Received: by 10.194.145.83 with SMTP id ss19mr6396477wjb.67.1482346017616; Wed, 21 Dec 2016 10:46:57 -0800 (PST) Received: from cassiopeia.fritz.box ([2001:a61:110a:1701:2511:6687:1ba5:b45b]) by smtp.gmail.com with ESMTPSA id ke6sm31840526wjb.21.2016.12.21.10.46.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 21 Dec 2016 10:46:57 -0800 (PST) From: =?UTF-8?q?Nicolai=20H=C3=A4hnle?= To: linux-kernel@vger.kernel.org Subject: [PATCH v3 09/12] locking/ww_mutex: Re-check ww->ctx in the inner optimistic spin loop Date: Wed, 21 Dec 2016 19:46:37 +0100 Message-Id: <1482346000-9927-10-git-send-email-nhaehnle@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1482346000-9927-1-git-send-email-nhaehnle@gmail.com> References: <1482346000-9927-1-git-send-email-nhaehnle@gmail.com> MIME-Version: 1.0 Cc: Maarten Lankhorst , =?UTF-8?q?Nicolai=20H=C3=A4hnle?= , Peter Zijlstra , dri-devel@lists.freedesktop.org, Ingo Molnar X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Nicolai Hähnle In the following scenario, thread #1 should back off its attempt to lock ww1 and unlock ww2 (assuming the acquire context stamps are ordered accordingly). Thread #0 Thread #1 --------- --------- successfully lock ww2 set ww1->base.owner attempt to lock ww1 confirm ww1->ctx == NULL enter mutex_spin_on_owner set ww1->ctx What was likely to happen previously is: attempt to lock ww2 refuse to spin because ww2->ctx != NULL schedule() detect thread #0 is off CPU stop optimistic spin return -EDEADLK unlock ww2 wakeup thread #0 lock ww2 Now, we are more likely to see: detect ww1->ctx != NULL stop optimistic spin return -EDEADLK unlock ww2 successfully lock ww2 ... because thread #1 will stop its optimistic spin as soon as possible. The whole scenario is quite unlikely, since it requires thread #1 to get between thread #0 setting the owner and setting the ctx. But since we're idling here anyway, the additional check is basically free. Found by inspection. Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Maarten Lankhorst Cc: Daniel Vetter Cc: Chris Wilson Cc: dri-devel@lists.freedesktop.org Signed-off-by: Nicolai Hähnle --- kernel/locking/mutex.c | 44 ++++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index c3f70dd..6f62695 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -373,7 +373,8 @@ ww_mutex_set_context_slowpath(struct ww_mutex *lock, * access and not reliable. */ static noinline -bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner) +bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner, + bool use_ww_ctx, struct ww_acquire_ctx *ww_ctx) { bool ret = true; @@ -396,6 +397,28 @@ bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner) break; } + if (use_ww_ctx && ww_ctx && ww_ctx->acquired > 0) { + struct ww_mutex *ww; + + ww = container_of(lock, struct ww_mutex, base); + + /* + * If ww->ctx is set the contents are undefined, only + * by acquiring wait_lock there is a guarantee that + * they are not invalid when reading. + * + * As such, when deadlock detection needs to be + * performed the optimistic spinning cannot be done. + * + * Check this in every inner iteration because we may + * be racing against another thread's ww_mutex_lock. + */ + if (READ_ONCE(ww->ctx)) { + ret = false; + break; + } + } + cpu_relax(); } rcu_read_unlock(); @@ -483,22 +506,6 @@ static bool mutex_optimistic_spin(struct mutex *lock, for (;;) { struct task_struct *owner; - if (use_ww_ctx && ww_ctx && ww_ctx->acquired > 0) { - struct ww_mutex *ww; - - ww = container_of(lock, struct ww_mutex, base); - /* - * If ww->ctx is set the contents are undefined, only - * by acquiring wait_lock there is a guarantee that - * they are not invalid when reading. - * - * As such, when deadlock detection needs to be - * performed the optimistic spinning cannot be done. - */ - if (READ_ONCE(ww->ctx)) - goto fail_unlock; - } - /* * If there's an owner, wait for it to either * release the lock or go to sleep. @@ -510,7 +517,8 @@ static bool mutex_optimistic_spin(struct mutex *lock, break; } - if (!mutex_spin_on_owner(lock, owner)) + if (!mutex_spin_on_owner(lock, owner, use_ww_ctx, + ww_ctx)) goto fail_unlock; }