From patchwork Wed Jun 7 16:24:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 9771927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 295566034B for ; Wed, 7 Jun 2017 16:27:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 044E3284E4 for ; Wed, 7 Jun 2017 16:27:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED09128521; Wed, 7 Jun 2017 16:27:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DF8EF2850E for ; Wed, 7 Jun 2017 16:27:04 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dIdko-0003eh-6u; Wed, 07 Jun 2017 16:24:18 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dIdkm-0003eb-Rm for xen-devel@lists.xenproject.org; Wed, 07 Jun 2017 16:24:16 +0000 Received: from [85.158.143.35] by server-3.bemta-6.messagelabs.com id B5/53-03058-0B828395; Wed, 07 Jun 2017 16:24:16 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrFLMWRWlGSWpSXmKPExsVyuP0Ov+56DYt Ig4Wdshbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8b2ribmgn6JirP/2tkaGN8KdzFyckgIGEm8 nfiPqYuRi0NIYCGjxLkH55lAEmwCqhIbrp9iBbFFBBwlVrctBbOZBYokbk37ywhiCwuYSlw6P QHMZgGqX7l3NguIzStgLLHo+HYmiAXyEh0HJrNMYORcwMiwilGjOLWoLLVI18hcL6koMz2jJD cxM0fX0MBMLze1uDgxPTUnMalYLzk/dxMj0F8MQLCDcfHawEOMkhxMSqK8D6QtIoX4kvJTKjM SizPii0pzUosPMcpwcChJ8F5UB8oJFqWmp1akZeYAAwcmLcHBoyTCK6sMlOYtLkjMLc5Mh0id YtTl2LB6/RcmIZa8/LxUKXHeYyAzBECKMkrz4EbAgvgSo6yUMC8j0FFCPAWpRbmZJajyrxjFO RiVhHlrVYGm8GTmlcBtegV0BBPQEXyXTECOKElESEk1MNaE/tfc+6FKz5RXoW8fY3Rx0hRb/r xP51fxTL6bu4P3o9W5RZlMD3q1ttboNCY3BBzdUFnH2MTnxLqafYtOxfoTh3S/Pg8Ru/Tz+OZ bK7uDjs7lm3vKSTgtrOm16nfl/Fo5Q87toQ8O5gnHeM+/dPZYr+gSriLrEqNQnU2Jd05s8P/B Y7XZTImlOCPRUIu5qDgRAKrlCmpdAgAA X-Env-Sender: jgross@suse.com X-Msg-Ref: server-8.tower-21.messagelabs.com!1496852655!72647537!1 X-Originating-IP: [195.135.220.15] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.19; banners=-,-,- X-VirusChecked: Checked Received: (qmail 19405 invoked from network); 7 Jun 2017 16:24:15 -0000 Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by server-8.tower-21.messagelabs.com with DHE-RSA-CAMELLIA256-SHA encrypted SMTP; 7 Jun 2017 16:24:15 -0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id E9492AAB4; Wed, 7 Jun 2017 16:24:14 +0000 (UTC) From: Juergen Gross To: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org Date: Wed, 7 Jun 2017 18:24:12 +0200 Message-Id: <20170607162412.8432-1-jgross@suse.com> X-Mailer: git-send-email 2.12.3 Cc: Juergen Gross , andre.przywara@arm.com, boris.ostrovsky@oracle.com, stable@vger.kernel.org Subject: [Xen-devel] [PATCH] xen: avoid deadlock in xenbus driver X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP There has been a report about a deadlock in the xenbus driver: [ 247.979498] ====================================================== [ 247.985688] WARNING: possible circular locking dependency detected [ 247.991882] 4.12.0-rc4-00022-gc4b25c0 #575 Not tainted [ 247.997040] ------------------------------------------------------ [ 248.003232] xenbus/91 is trying to acquire lock: [ 248.007875] (&u->msgbuffer_mutex){+.+.+.}, at: [] xenbus_dev_queue_reply+0x3c/0x230 [ 248.017163] [ 248.017163] but task is already holding lock: [ 248.023096] (xb_write_mutex){+.+...}, at: [] xenbus_thread+0x5f0/0x798 [ 248.031267] [ 248.031267] which lock already depends on the new lock. [ 248.031267] [ 248.039615] [ 248.039615] the existing dependency chain (in reverse order) is: [ 248.047176] [ 248.047176] -> #1 (xb_write_mutex){+.+...}: [ 248.052943] __lock_acquire+0x1728/0x1778 [ 248.057498] lock_acquire+0xc4/0x288 [ 248.061630] __mutex_lock+0x84/0x868 [ 248.065755] mutex_lock_nested+0x3c/0x50 [ 248.070227] xs_send+0x164/0x1f8 [ 248.074015] xenbus_dev_request_and_reply+0x6c/0x88 [ 248.079427] xenbus_file_write+0x260/0x420 [ 248.084073] __vfs_write+0x48/0x138 [ 248.088113] vfs_write+0xa8/0x1b8 [ 248.091983] SyS_write+0x54/0xb0 [ 248.095768] el0_svc_naked+0x24/0x28 [ 248.099897] [ 248.099897] -> #0 (&u->msgbuffer_mutex){+.+.+.}: [ 248.106088] print_circular_bug+0x80/0x2e0 [ 248.110730] __lock_acquire+0x1768/0x1778 [ 248.115288] lock_acquire+0xc4/0x288 [ 248.119417] __mutex_lock+0x84/0x868 [ 248.123545] mutex_lock_nested+0x3c/0x50 [ 248.128016] xenbus_dev_queue_reply+0x3c/0x230 [ 248.133005] xenbus_thread+0x788/0x798 [ 248.137306] kthread+0x110/0x140 [ 248.141087] ret_from_fork+0x10/0x40 It is rather easy to avoid by dropping xb_write_mutex before calling xenbus_dev_queue_reply(). Fixes fd8aa9095a95c02dcc35540a263267c29b8fda9d ("xen: optimize xenbus driver for multiple concurrent xenstore accesses"). Cc: # 4.11 Reported-by: Andre Przywara Signed-off-by: Juergen Gross --- drivers/xen/xenbus/xenbus_comms.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_comms.c b/drivers/xen/xenbus/xenbus_comms.c index 856ada5d39c9..a44bcdbf6533 100644 --- a/drivers/xen/xenbus/xenbus_comms.c +++ b/drivers/xen/xenbus/xenbus_comms.c @@ -305,18 +305,21 @@ static int process_msg(void) req->body = state.body; req->state = xb_req_state_got_reply; list_del(&req->list); + mutex_unlock(&xb_write_mutex); req->cb(req); } else { list_del(&req->list); + mutex_unlock(&xb_write_mutex); kfree(req); } err = 0; break; } } - mutex_unlock(&xb_write_mutex); - if (err) + if (err) { + mutex_unlock(&xb_write_mutex); goto out; + } } mutex_unlock(&xs_response_mutex);