From patchwork Mon Dec 10 00:46:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 10720453 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96600112E for ; Mon, 10 Dec 2018 00:47:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 85D3F29D75 for ; Mon, 10 Dec 2018 00:47:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7A1A329D9D; Mon, 10 Dec 2018 00:47:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 25B0E29D75 for ; Mon, 10 Dec 2018 00:47:29 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C182221F69E; Sun, 9 Dec 2018 16:47:28 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3E02021F553 for ; Sun, 9 Dec 2018 16:47:26 -0800 (PST) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 5320BAF5D; Mon, 10 Dec 2018 00:47:25 +0000 (UTC) From: NeilBrown To: James Simmons , Oleg Drokin , Andreas Dilger Date: Mon, 10 Dec 2018 11:46:16 +1100 Message-ID: <154440277667.29887.4836855598151465010.stgit@noble> In-Reply-To: <154440246926.29887.1667505755325904791.stgit@noble> References: <154440246926.29887.1667505755325904791.stgit@noble> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 3/4] lustre: use bit-locking in echo_client. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" X-Virus-Scanned: ClamAV using ClamSMTP The ep_lock used by echo client causes lockdep to complain. Multiple locks of the same class are taken concurrently which appear to lockdep to be prone to deadlocking, and can fill up lockdep's fixed size stack for locks. Ass ep_lock is taken on multiple pages always in ascending page order, deadlocks don't happen, so this is a false-positive. The function of the ep_lock is the same as thats for page_lock(), which is implemented as a bit-lock using wait_on_bit(). lockdep cannot see these locks, and doesn't really need to. So convert ep_lock to a simple bit-lock using wait_on_bit for waiting. This provides similar functionality, matches how page_lock() works, and avoids lockdep problems. Signed-off-by: NeilBrown Reviewed-by: James Simmons --- .../staging/lustre/lustre/obdecho/echo_client.c | 29 +++++++++++++------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c index 1ddb4a6dd8f3..887df7ce6b5c 100644 --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c @@ -78,7 +78,7 @@ struct echo_object_conf { struct echo_page { struct cl_page_slice ep_cl; - struct mutex ep_lock; + unsigned long ep_lock; }; struct echo_lock { @@ -217,10 +217,13 @@ static int echo_page_own(const struct lu_env *env, { struct echo_page *ep = cl2echo_page(slice); - if (!nonblock) - mutex_lock(&ep->ep_lock); - else if (!mutex_trylock(&ep->ep_lock)) - return -EAGAIN; + if (nonblock) { + if (test_and_set_bit(0, &ep->ep_lock)) + return -EAGAIN; + } else { + while (test_and_set_bit(0, &ep->ep_lock)) + wait_on_bit(&ep->ep_lock, 0, TASK_UNINTERRUPTIBLE); + } return 0; } @@ -230,8 +233,8 @@ static void echo_page_disown(const struct lu_env *env, { struct echo_page *ep = cl2echo_page(slice); - LASSERT(mutex_is_locked(&ep->ep_lock)); - mutex_unlock(&ep->ep_lock); + LASSERT(test_bit(0, &ep->ep_lock)); + clear_and_wake_up_bit(0, &ep->ep_lock); } static void echo_page_discard(const struct lu_env *env, @@ -244,7 +247,7 @@ static void echo_page_discard(const struct lu_env *env, static int echo_page_is_vmlocked(const struct lu_env *env, const struct cl_page_slice *slice) { - if (mutex_is_locked(&cl2echo_page(slice)->ep_lock)) + if (test_bit(0, &cl2echo_page(slice)->ep_lock)) return -EBUSY; return -ENODATA; } @@ -279,7 +282,7 @@ static int echo_page_print(const struct lu_env *env, struct echo_page *ep = cl2echo_page(slice); (*printer)(env, cookie, LUSTRE_ECHO_CLIENT_NAME "-page@%p %d vm@%p\n", - ep, mutex_is_locked(&ep->ep_lock), + ep, test_bit(0, &ep->ep_lock), slice->cpl_page->cp_vmpage); return 0; } @@ -339,7 +342,13 @@ static int echo_page_init(const struct lu_env *env, struct cl_object *obj, struct echo_object *eco = cl2echo_obj(obj); get_page(page->cp_vmpage); - mutex_init(&ep->ep_lock); + /* + * ep_lock is similar to the lock_page() lock, and + * cannot usefully be monitored by lockdep. + * So just a bit in an "unsigned long" and use the + * wait_on_bit() interface to wait for the bit to be clera. + */ + ep->ep_lock = 0; cl_page_slice_add(page, &ep->ep_cl, obj, index, &echo_page_ops); atomic_inc(&eco->eo_npages); return 0;