From patchwork Wed Jul 19 22:09:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 9853267 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B46C7602BD for ; Wed, 19 Jul 2017 22:10:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A5A8028700 for ; Wed, 19 Jul 2017 22:10:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9945B28717; Wed, 19 Jul 2017 22:10:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 367C428700 for ; Wed, 19 Jul 2017 22:10:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754584AbdGSWKJ (ORCPT ); Wed, 19 Jul 2017 18:10:09 -0400 Received: from mail-it0-f65.google.com ([209.85.214.65]:32994 "EHLO mail-it0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753904AbdGSWKI (ORCPT ); Wed, 19 Jul 2017 18:10:08 -0400 Received: by mail-it0-f65.google.com with SMTP id t6so860436itb.0 for ; Wed, 19 Jul 2017 15:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:in-reply-to:references; bh=Cr9mk9+z6iZ3p4C6EZvFVIHsPUmXJsgZKdKwsxPrMGQ=; b=Hy8LQGEX6iH1lj0t8DvWVgJ47UYCkImOjipiCkIrp/cAhO4DkvUgNZUtP741b/OvG7 6OG1OKlSMvjTJKSHbTkyX9J2LLosWg3cpf4EQytHfxC33iF+kRovQm9FIkg8oWM+gZGQ q3MPQmRSsQlqpTJv4BU5mdX+YHwWqhoBWPPsmhYRbnFGImq8gd4CCTzxmDFJpVT/Qt3E 6gfHSW14mhekNNABcOLFYoYUtmSLFVp1Ncl9AzxZyPWMluf/VrsqXwubda4OVyx79Jjl egYm7qvosaTgIV6ZPzxxh0rpbqmnGxdcH5CIP/UmYlRbRi5itqLvRzUQg8sdftBbCjBL moRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :in-reply-to:references; bh=Cr9mk9+z6iZ3p4C6EZvFVIHsPUmXJsgZKdKwsxPrMGQ=; b=lgNpk1grdnIasKwuXlj0+Q4dsAoXwTJlLINatD+1/wHw9hLkSBicMWWlenkVcfRAXh XiQWN33adN7wcDXUSd8Kg8OuVc9Y/mdM6oihs4G6tpSRm/C43ffYXcvFsMsMGArzQeZw H0xridmwa/nc5XdjeKty9cYCrKe8EyGwd5Ihc/3Za7YqW19Ammjn8Yn9E1oRb+QbFQ3q YvvMlXRZdL1G9oSWweWv6og2sK5e/nxNuh0XwD0X/ycnL/fXd3OJHzImj/5JCtx7HnnZ PCYa+gAU8z6wPYtBAv3amCtXVuxEibajUpYh4VJuZ2hSpQsZlT6ucIOCrCVPJr5dm5rK kS2g== X-Gm-Message-State: AIVw111JH/daSYtkzOPEYb9bdaQeplrLG4D2Z6fTo3tuWM8qJ1MEgjat n7uhmKAWBlGlGaV4bOk= X-Received: by 10.36.222.137 with SMTP id d131mr1393814itg.47.1500502207921; Wed, 19 Jul 2017 15:10:07 -0700 (PDT) Received: from leira.trondhjem.org (c-68-49-162-121.hsd1.mi.comcast.net. [68.49.162.121]) by smtp.gmail.com with ESMTPSA id 9sm518333iop.9.2017.07.19.15.10.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 19 Jul 2017 15:10:07 -0700 (PDT) From: Trond Myklebust To: Chuck Lever , linux-nfs@vger.kernel.org Subject: [PATCH 06/20] NFS: Fix an ABBA issue in nfs_lock_and_join_requests() Date: Wed, 19 Jul 2017 18:09:41 -0400 Message-Id: <20170719220955.58210-7-trond.myklebust@primarydata.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170719220955.58210-6-trond.myklebust@primarydata.com> References: <20170719220955.58210-1-trond.myklebust@primarydata.com> <20170719220955.58210-2-trond.myklebust@primarydata.com> <20170719220955.58210-3-trond.myklebust@primarydata.com> <20170719220955.58210-4-trond.myklebust@primarydata.com> <20170719220955.58210-5-trond.myklebust@primarydata.com> <20170719220955.58210-6-trond.myklebust@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP All other callers of nfs_page_group_lock() appear to already hold the page lock on the head page, so doing it in the opposite order here is inefficient, although not deadlock prone since we roll back all locks on contention. Signed-off-by: Trond Myklebust --- fs/nfs/write.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 1ca759719429..c940e615f5dc 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -383,7 +383,7 @@ nfs_unroll_locks_and_wait(struct inode *inode, struct nfs_page *head, int ret; /* relinquish all the locks successfully grabbed this run */ - for (tmp = head ; tmp != req; tmp = tmp->wb_this_page) + for (tmp = head->wb_this_page ; tmp != req; tmp = tmp->wb_this_page) nfs_unlock_request(tmp); WARN_ON_ONCE(test_bit(PG_TEARDOWN, &req->wb_flags)); @@ -395,7 +395,7 @@ nfs_unroll_locks_and_wait(struct inode *inode, struct nfs_page *head, spin_unlock(&inode->i_lock); /* release ref from nfs_page_find_head_request_locked */ - nfs_release_request(head); + nfs_unlock_and_release_request(head); ret = nfs_wait_on_request(req); nfs_release_request(req); @@ -484,10 +484,6 @@ nfs_lock_and_join_requests(struct page *page) int ret; try_again: - total_bytes = 0; - - WARN_ON_ONCE(destroy_list); - spin_lock(&inode->i_lock); /* @@ -502,6 +498,16 @@ nfs_lock_and_join_requests(struct page *page) return NULL; } + /* lock the page head first in order to avoid an ABBA inefficiency */ + if (!nfs_lock_request(head)) { + spin_unlock(&inode->i_lock); + ret = nfs_wait_on_request(head); + nfs_release_request(head); + if (ret < 0) + return ERR_PTR(ret); + goto try_again; + } + /* holding inode lock, so always make a non-blocking call to try the * page group lock */ ret = nfs_page_group_lock(head, true); @@ -509,13 +515,14 @@ nfs_lock_and_join_requests(struct page *page) spin_unlock(&inode->i_lock); nfs_page_group_lock_wait(head); - nfs_release_request(head); + nfs_unlock_and_release_request(head); goto try_again; } /* lock each request in the page group */ - subreq = head; - do { + total_bytes = head->wb_bytes; + for (subreq = head->wb_this_page; subreq != head; + subreq = subreq->wb_this_page) { /* * Subrequests are always contiguous, non overlapping * and in order - but may be repeated (mirrored writes). @@ -541,9 +548,7 @@ nfs_lock_and_join_requests(struct page *page) return ERR_PTR(ret); } - - subreq = subreq->wb_this_page; - } while (subreq != head); + } /* Now that all requests are locked, make sure they aren't on any list. * Commit list removal accounting is done after locks are dropped */