From patchwork Fri Feb 6 04:45:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 5788391 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C108CBF440 for ; Fri, 6 Feb 2015 04:45:22 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E33BB20219 for ; Fri, 6 Feb 2015 04:45:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0B42E20222 for ; Fri, 6 Feb 2015 04:45:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753747AbbBFEpS (ORCPT ); Thu, 5 Feb 2015 23:45:18 -0500 Received: from mail-ig0-f179.google.com ([209.85.213.179]:63650 "EHLO mail-ig0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754067AbbBFEpO (ORCPT ); Thu, 5 Feb 2015 23:45:14 -0500 Received: by mail-ig0-f179.google.com with SMTP id l13so5404818iga.0 for ; Thu, 05 Feb 2015 20:45:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=p6FROqTCm6iOaWjaiijD0xIYKaRViDqshuEvkkafvGg=; b=XHhusCZshpJiox7AGtpDcGHOw/tip1+K2TMZeoML37TkhtN111kBWuq++BTPAKr+bU wjAr6qtDi2uu3Az04Ub8bnJipuylhl8nYlQVV0Xct8RL1XyFmfSwpAubAuo1pViAiIHn KqP8kwHRQyAlpxmgVF/ys2/ezRcsohulA3/QDjhXro3kSd8FfakTu+iIhcW0unEfPd8S 9IGACSwQ7WlDQRGbvmtDwMfUFjzNv1wMYVN6EPWZYHn5asfioz0UBVf2SutMHMncHeGd TFVV8OXmIBplJ8cMx8ulHQL16fgd2+0YlPva3n+P1cKWDZbbQAXgg74qkYZ+deEdi3e6 pirA== X-Gm-Message-State: ALoCoQlfn0ilm9CzzuNcAeA2kUXZcaNsoCwJyGZ7aUsyVwGBBwNfjT5WJRqnMU0sxBlptFJ2/QEc X-Received: by 10.107.17.142 with SMTP id 14mr8796809ior.69.1423197913476; Thu, 05 Feb 2015 20:45:13 -0800 (PST) Received: from leira.trondhjem.org.localdomain (c-68-40-185-14.hsd1.mi.comcast.net. [68.40.185.14]) by mx.google.com with ESMTPSA id g11sm699830iod.28.2015.02.05.20.45.12 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Feb 2015 20:45:12 -0800 (PST) From: Trond Myklebust To: linux-nfs@vger.kernel.org Cc: Peng Tao Subject: [PATCH v3 5/5] NFSv4.1: Fix pnfs_put_lseg races Date: Thu, 5 Feb 2015 23:45:07 -0500 Message-Id: <1423197907-75541-5-git-send-email-trond.myklebust@primarydata.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1423197907-75541-4-git-send-email-trond.myklebust@primarydata.com> References: <1423197907-75541-1-git-send-email-trond.myklebust@primarydata.com> <1423197907-75541-2-git-send-email-trond.myklebust@primarydata.com> <1423197907-75541-3-git-send-email-trond.myklebust@primarydata.com> <1423197907-75541-4-git-send-email-trond.myklebust@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP pnfs_layoutreturn_free_lseg_async() can also race with inode put in the general case. We can now fix this, and also simplify the code. Cc: Peng Tao Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.c | 53 +++++++++++++++++++---------------------------------- 1 file changed, 19 insertions(+), 34 deletions(-) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index a1d8620e8cb7..107b321be7d4 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -361,14 +361,9 @@ pnfs_layout_need_return(struct pnfs_layout_hdr *lo, return true; } -static void pnfs_layoutreturn_free_lseg(struct work_struct *work) +static void pnfs_layoutreturn_before_put_lseg(struct pnfs_layout_segment *lseg, + struct pnfs_layout_hdr *lo, struct inode *inode) { - struct pnfs_layout_segment *lseg; - struct pnfs_layout_hdr *lo; - struct inode *inode; - - lseg = container_of(work, struct pnfs_layout_segment, pls_work); - WARN_ON(atomic_read(&lseg->pls_refcount)); lo = lseg->pls_layout; inode = lo->plh_inode; @@ -383,24 +378,12 @@ static void pnfs_layoutreturn_free_lseg(struct work_struct *work) lo->plh_block_lgets++; lo->plh_return_iomode = 0; spin_unlock(&inode->i_lock); + pnfs_get_layout_hdr(lo); - pnfs_send_layoutreturn(lo, stateid, iomode, true); - spin_lock(&inode->i_lock); + /* Send an async layoutreturn so we dont deadlock */ + pnfs_send_layoutreturn(lo, stateid, iomode, false); } else - /* match pnfs_get_layout_hdr #2 in pnfs_put_lseg */ - pnfs_put_layout_hdr(lo); - pnfs_layout_remove_lseg(lo, lseg); - spin_unlock(&inode->i_lock); - pnfs_free_lseg(lseg); - /* match pnfs_get_layout_hdr #1 in pnfs_put_lseg */ - pnfs_put_layout_hdr(lo); -} - -static void -pnfs_layoutreturn_free_lseg_async(struct pnfs_layout_segment *lseg) -{ - INIT_WORK(&lseg->pls_work, pnfs_layoutreturn_free_lseg); - queue_work(nfsiod_workqueue, &lseg->pls_work); + spin_unlock(&inode->i_lock); } void @@ -415,21 +398,23 @@ pnfs_put_lseg(struct pnfs_layout_segment *lseg) dprintk("%s: lseg %p ref %d valid %d\n", __func__, lseg, atomic_read(&lseg->pls_refcount), test_bit(NFS_LSEG_VALID, &lseg->pls_flags)); + + /* Handle the case where refcount != 1 */ + if (atomic_add_unless(&lseg->pls_refcount, -1, 1)) + return; + lo = lseg->pls_layout; inode = lo->plh_inode; + /* Do we need a layoutreturn? */ + if (test_bit(NFS_LSEG_LAYOUTRETURN, &lseg->pls_flags)) + pnfs_layoutreturn_before_put_lseg(lseg, lo, inode); + if (atomic_dec_and_lock(&lseg->pls_refcount, &inode->i_lock)) { pnfs_get_layout_hdr(lo); - if (pnfs_layout_need_return(lo, lseg)) { - spin_unlock(&inode->i_lock); - /* hdr reference dropped in nfs4_layoutreturn_release */ - pnfs_get_layout_hdr(lo); - pnfs_layoutreturn_free_lseg_async(lseg); - } else { - pnfs_layout_remove_lseg(lo, lseg); - spin_unlock(&inode->i_lock); - pnfs_free_lseg(lseg); - pnfs_put_layout_hdr(lo); - } + pnfs_layout_remove_lseg(lo, lseg); + spin_unlock(&inode->i_lock); + pnfs_free_lseg(lseg); + pnfs_put_layout_hdr(lo); } } EXPORT_SYMBOL_GPL(pnfs_put_lseg);