From patchwork Wed Jul 30 01:34:13 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 4644091 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 20491C0338 for ; Wed, 30 Jul 2014 01:35:11 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 2DCA72012B for ; Wed, 30 Jul 2014 01:35:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2D0AC20142 for ; Wed, 30 Jul 2014 01:35:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754251AbaG3BfG (ORCPT ); Tue, 29 Jul 2014 21:35:06 -0400 Received: from mail-qa0-f42.google.com ([209.85.216.42]:52777 "EHLO mail-qa0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754491AbaG3BfF (ORCPT ); Tue, 29 Jul 2014 21:35:05 -0400 Received: by mail-qa0-f42.google.com with SMTP id j15so578965qaq.29 for ; Tue, 29 Jul 2014 18:35:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=7WmcsbTmUK6DBwpXdf5s1ZgyifHBe78fqPcrgZTlpxg=; b=hI/fqyJizHJZOiIoSMqYNncIw5Gv8AThr8YxmzzJTAMGlck+ZHPOhUElWjsYvTmrW8 /sUwfuJwU0S6hDnafU1NI8lyEbs5dupcmb8fNBEFkPMa5ENT36JvC4Txm98Ne5y+L8o+ /n/VCNOeDrBeGUkJ3SYpkoj2jA1iyKimTlSlOH0aOOXPpeOqbqqcxk0fvyY4JdDRuD14 G3X0Hd98MVbVzDnDG74teupL49FZYaUA1j3y3swBdg6+CB91u31nBVpyic3SkrOtYuYl z6FrGt0ooYfxU9OTWwZatSrpSENSwJGdVlFbfvKfp/J5T/Z3TWyIH/AY3z/Px017EQcO 1Zsg== X-Gm-Message-State: ALoCoQmILM110xzRvnvTRaZvGZ9rUAvJb+5cuoq7oOyvNbejEwkCvoEk1kkSTB7rmM03EGDbImTV X-Received: by 10.140.23.198 with SMTP id 64mr1190312qgp.84.1406684101510; Tue, 29 Jul 2014 18:35:01 -0700 (PDT) Received: from tlielax.poochiereds.net ([2001:470:8:d63:3a60:77ff:fe93:a95d]) by mx.google.com with ESMTPSA id 81sm860946qgw.21.2014.07.29.18.35.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jul 2014 18:35:00 -0700 (PDT) From: Jeff Layton To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, hch@infradead.org Subject: [PATCH v3 08/38] nfsd: clean up races in lock stateid searching and creation Date: Tue, 29 Jul 2014 21:34:13 -0400 Message-Id: <1406684083-19736-9-git-send-email-jlayton@primarydata.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1406684083-19736-1-git-send-email-jlayton@primarydata.com> References: <1406684083-19736-1-git-send-email-jlayton@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Preparation for removal of the client_mutex. Currently, no lock aside from the client_mutex is held when calling find_lock_state. Ensure that the cl_lock is held by adding a lockdep assertion. Once we remove the client_mutex, it'll be possible for another thread to race in and insert a lock state for the same file after we search but before we insert a new one. Ensure that doesn't happen by redoing the search after allocating a new stid that we plan to insert. If one is found just put the one that was allocated. Signed-off-by: Jeff Layton --- fs/nfsd/nfs4state.c | 71 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 49 insertions(+), 22 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 3ac6e2fdabe5..59d44873b68b 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -4719,20 +4719,15 @@ alloc_init_lock_stateowner(unsigned int strhashval, struct nfs4_client *clp, str return lo; } -static struct nfs4_ol_stateid * -alloc_init_lock_stateid(struct nfs4_lockowner *lo, struct nfs4_file *fp, - struct inode *inode, - struct nfs4_ol_stateid *open_stp) +static void +init_lock_stateid(struct nfs4_ol_stateid *stp, struct nfs4_lockowner *lo, + struct nfs4_file *fp, struct inode *inode, + struct nfs4_ol_stateid *open_stp) { - struct nfs4_stid *s; - struct nfs4_openowner *oo = openowner(open_stp->st_stateowner); - struct nfs4_ol_stateid *stp; struct nfs4_client *clp = lo->lo_owner.so_client; - s = nfs4_alloc_stid(clp, stateid_slab); - if (s == NULL) - return NULL; - stp = openlockstateid(s); + lockdep_assert_held(&clp->cl_lock); + stp->st_stid.sc_type = NFS4_LOCK_STID; stp->st_stateowner = &lo->lo_owner; get_nfs4_file(fp); @@ -4741,20 +4736,20 @@ alloc_init_lock_stateid(struct nfs4_lockowner *lo, struct nfs4_file *fp, stp->st_access_bmap = 0; stp->st_deny_bmap = open_stp->st_deny_bmap; stp->st_openstp = open_stp; - spin_lock(&oo->oo_owner.so_client->cl_lock); list_add(&stp->st_locks, &open_stp->st_locks); list_add(&stp->st_perstateowner, &lo->lo_owner.so_stateids); spin_lock(&fp->fi_lock); list_add(&stp->st_perfile, &fp->fi_stateids); spin_unlock(&fp->fi_lock); - spin_unlock(&oo->oo_owner.so_client->cl_lock); - return stp; } static struct nfs4_ol_stateid * find_lock_stateid(struct nfs4_lockowner *lo, struct nfs4_file *fp) { struct nfs4_ol_stateid *lst; + struct nfs4_client *clp = lo->lo_owner.so_client; + + lockdep_assert_held(&clp->cl_lock); list_for_each_entry(lst, &lo->lo_owner.so_stateids, st_perstateowner) { if (lst->st_stid.sc_file == fp) @@ -4763,6 +4758,38 @@ find_lock_stateid(struct nfs4_lockowner *lo, struct nfs4_file *fp) return NULL; } +static struct nfs4_ol_stateid * +find_or_create_lock_stateid(struct nfs4_lockowner *lo, struct nfs4_file *fi, + struct inode *inode, struct nfs4_ol_stateid *ost, + bool *new) +{ + struct nfs4_stid *ns = NULL; + struct nfs4_ol_stateid *lst; + struct nfs4_openowner *oo = openowner(ost->st_stateowner); + struct nfs4_client *clp = oo->oo_owner.so_client; + + spin_lock(&clp->cl_lock); + lst = find_lock_stateid(lo, fi); + if (lst == NULL) { + spin_unlock(&clp->cl_lock); + ns = nfs4_alloc_stid(clp, stateid_slab); + if (ns == NULL) + return NULL; + + spin_lock(&clp->cl_lock); + lst = find_lock_stateid(lo, fi); + if (likely(!lst)) { + lst = openlockstateid(ns); + init_lock_stateid(lst, lo, fi, inode, ost); + ns = NULL; + *new = true; + } + } + spin_unlock(&clp->cl_lock); + if (ns) + nfs4_put_stid(ns); + return lst; +} static int check_lock_length(u64 offset, u64 length) @@ -4783,7 +4810,11 @@ static void get_lock_access(struct nfs4_ol_stateid *lock_stp, u32 access) set_access(access, lock_stp); } -static __be32 lookup_or_create_lock_state(struct nfsd4_compound_state *cstate, struct nfs4_ol_stateid *ost, struct nfsd4_lock *lock, struct nfs4_ol_stateid **lst, bool *new) +static __be32 +lookup_or_create_lock_state(struct nfsd4_compound_state *cstate, + struct nfs4_ol_stateid *ost, + struct nfsd4_lock *lock, + struct nfs4_ol_stateid **lst, bool *new) { struct nfs4_file *fi = ost->st_stid.sc_file; struct nfs4_openowner *oo = openowner(ost->st_stateowner); @@ -4807,14 +4838,10 @@ static __be32 lookup_or_create_lock_state(struct nfsd4_compound_state *cstate, s return nfserr_bad_seqid; } - *lst = find_lock_stateid(lo, fi); + *lst = find_or_create_lock_stateid(lo, fi, inode, ost, new); if (*lst == NULL) { - *lst = alloc_init_lock_stateid(lo, fi, inode, ost); - if (*lst == NULL) { - release_lockowner_if_empty(lo); - return nfserr_jukebox; - } - *new = true; + release_lockowner_if_empty(lo); + return nfserr_jukebox; } return nfs_ok; }