From patchwork Wed Jul 30 01:34:27 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 4644201 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id A3D259F36A for ; Wed, 30 Jul 2014 01:35:36 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7A18B2012B for ; Wed, 30 Jul 2014 01:35:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6062020142 for ; Wed, 30 Jul 2014 01:35:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754734AbaG3BfZ (ORCPT ); Tue, 29 Jul 2014 21:35:25 -0400 Received: from mail-qg0-f53.google.com ([209.85.192.53]:46223 "EHLO mail-qg0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754491AbaG3BfY (ORCPT ); Tue, 29 Jul 2014 21:35:24 -0400 Received: by mail-qg0-f53.google.com with SMTP id q107so681384qgd.12 for ; Tue, 29 Jul 2014 18:35:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=ahhm7HzdRLZmmBWPnueifkH8cEe11hugQhsC5hGoaYg=; b=cYtedxf0iMSATNxVkksIZ/h8LhXgHR9WaGD876fMKFpgEn+lxrEJ2RGwcueCC5iLiv GmP2Q/8y9hdBakg1rrw3B7zr53ykBezR/BiPDNlXbTpw3HVlHmAMdWsFXhmStUGyWYdL YZJ0LO79dvIytfbkcdjK5T2IxM4Xivl1WQg3N6mkKwHuHfPdBAdpVizkIlbdztty1Yo0 1uRVOLqCp+DoShvoSj2bCazlelwumi5tgoTdVmXzwzP6lL7IK0Tu3sO7yd3HQM2fArQd 22L5c4lW5/ItEv/3BQI/Xbx/Dd4Mpcek7mKl2EurvvFfAq9dRhxU7ER4d7J2HTLcq8vw hXjA== X-Gm-Message-State: ALoCoQn1b8DLNKiST9IIL6Q06mIIzmqk/jnKy4yafM1zlPS4TVTwRI9DXxS24Z0NODfVzjUl6va3 X-Received: by 10.140.108.200 with SMTP id j66mr1090859qgf.43.1406684123224; Tue, 29 Jul 2014 18:35:23 -0700 (PDT) Received: from tlielax.poochiereds.net ([2001:470:8:d63:3a60:77ff:fe93:a95d]) by mx.google.com with ESMTPSA id 81sm860946qgw.21.2014.07.29.18.35.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Jul 2014 18:35:22 -0700 (PDT) From: Jeff Layton To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, hch@infradead.org Subject: [PATCH v3 22/38] nfsd: Add a mutex to protect the NFSv4.0 open owner replay cache Date: Tue, 29 Jul 2014 21:34:27 -0400 Message-Id: <1406684083-19736-23-git-send-email-jlayton@primarydata.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1406684083-19736-1-git-send-email-jlayton@primarydata.com> References: <1406684083-19736-1-git-send-email-jlayton@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't want to rely on the client_mutex for protection in the case of NFSv4 open owners. Instead, we add a mutex that will only be taken for NFSv4.0 state mutating operations, and that will be released once the entire compound is done. Also, ensure that nfsd4_cstate_assign_replay/nfsd4_cstate_clear_replay take a reference to the stateowner when they are using it for NFSv4.0 open and lock replay caching. Signed-off-by: Trond Myklebust Signed-off-by: Jeff Layton --- fs/nfsd/nfs4proc.c | 12 +++--------- fs/nfsd/nfs4state.c | 47 +++++++++++++++++++++++++++++++++-------------- fs/nfsd/nfs4xdr.c | 2 -- fs/nfsd/state.h | 1 + fs/nfsd/xdr4.h | 5 ++++- 5 files changed, 41 insertions(+), 26 deletions(-) diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c index 8611585f739d..29cf395b694e 100644 --- a/fs/nfsd/nfs4proc.c +++ b/fs/nfsd/nfs4proc.c @@ -469,12 +469,9 @@ out: fh_put(resfh); kfree(resfh); } - nfsd4_cleanup_open_state(open, status); - if (open->op_openowner && !nfsd4_has_session(cstate)) - cstate->replay_owner = &open->op_openowner->oo_owner; + nfsd4_cleanup_open_state(cstate, open, status); nfsd4_bump_seqid(cstate, status); - if (!cstate->replay_owner) - nfs4_unlock_state(); + nfs4_unlock_state(); return status; } @@ -1395,10 +1392,7 @@ encode_op: args->ops, args->opcnt, resp->opcnt, op->opnum, be32_to_cpu(status)); - if (cstate->replay_owner) { - nfs4_unlock_state(); - cstate->replay_owner = NULL; - } + nfsd4_cstate_clear_replay(cstate); /* XXX Ugh, we need to get rid of this kind of special case: */ if (op->opnum == OP_READ && op->u.read.rd_filp) fput(op->u.read.rd_filp); diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 5a93e5fafd4a..749608b914b4 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -1069,7 +1069,7 @@ void nfsd4_bump_seqid(struct nfsd4_compound_state *cstate, __be32 nfserr) return; if (!seqid_mutating_err(ntohl(nfserr))) { - cstate->replay_owner = NULL; + nfsd4_cstate_clear_replay(cstate); return; } if (!so) @@ -2940,6 +2940,28 @@ static void init_nfs4_replay(struct nfs4_replay *rp) rp->rp_status = nfserr_serverfault; rp->rp_buflen = 0; rp->rp_buf = rp->rp_ibuf; + mutex_init(&rp->rp_mutex); +} + +static void nfsd4_cstate_assign_replay(struct nfsd4_compound_state *cstate, + struct nfs4_stateowner *so) +{ + if (!nfsd4_has_session(cstate)) { + mutex_lock(&so->so_replay.rp_mutex); + cstate->replay_owner = so; + atomic_inc(&so->so_count); + } +} + +void nfsd4_cstate_clear_replay(struct nfsd4_compound_state *cstate) +{ + struct nfs4_stateowner *so = cstate->replay_owner; + + if (so != NULL) { + cstate->replay_owner = NULL; + mutex_unlock(&so->so_replay.rp_mutex); + nfs4_put_stateowner(so); + } } static inline void *alloc_stateowner(struct kmem_cache *slab, struct xdr_netobj *owner, struct nfs4_client *clp) @@ -3855,7 +3877,8 @@ out: return status; } -void nfsd4_cleanup_open_state(struct nfsd4_open *open, __be32 status) +void nfsd4_cleanup_open_state(struct nfsd4_compound_state *cstate, + struct nfsd4_open *open, __be32 status) { if (open->op_openowner) { struct nfs4_openowner *oo = open->op_openowner; @@ -3869,6 +3892,8 @@ void nfsd4_cleanup_open_state(struct nfsd4_open *open, __be32 status) } else oo->oo_flags &= ~NFS4_OO_NEW; } + if (open->op_openowner) + nfsd4_cstate_assign_replay(cstate, &oo->oo_owner); } if (open->op_file) nfsd4_free_file(open->op_file); @@ -4399,8 +4424,7 @@ nfs4_preprocess_seqid_op(struct nfsd4_compound_state *cstate, u32 seqid, if (status) return status; stp = openlockstateid(s); - if (!nfsd4_has_session(cstate)) - cstate->replay_owner = stp->st_stateowner; + nfsd4_cstate_assign_replay(cstate, stp->st_stateowner); status = nfs4_seqid_op_checks(cstate, stateid, seqid, stp); if (!status) @@ -4469,8 +4493,7 @@ put_stateid: nfs4_put_stid(&stp->st_stid); out: nfsd4_bump_seqid(cstate, status); - if (!cstate->replay_owner) - nfs4_unlock_state(); + nfs4_unlock_state(); return status; } @@ -4544,8 +4567,7 @@ put_stateid: nfs4_put_stid(&stp->st_stid); out: nfsd4_bump_seqid(cstate, status); - if (!cstate->replay_owner) - nfs4_unlock_state(); + nfs4_unlock_state(); return status; } @@ -4610,8 +4632,7 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, /* put reference from nfs4_preprocess_seqid_op */ nfs4_put_stid(&stp->st_stid); out: - if (!cstate->replay_owner) - nfs4_unlock_state(); + nfs4_unlock_state(); return status; } @@ -5071,8 +5092,7 @@ out: if (status && new_state) release_lock_stateid(lock_stp); nfsd4_bump_seqid(cstate, status); - if (!cstate->replay_owner) - nfs4_unlock_state(); + nfs4_unlock_state(); if (file_lock) locks_free_lock(file_lock); if (conflock) @@ -5236,8 +5256,7 @@ put_stateid: nfs4_put_stid(&stp->st_stid); out: nfsd4_bump_seqid(cstate, status); - if (!cstate->replay_owner) - nfs4_unlock_state(); + nfs4_unlock_state(); if (file_lock) locks_free_lock(file_lock); return status; diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 628b430e743e..72a2a82e11a4 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -3925,8 +3925,6 @@ status: * * XDR note: do not encode rp->rp_buflen: the buffer contains the * previously sent already encoded operation. - * - * called with nfs4_lock_state() held */ void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op) diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index dc725deb4aa8..9cba295812f6 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -328,6 +328,7 @@ struct nfs4_replay { unsigned int rp_buflen; char *rp_buf; struct knfsd_fh rp_openfh; + struct mutex rp_mutex; char rp_ibuf[NFSD4_REPLAY_ISIZE]; }; diff --git a/fs/nfsd/xdr4.h b/fs/nfsd/xdr4.h index 5abf6c942ddf..465e7799742a 100644 --- a/fs/nfsd/xdr4.h +++ b/fs/nfsd/xdr4.h @@ -599,7 +599,9 @@ extern __be32 nfsd4_process_open1(struct nfsd4_compound_state *, struct nfsd4_open *open, struct nfsd_net *nn); extern __be32 nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nfsd4_open *open); -extern void nfsd4_cleanup_open_state(struct nfsd4_open *open, __be32 status); +extern void nfsd4_cstate_clear_replay(struct nfsd4_compound_state *cstate); +extern void nfsd4_cleanup_open_state(struct nfsd4_compound_state *cstate, + struct nfsd4_open *open, __be32 status); extern __be32 nfsd4_open_confirm(struct svc_rqst *rqstp, struct nfsd4_compound_state *, struct nfsd4_open_confirm *oc); extern __be32 nfsd4_close(struct svc_rqst *rqstp, @@ -630,6 +632,7 @@ extern __be32 nfsd4_test_stateid(struct svc_rqst *rqstp, extern __be32 nfsd4_free_stateid(struct svc_rqst *rqstp, struct nfsd4_compound_state *, struct nfsd4_free_stateid *free_stateid); extern void nfsd4_bump_seqid(struct nfsd4_compound_state *, __be32 nfserr); + #endif /*