From patchwork Tue Nov 17 11:52:45 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 7636571 Return-Path: X-Original-To: patchwork-linux-fsdevel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 75BC2BF90C for ; Tue, 17 Nov 2015 11:55:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 3CF4D204EC for ; Tue, 17 Nov 2015 11:55:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E320F20501 for ; Tue, 17 Nov 2015 11:55:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753512AbbKQLyq (ORCPT ); Tue, 17 Nov 2015 06:54:46 -0500 Received: from mail-qk0-f181.google.com ([209.85.220.181]:35656 "EHLO mail-qk0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753462AbbKQLxm (ORCPT ); Tue, 17 Nov 2015 06:53:42 -0500 Received: by qkao63 with SMTP id o63so1677013qka.2 for ; Tue, 17 Nov 2015 03:53:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=poochiereds_net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PupnoLHDjVfvadZIwIFP44BIHVpplydabGLiMuLPwQ8=; b=GR0VHAD67wImK+ygLTwTTvmTXwevIkvR6hPMfxyh9ZgFGbRpNvzsrIkwNBPezP0GhI F0YDHWGCQPRKJYBs7cNT+/Ch3/m/ghTDEv+n2J1OIL/sRFzT4+dCOtUzvAN6i22lKS0Y HZIjGhYkves3wQjvC+eN5SDpgsf+Mqt7gxZkGKvzgJLJ4QMvtPyuH/WP999mLnjUtp4q tol3aMX5KOjF7I180WtxBmeTpIU5Ewe5kMOUArNf+VGXLNm3fYlsQuto8M1lODOgcDWR htYC8tcNKNyMRtpgv2tq27y1eUOVy4jbzYq0qw1Q5Benj2uv10vS3y0q842Z6Cebpicn +GSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PupnoLHDjVfvadZIwIFP44BIHVpplydabGLiMuLPwQ8=; b=hBtx7HQt07GGJzptDlCpd0uMwPqHkcX4/xxsY9Sz/AEVGLY9Q/kZHWWjRlaH6Q1BWg RYps+zjzH0PWmZwuitrMBdhcdNhc4j36kqNRjbWlnRqnizltJfqenjcaCC9u32ohAVZ2 gP+FUtsLAOWvewYmLoGl+tKM9yNxJrKwxchVfJiPQakSOb6dVwwdvOg3bCTpFAFjzFkL qMLqL5D56Yv21xUkXOLwEtUWXDxbHICHr5Px45L1nkDhkSaNxazaSiukTEsqKlSN4Tdh HTXM4AEWU8Ro33XIQsiu1UTRGJsFwRCXkeBy8JAX8VmukLEo0aJJcBCTg7a+NUjAmFhI Q1KQ== X-Gm-Message-State: ALoCoQm6zVtjch04FqSmll+6tbAY/APof07f7ezBV6hRRO/a9AIswKFcLj/GIjZUn3yZny+oVl5U X-Received: by 10.55.72.70 with SMTP id v67mr40713692qka.83.1447761221651; Tue, 17 Nov 2015 03:53:41 -0800 (PST) Received: from tlielax.poochiereds.net ([2606:a000:1125:4075::d5a]) by smtp.googlemail.com with ESMTPSA id w10sm1583910qhc.16.2015.11.17.03.53.40 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 17 Nov 2015 03:53:40 -0800 (PST) From: Jeff Layton X-Google-Original-From: Jeff Layton To: bfields@fieldses.org, trond.myklebust@primarydata.com Cc: linux-nfs@vger.kernel.org, Eric Paris , Alexander Viro , linux-fsdevel@vger.kernel.org Subject: [PATCH v1 23/38] nfsd: add a new EXPORT_OP_NOWCC flag to struct export_operations Date: Tue, 17 Nov 2015 06:52:45 -0500 Message-Id: <1447761180-4250-24-git-send-email-jeff.layton@primarydata.com> X-Mailer: git-send-email 2.4.3 In-Reply-To: <1447761180-4250-1-git-send-email-jeff.layton@primarydata.com> References: <1447761180-4250-1-git-send-email-jeff.layton@primarydata.com> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Spam-Status: No, score=-7.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With NFSv3 nfsd will always attempt to send along WCC data to the client. This generally involves saving off the in-core inode information prior to doing the operation on the given filehandle, and then issuing a vfs_getattr to it after the op. Some filesystems (particularly clustered or networked ones) have an expensive ->getattr inode operation. Atomicitiy is also often difficult or impossible to guarantee on such filesystems. For those, we're best off not trying to provide WCC information to the client at all, and to simply allow it to poll for that information as needed with a GETATTR RPC. This patch adds a new flags field to struct export_operations, and defines a new EXPORT_OP_NOWCC flag that filesystems can use to indicate that nfsd should not attempt to provide WCC info in NFSv3 replies. It also adds a blurb about the new flags field and flag to the exporting documentation. The server will also now skip collecting this information for NFSv2 as well, since that info is never used there anyway. Note that this patch does not add this flag to any filesystem export_operations structures. This was originally developed to allow reexporting nfs via nfsd. That code is not (and may never be) suitable for merging into mainline. Other filesystems may want to consider enabling this flag too. It's hard to tell however which ones have export operations to enable export via knfsd and which ones mostly rely on them for open-by-filehandle support, so I'm leaving that up to the individual maintainers to decide. I am cc'ing the relevant lists for those filesystems that I think may want to consider adding this though. Cc: HPDD-discuss@lists.01.org Cc: ceph-devel@vger.kernel.org Cc: cluster-devel@redhat.com Cc: fuse-devel@lists.sourceforge.net Cc: ocfs2-devel@oss.oracle.com Signed-off-by: Jeff Layton --- Documentation/filesystems/nfs/Exporting | 27 +++++++++++++++++++++++++++ fs/nfsd/nfs3xdr.c | 5 ++++- fs/nfsd/nfsfh.c | 14 ++++++++++++++ fs/nfsd/nfsfh.h | 5 ++++- include/linux/exportfs.h | 2 ++ 5 files changed, 51 insertions(+), 2 deletions(-) diff --git a/Documentation/filesystems/nfs/Exporting b/Documentation/filesystems/nfs/Exporting index 520a4becb75c..fa636cde3907 100644 --- a/Documentation/filesystems/nfs/Exporting +++ b/Documentation/filesystems/nfs/Exporting @@ -138,6 +138,11 @@ struct which has the following members: to find potential names, and matches inode numbers to find the correct match. + flags + Some filesystems may need to be handled differently than others. The + export_operations struct also includes a flags field that allows the + filesystem to communicate such information to nfsd. See the Export + Operations Flags section below for more explanation. A filehandle fragment consists of an array of 1 or more 4byte words, together with a one byte "type". @@ -147,3 +152,25 @@ generated by encode_fh, in which case it will have been padded with nuls. Rather, the encode_fh routine should choose a "type" which indicates the decode_fh how much of the filehandle is valid, and how it should be interpreted. + +Export Operations Flags +----------------------- +In addition to the operation vector pointers, struct export_operations also +contains a "flags" field that allows the filesystem to communicate to nfsd +that it may want to do things differently when dealing with it. The +following flags are defined: + + EXPORT_OP_NOWCC + RFC 1813 recommends that servers always send weak cache consistency + (WCC) data to the client after each operation. The server should + atomically collect attributes about the inode, do an operation on it, + and then collect the attributes afterward. This allows the client to + skip issuing GETATTRs in some situations but means that the server + is calling vfs_getattr for almost all RPCs. On some filesystems + (particularly those that are clustered or networked) this is expensive + and atomicity is difficult to guarantee. This flag indicates to nfsd + that it should skip providing WCC attributes to the client in NFSv3 + replies when doing operations on this filesystem. Consider enabling + this on filesystems that have an expensive ->getattr inode operation, + or when atomicity between pre and post operation attribute collection + is impossible to guarantee. diff --git a/fs/nfsd/nfs3xdr.c b/fs/nfsd/nfs3xdr.c index 00575d776d91..6420e28a4d2f 100644 --- a/fs/nfsd/nfs3xdr.c +++ b/fs/nfsd/nfs3xdr.c @@ -203,7 +203,7 @@ static __be32 * encode_post_op_attr(struct svc_rqst *rqstp, __be32 *p, struct svc_fh *fhp) { struct dentry *dentry = fhp->fh_dentry; - if (dentry && d_really_is_positive(dentry)) { + if (!fhp->fh_no_wcc && dentry && d_really_is_positive(dentry)) { __be32 err; struct kstat stat; @@ -256,6 +256,9 @@ void fill_post_wcc(struct svc_fh *fhp) { __be32 err; + if (fhp->fh_no_wcc) + return; + if (fhp->fh_post_saved) printk("nfsd: inode locked twice during operation.\n"); diff --git a/fs/nfsd/nfsfh.c b/fs/nfsd/nfsfh.c index c1681ce894c5..f56f938a2e3f 100644 --- a/fs/nfsd/nfsfh.c +++ b/fs/nfsd/nfsfh.c @@ -267,6 +267,16 @@ static __be32 nfsd_set_fh_dentry(struct svc_rqst *rqstp, struct svc_fh *fhp) fhp->fh_dentry = dentry; fhp->fh_export = exp; + + switch (rqstp->rq_vers) { + case 3: + if (!(dentry->d_sb->s_export_op->flags & EXPORT_OP_NOWCC)) + break; + /* Fallthrough */ + case 2: + fhp->fh_no_wcc = true; + } + return 0; out: exp_put(exp); @@ -535,6 +545,9 @@ fh_compose(struct svc_fh *fhp, struct svc_export *exp, struct dentry *dentry, */ set_version_and_fsid_type(fhp, exp, ref_fh); + /* If we have a ref_fh, then copy the fh_no_wcc setting from it. */ + fhp->fh_no_wcc = ref_fh ? ref_fh->fh_no_wcc : false; + if (ref_fh == fhp) fh_put(ref_fh); @@ -638,6 +651,7 @@ fh_put(struct svc_fh *fhp) exp_put(exp); fhp->fh_export = NULL; } + fhp->fh_no_wcc = false; return; } diff --git a/fs/nfsd/nfsfh.h b/fs/nfsd/nfsfh.h index 0770bcb543c8..1cca265ad3e9 100644 --- a/fs/nfsd/nfsfh.h +++ b/fs/nfsd/nfsfh.h @@ -33,6 +33,7 @@ typedef struct svc_fh { bool fh_locked; /* inode locked by us */ bool fh_want_write; /* remount protection taken */ + bool fh_no_wcc; /* no wcc data needed */ #ifdef CONFIG_NFSD_V3 bool fh_post_saved; /* post-op attrs saved */ @@ -52,7 +53,6 @@ typedef struct svc_fh { struct kstat fh_post_attr; /* full attrs after operation */ u64 fh_post_change; /* nfsv4 change; see above */ #endif /* CONFIG_NFSD_V3 */ - } svc_fh; enum nfsd_fsid { @@ -248,6 +248,9 @@ fill_pre_wcc(struct svc_fh *fhp) { struct inode *inode; + if (fhp->fh_no_wcc) + return; + inode = d_inode(fhp->fh_dentry); if (!fhp->fh_pre_saved) { fhp->fh_pre_mtime = inode->i_mtime; diff --git a/include/linux/exportfs.h b/include/linux/exportfs.h index fa05e04c5531..600c3fccc999 100644 --- a/include/linux/exportfs.h +++ b/include/linux/exportfs.h @@ -214,6 +214,8 @@ struct export_operations { bool write, u32 *device_generation); int (*commit_blocks)(struct inode *inode, struct iomap *iomaps, int nr_iomaps, struct iattr *iattr); +#define EXPORT_OP_NOWCC (0x1) /* Don't collect wcc data for NFSv3 replies */ + unsigned long flags; }; extern int exportfs_encode_inode_fh(struct inode *inode, struct fid *fid,