From patchwork Thu Feb 17 13:15:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Weinberger X-Patchwork-Id: 12750033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60A53C433FE for ; Thu, 17 Feb 2022 13:22:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240666AbiBQNXK (ORCPT ); Thu, 17 Feb 2022 08:23:10 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:51032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238854AbiBQNXJ (ORCPT ); Thu, 17 Feb 2022 08:23:09 -0500 Received: from lithops.sigma-star.at (lithops.sigma-star.at [195.201.40.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA91799ECF for ; Thu, 17 Feb 2022 05:22:52 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by lithops.sigma-star.at (Postfix) with ESMTP id C33CA608898A; Thu, 17 Feb 2022 14:16:39 +0100 (CET) Received: from lithops.sigma-star.at ([127.0.0.1]) by localhost (lithops.sigma-star.at [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id GlwiPVsXDZHd; Thu, 17 Feb 2022 14:16:39 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by lithops.sigma-star.at (Postfix) with ESMTP id F33DE608A38A; Thu, 17 Feb 2022 14:16:38 +0100 (CET) Received: from lithops.sigma-star.at ([127.0.0.1]) by localhost (lithops.sigma-star.at [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 86VoNR_wcpn6; Thu, 17 Feb 2022 14:16:38 +0100 (CET) Received: from blindfold.corp.sigma-star.at (213-47-184-186.cable.dynamic.surfer.at [213.47.184.186]) by lithops.sigma-star.at (Postfix) with ESMTPSA id 8A069605DEBB; Thu, 17 Feb 2022 14:16:38 +0100 (CET) From: Richard Weinberger To: linux-nfs@vger.kernel.org Cc: david@sigma-star.at, bfields@fieldses.org, luis.turcitu@appsbroker.com, david.young@appsbroker.com, david.oberhollenzer@sigma-star.at, trond.myklebust@hammerspace.com, anna.schumaker@netapp.com, chris.chilvers@appsbroker.com, Richard Weinberger Subject: [RFC PATCH 2/6] exports: Implement new export option reexport= Date: Thu, 17 Feb 2022 14:15:27 +0100 Message-Id: <20220217131531.2890-3-richard@nod.at> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220217131531.2890-1-richard@nod.at> References: <20220217131531.2890-1-richard@nod.at> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org When re-exporting a NFS volume it is mandatory to specify either a UUID or numerical fsid= option because nfsd is unable to derive a identifier on its own. For NFS cross mounts this becomes a problem because nfsd also needs a identifier for every crossed mount. A common workaround is stating every single subvolume in the exports list too. But this defeats the purpose of the crossmnt option and is tedious. This is where the reexport= tries to help. It offers various strategies to automatically derive a identifier for NFS volumes and sub volumes. Each have their pros and cons. Currently three modes are implemented: 1. auto-fsidnum In this mode mountd/exportd will create a new numerical fsid for a NFS volume and subvolume. The numbers are stored in a database such that the server will always use the same fsid. The entry in the exports file allowed to skip fsid= entiry but stating a UUID is allowed, if needed. This mode has the obvious downside that load balancing is not possible since multiple re-exporting NFS servers would generate different ids. 2. predefined-fsidnum This mode works just like auto-fsidnum but does not generate ids for you. It helps in the load balancing case. A system administrator has to manually maintain the database and install it on all re-exporting NFS servers. If you have a massive amount of subvolumes this mode will help because you don't have to bloat the exports list. 3. remote-devfsid If this mode is selected mountd/exportd will derive an UUID from the re-exported NFS volume's fsid (rfc7530 section-5.8.1.9). No further local state is needed on the re-exporting server. The export list entry still needs a fsid= setting because while parsing the exports file the NFS mounts might be not there yet. This mode is dangerous, use only of you're absolutely sure that the NFS server you're re-exporting has a stable fsid. Chances are good that it can change. Since an UUID is derived, reexporting from NFSv3 to NFSv3 is not possible. The file handle space is too small. NFSv3 to NFSv4 works, though. Signed-off-by: Richard Weinberger --- support/include/nfslib.h | 1 + support/nfs/Makefile.am | 1 + support/nfs/exports.c | 73 ++++++++++++++++++++++++++++++++++++++ utils/exportfs/Makefile.am | 4 +++ utils/mount/Makefile.am | 6 ++++ 5 files changed, 85 insertions(+) diff --git a/support/include/nfslib.h b/support/include/nfslib.h index 6faba71b..0465a1ff 100644 --- a/support/include/nfslib.h +++ b/support/include/nfslib.h @@ -85,6 +85,7 @@ struct exportent { struct sec_entry e_secinfo[SECFLAVOR_COUNT+1]; unsigned int e_ttl; char * e_realpath; + int e_reexport; }; struct rmtabent { diff --git a/support/nfs/Makefile.am b/support/nfs/Makefile.am index 67e3a8e1..c4357e7d 100644 --- a/support/nfs/Makefile.am +++ b/support/nfs/Makefile.am @@ -9,6 +9,7 @@ libnfs_la_SOURCES = exports.c rmtab.c xio.c rpcmisc.c rpcdispatch.c \ svc_socket.c cacheio.c closeall.c nfs_mntent.c \ svc_create.c atomicio.c strlcat.c strlcpy.c libnfs_la_LIBADD = libnfsconf.la +libnfs_la_CPPFLAGS = -I$(top_srcdir)/support/reexport libnfsconf_la_SOURCES = conffile.c xlog.c diff --git a/support/nfs/exports.c b/support/nfs/exports.c index 2c8f0752..13129d68 100644 --- a/support/nfs/exports.c +++ b/support/nfs/exports.c @@ -31,6 +31,7 @@ #include "xlog.h" #include "xio.h" #include "pseudoflavors.h" +#include "reexport.h" #define EXPORT_DEFAULT_FLAGS \ (NFSEXP_READONLY|NFSEXP_ROOTSQUASH|NFSEXP_GATHERED_WRITES|NFSEXP_NOSUBTREECHECK) @@ -103,6 +104,7 @@ static void init_exportent (struct exportent *ee, int fromkernel) ee->e_nsqgids = 0; ee->e_uuid = NULL; ee->e_ttl = default_ttl; + ee->e_reexport = REEXP_NONE; } struct exportent * @@ -302,6 +304,26 @@ putexportent(struct exportent *ep) } if (ep->e_uuid) fprintf(fp, "fsid=%s,", ep->e_uuid); + + if (ep->e_reexport) { + fprintf(fp, "reexport="); + switch (ep->e_reexport) { + case REEXP_AUTO_FSIDNUM: + fprintf(fp, "auto-fsidnum"); + break; + case REEXP_PREDEFINED_FSIDNUM: + fprintf(fp, "predefined-fsidnum"); + break; + case REEXP_REMOTE_DEVFSID: + fprintf(fp, "remote-devfsid"); + break; + default: + xlog(L_ERROR, "unknown reexport method %i", ep->e_reexport); + fprintf(fp, "none"); + } + fprintf(fp, ","); + } + if (ep->e_mountpoint) fprintf(fp, "mountpoint%s%s,", ep->e_mountpoint[0]?"=":"", ep->e_mountpoint); @@ -538,6 +560,7 @@ parseopts(char *cp, struct exportent *ep, int warn, int *had_subtree_opt_ptr) char *flname = efname?efname:"command line"; int flline = efp?efp->x_line:0; unsigned int active = 0; + int saw_reexport = 0; squids = ep->e_squids; nsquids = ep->e_nsquids; sqgids = ep->e_sqgids; nsqgids = ep->e_nsqgids; @@ -644,6 +667,13 @@ bad_option: } } else if (strncmp(opt, "fsid=", 5) == 0) { char *oe; + + if (saw_reexport) { + xlog(L_ERROR, "%s:%d: 'fsid=' has to be after 'reexport=' %s\n", + flname, flline, opt); + goto bad_option; + } + if (strcmp(opt+5, "root") == 0) { ep->e_fsid = 0; setflags(NFSEXP_FSID, active, ep); @@ -688,6 +718,49 @@ bad_option: active = parse_flavors(opt+4, ep); if (!active) goto bad_option; + } else if (strncmp(opt, "reexport=", 9) == 0) { +#ifdef HAVE_REEXPORT_SUPPORT + char *strategy = strchr(opt, '='); + + if (!strategy) { + xlog(L_ERROR, "%s:%d: bad option %s\n", + flname, flline, opt); + goto bad_option; + } + strategy++; + + if (saw_reexport) { + xlog(L_ERROR, "%s:%d: only one 'reexport=' is allowed%s\n", + flname, flline, opt); + goto bad_option; + } + + if (strcmp(strategy, "auto-fsidnum") == 0) { + ep->e_reexport = REEXP_AUTO_FSIDNUM; + } else if (strcmp(strategy, "predefined-fsidnum") == 0) { + ep->e_reexport = REEXP_PREDEFINED_FSIDNUM; + } else if (strcmp(strategy, "remote-devfsid") == 0) { + ep->e_reexport = REEXP_REMOTE_DEVFSID; + } else if (strcmp(strategy, "none") == 0) { + ep->e_reexport = REEXP_NONE; + } else { + xlog(L_ERROR, "%s:%d: bad option %s\n", + flname, flline, strategy); + goto bad_option; + } + + if (reexpdb_apply_reexport_settings(ep, flname, flline) != 0) + goto bad_option; + + if (ep->e_fsid) + setflags(NFSEXP_FSID, active, ep); + + saw_reexport = 1; +#else + xlog(L_ERROR, "%s:%d: 'reexport=' not available, rebuild with --enable-reexport\n", + flname, flline); + goto bad_option; +#endif } else { xlog(L_ERROR, "%s:%d: unknown keyword \"%s\"\n", flname, flline, opt); diff --git a/utils/exportfs/Makefile.am b/utils/exportfs/Makefile.am index 96524c72..9eabef14 100644 --- a/utils/exportfs/Makefile.am +++ b/utils/exportfs/Makefile.am @@ -12,4 +12,8 @@ exportfs_LDADD = ../../support/export/libexport.a \ ../../support/misc/libmisc.a \ $(LIBWRAP) $(LIBNSL) $(LIBPTHREAD) +if CONFIG_REEXPORT +exportfs_LDADD += ../../support/reexport/libreexport.a $(LIBSQLITE) -lrt +endif + MAINTAINERCLEANFILES = Makefile.in diff --git a/utils/mount/Makefile.am b/utils/mount/Makefile.am index 3101f7ab..f4d5b182 100644 --- a/utils/mount/Makefile.am +++ b/utils/mount/Makefile.am @@ -32,6 +32,12 @@ mount_nfs_LDADD = ../../support/nfs/libnfs.la \ ../../support/misc/libmisc.a \ $(LIBTIRPC) +if CONFIG_REEXPORT +mount_nfs_LDADD += ../../support/reexport/libreexport.a \ + $(LIBSQLITE) -lrt $(LIBPTHREAD) +endif + + mount_nfs_SOURCES = $(mount_common) if CONFIG_LIBMOUNT