| Message ID | 20190521124701.61849-1-trond.myklebust@hammerspace.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Add a root_dir option to nfs.conf | expand |
Hi Trond - > On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com> wrote: > > The following patchset adds support for the 'root_dir' configuration > option for nfsd in nfs.conf. If a user sets this option to a valid > directory path, then nfsd will act as if it is confined to a chroot > jail based on that directory. All paths in /etc/exporfs and from > exportfs are then resolved relative to that directory. What about files under /proc that mountd might access? I assume these pathnames are not affected. Aren't there also one or two other files that maintain export state like /var/lib/nfs/rmtab? Are those affected? IMHO it could be less confusing to administrators to make root_dir an [exportfs] option instead of a [mountd] option, if this is not a true chroot of mountd. > Trond Myklebust (7): > mountd: Ensure we don't share cache file descriptors among processes. > Add a simple workqueue mechanism > Add utilities for resolving nfsd paths and stat()ing them > Add a helper to return the real path given an export entry > Add helpers to read/write to a file through the chrooted thread > Add support for the nfsd rootdir configuration option to rpc.mountd > Add support for the nfsd root directory to exportfs > > aclocal/libpthread.m4 | 13 +- > configure.ac | 6 +- > nfs.conf | 1 + > support/export/export.c | 24 +++ > support/include/Makefile.am | 2 + > support/include/exportfs.h | 1 + > support/include/nfsd_path.h | 17 ++ > support/include/nfslib.h | 1 + > support/include/workqueue.h | 22 +++ > support/misc/Makefile.am | 3 +- > support/misc/mountpoint.c | 5 +- > support/misc/nfsd_path.c | 175 +++++++++++++++++++++ > support/misc/workqueue.c | 306 ++++++++++++++++++++++++++++++++++++ > support/nfs/exports.c | 4 + > systemd/nfs.conf.man | 3 +- > utils/exportfs/Makefile.am | 2 +- > utils/exportfs/exportfs.c | 32 +++- > utils/mountd/Makefile.am | 3 +- > utils/mountd/cache.c | 79 +++++++--- > utils/mountd/mountd.c | 13 +- > utils/nfsd/nfsd.man | 6 + > 21 files changed, 676 insertions(+), 42 deletions(-) > create mode 100644 support/include/nfsd_path.h > create mode 100644 support/include/workqueue.h > create mode 100644 support/misc/nfsd_path.c > create mode 100644 support/misc/workqueue.c > > -- > 2.21.0 > -- Chuck Lever chucklever@gmail.com
On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: > Hi Trond - > > > On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com> > > wrote: > > > > The following patchset adds support for the 'root_dir' > > configuration > > option for nfsd in nfs.conf. If a user sets this option to a valid > > directory path, then nfsd will act as if it is confined to a chroot > > jail based on that directory. All paths in /etc/exporfs and from > > exportfs are then resolved relative to that directory. > > What about files under /proc that mountd might access? I assume these > pathnames are not affected. > That's why we have 2 threads. One thread is root jailed using chroot, and is used to talk to knfsd. The other thread is not root jailed (or at least not by root_dir) and so has full access to /etc, /proc, /var, ... > Aren't there also one or two other files that maintain export state > like /var/lib/nfs/rmtab? Are those affected? See above. They are not affected. > IMHO it could be less confusing to administrators to make root_dir an > [exportfs] option instead of a [mountd] option, if this is not a true > chroot of mountd. It is neither. I made in a [nfsd] option, since it governs the way that both exportfs and mountd talk to nfsd. Cheers Trond
On Tue, 2019-05-21 at 18:17 +0000, Trond Myklebust wrote: > On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: > > Hi Trond - > > > > > On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com> > > > wrote: > > > > > > The following patchset adds support for the 'root_dir' > > > configuration > > > option for nfsd in nfs.conf. If a user sets this option to a > > > valid > > > directory path, then nfsd will act as if it is confined to a > > > chroot > > > jail based on that directory. All paths in /etc/exporfs and from > > > exportfs are then resolved relative to that directory. > > > > What about files under /proc that mountd might access? I assume > > these > > pathnames are not affected. > > > That's why we have 2 threads. One thread is root jailed using chroot, > and is used to talk to knfsd. The other thread is not root jailed (or > at least not by root_dir) and so has full access to /etc, /proc, > /var, > ... I should perhaps note that the reason why I used a second thread, rather than using fork()ed processes like the rest of the mountd code is to allow the sharing of file descriptors, so that the unconfined thread can open files that can then be easily used by the root jailed thread. This means that if you have an old glibc that does not support POSIX threads, then the 'root_dir' functionality is disabled. Ditto if you have a kernel that does not support the unshare() system call or if it does not support openat()+fstatat(). > > Aren't there also one or two other files that maintain export state > > like /var/lib/nfs/rmtab? Are those affected? > > See above. They are not affected. > > > IMHO it could be less confusing to administrators to make root_dir > > an > > [exportfs] option instead of a [mountd] option, if this is not a > > true > > chroot of mountd. > > It is neither. I made in a [nfsd] option, since it governs the way > that > both exportfs and mountd talk to nfsd. >
> On May 21, 2019, at 2:17 PM, Trond Myklebust <trondmy@hammerspace.com> wrote: > > On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: >> Hi Trond - >> >>> On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com> >>> wrote: >>> >>> The following patchset adds support for the 'root_dir' >>> configuration >>> option for nfsd in nfs.conf. If a user sets this option to a valid >>> directory path, then nfsd will act as if it is confined to a chroot >>> jail based on that directory. All paths in /etc/exporfs and from >>> exportfs are then resolved relative to that directory. >> >> What about files under /proc that mountd might access? I assume these >> pathnames are not affected. >> > That's why we have 2 threads. One thread is root jailed using chroot, > and is used to talk to knfsd. The other thread is not root jailed (or > at least not by root_dir) and so has full access to /etc, /proc, /var, > ... > >> Aren't there also one or two other files that maintain export state >> like /var/lib/nfs/rmtab? Are those affected? > > See above. They are not affected. > >> IMHO it could be less confusing to administrators to make root_dir an >> [exportfs] option instead of a [mountd] option, if this is not a true >> chroot of mountd. > > It is neither. I made in a [nfsd] option, since it governs the way that > both exportfs and mountd talk to nfsd. My point is not about implementation, it's about how this functionality is presented to administrators. In nfs.conf, [nfsd] looks like it controls what options are passed via rpc.nfsd. That still seems like a confusing admin interface. IMO admins won't care about who is talking to whom. They will care about how the export pathnames are interpreted. That seems like it belongs squarely with the exportfs interface. -- Chuck Lever chucklever@gmail.com
On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: > > On May 21, 2019, at 2:17 PM, Trond Myklebust < > > trondmy@hammerspace.com> wrote: > > > > On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: > > > Hi Trond - > > > > > > > On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com > > > > > > > > > wrote: > > > > > > > > The following patchset adds support for the 'root_dir' > > > > configuration > > > > option for nfsd in nfs.conf. If a user sets this option to a > > > > valid > > > > directory path, then nfsd will act as if it is confined to a > > > > chroot > > > > jail based on that directory. All paths in /etc/exporfs and > > > > from > > > > exportfs are then resolved relative to that directory. > > > > > > What about files under /proc that mountd might access? I assume > > > these > > > pathnames are not affected. > > > > > That's why we have 2 threads. One thread is root jailed using > > chroot, > > and is used to talk to knfsd. The other thread is not root jailed > > (or > > at least not by root_dir) and so has full access to /etc, /proc, > > /var, > > ... > > > > > Aren't there also one or two other files that maintain export > > > state > > > like /var/lib/nfs/rmtab? Are those affected? > > > > See above. They are not affected. > > > > > IMHO it could be less confusing to administrators to make > > > root_dir an > > > [exportfs] option instead of a [mountd] option, if this is not a > > > true > > > chroot of mountd. > > > > It is neither. I made in a [nfsd] option, since it governs the way > > that > > both exportfs and mountd talk to nfsd. > > My point is not about implementation, it's about how this > functionality > is presented to administrators. > > In nfs.conf, [nfsd] looks like it controls what options are passed > via > rpc.nfsd. That still seems like a confusing admin interface. > > IMO admins won't care about who is talking to whom. They will care > about > how the export pathnames are interpreted. That seems like it belongs > squarely with the exportfs interface. > With the exportfs interface, yes. However it is not specific to the exportfs utility, so to me [exportfs] is more confusing than what exists now. OK, so what if we put it in [general] instead, and perhaps rename it "export_rootdir"?
On 5/21/19 3:58 PM, Trond Myklebust wrote: > On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: >>> On May 21, 2019, at 2:17 PM, Trond Myklebust < >>> trondmy@hammerspace.com> wrote: >>> >>> On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: >>>> Hi Trond - >>>> >>>>> On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com >>>>>> >>>>> wrote: >>>>> >>>>> The following patchset adds support for the 'root_dir' >>>>> configuration >>>>> option for nfsd in nfs.conf. If a user sets this option to a >>>>> valid >>>>> directory path, then nfsd will act as if it is confined to a >>>>> chroot >>>>> jail based on that directory. All paths in /etc/exporfs and >>>>> from >>>>> exportfs are then resolved relative to that directory. >>>> >>>> What about files under /proc that mountd might access? I assume >>>> these >>>> pathnames are not affected. >>>> >>> That's why we have 2 threads. One thread is root jailed using >>> chroot, >>> and is used to talk to knfsd. The other thread is not root jailed >>> (or >>> at least not by root_dir) and so has full access to /etc, /proc, >>> /var, >>> ... >>> >>>> Aren't there also one or two other files that maintain export >>>> state >>>> like /var/lib/nfs/rmtab? Are those affected? >>> >>> See above. They are not affected. >>> >>>> IMHO it could be less confusing to administrators to make >>>> root_dir an >>>> [exportfs] option instead of a [mountd] option, if this is not a >>>> true >>>> chroot of mountd. >>> >>> It is neither. I made in a [nfsd] option, since it governs the way >>> that >>> both exportfs and mountd talk to nfsd. >> >> My point is not about implementation, it's about how this >> functionality >> is presented to administrators. >> >> In nfs.conf, [nfsd] looks like it controls what options are passed >> via >> rpc.nfsd. That still seems like a confusing admin interface. >> >> IMO admins won't care about who is talking to whom. They will care >> about >> how the export pathnames are interpreted. That seems like it belongs >> squarely with the exportfs interface. >> > > With the exportfs interface, yes. However it is not specific to the > exportfs utility, so to me [exportfs] is more confusing than what > exists now. > > OK, so what if we put it in [general] instead, and perhaps rename it > "export_rootdir"? > I'm just catching up... my apologies tartness... So setting root_dir effects *all* exports in /etc/exports? If that is the case, that one variable can change hundreds of export... is that what we really want? Wouldn't be better to have a little more granularity? As for where root_dir should go, I think it makes senses to create a new [exportfs] section and have mountd read it from there. I think that would be more straightforward if we continue with the big hammer approach where any and all exports are effected. steved.
> On May 21, 2019, at 3:58 PM, Trond Myklebust <trondmy@hammerspace.com> wrote: > > On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: >>> On May 21, 2019, at 2:17 PM, Trond Myklebust < >>> trondmy@hammerspace.com> wrote: >>> >>> On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: >>>> Hi Trond - >>>> >>>>> On May 21, 2019, at 8:46 AM, Trond Myklebust <trondmy@gmail.com >>>>>> >>>>> wrote: >>>>> >>>>> The following patchset adds support for the 'root_dir' >>>>> configuration >>>>> option for nfsd in nfs.conf. If a user sets this option to a >>>>> valid >>>>> directory path, then nfsd will act as if it is confined to a >>>>> chroot >>>>> jail based on that directory. All paths in /etc/exporfs and >>>>> from >>>>> exportfs are then resolved relative to that directory. >>>> >>>> What about files under /proc that mountd might access? I assume >>>> these >>>> pathnames are not affected. >>>> >>> That's why we have 2 threads. One thread is root jailed using >>> chroot, >>> and is used to talk to knfsd. The other thread is not root jailed >>> (or >>> at least not by root_dir) and so has full access to /etc, /proc, >>> /var, >>> ... >>> >>>> Aren't there also one or two other files that maintain export >>>> state >>>> like /var/lib/nfs/rmtab? Are those affected? >>> >>> See above. They are not affected. >>> >>>> IMHO it could be less confusing to administrators to make >>>> root_dir an >>>> [exportfs] option instead of a [mountd] option, if this is not a >>>> true >>>> chroot of mountd. >>> >>> It is neither. I made in a [nfsd] option, since it governs the way >>> that >>> both exportfs and mountd talk to nfsd. >> >> My point is not about implementation, it's about how this >> functionality >> is presented to administrators. >> >> In nfs.conf, [nfsd] looks like it controls what options are passed >> via >> rpc.nfsd. That still seems like a confusing admin interface. >> >> IMO admins won't care about who is talking to whom. They will care >> about >> how the export pathnames are interpreted. That seems like it belongs >> squarely with the exportfs interface. >> > > With the exportfs interface, yes. However it is not specific to the > exportfs utility, so to me [exportfs] is more confusing than what > exists now. > > OK, so what if we put it in [general] instead, and perhaps rename it > "export_rootdir"? For the record, I prefer export_rootdir to root_dir. I haven't thought of a better choice. I also like not putting this option under [nfsd]. [general] is a better choice IMO, though I'm still not crazy about it. It might be nice to have a new section called "[exports]" (no "f"). -- Chuck Lever
On Tue, 2019-05-28 at 11:25 -0400, Steve Dickson wrote: > > On 5/21/19 3:58 PM, Trond Myklebust wrote: > > On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: > > > > On May 21, 2019, at 2:17 PM, Trond Myklebust < > > > > trondmy@hammerspace.com> wrote: > > > > > > > > On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: > > > > > Hi Trond - > > > > > > > > > > > On May 21, 2019, at 8:46 AM, Trond Myklebust < > > > > > > trondmy@gmail.com > > > > > > wrote: > > > > > > > > > > > > The following patchset adds support for the 'root_dir' > > > > > > configuration > > > > > > option for nfsd in nfs.conf. If a user sets this option to > > > > > > a > > > > > > valid > > > > > > directory path, then nfsd will act as if it is confined to > > > > > > a > > > > > > chroot > > > > > > jail based on that directory. All paths in /etc/exporfs and > > > > > > from > > > > > > exportfs are then resolved relative to that directory. > > > > > > > > > > What about files under /proc that mountd might access? I > > > > > assume > > > > > these > > > > > pathnames are not affected. > > > > > > > > > That's why we have 2 threads. One thread is root jailed using > > > > chroot, > > > > and is used to talk to knfsd. The other thread is not root > > > > jailed > > > > (or > > > > at least not by root_dir) and so has full access to /etc, > > > > /proc, > > > > /var, > > > > ... > > > > > > > > > Aren't there also one or two other files that maintain export > > > > > state > > > > > like /var/lib/nfs/rmtab? Are those affected? > > > > > > > > See above. They are not affected. > > > > > > > > > IMHO it could be less confusing to administrators to make > > > > > root_dir an > > > > > [exportfs] option instead of a [mountd] option, if this is > > > > > not a > > > > > true > > > > > chroot of mountd. > > > > > > > > It is neither. I made in a [nfsd] option, since it governs the > > > > way > > > > that > > > > both exportfs and mountd talk to nfsd. > > > > > > My point is not about implementation, it's about how this > > > functionality > > > is presented to administrators. > > > > > > In nfs.conf, [nfsd] looks like it controls what options are > > > passed > > > via > > > rpc.nfsd. That still seems like a confusing admin interface. > > > > > > IMO admins won't care about who is talking to whom. They will > > > care > > > about > > > how the export pathnames are interpreted. That seems like it > > > belongs > > > squarely with the exportfs interface. > > > > > > > With the exportfs interface, yes. However it is not specific to the > > exportfs utility, so to me [exportfs] is more confusing than what > > exists now. > > > > OK, so what if we put it in [general] instead, and perhaps rename > > it > > "export_rootdir"? > > > I'm just catching up... my apologies tartness... > > So setting root_dir effects *all* exports in /etc/exports? > If that is the case, that one variable can change hundreds > of export... is that what we really want? > > Wouldn't be better to have a little more granularity? Can you explain what you mean? The intention here is that if you have all your exported filesystems set up in a subtree under /mnt/my/exports, then you can remove that unnecessary prefix. So, for instance, if I'm trying to export /mnt/my/exports/foo and /mnt/my/exports/bar, then I can make those two filesystems appear as /foo, and /bar to the remote clients. If an admin wants to rearrange all the paths in /etc/exports, and make a custom namespace, then that is possible using bind mounts: just create a directory /my_exports, and use mount --bind to attach the necessary mountpoints into the right spots in /my_exports, then use export_rootdir to remove the /my_exports prefix. > As for where root_dir should go, I think it makes senses > to create a new [exportfs] section and have mountd read it > from there. I think that would be more straightforward if > we continue with the big hammer approach where any and all > exports are effected. > Fair enough, I can add the [exports] section if you all agree that is an appropriate place.
> On May 28, 2019, at 12:44 PM, Trond Myklebust <trondmy@hammerspace.com> wrote: > > On Tue, 2019-05-28 at 11:25 -0400, Steve Dickson wrote: >> >> On 5/21/19 3:58 PM, Trond Myklebust wrote: >>> On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: >>>>> On May 21, 2019, at 2:17 PM, Trond Myklebust < >>>>> trondmy@hammerspace.com> wrote: >>>>> >>>>> On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: >>>>>> Hi Trond - >>>>>> >>>>>>> On May 21, 2019, at 8:46 AM, Trond Myklebust < >>>>>>> trondmy@gmail.com >>>>>>> wrote: >>>>>>> >>>>>>> The following patchset adds support for the 'root_dir' >>>>>>> configuration >>>>>>> option for nfsd in nfs.conf. If a user sets this option to >>>>>>> a >>>>>>> valid >>>>>>> directory path, then nfsd will act as if it is confined to >>>>>>> a >>>>>>> chroot >>>>>>> jail based on that directory. All paths in /etc/exporfs and >>>>>>> from >>>>>>> exportfs are then resolved relative to that directory. >>>>>> >>>>>> What about files under /proc that mountd might access? I >>>>>> assume >>>>>> these >>>>>> pathnames are not affected. >>>>>> >>>>> That's why we have 2 threads. One thread is root jailed using >>>>> chroot, >>>>> and is used to talk to knfsd. The other thread is not root >>>>> jailed >>>>> (or >>>>> at least not by root_dir) and so has full access to /etc, >>>>> /proc, >>>>> /var, >>>>> ... >>>>> >>>>>> Aren't there also one or two other files that maintain export >>>>>> state >>>>>> like /var/lib/nfs/rmtab? Are those affected? >>>>> >>>>> See above. They are not affected. >>>>> >>>>>> IMHO it could be less confusing to administrators to make >>>>>> root_dir an >>>>>> [exportfs] option instead of a [mountd] option, if this is >>>>>> not a >>>>>> true >>>>>> chroot of mountd. >>>>> >>>>> It is neither. I made in a [nfsd] option, since it governs the >>>>> way >>>>> that >>>>> both exportfs and mountd talk to nfsd. >>>> >>>> My point is not about implementation, it's about how this >>>> functionality >>>> is presented to administrators. >>>> >>>> In nfs.conf, [nfsd] looks like it controls what options are >>>> passed >>>> via >>>> rpc.nfsd. That still seems like a confusing admin interface. >>>> >>>> IMO admins won't care about who is talking to whom. They will >>>> care >>>> about >>>> how the export pathnames are interpreted. That seems like it >>>> belongs >>>> squarely with the exportfs interface. >>>> >>> >>> With the exportfs interface, yes. However it is not specific to the >>> exportfs utility, so to me [exportfs] is more confusing than what >>> exists now. >>> >>> OK, so what if we put it in [general] instead, and perhaps rename >>> it >>> "export_rootdir"? >>> >> I'm just catching up... my apologies tartness... >> >> So setting root_dir effects *all* exports in /etc/exports? >> If that is the case, that one variable can change hundreds >> of export... is that what we really want? >> >> Wouldn't be better to have a little more granularity? > > Can you explain what you mean? The intention here is that if you have > all your exported filesystems set up in a subtree under > /mnt/my/exports, then you can remove that unnecessary prefix. > > So, for instance, if I'm trying to export /mnt/my/exports/foo and > /mnt/my/exports/bar, then I can make those two filesystems appear as > /foo, and /bar to the remote clients. > > If an admin wants to rearrange all the paths in /etc/exports, and make > a custom namespace, then that is possible using bind mounts: just > create a directory /my_exports, and use mount --bind to attach the > necessary mountpoints into the right spots in /my_exports, then use > export_rootdir to remove the /my_exports prefix. Just to be clear, do you expect that each mount namespace on a Linux NFS server would have its own /etc/exports and /etc/nfs.conf ? Maybe you stated that before, and I missed it. >> As for where root_dir should go, I think it makes senses >> to create a new [exportfs] section and have mountd read it >> from there. I think that would be more straightforward if >> we continue with the big hammer approach where any and all >> exports are effected. >> > > Fair enough, I can add the [exports] section if you all agree that is > an appropriate place. > > -- > Trond Myklebust > Linux NFS client maintainer, Hammerspace > trond.myklebust@hammerspace.com -- Chuck Lever chucklever@gmail.com
On Tue, 2019-05-28 at 12:47 -0400, Chuck Lever wrote: > > On May 28, 2019, at 12:44 PM, Trond Myklebust < > > trondmy@hammerspace.com> wrote: > > > > On Tue, 2019-05-28 at 11:25 -0400, Steve Dickson wrote: > > > On 5/21/19 3:58 PM, Trond Myklebust wrote: > > > > On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: > > > > > > On May 21, 2019, at 2:17 PM, Trond Myklebust < > > > > > > trondmy@hammerspace.com> wrote: > > > > > > > > > > > > On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: > > > > > > > Hi Trond - > > > > > > > > > > > > > > > On May 21, 2019, at 8:46 AM, Trond Myklebust < > > > > > > > > trondmy@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > > > > The following patchset adds support for the 'root_dir' > > > > > > > > configuration > > > > > > > > option for nfsd in nfs.conf. If a user sets this option > > > > > > > > to > > > > > > > > a > > > > > > > > valid > > > > > > > > directory path, then nfsd will act as if it is confined > > > > > > > > to > > > > > > > > a > > > > > > > > chroot > > > > > > > > jail based on that directory. All paths in /etc/exporfs > > > > > > > > and > > > > > > > > from > > > > > > > > exportfs are then resolved relative to that directory. > > > > > > > > > > > > > > What about files under /proc that mountd might access? I > > > > > > > assume > > > > > > > these > > > > > > > pathnames are not affected. > > > > > > > > > > > > > That's why we have 2 threads. One thread is root jailed > > > > > > using > > > > > > chroot, > > > > > > and is used to talk to knfsd. The other thread is not root > > > > > > jailed > > > > > > (or > > > > > > at least not by root_dir) and so has full access to /etc, > > > > > > /proc, > > > > > > /var, > > > > > > ... > > > > > > > > > > > > > Aren't there also one or two other files that maintain > > > > > > > export > > > > > > > state > > > > > > > like /var/lib/nfs/rmtab? Are those affected? > > > > > > > > > > > > See above. They are not affected. > > > > > > > > > > > > > IMHO it could be less confusing to administrators to make > > > > > > > root_dir an > > > > > > > [exportfs] option instead of a [mountd] option, if this > > > > > > > is > > > > > > > not a > > > > > > > true > > > > > > > chroot of mountd. > > > > > > > > > > > > It is neither. I made in a [nfsd] option, since it governs > > > > > > the > > > > > > way > > > > > > that > > > > > > both exportfs and mountd talk to nfsd. > > > > > > > > > > My point is not about implementation, it's about how this > > > > > functionality > > > > > is presented to administrators. > > > > > > > > > > In nfs.conf, [nfsd] looks like it controls what options are > > > > > passed > > > > > via > > > > > rpc.nfsd. That still seems like a confusing admin interface. > > > > > > > > > > IMO admins won't care about who is talking to whom. They will > > > > > care > > > > > about > > > > > how the export pathnames are interpreted. That seems like it > > > > > belongs > > > > > squarely with the exportfs interface. > > > > > > > > > > > > > With the exportfs interface, yes. However it is not specific to > > > > the > > > > exportfs utility, so to me [exportfs] is more confusing than > > > > what > > > > exists now. > > > > > > > > OK, so what if we put it in [general] instead, and perhaps > > > > rename > > > > it > > > > "export_rootdir"? > > > > > > > I'm just catching up... my apologies tartness... > > > > > > So setting root_dir effects *all* exports in /etc/exports? > > > If that is the case, that one variable can change hundreds > > > of export... is that what we really want? > > > > > > Wouldn't be better to have a little more granularity? > > > > Can you explain what you mean? The intention here is that if you > > have > > all your exported filesystems set up in a subtree under > > /mnt/my/exports, then you can remove that unnecessary prefix. > > > > So, for instance, if I'm trying to export /mnt/my/exports/foo and > > /mnt/my/exports/bar, then I can make those two filesystems appear > > as > > /foo, and /bar to the remote clients. > > > > If an admin wants to rearrange all the paths in /etc/exports, and > > make > > a custom namespace, then that is possible using bind mounts: just > > create a directory /my_exports, and use mount --bind to attach the > > necessary mountpoints into the right spots in /my_exports, then use > > export_rootdir to remove the /my_exports prefix. > > Just to be clear, do you expect that each mount namespace on a > Linux NFS server would have its own /etc/exports and /etc/nfs.conf ? > > Maybe you stated that before, and I missed it. Yes, if you are running in a containerised environment, then you will have your own /etc/exports, /etc/nfs.conf, etc.
On 5/28/19 12:44 PM, Trond Myklebust wrote: > On Tue, 2019-05-28 at 11:25 -0400, Steve Dickson wrote: >> >> On 5/21/19 3:58 PM, Trond Myklebust wrote: >>> On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: >>>>> On May 21, 2019, at 2:17 PM, Trond Myklebust < >>>>> trondmy@hammerspace.com> wrote: >>>>> >>>>> On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: >>>>>> Hi Trond - >>>>>> >>>>>>> On May 21, 2019, at 8:46 AM, Trond Myklebust < >>>>>>> trondmy@gmail.com >>>>>>> wrote: >>>>>>> >>>>>>> The following patchset adds support for the 'root_dir' >>>>>>> configuration >>>>>>> option for nfsd in nfs.conf. If a user sets this option to >>>>>>> a >>>>>>> valid >>>>>>> directory path, then nfsd will act as if it is confined to >>>>>>> a >>>>>>> chroot >>>>>>> jail based on that directory. All paths in /etc/exporfs and >>>>>>> from >>>>>>> exportfs are then resolved relative to that directory. >>>>>> >>>>>> What about files under /proc that mountd might access? I >>>>>> assume >>>>>> these >>>>>> pathnames are not affected. >>>>>> >>>>> That's why we have 2 threads. One thread is root jailed using >>>>> chroot, >>>>> and is used to talk to knfsd. The other thread is not root >>>>> jailed >>>>> (or >>>>> at least not by root_dir) and so has full access to /etc, >>>>> /proc, >>>>> /var, >>>>> ... >>>>> >>>>>> Aren't there also one or two other files that maintain export >>>>>> state >>>>>> like /var/lib/nfs/rmtab? Are those affected? >>>>> >>>>> See above. They are not affected. >>>>> >>>>>> IMHO it could be less confusing to administrators to make >>>>>> root_dir an >>>>>> [exportfs] option instead of a [mountd] option, if this is >>>>>> not a >>>>>> true >>>>>> chroot of mountd. >>>>> >>>>> It is neither. I made in a [nfsd] option, since it governs the >>>>> way >>>>> that >>>>> both exportfs and mountd talk to nfsd. >>>> >>>> My point is not about implementation, it's about how this >>>> functionality >>>> is presented to administrators. >>>> >>>> In nfs.conf, [nfsd] looks like it controls what options are >>>> passed >>>> via >>>> rpc.nfsd. That still seems like a confusing admin interface. >>>> >>>> IMO admins won't care about who is talking to whom. They will >>>> care >>>> about >>>> how the export pathnames are interpreted. That seems like it >>>> belongs >>>> squarely with the exportfs interface. >>>> >>> >>> With the exportfs interface, yes. However it is not specific to the >>> exportfs utility, so to me [exportfs] is more confusing than what >>> exists now. >>> >>> OK, so what if we put it in [general] instead, and perhaps rename >>> it >>> "export_rootdir"? >>> >> I'm just catching up... my apologies tartness... >> >> So setting root_dir effects *all* exports in /etc/exports? >> If that is the case, that one variable can change hundreds >> of export... is that what we really want? >> >> Wouldn't be better to have a little more granularity? > > Can you explain what you mean? The intention here is that if you have > all your exported filesystems set up in a subtree under > /mnt/my/exports, then you can remove that unnecessary prefix. > > So, for instance, if I'm trying to export /mnt/my/exports/foo and > /mnt/my/exports/bar, then I can make those two filesystems appear as > /foo, and /bar to the remote clients. By granularity I meant have different roots for different exports. Meaning /mnt/foo/exports/foo and /mnt/bar/exports/bar would still appear as /foo and /bar As you explain later in this thread, there is going to be a nfs.conf and exports for each container so maybe this is not necessary?? Maybe I'm misunderstanding how this feature should/will be used. > > If an admin wants to rearrange all the paths in /etc/exports, and make > a custom namespace, then that is possible using bind mounts: just > create a directory /my_exports, and use mount --bind to attach the > necessary mountpoints into the right spots in /my_exports, then use > export_rootdir to remove the /my_exports prefix. > >> As for where root_dir should go, I think it makes senses >> to create a new [exportfs] section and have mountd read it >> from there. I think that would be more straightforward if >> we continue with the big hammer approach where any and all >> exports are effected. >> > > Fair enough, I can add the [exports] section if you all agree that is > an appropriate place. > I think a new exports sections with a rootdir variable makes sense. It is changing the root of the exports... But I could also live with a export_rootdir in the general section. Question: How is this different than pseudo root? Isn't this basically a way to set the pseudo for v3? What is going to override whom? Meaning if both fsid=/mnt/foo and rootdir=/mnt/bar which one will be used? steved. steved.
On Tue, 2019-05-28 at 13:40 -0400, Steve Dickson wrote: > > On 5/28/19 12:44 PM, Trond Myklebust wrote: > > On Tue, 2019-05-28 at 11:25 -0400, Steve Dickson wrote: > > > On 5/21/19 3:58 PM, Trond Myklebust wrote: > > > > On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: > > > > > > On May 21, 2019, at 2:17 PM, Trond Myklebust < > > > > > > trondmy@hammerspace.com> wrote: > > > > > > > > > > > > On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: > > > > > > > Hi Trond - > > > > > > > > > > > > > > > On May 21, 2019, at 8:46 AM, Trond Myklebust < > > > > > > > > trondmy@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > > > > The following patchset adds support for the 'root_dir' > > > > > > > > configuration > > > > > > > > option for nfsd in nfs.conf. If a user sets this option > > > > > > > > to > > > > > > > > a > > > > > > > > valid > > > > > > > > directory path, then nfsd will act as if it is confined > > > > > > > > to > > > > > > > > a > > > > > > > > chroot > > > > > > > > jail based on that directory. All paths in /etc/exporfs > > > > > > > > and > > > > > > > > from > > > > > > > > exportfs are then resolved relative to that directory. > > > > > > > > > > > > > > What about files under /proc that mountd might access? I > > > > > > > assume > > > > > > > these > > > > > > > pathnames are not affected. > > > > > > > > > > > > > That's why we have 2 threads. One thread is root jailed > > > > > > using > > > > > > chroot, > > > > > > and is used to talk to knfsd. The other thread is not root > > > > > > jailed > > > > > > (or > > > > > > at least not by root_dir) and so has full access to /etc, > > > > > > /proc, > > > > > > /var, > > > > > > ... > > > > > > > > > > > > > Aren't there also one or two other files that maintain > > > > > > > export > > > > > > > state > > > > > > > like /var/lib/nfs/rmtab? Are those affected? > > > > > > > > > > > > See above. They are not affected. > > > > > > > > > > > > > IMHO it could be less confusing to administrators to make > > > > > > > root_dir an > > > > > > > [exportfs] option instead of a [mountd] option, if this > > > > > > > is > > > > > > > not a > > > > > > > true > > > > > > > chroot of mountd. > > > > > > > > > > > > It is neither. I made in a [nfsd] option, since it governs > > > > > > the > > > > > > way > > > > > > that > > > > > > both exportfs and mountd talk to nfsd. > > > > > > > > > > My point is not about implementation, it's about how this > > > > > functionality > > > > > is presented to administrators. > > > > > > > > > > In nfs.conf, [nfsd] looks like it controls what options are > > > > > passed > > > > > via > > > > > rpc.nfsd. That still seems like a confusing admin interface. > > > > > > > > > > IMO admins won't care about who is talking to whom. They will > > > > > care > > > > > about > > > > > how the export pathnames are interpreted. That seems like it > > > > > belongs > > > > > squarely with the exportfs interface. > > > > > > > > > > > > > With the exportfs interface, yes. However it is not specific to > > > > the > > > > exportfs utility, so to me [exportfs] is more confusing than > > > > what > > > > exists now. > > > > > > > > OK, so what if we put it in [general] instead, and perhaps > > > > rename > > > > it > > > > "export_rootdir"? > > > > > > > I'm just catching up... my apologies tartness... > > > > > > So setting root_dir effects *all* exports in /etc/exports? > > > If that is the case, that one variable can change hundreds > > > of export... is that what we really want? > > > > > > Wouldn't be better to have a little more granularity? > > > > Can you explain what you mean? The intention here is that if you > > have > > all your exported filesystems set up in a subtree under > > /mnt/my/exports, then you can remove that unnecessary prefix. > > > > So, for instance, if I'm trying to export /mnt/my/exports/foo and > > /mnt/my/exports/bar, then I can make those two filesystems appear > > as > > /foo, and /bar to the remote clients. > By granularity I meant have different roots for different exports. > Meaning /mnt/foo/exports/foo and /mnt/bar/exports/bar > would still appear as /foo and /bar No. That should be done using bind mounts. Otherwise we end up with /etc/nfs.conf and /etc/exports depending on being mutually consistent. That would be awkward. > As you explain later in this thread, there is going to be a nfs.conf > and exports for each container so maybe this is not necessary?? > > Maybe I'm misunderstanding how this feature should/will be used. As I've already said, it can be used to do what you are proposing, but only in conjunction with bind mounts. > > > If an admin wants to rearrange all the paths in /etc/exports, and > > make > > a custom namespace, then that is possible using bind mounts: just > > create a directory /my_exports, and use mount --bind to attach the > > necessary mountpoints into the right spots in /my_exports, then use > > export_rootdir to remove the /my_exports prefix. > > > > > As for where root_dir should go, I think it makes senses > > > to create a new [exportfs] section and have mountd read it > > > from there. I think that would be more straightforward if > > > we continue with the big hammer approach where any and all > > > exports are effected. > > > > > > > Fair enough, I can add the [exports] section if you all agree that > > is > > an appropriate place. > > > I think a new exports sections with a rootdir variable makes sense. > It is changing the root of the exports... > > But I could also live with a export_rootdir in the general section. > > Question: > How is this different than pseudo root? > > Isn't this basically a way to set the pseudo for v3? Sort of, yes. > What is going to override whom? Meaning if both > fsid=/mnt/foo and rootdir=/mnt/bar which one will be used? > > Both. However the entry in /etc/exports will be relative to /mnt/bar. In other words, the NFSv4 root would be fsid=/mnt/foo, which translates as /mnt/bar/mnt/foo in the 'init' namespace.
On 5/28/19 2:19 PM, Trond Myklebust wrote: > On Tue, 2019-05-28 at 13:40 -0400, Steve Dickson wrote: >> >> On 5/28/19 12:44 PM, Trond Myklebust wrote: >>> On Tue, 2019-05-28 at 11:25 -0400, Steve Dickson wrote: >>>> On 5/21/19 3:58 PM, Trond Myklebust wrote: >>>>> On Tue, 2019-05-21 at 15:06 -0400, Chuck Lever wrote: >>>>>>> On May 21, 2019, at 2:17 PM, Trond Myklebust < >>>>>>> trondmy@hammerspace.com> wrote: >>>>>>> >>>>>>> On Tue, 2019-05-21 at 13:40 -0400, Chuck Lever wrote: >>>>>>>> Hi Trond - >>>>>>>> >>>>>>>>> On May 21, 2019, at 8:46 AM, Trond Myklebust < >>>>>>>>> trondmy@gmail.com >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> The following patchset adds support for the 'root_dir' >>>>>>>>> configuration >>>>>>>>> option for nfsd in nfs.conf. If a user sets this option >>>>>>>>> to >>>>>>>>> a >>>>>>>>> valid >>>>>>>>> directory path, then nfsd will act as if it is confined >>>>>>>>> to >>>>>>>>> a >>>>>>>>> chroot >>>>>>>>> jail based on that directory. All paths in /etc/exporfs >>>>>>>>> and >>>>>>>>> from >>>>>>>>> exportfs are then resolved relative to that directory. >>>>>>>> >>>>>>>> What about files under /proc that mountd might access? I >>>>>>>> assume >>>>>>>> these >>>>>>>> pathnames are not affected. >>>>>>>> >>>>>>> That's why we have 2 threads. One thread is root jailed >>>>>>> using >>>>>>> chroot, >>>>>>> and is used to talk to knfsd. The other thread is not root >>>>>>> jailed >>>>>>> (or >>>>>>> at least not by root_dir) and so has full access to /etc, >>>>>>> /proc, >>>>>>> /var, >>>>>>> ... >>>>>>> >>>>>>>> Aren't there also one or two other files that maintain >>>>>>>> export >>>>>>>> state >>>>>>>> like /var/lib/nfs/rmtab? Are those affected? >>>>>>> >>>>>>> See above. They are not affected. >>>>>>> >>>>>>>> IMHO it could be less confusing to administrators to make >>>>>>>> root_dir an >>>>>>>> [exportfs] option instead of a [mountd] option, if this >>>>>>>> is >>>>>>>> not a >>>>>>>> true >>>>>>>> chroot of mountd. >>>>>>> >>>>>>> It is neither. I made in a [nfsd] option, since it governs >>>>>>> the >>>>>>> way >>>>>>> that >>>>>>> both exportfs and mountd talk to nfsd. >>>>>> >>>>>> My point is not about implementation, it's about how this >>>>>> functionality >>>>>> is presented to administrators. >>>>>> >>>>>> In nfs.conf, [nfsd] looks like it controls what options are >>>>>> passed >>>>>> via >>>>>> rpc.nfsd. That still seems like a confusing admin interface. >>>>>> >>>>>> IMO admins won't care about who is talking to whom. They will >>>>>> care >>>>>> about >>>>>> how the export pathnames are interpreted. That seems like it >>>>>> belongs >>>>>> squarely with the exportfs interface. >>>>>> >>>>> >>>>> With the exportfs interface, yes. However it is not specific to >>>>> the >>>>> exportfs utility, so to me [exportfs] is more confusing than >>>>> what >>>>> exists now. >>>>> >>>>> OK, so what if we put it in [general] instead, and perhaps >>>>> rename >>>>> it >>>>> "export_rootdir"? >>>>> >>>> I'm just catching up... my apologies tartness... >>>> >>>> So setting root_dir effects *all* exports in /etc/exports? >>>> If that is the case, that one variable can change hundreds >>>> of export... is that what we really want? >>>> >>>> Wouldn't be better to have a little more granularity? >>> >>> Can you explain what you mean? The intention here is that if you >>> have >>> all your exported filesystems set up in a subtree under >>> /mnt/my/exports, then you can remove that unnecessary prefix. >>> >>> So, for instance, if I'm trying to export /mnt/my/exports/foo and >>> /mnt/my/exports/bar, then I can make those two filesystems appear >>> as >>> /foo, and /bar to the remote clients. >> By granularity I meant have different roots for different exports. >> Meaning /mnt/foo/exports/foo and /mnt/bar/exports/bar >> would still appear as /foo and /bar > > No. That should be done using bind mounts. Otherwise we end up with > /etc/nfs.conf and /etc/exports depending on being mutually consistent. > That would be awkward. Fine... > >> As you explain later in this thread, there is going to be a nfs.conf >> and exports for each container so maybe this is not necessary?? >> >> Maybe I'm misunderstanding how this feature should/will be used. > > As I've already said, it can be used to do what you are proposing, but > only in conjunction with bind mounts. > >> >>> If an admin wants to rearrange all the paths in /etc/exports, and >>> make >>> a custom namespace, then that is possible using bind mounts: just >>> create a directory /my_exports, and use mount --bind to attach the >>> necessary mountpoints into the right spots in /my_exports, then use >>> export_rootdir to remove the /my_exports prefix. >>> >>>> As for where root_dir should go, I think it makes senses >>>> to create a new [exportfs] section and have mountd read it >>>> from there. I think that would be more straightforward if >>>> we continue with the big hammer approach where any and all >>>> exports are effected. >>>> >>> >>> Fair enough, I can add the [exports] section if you all agree that >>> is >>> an appropriate place. >>> >> I think a new exports sections with a rootdir variable makes sense. >> It is changing the root of the exports... >> >> But I could also live with a export_rootdir in the general section. >> >> Question: >> How is this different than pseudo root? >> >> Isn't this basically a way to set the pseudo for v3? > > Sort of, yes. > >> What is going to override whom? Meaning if both >> fsid=/mnt/foo and rootdir=/mnt/bar which one will be used? >> >> > Both. However the entry in /etc/exports will be relative to /mnt/bar. > In other words, the NFSv4 root would be fsid=/mnt/foo, which translates > as /mnt/bar/mnt/foo in the 'init' namespace. > Ok... So what do you want to do... [exports] rootdir=/mnt/foo or [general] export_rootdir=/mnt/bar steved.