| Message ID | 162035212343.24322.12361160756597283121@noble.neil.brown.name (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [PATCH/RFC,nfs-utils] Fix NFSv4 export of tmpfs filesystems. | expand |
Hi Neil, > [[This is a proposed fix. It seems to work. I'd like > some review comments before it is committed. > Petr: it would be great if you could test it to confirm > it actually works in your case. > ]] Thanks for a quick fix. It runs nicely in newer kernels (5.11.12-1-default openSUSE and 5.10.0-6-amd64 Debian). But it somehow fails on older ones (SLES 5.3.18-54-default heavily patched and 4.9.0-11-amd64). I have some problem on Debian with 4.9.0-11-amd64 fails on both tmpfs and ext4, others work fine (testing tmpfs, btrfs and ext4). But maybe I did something wrong during testing. I did: cp ./utils/mountd/mountd /usr/sbin/rpc.mountd systemctl restart nfs-mountd.service Failure is regardless I use new mount.nfs (master) or the original from Debian (1.3.3). strace looks nearly the same on tmpfs and ext4: socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 5 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 connect(5, {sa_family=AF_INET, sin_port=htons(111), sin_addr=inet_addr("10.0.0.2")}, 16) = -1 EINPROGRESS (Operation now in progress) select(6, NULL, [5], NULL, {tv_sec=10, tv_usec=0}) = 1 (out [5], left {tv_sec=9, tv_usec=999997}) getsockopt(5, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 fcntl(5, F_SETFL, O_RDWR) = 0 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 getpeername(5, {sa_family=AF_INET, sin_port=htons(111), sin_addr=inet_addr("10.0.0.2")}, [128->16]) = 0 getsockname(5, {sa_family=AF_INET, sin_port=htons(54140), sin_addr=inet_addr("10.0.0.1")}, [128->16]) = 0 getsockopt(5, SOL_SOCKET, SO_TYPE, [1], [4]) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 getpid() = 920 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 write(5, "\200\0\0008bZ\360\303\0\0\0\0\0\0\0\2\0\1\206\240\0\0\0\2\0\0\0\3\0\0\0\0"..., 60) = 60 poll([{fd=5, events=POLLIN}], 1, 9999) = 1 ([{fd=5, revents=POLLIN}]) read(5, "\200\0\0\34bZ\360\303\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\10\1", 65536) = 32 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 close(5) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 5 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 Kind regards, Petr
> On May 6, 2021, at 9:48 PM, NeilBrown <neilb@suse.de> wrote: > > > [[This is a proposed fix. It seems to work. I'd like > some review comments before it is committed. > Petr: it would be great if you could test it to confirm > it actually works in your case. > ]] > > Some filesystems cannot be exported without an fsid or uuid. > tmpfs is the main example. > > When mountd creates nfsv4 pseudo-root exports for the path leading down > to an export point it exports each directory without any fsid or uuid. > If one of these directories is on tmp, that will fail. > > The net result is that exporting a subdirectory of a tmpfs filesystem > will not work over NFSv4 as the parents within the filesystem cannot be > exported. It will either fail, or fall-back to NFSv3 (depending on the > version of the mount.nfs program). > > To fix this we need to provide an fsid or uuid for these pseudo-root > exports. This patch does that by creating a UUID with the first 4 bytes > 0xFFFFFFFF and the remaining 12 bytes form from the path name, xoring > bytes together if the path is longer than 12 characters. > Hopefully no filesystem uses a UUID like this.... That's not really a UUID, as per RFC 4122. I'm guessing it's possible for a collision to occur pretty quickly, for instance. It would be nicer if a conformant UUID could be used here. Is there a problem with specifying the export's fsid in /etc/exports? > The patch borrows some code from exportfs. Maybe that code should be > move to a library.. > > Signed-off-by: NeilBrown <neilb@suse.de> > --- > support/export/v4root.c | 57 +++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 57 insertions(+) > > diff --git a/support/export/v4root.c b/support/export/v4root.c > index 3654bd7c10c0..fd36eb704441 100644 > --- a/support/export/v4root.c > +++ b/support/export/v4root.c > @@ -11,6 +11,7 @@ > #include <config.h> > #endif > > +#include <fcntl.h> > #include <sys/types.h> > #include <sys/stat.h> > #include <sys/queue.h> > @@ -21,6 +22,7 @@ > #include <unistd.h> > #include <errno.h> > > +#include "nfsd_path.h" > #include "xlog.h" > #include "exportfs.h" > #include "nfslib.h" > @@ -73,6 +75,38 @@ set_pseudofs_security(struct exportent *pseudo) > } > } > > +static ssize_t exportfs_write(int fd, const char *buf, size_t len) > +{ > + return nfsd_path_write(fd, buf, len); > +} > + > +static int test_export(struct exportent *eep, int with_fsid) > +{ > + char *path = eep->e_path; > + int flags = eep->e_flags | (with_fsid ? NFSEXP_FSID : 0); > + /* beside max path, buf size should take protocol str into account */ > + char buf[NFS_MAXPATHLEN+1+64] = { 0 }; > + char *bp = buf; > + int len = sizeof(buf); > + int fd, n; > + > + n = snprintf(buf, len, "-test-client- "); > + bp += n; > + len -= n; > + qword_add(&bp, &len, path); > + if (len < 1) > + return 0; > + snprintf(bp, len, " 3 %d 65534 65534 0\n", flags); > + fd = open("/proc/net/rpc/nfsd.export/channel", O_WRONLY); > + if (fd < 0) > + return 0; > + n = exportfs_write(fd, buf, strlen(buf)); > + close(fd); > + if (n < 0) > + return 0; > + return 1; > +} > + > /* > * Create a pseudo export > */ > @@ -82,6 +116,7 @@ v4root_create(char *path, nfs_export *export) > nfs_export *exp; > struct exportent eep; > struct exportent *curexp = &export->m_export; > + char uuid[33]; > > dupexportent(&eep, &pseudo_root.m_export); > eep.e_ttl = default_ttl; > @@ -89,6 +124,28 @@ v4root_create(char *path, nfs_export *export) > strncpy(eep.e_path, path, sizeof(eep.e_path)-1); > if (strcmp(path, "/") != 0) > eep.e_flags &= ~NFSEXP_FSID; > + if (strcmp(path, "/") != 0 && > + !test_export(&eep, 0)) { > + /* Need a uuid - base it on path */ > + char buf[12], *pp = path; > + unsigned int i = 0; > + > + memset(buf, 0, sizeof(buf)); > + while (*pp) { > + buf[i] ^= *pp++; > + i += 1; > + if (i >= sizeof(buf)) > + i = 0; > + } > + memset(uuid, 'F', 32); > + uuid[32] = '\0'; > + pp = uuid + 32 - sizeof(buf) * 2; > + for (i = 0; i < sizeof(buf); i++) { > + snprintf(pp, 3, "%02X", buf[i]); > + pp += 2; > + } > + eep.e_uuid = uuid; > + } > set_pseudofs_security(&eep); > exp = export_create(&eep, 0); > if (exp == NULL) > -- > 2.31.1 > -- Chuck Lever
On Fri, 07 May 2021, Chuck Lever III wrote: > > That's not really a UUID, as per RFC 4122. I'm guessing it's possible > for a collision to occur pretty quickly, for instance. It would be nicer > if a conformant UUID could be used here. That sounds like a sensible approach. I'll go and read RFC 4122 and see what I can learn. > > Is there a problem with specifying the export's fsid in /etc/exports? Each ancestor directory of any export point needs to be exported with the v4root flag, and those that are on tmpfs each need a unique uuid or fsid. Requiring that to be specified in /etc/exports is an extra burden to impose on admin So yes, I think it s a problem with requiring that specification. (I'm not even sure if NFSEXP_V4ROOT exports can be specified in /etc/exports, but that would be easy to fix of course. Thanks, NeilBrown > > > > The patch borrows some code from exportfs. Maybe that code should be > > move to a library.. > > > > Signed-off-by: NeilBrown <neilb@suse.de> > > --- > > support/export/v4root.c | 57 +++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 57 insertions(+) > > > > diff --git a/support/export/v4root.c b/support/export/v4root.c > > index 3654bd7c10c0..fd36eb704441 100644 > > --- a/support/export/v4root.c > > +++ b/support/export/v4root.c > > @@ -11,6 +11,7 @@ > > #include <config.h> > > #endif > > > > +#include <fcntl.h> > > #include <sys/types.h> > > #include <sys/stat.h> > > #include <sys/queue.h> > > @@ -21,6 +22,7 @@ > > #include <unistd.h> > > #include <errno.h> > > > > +#include "nfsd_path.h" > > #include "xlog.h" > > #include "exportfs.h" > > #include "nfslib.h" > > @@ -73,6 +75,38 @@ set_pseudofs_security(struct exportent *pseudo) > > } > > } > > > > +static ssize_t exportfs_write(int fd, const char *buf, size_t len) > > +{ > > + return nfsd_path_write(fd, buf, len); > > +} > > + > > +static int test_export(struct exportent *eep, int with_fsid) > > +{ > > + char *path = eep->e_path; > > + int flags = eep->e_flags | (with_fsid ? NFSEXP_FSID : 0); > > + /* beside max path, buf size should take protocol str into account */ > > + char buf[NFS_MAXPATHLEN+1+64] = { 0 }; > > + char *bp = buf; > > + int len = sizeof(buf); > > + int fd, n; > > + > > + n = snprintf(buf, len, "-test-client- "); > > + bp += n; > > + len -= n; > > + qword_add(&bp, &len, path); > > + if (len < 1) > > + return 0; > > + snprintf(bp, len, " 3 %d 65534 65534 0\n", flags); > > + fd = open("/proc/net/rpc/nfsd.export/channel", O_WRONLY); > > + if (fd < 0) > > + return 0; > > + n = exportfs_write(fd, buf, strlen(buf)); > > + close(fd); > > + if (n < 0) > > + return 0; > > + return 1; > > +} > > + > > /* > > * Create a pseudo export > > */ > > @@ -82,6 +116,7 @@ v4root_create(char *path, nfs_export *export) > > nfs_export *exp; > > struct exportent eep; > > struct exportent *curexp = &export->m_export; > > + char uuid[33]; > > > > dupexportent(&eep, &pseudo_root.m_export); > > eep.e_ttl = default_ttl; > > @@ -89,6 +124,28 @@ v4root_create(char *path, nfs_export *export) > > strncpy(eep.e_path, path, sizeof(eep.e_path)-1); > > if (strcmp(path, "/") != 0) > > eep.e_flags &= ~NFSEXP_FSID; > > + if (strcmp(path, "/") != 0 && > > + !test_export(&eep, 0)) { > > + /* Need a uuid - base it on path */ > > + char buf[12], *pp = path; > > + unsigned int i = 0; > > + > > + memset(buf, 0, sizeof(buf)); > > + while (*pp) { > > + buf[i] ^= *pp++; > > + i += 1; > > + if (i >= sizeof(buf)) > > + i = 0; > > + } > > + memset(uuid, 'F', 32); > > + uuid[32] = '\0'; > > + pp = uuid + 32 - sizeof(buf) * 2; > > + for (i = 0; i < sizeof(buf); i++) { > > + snprintf(pp, 3, "%02X", buf[i]); > > + pp += 2; > > + } > > + eep.e_uuid = uuid; > > + } > > set_pseudofs_security(&eep); > > exp = export_create(&eep, 0); > > if (exp == NULL) > > -- > > 2.31.1 > > > > -- > Chuck Lever > > > >
On Fri, 07 May 2021, Petr Vorel wrote: > Hi Neil, > > > [[This is a proposed fix. It seems to work. I'd like > > some review comments before it is committed. > > Petr: it would be great if you could test it to confirm > > it actually works in your case. > > ]] > Thanks for a quick fix. It runs nicely in newer kernels (5.11.12-1-default > openSUSE and 5.10.0-6-amd64 Debian). But it somehow fails on older ones > (SLES 5.3.18-54-default heavily patched and 4.9.0-11-amd64). > > I have some problem on Debian with 4.9.0-11-amd64 fails on both tmpfs and ext4, > others work fine (testing tmpfs, btrfs and ext4). But maybe I did something > wrong during testing. I did: > cp ./utils/mountd/mountd /usr/sbin/rpc.mountd > systemctl restart nfs-mountd.service That is the correct procedure. It should work... > > Failure is regardless I use new mount.nfs (master) or the original from > Debian (1.3.3). What error message do you get on failure? It might help to add "-v" to the mount command to see more messages. > > strace looks nearly the same on tmpfs and ext4: This shows mount.nfs connecting to rpcbind, sending a request, getting a reply, and maybe looping around and trying again? There doesn't seem to be anything kernel related that would affect anything there so I cannot think why on older kernel would make a difference. Or an older rpcbind... Maybe I'll experiment on a SLE12 kernel. NeilBrown
Hi Neil, > On Fri, 07 May 2021, Petr Vorel wrote: > > Hi Neil, > > > [[This is a proposed fix. It seems to work. I'd like > > > some review comments before it is committed. > > > Petr: it would be great if you could test it to confirm > > > it actually works in your case. > > > ]] > > Thanks for a quick fix. It runs nicely in newer kernels (5.11.12-1-default > > openSUSE and 5.10.0-6-amd64 Debian). But it somehow fails on older ones > > (SLES 5.3.18-54-default heavily patched and 4.9.0-11-amd64). > > I have some problem on Debian with 4.9.0-11-amd64 fails on both tmpfs and ext4, > > others work fine (testing tmpfs, btrfs and ext4). But maybe I did something > > wrong during testing. I did: > > cp ./utils/mountd/mountd /usr/sbin/rpc.mountd > > systemctl restart nfs-mountd.service > That is the correct procedure. It should work... Thanks for a confirmation. > > Failure is regardless I use new mount.nfs (master) or the original from > > Debian (1.3.3). > What error message do you get on failure? It might help to add "-v" to > the mount command to see more messages. +1. I'll add it to the tests (+ printing mount.nfs version). Here is debug info (when using nfs-utils 2.5.3 - master, with your patch): * NFSv3 on ext4 (OK) nfs01 1 TINFO: setup NFSv3, socket type tcp nfs01 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=3 10.0.0.2:/var/tmp/LTP_nfs01.14F4lC51P6/3/tcp /var/tmp/LTP_nfs01.14F4lC51P6/3/0 mount.nfs: trying 10.0.0.2 prog 100003 vers 3 prot TCP port 2049 mount.nfs: trying 10.0.0.2 prog 100005 vers 3 prot TCP port 58997 mount.nfs: timeout set for Thu May 13 08:22:46 2021 mount.nfs: trying text-based options 'proto=tcp,vers=3,addr=10.0.0.2' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: prog 100005, trying vers=3, prot=6 nfs01 1 TINFO: starting 'nfs01_open_files 10' * NFSv3 on tmpfs (OK) nfs01 1 TINFO: setup NFSv3, socket type tcp nfs01 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=3 10.0.0.2:/tmp/LTP_nfs01.oJEz2xd9ZI/3/tcp /tmp/LTP_nfs01.oJEz2xd9ZI/3/0 mount.nfs: trying 10.0.0.2 prog 100003 vers 3 prot TCP port 2049 mount.nfs: trying 10.0.0.2 prog 100005 vers 3 prot TCP port 58997 mount.nfs: timeout set for Thu May 13 08:23:01 2021 mount.nfs: trying text-based options 'proto=tcp,vers=3,addr=10.0.0.2' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: prog 100005, trying vers=3, prot=6 nfs01 1 TINFO: starting 'nfs01_open_files 10' * NFSv4.1 on tmpfs (FAIL) nfs01 1 TINFO: setup NFSv4.1, socket type tcp nfs01 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=4.1 10.0.0.2:/tmp/LTP_nfs01.QnnFGEV4rs/4.1/tcp /tmp/LTP_nfs01.QnnFGEV4rs/4.1/0 mount.nfs: mount(2): No such file or directory mount.nfs: mounting 10.0.0.2:/tmp/LTP_nfs01.QnnFGEV4rs/4.1/tcp failed, reason given by server: No such file or directory mount.nfs: timeout set for Thu May 13 08:23:02 2021 mount.nfs: trying text-based options 'proto=tcp,vers=4.1,addr=10.0.0.2,clientaddr=10.0.0.1' nfs01 1 TBROK: mount command failed nfs01 1 TINFO: Cleaning up testcase > > strace looks nearly the same on tmpfs and ext4: > This shows mount.nfs connecting to rpcbind, sending a request, getting a > reply, and maybe looping around and trying again? > There doesn't seem to be anything kernel related that would affect > anything there so I cannot think why on older kernel would make a > difference. Or an older rpcbind... Well, it uses 1.2.5-0.3+deb10u1, i.e. nearly the latest version (Steve released 1.2.6 3 days ago). And it's the same version as in openSUSE Tumbleweed where it works well. > Maybe I'll experiment on a SLE12 kernel. Thanks! > NeilBrown Kind regards, Petr
Hi Neil, all, [Cc also Debian kernel folks as we're trying to fix issues which could hit them. See https://lore.kernel.org/linux-nfs/YILQip3nAxhpXP9+@pevik/T/#t ] > Hi Neil, > > On Fri, 07 May 2021, Petr Vorel wrote: > > > Hi Neil, > > > > [[This is a proposed fix. It seems to work. I'd like > > > > some review comments before it is committed. > > > > Petr: it would be great if you could test it to confirm > > > > it actually works in your case. > > > > ]] > > > Thanks for a quick fix. It runs nicely in newer kernels (5.11.12-1-default > > > openSUSE and 5.10.0-6-amd64 Debian). But it somehow fails on older ones > > > (SLES 5.3.18-54-default heavily patched and 4.9.0-11-amd64). > > > I have some problem on Debian with 4.9.0-11-amd64 fails on both tmpfs and ext4, > > > others work fine (testing tmpfs, btrfs and ext4). But maybe I did something > > > wrong during testing. I did: > > > cp ./utils/mountd/mountd /usr/sbin/rpc.mountd > > > systemctl restart nfs-mountd.service > > That is the correct procedure. It should work... > Thanks for a confirmation. > > > Failure is regardless I use new mount.nfs (master) or the original from > > > Debian (1.3.3). > > What error message do you get on failure? It might help to add "-v" to > > the mount command to see more messages. > +1. I'll add it to the tests (+ printing mount.nfs version). > Here is debug info (when using nfs-utils 2.5.3 - master, with your patch): > * NFSv3 on ext4 (OK) > nfs01 1 TINFO: setup NFSv3, socket type tcp > nfs01 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=3 10.0.0.2:/var/tmp/LTP_nfs01.14F4lC51P6/3/tcp /var/tmp/LTP_nfs01.14F4lC51P6/3/0 > mount.nfs: trying 10.0.0.2 prog 100003 vers 3 prot TCP port 2049 > mount.nfs: trying 10.0.0.2 prog 100005 vers 3 prot TCP port 58997 > mount.nfs: timeout set for Thu May 13 08:22:46 2021 > mount.nfs: trying text-based options 'proto=tcp,vers=3,addr=10.0.0.2' > mount.nfs: prog 100003, trying vers=3, prot=6 > mount.nfs: prog 100005, trying vers=3, prot=6 > nfs01 1 TINFO: starting 'nfs01_open_files 10' I'm sorry, this one is when running the original rpc.nfsd. When running your patched, it fail (as I previously reported). > * NFSv3 on tmpfs (OK) > nfs01 1 TINFO: setup NFSv3, socket type tcp > nfs01 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=3 10.0.0.2:/tmp/LTP_nfs01.oJEz2xd9ZI/3/tcp /tmp/LTP_nfs01.oJEz2xd9ZI/3/0 > mount.nfs: trying 10.0.0.2 prog 100003 vers 3 prot TCP port 2049 > mount.nfs: trying 10.0.0.2 prog 100005 vers 3 prot TCP port 58997 > mount.nfs: timeout set for Thu May 13 08:23:01 2021 > mount.nfs: trying text-based options 'proto=tcp,vers=3,addr=10.0.0.2' > mount.nfs: prog 100003, trying vers=3, prot=6 > mount.nfs: prog 100005, trying vers=3, prot=6 > nfs01 1 TINFO: starting 'nfs01_open_files 10' The same here. > * NFSv4.1 on tmpfs (FAIL) > nfs01 1 TINFO: setup NFSv4.1, socket type tcp > nfs01 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=4.1 10.0.0.2:/tmp/LTP_nfs01.QnnFGEV4rs/4.1/tcp /tmp/LTP_nfs01.QnnFGEV4rs/4.1/0 > mount.nfs: mount(2): No such file or directory > mount.nfs: mounting 10.0.0.2:/tmp/LTP_nfs01.QnnFGEV4rs/4.1/tcp failed, reason given by server: No such file or directory > mount.nfs: timeout set for Thu May 13 08:23:02 2021 > mount.nfs: trying text-based options 'proto=tcp,vers=4.1,addr=10.0.0.2,clientaddr=10.0.0.1' > nfs01 1 TBROK: mount command failed > nfs01 1 TINFO: Cleaning up testcase The failure has really something to do with rpcbind ("mount.nfs: portmap query failed:"): rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 write(2, "mount.nfs: trying 10.0.0.2 prog "..., 66) = 66 socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 5 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 connect(5, {sa_family=AF_INET, sin_port=htons(37873), sin_addr=inet_addr("10.0.0.2")}, 16) = -1 EINPROGRESS (Operation now in progress) select(6, NULL, [5], NULL, {tv_sec=10, tv_usec=0}) = 1 (out [5], left {tv_sec=9, tv_usec=999998}) getsockopt(5, SOL_SOCKET, SO_ERROR, [111], [4]) = 0 fcntl(5, F_SETFL, O_RDWR) = 0 close(5) = 0 write(2, "mount.nfs: portmap query failed:"..., 79) = 79 Kind regards, Petr > > > strace looks nearly the same on tmpfs and ext4: > > This shows mount.nfs connecting to rpcbind, sending a request, getting a > > reply, and maybe looping around and trying again? > > There doesn't seem to be anything kernel related that would affect > > anything there so I cannot think why on older kernel would make a > > difference. Or an older rpcbind... > Well, it uses 1.2.5-0.3+deb10u1, i.e. nearly the latest version (Steve released > 1.2.6 3 days ago). And it's the same version as in openSUSE Tumbleweed where it > works well. > > Maybe I'll experiment on a SLE12 kernel. > Thanks! > > NeilBrown > Kind regards, > Petr
On Fri, 14 May 2021, Petr Vorel wrote: > > The failure has really something to do with rpcbind ("mount.nfs: portmap query > failed:"): > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > write(2, "mount.nfs: trying 10.0.0.2 prog "..., 66) = 66 > socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 5 > fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) > fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > connect(5, {sa_family=AF_INET, sin_port=htons(37873), sin_addr=inet_addr("10.0.0.2")}, 16) = -1 EINPROGRESS (Operation now in progress) > select(6, NULL, [5], NULL, {tv_sec=10, tv_usec=0}) = 1 (out [5], left {tv_sec=9, tv_usec=999998}) > getsockopt(5, SOL_SOCKET, SO_ERROR, [111], [4]) = 0 > fcntl(5, F_SETFL, O_RDWR) = 0 > close(5) = 0 > write(2, "mount.nfs: portmap query failed:"..., 79) = 79 The "111" from getsockopt...SO_ERROR is ECONNREFUSED. That suggests that rpcbind wasn't even running. This is different to the first strace you reported where mount.nfs successfully connected to rpcbind, sent and request and got a response, and then fail the mount. That would happen if, for example, rpc.mountd wasn't running. So I think these failures are caused by some problem with restarting the services and aren't actually testing the code at all. Could you try again and make sure rpcbind and rpc.mountd are running on the server before attempting the mount? Thanks, NeilBrown
Hi Neil, > On Fri, 14 May 2021, Petr Vorel wrote: > > The failure has really something to do with rpcbind ("mount.nfs: portmap query > > failed:"): > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > write(2, "mount.nfs: trying 10.0.0.2 prog "..., 66) = 66 > > socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 5 > > fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) > > fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 > > connect(5, {sa_family=AF_INET, sin_port=htons(37873), sin_addr=inet_addr("10.0.0.2")}, 16) = -1 EINPROGRESS (Operation now in progress) > > select(6, NULL, [5], NULL, {tv_sec=10, tv_usec=0}) = 1 (out [5], left {tv_sec=9, tv_usec=999998}) > > getsockopt(5, SOL_SOCKET, SO_ERROR, [111], [4]) = 0 > > fcntl(5, F_SETFL, O_RDWR) = 0 > > close(5) = 0 > > write(2, "mount.nfs: portmap query failed:"..., 79) = 79 > The "111" from getsockopt...SO_ERROR is ECONNREFUSED. That suggests > that rpcbind wasn't even running. > This is different to the first strace you reported where mount.nfs > successfully connected to rpcbind, sent and request and got a response, > and then fail the mount. That would happen if, for example, rpc.mountd > wasn't running. > So I think these failures are caused by some problem with restarting the > services and aren't actually testing the code at all. > Could you try again and make sure rpcbind and rpc.mountd are running on > the server before attempting the mount? I'm sorry, you're right, indeed rpc.mountd was not running and previously probably by rpcbind not running. Not sure what exactly was wrong I'll test your v2 probably only on openSUSE. BTW apart from checking whether rpcbind is running I'd check only rpc.nfsd. Or should be anything else tested? IMHO rpc.mountd, rpc.idmapd and rpc.statd are nfs-server.service dependencies (the service, which starts also rpc.nfsd). Kind regards, Petr > Thanks, > NeilBrown
diff --git a/support/export/v4root.c b/support/export/v4root.c index 3654bd7c10c0..fd36eb704441 100644 --- a/support/export/v4root.c +++ b/support/export/v4root.c @@ -11,6 +11,7 @@ #include <config.h> #endif +#include <fcntl.h> #include <sys/types.h> #include <sys/stat.h> #include <sys/queue.h> @@ -21,6 +22,7 @@ #include <unistd.h> #include <errno.h> +#include "nfsd_path.h" #include "xlog.h" #include "exportfs.h" #include "nfslib.h" @@ -73,6 +75,38 @@ set_pseudofs_security(struct exportent *pseudo) } } +static ssize_t exportfs_write(int fd, const char *buf, size_t len) +{ + return nfsd_path_write(fd, buf, len); +} + +static int test_export(struct exportent *eep, int with_fsid) +{ + char *path = eep->e_path; + int flags = eep->e_flags | (with_fsid ? NFSEXP_FSID : 0); + /* beside max path, buf size should take protocol str into account */ + char buf[NFS_MAXPATHLEN+1+64] = { 0 }; + char *bp = buf; + int len = sizeof(buf); + int fd, n; + + n = snprintf(buf, len, "-test-client- "); + bp += n; + len -= n; + qword_add(&bp, &len, path); + if (len < 1) + return 0; + snprintf(bp, len, " 3 %d 65534 65534 0\n", flags); + fd = open("/proc/net/rpc/nfsd.export/channel", O_WRONLY); + if (fd < 0) + return 0; + n = exportfs_write(fd, buf, strlen(buf)); + close(fd); + if (n < 0) + return 0; + return 1; +} + /* * Create a pseudo export */ @@ -82,6 +116,7 @@ v4root_create(char *path, nfs_export *export) nfs_export *exp; struct exportent eep; struct exportent *curexp = &export->m_export; + char uuid[33]; dupexportent(&eep, &pseudo_root.m_export); eep.e_ttl = default_ttl; @@ -89,6 +124,28 @@ v4root_create(char *path, nfs_export *export) strncpy(eep.e_path, path, sizeof(eep.e_path)-1); if (strcmp(path, "/") != 0) eep.e_flags &= ~NFSEXP_FSID; + if (strcmp(path, "/") != 0 && + !test_export(&eep, 0)) { + /* Need a uuid - base it on path */ + char buf[12], *pp = path; + unsigned int i = 0; + + memset(buf, 0, sizeof(buf)); + while (*pp) { + buf[i] ^= *pp++; + i += 1; + if (i >= sizeof(buf)) + i = 0; + } + memset(uuid, 'F', 32); + uuid[32] = '\0'; + pp = uuid + 32 - sizeof(buf) * 2; + for (i = 0; i < sizeof(buf); i++) { + snprintf(pp, 3, "%02X", buf[i]); + pp += 2; + } + eep.e_uuid = uuid; + } set_pseudofs_security(&eep); exp = export_create(&eep, 0); if (exp == NULL)
[[This is a proposed fix. It seems to work. I'd like some review comments before it is committed. Petr: it would be great if you could test it to confirm it actually works in your case. ]] Some filesystems cannot be exported without an fsid or uuid. tmpfs is the main example. When mountd creates nfsv4 pseudo-root exports for the path leading down to an export point it exports each directory without any fsid or uuid. If one of these directories is on tmp, that will fail. The net result is that exporting a subdirectory of a tmpfs filesystem will not work over NFSv4 as the parents within the filesystem cannot be exported. It will either fail, or fall-back to NFSv3 (depending on the version of the mount.nfs program). To fix this we need to provide an fsid or uuid for these pseudo-root exports. This patch does that by creating a UUID with the first 4 bytes 0xFFFFFFFF and the remaining 12 bytes form from the path name, xoring bytes together if the path is longer than 12 characters. Hopefully no filesystem uses a UUID like this.... The patch borrows some code from exportfs. Maybe that code should be move to a library.. Signed-off-by: NeilBrown <neilb@suse.de> --- support/export/v4root.c | 57 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+)