| Message ID | 496666FE.80206@cs.indiana.edu (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Jeff Layton wrote: > Rob, > > Your problem sounds like a different problem from what Leonardo > described, and is probably just poor error handling in the cifs_mount > code. We've done a lot of cleanup to that code recently, so results > with more recent kernels might be better. If you have a place to test > them, please test the kernels on my people.redhat.com page and report > the results: > > http://people.redhat.com/jlayton/ > > ...they have updated CIFS code. Both the RHEL4 and RHEL5 kernels should > have the cifs_mount overhaul. > > Thanks, Jeff. I just grabbed and installed 2.6.18-128.el5.jtltest.57PAE and didn't have any luck. The cifs mounts fail completely (even with just one user) and tcpdump shows they aren't generating any traffic to the samba server. I ran the mount via strace and it says it is panicing. Here are the results: # strace -f -v mount -t cifs //curie.cs.indiana.edu/robh /u/robh ... lots of output deleted ... [pid 6310] write(3, "Password: ", 10Password: ) = 10 [pid 6310] read(3, "password removed\n", 4096) = 17 [pid 6310] write(3, "\n", 1 ) = 1 [pid 6310] ioctl(3, SNDCTL_TMR_CONTINUE or TCSETSF, {c_iflags=0x500, c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) = 0 [pid 6310] close(3) = 0 [pid 6310] munmap(0xb7f7f000, 4096) = 0 [pid 6310] mount("//curie.cs.indiana.edu/robh", "/u/robh", "cifs"..., MS_MANDLOCK, "unc=//curie.cs.indiana.edu\\robh,"...PANIC: attached pid 6310 exited <unfinished ... exit status 0> Process 6309 resumed <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 6310 --- SIGCHLD (Child exited) @ 0 (0) --- exit_group(0) = ? I don't see anything being syslogged when I do this. Please let me know what else I can I do to help debug this? Thanks! --Rob
On Thu, 08 Jan 2009 17:04:35 -0500 Rob Henderson <robh@cs.indiana.edu> wrote: > > > Jeff Layton wrote: > > Rob, > > > > Your problem sounds like a different problem from what Leonardo > > described, and is probably just poor error handling in the cifs_mount > > code. We've done a lot of cleanup to that code recently, so results > > with more recent kernels might be better. If you have a place to test > > them, please test the kernels on my people.redhat.com page and report > > the results: > > > > http://people.redhat.com/jlayton/ > > > > ...they have updated CIFS code. Both the RHEL4 and RHEL5 kernels should > > have the cifs_mount overhaul. > > > > > > Thanks, Jeff. I just grabbed and installed 2.6.18-128.el5.jtltest.57PAE > and didn't have any luck. The cifs mounts fail completely (even with > just one user) and tcpdump shows they aren't generating any traffic to > the samba server. I ran the mount via strace and it says it is > panicing. Here are the results: > > # strace -f -v mount -t cifs //curie.cs.indiana.edu/robh /u/robh > ... lots of output deleted ... > [pid 6310] write(3, "Password: ", 10Password: ) = 10 > [pid 6310] read(3, "password removed\n", 4096) = 17 > [pid 6310] write(3, "\n", 1 > ) = 1 > [pid 6310] ioctl(3, SNDCTL_TMR_CONTINUE or TCSETSF, {c_iflags=0x500, > c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, > c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) > = 0 > [pid 6310] close(3) = 0 > [pid 6310] munmap(0xb7f7f000, 4096) = 0 > [pid 6310] mount("//curie.cs.indiana.edu/robh", "/u/robh", "cifs"..., > MS_MANDLOCK, "unc=//curie.cs.indiana.edu\\robh,"...PANIC: attached pid > 6310 exited > <unfinished ... exit status 0> > Process 6309 resumed > <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 6310 > --- SIGCHLD (Child exited) @ 0 (0) --- > exit_group(0) = ? > > > I don't see anything being syslogged when I do this. Please let me know > what else I can I do to help debug this? > Huh. Looks like the userspace helper errored out (though maybe the problem is something else). Did the mount attempt generate any error messages in dmesg? What version of samba-client package do you have installed?
Jeff Layton wrote: > On Thu, 08 Jan 2009 17:04:35 -0500 > Rob Henderson <robh@cs.indiana.edu> wrote: > > >> Jeff Layton wrote: >> >>> Rob, >>> >>> Your problem sounds like a different problem from what Leonardo >>> described, and is probably just poor error handling in the cifs_mount >>> code. We've done a lot of cleanup to that code recently, so results >>> with more recent kernels might be better. If you have a place to test >>> them, please test the kernels on my people.redhat.com page and report >>> the results: >>> >>> http://people.redhat.com/jlayton/ >>> >>> ...they have updated CIFS code. Both the RHEL4 and RHEL5 kernels should >>> have the cifs_mount overhaul. >>> >>> >>> >> Thanks, Jeff. I just grabbed and installed 2.6.18-128.el5.jtltest.57PAE >> and didn't have any luck. The cifs mounts fail completely (even with >> just one user) and tcpdump shows they aren't generating any traffic to >> the samba server. I ran the mount via strace and it says it is >> panicing. Here are the results: >> >> # strace -f -v mount -t cifs //curie.cs.indiana.edu/robh /u/robh >> ... lots of output deleted ... >> [pid 6310] write(3, "Password: ", 10Password: ) = 10 >> [pid 6310] read(3, "password removed\n", 4096) = 17 >> [pid 6310] write(3, "\n", 1 >> ) = 1 >> [pid 6310] ioctl(3, SNDCTL_TMR_CONTINUE or TCSETSF, {c_iflags=0x500, >> c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, >> c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) >> = 0 >> [pid 6310] close(3) = 0 >> [pid 6310] munmap(0xb7f7f000, 4096) = 0 >> [pid 6310] mount("//curie.cs.indiana.edu/robh", "/u/robh", "cifs"..., >> MS_MANDLOCK, "unc=//curie.cs.indiana.edu\\robh,"...PANIC: attached pid >> 6310 exited >> <unfinished ... exit status 0> >> Process 6309 resumed >> <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 6310 >> --- SIGCHLD (Child exited) @ 0 (0) --- >> exit_group(0) = ? >> >> >> I don't see anything being syslogged when I do this. Please let me know >> what else I can I do to help debug this? >> >> > > Huh. Looks like the userspace helper errored out (though maybe the > problem is something else). Did the mount attempt generate any error > messages in dmesg? What version of samba-client package do you have > installed? > > No, there isn't anything showing up in dmesg. Here is the samba version info: [root@test ~]# rpm -qa samba\* samba-client-3.0.28-1.el5_2.2 samba-3.0.28-1.el5_2.2 samba-common-3.0.28-1.el5_2.2 samba-swat-3.0.28-1.el5_2.2 Please let me know if there is a newer samba version I should be trying. Thanks, --Rob
On Thu, 08 Jan 2009 22:18:44 -0500 Rob Henderson <robh@cs.indiana.edu> wrote: > > > Jeff Layton wrote: > > On Thu, 08 Jan 2009 17:04:35 -0500 > > Rob Henderson <robh@cs.indiana.edu> wrote: > > > > > >> Jeff Layton wrote: > >> > >>> Rob, > >>> > >>> Your problem sounds like a different problem from what Leonardo > >>> described, and is probably just poor error handling in the cifs_mount > >>> code. We've done a lot of cleanup to that code recently, so results > >>> with more recent kernels might be better. If you have a place to test > >>> them, please test the kernels on my people.redhat.com page and report > >>> the results: > >>> > >>> http://people.redhat.com/jlayton/ > >>> > >>> ...they have updated CIFS code. Both the RHEL4 and RHEL5 kernels should > >>> have the cifs_mount overhaul. > >>> > >>> > >>> > >> Thanks, Jeff. I just grabbed and installed 2.6.18-128.el5.jtltest.57PAE > >> and didn't have any luck. The cifs mounts fail completely (even with > >> just one user) and tcpdump shows they aren't generating any traffic to > >> the samba server. I ran the mount via strace and it says it is > >> panicing. Here are the results: > >> > >> # strace -f -v mount -t cifs //curie.cs.indiana.edu/robh /u/robh > >> ... lots of output deleted ... > >> [pid 6310] write(3, "Password: ", 10Password: ) = 10 > >> [pid 6310] read(3, "password removed\n", 4096) = 17 > >> [pid 6310] write(3, "\n", 1 > >> ) = 1 > >> [pid 6310] ioctl(3, SNDCTL_TMR_CONTINUE or TCSETSF, {c_iflags=0x500, > >> c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, > >> c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) > >> = 0 > >> [pid 6310] close(3) = 0 > >> [pid 6310] munmap(0xb7f7f000, 4096) = 0 > >> [pid 6310] mount("//curie.cs.indiana.edu/robh", "/u/robh", "cifs"..., > >> MS_MANDLOCK, "unc=//curie.cs.indiana.edu\\robh,"...PANIC: attached pid > >> 6310 exited > >> <unfinished ... exit status 0> > >> Process 6309 resumed > >> <... wait4 resumed> [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 6310 > >> --- SIGCHLD (Child exited) @ 0 (0) --- > >> exit_group(0) = ? > >> > >> > >> I don't see anything being syslogged when I do this. Please let me know > >> what else I can I do to help debug this? > >> > >> > > > > Huh. Looks like the userspace helper errored out (though maybe the > > problem is something else). Did the mount attempt generate any error > > messages in dmesg? What version of samba-client package do you have > > installed? > > > > > > No, there isn't anything showing up in dmesg. Here is the samba version > info: > > [root@test ~]# rpm -qa samba\* > samba-client-3.0.28-1.el5_2.2 > samba-3.0.28-1.el5_2.2 > samba-common-3.0.28-1.el5_2.2 > samba-swat-3.0.28-1.el5_2.2 > > Please let me know if there is a newer samba version I should be trying. No, that should be OK I think... At this point, I'm going to ask you to go ahead and file a RHBZ: http://bugzilla.redhat.com/ We're likely to need to go back and forth on this a few times and collect some info and that's difficult to track over email. Once you get a bug filed, go ahead and CC me on it or email me with the bug number and I'll plan to grab it. Thanks,
Jeff Layton wrote: > At this point, I'm going to ask you to go ahead and file a RHBZ: > > http://bugzilla.redhat.com/ > > We're likely to need to go back and forth on this a few times and > collect some info and that's difficult to track over email. Once you > get a bug filed, go ahead and CC me on it or email me with the bug > number and I'll plan to grab it. > > Thanks, > Ok, done: https://bugzilla.redhat.com/show_bug.cgi?id=479418 Thanks for your help! --Rob
> [root@wink ~]# mount -t cifs //curie.cs.indiana.edu/robh /mnt/robh -o > [root@wink ~]# mount -t cifs //curie.cs.indiana.edu/somebogususer > /mnt/somebogususer -o user=somebogususer Hi Rob, Although similar, I don't think it is the same issue. The problem I'm seeing here occurs specifically when you try to mount the same share twice. Also, I couldn't duplicate the issue with Samba servers, only with Windows 2003 or greater. Thanks! Leonardo
> Ok, done: > > https://bugzilla.redhat.com/show_bug.cgi?id=479418 > > Thanks for your help! Jeff, I understand it is a different issue, but I tested kernel-2.6.18-128.el5.jtltest.59 (on Centos 5.3, as I don't have a RHEL subscription) to see if some of your patches would help, but I could still reproduce the problem. [root@localhost smb]# uname -a Linux localhost.localdomain 2.6.18-128.el5.jtltest.59 #1 SMP Fri Jan 9 15:42:22 EST 2009 x86_64 x86_64 x86_64 GNU/Linux [root@localhost smb]# mount -t cifs //192.168.8.142/Shared /smb/a -o noacl,noperm,user=aaa,dir_mode=0700,domain=DOMAIN,rw Password: [root@localhost smb]# mount -t cifs //192.168.8.142/Shared /smb/b -o noacl,noperm,user=bbb,dir_mode=0700,domain=DOMAIN,rw Password: [root@localhost smb]# ls -la total 16 drwxr-xr-x 4 root root 4096 Jan 6 12:14 . drwxr-xr-x 25 root root 4096 Jan 12 08:39 .. ?--------- ? ? ? ? ? a drwx------ 1 root root 0 Jan 5 20:07 b [root@localhost smb]# Thanks for your help, Leonardo
difference in that my first mount gets I/O errors rather than permission
denied errors after the second mount steps on it. See below for a
complete transcript demonstrating the problem.
I've tested this using a diverse set of servers and clients. I've tried
most combinations of the following mix of clients and servers and I have
yet to find any that don't fail. It doesn't seem to matter whether I
have /proc/fs/cifs/MultiuserMount set to 0 or 1.
Servers:
Red Hat EL4 running samba-3.0.28-0.el4.9 using ADS authentication
(security=ADS)
Red Hat EL5 running samba-3.0.28-1.el5_2.1 using ADS authentication
(security=ADS)
Red Hat EL5 running samba-3.0.28-1.el5_2.1 using smbpasswd
authentication (security=user)
Windows 2003 Server (R2)
Clients (with the CIFS version as reported by /proc/fs/cifs/DebugData):
Red Hat EL4 with kernel 2.6.9-78.0.8, samba version
samba-3.0.28-0.el4.9, and CIFS version 1.50cRH
Red Hat EL5 with kernel 2.6.18-92.1.22, samba version
samba-3.0.28-1.el5_2.1, and CIFS version 1.50cRH
Red Hat EL5 with kernel 2.6.18-92.1.10, samba version
samba-3.0.28-1.el5_2.1, and CIFS version 1.50cRH
Ubuntu 8.04.1 with kernel 2.6.24-22-rt, samba version
3.0.28a-1ubuntu4.7, and CIFS version 1.52
We are trying to deploy a lab of RHEL workstations that get homedirs via
samba+cifs from a RHEL file server with pam_mount managing the mounts at
login. But, since we need to support multiple remote logins this
problem has us dead in the water.
I tried the equivalent patch to fs/cifs/connect.c that Leonardo
suggested and so far everything seems to work! I really don't know what
the downside of this may be but but it _does_ seem to be working. The
RHEL5 ver is a little different than the code Leonardo was dealing with
and here is the patch I applied:
--- fs/cifs/connect.c.orig 2009-01-08 15:05:35.000000000 -0500
+++ fs/cifs/connect.c 2009-01-08 15:11:42.000000000 -0500
@@ -1974,7 +1974,7 @@
return -EINVAL;
}
- if (srvTcp) {
+ if (0) {
cFYI(1, ("Existing tcp session with server found"));
} else { /* create socket */
if (volume_info.port)
I wanted to add a 'me too' but also provide some more details about what I am seeing. Since my setup involves RHEL samba servers and RHEL cifs clients it may provide a good environment for debugging. My problem looks to be the same as Leonardo with just one slight Please let me know if there is any further debugging information I can provide. I would also be interested to know the downside of using this patch since it seems to work, albeit non-optimally. Thanks! --Rob ====================== transcript demonstrating failure =========================== [root@wink ~]# cat /proc/fs/cifs/MultiuserMount 1 [root@wink ~]# mount -t cifs //curie.cs.indiana.edu/robh /mnt/robh -o user=robh Password: [root@wink ~]# df /mnt/robh Filesystem 1K-blocks Used Available Use% Mounted on //curie.cs.indiana.edu/robh 1692109088 537737952 1068416960 34% /mnt/robh [root@wink ~]# cat /proc/fs/cifs/DebugData Display Internal CIFS Data Structures for Debugging --------------------------------------------------- CIFS Version 1.50cRH Active VFS Requests: 0 Servers: 1) Name: 129.79.246.140 Domain: WORKGROUP Mounts: 1 OS: Unix NOS: Samba 3.0.28-1.el5_2.1 Capability: 0x80f3fd SMB session status: 1 TCP status: 1 Local Users To Server: 1 SecMode: 0x3 Req On Wire: 0 MIDs: Shares: 1) \\curie.cs.indiana.edu\robh Uses: 1 Type: NTFS DevInfo: 0x0 Attributes: 0x2b PathComponentMax: 255 Status: 1 type: 0 [root@wink ~]# mount -t cifs //curie.cs.indiana.edu/somebogususer /mnt/somebogususer -o user=somebogususer Password: mount error 13 = Permission denied Refer to the mount.cifs(8) manual page (e.g.man mount.cifs) -> This was just a bogus username and I just hit enter at the password prompt so authentication failed [root@wink ~]# df /mnt/robh df: `/mnt/robh': Input/output error [root@wink ~]# ls /mnt/robh ls: /mnt/robh: Input/output error [root@wink ~]# cat /proc/fs/cifs/DebugData Display Internal CIFS Data Structures for Debugging --------------------------------------------------- CIFS Version 1.50cRH Active VFS Requests: 0 Servers: 1) Name: 129.79.246.140 Domain: WORKGROUP Mounts: 1 OS: Unix NOS: Samba 3.0.28-1.el5_2.1 Capability: 0x80f3fd SMB session status: 2 Shares: 1) \\curie.cs.indiana.edu\robh Uses: 1 Type: NTFS DevInfo: 0x0 Attributes: 0x2b PathComponentMax: 255 Status: 1 type: 0 NOTES: 1) It doesn't seem to matter what I do with the second mount command. In this case, I just used some nonexistant username but the behavior is the same if I use a real user, with or without successful authentication. 2) The diffs in the DebugData output show that the "SMB session status" changes from 1 to 2 and the "Local Users To Server" disappears as does the (empty) "MIDs:" section. Jeff Layton wrote: > On Tue, 6 Jan 2009 18:24:53 -0200 > "Leonardo Chiquitto" <leonardo.lists@gmail.com> wrote: > > >> On Fri, Dec 19, 2008 at 11:12 AM, Leonardo Chiquitto >> <leonardo.lists@gmail.com> wrote: >> >>> Hello, >>> >>> I'm doing some tests with CIFS client 1.54, trying to discover what >>> changed with Windows 2003 server that is resulting in a different >>> behavior from previous Windows versions. In the end, the goal is >>> to understand whether this is something we can workaround / avoid >>> on the client side. >>> >>> The test: >>> - the same share is mounted twice with different user names >>> - the server is not a domain controller or member server >>> - the server doesn't "know" the users >>> - "Everyone" and "Guest" have full permissions in the share >>> >>> The following commands were used: >>> >>> # mount -t cifs //192.168.8.1/Shared /mnt/a -o >>> noacl,noperm,user=john,dir_mode=0700,domain=DOMAIN,rw >>> # mount -t cifs //192.168.8.1/Shared /mnt/b -o >>> noacl,noperm,user=jeff,dir_mode=0700,domain=DOMAIN,rw >>> >>> Results with Windows 2000 Server: >>> >>> - Both mount points are accessible in the client: >>> >>> client:/mnt # ls -la >>> total 8 >>> drwxr-xr-x 4 root root 4096 Dec 9 19:14 . >>> drwxr-xr-x 31 root root 4096 Dec 15 11:40 .. >>> drwx------ 1 root root 0 Dec 15 2008 a >>> drwx------ 1 root root 0 Dec 15 2008 b >>> >>> - "net session" output from server: >>> >>> Computer User name Client Type Opens Idle time >>> ------------------------------------------------------------------------------- >>> \\192.168.8.1 JOHN Linux version 2.6 0 00:01:40 >>> \\192.168.8.1 JEFF Linux version 2.6 0 00:01:40 >>> >>> * We can see that Windows 2000 Server keeps the user name provided >>> by the client. >>> >>> Results with Windows 2003 Server: >>> >>> - Just the mount point mounted last is accessible: >>> >>> client:/mnt # ls -la >>> ls: cannot access a: Permission denied >>> total 8 >>> drwxr-xr-x 4 root root 4096 Dec 9 19:14 . >>> drwxr-xr-x 31 root root 4096 Dec 15 11:40 .. >>> d????????? ? ? ? ? ? a >>> drwx------ 1 root root 0 Dec 15 2008 b >>> >>> - "net session" output from the server: >>> >>> Computer User name Client Type Opens Idle time >>> ------------------------------------------------------------------------------- >>> 192.168.8.1 GUEST Linux version 2.6 0 000012 >>> >>> * We can see that just one session remains and the user name was mapped to >>> GUEST. >>> >>> I suppose this is expected, but I really would like to understand what is >>> the underlying cause. >>> >> I couldn't find a solution for this problem yet, but I have more information >> to share: >> >> - The problem also happens with Windows 2008 Server and earlier versions >> of the CIFS client (tested with 1.20). >> >> Playing with the code, I found that disabling TCP connection sharing by >> applying the following patch, will workaround the problem: I can now >> mount the same share twice with different users and still have access to >> both. >> >> --- linux.orig/fs/cifs/connect.c >> +++ linux/fs/cifs/connect.c >> @@ -1463,9 +1463,11 @@ cifs_get_tcp_session(struct smb_vol *vol >> } >> >> /* see if we already have a matching tcp_ses */ >> +#if 0 >> tcp_ses = cifs_find_tcp_session(&addr); >> if (tcp_ses) >> return tcp_ses; >> +#endif >> >> tcp_ses = kzalloc(sizeof(struct TCP_Server_Info), GFP_KERNEL); >> if (!tcp_ses) { >> >> I understand that this is not a valid fix, it was made only for debugging. >> >> It's also interesting that even when not sharing the TCP connection, >> "net session" output from Windows 2003 still shows only one session >> opened. Don't laugh, but is there any documentation that describes what >> defines an unique SMB session for different versions of Windows? >> >> > > Not that I know of... > > The Linux CIFS code has 3 layered "pieces" when setting up a mount: > > TCP Session (basically the socket) > SMB Session (generally separated by username) > Tree Connection (sort of the actual "mount") > > These are all "many to one". i.e. a TCP session can have several SMB > sessions, and a SMB session can have several tcons. > > It sounds almost like the server is squishing the SMB sessions together > and getting them confused for some reason. You might want to look at > what's going on on the wire with the SMB sessions. > > I also wonder whether the server may be presenting share-level security > mounts, but that would be very odd for a server as new as win2k3. > >