| Message ID | 1383839253-8223-1-git-send-email-jlayton@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Nov 07, 2013 at 10:47:33AM -0500, Jeff Layton wrote: > Because the kernel now looks for krb5 creds by default when mounting, we > need to always have clients run rpc.gssd to avoid the 15s hang on the > first mount attempt. Not sure if it's the same hangs, but I see about the same length of a hang on my Debian wheezy systems when accessing the first file for a data operation, with rpc.gssd running or not. Funnily enough this also happens when trying to read a non-existand file, but not for things like a simple readdir. Still looking into it, but being a non-interruptible sleep it's fairly annoying. That being said Debian wheezy also only starts rpc.gssd from the init scripts if it sees a krb option in /etc/fstab, which I don't have. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 7 Nov 2013 07:58:37 -0800 Christoph Hellwig <hch@infradead.org> wrote: > On Thu, Nov 07, 2013 at 10:47:33AM -0500, Jeff Layton wrote: > > Because the kernel now looks for krb5 creds by default when mounting, we > > need to always have clients run rpc.gssd to avoid the 15s hang on the > > first mount attempt. > > Not sure if it's the same hangs, but I see about the same length of a > hang on my Debian wheezy systems when accessing the first file for a > data operation, with rpc.gssd running or not. Funnily enough this also > happens when trying to read a non-existand file, but not for things like > a simple readdir. > > Still looking into it, but being a non-interruptible sleep it's fairly > annoying. > > That being said Debian wheezy also only starts rpc.gssd from the init > scripts if it sees a krb option in /etc/fstab, which I don't have. > It may be the same issue. IIRC, the attempt to use krb5i by default is done in the SETCLIENTID call (see kernel commit 4edaa30888 for some background, but there were some later tweaks to this approach).
On Thu, Nov 07, 2013 at 11:09:28AM -0500, Jeff Layton wrote: > It may be the same issue. IIRC, the attempt to use krb5i by default > is done in the SETCLIENTID call (see kernel commit 4edaa30888 for some > background, but there were some later tweaks to this approach). I'm still seing my issue after reverting that commit by hand. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 7 Nov 2013 08:21:23 -0800 Christoph Hellwig <hch@infradead.org> wrote: > On Thu, Nov 07, 2013 at 11:09:28AM -0500, Jeff Layton wrote: > > It may be the same issue. IIRC, the attempt to use krb5i by default > > is done in the SETCLIENTID call (see kernel commit 4edaa30888 for some > > background, but there were some later tweaks to this approach). > > I'm still seing my issue after reverting that commit by hand. Hrm... You'd also see these messages in the log if it were the same problem: RPC: AUTH_GSS upcall timed out. ...it could also be a delay when trying to set up the callback channel if you're running v4.0. Do you have the callback port firewalled off?
On Thu, Nov 07, 2013 at 11:33:10AM -0500, Jeff Layton wrote: > > I'm still seing my issue after reverting that commit by hand. > > Hrm... > > You'd also see these messages in the log if it were the same problem: > > RPC: AUTH_GSS upcall timed out. > > ...it could also be a delay when trying to set up the callback channel > if you're running v4.0. Do you have the callback port firewalled off? Not seeing any of those, and running with 4.1 at the moment, although I could try 4.0. Talking to a server on the same host, so firewalling is unlikely. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 7 Nov 2013 08:35:20 -0800 Christoph Hellwig <hch@infradead.org> wrote: > On Thu, Nov 07, 2013 at 11:33:10AM -0500, Jeff Layton wrote: > > > I'm still seing my issue after reverting that commit by hand. > > > > Hrm... > > > > You'd also see these messages in the log if it were the same problem: > > > > RPC: AUTH_GSS upcall timed out. > > > > ...it could also be a delay when trying to set up the callback channel > > if you're running v4.0. Do you have the callback port firewalled off? > > Not seeing any of those, and running with 4.1 at the moment, although I > could try 4.0. Talking to a server on the same host, so firewalling is > unlikely. > No idea then, sorry...
On Thu, Nov 07, 2013 at 11:36:56AM -0500, Jeff Layton wrote:
> No idea then, sorry...
4.0 is also affected, interestingly it's only reproducible on the first
mount of a volume after the server started, so I'll assume a server
issue. I'm trying to drill it down time permitting, but so far I have
no clue what's going on. Doesn't help that recent versions of
latencytop are unusable..
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hello, My thoughts... On 07/11/13 10:47, Jeff Layton wrote: > Because the kernel now looks for krb5 creds by default when mounting, we > need to always have clients run rpc.gssd to avoid the 15s hang on the > first mount attempt. > > In situations however where people don't have krb5 set up, this leads to > a lot of log spamming on mount attempts. Try to cut down on some of the > log chatter by lowering the priority of the log messages in > gssd_search_krb5_keytab(). > > Signed-off-by: Jeff Layton <jlayton@redhat.com> > --- > utils/gssd/krb5_util.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c > index 697d1d2..2ca3adc 100644 > --- a/utils/gssd/krb5_util.c > +++ b/utils/gssd/krb5_util.c > @@ -711,13 +711,13 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, > */ > if ((code = krb5_kt_get_name(context, kt, kt_name, BUFSIZ))) { > k5err = gssd_k5_err_msg(context, code); > - printerr(0, "ERROR: %s attempting to get keytab name\n", k5err); > + printerr(1, "ERROR: %s attempting to get keytab name\n", k5err); > retval = code; > goto out; > } > if ((code = krb5_kt_start_seq_get(context, kt, &cursor))) { > k5err = gssd_k5_err_msg(context, code); > - printerr(0, "ERROR: %s while beginning keytab scan " > + printerr(1, "ERROR: %s while beginning keytab scan " > "for keytab '%s'\n", k5err, kt_name); > retval = code; > goto out; I'm thinking these need to stay... I don't think is a good idea to mask out messages describing failures... > @@ -727,7 +727,7 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, > if ((code = krb5_unparse_name(context, kte->principal, > &pname))) { > k5err = gssd_k5_err_msg(context, code); > - printerr(0, "WARNING: Skipping keytab entry because " > + printerr(1, "WARNING: Skipping keytab entry because " > "we failed to unparse principal name: %s\n", > k5err); > k5_free_kt_entry(context, kte); > @@ -768,7 +768,7 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, > > if ((code = krb5_kt_end_seq_get(context, kt, &cursor))) { > k5err = gssd_k5_err_msg(context, code); > - printerr(0, "WARNING: %s while ending keytab scan for " > + printerr(1, "WARNING: %s while ending keytab scan for " > "keytab '%s'\n", k5err, kt_name); > } I think warnings would be ok to mask out... steved. > > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 07 Nov 2013 11:47:55 -0500 Steve Dickson <SteveD@redhat.com> wrote: > Hello, > > My thoughts... > > On 07/11/13 10:47, Jeff Layton wrote: > > Because the kernel now looks for krb5 creds by default when mounting, we > > need to always have clients run rpc.gssd to avoid the 15s hang on the > > first mount attempt. > > > > In situations however where people don't have krb5 set up, this leads to > > a lot of log spamming on mount attempts. Try to cut down on some of the > > log chatter by lowering the priority of the log messages in > > gssd_search_krb5_keytab(). > > > > Signed-off-by: Jeff Layton <jlayton@redhat.com> > > --- > > utils/gssd/krb5_util.c | 8 ++++---- > > 1 file changed, 4 insertions(+), 4 deletions(-) > > > > diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c > > index 697d1d2..2ca3adc 100644 > > --- a/utils/gssd/krb5_util.c > > +++ b/utils/gssd/krb5_util.c > > @@ -711,13 +711,13 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, > > */ > > if ((code = krb5_kt_get_name(context, kt, kt_name, BUFSIZ))) { > > k5err = gssd_k5_err_msg(context, code); > > - printerr(0, "ERROR: %s attempting to get keytab name\n", k5err); > > + printerr(1, "ERROR: %s attempting to get keytab name\n", k5err); > > retval = code; > > goto out; > > } > > if ((code = krb5_kt_start_seq_get(context, kt, &cursor))) { > > k5err = gssd_k5_err_msg(context, code); > > - printerr(0, "ERROR: %s while beginning keytab scan " > > + printerr(1, "ERROR: %s while beginning keytab scan " > > "for keytab '%s'\n", k5err, kt_name); > > retval = code; > > goto out; > I'm thinking these need to stay... I don't think is a good idea > to mask out messages describing failures... > The above message is the one that's the problem when there is no keytab. I just went ahead and lowered the priority on the others for good measure. If you don't want to take the above hunk, then there's probably no need to bother with this patch at all. > > @@ -727,7 +727,7 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, > > if ((code = krb5_unparse_name(context, kte->principal, > > &pname))) { > > k5err = gssd_k5_err_msg(context, code); > > - printerr(0, "WARNING: Skipping keytab entry because " > > + printerr(1, "WARNING: Skipping keytab entry because " > > "we failed to unparse principal name: %s\n", > > k5err); > > k5_free_kt_entry(context, kte); > > @@ -768,7 +768,7 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, > > > > if ((code = krb5_kt_end_seq_get(context, kt, &cursor))) { > > k5err = gssd_k5_err_msg(context, code); > > - printerr(0, "WARNING: %s while ending keytab scan for " > > + printerr(1, "WARNING: %s while ending keytab scan for " > > "keytab '%s'\n", k5err, kt_name); > > } > I think warnings would be ok to mask out... > > steved. > > > > >
diff --git a/utils/gssd/krb5_util.c b/utils/gssd/krb5_util.c index 697d1d2..2ca3adc 100644 --- a/utils/gssd/krb5_util.c +++ b/utils/gssd/krb5_util.c @@ -711,13 +711,13 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, */ if ((code = krb5_kt_get_name(context, kt, kt_name, BUFSIZ))) { k5err = gssd_k5_err_msg(context, code); - printerr(0, "ERROR: %s attempting to get keytab name\n", k5err); + printerr(1, "ERROR: %s attempting to get keytab name\n", k5err); retval = code; goto out; } if ((code = krb5_kt_start_seq_get(context, kt, &cursor))) { k5err = gssd_k5_err_msg(context, code); - printerr(0, "ERROR: %s while beginning keytab scan " + printerr(1, "ERROR: %s while beginning keytab scan " "for keytab '%s'\n", k5err, kt_name); retval = code; goto out; @@ -727,7 +727,7 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, if ((code = krb5_unparse_name(context, kte->principal, &pname))) { k5err = gssd_k5_err_msg(context, code); - printerr(0, "WARNING: Skipping keytab entry because " + printerr(1, "WARNING: Skipping keytab entry because " "we failed to unparse principal name: %s\n", k5err); k5_free_kt_entry(context, kte); @@ -768,7 +768,7 @@ gssd_search_krb5_keytab(krb5_context context, krb5_keytab kt, if ((code = krb5_kt_end_seq_get(context, kt, &cursor))) { k5err = gssd_k5_err_msg(context, code); - printerr(0, "WARNING: %s while ending keytab scan for " + printerr(1, "WARNING: %s while ending keytab scan for " "keytab '%s'\n", k5err, kt_name); }
Because the kernel now looks for krb5 creds by default when mounting, we need to always have clients run rpc.gssd to avoid the 15s hang on the first mount attempt. In situations however where people don't have krb5 set up, this leads to a lot of log spamming on mount attempts. Try to cut down on some of the log chatter by lowering the priority of the log messages in gssd_search_krb5_keytab(). Signed-off-by: Jeff Layton <jlayton@redhat.com> --- utils/gssd/krb5_util.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)