Adding the nfs4_use_min_auth module parameter

Commit Message

Steve Dickson Nov. 7, 2013, 7:09 p.m. UTC

This new module parameter makes the v4 client
use the minimal authentication flavor (AUTH_UNIX)
when establishing NFSV4 state and doing the
pseudoroot lookup

Signed-off-by: Steve Dickson <[e-mail]>
---
 fs/nfs/nfs4_fs.h    |    1 +
 fs/nfs/nfs4client.c |    8 ++++++--
 fs/nfs/nfs4proc.c   |    4 +++-
 fs/nfs/super.c      |    6 +++++-
 4 files changed, 15 insertions(+), 4 deletions(-)

Comments

Chuck Lever III Nov. 7, 2013, 7:25 p.m. UTC | #1

Hi Steve-

On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:

> This new module parameter makes the v4 client
> use the minimal authentication flavor (AUTH_UNIX)
> when establishing NFSV4 state and doing the
> pseudoroot lookup

The patch description doesn't say, but is this change to work around the 15 second GSSD upcall timeout? Have we completely given up on fixing the upcall?


> Signed-off-by: Steve Dickson <[e-mail]>
> ---
> fs/nfs/nfs4_fs.h    |    1 +
> fs/nfs/nfs4client.c |    8 ++++++--
> fs/nfs/nfs4proc.c   |    4 +++-
> fs/nfs/super.c      |    6 +++++-
> 4 files changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
> index 28842ab..20bf925 100644
> --- a/fs/nfs/nfs4_fs.h
> +++ b/fs/nfs/nfs4_fs.h
> @@ -438,6 +438,7 @@ extern bool nfs4_disable_idmapping;
> extern unsigned short max_session_slots;
> extern unsigned short send_implementation_id;
> extern bool recover_lost_locks;
> +extern bool nfs4_use_min_auth;
> 
> #define NFS4_CLIENT_ID_UNIQ_LEN		(64)
> extern char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN];
> diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
> index a860ab5..ff85991 100644
> --- a/fs/nfs/nfs4client.c
> +++ b/fs/nfs/nfs4client.c
> @@ -355,6 +355,7 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
> 	char buf[INET6_ADDRSTRLEN + 1];
> 	struct nfs_client *old;
> 	int error;
> +	rpc_authflavor_t flavor = RPC_AUTH_GSS_KRB5I;
> 
> 	if (clp->cl_cons_state == NFS_CS_READY) {
> 		/* the client is initialised already */
> @@ -368,8 +369,11 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
> 	if (clp->cl_minorversion != 0)
> 		__set_bit(NFS_CS_INFINITE_SLOTS, &clp->cl_flags);
> 	__set_bit(NFS_CS_DISCRTRY, &clp->cl_flags);
> -	error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_GSS_KRB5I);
> -	if (error == -EINVAL)
> +
> +	if (nfs4_use_min_auth)
> +		flavor = RPC_AUTH_UNIX;
> +	error = nfs_create_rpc_client(clp, timeparms, flavor);
> +	if (error == -EINVAL && flavor != RPC_AUTH_UNIX)
> 		error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_UNIX);
> 	if (error < 0)
> 		goto error;
> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> index d53d678..00162cb 100644
> --- a/fs/nfs/nfs4proc.c
> +++ b/fs/nfs/nfs4proc.c
> @@ -2864,7 +2864,9 @@ static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
> 	int status = -EPERM;
> 	size_t i;
> 
> -	for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
> +	if (nfs4_use_min_auth)
> +		status = nfs4_lookup_root_sec(server, fhandle, info, RPC_AUTH_UNIX);
> +	else for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
> 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
> 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
> 			continue;
> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
> index a03b9c6..42b4f9b 100644
> --- a/fs/nfs/super.c
> +++ b/fs/nfs/super.c
> @@ -2791,6 +2791,7 @@ unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
> unsigned short send_implementation_id = 1;
> char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
> bool recover_lost_locks = false;
> +bool nfs4_use_min_auth = false;
> 
> EXPORT_SYMBOL_GPL(nfs_callback_set_tcpport);
> EXPORT_SYMBOL_GPL(nfs_callback_tcpport);
> @@ -2800,6 +2801,7 @@ EXPORT_SYMBOL_GPL(max_session_slots);
> EXPORT_SYMBOL_GPL(send_implementation_id);
> EXPORT_SYMBOL_GPL(nfs4_client_id_uniquifier);
> EXPORT_SYMBOL_GPL(recover_lost_locks);
> +EXPORT_SYMBOL_GPL(nfs4_use_min_auth);
> 
> #define NFS_CALLBACK_MAXPORTNR (65535U)
> 
> @@ -2842,5 +2844,7 @@ MODULE_PARM_DESC(recover_lost_locks,
> 		 "If the server reports that a lock might be lost, "
> 		 "try to recover it risking data corruption.");
> 
> -
> +module_param(nfs4_use_min_auth, bool, 0644);
> +MODULE_PARM_DESC(nfs4_use_min_auth,
> +		"Use mimnal auth in SETCLIENTID operation");
> #endif /* CONFIG_NFS_V4 */
> -- 
> 1.7.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jeff Layton Nov. 7, 2013, 9:01 p.m. UTC | #2

On Thu, 7 Nov 2013 11:25:19 -0800
Chuck Lever <[e-mail]> wrote:

> Hi Steve-
> 
> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> 
> > This new module parameter makes the v4 client
> > use the minimal authentication flavor (AUTH_UNIX)
> > when establishing NFSV4 state and doing the
> > pseudoroot lookup
> 
> The patch description doesn't say, but is this change to work around the 15 second GSSD upcall timeout? Have we completely given up on fixing the upcall?
> 

That would be my preferred solution too. The whole problem is that this
upcall takes too damned long to time out.

But...how can it be fixed?

Due to the way that rpc_pipefs interface works, there's no way that I
can see to make that fail immediately if no one is listening on the
pipe. You could reduce the timeout I guess but that's sort of a
half-assed solution.

IMO, we're long past due for a new upcall that uses a different design
altogether. Maybe something based on call_usermodehelper that doesn't
require a running daemon?

Steve Dickson Nov. 7, 2013, 9:35 p.m. UTC | #3

Hey mrchuck... 

On 07/11/13 14:25, Chuck Lever wrote:
> Hi Steve-
> 
> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> 
>> This new module parameter makes the v4 client
>> use the minimal authentication flavor (AUTH_UNIX)
>> when establishing NFSV4 state and doing the
>> pseudoroot lookup
> 
> The patch description doesn't say, but is this change to work 
> around the 15 second GSSD upcall timeout? 
Yes. A 15 second delay on every mount due to security that
nobody is requesting is just not good.. IMHO.. Also running
a security daemon for non-secure mounts just seems wrong to me.

> Have we completely given up on fixing the upcall?
Back when we were looking on to introduce gss-proxy into
the client, it was decided the upcall on the client
was not going to change. I'm just going with that!

steved.


> 
> 
>> Signed-off-by: Steve Dickson <[e-mail]>
>> ---
>> fs/nfs/nfs4_fs.h    |    1 +
>> fs/nfs/nfs4client.c |    8 ++++++--
>> fs/nfs/nfs4proc.c   |    4 +++-
>> fs/nfs/super.c      |    6 +++++-
>> 4 files changed, 15 insertions(+), 4 deletions(-)
>>
>> diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
>> index 28842ab..20bf925 100644
>> --- a/fs/nfs/nfs4_fs.h
>> +++ b/fs/nfs/nfs4_fs.h
>> @@ -438,6 +438,7 @@ extern bool nfs4_disable_idmapping;
>> extern unsigned short max_session_slots;
>> extern unsigned short send_implementation_id;
>> extern bool recover_lost_locks;
>> +extern bool nfs4_use_min_auth;
>>
>> #define NFS4_CLIENT_ID_UNIQ_LEN		(64)
>> extern char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN];
>> diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
>> index a860ab5..ff85991 100644
>> --- a/fs/nfs/nfs4client.c
>> +++ b/fs/nfs/nfs4client.c
>> @@ -355,6 +355,7 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
>> 	char buf[INET6_ADDRSTRLEN + 1];
>> 	struct nfs_client *old;
>> 	int error;
>> +	rpc_authflavor_t flavor = RPC_AUTH_GSS_KRB5I;
>>
>> 	if (clp->cl_cons_state == NFS_CS_READY) {
>> 		/* the client is initialised already */
>> @@ -368,8 +369,11 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
>> 	if (clp->cl_minorversion != 0)
>> 		__set_bit(NFS_CS_INFINITE_SLOTS, &clp->cl_flags);
>> 	__set_bit(NFS_CS_DISCRTRY, &clp->cl_flags);
>> -	error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_GSS_KRB5I);
>> -	if (error == -EINVAL)
>> +
>> +	if (nfs4_use_min_auth)
>> +		flavor = RPC_AUTH_UNIX;
>> +	error = nfs_create_rpc_client(clp, timeparms, flavor);
>> +	if (error == -EINVAL && flavor != RPC_AUTH_UNIX)
>> 		error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_UNIX);
>> 	if (error < 0)
>> 		goto error;
>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
>> index d53d678..00162cb 100644
>> --- a/fs/nfs/nfs4proc.c
>> +++ b/fs/nfs/nfs4proc.c
>> @@ -2864,7 +2864,9 @@ static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
>> 	int status = -EPERM;
>> 	size_t i;
>>
>> -	for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
>> +	if (nfs4_use_min_auth)
>> +		status = nfs4_lookup_root_sec(server, fhandle, info, RPC_AUTH_UNIX);
>> +	else for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
>> 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
>> 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
>> 			continue;
>> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
>> index a03b9c6..42b4f9b 100644
>> --- a/fs/nfs/super.c
>> +++ b/fs/nfs/super.c
>> @@ -2791,6 +2791,7 @@ unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
>> unsigned short send_implementation_id = 1;
>> char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
>> bool recover_lost_locks = false;
>> +bool nfs4_use_min_auth = false;
>>
>> EXPORT_SYMBOL_GPL(nfs_callback_set_tcpport);
>> EXPORT_SYMBOL_GPL(nfs_callback_tcpport);
>> @@ -2800,6 +2801,7 @@ EXPORT_SYMBOL_GPL(max_session_slots);
>> EXPORT_SYMBOL_GPL(send_implementation_id);
>> EXPORT_SYMBOL_GPL(nfs4_client_id_uniquifier);
>> EXPORT_SYMBOL_GPL(recover_lost_locks);
>> +EXPORT_SYMBOL_GPL(nfs4_use_min_auth);
>>
>> #define NFS_CALLBACK_MAXPORTNR (65535U)
>>
>> @@ -2842,5 +2844,7 @@ MODULE_PARM_DESC(recover_lost_locks,
>> 		 "If the server reports that a lock might be lost, "
>> 		 "try to recover it risking data corruption.");
>>
>> -
>> +module_param(nfs4_use_min_auth, bool, 0644);
>> +MODULE_PARM_DESC(nfs4_use_min_auth,
>> +		"Use mimnal auth in SETCLIENTID operation");
>> #endif /* CONFIG_NFS_V4 */
>> -- 
>> 1.7.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [e-mail]
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 7, 2013, 9:40 p.m. UTC | #4

On 07/11/13 16:01, Jeff Layton wrote:
> On Thu, 7 Nov 2013 11:25:19 -0800
> Chuck Lever <[e-mail]> wrote:
> 
>> Hi Steve-
>>
>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>
>>> This new module parameter makes the v4 client
>>> use the minimal authentication flavor (AUTH_UNIX)
>>> when establishing NFSV4 state and doing the
>>> pseudoroot lookup
>>
>> The patch description doesn't say, but is this change to work around the 15 second GSSD upcall timeout? Have we completely given up on fixing the upcall?
>>
> 
> That would be my preferred solution too. The whole problem is that this
> upcall takes too damned long to time out.
> 
> But...how can it be fixed?
> 
> Due to the way that rpc_pipefs interface works, there's no way that I
> can see to make that fail immediately if no one is listening on the
> pipe. You could reduce the timeout I guess but that's sort of a
> half-assed solution.
> 
> IMO, we're long past due for a new upcall that uses a different design
> altogether. Maybe something based on call_usermodehelper that doesn't
> require a running daemon?
Bingo! I think it would be huge for all these upcalls to work like the idmapping
does on the client (aka via call_usermodehelper)... but which the likes
of Kerberos it may not be possible... 

steved.

> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jeff Layton Nov. 7, 2013, 10:04 p.m. UTC | #5

On Thu, 07 Nov 2013 16:40:18 -0500
Steve Dickson <[e-mail]> wrote:

> 
> 
> On 07/11/13 16:01, Jeff Layton wrote:
> > On Thu, 7 Nov 2013 11:25:19 -0800
> > Chuck Lever <[e-mail]> wrote:
> > 
> >> Hi Steve-
> >>
> >> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> >>
> >>> This new module parameter makes the v4 client
> >>> use the minimal authentication flavor (AUTH_UNIX)
> >>> when establishing NFSV4 state and doing the
> >>> pseudoroot lookup
> >>
> >> The patch description doesn't say, but is this change to work around the 15 second GSSD upcall timeout? Have we completely given up on fixing the upcall?
> >>
> > 
> > That would be my preferred solution too. The whole problem is that this
> > upcall takes too damned long to time out.
> > 
> > But...how can it be fixed?
> > 
> > Due to the way that rpc_pipefs interface works, there's no way that I
> > can see to make that fail immediately if no one is listening on the
> > pipe. You could reduce the timeout I guess but that's sort of a
> > half-assed solution.
> > 
> > IMO, we're long past due for a new upcall that uses a different design
> > altogether. Maybe something based on call_usermodehelper that doesn't
> > require a running daemon?
> Bingo! I think it would be huge for all these upcalls to work like the idmapping
> does on the client (aka via call_usermodehelper)... but which the likes
> of Kerberos it may not be possible... 
> 

No, it's possible.

We don't currently handle multi-stage GSSAPI negotiation across the
kernel/userland boundary anyway, so there's no problem with state
preservation across upcalls or anything.

The tricky part is how to handle the downcall. call_usermodehelper
doesn't have a way to pass anything but a return code back to the
kernel, so you'd need to send the context handle and such via another
mechanism.

Hmm...and containers too. The UMH infrastructure is not container-aware,
whereas rpc_pipefs is. I guess you could make the program use setns(),
but you have to come up with some way to pass a fd to it.

Either way, not a trivial task. It'd be best to get everyone on
board with the design before trying to do the work.

Chuck Lever III Nov. 7, 2013, 11:05 p.m. UTC | #6

On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:

> Hey mrchuck... 
> 
> On 07/11/13 14:25, Chuck Lever wrote:
>> Hi Steve-
>> 
>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>> 
>>> This new module parameter makes the v4 client
>>> use the minimal authentication flavor (AUTH_UNIX)
>>> when establishing NFSV4 state and doing the
>>> pseudoroot lookup
>> 
>> The patch description doesn't say, but is this change to work 
>> around the 15 second GSSD upcall timeout? 
> Yes. A 15 second delay on every mount due to security that
> nobody is requesting is just not good.. IMHO..

One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when you expect the mount to work (ie why would it be good for any properly configured environment to take 15 seconds to establish a GSS context?).

In other words, there are still cases where users wait 15 seconds unnecessarily, and not because of the use of krb5i for lease management.  Aren't those of concern?

> Also running
> a security daemon for non-secure mounts just seems wrong to me.

It seems wrong to me to keep auth_rpcgss loaded if no mounts use Kerberos security.  What's the difference between that and running gssd all the time?

The new SETCLIENTID architecture would be transparent if our security infrastructure was reliable.  We really need to address the upcall issue.  We've been working around it for a very long time.

> 
>> Have we completely given up on fixing the upcall?
> Back when we were looking on to introduce gss-proxy into
> the client, it was decided the upcall on the client
> was not going to change. I'm just going with that!

I don't remember that discussion.  Or if it occurred, I didn't realize at the time that it was referring to fixing bugs.  Do you have a link to the thread?


> steved.
> 
> 
>> 
>> 
>>> Signed-off-by: Steve Dickson <[e-mail]>
>>> ---
>>> fs/nfs/nfs4_fs.h    |    1 +
>>> fs/nfs/nfs4client.c |    8 ++++++--
>>> fs/nfs/nfs4proc.c   |    4 +++-
>>> fs/nfs/super.c      |    6 +++++-
>>> 4 files changed, 15 insertions(+), 4 deletions(-)
>>> 
>>> diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
>>> index 28842ab..20bf925 100644
>>> --- a/fs/nfs/nfs4_fs.h
>>> +++ b/fs/nfs/nfs4_fs.h
>>> @@ -438,6 +438,7 @@ extern bool nfs4_disable_idmapping;
>>> extern unsigned short max_session_slots;
>>> extern unsigned short send_implementation_id;
>>> extern bool recover_lost_locks;
>>> +extern bool nfs4_use_min_auth;
>>> 
>>> #define NFS4_CLIENT_ID_UNIQ_LEN		(64)
>>> extern char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN];
>>> diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
>>> index a860ab5..ff85991 100644
>>> --- a/fs/nfs/nfs4client.c
>>> +++ b/fs/nfs/nfs4client.c
>>> @@ -355,6 +355,7 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
>>> 	char buf[INET6_ADDRSTRLEN + 1];
>>> 	struct nfs_client *old;
>>> 	int error;
>>> +	rpc_authflavor_t flavor = RPC_AUTH_GSS_KRB5I;
>>> 
>>> 	if (clp->cl_cons_state == NFS_CS_READY) {
>>> 		/* the client is initialised already */
>>> @@ -368,8 +369,11 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
>>> 	if (clp->cl_minorversion != 0)
>>> 		__set_bit(NFS_CS_INFINITE_SLOTS, &clp->cl_flags);
>>> 	__set_bit(NFS_CS_DISCRTRY, &clp->cl_flags);
>>> -	error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_GSS_KRB5I);
>>> -	if (error == -EINVAL)
>>> +
>>> +	if (nfs4_use_min_auth)
>>> +		flavor = RPC_AUTH_UNIX;
>>> +	error = nfs_create_rpc_client(clp, timeparms, flavor);
>>> +	if (error == -EINVAL && flavor != RPC_AUTH_UNIX)
>>> 		error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_UNIX);
>>> 	if (error < 0)
>>> 		goto error;
>>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
>>> index d53d678..00162cb 100644
>>> --- a/fs/nfs/nfs4proc.c
>>> +++ b/fs/nfs/nfs4proc.c
>>> @@ -2864,7 +2864,9 @@ static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
>>> 	int status = -EPERM;
>>> 	size_t i;
>>> 
>>> -	for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
>>> +	if (nfs4_use_min_auth)
>>> +		status = nfs4_lookup_root_sec(server, fhandle, info, RPC_AUTH_UNIX);
>>> +	else for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
>>> 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
>>> 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
>>> 			continue;
>>> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
>>> index a03b9c6..42b4f9b 100644
>>> --- a/fs/nfs/super.c
>>> +++ b/fs/nfs/super.c
>>> @@ -2791,6 +2791,7 @@ unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
>>> unsigned short send_implementation_id = 1;
>>> char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
>>> bool recover_lost_locks = false;
>>> +bool nfs4_use_min_auth = false;
>>> 
>>> EXPORT_SYMBOL_GPL(nfs_callback_set_tcpport);
>>> EXPORT_SYMBOL_GPL(nfs_callback_tcpport);
>>> @@ -2800,6 +2801,7 @@ EXPORT_SYMBOL_GPL(max_session_slots);
>>> EXPORT_SYMBOL_GPL(send_implementation_id);
>>> EXPORT_SYMBOL_GPL(nfs4_client_id_uniquifier);
>>> EXPORT_SYMBOL_GPL(recover_lost_locks);
>>> +EXPORT_SYMBOL_GPL(nfs4_use_min_auth);
>>> 
>>> #define NFS_CALLBACK_MAXPORTNR (65535U)
>>> 
>>> @@ -2842,5 +2844,7 @@ MODULE_PARM_DESC(recover_lost_locks,
>>> 		 "If the server reports that a lock might be lost, "
>>> 		 "try to recover it risking data corruption.");
>>> 
>>> -
>>> +module_param(nfs4_use_min_auth, bool, 0644);
>>> +MODULE_PARM_DESC(nfs4_use_min_auth,
>>> +		"Use mimnal auth in SETCLIENTID operation");
>>> #endif /* CONFIG_NFS_V4 */
>>> -- 
>>> 1.7.1
>>> 
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to [e-mail]
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>> --
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>> 
>> 
>> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 12:41 p.m. UTC | #7

On 07/11/13 18:05, Chuck Lever wrote:
> 
> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
> 
>> Hey mrchuck... 
>>
>> On 07/11/13 14:25, Chuck Lever wrote:
>>> Hi Steve-
>>>
>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>
>>>> This new module parameter makes the v4 client
>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>> when establishing NFSV4 state and doing the
>>>> pseudoroot lookup
>>>
>>> The patch description doesn't say, but is this change to work 
>>> around the 15 second GSSD upcall timeout? 
>> Yes. A 15 second delay on every mount due to security that
>> nobody is requesting is just not good.. IMHO..
> 
> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
> you expect the mount to work (ie why would it be good for any properly configured 
> environment to take 15 seconds to establish a GSS context?).
> 
> In other words, there are still cases where users wait 15 seconds unnecessarily, 
> and not because of the use of krb5i for lease management.  Aren't those of concern?
No. I think the concern here, at least my concern, is the lack of management.
We are forcing admins to use krb5i in lease management when its not necessary
and there is no way to turn it off.
  
> 
>> Also running
>> a security daemon for non-secure mounts just seems wrong to me.
> 
> It seems wrong to me to keep auth_rpcgss loaded if no mounts use Kerberos security.  
> What's the difference between that and running gssd all the time?
You can say thing about %99 of all the modules load in a Fedora kernel today.
lsmod | wc -l show there is 107 modules load on a f19 system. Do you think
all those modules are used? Of course not, but they are not spewing 25
log messages per mount or sucking up CPU cycles unnecessarily. 

> 
> The new SETCLIENTID architecture would be transparent if our security 
> infrastructure was reliable.  We really need to address the upcall issue.  
> We've been working around it for a very long time.
But its not transparent at this point. So we need a way to disable it
until it comes transparent. That is what this module parameter does.

> 
>>
>>> Have we completely given up on fixing the upcall?
>> Back when we were looking on to introduce gss-proxy into
>> the client, it was decided the upcall on the client
>> was not going to change. I'm just going with that!
> 
> I don't remember that discussion.  Or if it occurred, I didn't 
> realize at the time that it was referring to fixing bugs.  
> Do you have a link to the thread?
It was when we were discussing how to use gss-proxy. If I 
remember correctly it was a verbal discussion... 

steved.
 
> 
> 
>> steved.
>>
>>
>>>
>>>
>>>> Signed-off-by: Steve Dickson <[e-mail]>
>>>> ---
>>>> fs/nfs/nfs4_fs.h    |    1 +
>>>> fs/nfs/nfs4client.c |    8 ++++++--
>>>> fs/nfs/nfs4proc.c   |    4 +++-
>>>> fs/nfs/super.c      |    6 +++++-
>>>> 4 files changed, 15 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
>>>> index 28842ab..20bf925 100644
>>>> --- a/fs/nfs/nfs4_fs.h
>>>> +++ b/fs/nfs/nfs4_fs.h
>>>> @@ -438,6 +438,7 @@ extern bool nfs4_disable_idmapping;
>>>> extern unsigned short max_session_slots;
>>>> extern unsigned short send_implementation_id;
>>>> extern bool recover_lost_locks;
>>>> +extern bool nfs4_use_min_auth;
>>>>
>>>> #define NFS4_CLIENT_ID_UNIQ_LEN		(64)
>>>> extern char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN];
>>>> diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
>>>> index a860ab5..ff85991 100644
>>>> --- a/fs/nfs/nfs4client.c
>>>> +++ b/fs/nfs/nfs4client.c
>>>> @@ -355,6 +355,7 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
>>>> 	char buf[INET6_ADDRSTRLEN + 1];
>>>> 	struct nfs_client *old;
>>>> 	int error;
>>>> +	rpc_authflavor_t flavor = RPC_AUTH_GSS_KRB5I;
>>>>
>>>> 	if (clp->cl_cons_state == NFS_CS_READY) {
>>>> 		/* the client is initialised already */
>>>> @@ -368,8 +369,11 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
>>>> 	if (clp->cl_minorversion != 0)
>>>> 		__set_bit(NFS_CS_INFINITE_SLOTS, &clp->cl_flags);
>>>> 	__set_bit(NFS_CS_DISCRTRY, &clp->cl_flags);
>>>> -	error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_GSS_KRB5I);
>>>> -	if (error == -EINVAL)
>>>> +
>>>> +	if (nfs4_use_min_auth)
>>>> +		flavor = RPC_AUTH_UNIX;
>>>> +	error = nfs_create_rpc_client(clp, timeparms, flavor);
>>>> +	if (error == -EINVAL && flavor != RPC_AUTH_UNIX)
>>>> 		error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_UNIX);
>>>> 	if (error < 0)
>>>> 		goto error;
>>>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
>>>> index d53d678..00162cb 100644
>>>> --- a/fs/nfs/nfs4proc.c
>>>> +++ b/fs/nfs/nfs4proc.c
>>>> @@ -2864,7 +2864,9 @@ static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
>>>> 	int status = -EPERM;
>>>> 	size_t i;
>>>>
>>>> -	for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
>>>> +	if (nfs4_use_min_auth)
>>>> +		status = nfs4_lookup_root_sec(server, fhandle, info, RPC_AUTH_UNIX);
>>>> +	else for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
>>>> 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
>>>> 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
>>>> 			continue;
>>>> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
>>>> index a03b9c6..42b4f9b 100644
>>>> --- a/fs/nfs/super.c
>>>> +++ b/fs/nfs/super.c
>>>> @@ -2791,6 +2791,7 @@ unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
>>>> unsigned short send_implementation_id = 1;
>>>> char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
>>>> bool recover_lost_locks = false;
>>>> +bool nfs4_use_min_auth = false;
>>>>
>>>> EXPORT_SYMBOL_GPL(nfs_callback_set_tcpport);
>>>> EXPORT_SYMBOL_GPL(nfs_callback_tcpport);
>>>> @@ -2800,6 +2801,7 @@ EXPORT_SYMBOL_GPL(max_session_slots);
>>>> EXPORT_SYMBOL_GPL(send_implementation_id);
>>>> EXPORT_SYMBOL_GPL(nfs4_client_id_uniquifier);
>>>> EXPORT_SYMBOL_GPL(recover_lost_locks);
>>>> +EXPORT_SYMBOL_GPL(nfs4_use_min_auth);
>>>>
>>>> #define NFS_CALLBACK_MAXPORTNR (65535U)
>>>>
>>>> @@ -2842,5 +2844,7 @@ MODULE_PARM_DESC(recover_lost_locks,
>>>> 		 "If the server reports that a lock might be lost, "
>>>> 		 "try to recover it risking data corruption.");
>>>>
>>>> -
>>>> +module_param(nfs4_use_min_auth, bool, 0644);
>>>> +MODULE_PARM_DESC(nfs4_use_min_auth,
>>>> +		"Use mimnal auth in SETCLIENTID operation");
>>>> #endif /* CONFIG_NFS_V4 */
>>>> -- 
>>>> 1.7.1
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to [e-mail]
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> --
>>> Chuck Lever
>>> chuck[dot]lever[at]oracle[dot]com
>>>
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [e-mail]
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jeff Layton Nov. 8, 2013, 1:22 p.m. UTC | #8

On Fri, 08 Nov 2013 07:41:32 -0500
Steve Dickson <[e-mail]> wrote:

> 
> 
> On 07/11/13 18:05, Chuck Lever wrote:
> > 
> > On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
> > 
> >> Hey mrchuck... 
> >>
> >> On 07/11/13 14:25, Chuck Lever wrote:
> >>> Hi Steve-
> >>>
> >>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> >>>
> >>>> This new module parameter makes the v4 client
> >>>> use the minimal authentication flavor (AUTH_UNIX)
> >>>> when establishing NFSV4 state and doing the
> >>>> pseudoroot lookup
> >>>
> >>> The patch description doesn't say, but is this change to work 
> >>> around the 15 second GSSD upcall timeout? 
> >> Yes. A 15 second delay on every mount due to security that
> >> nobody is requesting is just not good.. IMHO..
> > 
> > One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
> > as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
> > you expect the mount to work (ie why would it be good for any properly configured 
> > environment to take 15 seconds to establish a GSS context?).
> > 
> > In other words, there are still cases where users wait 15 seconds unnecessarily, 
> > and not because of the use of krb5i for lease management.  Aren't those of concern?
> No. I think the concern here, at least my concern, is the lack of management.
> We are forcing admins to use krb5i in lease management when its not necessary
> and there is no way to turn it off.
>   

I don't think that's really the case. The idea was to have the client
attempt to use krb5i if it's available, and then to fall back to
AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
GSSAPI upcall succeeded or failed immediately instead of requiring this
timeout when the daemon isn't running.

> > 
> >> Also running
> >> a security daemon for non-secure mounts just seems wrong to me.
> > 
> > It seems wrong to me to keep auth_rpcgss loaded if no mounts use Kerberos security.  
> > What's the difference between that and running gssd all the time?
> You can say thing about %99 of all the modules load in a Fedora kernel today.
> lsmod | wc -l show there is 107 modules load on a f19 system. Do you think
> all those modules are used? Of course not, but they are not spewing 25
> log messages per mount or sucking up CPU cycles unnecessarily. 
> 

rpc.gssd spends most of its time sleeping, unless there is
nfs mount or "real" gssapi activity. CPU cycles shouldn't be an issue.
The RSS for the process on my 64-bit box is ~2k so I wouldn't even be
concerned about memory usage. For comparison, we don't worry much about
running rpc.statd these days and it's of similar size and duty cycle.

The log messages are another matter. rpc.gssd is just too chatty by
default. That's fixable, but it would mean that we'd need to tell
people to run it in verbose mode when tracking down problems instead of
assuming that all of those messages would go to the logs. That seems
like a reasonable thing to do anyway. Most people don't care about
rpc.gssd log messages once they have kerberized NFS working.


> > 
> > The new SETCLIENTID architecture would be transparent if our security 
> > infrastructure was reliable.  We really need to address the upcall issue.  
> > We've been working around it for a very long time.
> But its not transparent at this point. So we need a way to disable it
> until it comes transparent. That is what this module parameter does.
> 
> > 
> >>
> >>> Have we completely given up on fixing the upcall?
> >> Back when we were looking on to introduce gss-proxy into
> >> the client, it was decided the upcall on the client
> >> was not going to change. I'm just going with that!
> > 
> > I don't remember that discussion.  Or if it occurred, I didn't 
> > realize at the time that it was referring to fixing bugs.  
> > Do you have a link to the thread?
> It was when we were discussing how to use gss-proxy. If I 
> remember correctly it was a verbal discussion... 
> 
> steved.
>  
> > 
> > 
> >> steved.
> >>
> >>
> >>>
> >>>
> >>>> Signed-off-by: Steve Dickson <[e-mail]>
> >>>> ---
> >>>> fs/nfs/nfs4_fs.h    |    1 +
> >>>> fs/nfs/nfs4client.c |    8 ++++++--
> >>>> fs/nfs/nfs4proc.c   |    4 +++-
> >>>> fs/nfs/super.c      |    6 +++++-
> >>>> 4 files changed, 15 insertions(+), 4 deletions(-)
> >>>>
> >>>> diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
> >>>> index 28842ab..20bf925 100644
> >>>> --- a/fs/nfs/nfs4_fs.h
> >>>> +++ b/fs/nfs/nfs4_fs.h
> >>>> @@ -438,6 +438,7 @@ extern bool nfs4_disable_idmapping;
> >>>> extern unsigned short max_session_slots;
> >>>> extern unsigned short send_implementation_id;
> >>>> extern bool recover_lost_locks;
> >>>> +extern bool nfs4_use_min_auth;
> >>>>
> >>>> #define NFS4_CLIENT_ID_UNIQ_LEN		(64)
> >>>> extern char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN];
> >>>> diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
> >>>> index a860ab5..ff85991 100644
> >>>> --- a/fs/nfs/nfs4client.c
> >>>> +++ b/fs/nfs/nfs4client.c
> >>>> @@ -355,6 +355,7 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
> >>>> 	char buf[INET6_ADDRSTRLEN + 1];
> >>>> 	struct nfs_client *old;
> >>>> 	int error;
> >>>> +	rpc_authflavor_t flavor = RPC_AUTH_GSS_KRB5I;
> >>>>
> >>>> 	if (clp->cl_cons_state == NFS_CS_READY) {
> >>>> 		/* the client is initialised already */
> >>>> @@ -368,8 +369,11 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
> >>>> 	if (clp->cl_minorversion != 0)
> >>>> 		__set_bit(NFS_CS_INFINITE_SLOTS, &clp->cl_flags);
> >>>> 	__set_bit(NFS_CS_DISCRTRY, &clp->cl_flags);
> >>>> -	error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_GSS_KRB5I);
> >>>> -	if (error == -EINVAL)
> >>>> +
> >>>> +	if (nfs4_use_min_auth)
> >>>> +		flavor = RPC_AUTH_UNIX;
> >>>> +	error = nfs_create_rpc_client(clp, timeparms, flavor);
> >>>> +	if (error == -EINVAL && flavor != RPC_AUTH_UNIX)
> >>>> 		error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_UNIX);
> >>>> 	if (error < 0)
> >>>> 		goto error;
> >>>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> >>>> index d53d678..00162cb 100644
> >>>> --- a/fs/nfs/nfs4proc.c
> >>>> +++ b/fs/nfs/nfs4proc.c
> >>>> @@ -2864,7 +2864,9 @@ static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
> >>>> 	int status = -EPERM;
> >>>> 	size_t i;
> >>>>
> >>>> -	for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
> >>>> +	if (nfs4_use_min_auth)
> >>>> +		status = nfs4_lookup_root_sec(server, fhandle, info, RPC_AUTH_UNIX);
> >>>> +	else for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
> >>>> 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
> >>>> 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
> >>>> 			continue;
> >>>> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
> >>>> index a03b9c6..42b4f9b 100644
> >>>> --- a/fs/nfs/super.c
> >>>> +++ b/fs/nfs/super.c
> >>>> @@ -2791,6 +2791,7 @@ unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
> >>>> unsigned short send_implementation_id = 1;
> >>>> char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
> >>>> bool recover_lost_locks = false;
> >>>> +bool nfs4_use_min_auth = false;
> >>>>
> >>>> EXPORT_SYMBOL_GPL(nfs_callback_set_tcpport);
> >>>> EXPORT_SYMBOL_GPL(nfs_callback_tcpport);
> >>>> @@ -2800,6 +2801,7 @@ EXPORT_SYMBOL_GPL(max_session_slots);
> >>>> EXPORT_SYMBOL_GPL(send_implementation_id);
> >>>> EXPORT_SYMBOL_GPL(nfs4_client_id_uniquifier);
> >>>> EXPORT_SYMBOL_GPL(recover_lost_locks);
> >>>> +EXPORT_SYMBOL_GPL(nfs4_use_min_auth);
> >>>>
> >>>> #define NFS_CALLBACK_MAXPORTNR (65535U)
> >>>>
> >>>> @@ -2842,5 +2844,7 @@ MODULE_PARM_DESC(recover_lost_locks,
> >>>> 		 "If the server reports that a lock might be lost, "
> >>>> 		 "try to recover it risking data corruption.");
> >>>>
> >>>> -
> >>>> +module_param(nfs4_use_min_auth, bool, 0644);
> >>>> +MODULE_PARM_DESC(nfs4_use_min_auth,
> >>>> +		"Use mimnal auth in SETCLIENTID operation");
> >>>> #endif /* CONFIG_NFS_V4 */
> >>>> -- 
> >>>> 1.7.1
> >>>>
> >>>> --
> >>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> >>>> the body of a message to [e-mail]
> >>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>>
> >>> --
> >>> Chuck Lever
> >>> chuck[dot]lever[at]oracle[dot]com
> >>>
> >>>
> >>>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> >> the body of a message to [e-mail]
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> > --
> > Chuck Lever
> > chuck[dot]lever[at]oracle[dot]com
> > 
> > 
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Trond Myklebust Nov. 8, 2013, 2:30 p.m. UTC | #9

On Nov 8, 2013, at 7:21, Steve Dickson <[e-mail]> wrote:

> 
> 
> On 07/11/13 17:29, Myklebust, Trond wrote:
>>> Again, what servers, today, support this type of secure state establishment?
>>>> Having this type of security in the client I think is good... but if
>>>> the client is not talking with any servers that support this type 
>>>> of security, why not have a way to turn it off?
>> I don't understand. Servers are _required_ to support RPCSEC_GSS with
>> krb5 by both RFC3530 and RFC5661. AUTH_SYS is, in fact, the optional
>> flavour.
> Agreed... 100% of the NFSv4 server have to support RPCSEC_GSS. its mandated
> by the spec(s). 
> 
>> 
>> The problem here is that sometimes kerberos isn't configured by the
>> admin, who then expects that it shouldn't be necessary to run rpc.gssd
>> or rpc.svcgssd. It is necessary because we first try the
>> mandatory-to-implement and secure RPCSEC_GSS/krb5i flavour before
>> falling back to the less secure AUTH_SYS...
> Sometimes? Its generally not.. from my experience... 
> 
> Basically how I interpret this last paragraph, is we will be requiring 
> admins set up secure mounts for them to avoid the 15sec delay mount
> times... aka... running a daemon that will say "no, no there is no 
> security here" while spewing of log messages when Kerberos is not setup...

No. All we are requiring is that they run rpc.gssd.

--
Trond Myklebust
Linux NFS client maintainer

NetApp
[e-mail]
www.netapp.com

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 3 p.m. UTC | #10

On 08/11/13 08:22, Jeff Layton wrote:
> On Fri, 08 Nov 2013 07:41:32 -0500
> Steve Dickson <[e-mail]> wrote:
> 
>>
>>
>> On 07/11/13 18:05, Chuck Lever wrote:
>>>
>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>
>>>> Hey mrchuck... 
>>>>
>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>> Hi Steve-
>>>>>
>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>
>>>>>> This new module parameter makes the v4 client
>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>> when establishing NFSV4 state and doing the
>>>>>> pseudoroot lookup
>>>>>
>>>>> The patch description doesn't say, but is this change to work 
>>>>> around the 15 second GSSD upcall timeout? 
>>>> Yes. A 15 second delay on every mount due to security that
>>>> nobody is requesting is just not good.. IMHO..
>>>
>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>> you expect the mount to work (ie why would it be good for any properly configured 
>>> environment to take 15 seconds to establish a GSS context?).
>>>
>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>> No. I think the concern here, at least my concern, is the lack of management.
>> We are forcing admins to use krb5i in lease management when its not necessary
>> and there is no way to turn it off.
>>   
> 
> I don't think that's really the case. The idea was to have the client
> attempt to use krb5i if it's available, and then to fall back to
> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
> GSSAPI upcall succeeded or failed immediately instead of requiring this
> timeout when the daemon isn't running.
What server makes krb5i available today in state setup and pseudoroot lookups?

> 
>>>
>>>> Also running
>>>> a security daemon for non-secure mounts just seems wrong to me.
>>>
>>> It seems wrong to me to keep auth_rpcgss loaded if no mounts use Kerberos security.  
>>> What's the difference between that and running gssd all the time?
>> You can say thing about %99 of all the modules load in a Fedora kernel today.
>> lsmod | wc -l show there is 107 modules load on a f19 system. Do you think
>> all those modules are used? Of course not, but they are not spewing 25
>> log messages per mount or sucking up CPU cycles unnecessarily. 
>>
> 
> rpc.gssd spends most of its time sleeping, unless there is
> nfs mount or "real" gssapi activity. CPU cycles shouldn't be an issue.
> The RSS for the process on my 64-bit box is ~2k so I wouldn't even be
> concerned about memory usage. For comparison, we don't worry much about
> running rpc.statd these days and it's of similar size and duty cycle.
rpc.statd is needed by the V3 protocol to support file locking which
is a bit more needed that a rpc.gssd to do unsecured mounts.

> 
> The log messages are another matter. rpc.gssd is just too chatty by
> default. That's fixable, but it would mean that we'd need to tell
> people to run it in verbose mode when tracking down problems instead of
> assuming that all of those messages would go to the logs. That seems
> like a reasonable thing to do anyway. Most people don't care about
> rpc.gssd log messages once they have kerberized NFS working.
So we should remove error and warning messages that in the
pass help us debug very difficult code so we can run rpc.gssd
just for normal unsecured mounts? 

My apologies... but that makes absolutely no sense to me. 

Let talk about scalability... Does anybody have idea what
this needless upcall will cost on a client that does 
a very large number of mounts all at once? 

Please let me reiterate my point. The new securing of
SETCLIENTIDs and pseudoroot lookups is good! I have
no problem with the actual technology. What I'm having
a problem with is I can not manage this new technology.

Could you image the push back there would be if, 
back in the day when secure mounts became available, 
we required *everyone* to use secure mounts. The 
only way you could use NFS  was through a secure mount.

That's basically what we are doing today! We are requiring
every mount to try a secure flavor that fails %100 time
because there is no server support for this technology. 

Again its not the technology, its the management of the
technology that this new module parameter address. All
I'm looking for is switch that enables or disables 
this new technology... 

steved.


--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 3:04 p.m. UTC | #11

On Fri, Nov 08, 2013 at 08:22:02AM -0500, Jeff Layton wrote:
> On Fri, 08 Nov 2013 07:41:32 -0500
> Steve Dickson <[e-mail]> wrote:
> > No. I think the concern here, at least my concern, is the lack of management.
> > We are forcing admins to use krb5i in lease management when its not necessary
> > and there is no way to turn it off.
> >   
> 
> I don't think that's really the case. The idea was to have the client
> attempt to use krb5i if it's available, and then to fall back to
> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
> GSSAPI upcall succeeded or failed immediately instead of requiring this
> timeout when the daemon isn't running.

I'm also still a little confused about the security model.  We discussed
it before but I can't remember if it was really resolved.

It makes sense to me as long as we insist on krb5i whenever we find a
keytab.

But my understanding was that with the current implementation it's
possible we could find a keytab, attempt the krb5i connection, and
*then* fallback silently on auth_sys if krb5i fails.  Is that right?

In that case I don't see the point of the krb5i any more: any attacker
that can spoof rpc replies can force the fallback to auth_sys.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 3:08 p.m. UTC | #12

On 08/11/13 09:30, Myklebust, Trond wrote:
> 
> On Nov 8, 2013, at 7:21, Steve Dickson <[e-mail]> wrote:
> 
>>
>>
>> On 07/11/13 17:29, Myklebust, Trond wrote:
>>>> Again, what servers, today, support this type of secure state establishment?
>>>>> Having this type of security in the client I think is good... but if
>>>>> the client is not talking with any servers that support this type 
>>>>> of security, why not have a way to turn it off?
>>> I don't understand. Servers are _required_ to support RPCSEC_GSS with
>>> krb5 by both RFC3530 and RFC5661. AUTH_SYS is, in fact, the optional
>>> flavour.
>> Agreed... 100% of the NFSv4 server have to support RPCSEC_GSS. its mandated
>> by the spec(s). 
>>
>>>
>>> The problem here is that sometimes kerberos isn't configured by the
>>> admin, who then expects that it shouldn't be necessary to run rpc.gssd
>>> or rpc.svcgssd. It is necessary because we first try the
>>> mandatory-to-implement and secure RPCSEC_GSS/krb5i flavour before
>>> falling back to the less secure AUTH_SYS...
>> Sometimes? Its generally not.. from my experience... 
>>
>> Basically how I interpret this last paragraph, is we will be requiring 
>> admins set up secure mounts for them to avoid the 15sec delay mount
>> times... aka... running a daemon that will say "no, no there is no 
>> security here" while spewing of log messages when Kerberos is not setup...
> 
> No. All we are requiring is that they run rpc.gssd.
Even when they do not want any secure mounts at all?

What is that justification?


steved.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jeff Layton Nov. 8, 2013, 3:12 p.m. UTC | #13

On Fri, 08 Nov 2013 10:00:02 -0500
Steve Dickson <[e-mail]> wrote:

> 
> 
> On 08/11/13 08:22, Jeff Layton wrote:
> > On Fri, 08 Nov 2013 07:41:32 -0500
> > Steve Dickson <[e-mail]> wrote:
> > 
> >>
> >>
> >> On 07/11/13 18:05, Chuck Lever wrote:
> >>>
> >>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
> >>>
> >>>> Hey mrchuck... 
> >>>>
> >>>> On 07/11/13 14:25, Chuck Lever wrote:
> >>>>> Hi Steve-
> >>>>>
> >>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> >>>>>
> >>>>>> This new module parameter makes the v4 client
> >>>>>> use the minimal authentication flavor (AUTH_UNIX)
> >>>>>> when establishing NFSV4 state and doing the
> >>>>>> pseudoroot lookup
> >>>>>
> >>>>> The patch description doesn't say, but is this change to work 
> >>>>> around the 15 second GSSD upcall timeout? 
> >>>> Yes. A 15 second delay on every mount due to security that
> >>>> nobody is requesting is just not good.. IMHO..
> >>>
> >>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
> >>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
> >>> you expect the mount to work (ie why would it be good for any properly configured 
> >>> environment to take 15 seconds to establish a GSS context?).
> >>>
> >>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
> >>> and not because of the use of krb5i for lease management.  Aren't those of concern?
> >> No. I think the concern here, at least my concern, is the lack of management.
> >> We are forcing admins to use krb5i in lease management when its not necessary
> >> and there is no way to turn it off.
> >>   
> > 
> > I don't think that's really the case. The idea was to have the client
> > attempt to use krb5i if it's available, and then to fall back to
> > AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
> > GSSAPI upcall succeeded or failed immediately instead of requiring this
> > timeout when the daemon isn't running.
> What server makes krb5i available today in state setup and pseudoroot lookups?
> 

That I don't know...sorry...

> > 
> >>>
> >>>> Also running
> >>>> a security daemon for non-secure mounts just seems wrong to me.
> >>>
> >>> It seems wrong to me to keep auth_rpcgss loaded if no mounts use Kerberos security.  
> >>> What's the difference between that and running gssd all the time?
> >> You can say thing about %99 of all the modules load in a Fedora kernel today.
> >> lsmod | wc -l show there is 107 modules load on a f19 system. Do you think
> >> all those modules are used? Of course not, but they are not spewing 25
> >> log messages per mount or sucking up CPU cycles unnecessarily. 
> >>
> > 
> > rpc.gssd spends most of its time sleeping, unless there is
> > nfs mount or "real" gssapi activity. CPU cycles shouldn't be an issue.
> > The RSS for the process on my 64-bit box is ~2k so I wouldn't even be
> > concerned about memory usage. For comparison, we don't worry much about
> > running rpc.statd these days and it's of similar size and duty cycle.
> rpc.statd is needed by the V3 protocol to support file locking which
> is a bit more needed that a rpc.gssd to do unsecured mounts.
> 

Right, but we've defaulted to v4 for *years* now. Most people run statd
and don't actually use it. No one seems to get bent out of shape about
that though.

> > 
> > The log messages are another matter. rpc.gssd is just too chatty by
> > default. That's fixable, but it would mean that we'd need to tell
> > people to run it in verbose mode when tracking down problems instead of
> > assuming that all of those messages would go to the logs. That seems
> > like a reasonable thing to do anyway. Most people don't care about
> > rpc.gssd log messages once they have kerberized NFS working.
>
> So we should remove error and warning messages that in the
> pass help us debug very difficult code so we can run rpc.gssd
> just for normal unsecured mounts? 
> 

Yes. Because in the case where you don't have a keytab or a credcache,
failure is expected. There's no need to log messages due to those sorts
of problems. Once you have it working correctly, there's just no need to
keep logging that junk. Most of those log messages should only be
enabled when rpc.gssd is run with -v(vvv). Admins can (and often do)
run rpc.gssd in foreground with those flags while they sort through the
setup.

> My apologies... but that makes absolutely no sense to me. 
> 
> Let talk about scalability... Does anybody have idea what
> this needless upcall will cost on a client that does 
> a very large number of mounts all at once? 
> 

Virtually nothing -- a couple of pipe reads/writes.

> Please let me reiterate my point. The new securing of
> SETCLIENTIDs and pseudoroot lookups is good! I have
> no problem with the actual technology. What I'm having
> a problem with is I can not manage this new technology.
> 
> Could you image the push back there would be if, 
> back in the day when secure mounts became available, 
> we required *everyone* to use secure mounts. The 
> only way you could use NFS  was through a secure mount.
> 
> That's basically what we are doing today! We are requiring
> every mount to try a secure flavor that fails %100 time
> because there is no server support for this technology. 
> 
> Again its not the technology, its the management of the
> technology that this new module parameter address. All
> I'm looking for is switch that enables or disables 
> this new technology... 
> 

No one is requiring anyone to do anything. IMO, running rpc.gssd by
default is simply a reasonable workaround for this long delay in
mounting until we have a real solution for this problem.

I'd like to see us transition to a more sensible upcall in the future
that doesn't require this timeout, but we're not there yet. Once we
have that, we won't need to run rpc.gssd at all anymore.

FWIW, I'm perfectly happy do the work on such an upcall. I think
call_usermodehelper (or maybe keys API) makes a lot of sense for this.
The only real problem I can see with doing this today is the damnable
namespaces. The nfs client is currently net-namespacified, but running
something like call_usermodehelper sort of requires that you switch the
mnt namespace too.

Trond Myklebust Nov. 8, 2013, 3:16 p.m. UTC | #14

On Nov 8, 2013, at 10:08, Steve Dickson <[e-mail]> wrote:

> 
> 
> On 08/11/13 09:30, Myklebust, Trond wrote:
>> 
>> On Nov 8, 2013, at 7:21, Steve Dickson <[e-mail]> wrote:
>> 
>>> 
>>> 
>>> On 07/11/13 17:29, Myklebust, Trond wrote:
>>>>> Again, what servers, today, support this type of secure state establishment?
>>>>>> Having this type of security in the client I think is good... but if
>>>>>> the client is not talking with any servers that support this type 
>>>>>> of security, why not have a way to turn it off?
>>>> I don't understand. Servers are _required_ to support RPCSEC_GSS with
>>>> krb5 by both RFC3530 and RFC5661. AUTH_SYS is, in fact, the optional
>>>> flavour.
>>> Agreed... 100% of the NFSv4 server have to support RPCSEC_GSS. its mandated
>>> by the spec(s). 
>>> 
>>>> 
>>>> The problem here is that sometimes kerberos isn't configured by the
>>>> admin, who then expects that it shouldn't be necessary to run rpc.gssd
>>>> or rpc.svcgssd. It is necessary because we first try the
>>>> mandatory-to-implement and secure RPCSEC_GSS/krb5i flavour before
>>>> falling back to the less secure AUTH_SYS...
>>> Sometimes? Its generally not.. from my experience... 
>>> 
>>> Basically how I interpret this last paragraph, is we will be requiring 
>>> admins set up secure mounts for them to avoid the 15sec delay mount
>>> times... aka... running a daemon that will say "no, no there is no 
>>> security here" while spewing of log messages when Kerberos is not setup...
>> 
>> No. All we are requiring is that they run rpc.gssd.
> Even when they do not want any secure mounts at all?
> 
> What is that justification?

They get to skip a 15 second wait without having to blacklist the krb5 module.

So, this is what I mean when I say that we _might_ need a mount option to specify the security flavour that the client uses for the lease. It solves the problem that Bruce mentioned about what to do when krb5i fails, and it allows admins to not have to run rpc.gssd or blacklist any modules.

Trond

--
Trond Myklebust
Linux NFS client maintainer

NetApp
[e-mail]
www.netapp.com

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Chuck Lever III Nov. 8, 2013, 3:46 p.m. UTC | #15

On Nov 8, 2013, at 4:41 AM, Steve Dickson <[e-mail]> wrote:

> 
> 
> On 07/11/13 18:05, Chuck Lever wrote:
>> 
>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>> 
>>> Hey mrchuck... 
>>> 
>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>> Hi Steve-
>>>> 
>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>> 
>>>>> This new module parameter makes the v4 client
>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>> when establishing NFSV4 state and doing the
>>>>> pseudoroot lookup
>>>> 
>>>> The patch description doesn't say, but is this change to work 
>>>> around the 15 second GSSD upcall timeout? 
>>> Yes. A 15 second delay on every mount due to security that
>>> nobody is requesting is just not good.. IMHO..
>> 
>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>> you expect the mount to work (ie why would it be good for any properly configured 
>> environment to take 15 seconds to establish a GSS context?).
>> 
>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>> and not because of the use of krb5i for lease management.  Aren't those of concern?
> No. I think the concern here, at least my concern, is the lack of management.
> We are forcing admins to use krb5i in lease management when its not necessary
> and there is no way to turn it off.

Before, we were "forcing" admins to use AUTH_SYS on some mounts even though they had chosen to use sec=krb5 on some mounts.  How is that a good thing?

Now, we are "forcing" admins to use security, but only when it is available.  In other words, our default now is to be more secure rather than insecure.  Why is that a bad thing?

The IETF has now reached consensus to require security in every new protocol it specifies.  They want to enable security and not give ways to turn it off.  They are even considering re-specifying existing widely deployed protocols to increase their security.

"it's not necessary" is no longer an excuse not to use security.  Read the minutes of Wednesday's IAB technical plenary if you doubt me (yes, someone actually said that during the open mic, and there was some applause).

Earlier Trond noted that NFSv4 REQUIRES implementations to support RPCSEC GSS.  That requirement has been there since RFC 3010.  There is a reason for that.

The only reason you are arguing to disable security here is because it is a mild annoyance.  And instead of addressing the annoyance, you want to allow users to disable security.

I'd like a solution that chooses to use security when its available, rather than blindly sticking with an insecure default.  I'd like a solution that makes that choice automatically without adding yet another obscure setting to our panoply of obscure settings.  It should "just work."

We are "this close" to getting that.  Let's not take a step backwards.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Chuck Lever III Nov. 8, 2013, 3:54 p.m. UTC | #16

On Nov 8, 2013, at 7:04 AM, J. Bruce Fields <[e-mail]> wrote:

> On Fri, Nov 08, 2013 at 08:22:02AM -0500, Jeff Layton wrote:
>> On Fri, 08 Nov 2013 07:41:32 -0500
>> Steve Dickson <[e-mail]> wrote:
>>> No. I think the concern here, at least my concern, is the lack of management.
>>> We are forcing admins to use krb5i in lease management when its not necessary
>>> and there is no way to turn it off.
>>> 
>> 
>> I don't think that's really the case. The idea was to have the client
>> attempt to use krb5i if it's available, and then to fall back to
>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>> timeout when the daemon isn't running.
> 
> I'm also still a little confused about the security model.  We discussed
> it before but I can't remember if it was really resolved.
> 
> It makes sense to me as long as we insist on krb5i whenever we find a
> keytab.
> 
> But my understanding was that with the current implementation it's
> possible we could find a keytab, attempt the krb5i connection, and
> *then* fallback silently on auth_sys if krb5i fails.  Is that right?
> 
> In that case I don't see the point of the krb5i any more: any attacker
> that can spoof rpc replies can force the fallback to auth_sys.

The point is to use a consistent security flavor for lease management to avoid NFS4ERR_CLID_INUSE.  This really isn't about thwarting a MiTM attack on lease management.

The fallback mechanism can be fixed, somewhat.  I've got a patch to have gssd return ENOKEY if it can't find a machine credential.  Then the kernel will use AUTH_SYS.

The issue though is what to do in the other cases.  Can gssd distinguish between the case where the server has no Kerberos principal, and the case where gssd simply failed to establish a GSS context?  Or should we simply use the "sec=" setting in this case and call it a day?

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:10 p.m. UTC | #17

On 08/11/13 10:12, Jeff Layton wrote:
> On Fri, 08 Nov 2013 10:00:02 -0500
> Steve Dickson <[e-mail]> wrote:
> 
>>
>>
>> On 08/11/13 08:22, Jeff Layton wrote:
>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>> Steve Dickson <[e-mail]> wrote:
>>>
>>>>
>>>>
>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>
>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>
>>>>>> Hey mrchuck... 
>>>>>>
>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>> Hi Steve-
>>>>>>>
>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>
>>>>>>>> This new module parameter makes the v4 client
>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>> pseudoroot lookup
>>>>>>>
>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>
>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>
>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>> and there is no way to turn it off.
>>>>   
>>>
>>> I don't think that's really the case. The idea was to have the client
>>> attempt to use krb5i if it's available, and then to fall back to
>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>> timeout when the daemon isn't running.
>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>
> 
> That I don't know...sorry...
Then what is the justification to take all these extra steps
there they going to fail %100 of the time??

 
> 
>>>
>>>>>
>>>>>> Also running
>>>>>> a security daemon for non-secure mounts just seems wrong to me.
>>>>>
>>>>> It seems wrong to me to keep auth_rpcgss loaded if no mounts use Kerberos security.  
>>>>> What's the difference between that and running gssd all the time?
>>>> You can say thing about %99 of all the modules load in a Fedora kernel today.
>>>> lsmod | wc -l show there is 107 modules load on a f19 system. Do you think
>>>> all those modules are used? Of course not, but they are not spewing 25
>>>> log messages per mount or sucking up CPU cycles unnecessarily. 
>>>>
>>>
>>> rpc.gssd spends most of its time sleeping, unless there is
>>> nfs mount or "real" gssapi activity. CPU cycles shouldn't be an issue.
>>> The RSS for the process on my 64-bit box is ~2k so I wouldn't even be
>>> concerned about memory usage. For comparison, we don't worry much about
>>> running rpc.statd these days and it's of similar size and duty cycle.
>> rpc.statd is needed by the V3 protocol to support file locking which
>> is a bit more needed that a rpc.gssd to do unsecured mounts.
>>
> 
> Right, but we've defaulted to v4 for *years* now. Most people run statd
> and don't actually use it. No one seems to get bent out of shape about
> that though.
So you using that justify another needless daemon? 

> 
>>>
>>> The log messages are another matter. rpc.gssd is just too chatty by
>>> default. That's fixable, but it would mean that we'd need to tell
>>> people to run it in verbose mode when tracking down problems instead of
>>> assuming that all of those messages would go to the logs. That seems
>>> like a reasonable thing to do anyway. Most people don't care about
>>> rpc.gssd log messages once they have kerberized NFS working.
>>
>> So we should remove error and warning messages that in the
>> pass help us debug very difficult code so we can run rpc.gssd
>> just for normal unsecured mounts? 
>>
> 
> Yes. Because in the case where you don't have a keytab or a credcache,
> failure is expected. There's no need to log messages due to those sorts
> of problems. Once you have it working correctly, there's just no need to
> keep logging that junk. Most of those log messages should only be
> enabled when rpc.gssd is run with -v(vvv). Admins can (and often do)
> run rpc.gssd in foreground with those flags while they sort through the
> setup.
When there is no keytab or credcache there should be to start rpc.gssd.
Non-secure mount should not depend on the existence of a user level 
daemon... When has ever been the case a NFS mount was dependent on
user level daemon? 



> 
>> My apologies... but that makes absolutely no sense to me. 
>>
>> Let talk about scalability... Does anybody have idea what
>> this needless upcall will cost on a client that does 
>> a very large number of mounts all at once? 
>>
> 
> Virtually nothing -- a couple of pipe reads/writes.
Until rpc.gssd dies and everything comes to a halt... ;-)

> 
>> Please let me reiterate my point. The new securing of
>> SETCLIENTIDs and pseudoroot lookups is good! I have
>> no problem with the actual technology. What I'm having
>> a problem with is I can not manage this new technology.
>>
>> Could you image the push back there would be if, 
>> back in the day when secure mounts became available, 
>> we required *everyone* to use secure mounts. The 
>> only way you could use NFS  was through a secure mount.
>>
>> That's basically what we are doing today! We are requiring
>> every mount to try a secure flavor that fails %100 time
>> because there is no server support for this technology. 
>>
>> Again its not the technology, its the management of the
>> technology that this new module parameter address. All
>> I'm looking for is switch that enables or disables 
>> this new technology... 
>>
> 
> No one is requiring anyone to do anything. IMO, running rpc.gssd by
> default is simply a reasonable workaround for this long delay in
> mounting until we have a real solution for this problem.
The client code is hard code to try security flavor on every
mount. There is no way around it. That is I mean by requiring
all mounts to use security flavor.
> 
> I'd like to see us transition to a more sensible upcall in the future
> that doesn't require this timeout, but we're not there yet. Once we
> have that, we won't need to run rpc.gssd at all anymore.
So we all agree, this code needs work... So why not have 
a way to disable the trying of security flavor until
its ready for prime time??? We done that in the past??

Or may this... Disable it until there is server support!
What is wrong with that??? 

steved.

> 
> FWIW, I'm perfectly happy do the work on such an upcall. I think
> call_usermodehelper (or maybe keys API) makes a lot of sense for this.
> The only real problem I can see with doing this today is the damnable
> namespaces. The nfs client is currently net-namespacified, but running
> something like call_usermodehelper sort of requires that you switch the
> mnt namespace too.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 4:14 p.m. UTC | #18

On Fri, Nov 08, 2013 at 07:54:19AM -0800, Chuck Lever wrote:
> 
> On Nov 8, 2013, at 7:04 AM, J. Bruce Fields <[e-mail]>
> wrote:
> 
> > On Fri, Nov 08, 2013 at 08:22:02AM -0500, Jeff Layton wrote:
> >> On Fri, 08 Nov 2013 07:41:32 -0500 Steve Dickson
> >> <[e-mail]> wrote:
> >>> No. I think the concern here, at least my concern, is the lack of
> >>> management.  We are forcing admins to use krb5i in lease
> >>> management when its not necessary and there is no way to turn it
> >>> off.
> >>> 
> >> 
> >> I don't think that's really the case. The idea was to have the
> >> client attempt to use krb5i if it's available, and then to fall
> >> back to AUTH_SYS if it isn't. This would be *absolutely* no big
> >> deal if the GSSAPI upcall succeeded or failed immediately instead
> >> of requiring this timeout when the daemon isn't running.
> > 
> > I'm also still a little confused about the security model.  We
> > discussed it before but I can't remember if it was really resolved.
> > 
> > It makes sense to me as long as we insist on krb5i whenever we find
> > a keytab.
> > 
> > But my understanding was that with the current implementation it's
> > possible we could find a keytab, attempt the krb5i connection, and
> > *then* fallback silently on auth_sys if krb5i fails.  Is that right?
> > 
> > In that case I don't see the point of the krb5i any more: any
> > attacker that can spoof rpc replies can force the fallback to
> > auth_sys.
> 
> The point is to use a consistent security flavor for lease management
> to avoid NFS4ERR_CLID_INUSE.  This really isn't about thwarting a MiTM
> attack on lease management.

OK, but then you still do want to ensure you thwart that attack in the
case krb5 is explicitly asked for.

So for example does a krb5 mount fail if a previous non-krb5 mount fell
back on auth_sys for clientid establishment?

> The fallback mechanism can be fixed, somewhat.  I've got a patch to
> have gssd return ENOKEY if it can't find a machine credential.  Then
> the kernel will use AUTH_SYS.

To get that right you'd then need to fail on errors *other* than ENOKEY.
Since old gssd doesn't return ENOKEY that would mean failing all
auth_sys mounts on kernel upgrade, wouldn't it?

> The issue though is what to do in the other cases.  Can gssd
> distinguish between the case where the server has no Kerberos
> principal, and the case where gssd simply failed to establish a GSS
> context?

I don't know.  I suppose the client should get some kind of
no-such-principal error and use that.

> Or should we simply use the "sec=" setting in this case and
> call it a day?

I think falling back works as long as we still are strict in the case
the mount explicitly requests some security and don't just use
whatever's already established in that case.

And if we do that then I don't think the ENOKEY/no-such-principal cases
are worth sorting out.

OK, sorry for the noise.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 4:17 p.m. UTC | #19

On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
> 
> 
> On 08/11/13 10:12, Jeff Layton wrote:
> > On Fri, 08 Nov 2013 10:00:02 -0500
> > Steve Dickson <[e-mail]> wrote:
> > 
> >>
> >>
> >> On 08/11/13 08:22, Jeff Layton wrote:
> >>> On Fri, 08 Nov 2013 07:41:32 -0500
> >>> Steve Dickson <[e-mail]> wrote:
> >>>
> >>>>
> >>>>
> >>>> On 07/11/13 18:05, Chuck Lever wrote:
> >>>>>
> >>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
> >>>>>
> >>>>>> Hey mrchuck... 
> >>>>>>
> >>>>>> On 07/11/13 14:25, Chuck Lever wrote:
> >>>>>>> Hi Steve-
> >>>>>>>
> >>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> >>>>>>>
> >>>>>>>> This new module parameter makes the v4 client
> >>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
> >>>>>>>> when establishing NFSV4 state and doing the
> >>>>>>>> pseudoroot lookup
> >>>>>>>
> >>>>>>> The patch description doesn't say, but is this change to work 
> >>>>>>> around the 15 second GSSD upcall timeout? 
> >>>>>> Yes. A 15 second delay on every mount due to security that
> >>>>>> nobody is requesting is just not good.. IMHO..
> >>>>>
> >>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
> >>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
> >>>>> you expect the mount to work (ie why would it be good for any properly configured 
> >>>>> environment to take 15 seconds to establish a GSS context?).
> >>>>>
> >>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
> >>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
> >>>> No. I think the concern here, at least my concern, is the lack of management.
> >>>> We are forcing admins to use krb5i in lease management when its not necessary
> >>>> and there is no way to turn it off.
> >>>>   
> >>>
> >>> I don't think that's really the case. The idea was to have the client
> >>> attempt to use krb5i if it's available, and then to fall back to
> >>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
> >>> GSSAPI upcall succeeded or failed immediately instead of requiring this
> >>> timeout when the daemon isn't running.
> >> What server makes krb5i available today in state setup and pseudoroot lookups?
> >>
> > 
> > That I don't know...sorry...
> Then what is the justification to take all these extra steps
> there they going to fail %100 of the time??

Any server can support krb5 for state setup and pseudoroot operations if
it's configured.  This isn't a problem.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:17 p.m. UTC | #20

On 08/11/13 10:54, Chuck Lever wrote:
> 
> On Nov 8, 2013, at 7:04 AM, J. Bruce Fields <[e-mail]> wrote:
> 
>> On Fri, Nov 08, 2013 at 08:22:02AM -0500, Jeff Layton wrote:
>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>> Steve Dickson <[e-mail]> wrote:
>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>> and there is no way to turn it off.
>>>>
>>>
>>> I don't think that's really the case. The idea was to have the client
>>> attempt to use krb5i if it's available, and then to fall back to
>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>> timeout when the daemon isn't running.
>>
>> I'm also still a little confused about the security model.  We discussed
>> it before but I can't remember if it was really resolved.
>>
>> It makes sense to me as long as we insist on krb5i whenever we find a
>> keytab.
>>
>> But my understanding was that with the current implementation it's
>> possible we could find a keytab, attempt the krb5i connection, and
>> *then* fallback silently on auth_sys if krb5i fails.  Is that right?
>>
>> In that case I don't see the point of the krb5i any more: any attacker
>> that can spoof rpc replies can force the fallback to auth_sys.
> 
> The point is to use a consistent security flavor for lease management to avoid 
> NFS4ERR_CLID_INUSE.  This really isn't about thwarting a MiTM attack on lease 
> management.
Again, its not the technology I'm arguing because its good stuff... I would
just like a way to manage it.
 
> 
> The fallback mechanism can be fixed, somewhat.  I've got a patch to have gssd return 
> ENOKEY if it can't find a machine credential.  Then the kernel will use AUTH_SYS.
We should not have to start a daemon to do a normal NFS client mount. That
is just not a good thing... IMHO... 

> 
> The issue though is what to do in the other cases.  Can gssd distinguish between the 
> case where the server has no Kerberos principal, and the case where gssd simply failed to 
> establish a GSS context?  Or should we simply use the "sec=" setting in this 
> case and call it a day?
Good questions.... IDK... I see this is another justification for use to
have away to manage this code until these questions are answered.

steved.
 
> 
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:19 p.m. UTC | #21

On 08/11/13 11:17, J. Bruce Fields wrote:
> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
>>
>>
>> On 08/11/13 10:12, Jeff Layton wrote:
>>> On Fri, 08 Nov 2013 10:00:02 -0500
>>> Steve Dickson <[e-mail]> wrote:
>>>
>>>>
>>>>
>>>> On 08/11/13 08:22, Jeff Layton wrote:
>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>>>
>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>>>
>>>>>>>> Hey mrchuck... 
>>>>>>>>
>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>>>> Hi Steve-
>>>>>>>>>
>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>
>>>>>>>>>> This new module parameter makes the v4 client
>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>>>> pseudoroot lookup
>>>>>>>>>
>>>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>>>
>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>>>
>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>>>> and there is no way to turn it off.
>>>>>>   
>>>>>
>>>>> I don't think that's really the case. The idea was to have the client
>>>>> attempt to use krb5i if it's available, and then to fall back to
>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>>>> timeout when the daemon isn't running.
>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>>>
>>>
>>> That I don't know...sorry...
>> Then what is the justification to take all these extra steps
>> there they going to fail %100 of the time??
> 
> Any server can support krb5 for state setup and pseudoroot operations if
> it's configured.  This isn't a problem.
Would is this done on a Linux server? Is there a wiki?

steved.

> 
> --b.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 4:22 p.m. UTC | #22

On Fri, Nov 08, 2013 at 11:19:45AM -0500, Steve Dickson wrote:
> 
> 
> On 08/11/13 11:17, J. Bruce Fields wrote:
> > On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
> >>
> >>
> >> On 08/11/13 10:12, Jeff Layton wrote:
> >>> On Fri, 08 Nov 2013 10:00:02 -0500
> >>> Steve Dickson <[e-mail]> wrote:
> >>>
> >>>>
> >>>>
> >>>> On 08/11/13 08:22, Jeff Layton wrote:
> >>>>> On Fri, 08 Nov 2013 07:41:32 -0500
> >>>>> Steve Dickson <[e-mail]> wrote:
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 07/11/13 18:05, Chuck Lever wrote:
> >>>>>>>
> >>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
> >>>>>>>
> >>>>>>>> Hey mrchuck... 
> >>>>>>>>
> >>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
> >>>>>>>>> Hi Steve-
> >>>>>>>>>
> >>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> >>>>>>>>>
> >>>>>>>>>> This new module parameter makes the v4 client
> >>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
> >>>>>>>>>> when establishing NFSV4 state and doing the
> >>>>>>>>>> pseudoroot lookup
> >>>>>>>>>
> >>>>>>>>> The patch description doesn't say, but is this change to work 
> >>>>>>>>> around the 15 second GSSD upcall timeout? 
> >>>>>>>> Yes. A 15 second delay on every mount due to security that
> >>>>>>>> nobody is requesting is just not good.. IMHO..
> >>>>>>>
> >>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
> >>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
> >>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
> >>>>>>> environment to take 15 seconds to establish a GSS context?).
> >>>>>>>
> >>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
> >>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
> >>>>>> No. I think the concern here, at least my concern, is the lack of management.
> >>>>>> We are forcing admins to use krb5i in lease management when its not necessary
> >>>>>> and there is no way to turn it off.
> >>>>>>   
> >>>>>
> >>>>> I don't think that's really the case. The idea was to have the client
> >>>>> attempt to use krb5i if it's available, and then to fall back to
> >>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
> >>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
> >>>>> timeout when the daemon isn't running.
> >>>> What server makes krb5i available today in state setup and pseudoroot lookups?
> >>>>
> >>>
> >>> That I don't know...sorry...
> >> Then what is the justification to take all these extra steps
> >> there they going to fail %100 of the time??
> > 
> > Any server can support krb5 for state setup and pseudoroot operations if
> > it's configured.  This isn't a problem.
> Would is this done on a Linux server? Is there a wiki?

It's allowed by default, there should be nothing to configure beyond the
usual krb5 setup.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Weston Andros Adamson Nov. 8, 2013, 4:27 p.m. UTC | #23

On Nov 8, 2013, at 10:00 AM, Steve Dickson <[e-mail]> wrote:

> What server makes krb5i available today in state setup and pseudoroot lookups?

Linux nfsd, among others…

The real issue I see here is what Trond was mentioning earlier - the order of multiple mounts of the same server matters, i.e.:

1) mount sec=krb5i server:/foo /mnt1
2) mount sec=sys server:/foo /mnt2

This leads to the state operations to server using krb5i, but:

1) mount sec=sys server:/foo /mnt2
2) mount sec=krb5i server:/foo /mnt1

this leads to the state operations to server using AUTH_SYS. yuck.

I don’t think we can just upgrade the state connection from AUTH_SYS to krb5i when this happens, that is why we try krb5i first, then fall back to AUTH_SYS.

-dros 

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:28 p.m. UTC | #24

On 08/11/13 11:22, J. Bruce Fields wrote:
> On Fri, Nov 08, 2013 at 11:19:45AM -0500, Steve Dickson wrote:
>>
>>
>> On 08/11/13 11:17, J. Bruce Fields wrote:
>>> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
>>>>
>>>>
>>>> On 08/11/13 10:12, Jeff Layton wrote:
>>>>> On Fri, 08 Nov 2013 10:00:02 -0500
>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 08/11/13 08:22, Jeff Layton wrote:
>>>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>>>>>
>>>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>
>>>>>>>>>> Hey mrchuck... 
>>>>>>>>>>
>>>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>>>>>> Hi Steve-
>>>>>>>>>>>
>>>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> This new module parameter makes the v4 client
>>>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>>>>>> pseudoroot lookup
>>>>>>>>>>>
>>>>>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>>>>>
>>>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>>>>>
>>>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>>>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>>>>>> and there is no way to turn it off.
>>>>>>>>   
>>>>>>>
>>>>>>> I don't think that's really the case. The idea was to have the client
>>>>>>> attempt to use krb5i if it's available, and then to fall back to
>>>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>>>>>> timeout when the daemon isn't running.
>>>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>>>>>
>>>>>
>>>>> That I don't know...sorry...
>>>> Then what is the justification to take all these extra steps
>>>> there they going to fail %100 of the time??
>>>
>>> Any server can support krb5 for state setup and pseudoroot operations if
>>> it's configured.  This isn't a problem.
>> Would is this done on a Linux server? Is there a wiki?
> 
> It's allowed by default, there should be nothing to configure beyond the
> usual krb5 setup.
Great! So you are saying when rpc.gssd is up and  Kerberos is correctly 
configured on both the server and client the state setup and pseudoroot
become secured? 

And this is the case with other non-Linux servers? 

steved.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:31 p.m. UTC | #25

On 08/11/13 10:16, Myklebust, Trond wrote:
> 
> On Nov 8, 2013, at 10:08, Steve Dickson <[e-mail]> wrote:
> 
>>
>>
>> On 08/11/13 09:30, Myklebust, Trond wrote:
>>>
>>> On Nov 8, 2013, at 7:21, Steve Dickson <[e-mail]> wrote:
>>>
>>>>
>>>>
>>>> On 07/11/13 17:29, Myklebust, Trond wrote:
>>>>>> Again, what servers, today, support this type of secure state establishment?
>>>>>>> Having this type of security in the client I think is good... but if
>>>>>>> the client is not talking with any servers that support this type 
>>>>>>> of security, why not have a way to turn it off?
>>>>> I don't understand. Servers are _required_ to support RPCSEC_GSS with
>>>>> krb5 by both RFC3530 and RFC5661. AUTH_SYS is, in fact, the optional
>>>>> flavour.
>>>> Agreed... 100% of the NFSv4 server have to support RPCSEC_GSS. its mandated
>>>> by the spec(s). 
>>>>
>>>>>
>>>>> The problem here is that sometimes kerberos isn't configured by the
>>>>> admin, who then expects that it shouldn't be necessary to run rpc.gssd
>>>>> or rpc.svcgssd. It is necessary because we first try the
>>>>> mandatory-to-implement and secure RPCSEC_GSS/krb5i flavour before
>>>>> falling back to the less secure AUTH_SYS...
>>>> Sometimes? Its generally not.. from my experience... 
>>>>
>>>> Basically how I interpret this last paragraph, is we will be requiring 
>>>> admins set up secure mounts for them to avoid the 15sec delay mount
>>>> times... aka... running a daemon that will say "no, no there is no 
>>>> security here" while spewing of log messages when Kerberos is not setup...
>>>
>>> No. All we are requiring is that they run rpc.gssd.
>> Even when they do not want any secure mounts at all?
>>
>> What is that justification?
> 
> They get to skip a 15 second wait without having to blacklist the krb5 module.
> 
> So, this is what I mean when I say that we _might_ need a mount option to 
> specify the security flavour that the client uses for the lease. It solves the 
> problem that Bruce mentioned about what to do when krb5i fails, and it allows admins 
> to not have to run rpc.gssd or blacklist any modules.
Fine... Would mind doing a brain dump of what this mount might look like
and how it work in a different thread... Since this is pretty beaten down
at this point... ;-) 

steved.

> 
> Trond
> 
> --
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> [e-mail]
> www.netapp.com
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:38 p.m. UTC | #26

On 08/11/13 11:27, Weston Andros Adamson wrote:
> 
> On Nov 8, 2013, at 10:00 AM, Steve Dickson <[e-mail]> wrote:
> 
>> What server makes krb5i available today in state setup and pseudoroot lookups?
> 
> Linux nfsd, among others…
> 
> The real issue I see here is what Trond was mentioning earlier - the order of multiple mounts of the same server matters, i.e.:
> 
> 1) mount sec=krb5i server:/foo /mnt1
> 2) mount sec=sys server:/foo /mnt2
> 
> This leads to the state operations to server using krb5i, but:
> 
> 1) mount sec=sys server:/foo /mnt2
> 2) mount sec=krb5i server:/foo /mnt1
> 
> this leads to the state operations to server using AUTH_SYS. yuck.
> 
> I don’t think we can just upgrade the state connection from AUTH_SYS to krb5i 
> when this happens, that is why we try krb5i first, then fall back to AUTH_SYS.

Excellent explanation! Thanks you! But... ;-) This assumes the admin is
actually trying to krb5i which means he/she has set up a functioning 
Kerberos environment. But we can't assume every client has a valid 
Kerberos environment, which is what the code is doing today!

steved.

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 4:39 p.m. UTC | #27

On Fri, Nov 08, 2013 at 11:28:02AM -0500, Steve Dickson wrote:
> 
> 
> On 08/11/13 11:22, J. Bruce Fields wrote:
> > On Fri, Nov 08, 2013 at 11:19:45AM -0500, Steve Dickson wrote:
> >>
> >>
> >> On 08/11/13 11:17, J. Bruce Fields wrote:
> >>> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
> >>>>
> >>>>
> >>>> On 08/11/13 10:12, Jeff Layton wrote:
> >>>>> On Fri, 08 Nov 2013 10:00:02 -0500
> >>>>> Steve Dickson <[e-mail]> wrote:
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 08/11/13 08:22, Jeff Layton wrote:
> >>>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
> >>>>>>> Steve Dickson <[e-mail]> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
> >>>>>>>>>
> >>>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hey mrchuck... 
> >>>>>>>>>>
> >>>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
> >>>>>>>>>>> Hi Steve-
> >>>>>>>>>>>
> >>>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> This new module parameter makes the v4 client
> >>>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
> >>>>>>>>>>>> when establishing NFSV4 state and doing the
> >>>>>>>>>>>> pseudoroot lookup
> >>>>>>>>>>>
> >>>>>>>>>>> The patch description doesn't say, but is this change to work 
> >>>>>>>>>>> around the 15 second GSSD upcall timeout? 
> >>>>>>>>>> Yes. A 15 second delay on every mount due to security that
> >>>>>>>>>> nobody is requesting is just not good.. IMHO..
> >>>>>>>>>
> >>>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
> >>>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
> >>>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
> >>>>>>>>> environment to take 15 seconds to establish a GSS context?).
> >>>>>>>>>
> >>>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
> >>>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
> >>>>>>>> No. I think the concern here, at least my concern, is the lack of management.
> >>>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
> >>>>>>>> and there is no way to turn it off.
> >>>>>>>>   
> >>>>>>>
> >>>>>>> I don't think that's really the case. The idea was to have the client
> >>>>>>> attempt to use krb5i if it's available, and then to fall back to
> >>>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
> >>>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
> >>>>>>> timeout when the daemon isn't running.
> >>>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
> >>>>>>
> >>>>>
> >>>>> That I don't know...sorry...
> >>>> Then what is the justification to take all these extra steps
> >>>> there they going to fail %100 of the time??
> >>>
> >>> Any server can support krb5 for state setup and pseudoroot operations if
> >>> it's configured.  This isn't a problem.
> >> Would is this done on a Linux server? Is there a wiki?
> > 
> > It's allowed by default, there should be nothing to configure beyond the
> > usual krb5 setup.
> Great! So you are saying when rpc.gssd is up and  Kerberos is correctly 
> configured on both the server and client the state setup and pseudoroot
> become secured? 

Yes.

There's one possible exception: I think mountd currently may not allow
krb5 on the pseudoroot if it's not allowed on some export?

We could fix that if it's a problem here.  (I don't think it is, because
a client mounting with auth_sys will fall back on auth_sys in this case,
and a client mounting with sec=krb5 is going to fail eventually anyway.)

The server will allow state setup regardless.

> And this is the case with other non-Linux servers? 

Yes, the client's always used krb5 for all that setup at least in the
case where the first mount uses sec=krb5.  And we test that at
connectathon/bakeathon, and it's probably one of the first things anyone
writing new gss support for a server would try.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 4:45 p.m. UTC | #28

On 08/11/13 11:39, J. Bruce Fields wrote:
> On Fri, Nov 08, 2013 at 11:28:02AM -0500, Steve Dickson wrote:
>>
>>
>> On 08/11/13 11:22, J. Bruce Fields wrote:
>>> On Fri, Nov 08, 2013 at 11:19:45AM -0500, Steve Dickson wrote:
>>>>
>>>>
>>>> On 08/11/13 11:17, J. Bruce Fields wrote:
>>>>> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
>>>>>>
>>>>>>
>>>>>> On 08/11/13 10:12, Jeff Layton wrote:
>>>>>>> On Fri, 08 Nov 2013 10:00:02 -0500
>>>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 08/11/13 08:22, Jeff Layton wrote:
>>>>>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>>>>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey mrchuck... 
>>>>>>>>>>>>
>>>>>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>>>>>>>> Hi Steve-
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> This new module parameter makes the v4 client
>>>>>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>>>>>>>> pseudoroot lookup
>>>>>>>>>>>>>
>>>>>>>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>>>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>>>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>>>>>>>
>>>>>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>>>>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>>>>>>>
>>>>>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>>>>>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>>>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>>>>>>>> and there is no way to turn it off.
>>>>>>>>>>   
>>>>>>>>>
>>>>>>>>> I don't think that's really the case. The idea was to have the client
>>>>>>>>> attempt to use krb5i if it's available, and then to fall back to
>>>>>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>>>>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>>>>>>>> timeout when the daemon isn't running.
>>>>>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>>>>>>>
>>>>>>>
>>>>>>> That I don't know...sorry...
>>>>>> Then what is the justification to take all these extra steps
>>>>>> there they going to fail %100 of the time??
>>>>>
>>>>> Any server can support krb5 for state setup and pseudoroot operations if
>>>>> it's configured.  This isn't a problem.
>>>> Would is this done on a Linux server? Is there a wiki?
>>>
>>> It's allowed by default, there should be nothing to configure beyond the
>>> usual krb5 setup.
>> Great! So you are saying when rpc.gssd is up and  Kerberos is correctly 
>> configured on both the server and client the state setup and pseudoroot
>> become secured? 
> 
> Yes.
> 
> There's one possible exception: I think mountd currently may not allow
> krb5 on the pseudoroot if it's not allowed on some export?
> 
> We could fix that if it's a problem here.  (I don't think it is, because
> a client mounting with auth_sys will fall back on auth_sys in this case,
> and a client mounting with sec=krb5 is going to fail eventually anyway.)
> 
> The server will allow state setup regardless.
This is good to know... Thanks!

> 
>> And this is the case with other non-Linux servers? 
> 
> Yes, the client's always used krb5 for all that setup at least in the
> case where the first mount uses sec=krb5.  And we test that at
> connectathon/bakeathon, and it's probably one of the first things anyone
> writing new gss support for a server would try.
It would be good to get all this documented... If its not already...

steved.

> 
> --b.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Chuck Lever III Nov. 8, 2013, 5:58 p.m. UTC | #29

On Nov 8, 2013, at 8:14 AM, "J. Bruce Fields" <[e-mail]> wrote:

> On Fri, Nov 08, 2013 at 07:54:19AM -0800, Chuck Lever wrote:
>> 
>> On Nov 8, 2013, at 7:04 AM, J. Bruce Fields <[e-mail]>
>> wrote:
>> 
>>> On Fri, Nov 08, 2013 at 08:22:02AM -0500, Jeff Layton wrote:
>>>> On Fri, 08 Nov 2013 07:41:32 -0500 Steve Dickson
>>>> <[e-mail]> wrote:
>>>>> No. I think the concern here, at least my concern, is the lack of
>>>>> management.  We are forcing admins to use krb5i in lease
>>>>> management when its not necessary and there is no way to turn it
>>>>> off.
>>>>> 
>>>> 
>>>> I don't think that's really the case. The idea was to have the
>>>> client attempt to use krb5i if it's available, and then to fall
>>>> back to AUTH_SYS if it isn't. This would be *absolutely* no big
>>>> deal if the GSSAPI upcall succeeded or failed immediately instead
>>>> of requiring this timeout when the daemon isn't running.
>>> 
>>> I'm also still a little confused about the security model.  We
>>> discussed it before but I can't remember if it was really resolved.
>>> 
>>> It makes sense to me as long as we insist on krb5i whenever we find
>>> a keytab.
>>> 
>>> But my understanding was that with the current implementation it's
>>> possible we could find a keytab, attempt the krb5i connection, and
>>> *then* fallback silently on auth_sys if krb5i fails.  Is that right?
>>> 
>>> In that case I don't see the point of the krb5i any more: any
>>> attacker that can spoof rpc replies can force the fallback to
>>> auth_sys.
>> 
>> The point is to use a consistent security flavor for lease management
>> to avoid NFS4ERR_CLID_INUSE.  This really isn't about thwarting a MiTM
>> attack on lease management.
> 
> OK, but then you still do want to ensure you thwart that attack in the
> case krb5 is explicitly asked for.
> 
> So for example does a krb5 mount fail if a previous non-krb5 mount fell
> back on auth_sys for clientid establishment?
> 
>> The fallback mechanism can be fixed, somewhat.  I've got a patch to
>> have gssd return ENOKEY if it can't find a machine credential.  Then
>> the kernel will use AUTH_SYS.
> 
> To get that right you'd then need to fail on errors *other* than ENOKEY.
> Since old gssd doesn't return ENOKEY that would mean failing all
> auth_sys mounts on kernel upgrade, wouldn't it?
> 
>> The issue though is what to do in the other cases.  Can gssd
>> distinguish between the case where the server has no Kerberos
>> principal, and the case where gssd simply failed to establish a GSS
>> context?
> 
> I don't know.  I suppose the client should get some kind of
> no-such-principal error and use that.
> 
>> Or should we simply use the "sec=" setting in this case and
>> call it a day?
> 
> I think falling back works as long as we still are strict in the case
> the mount explicitly requests some security and don't just use
> whatever's already established in that case.

Agreed.  That is what we want to strive for.

So: use the fallback mechanism only if Kerberos was _not_ specified via the mount options, I think?

> And if we do that then I don't think the ENOKEY/no-such-principal cases
> are worth sorting out.
> 
> OK, sorry for the noise.
> 
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Chuck Lever III Nov. 8, 2013, 6:09 p.m. UTC | #30

On Nov 8, 2013, at 8:17 AM, "J. Bruce Fields" <[e-mail]> wrote:

> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
>> 
>> 
>> On 08/11/13 10:12, Jeff Layton wrote:
>>> On Fri, 08 Nov 2013 10:00:02 -0500
>>> Steve Dickson <[e-mail]> wrote:
>>> 
>>>> 
>>>> 
>>>> On 08/11/13 08:22, Jeff Layton wrote:
>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>>>> Steve Dickson <[e-mail]> wrote:
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>>> 
>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>>> 
>>>>>>>> Hey mrchuck... 
>>>>>>>> 
>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>>>> Hi Steve-
>>>>>>>>> 
>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>> 
>>>>>>>>>> This new module parameter makes the v4 client
>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>>>> pseudoroot lookup
>>>>>>>>> 
>>>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>>> 
>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>>> 
>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>>>> and there is no way to turn it off.
>>>>>> 
>>>>> 
>>>>> I don't think that's really the case. The idea was to have the client
>>>>> attempt to use krb5i if it's available, and then to fall back to
>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>>>> timeout when the daemon isn't running.
>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>>> 
>>> 
>>> That I don't know...sorry...
>> Then what is the justification to take all these extra steps
>> there they going to fail %100 of the time??
> 
> Any server can support krb5 for state setup and pseudoroot operations if
> it's configured.  This isn't a problem.

I let this pass earlier, but...

The krb5i setting is _ONLY_ for lease management, not for data access.  Traversing the pseudo-fs counts as data access.  Our client is supposed to use the security flavor specified on the mount command line for the pseudo-fs.  (That's why the pseudo-fs security policy is the union of all the real exports on the server, right?)

If no flavor is specified by the client administrator, we have SECINFO_NONAME for negotiating the pseudo-fs security flavor in NFSv4.1, and some roughly equivalent heuristics for this in NFSv4.0, which doesn't have the SECINFO_NONAME operation.  Since 3.11, I believe, our client should be using these mechanisms instead of just plowing ahead with AUTH_SYS.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Chuck Lever III Nov. 8, 2013, 6:12 p.m. UTC | #31

On Nov 8, 2013, at 8:39 AM, "J. Bruce Fields" <[e-mail]> wrote:

> On Fri, Nov 08, 2013 at 11:28:02AM -0500, Steve Dickson wrote:
>> 
>> 
>> On 08/11/13 11:22, J. Bruce Fields wrote:
>>> On Fri, Nov 08, 2013 at 11:19:45AM -0500, Steve Dickson wrote:
>>>> 
>>>> 
>>>> On 08/11/13 11:17, J. Bruce Fields wrote:
>>>>> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
>>>>>> 
>>>>>> 
>>>>>> On 08/11/13 10:12, Jeff Layton wrote:
>>>>>>> On Fri, 08 Nov 2013 10:00:02 -0500
>>>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 08/11/13 08:22, Jeff Layton wrote:
>>>>>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>>>>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>>>>>>> 
>>>>>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hey mrchuck... 
>>>>>>>>>>>> 
>>>>>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>>>>>>>> Hi Steve-
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> This new module parameter makes the v4 client
>>>>>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>>>>>>>> pseudoroot lookup
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>>>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>>>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>>>>>>> 
>>>>>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>>>>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>>>>>>> 
>>>>>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>>>>>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>>>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>>>>>>>> and there is no way to turn it off.
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> I don't think that's really the case. The idea was to have the client
>>>>>>>>> attempt to use krb5i if it's available, and then to fall back to
>>>>>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>>>>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>>>>>>>> timeout when the daemon isn't running.
>>>>>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>>>>>>> 
>>>>>>> 
>>>>>>> That I don't know...sorry...
>>>>>> Then what is the justification to take all these extra steps
>>>>>> there they going to fail %100 of the time??
>>>>> 
>>>>> Any server can support krb5 for state setup and pseudoroot operations if
>>>>> it's configured.  This isn't a problem.
>>>> Would is this done on a Linux server? Is there a wiki?
>>> 
>>> It's allowed by default, there should be nothing to configure beyond the
>>> usual krb5 setup.
>> Great! So you are saying when rpc.gssd is up and  Kerberos is correctly 
>> configured on both the server and client the state setup and pseudoroot
>> become secured? 
> 
> Yes.
> 
> There's one possible exception: I think mountd currently may not allow
> krb5 on the pseudoroot if it's not allowed on some export?

See earlier e-mail:  pseudo-fs access is done using the mount's specified security flavor, not by the lease management security flavor.

> We could fix that if it's a problem here.  (I don't think it is, because
> a client mounting with auth_sys will fall back on auth_sys in this case,
> and a client mounting with sec=krb5 is going to fail eventually anyway.)
> 
> The server will allow state setup regardless.

Correct.

>> And this is the case with other non-Linux servers? 
> 
> Yes, the client's always used krb5 for all that setup at least in the
> case where the first mount uses sec=krb5.  And we test that at
> connectathon/bakeathon, and it's probably one of the first things anyone
> writing new gss support for a server would try.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Chuck Lever III Nov. 8, 2013, 6:46 p.m. UTC | #32

On Nov 8, 2013, at 9:58 AM, Chuck Lever <[e-mail]> wrote:

> 
> On Nov 8, 2013, at 8:14 AM, "J. Bruce Fields" <[e-mail]> wrote:
> 
>> On Fri, Nov 08, 2013 at 07:54:19AM -0800, Chuck Lever wrote:
>>> 
>>> On Nov 8, 2013, at 7:04 AM, J. Bruce Fields <[e-mail]>
>>> wrote:
>>> 
>>>> On Fri, Nov 08, 2013 at 08:22:02AM -0500, Jeff Layton wrote:
>>>>> On Fri, 08 Nov 2013 07:41:32 -0500 Steve Dickson
>>>>> <[e-mail]> wrote:
>>>>>> No. I think the concern here, at least my concern, is the lack of
>>>>>> management.  We are forcing admins to use krb5i in lease
>>>>>> management when its not necessary and there is no way to turn it
>>>>>> off.
>>>>>> 
>>>>> 
>>>>> I don't think that's really the case. The idea was to have the
>>>>> client attempt to use krb5i if it's available, and then to fall
>>>>> back to AUTH_SYS if it isn't. This would be *absolutely* no big
>>>>> deal if the GSSAPI upcall succeeded or failed immediately instead
>>>>> of requiring this timeout when the daemon isn't running.
>>>> 
>>>> I'm also still a little confused about the security model.  We
>>>> discussed it before but I can't remember if it was really resolved.
>>>> 
>>>> It makes sense to me as long as we insist on krb5i whenever we find
>>>> a keytab.
>>>> 
>>>> But my understanding was that with the current implementation it's
>>>> possible we could find a keytab, attempt the krb5i connection, and
>>>> *then* fallback silently on auth_sys if krb5i fails.  Is that right?
>>>> 
>>>> In that case I don't see the point of the krb5i any more: any
>>>> attacker that can spoof rpc replies can force the fallback to
>>>> auth_sys.
>>> 
>>> The point is to use a consistent security flavor for lease management
>>> to avoid NFS4ERR_CLID_INUSE.  This really isn't about thwarting a MiTM
>>> attack on lease management.
>> 
>> OK, but then you still do want to ensure you thwart that attack in the
>> case krb5 is explicitly asked for.
>> 
>> So for example does a krb5 mount fail if a previous non-krb5 mount fell
>> back on auth_sys for clientid establishment?
>> 
>>> The fallback mechanism can be fixed, somewhat.  I've got a patch to
>>> have gssd return ENOKEY if it can't find a machine credential.  Then
>>> the kernel will use AUTH_SYS.
>> 
>> To get that right you'd then need to fail on errors *other* than ENOKEY.
>> Since old gssd doesn't return ENOKEY that would mean failing all
>> auth_sys mounts on kernel upgrade, wouldn't it?
>> 
>>> The issue though is what to do in the other cases.  Can gssd
>>> distinguish between the case where the server has no Kerberos
>>> principal, and the case where gssd simply failed to establish a GSS
>>> context?
>> 
>> I don't know.  I suppose the client should get some kind of
>> no-such-principal error and use that.
>> 
>>> Or should we simply use the "sec=" setting in this case and
>>> call it a day?
>> 
>> I think falling back works as long as we still are strict in the case
>> the mount explicitly requests some security and don't just use
>> whatever's already established in that case.
> 
> Agreed.  That is what we want to strive for.
> 
> So: use the fallback mechanism only if Kerberos was _not_ specified via the mount options, I think?
> 
>> And if we do that then I don't think the ENOKEY/no-such-principal cases
>> are worth sorting out.

The fly in this ointment is allowing clients with no keytab to mount with sec=krb5.

We can use ENOKEY to allow lease management with AUTH_SYS but data access using Kerberos and a user's credential.

Otherwise, a user has to login as root, kinit as themselves, and then mount.  That makes automounter configurations a little dodgy.


--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 8:14 p.m. UTC | #33

On Fri, Nov 08, 2013 at 10:09:18AM -0800, Chuck Lever wrote:
> I let this pass earlier, but...
> 
> The krb5i setting is _ONLY_ for lease management, not for data access.  Traversing the pseudo-fs counts as data access.  Our client is supposed to use the security flavor specified on the mount command line for the pseudo-fs.  (That's why the pseudo-fs security policy is the union of all the real exports on the server, right?)
> 
> If no flavor is specified by the client administrator, we have SECINFO_NONAME for negotiating the pseudo-fs security flavor in NFSv4.1, and some roughly equivalent heuristics for this in NFSv4.0, which doesn't have the SECINFO_NONAME operation.  Since 3.11, I believe, our client should be using these mechanisms instead of just plowing ahead with AUTH_SYS.

That makes sense.

(By the way: if a mount is done with krb5*, are we guaranteed the entire
export path is looked up with security at least as strong, or is it
possible we trust cached lookups possibly originally obtained with
weaker security?)

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 8:32 p.m. UTC | #34

On 08/11/13 11:17, J. Bruce Fields wrote:
> On Fri, Nov 08, 2013 at 11:10:14AM -0500, Steve Dickson wrote:
>>
>>
>> On 08/11/13 10:12, Jeff Layton wrote:
>>> On Fri, 08 Nov 2013 10:00:02 -0500
>>> Steve Dickson <[e-mail]> wrote:
>>>
>>>>
>>>>
>>>> On 08/11/13 08:22, Jeff Layton wrote:
>>>>> On Fri, 08 Nov 2013 07:41:32 -0500
>>>>> Steve Dickson <[e-mail]> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 07/11/13 18:05, Chuck Lever wrote:
>>>>>>>
>>>>>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>>>>>
>>>>>>>> Hey mrchuck... 
>>>>>>>>
>>>>>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>>>>>> Hi Steve-
>>>>>>>>>
>>>>>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>>>>>
>>>>>>>>>> This new module parameter makes the v4 client
>>>>>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>>>>>> when establishing NFSV4 state and doing the
>>>>>>>>>> pseudoroot lookup
>>>>>>>>>
>>>>>>>>> The patch description doesn't say, but is this change to work 
>>>>>>>>> around the 15 second GSSD upcall timeout? 
>>>>>>>> Yes. A 15 second delay on every mount due to security that
>>>>>>>> nobody is requesting is just not good.. IMHO..
>>>>>>>
>>>>>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>>>>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>>>>>> you expect the mount to work (ie why would it be good for any properly configured 
>>>>>>> environment to take 15 seconds to establish a GSS context?).
>>>>>>>
>>>>>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>>>>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>>>>>> No. I think the concern here, at least my concern, is the lack of management.
>>>>>> We are forcing admins to use krb5i in lease management when its not necessary
>>>>>> and there is no way to turn it off.
>>>>>>   
>>>>>
>>>>> I don't think that's really the case. The idea was to have the client
>>>>> attempt to use krb5i if it's available, and then to fall back to
>>>>> AUTH_SYS if it isn't. This would be *absolutely* no big deal if the
>>>>> GSSAPI upcall succeeded or failed immediately instead of requiring this
>>>>> timeout when the daemon isn't running.
>>>> What server makes krb5i available today in state setup and pseudoroot lookups?
>>>>
>>>
>>> That I don't know...sorry...
>> Then what is the justification to take all these extra steps
>> there they going to fail %100 of the time??
> 
> Any server can support krb5 for state setup and pseudoroot operations if
> it's configured.  This isn't a problem.
Thank you for point this out... That does change things... IMHO...

steved.

> 
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

J. Bruce Fields Nov. 8, 2013, 9:09 p.m. UTC | #35

On Fri, Nov 08, 2013 at 10:46:26AM -0800, Chuck Lever wrote:
> The fly in this ointment is allowing clients with no keytab to mount
> with sec=krb5.
> 
> We can use ENOKEY to allow lease management with AUTH_SYS but data
> access using Kerberos and a user's credential.

So "mount -osec=krb5*" means "use krb5* or stronger for everything",
unless you have no keytab in which case it means "use krb5* for all file
access but allow auth_sys for lease maintenance".  Huh.  I guess that
works.

--b.

> 
> Otherwise, a user has to login as root, kinit as themselves, and then
> mount.  That makes automounter configurations a little dodgy.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Steve Dickson Nov. 8, 2013, 9:25 p.m. UTC | #36

Hey,

On 08/11/13 10:46, Chuck Lever wrote:
> 
> On Nov 8, 2013, at 4:41 AM, Steve Dickson <[e-mail]> wrote:
> 
>>
>>
>> On 07/11/13 18:05, Chuck Lever wrote:
>>>
>>> On Nov 7, 2013, at 1:35 PM, Steve Dickson <[e-mail]> wrote:
>>>
>>>> Hey mrchuck... 
>>>>
>>>> On 07/11/13 14:25, Chuck Lever wrote:
>>>>> Hi Steve-
>>>>>
>>>>> On Nov 7, 2013, at 11:09 AM, Steve Dickson <[e-mail]> wrote:
>>>>>
>>>>>> This new module parameter makes the v4 client
>>>>>> use the minimal authentication flavor (AUTH_UNIX)
>>>>>> when establishing NFSV4 state and doing the
>>>>>> pseudoroot lookup
>>>>>
>>>>> The patch description doesn't say, but is this change to work 
>>>>> around the 15 second GSSD upcall timeout? 
>>>> Yes. A 15 second delay on every mount due to security that
>>>> nobody is requesting is just not good.. IMHO..
>>>
>>> One thing we haven't discussed is reducing the upcall timeout to 5 seconds or less, 
>>> as a form of immediate relief.  15 seconds is arbitrary, and is onerous even when 
>>> you expect the mount to work (ie why would it be good for any properly configured 
>>> environment to take 15 seconds to establish a GSS context?).
>>>
>>> In other words, there are still cases where users wait 15 seconds unnecessarily, 
>>> and not because of the use of krb5i for lease management.  Aren't those of concern?
>> No. I think the concern here, at least my concern, is the lack of management.
>> We are forcing admins to use krb5i in lease management when its not necessary
>> and there is no way to turn it off.
> 
> Before, we were "forcing" admins to use AUTH_SYS on some mounts even 
> though they had chosen to use sec=krb5 on some mounts.  How is that a good thing?
It was not, but it was a small window of where AUTH_SYS was used then
krb5* took over... I think that was better than a 15 sec delay.

> 
> Now, we are "forcing" admins to use security, but only when it is available.  
> In other words, our default now is to be more secure rather than insecure.  Why is that a bad thing?
Again its not... But if the technology is not there to make it a
seamless transition then we should wait until the transition is  seamless 
before we force things on admins... IMHO..
 
> 
> The IETF has now reached consensus to require security in every new 
> protocol it specifies.  They want to enable security and not give ways to turn it off.
> They are even considering re-specifying existing widely deployed protocols to increase 
> their security.
I'll refer to my previous comments... When it becomes seamless... its good to go... 

Remember when SELinux first came out? How it caused everything to fail, especially
on the server side? The first they did was to give us a way to turn in off. How
many times did you use setenforce 0 ? ;-) Heck, I think I'm the reason they
came up with that command! 8-) 
 
> 
> "it's not necessary" is no longer an excuse not to use security. Read the minutes of
> Wednesday's IAB technical plenary if you doubt me (yes, someone actually said that during t
> he open mic, and there was some applause).
Dictating or taking away choice on how people can configure their environment 
is not a good thing IMHO... One size security does not fit all...
 
> 
> Earlier Trond noted that NFSv4 REQUIRES implementations to support RPCSEC GSS.  That 
> requirement has been there since RFC 3010.  There is a reason for that.
Yes it is a requirement, which was a good thing, but admins had a *choice* 
on whether they wanted to use it... I disagree with taking that choice away.

> 
> The only reason you are arguing to disable security here is because it is a mild annoyance.
Mild? You are saying a 15 second delay on every mount is mild? :-) Boy I would
hate to see you though disruptive was... ;-)

> And instead of addressing the annoyance, you want to allow users to disable security.
Yes, I want to give admins the same choice they the have always had.. On or off.
and that is bad? 

> 
> I'd like a solution that chooses to use security when its available, rather than 
> blindly sticking with an insecure default.  I'd like a solution that makes that choice 
> automatically without adding yet another obscure setting to our panoply of obscure settings.
> It should "just work."
If it did "just work" fine.. but it does not. Why not give a bridge to people until 
we get things to "just work"?

> 
> We are "this close" to getting that.  Let's not take a step backwards.
But we are not there yet... So sometimes you have to take one step backwards to 
take two step forwards as the old saying goes... 

steved.

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

NeilBrown Nov. 9, 2013, 2:04 a.m. UTC | #37

On Fri, 8 Nov 2013 10:12:32 -0500 Jeff Layton <[e-mail]> wrote:


> Right, but we've defaulted to v4 for *years* now. Most people run statd
> and don't actually use it. No one seems to get bent out of shape about
> that though.

Given that statd can be started on demand by mount.nfs, this sounds like a
configuration error.


NeilBrown

Patch

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 28842ab..20bf925 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -438,6 +438,7 @@  extern bool nfs4_disable_idmapping;
 extern unsigned short max_session_slots;
 extern unsigned short send_implementation_id;
 extern bool recover_lost_locks;
+extern bool nfs4_use_min_auth;
 
 #define NFS4_CLIENT_ID_UNIQ_LEN		(64)
 extern char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN];
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index a860ab5..ff85991 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -355,6 +355,7 @@  struct nfs_client *nfs4_init_client(struct nfs_client *clp,
 	char buf[INET6_ADDRSTRLEN + 1];
 	struct nfs_client *old;
 	int error;
+	rpc_authflavor_t flavor = RPC_AUTH_GSS_KRB5I;
 
 	if (clp->cl_cons_state == NFS_CS_READY) {
 		/* the client is initialised already */
@@ -368,8 +369,11 @@  struct nfs_client *nfs4_init_client(struct nfs_client *clp,
 	if (clp->cl_minorversion != 0)
 		__set_bit(NFS_CS_INFINITE_SLOTS, &clp->cl_flags);
 	__set_bit(NFS_CS_DISCRTRY, &clp->cl_flags);
-	error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_GSS_KRB5I);
-	if (error == -EINVAL)
+
+	if (nfs4_use_min_auth)
+		flavor = RPC_AUTH_UNIX;
+	error = nfs_create_rpc_client(clp, timeparms, flavor);
+	if (error == -EINVAL && flavor != RPC_AUTH_UNIX)
 		error = nfs_create_rpc_client(clp, timeparms, RPC_AUTH_UNIX);
 	if (error < 0)
 		goto error;
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index d53d678..00162cb 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -2864,7 +2864,9 @@  static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
 	int status = -EPERM;
 	size_t i;
 
-	for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
+	if (nfs4_use_min_auth)
+		status = nfs4_lookup_root_sec(server, fhandle, info, RPC_AUTH_UNIX);
+	else for (i = 0; i < ARRAY_SIZE(flav_array); i++) {
 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
 			continue;
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index a03b9c6..42b4f9b 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -2791,6 +2791,7 @@  unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
 unsigned short send_implementation_id = 1;
 char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
 bool recover_lost_locks = false;
+bool nfs4_use_min_auth = false;
 
 EXPORT_SYMBOL_GPL(nfs_callback_set_tcpport);
 EXPORT_SYMBOL_GPL(nfs_callback_tcpport);
@@ -2800,6 +2801,7 @@  EXPORT_SYMBOL_GPL(max_session_slots);
 EXPORT_SYMBOL_GPL(send_implementation_id);
 EXPORT_SYMBOL_GPL(nfs4_client_id_uniquifier);
 EXPORT_SYMBOL_GPL(recover_lost_locks);
+EXPORT_SYMBOL_GPL(nfs4_use_min_auth);
 
 #define NFS_CALLBACK_MAXPORTNR (65535U)
 
@@ -2842,5 +2844,7 @@  MODULE_PARM_DESC(recover_lost_locks,
 		 "If the server reports that a lock might be lost, "
 		 "try to recover it risking data corruption.");
 
-
+module_param(nfs4_use_min_auth, bool, 0644);
+MODULE_PARM_DESC(nfs4_use_min_auth,
+		"Use mimnal auth in SETCLIENTID operation");
 #endif /* CONFIG_NFS_V4 */