diff mbox

[2/2] mount: RPC_PROGNOTREGISTERED should not be a permanent error

Message ID 87poll93s7.fsf@notabene.neil.brown.name (mailing list archive)
State New, archived
Headers show

Commit Message

NeilBrown Nov. 23, 2016, 11:26 p.m. UTC
On Thu, Nov 24 2016, Steve Dickson wrote:

> On 11/22/2016 05:43 PM, NeilBrown wrote:
>> On Wed, Nov 23 2016, Steve Dickson wrote:
>> 
>>> [Resent due to mailman rejecting the HTML subpart]
>> (and the resend included HTML too ... how embarrassing :-)
> Yeah... :-) I guess an upgrade turned it on.. 
>
>> 
>>>
>>> Hey Neil,
>>>
>>>
>>> On 08/18/2016 09:45 PM, NeilBrown wrote:
>>>> Commit: bf66c9facb8e ("mounts.nfs: v2 and v3 background mounts should retry when server is down.")
>>>>
>>>> changed the behaviour of "bg" mounts so that RPC_PROGNOTREGISTERED,
>>>> which maps to EOPNOTSUPP, is not a permanent error.
>>>> This useful because when an NFS server starts up there is a small window between
>>>> the moment that rpcbind (or portmap) starts responding to lookup requests,
>>>> and the moment when nfsd registers with rpcbind.  During that window
>>>> rpcbind will reply with RPC_PROGNOTREGISTERED, but mount should not give up.
>>>>
>>>> This same reasoning applies to foreground mounts.  They don't wait for
>>>> as long, but could still hit the window and fail prematurely.
>>>>
>>>> So revert the above patch and instead add EOPNOTSUPP to the list of
>>>> temporary errors known to nfs_is_permanent_error.
>>>>
>>>> Signed-off-by: NeilBrown <neilb@suse.com>
>>>> ---
>>>>  utils/mount/stropts.c |    7 +++----
>>>>  1 file changed, 3 insertions(+), 4 deletions(-)
>>>>
>>>> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c
>>>> index 9de6794c6177..d5dfb5e4a669 100644
>>>> --- a/utils/mount/stropts.c
>>>> +++ b/utils/mount/stropts.c
>>>> @@ -948,6 +948,7 @@ static int nfs_is_permanent_error(int error)
>>>>  	case ETIMEDOUT:
>>>>  	case ECONNREFUSED:
>>>>  	case EHOSTUNREACH:
>>>> +	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
>>> I think this introduced a regression... When the server does not support
>>> a protocol, say UDP, this patch cause the mount to hang forever,
>>> which I don't think we want.
>> 
>> 
>> I think we do want it to wait a while so that the nfs server has a
>> chance to start up.  We have no guarantee that the NFS server will be
>> registered with rpcbind before rpcbind responds to requests.
> I do see this race but there it has to be a small window. With
> Fedora its under seconds between the time rpcbind started
> and the NFS server.
>
>> 
>> I disagree with the "hang forever" description.  I just tested after
>> disabling UDP on an nfs server, and the delay was 2 minutes, 5 seconds
>> before a failure was reported.  It might be longer when trying TCP on a
>> server that only supports UDP.
> Yeah I did not wait that long... You are much more of a patient man than I ;-) 
> I do think this is a regression. Going an from an instant failure to one
> that takes over 2min is not a good thing... IMHO.
>
>> 
>> So I think the current behavior is correct.  You might be able to argue
>> that certain error codes should trigger a shorter timeout, but it would
>> need a strong argument.
> Going with the theory the window is very small, how about 
> a retry with a timeout then a failure? 

I started looking at changing the timeout and it wouldn't be too hard
(if we can agree on a suitable delay), but I feel I must ask why this is
important.
In what situation are you likely to mount with the wrong protocol, that
you aren't able to just Ctrl-C when you realized what a dumb thing you
just did?

If rpcbind isn't running, which is arguably a very similar situation
(no protocols are register) we have always had a long timeout. Why is
"just one protocol not registered" any different?


Anyway, below is the patch I was working on.  I stopped when I wasn't
sure how to handle ECONNREFUSED.

NeilBrown

Comments

Steve Dickson Nov. 28, 2016, 5:24 p.m. UTC | #1
My apologies for the delayed response... I just saw this... 

On 11/23/2016 06:26 PM, NeilBrown wrote:
> On Thu, Nov 24 2016, Steve Dickson wrote:
>>>
>>> So I think the current behavior is correct.  You might be able to argue
>>> that certain error codes should trigger a shorter timeout, but it would
>>> need a strong argument.
>> Going with the theory the window is very small, how about 
>> a retry with a timeout then a failure? 
> 
> I started looking at changing the timeout and it wouldn't be too hard
> (if we can agree on a suitable delay), but I feel I must ask why this is
> important.
Over the last few Connectathon and bakeathons I've been
floating the idea of dismantling the UDP support and
nobody to objected... The main reason is to cut the testing 
matrix in half.

I keep getting these goofy UDP bugs from our QE guys that
nobody is going to fix... and I have not seen a UDP bug
from a customer in years.. I really don't think 
it being used so why continue to support it?

> In what situation are you likely to mount with the wrong protocol, that
> you aren't able to just Ctrl-C when you realized what a dumb thing you
> just did?
I turned off the UDP support in rpc.nfsd and mounts started to hang.

> 
> If rpcbind isn't running, which is arguably a very similar situation
> (no protocols are register) we have always had a long timeout. Why is
> "just one protocol not registered" any different?
ECONNREFUSED can also me the server is not up so we 
need to wait. 

> 
> 
> Anyway, below is the patch I was working on.  I stopped when I wasn't
> sure how to handle ECONNREFUSED.
I've quick took and it does look a little messy or we
revert the EOPNOTSUPP commit... 

But at least you know my motivation.

steved. 
> 
> NeilBrown
> 
> 
> 
> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c
> index d5dfb5e4a669..084776115b9>>> I disagree with the "hang forever" description.  I just tested after
>>> disabling UDP on an nfs server, and the delay was 2 minutes, 5 seconds
>>> before a failure was reported.  It might be longer when trying TCP on a
>>> server that only supports UDP.
>> Yeah I did not wait that long... You are much more of a patient man than I ;-) 
>> I do think this is a regression. Going an from an instant failure to one
>> that takes over 2min is not a good thing... IMHO.
>>f 100644
> --- a/utils/mount/stropts.c
> +++ b/utils/mount/stropts.c
> @@ -935,24 +935,30 @@ static int nfs_try_mount(struct nfsmount_info *mi)
>   * failed so far, but fail immediately if there is a local
>   * error (like a bad mount option).
>   *
> - * ESTALE is also a temporary error because some servers
> - * return ESTALE when a share is temporarily offline.
> + * If there is a remote error, like ESTALE or RPC_PROGNOTREGISTERED
> + * then it is probably permanent, but there is a small chance
> + * the it is temporary can we caught the server at an awkward
> + * time during start-up.  A shorter timeout is best for such
> + * circumstances, so return a distinct status.
>   *
> - * Returns 1 if we should fail immediately, or 0 if we
> - * should retry.
> + * Returns PERMANENT if we should fail immediately,
> + * TEMPORARY if we should retry normally, or
> + * REMOTE if we should retry with shorter timeout.
>   */
> -static int nfs_is_permanent_error(int error)
> +enum error_type { PERMANENT, TEMPORARY, REMOTE };
> +static enum error_type nfs_error_type(int error)
>  {
>  	switch (error) {
>  	case ESTALE:
> +	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
> +		return REMOTE;
>  	case ETIMEDOUT:
>  	case ECONNREFUSED:
>  	case EHOSTUNREACH:
> -	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
>  	case EAGAIN:
> -		return 0;	/* temporary */
> +		return TEMPORARY;
>  	default:
> -		return 1;	/* permanent */
> +		return PERMANENT;
>  	}
>  }
>  
> @@ -967,6 +973,7 @@ static int nfsmount_fg(struct nfsmount_info *mi)
>  {
>  	unsigned int secs = 1;
>  	time_t timeout;
> +	int last_errno = 0;
>  
>  	timeout = nfs_parse_retry_option(mi->options,
>  					 NFS_DEF_FG_TIMEOUT_MINUTES);
> @@ -987,13 +994,22 @@ static int nfsmount_fg(struct nfsmount_info *mi)
>  			 */
>  			return EX_SUCCESS;
>  
> -		if (nfs_is_permanent_error(errno))
> +		switch(nfs_error_type(errno)) {
> +		case PERMANENT:
> +			timeout = 0;
>  			break;
> -
> -		if (time(NULL) > timeout) {
> +		case REMOTE:
> +			if (errno == last_errno)
> +				timeout = 0;
> +			break;
> +		case TEMPORARY:
>  			errno = ETIMEDOUT;
>  			break;
>  		}
> +		last_errno = errno;
> +
> +		if (time(NULL) > timeout)
> +			break;
>  
>  		if (errno != ETIMEDOUT) {
>  			if (sleep(secs))
> @@ -1020,7 +1036,7 @@ static int nfsmount_parent(struct nfsmount_info *mi)
>  	if (nfs_try_mount(mi))
>  		return EX_SUCCESS;
>  
> -	if (nfs_is_permanent_error(errno)) {
> +	if (nfs_error_type(errno) == PERMANENT) {
>  		mount_error(mi->spec, mi->node, errno);
>  		return EX_FAIL;
>  	}
> @@ -1055,8 +1071,14 @@ static int nfsmount_child(struct nfsmount_info *mi)
>  		if (nfs_try_mount(mi))
>  			return EX_SUCCESS;
>  
> -		if (nfs_is_permanent_error(errno))
> +		switch (nfs_error_type(errno)) {
> +		case REMOTE: /* Doesn't hurt to keep trying remote errors
> +			      * when in the background
> +			      */
> +		case PERMANENT:
> +			timeout = 0;
>  			break;
> +		}
>  
>  		if (time(NULL) > timeout)
>  			break;
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
NeilBrown Nov. 29, 2016, 9:36 p.m. UTC | #2
On Tue, Nov 29 2016, Steve Dickson wrote:

> My apologies for the delayed response... I just saw this... 


Yeah, life happens.

>
> On 11/23/2016 06:26 PM, NeilBrown wrote:
>> On Thu, Nov 24 2016, Steve Dickson wrote:
>>>>
>>>> So I think the current behavior is correct.  You might be able to argue
>>>> that certain error codes should trigger a shorter timeout, but it would
>>>> need a strong argument.
>>> Going with the theory the window is very small, how about 
>>> a retry with a timeout then a failure? 
>> 
>> I started looking at changing the timeout and it wouldn't be too hard
>> (if we can agree on a suitable delay), but I feel I must ask why this is
>> important.
> Over the last few Connectathon and bakeathons I've been
> floating the idea of dismantling the UDP support and
> nobody to objected... The main reason is to cut the testing 
> matrix in half.

Dismantling?  As in: removing the code?  I wouldn't like that.
Certainly switch the default and even pop up a warning "UDP bad - use
TCP ok?".

>
> I keep getting these goofy UDP bugs from our QE guys that
> nobody is going to fix... and I have not seen a UDP bug
> from a customer in years.. I really don't think 
> it being used so why continue to support it?

Completely happy for distros to choose not to support it.  But there are
more users of Linux than we distro people hear from.

>
>> In what situation are you likely to mount with the wrong protocol, that
>> you aren't able to just Ctrl-C when you realized what a dumb thing you
>> just did?
> I turned off the UDP support in rpc.nfsd and mounts started to hang.

Mounts that would otherwise fail - correct?
Exactly the same behavior as if you disabled NFS service on the server.

I have difficulty seeing this as being a serious problem.  It gets
notices, someone does "s/udp/tcp/" on /etc/fstab.  Problem gone.


>
>> 
>> If rpcbind isn't running, which is arguably a very similar situation
>> (no protocols are register) we have always had a long timeout. Why is
>> "just one protocol not registered" any different?
> ECONNREFUSED can also me the server is not up so we 
> need to wait.

If there servers in not up you get EHOSTUNREACHABLE (or however it is
spelled) or a timeout.
We need to wait on RPC_PROGNOTREGISTERED for exactly the same reason
that we need to wait for ECONNREFUSED: the server might still be coming
up and hasn't quite got everything registered properly yet.

>
>> 
>> 
>> Anyway, below is the patch I was working on.  I stopped when I wasn't
>> sure how to handle ECONNREFUSED.
> I've quick took and it does look a little messy or we
> revert the EOPNOTSUPP commit...

The EOPNOTSUPP commit fixed a real bug.  The bug would result in a mount
attempt failing when it should succeed.  Reverting that is not an
option.
Your symptom is not so much a bug and an inconvenience.  You issue a
command that will fail, and it takes a bit longer to fail than you would
like.  You still get the correct end result.

I don't think the patch is messy ... or at least, the resulting code
isn't.
I'll have another go and submit a proper patch.

Thanks,
NeilBrown

>
> But at least you know my motivation.
>
> steved. 
>> 
>> NeilBrown
>> 
>> 
>> 
>> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c
>> index d5dfb5e4a669..084776115b9>>> I disagree with the "hang forever" description.  I just tested after
>>>> disabling UDP on an nfs server, and the delay was 2 minutes, 5 seconds
>>>> before a failure was reported.  It might be longer when trying TCP on a
>>>> server that only supports UDP.
>>> Yeah I did not wait that long... You are much more of a patient man than I ;-) 
>>> I do think this is a regression. Going an from an instant failure to one
>>> that takes over 2min is not a good thing... IMHO.
>>>f 100644
>> --- a/utils/mount/stropts.c
>> +++ b/utils/mount/stropts.c
>> @@ -935,24 +935,30 @@ static int nfs_try_mount(struct nfsmount_info *mi)
>>   * failed so far, but fail immediately if there is a local
>>   * error (like a bad mount option).
>>   *
>> - * ESTALE is also a temporary error because some servers
>> - * return ESTALE when a share is temporarily offline.
>> + * If there is a remote error, like ESTALE or RPC_PROGNOTREGISTERED
>> + * then it is probably permanent, but there is a small chance
>> + * the it is temporary can we caught the server at an awkward
>> + * time during start-up.  A shorter timeout is best for such
>> + * circumstances, so return a distinct status.
>>   *
>> - * Returns 1 if we should fail immediately, or 0 if we
>> - * should retry.
>> + * Returns PERMANENT if we should fail immediately,
>> + * TEMPORARY if we should retry normally, or
>> + * REMOTE if we should retry with shorter timeout.
>>   */
>> -static int nfs_is_permanent_error(int error)
>> +enum error_type { PERMANENT, TEMPORARY, REMOTE };
>> +static enum error_type nfs_error_type(int error)
>>  {
>>  	switch (error) {
>>  	case ESTALE:
>> +	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
>> +		return REMOTE;
>>  	case ETIMEDOUT:
>>  	case ECONNREFUSED:
>>  	case EHOSTUNREACH:
>> -	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
>>  	case EAGAIN:
>> -		return 0;	/* temporary */
>> +		return TEMPORARY;
>>  	default:
>> -		return 1;	/* permanent */
>> +		return PERMANENT;
>>  	}
>>  }
>>  
>> @@ -967,6 +973,7 @@ static int nfsmount_fg(struct nfsmount_info *mi)
>>  {
>>  	unsigned int secs = 1;
>>  	time_t timeout;
>> +	int last_errno = 0;
>>  
>>  	timeout = nfs_parse_retry_option(mi->options,
>>  					 NFS_DEF_FG_TIMEOUT_MINUTES);
>> @@ -987,13 +994,22 @@ static int nfsmount_fg(struct nfsmount_info *mi)
>>  			 */
>>  			return EX_SUCCESS;
>>  
>> -		if (nfs_is_permanent_error(errno))
>> +		switch(nfs_error_type(errno)) {
>> +		case PERMANENT:
>> +			timeout = 0;
>>  			break;
>> -
>> -		if (time(NULL) > timeout) {
>> +		case REMOTE:
>> +			if (errno == last_errno)
>> +				timeout = 0;
>> +			break;
>> +		case TEMPORARY:
>>  			errno = ETIMEDOUT;
>>  			break;
>>  		}
>> +		last_errno = errno;
>> +
>> +		if (time(NULL) > timeout)
>> +			break;
>>  
>>  		if (errno != ETIMEDOUT) {
>>  			if (sleep(secs))
>> @@ -1020,7 +1036,7 @@ static int nfsmount_parent(struct nfsmount_info *mi)
>>  	if (nfs_try_mount(mi))
>>  		return EX_SUCCESS;
>>  
>> -	if (nfs_is_permanent_error(errno)) {
>> +	if (nfs_error_type(errno) == PERMANENT) {
>>  		mount_error(mi->spec, mi->node, errno);
>>  		return EX_FAIL;
>>  	}
>> @@ -1055,8 +1071,14 @@ static int nfsmount_child(struct nfsmount_info *mi)
>>  		if (nfs_try_mount(mi))
>>  			return EX_SUCCESS;
>>  
>> -		if (nfs_is_permanent_error(errno))
>> +		switch (nfs_error_type(errno)) {
>> +		case REMOTE: /* Doesn't hurt to keep trying remote errors
>> +			      * when in the background
>> +			      */
>> +		case PERMANENT:
>> +			timeout = 0;
>>  			break;
>> +		}
>>  
>>  		if (time(NULL) > timeout)
>>  			break;
>>
Steve Dickson Nov. 29, 2016, 11:05 p.m. UTC | #3
On 11/29/2016 04:36 PM, NeilBrown wrote:
> On Tue, Nov 29 2016, Steve Dickson wrote:
> 
>> My apologies for the delayed response... I just saw this... 
> 
> 
> Yeah, life happens.
> 
>>
>> On 11/23/2016 06:26 PM, NeilBrown wrote:
>>> On Thu, Nov 24 2016, Steve Dickson wrote:
>>>>>
>>>>> So I think the current behavior is correct.  You might be able to argue
>>>>> that certain error codes should trigger a shorter timeout, but it would
>>>>> need a strong argument.
>>>> Going with the theory the window is very small, how about 
>>>> a retry with a timeout then a failure? 
>>>
>>> I started looking at changing the timeout and it wouldn't be too hard
>>> (if we can agree on a suitable delay), but I feel I must ask why this is
>>> important.
>> Over the last few Connectathon and bakeathons I've been
>> floating the idea of dismantling the UDP support and
>> nobody to objected... The main reason is to cut the testing 
>> matrix in half.
> 
> Dismantling?  As in: removing the code?  I wouldn't like that.
No. Basically not to use the code. 

> Certainly switch the default and even pop up a warning "UDP bad - use
> TCP ok?".
Right... the first step on the client would be stop 
negotiating UDP mounts but still allow mount -o udp,v3 
to work, but not hang for 2.5 minutes before failing.

Then, I guess, not to have all the other daemons 
not to advertise UDP unless configured to do so,
but off by default. 

> 
>>
>> I keep getting these goofy UDP bugs from our QE guys that
>> nobody is going to fix... and I have not seen a UDP bug
>> from a customer in years.. I really don't think 
>> it being used so why continue to support it?
> 
> Completely happy for distros to choose not to support it.  But there are
> more users of Linux than we distro people hear from.
Fair enough... So leave the code in, but off by default
so it's not something that has to be tested. 

> 
>>
>>> In what situation are you likely to mount with the wrong protocol, that
>>> you aren't able to just Ctrl-C when you realized what a dumb thing you
>>> just did?
>> I turned off the UDP support in rpc.nfsd and mounts started to hang.
> 
> Mounts that would otherwise fail - correct?
> Exactly the same behavior as if you disabled NFS service on the server.
Yes, fail immediately which happen before commit df0b9998

> 
> I have difficulty seeing this as being a serious problem.  It gets
> notices, someone does "s/udp/tcp/" on /etc/fstab.  Problem gone.
I have not looked, but will it hang the reboot for a few minutes?

>>> If rpcbind isn't running, which is arguably a very similar situation
>>> (no protocols are register) we have always had a long timeout. Why is
>>> "just one protocol not registered" any different?
>> ECONNREFUSED can also me the server is not up so we 
>> need to wait.
> 
> If there servers in not up you get EHOSTUNREACHABLE (or however it is
> spelled) or a timeout.
> We need to wait on RPC_PROGNOTREGISTERED for exactly the same reason
> that we need to wait for ECONNREFUSED: the server might still be coming
> up and hasn't quite got everything registered properly yet.
Well when the server is down I'm seeing 
"mount.nfs: mount(2): Connection refused" on the v4 mount then mounts
then "RPC: Program not registered" on both the TCP and UDP v3 mounts.
And that cycles through for 2.5 mins until the failure.

So you are correct, there is no difference between when a server
is down and when a protocol is not supported. But this is where
we differ. A server could come up but a protocol is not all of
a sudden be supported. I think we should handle that case better. 
 
> 
>>
>>>
>>>
>>> Anyway, below is the patch I was working on.  I stopped when I wasn't
>>> sure how to handle ECONNREFUSED.
>> I've quick took and it does look a little messy or we
>> revert the EOPNOTSUPP commit...
> 
> The EOPNOTSUPP commit fixed a real bug.  The bug would result in a mount
> attempt failing when it should succeed.  Reverting that is not an
> option.
So the bug was that fact rpcbind was not up but the NFS server was?
Or both were on there way up?

> Your symptom is not so much a bug and an inconvenience.  You issue a
> command that will fail, and it takes a bit longer to fail than you would
> like.  You still get the correct end result.
No, I see it as a regression as how we used to handle a 
protocol not being supported.

> 
> I don't think the patch is messy ... or at least, the resulting code
> isn't.
> I'll have another go and submit a proper patch.
Thank you... I'm confident things will work out well.

steved.

> 
> Thanks,
> NeilBrown
> 
>>
>> But at least you know my motivation.
>>
>> steved. 
>>>
>>> NeilBrown
>>>
>>>
>>>
>>> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c
>>> index d5dfb5e4a669..084776115b9>>> I disagree with the "hang forever" description.  I just tested after
>>>>> disabling UDP on an nfs server, and the delay was 2 minutes, 5 seconds
>>>>> before a failure was reported.  It might be longer when trying TCP on a
>>>>> server that only supports UDP.
>>>> Yeah I did not wait that long... You are much more of a patient man than I ;-) 
>>>> I do think this is a regression. Going an from an instant failure to one
>>>> that takes over 2min is not a good thing... IMHO.
>>>> f 100644
>>> --- a/utils/mount/stropts.c
>>> +++ b/utils/mount/stropts.c
>>> @@ -935,24 +935,30 @@ static int nfs_try_mount(struct nfsmount_info *mi)
>>>   * failed so far, but fail immediately if there is a local
>>>   * error (like a bad mount option).
>>>   *
>>> - * ESTALE is also a temporary error because some servers
>>> - * return ESTALE when a share is temporarily offline.
>>> + * If there is a remote error, like ESTALE or RPC_PROGNOTREGISTERED
>>> + * then it is probably permanent, but there is a small chance
>>> + * the it is temporary can we caught the server at an awkward
>>> + * time during start-up.  A shorter timeout is best for such
>>> + * circumstances, so return a distinct status.
>>>   *
>>> - * Returns 1 if we should fail immediately, or 0 if we
>>> - * should retry.
>>> + * Returns PERMANENT if we should fail immediately,
>>> + * TEMPORARY if we should retry normally, or
>>> + * REMOTE if we should retry with shorter timeout.
>>>   */
>>> -static int nfs_is_permanent_error(int error)
>>> +enum error_type { PERMANENT, TEMPORARY, REMOTE };
>>> +static enum error_type nfs_error_type(int error)
>>>  {
>>>  	switch (error) {
>>>  	case ESTALE:
>>> +	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
>>> +		return REMOTE;
>>>  	case ETIMEDOUT:
>>>  	case ECONNREFUSED:
>>>  	case EHOSTUNREACH:
>>> -	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
>>>  	case EAGAIN:
>>> -		return 0;	/* temporary */
>>> +		return TEMPORARY;
>>>  	default:
>>> -		return 1;	/* permanent */
>>> +		return PERMANENT;
>>>  	}
>>>  }
>>>  
>>> @@ -967,6 +973,7 @@ static int nfsmount_fg(struct nfsmount_info *mi)
>>>  {
>>>  	unsigned int secs = 1;
>>>  	time_t timeout;
>>> +	int last_errno = 0;
>>>  
>>>  	timeout = nfs_parse_retry_option(mi->options,
>>>  					 NFS_DEF_FG_TIMEOUT_MINUTES);
>>> @@ -987,13 +994,22 @@ static int nfsmount_fg(struct nfsmount_info *mi)
>>>  			 */
>>>  			return EX_SUCCESS;
>>>  
>>> -		if (nfs_is_permanent_error(errno))
>>> +		switch(nfs_error_type(errno)) {
>>> +		case PERMANENT:
>>> +			timeout = 0;
>>>  			break;
>>> -
>>> -		if (time(NULL) > timeout) {
>>> +		case REMOTE:
>>> +			if (errno == last_errno)
>>> +				timeout = 0;
>>> +			break;
>>> +		case TEMPORARY:
>>>  			errno = ETIMEDOUT;
>>>  			break;
>>>  		}
>>> +		last_errno = errno;
>>> +
>>> +		if (time(NULL) > timeout)
>>> +			break;
>>>  
>>>  		if (errno != ETIMEDOUT) {
>>>  			if (sleep(secs))
>>> @@ -1020,7 +1036,7 @@ static int nfsmount_parent(struct nfsmount_info *mi)
>>>  	if (nfs_try_mount(mi))
>>>  		return EX_SUCCESS;
>>>  
>>> -	if (nfs_is_permanent_error(errno)) {
>>> +	if (nfs_error_type(errno) == PERMANENT) {
>>>  		mount_error(mi->spec, mi->node, errno);
>>>  		return EX_FAIL;
>>>  	}
>>> @@ -1055,8 +1071,14 @@ static int nfsmount_child(struct nfsmount_info *mi)
>>>  		if (nfs_try_mount(mi))
>>>  			return EX_SUCCESS;
>>>  
>>> -		if (nfs_is_permanent_error(errno))
>>> +		switch (nfs_error_type(errno)) {
>>> +		case REMOTE: /* Doesn't hurt to keep trying remote errors
>>> +			      * when in the background
>>> +			      */
>>> +		case PERMANENT:
>>> +			timeout = 0;
>>>  			break;
>>> +		}
>>>  
>>>  		if (time(NULL) > timeout)
>>>  			break;
>>>
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
NeilBrown Nov. 30, 2016, 1:33 a.m. UTC | #4
On Wed, Nov 30 2016, Steve Dickson wrote:

> On 11/29/2016 04:36 PM, NeilBrown wrote:
>> 
>> Completely happy for distros to choose not to support it.  But there are
>> more users of Linux than we distro people hear from.
> Fair enough... So leave the code in, but off by default
> so it's not something that has to be tested.

Thanks for clarifying.  Sounds good to me.

>> 
>> The EOPNOTSUPP commit fixed a real bug.  The bug would result in a mount
>> attempt failing when it should succeed.  Reverting that is not an
>> option.
> So the bug was that fact rpcbind was not up but the NFS server was?
> Or both were on there way up?

You could say that the bug was that rpcbind starts before the NFS server
has registered... but as the NFS server *cannot* register before rpcbind
starts, that would be a little harsh.

RPC_PROGNOTREGISTERED is a very close analogy to ECONNREFUSED.
They both say "The server process isn't listening (yet)".  One at the
TCP/IP level, one at the SUNRPC level.

If we started all daemons before enabling the network interface, and if
we could register all RPC services before rpcbind responds to external
requests, then these windows could be closed.  Unfortunately our
forefathers didn't have that foresight.

Thanks,
NeilBrown
diff mbox

Patch

diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c
index d5dfb5e4a669..084776115b9f 100644
--- a/utils/mount/stropts.c
+++ b/utils/mount/stropts.c
@@ -935,24 +935,30 @@  static int nfs_try_mount(struct nfsmount_info *mi)
  * failed so far, but fail immediately if there is a local
  * error (like a bad mount option).
  *
- * ESTALE is also a temporary error because some servers
- * return ESTALE when a share is temporarily offline.
+ * If there is a remote error, like ESTALE or RPC_PROGNOTREGISTERED
+ * then it is probably permanent, but there is a small chance
+ * the it is temporary can we caught the server at an awkward
+ * time during start-up.  A shorter timeout is best for such
+ * circumstances, so return a distinct status.
  *
- * Returns 1 if we should fail immediately, or 0 if we
- * should retry.
+ * Returns PERMANENT if we should fail immediately,
+ * TEMPORARY if we should retry normally, or
+ * REMOTE if we should retry with shorter timeout.
  */
-static int nfs_is_permanent_error(int error)
+enum error_type { PERMANENT, TEMPORARY, REMOTE };
+static enum error_type nfs_error_type(int error)
 {
 	switch (error) {
 	case ESTALE:
+	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
+		return REMOTE;
 	case ETIMEDOUT:
 	case ECONNREFUSED:
 	case EHOSTUNREACH:
-	case EOPNOTSUPP:	/* aka RPC_PROGNOTREGISTERED */
 	case EAGAIN:
-		return 0;	/* temporary */
+		return TEMPORARY;
 	default:
-		return 1;	/* permanent */
+		return PERMANENT;
 	}
 }
 
@@ -967,6 +973,7 @@  static int nfsmount_fg(struct nfsmount_info *mi)
 {
 	unsigned int secs = 1;
 	time_t timeout;
+	int last_errno = 0;
 
 	timeout = nfs_parse_retry_option(mi->options,
 					 NFS_DEF_FG_TIMEOUT_MINUTES);
@@ -987,13 +994,22 @@  static int nfsmount_fg(struct nfsmount_info *mi)
 			 */
 			return EX_SUCCESS;
 
-		if (nfs_is_permanent_error(errno))
+		switch(nfs_error_type(errno)) {
+		case PERMANENT:
+			timeout = 0;
 			break;
-
-		if (time(NULL) > timeout) {
+		case REMOTE:
+			if (errno == last_errno)
+				timeout = 0;
+			break;
+		case TEMPORARY:
 			errno = ETIMEDOUT;
 			break;
 		}
+		last_errno = errno;
+
+		if (time(NULL) > timeout)
+			break;
 
 		if (errno != ETIMEDOUT) {
 			if (sleep(secs))
@@ -1020,7 +1036,7 @@  static int nfsmount_parent(struct nfsmount_info *mi)
 	if (nfs_try_mount(mi))
 		return EX_SUCCESS;
 
-	if (nfs_is_permanent_error(errno)) {
+	if (nfs_error_type(errno) == PERMANENT) {
 		mount_error(mi->spec, mi->node, errno);
 		return EX_FAIL;
 	}
@@ -1055,8 +1071,14 @@  static int nfsmount_child(struct nfsmount_info *mi)
 		if (nfs_try_mount(mi))
 			return EX_SUCCESS;
 
-		if (nfs_is_permanent_error(errno))
+		switch (nfs_error_type(errno)) {
+		case REMOTE: /* Doesn't hurt to keep trying remote errors
+			      * when in the background
+			      */
+		case PERMANENT:
+			timeout = 0;
 			break;
+		}
 
 		if (time(NULL) > timeout)
 			break;