| Message ID | CAN-5tyEo3d18aWny5U65NWGP1G_4kdrKGw2VDEO8vipCoivAQw@mail.gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > > I'm looking for comments on the approach to deal with the following > denial-of-service issue. > > Currently, during the nfs4.0 mount, the code takes the content > supplied by the user in the mount command for "clientaddr" and that > becomes part of the content of the SETCLIENTID client id. There are no > verifications that the supplied address belongs to the client > initiating the mount. > > A denial of services comes from where there are 2 clients with IP A > and IP B (bad one). Client IP A mounts and has "IP A" in the > SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". > This causes the server to invalidate the lease for the legitimate > client IP A. Generally if this is a concern, Kerberos can be used during the SETCLIENTID to mutually authenticate the client and server. Shouldn't that prevent client B from tampering with client A's lease? > My suggested approach to fixing it, is to have nfs-utils do a sanity > checking that will check if the clientaddr that's suppose matches the > IP of the machine. Then currently, if it doesn't then it will ignore > the supplied value and use the IP of the machine. Whether this is > desirable vs say failing the mount and forcing the user to specify the > correct value is up for debate. Also, I'm not sure if the check for > the value of clientaddr should be done in the kernel itself instead of > the nfs-utils. > > Below is the rough fix to the nfs-utils. Please comment. One thing we want to be able to continue to do is specify "clientaddr=0.0.0.0" to disable NFSv4.0 callback (and thus delegation). > diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c > index 1217823..982927e 100644 > --- a/utils/mount/stropts.c > +++ b/utils/mount/stropts.c > @@ -242,11 +242,21 @@ static int nfs_append_clientaddr_option(const > struct sockaddr *sap, > struct sockaddr *my_addr = &address.sa; > socklen_t my_len = sizeof(address); > > - if (po_contains(options, "clientaddr") == PO_FOUND) > - return 1; > - > nfs_callback_address(sap, salen, my_addr, &my_len); > > + if (po_contains(options, "clientaddr") == PO_FOUND) { > + char *addr = po_get(options, "clientaddr"); > + char address[NI_MAXHOST]; > + > + if (!nfs_present_sockaddr(my_addr, my_len, address, > + sizeof(address))) > + goto out; > + > + if (strcmp(addr, address)) > + goto out; > + return 1; > + } > +out: > return nfs_append_generic_address_option(my_addr, my_len, > "clientaddr", options); > } > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: > > >> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> I'm looking for comments on the approach to deal with the following >> denial-of-service issue. >> >> Currently, during the nfs4.0 mount, the code takes the content >> supplied by the user in the mount command for "clientaddr" and that >> becomes part of the content of the SETCLIENTID client id. There are no >> verifications that the supplied address belongs to the client >> initiating the mount. >> >> A denial of services comes from where there are 2 clients with IP A >> and IP B (bad one). Client IP A mounts and has "IP A" in the >> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >> This causes the server to invalidate the lease for the legitimate >> client IP A. > > Generally if this is a concern, Kerberos can be used during > the SETCLIENTID to mutually authenticate the client and > server. Shouldn't that prevent client B from tampering with > client A's lease? It turns out to be a concern by folks (customers) that are using the code. Kerberos does not help here. Client IP B can have a valid Kerberos identity and still supply "clientaddr=" not belonging to it for the SETCLIENTID and interfere with the other's lease. >> My suggested approach to fixing it, is to have nfs-utils do a sanity >> checking that will check if the clientaddr that's suppose matches the >> IP of the machine. Then currently, if it doesn't then it will ignore >> the supplied value and use the IP of the machine. Whether this is >> desirable vs say failing the mount and forcing the user to specify the >> correct value is up for debate. Also, I'm not sure if the check for >> the value of clientaddr should be done in the kernel itself instead of >> the nfs-utils. >> >> Below is the rough fix to the nfs-utils. Please comment. > > One thing we want to be able to continue to do is specify > "clientaddr=0.0.0.0" to disable NFSv4.0 callback (and thus > delegation). Ok so some extra logic to check for the special value? > > >> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c >> index 1217823..982927e 100644 >> --- a/utils/mount/stropts.c >> +++ b/utils/mount/stropts.c >> @@ -242,11 +242,21 @@ static int nfs_append_clientaddr_option(const >> struct sockaddr *sap, >> struct sockaddr *my_addr = &address.sa; >> socklen_t my_len = sizeof(address); >> >> - if (po_contains(options, "clientaddr") == PO_FOUND) >> - return 1; >> - >> nfs_callback_address(sap, salen, my_addr, &my_len); >> >> + if (po_contains(options, "clientaddr") == PO_FOUND) { >> + char *addr = po_get(options, "clientaddr"); >> + char address[NI_MAXHOST]; >> + >> + if (!nfs_present_sockaddr(my_addr, my_len, address, >> + sizeof(address))) >> + goto out; >> + >> + if (strcmp(addr, address)) >> + goto out; >> + return 1; >> + } >> +out: >> return nfs_append_generic_address_option(my_addr, my_len, >> "clientaddr", options); >> } >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > Chuck Lever > > > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > > On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >> >> >>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>> >>> I'm looking for comments on the approach to deal with the following >>> denial-of-service issue. >>> >>> Currently, during the nfs4.0 mount, the code takes the content >>> supplied by the user in the mount command for "clientaddr" and that >>> becomes part of the content of the SETCLIENTID client id. There are no >>> verifications that the supplied address belongs to the client >>> initiating the mount. >>> >>> A denial of services comes from where there are 2 clients with IP A >>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>> This causes the server to invalidate the lease for the legitimate >>> client IP A. >> >> Generally if this is a concern, Kerberos can be used during >> the SETCLIENTID to mutually authenticate the client and >> server. Shouldn't that prevent client B from tampering with >> client A's lease? > > It turns out to be a concern by folks (customers) that are using the > code. Kerberos does not help here. Client IP B can have a valid > Kerberos identity and still supply "clientaddr=" not belonging to it > for the SETCLIENTID and interfere with the other's lease. SETCLIENTID is associated with a client ID string and a Kerberos principal. The server is supposed to deny a client with the same string (and perhaps the same callback information) but a different Kerberos identity from purging an existing lease belonging to a different principal. NFS4ERR_CLID_INUSE. Are you saying the two clients have exactly the same host principal? That seems... wrong. >>> My suggested approach to fixing it, is to have nfs-utils do a sanity >>> checking that will check if the clientaddr that's suppose matches the >>> IP of the machine. Then currently, if it doesn't then it will ignore >>> the supplied value and use the IP of the machine. Whether this is >>> desirable vs say failing the mount and forcing the user to specify the >>> correct value is up for debate. Also, I'm not sure if the check for >>> the value of clientaddr should be done in the kernel itself instead of >>> the nfs-utils. >>> >>> Below is the rough fix to the nfs-utils. Please comment. >> >> One thing we want to be able to continue to do is specify >> "clientaddr=0.0.0.0" to disable NFSv4.0 callback (and thus >> delegation). > > Ok so some extra logic to check for the special value? There is some value in stronger input validation here, outside of the particular issue you described above. I think such validation should allow INADDR_ANY (and the IPv6 equivalent). >>> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c >>> index 1217823..982927e 100644 >>> --- a/utils/mount/stropts.c >>> +++ b/utils/mount/stropts.c >>> @@ -242,11 +242,21 @@ static int nfs_append_clientaddr_option(const >>> struct sockaddr *sap, >>> struct sockaddr *my_addr = &address.sa; >>> socklen_t my_len = sizeof(address); >>> >>> - if (po_contains(options, "clientaddr") == PO_FOUND) >>> - return 1; >>> - >>> nfs_callback_address(sap, salen, my_addr, &my_len); >>> >>> + if (po_contains(options, "clientaddr") == PO_FOUND) { >>> + char *addr = po_get(options, "clientaddr"); >>> + char address[NI_MAXHOST]; >>> + >>> + if (!nfs_present_sockaddr(my_addr, my_len, address, >>> + sizeof(address))) >>> + goto out; >>> + >>> + if (strcmp(addr, address)) >>> + goto out; >>> + return 1; >>> + } >>> +out: >>> return nfs_append_generic_address_option(my_addr, my_len, >>> "clientaddr", options); >>> } >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> Chuck Lever -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: > > >> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>> >>> >>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>> >>>> I'm looking for comments on the approach to deal with the following >>>> denial-of-service issue. >>>> >>>> Currently, during the nfs4.0 mount, the code takes the content >>>> supplied by the user in the mount command for "clientaddr" and that >>>> becomes part of the content of the SETCLIENTID client id. There are no >>>> verifications that the supplied address belongs to the client >>>> initiating the mount. >>>> >>>> A denial of services comes from where there are 2 clients with IP A >>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>> This causes the server to invalidate the lease for the legitimate >>>> client IP A. >>> >>> Generally if this is a concern, Kerberos can be used during >>> the SETCLIENTID to mutually authenticate the client and >>> server. Shouldn't that prevent client B from tampering with >>> client A's lease? >> >> It turns out to be a concern by folks (customers) that are using the >> code. Kerberos does not help here. Client IP B can have a valid >> Kerberos identity and still supply "clientaddr=" not belonging to it >> for the SETCLIENTID and interfere with the other's lease. > > SETCLIENTID is associated with a client ID string and a Kerberos > principal. The server is supposed to deny a client with the same > string (and perhaps the same callback information) but a different > Kerberos identity from purging an existing lease belonging to a > different principal. NFS4ERR_CLID_INUSE. > > Are you saying the two clients have exactly the same host > principal? That seems... wrong. > Are you sure client ID is associated with a Kerberos principal? Looking ta the code that constructs the clientid content. I don't see that cl_nodename takes in principal identity. scnprintf(str, len, "Linux NFSv%u.%u %s", clp->rpc_ops->version, clp->cl_minorversion, clp->cl_rpcclient->cl_nodename); I have also tried to do a mount with and without Kerberos and the clientid string is that same has NFSv4.0 client ip/server ip. >>>> My suggested approach to fixing it, is to have nfs-utils do a sanity >>>> checking that will check if the clientaddr that's suppose matches the >>>> IP of the machine. Then currently, if it doesn't then it will ignore >>>> the supplied value and use the IP of the machine. Whether this is >>>> desirable vs say failing the mount and forcing the user to specify the >>>> correct value is up for debate. Also, I'm not sure if the check for >>>> the value of clientaddr should be done in the kernel itself instead of >>>> the nfs-utils. >>>> >>>> Below is the rough fix to the nfs-utils. Please comment. >>> >>> One thing we want to be able to continue to do is specify >>> "clientaddr=0.0.0.0" to disable NFSv4.0 callback (and thus >>> delegation). >> >> Ok so some extra logic to check for the special value? > > There is some value in stronger input validation here, > outside of the particular issue you described above. I > think such validation should allow INADDR_ANY (and the > IPv6 equivalent). Ok. > > >>>> diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c >>>> index 1217823..982927e 100644 >>>> --- a/utils/mount/stropts.c >>>> +++ b/utils/mount/stropts.c >>>> @@ -242,11 +242,21 @@ static int nfs_append_clientaddr_option(const >>>> struct sockaddr *sap, >>>> struct sockaddr *my_addr = &address.sa; >>>> socklen_t my_len = sizeof(address); >>>> >>>> - if (po_contains(options, "clientaddr") == PO_FOUND) >>>> - return 1; >>>> - >>>> nfs_callback_address(sap, salen, my_addr, &my_len); >>>> >>>> + if (po_contains(options, "clientaddr") == PO_FOUND) { >>>> + char *addr = po_get(options, "clientaddr"); >>>> + char address[NI_MAXHOST]; >>>> + >>>> + if (!nfs_present_sockaddr(my_addr, my_len, address, >>>> + sizeof(address))) >>>> + goto out; >>>> + >>>> + if (strcmp(addr, address)) >>>> + goto out; >>>> + return 1; >>>> + } >>>> +out: >>>> return nfs_append_generic_address_option(my_addr, my_len, >>>> "clientaddr", options); >>>> } >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> -- >>> Chuck Lever > > -- > Chuck Lever > > > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > > On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >> >> >>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>> >>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>> >>>> >>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>> >>>>> I'm looking for comments on the approach to deal with the following >>>>> denial-of-service issue. >>>>> >>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>> supplied by the user in the mount command for "clientaddr" and that >>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>> verifications that the supplied address belongs to the client >>>>> initiating the mount. >>>>> >>>>> A denial of services comes from where there are 2 clients with IP A >>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>> This causes the server to invalidate the lease for the legitimate >>>>> client IP A. >>>> >>>> Generally if this is a concern, Kerberos can be used during >>>> the SETCLIENTID to mutually authenticate the client and >>>> server. Shouldn't that prevent client B from tampering with >>>> client A's lease? >>> >>> It turns out to be a concern by folks (customers) that are using the >>> code. Kerberos does not help here. Client IP B can have a valid >>> Kerberos identity and still supply "clientaddr=" not belonging to it >>> for the SETCLIENTID and interfere with the other's lease. >> >> SETCLIENTID is associated with a client ID string and a Kerberos >> principal. The server is supposed to deny a client with the same >> string (and perhaps the same callback information) but a different >> Kerberos identity from purging an existing lease belonging to a >> different principal. NFS4ERR_CLID_INUSE. >> >> Are you saying the two clients have exactly the same host >> principal? That seems... wrong. >> > > Are you sure client ID is associated with a Kerberos principal? > > Looking ta the code that constructs the clientid content. I don't see > that cl_nodename takes in principal identity. > scnprintf(str, len, "Linux NFSv%u.%u %s", > clp->rpc_ops->version, clp->cl_minorversion, > clp->cl_rpcclient->cl_nodename); That's correct. Normally the Linux client picks up the host principal in the client's keytab and uses that as the credential for lease management operations like SETCLIENTID, without any regard to whether sec=sys or sec=krb5-yada is used on the mount command. The client ID string is not supposed to change between those cases. The server associates the client ID string with the Kerberos principal the client used to perform the SETCLIENTID. If a different Kerberos principal is used with a SETCLIENTID that bears the same client ID string as a client whose lease is still active, the server is supposed to reject that SETCLIENTID with NFS4ERR_CLID_INUSE. > I have also tried to do a mount with and without Kerberos and the > clientid string is that same has NFSv4.0 client ip/server ip. A quick way to disable the use of Kerberos for lease management is to sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak and then restart rpc.gssd. If the clients are using AUTH_UNIX credentials for SETCLIENTID, client A and client B would have to have the same cl_nodename to be able to futz with each others leases. Is that the case? There used to be a way to get the client to include a uniquifier in the client ID string. Has that logic been removed? -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 22, 2018 at 5:02 PM, Chuck Lever <chuck.lever@oracle.com> wrote: > > >> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>> >>> >>>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>> >>>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>> >>>>> >>>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>> >>>>>> I'm looking for comments on the approach to deal with the following >>>>>> denial-of-service issue. >>>>>> >>>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>>> supplied by the user in the mount command for "clientaddr" and that >>>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>>> verifications that the supplied address belongs to the client >>>>>> initiating the mount. >>>>>> >>>>>> A denial of services comes from where there are 2 clients with IP A >>>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>>> This causes the server to invalidate the lease for the legitimate >>>>>> client IP A. >>>>> >>>>> Generally if this is a concern, Kerberos can be used during >>>>> the SETCLIENTID to mutually authenticate the client and >>>>> server. Shouldn't that prevent client B from tampering with >>>>> client A's lease? >>>> >>>> It turns out to be a concern by folks (customers) that are using the >>>> code. Kerberos does not help here. Client IP B can have a valid >>>> Kerberos identity and still supply "clientaddr=" not belonging to it >>>> for the SETCLIENTID and interfere with the other's lease. >>> >>> SETCLIENTID is associated with a client ID string and a Kerberos >>> principal. The server is supposed to deny a client with the same >>> string (and perhaps the same callback information) but a different >>> Kerberos identity from purging an existing lease belonging to a >>> different principal. NFS4ERR_CLID_INUSE. >>> >>> Are you saying the two clients have exactly the same host >>> principal? That seems... wrong. >>> >> >> Are you sure client ID is associated with a Kerberos principal? >> >> Looking ta the code that constructs the clientid content. I don't see >> that cl_nodename takes in principal identity. >> scnprintf(str, len, "Linux NFSv%u.%u %s", >> clp->rpc_ops->version, clp->cl_minorversion, >> clp->cl_rpcclient->cl_nodename); > > That's correct. > > Normally the Linux client picks up the host principal in the > client's keytab and uses that as the credential for lease > management operations like SETCLIENTID, without any regard to > whether sec=sys or sec=krb5-yada is used on the mount command. > The client ID string is not supposed to change between those > cases. > > The server associates the client ID string with the Kerberos > principal the client used to perform the SETCLIENTID. I haven't checked the spec but is this required? > If a different Kerberos principal is used with a SETCLIENTID > that bears the same client ID string as a client whose lease > is still active, the server is supposed to reject that > SETCLIENTID with NFS4ERR_CLID_INUSE. I have tried (against the linux server), do a mount with krb5 and one without that used the clientaddr of the client with krb5 mount and I could get into the same lease revocation behavior. Which makes me question if indeed the servers do associate Kerberos principal in the SETCLIENTID handling. > > >> I have also tried to do a mount with and without Kerberos and the >> clientid string is that same has NFSv4.0 client ip/server ip. > > A quick way to disable the use of Kerberos for lease management > is to > > sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak > > and then restart rpc.gssd. > > If the clients are using AUTH_UNIX credentials for SETCLIENTID, > client A and client B would have to have the same cl_nodename > to be able to futz with each others leases. Is that the case? That is correct. Auth_unix mount can do it. But so it turns out to be with Kerberos/auth_unix mix. I haven't tried Kerberos/Kerberos but it makes me thing that it will also be a problem (since mix is a problem). > There used to be a way to get the client to include a uniquifier > in the client ID string. Has that logic been removed? I'm unaware of such logic. I wonder what that uniquer string used to be , a MAC address? The spec talks about how difficult it is to come up with a reboot persistent unique identifier. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On May 22, 2018, at 2:21 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > > On Tue, May 22, 2018 at 5:02 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >> >> >>> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>> >>> On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>> >>>> >>>>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>> >>>>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>>> >>>>>> >>>>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>>> >>>>>>> I'm looking for comments on the approach to deal with the following >>>>>>> denial-of-service issue. >>>>>>> >>>>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>>>> supplied by the user in the mount command for "clientaddr" and that >>>>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>>>> verifications that the supplied address belongs to the client >>>>>>> initiating the mount. >>>>>>> >>>>>>> A denial of services comes from where there are 2 clients with IP A >>>>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>>>> This causes the server to invalidate the lease for the legitimate >>>>>>> client IP A. >>>>>> >>>>>> Generally if this is a concern, Kerberos can be used during >>>>>> the SETCLIENTID to mutually authenticate the client and >>>>>> server. Shouldn't that prevent client B from tampering with >>>>>> client A's lease? >>>>> >>>>> It turns out to be a concern by folks (customers) that are using the >>>>> code. Kerberos does not help here. Client IP B can have a valid >>>>> Kerberos identity and still supply "clientaddr=" not belonging to it >>>>> for the SETCLIENTID and interfere with the other's lease. >>>> >>>> SETCLIENTID is associated with a client ID string and a Kerberos >>>> principal. The server is supposed to deny a client with the same >>>> string (and perhaps the same callback information) but a different >>>> Kerberos identity from purging an existing lease belonging to a >>>> different principal. NFS4ERR_CLID_INUSE. >>>> >>>> Are you saying the two clients have exactly the same host >>>> principal? That seems... wrong. >>>> >>> >>> Are you sure client ID is associated with a Kerberos principal? >>> >>> Looking ta the code that constructs the clientid content. I don't see >>> that cl_nodename takes in principal identity. >>> scnprintf(str, len, "Linux NFSv%u.%u %s", >>> clp->rpc_ops->version, clp->cl_minorversion, >>> clp->cl_rpcclient->cl_nodename); >> >> That's correct. >> >> Normally the Linux client picks up the host principal in the >> client's keytab and uses that as the credential for lease >> management operations like SETCLIENTID, without any regard to >> whether sec=sys or sec=krb5-yada is used on the mount command. >> The client ID string is not supposed to change between those >> cases. >> >> The server associates the client ID string with the Kerberos >> principal the client used to perform the SETCLIENTID. > > I haven't checked the spec but is this required? Yes, it is required. That's what the NFS4ERR_CLID_INUSE status code is for. RFC 7530 p. 291: For any confirmed record with the same id string x, if the recorded principal does not match that of the SETCLIENTID call, then the server returns an NFS4ERR_CLID_INUSE error. >> If a different Kerberos principal is used with a SETCLIENTID >> that bears the same client ID string as a client whose lease >> is still active, the server is supposed to reject that >> SETCLIENTID with NFS4ERR_CLID_INUSE. > > I have tried (against the linux server), do a mount with krb5 and one > without that used the clientaddr of the client with krb5 mount and I > could get into the same lease revocation behavior. Which makes me > question if indeed the servers do associate Kerberos principal in the > SETCLIENTID handling. That sounds like a bad server bug to me. Input validation on a client can't possibly be a reliable fix for this issue. Preventing lease tampering is exactly why the Linux client uses krb5i with the host principal for lease management whenever it can. >>> I have also tried to do a mount with and without Kerberos and the >>> clientid string is that same has NFSv4.0 client ip/server ip. >> >> A quick way to disable the use of Kerberos for lease management >> is to >> >> sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak >> >> and then restart rpc.gssd. >> >> If the clients are using AUTH_UNIX credentials for SETCLIENTID, >> client A and client B would have to have the same cl_nodename >> to be able to futz with each others leases. Is that the case? > > That is correct. Auth_unix mount can do it. But so it turns out to be > with Kerberos/auth_unix mix. I haven't tried Kerberos/Kerberos but it > makes me thing that it will also be a problem (since mix is a > problem). If an AUTH_UNIX client can tamper with a lease established by an AUTH_GSS client, that's a pretty serious server bug. Which server implementation is this? >> There used to be a way to get the client to include a uniquifier >> in the client ID string. Has that logic been removed? > > I'm unaware of such logic. I wonder what that uniquer string used to > be , a MAC address? The spec talks about how difficult it is to come > up with a reboot persistent unique identifier. Search for nfs4_client_id_uniquifier . It's meant to be a UUID, but it can be any random string. This can be set as a kernel boot parameter so it can be stored on a network boot server. -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 22, 2018 at 5:44 PM, Chuck Lever <chuck.lever@oracle.com> wrote: > > >> On May 22, 2018, at 2:21 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> On Tue, May 22, 2018 at 5:02 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>> >>> >>>> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>> >>>> On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>> >>>>> >>>>>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>> >>>>>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>>>> >>>>>>> >>>>>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>>>> >>>>>>>> I'm looking for comments on the approach to deal with the following >>>>>>>> denial-of-service issue. >>>>>>>> >>>>>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>>>>> supplied by the user in the mount command for "clientaddr" and that >>>>>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>>>>> verifications that the supplied address belongs to the client >>>>>>>> initiating the mount. >>>>>>>> >>>>>>>> A denial of services comes from where there are 2 clients with IP A >>>>>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>>>>> This causes the server to invalidate the lease for the legitimate >>>>>>>> client IP A. >>>>>>> >>>>>>> Generally if this is a concern, Kerberos can be used during >>>>>>> the SETCLIENTID to mutually authenticate the client and >>>>>>> server. Shouldn't that prevent client B from tampering with >>>>>>> client A's lease? >>>>>> >>>>>> It turns out to be a concern by folks (customers) that are using the >>>>>> code. Kerberos does not help here. Client IP B can have a valid >>>>>> Kerberos identity and still supply "clientaddr=" not belonging to it >>>>>> for the SETCLIENTID and interfere with the other's lease. >>>>> >>>>> SETCLIENTID is associated with a client ID string and a Kerberos >>>>> principal. The server is supposed to deny a client with the same >>>>> string (and perhaps the same callback information) but a different >>>>> Kerberos identity from purging an existing lease belonging to a >>>>> different principal. NFS4ERR_CLID_INUSE. >>>>> >>>>> Are you saying the two clients have exactly the same host >>>>> principal? That seems... wrong. >>>>> >>>> >>>> Are you sure client ID is associated with a Kerberos principal? >>>> >>>> Looking ta the code that constructs the clientid content. I don't see >>>> that cl_nodename takes in principal identity. >>>> scnprintf(str, len, "Linux NFSv%u.%u %s", >>>> clp->rpc_ops->version, clp->cl_minorversion, >>>> clp->cl_rpcclient->cl_nodename); >>> >>> That's correct. >>> >>> Normally the Linux client picks up the host principal in the >>> client's keytab and uses that as the credential for lease >>> management operations like SETCLIENTID, without any regard to >>> whether sec=sys or sec=krb5-yada is used on the mount command. >>> The client ID string is not supposed to change between those >>> cases. >>> >>> The server associates the client ID string with the Kerberos >>> principal the client used to perform the SETCLIENTID. >> >> I haven't checked the spec but is this required? > > Yes, it is required. That's what the NFS4ERR_CLID_INUSE status > code is for. > > RFC 7530 p. 291: > > For any confirmed > record with the same id string x, if the recorded principal does > not match that of the SETCLIENTID call, then the server returns an > NFS4ERR_CLID_INUSE error. > > >>> If a different Kerberos principal is used with a SETCLIENTID >>> that bears the same client ID string as a client whose lease >>> is still active, the server is supposed to reject that >>> SETCLIENTID with NFS4ERR_CLID_INUSE. >> >> I have tried (against the linux server), do a mount with krb5 and one >> without that used the clientaddr of the client with krb5 mount and I >> could get into the same lease revocation behavior. Which makes me >> question if indeed the servers do associate Kerberos principal in the >> SETCLIENTID handling. > > That sounds like a bad server bug to me. > > Input validation on a client can't possibly be a reliable fix > for this issue. But for auth_sys I believe it is helpful. > Preventing lease tampering is exactly why the > Linux client uses krb5i with the host principal for lease > management whenever it can. > > >>>> I have also tried to do a mount with and without Kerberos and the >>>> clientid string is that same has NFSv4.0 client ip/server ip. >>> >>> A quick way to disable the use of Kerberos for lease management >>> is to >>> >>> sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak >>> >>> and then restart rpc.gssd. >>> >>> If the clients are using AUTH_UNIX credentials for SETCLIENTID, >>> client A and client B would have to have the same cl_nodename >>> to be able to futz with each others leases. Is that the case? >> >> That is correct. Auth_unix mount can do it. But so it turns out to be >> with Kerberos/auth_unix mix. I haven't tried Kerberos/Kerberos but it >> makes me thing that it will also be a problem (since mix is a >> problem). > > If an AUTH_UNIX client can tamper with a lease established > by an AUTH_GSS client, that's a pretty serious server bug. > > Which server implementation is this? This is linux 4.16-rc1. >>> There used to be a way to get the client to include a uniquifier >>> in the client ID string. Has that logic been removed? >> >> I'm unaware of such logic. I wonder what that uniquer string used to >> be , a MAC address? The spec talks about how difficult it is to come >> up with a reboot persistent unique identifier. > > Search for nfs4_client_id_uniquifier . > > It's meant to be a UUID, but it can be any random string. > This can be set as a kernel boot parameter so it can be > stored on a network boot server. I see, ok thanks. > > > -- > Chuck Lever > > > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > > On Tue, May 22, 2018 at 5:44 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >> >> >>> On May 22, 2018, at 2:21 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>> >>> On Tue, May 22, 2018 at 5:02 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>> >>>> >>>>> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>> >>>>> On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>>> >>>>>> >>>>>>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>>> >>>>>>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>>>>> >>>>>>>>> I'm looking for comments on the approach to deal with the following >>>>>>>>> denial-of-service issue. >>>>>>>>> >>>>>>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>>>>>> supplied by the user in the mount command for "clientaddr" and that >>>>>>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>>>>>> verifications that the supplied address belongs to the client >>>>>>>>> initiating the mount. >>>>>>>>> >>>>>>>>> A denial of services comes from where there are 2 clients with IP A >>>>>>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>>>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>>>>>> This causes the server to invalidate the lease for the legitimate >>>>>>>>> client IP A. >>>>>>>> >>>>>>>> Generally if this is a concern, Kerberos can be used during >>>>>>>> the SETCLIENTID to mutually authenticate the client and >>>>>>>> server. Shouldn't that prevent client B from tampering with >>>>>>>> client A's lease? >>>>>>> >>>>>>> It turns out to be a concern by folks (customers) that are using the >>>>>>> code. Kerberos does not help here. Client IP B can have a valid >>>>>>> Kerberos identity and still supply "clientaddr=" not belonging to it >>>>>>> for the SETCLIENTID and interfere with the other's lease. >>>>>> >>>>>> SETCLIENTID is associated with a client ID string and a Kerberos >>>>>> principal. The server is supposed to deny a client with the same >>>>>> string (and perhaps the same callback information) but a different >>>>>> Kerberos identity from purging an existing lease belonging to a >>>>>> different principal. NFS4ERR_CLID_INUSE. >>>>>> >>>>>> Are you saying the two clients have exactly the same host >>>>>> principal? That seems... wrong. >>>>>> >>>>> >>>>> Are you sure client ID is associated with a Kerberos principal? >>>>> >>>>> Looking ta the code that constructs the clientid content. I don't see >>>>> that cl_nodename takes in principal identity. >>>>> scnprintf(str, len, "Linux NFSv%u.%u %s", >>>>> clp->rpc_ops->version, clp->cl_minorversion, >>>>> clp->cl_rpcclient->cl_nodename); >>>> >>>> That's correct. >>>> >>>> Normally the Linux client picks up the host principal in the >>>> client's keytab and uses that as the credential for lease >>>> management operations like SETCLIENTID, without any regard to >>>> whether sec=sys or sec=krb5-yada is used on the mount command. >>>> The client ID string is not supposed to change between those >>>> cases. >>>> >>>> The server associates the client ID string with the Kerberos >>>> principal the client used to perform the SETCLIENTID. >>> >>> I haven't checked the spec but is this required? >> >> Yes, it is required. That's what the NFS4ERR_CLID_INUSE status >> code is for. >> >> RFC 7530 p. 291: >> >> For any confirmed >> record with the same id string x, if the recorded principal does >> not match that of the SETCLIENTID call, then the server returns an >> NFS4ERR_CLID_INUSE error. >> >> >>>> If a different Kerberos principal is used with a SETCLIENTID >>>> that bears the same client ID string as a client whose lease >>>> is still active, the server is supposed to reject that >>>> SETCLIENTID with NFS4ERR_CLID_INUSE. >>> >>> I have tried (against the linux server), do a mount with krb5 and one >>> without that used the clientaddr of the client with krb5 mount and I >>> could get into the same lease revocation behavior. Which makes me >>> question if indeed the servers do associate Kerberos principal in the >>> SETCLIENTID handling. >> >> That sounds like a bad server bug to me. >> >> Input validation on a client can't possibly be a reliable fix >> for this issue. > > But for auth_sys I believe it is helpful. With AUTH_UNIX, the best you can do is try to ensure that all your clients have unique cl_nodenames. The question of whether the provided callback address is valid is a different matter. "Is this user-provided address one of my local interfaces or _ANY?" and then either warn or fail the mount if not. >> Preventing lease tampering is exactly why the >> Linux client uses krb5i with the host principal for lease >> management whenever it can. >> >> >>>>> I have also tried to do a mount with and without Kerberos and the >>>>> clientid string is that same has NFSv4.0 client ip/server ip. >>>> >>>> A quick way to disable the use of Kerberos for lease management >>>> is to >>>> >>>> sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak >>>> >>>> and then restart rpc.gssd. >>>> >>>> If the clients are using AUTH_UNIX credentials for SETCLIENTID, >>>> client A and client B would have to have the same cl_nodename >>>> to be able to futz with each others leases. Is that the case? >>> >>> That is correct. Auth_unix mount can do it. But so it turns out to be >>> with Kerberos/auth_unix mix. I haven't tried Kerberos/Kerberos but it >>> makes me thing that it will also be a problem (since mix is a >>> problem). >> >> If an AUTH_UNIX client can tamper with a lease established >> by an AUTH_GSS client, that's a pretty serious server bug. >> >> Which server implementation is this? > > This is linux 4.16-rc1. Would it be easy for you confirm if two AUTH_GSS clients are appropriately protected from each other? It would be good to file a bug on bugzilla.linux-nfs.org to document the full extent of the badness. >>>> There used to be a way to get the client to include a uniquifier >>>> in the client ID string. Has that logic been removed? >>> >>> I'm unaware of such logic. I wonder what that uniquer string used to >>> be , a MAC address? The spec talks about how difficult it is to come >>> up with a reboot persistent unique identifier. >> >> Search for nfs4_client_id_uniquifier . >> >> It's meant to be a UUID, but it can be any random string. >> This can be set as a kernel boot parameter so it can be >> stored on a network boot server. > > I see, ok thanks. >> >> >> -- >> Chuck Lever -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 22, 2018 at 6:36 PM, Chuck Lever <chuck.lever@oracle.com> wrote: > > >> On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> On Tue, May 22, 2018 at 5:44 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>> >>> >>>> On May 22, 2018, at 2:21 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>> >>>> On Tue, May 22, 2018 at 5:02 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>> >>>>> >>>>>> On May 22, 2018, at 1:38 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>> >>>>>> On Tue, May 22, 2018 at 4:22 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>>>> >>>>>>> >>>>>>>> On May 22, 2018, at 1:17 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>>>> >>>>>>>> On Tue, May 22, 2018 at 4:08 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On May 22, 2018, at 1:03 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>>>>>>>> >>>>>>>>>> I'm looking for comments on the approach to deal with the following >>>>>>>>>> denial-of-service issue. >>>>>>>>>> >>>>>>>>>> Currently, during the nfs4.0 mount, the code takes the content >>>>>>>>>> supplied by the user in the mount command for "clientaddr" and that >>>>>>>>>> becomes part of the content of the SETCLIENTID client id. There are no >>>>>>>>>> verifications that the supplied address belongs to the client >>>>>>>>>> initiating the mount. >>>>>>>>>> >>>>>>>>>> A denial of services comes from where there are 2 clients with IP A >>>>>>>>>> and IP B (bad one). Client IP A mounts and has "IP A" in the >>>>>>>>>> SETCLIENTID. Client IP B does a mount and specified "clientaddr=IP A". >>>>>>>>>> This causes the server to invalidate the lease for the legitimate >>>>>>>>>> client IP A. >>>>>>>>> >>>>>>>>> Generally if this is a concern, Kerberos can be used during >>>>>>>>> the SETCLIENTID to mutually authenticate the client and >>>>>>>>> server. Shouldn't that prevent client B from tampering with >>>>>>>>> client A's lease? >>>>>>>> >>>>>>>> It turns out to be a concern by folks (customers) that are using the >>>>>>>> code. Kerberos does not help here. Client IP B can have a valid >>>>>>>> Kerberos identity and still supply "clientaddr=" not belonging to it >>>>>>>> for the SETCLIENTID and interfere with the other's lease. >>>>>>> >>>>>>> SETCLIENTID is associated with a client ID string and a Kerberos >>>>>>> principal. The server is supposed to deny a client with the same >>>>>>> string (and perhaps the same callback information) but a different >>>>>>> Kerberos identity from purging an existing lease belonging to a >>>>>>> different principal. NFS4ERR_CLID_INUSE. >>>>>>> >>>>>>> Are you saying the two clients have exactly the same host >>>>>>> principal? That seems... wrong. >>>>>>> >>>>>> >>>>>> Are you sure client ID is associated with a Kerberos principal? >>>>>> >>>>>> Looking ta the code that constructs the clientid content. I don't see >>>>>> that cl_nodename takes in principal identity. >>>>>> scnprintf(str, len, "Linux NFSv%u.%u %s", >>>>>> clp->rpc_ops->version, clp->cl_minorversion, >>>>>> clp->cl_rpcclient->cl_nodename); >>>>> >>>>> That's correct. >>>>> >>>>> Normally the Linux client picks up the host principal in the >>>>> client's keytab and uses that as the credential for lease >>>>> management operations like SETCLIENTID, without any regard to >>>>> whether sec=sys or sec=krb5-yada is used on the mount command. >>>>> The client ID string is not supposed to change between those >>>>> cases. >>>>> >>>>> The server associates the client ID string with the Kerberos >>>>> principal the client used to perform the SETCLIENTID. >>>> >>>> I haven't checked the spec but is this required? >>> >>> Yes, it is required. That's what the NFS4ERR_CLID_INUSE status >>> code is for. >>> >>> RFC 7530 p. 291: >>> >>> For any confirmed >>> record with the same id string x, if the recorded principal does >>> not match that of the SETCLIENTID call, then the server returns an >>> NFS4ERR_CLID_INUSE error. >>> >>> >>>>> If a different Kerberos principal is used with a SETCLIENTID >>>>> that bears the same client ID string as a client whose lease >>>>> is still active, the server is supposed to reject that >>>>> SETCLIENTID with NFS4ERR_CLID_INUSE. >>>> >>>> I have tried (against the linux server), do a mount with krb5 and one >>>> without that used the clientaddr of the client with krb5 mount and I >>>> could get into the same lease revocation behavior. Which makes me >>>> question if indeed the servers do associate Kerberos principal in the >>>> SETCLIENTID handling. >>> >>> That sounds like a bad server bug to me. >>> >>> Input validation on a client can't possibly be a reliable fix >>> for this issue. >> >> But for auth_sys I believe it is helpful. > > With AUTH_UNIX, the best you can do is try to ensure > that all your clients have unique cl_nodenames. > The question of whether the provided callback address is > valid is a different matter. "Is this user-provided > address one of my local interfaces or _ANY?" and then > either warn or fail the mount if not. clientaddr is advertised to the users for the callback information and yet the code uses it to construct the client id string. It should either not do so and acquire the IP information independently from what was supplied or I think there should be a check on the user supplied input. Would you support a patch that does the check and then (I'm making a choice here) fails the mount if the check fails. >>> Preventing lease tampering is exactly why the >>> Linux client uses krb5i with the host principal for lease >>> management whenever it can. >>> >>> >>>>>> I have also tried to do a mount with and without Kerberos and the >>>>>> clientid string is that same has NFSv4.0 client ip/server ip. >>>>> >>>>> A quick way to disable the use of Kerberos for lease management >>>>> is to >>>>> >>>>> sudo mv /etc/krb5.keytab /etc/krb5.keytab.bak >>>>> >>>>> and then restart rpc.gssd. >>>>> >>>>> If the clients are using AUTH_UNIX credentials for SETCLIENTID, >>>>> client A and client B would have to have the same cl_nodename >>>>> to be able to futz with each others leases. Is that the case? >>>> >>>> That is correct. Auth_unix mount can do it. But so it turns out to be >>>> with Kerberos/auth_unix mix. I haven't tried Kerberos/Kerberos but it >>>> makes me thing that it will also be a problem (since mix is a >>>> problem). >>> >>> If an AUTH_UNIX client can tamper with a lease established >>> by an AUTH_GSS client, that's a pretty serious server bug. >>> >>> Which server implementation is this? >> >> This is linux 4.16-rc1. > > Would it be easy for you confirm if two AUTH_GSS clients are > appropriately protected from each other? It would be good to > file a bug on bugzilla.linux-nfs.org to document the full > extent of the badness. > > >>>>> There used to be a way to get the client to include a uniquifier >>>>> in the client ID string. Has that logic been removed? >>>> >>>> I'm unaware of such logic. I wonder what that uniquer string used to >>>> be , a MAC address? The spec talks about how difficult it is to come >>>> up with a reboot persistent unique identifier. >>> >>> Search for nfs4_client_id_uniquifier . >>> >>> It's meant to be a UUID, but it can be any random string. >>> This can be set as a kernel boot parameter so it can be >>> stored on a network boot server. >> >> I see, ok thanks. >>> >>> >>> -- >>> Chuck Lever > > -- > Chuck Lever > > > -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hey Olga- > On May 23, 2018, at 8:27 AM, Olga Kornievskaia <aglo@umich.edu> wrote: > > On Tue, May 22, 2018 at 6:36 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >> >> The question of whether the provided callback address is >> valid is a different matter. "Is this user-provided >> address one of my local interfaces or _ANY?" and then >> either warn or fail the mount if not. > > clientaddr is advertised to the users for the callback information and > yet the code uses it to construct the client id string. To be clear, the only way the server knows about callback information is because nfs4_proc_setclientid constructs a cb_client4 argument to SETCLIENTID. The server treats client ID strings as opaque blobs, used only for comparison with other client ID strings. It does not matter that the non-uniform client ID string we construct happens to have random bogus crap in it, just as long as it is different random bogus crap as any other client. ;-) What matters is what goes in cb_client4. > It should > either not do so and acquire the IP information independently from > what was supplied The normal situation is that "clientaddr=" is not specified by the administrator. Rather it is constructed by mount.nfs and passed into the kernel with the other mount options. (At least in the past) the kernel could not determine the callback information by itself, which is why we have clientaddr= in the first place. I'm not sure from your original post why the admin was supplying clientaddr= . I got the impression that it was malicious, but maybe I am mistaken? Was there some problem with the clientaddr= being supplied by mount.nfs ? > or I think there should be a check on the user > supplied input. It's rare that an administrator would need to explicitly add clientaddr= on the command line. But some input validation is reasonable in the case where an admin (and not mount.nfs) has specified this address. > Would you support a patch that does the check and then (I'm making a > choice here) fails the mount if the check fails. I would need to know specifically what you have in mind for "the check". -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, May 23, 2018 at 12:05 PM, Chuck Lever <chuck.lever@oracle.com> wrote: > Hey Olga- > >> On May 23, 2018, at 8:27 AM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> On Tue, May 22, 2018 at 6:36 PM, Chuck Lever <chuck.lever@oracle.com> wrote: >>> >>> The question of whether the provided callback address is >>> valid is a different matter. "Is this user-provided >>> address one of my local interfaces or _ANY?" and then >>> either warn or fail the mount if not. >> >> clientaddr is advertised to the users for the callback information and >> yet the code uses it to construct the client id string. > > To be clear, the only way the server knows about callback > information is because nfs4_proc_setclientid constructs a > cb_client4 argument to SETCLIENTID. The server treats > client ID strings as opaque blobs, used only for comparison > with other client ID strings. It does not matter that the > non-uniform client ID string we construct happens to have > random bogus crap in it, just as long as it is different > random bogus crap as any other client. ;-) > > What matters is what goes in cb_client4. > > >> It should >> either not do so and acquire the IP information independently from >> what was supplied > > The normal situation is that "clientaddr=" is not specified > by the administrator. Rather it is constructed by mount.nfs > and passed into the kernel with the other mount options. > > (At least in the past) the kernel could not determine the > callback information by itself, which is why we have > clientaddr= in the first place. > > I'm not sure from your original post why the admin was > supplying clientaddr= . I got the impression that it was > malicious, but maybe I am mistaken? In practice it was a simple misconfiguration on the part of the admin that input an incorrect address. Mistaken or malicious attempt will greatly interfere with a legitimate user. > Was there some problem > with the clientaddr= being supplied by mount.nfs ? The problem is that clientaddr is allowed to be specified by the user and have no checks before being used as an input to the client id content. >> or I think there should be a check on the user >> supplied input. > > It's rare that an administrator would need to explicitly add > clientaddr= on the command line. I guess not rare enough as it came up as a customer case. > But some input validation > is reasonable in the case where an admin (and not mount.nfs) > has specified this address. I believe so and that's what I'm trying to accomplish. >> Would you support a patch that does the check and then (I'm making a >> choice here) fails the mount if the check fails. > > I would need to know specifically what you have in mind for > "the check". I will make another attempt at the check to include allowing for the 0.0.0.0 and whatever equivalent of it is in ipv6. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: > > On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > >> If an AUTH_UNIX client can tamper with a lease established > >> by an AUTH_GSS client, that's a pretty serious server bug. > >> > >> Which server implementation is this? > > > > This is linux 4.16-rc1. > > Would it be easy for you confirm if two AUTH_GSS clients are > appropriately protected from each other? It would be good to > file a bug on bugzilla.linux-nfs.org to document the full > extent of the badness. If you try a setclientid with a client name matching an already-established client with state, then nfsd4_setclientid() should be returning CLID_INUSE: if (conf && client_has_state(conf)) { ... status = nfserr_clid_inuse; ... if (!same_creds(&conf->cl_cred, &rqstp-.rq_cred)) { ... goto out; } } So if you're seeing SETCLIENTID succeed then maybe same_creds() or client_has_state() is failing. Maybe client_has_state()?--that will fail (and allow the setclientid) if the v4.0 client doesn't currently have any opens or delegations. I think that's correct: https://tools.ietf.org/html/rfc7530#section-9.1.2 when the server gets a SETCLIENTID for a client ID that currently has no state, or it has state but the lease has expired, rather than returning NFS4ERR_CLID_INUSE, the server MUST allow the SETCLIENTID and confirm the new client ID if followed by the appropriate SETCLIENTID_CONFIRM. That's left out of the later breakdown of cases in 16.33.5, unfortunately. --b. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On May 29, 2018, at 3:56 PM, bfields@fieldses.org wrote: > > On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: >>> On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >>>> If an AUTH_UNIX client can tamper with a lease established >>>> by an AUTH_GSS client, that's a pretty serious server bug. >>>> >>>> Which server implementation is this? >>> >>> This is linux 4.16-rc1. >> >> Would it be easy for you confirm if two AUTH_GSS clients are >> appropriately protected from each other? It would be good to >> file a bug on bugzilla.linux-nfs.org to document the full >> extent of the badness. > > If you try a setclientid with a client name matching an > already-established client with state, then nfsd4_setclientid() should > be returning CLID_INUSE: > > if (conf && client_has_state(conf)) { > ... > status = nfserr_clid_inuse; > ... > if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) { > ... > goto out; > } > } > > So if you're seeing SETCLIENTID succeed then maybe same_creds() or > client_has_state() is failing. > > Maybe client_has_state()?--that will fail (and allow the setclientid) if > the v4.0 client doesn't currently have any opens or delegations. > > I think that's correct: > > https://tools.ietf.org/html/rfc7530#section-9.1.2 > > when the server gets a SETCLIENTID for a client ID that > currently has no state, or it has state but the lease has > expired, rather than returning NFS4ERR_CLID_INUSE, the server > MUST allow the SETCLIENTID and confirm the new client ID if > followed by the appropriate SETCLIENTID_CONFIRM. > > That's left out of the later breakdown of cases in 16.33.5, > unfortunately. That prevents certain denial of service attacks. A client can't lose state because of this. However, it can do a SETCLIENTID then later be prevented from doing its first OPEN? -- Chuck Lever -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 29, 2018 at 3:56 PM, J. Bruce Fields <bfields@fieldses.org> wrote: > On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: >> > On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> If an AUTH_UNIX client can tamper with a lease established >> >> by an AUTH_GSS client, that's a pretty serious server bug. >> >> >> >> Which server implementation is this? >> > >> > This is linux 4.16-rc1. >> >> Would it be easy for you confirm if two AUTH_GSS clients are >> appropriately protected from each other? It would be good to >> file a bug on bugzilla.linux-nfs.org to document the full >> extent of the badness. > > If you try a setclientid with a client name matching an > already-established client with state, then nfsd4_setclientid() should > be returning CLID_INUSE: This is not what I see. I see that the 2nd SETCLIENTID succeed. Then the RENEW from the 1st client gets ERR_EXPIRED. 1st client does the SETCLIENTID/SETCLIENT_CONFIRM. Then when the 2nd client's RENEW is sent it gets ERR_EXPIRED and it sends SETCLIENTID. > > if (conf && client_has_state(conf)) { > ... > status = nfserr_clid_inuse; > ... > if (!same_creds(&conf->cl_cred, &rqstp-.rq_cred)) { > ... > goto out; > } > } > > So if you're seeing SETCLIENTID succeed then maybe same_creds() or > client_has_state() is failing. > > Maybe client_has_state()?--that will fail (and allow the setclientid) if > the v4.0 client doesn't currently have any opens or delegations. > > I think that's correct: > > https://tools.ietf.org/html/rfc7530#section-9.1.2 > > when the server gets a SETCLIENTID for a client ID that > currently has no state, or it has state but the lease has > expired, rather than returning NFS4ERR_CLID_INUSE, the server > MUST allow the SETCLIENTID and confirm the new client ID if > followed by the appropriate SETCLIENTID_CONFIRM. At the time the "competing" SETCLIENTID arrives the other client definitely does not have an expired lease. > > That's left out of the later breakdown of cases in 16.33.5, > unfortunately. > > --b. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 29, 2018 at 04:03:46PM -0400, Chuck Lever wrote: > > > > On May 29, 2018, at 3:56 PM, bfields@fieldses.org wrote: > > > > On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: > >>> On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > >>>> If an AUTH_UNIX client can tamper with a lease established > >>>> by an AUTH_GSS client, that's a pretty serious server bug. > >>>> > >>>> Which server implementation is this? > >>> > >>> This is linux 4.16-rc1. > >> > >> Would it be easy for you confirm if two AUTH_GSS clients are > >> appropriately protected from each other? It would be good to > >> file a bug on bugzilla.linux-nfs.org to document the full > >> extent of the badness. > > > > If you try a setclientid with a client name matching an > > already-established client with state, then nfsd4_setclientid() should > > be returning CLID_INUSE: > > > > if (conf && client_has_state(conf)) { > > ... > > status = nfserr_clid_inuse; > > ... > > if (!same_creds(&conf->cl_cred, &rqstp->rq_cred)) { > > ... > > goto out; > > } > > } > > > > So if you're seeing SETCLIENTID succeed then maybe same_creds() or > > client_has_state() is failing. > > > > Maybe client_has_state()?--that will fail (and allow the setclientid) if > > the v4.0 client doesn't currently have any opens or delegations. > > > > I think that's correct: > > > > https://tools.ietf.org/html/rfc7530#section-9.1.2 > > > > when the server gets a SETCLIENTID for a client ID that > > currently has no state, or it has state but the lease has > > expired, rather than returning NFS4ERR_CLID_INUSE, the server > > MUST allow the SETCLIENTID and confirm the new client ID if > > followed by the appropriate SETCLIENTID_CONFIRM. > > > > That's left out of the later breakdown of cases in 16.33.5, > > unfortunately. > > That prevents certain denial of service attacks. A client can't > lose state because of this. However, it can do a SETCLIENTID > then later be prevented from doing its first OPEN? Yeah, I think so. Looking back at changelogs, the only motivation was to avoid annoyances in testing, where the same client might rapidly mount several different security flavors. The client may have been wrong not to change its identifier in that case, I don't know. All a bit academic for 4.1, where the client pretty much always has a session. --b. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 29, 2018 at 04:14:09PM -0400, Olga Kornievskaia wrote: > On Tue, May 29, 2018 at 3:56 PM, J. Bruce Fields <bfields@fieldses.org> wrote: > > On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: > >> > On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: > >> >> If an AUTH_UNIX client can tamper with a lease established > >> >> by an AUTH_GSS client, that's a pretty serious server bug. > >> >> > >> >> Which server implementation is this? > >> > > >> > This is linux 4.16-rc1. > >> > >> Would it be easy for you confirm if two AUTH_GSS clients are > >> appropriately protected from each other? It would be good to > >> file a bug on bugzilla.linux-nfs.org to document the full > >> extent of the badness. > > > > If you try a setclientid with a client name matching an > > already-established client with state, then nfsd4_setclientid() should > > be returning CLID_INUSE: > > This is not what I see. I see that the 2nd SETCLIENTID succeed. Then > the RENEW from the 1st client gets ERR_EXPIRED. 1st client does the > SETCLIENTID/SETCLIENT_CONFIRM. Then when the 2nd client's RENEW is > sent it gets ERR_EXPIRED and it sends SETCLIENTID. > > > > > if (conf && client_has_state(conf)) { > > ... > > status = nfserr_clid_inuse; > > ... > > if (!same_creds(&conf->cl_cred, &rqstp-.rq_cred)) { > > ... > > goto out; > > } > > } > > > > So if you're seeing SETCLIENTID succeed then maybe same_creds() or > > client_has_state() is failing. > > > > Maybe client_has_state()?--that will fail (and allow the setclientid) if > > the v4.0 client doesn't currently have any opens or delegations. > > > > I think that's correct: > > > > https://tools.ietf.org/html/rfc7530#section-9.1.2 > > > > when the server gets a SETCLIENTID for a client ID that > > currently has no state, or it has state but the lease has > > expired, rather than returning NFS4ERR_CLID_INUSE, the server > > MUST allow the SETCLIENTID and confirm the new client ID if > > followed by the appropriate SETCLIENTID_CONFIRM. > > At the time the "competing" SETCLIENTID arrives the other client > definitely does not have an expired lease. Right, but does it have opens or delegations? That's what I took as the definition of "state" for the code above. Could be wrong, I don't know. --b. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 29, 2018 at 4:36 PM, J. Bruce Fields <bfields@fieldses.org> wrote: > On Tue, May 29, 2018 at 04:14:09PM -0400, Olga Kornievskaia wrote: >> On Tue, May 29, 2018 at 3:56 PM, J. Bruce Fields <bfields@fieldses.org> wrote: >> > On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: >> >> > On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> >> If an AUTH_UNIX client can tamper with a lease established >> >> >> by an AUTH_GSS client, that's a pretty serious server bug. >> >> >> >> >> >> Which server implementation is this? >> >> > >> >> > This is linux 4.16-rc1. >> >> >> >> Would it be easy for you confirm if two AUTH_GSS clients are >> >> appropriately protected from each other? It would be good to >> >> file a bug on bugzilla.linux-nfs.org to document the full >> >> extent of the badness. >> > >> > If you try a setclientid with a client name matching an >> > already-established client with state, then nfsd4_setclientid() should >> > be returning CLID_INUSE: >> >> This is not what I see. I see that the 2nd SETCLIENTID succeed. Then >> the RENEW from the 1st client gets ERR_EXPIRED. 1st client does the >> SETCLIENTID/SETCLIENT_CONFIRM. Then when the 2nd client's RENEW is >> sent it gets ERR_EXPIRED and it sends SETCLIENTID. >> >> > >> > if (conf && client_has_state(conf)) { >> > ... >> > status = nfserr_clid_inuse; >> > ... >> > if (!same_creds(&conf->cl_cred, &rqstp-.rq_cred)) { >> > ... >> > goto out; >> > } >> > } >> > >> > So if you're seeing SETCLIENTID succeed then maybe same_creds() or >> > client_has_state() is failing. >> > >> > Maybe client_has_state()?--that will fail (and allow the setclientid) if >> > the v4.0 client doesn't currently have any opens or delegations. >> > >> > I think that's correct: >> > >> > https://tools.ietf.org/html/rfc7530#section-9.1.2 >> > >> > when the server gets a SETCLIENTID for a client ID that >> > currently has no state, or it has state but the lease has >> > expired, rather than returning NFS4ERR_CLID_INUSE, the server >> > MUST allow the SETCLIENTID and confirm the new client ID if >> > followed by the appropriate SETCLIENTID_CONFIRM. >> >> At the time the "competing" SETCLIENTID arrives the other client >> definitely does not have an expired lease. > > Right, but does it have opens or delegations? That's what I took as the > definition of "state" for the code above. Could be wrong, I don't know. > > --b. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 29, 2018 at 4:36 PM, J. Bruce Fields <bfields@fieldses.org> wrote: > On Tue, May 29, 2018 at 04:14:09PM -0400, Olga Kornievskaia wrote: >> On Tue, May 29, 2018 at 3:56 PM, J. Bruce Fields <bfields@fieldses.org> wrote: >> > On Tue, May 22, 2018 at 03:36:12PM -0700, Chuck Lever wrote: >> >> > On May 22, 2018, at 3:11 PM, Olga Kornievskaia <aglo@umich.edu> wrote: >> >> >> If an AUTH_UNIX client can tamper with a lease established >> >> >> by an AUTH_GSS client, that's a pretty serious server bug. >> >> >> >> >> >> Which server implementation is this? >> >> > >> >> > This is linux 4.16-rc1. >> >> >> >> Would it be easy for you confirm if two AUTH_GSS clients are >> >> appropriately protected from each other? It would be good to >> >> file a bug on bugzilla.linux-nfs.org to document the full >> >> extent of the badness. >> > >> > If you try a setclientid with a client name matching an >> > already-established client with state, then nfsd4_setclientid() should >> > be returning CLID_INUSE: >> >> This is not what I see. I see that the 2nd SETCLIENTID succeed. Then >> the RENEW from the 1st client gets ERR_EXPIRED. 1st client does the >> SETCLIENTID/SETCLIENT_CONFIRM. Then when the 2nd client's RENEW is >> sent it gets ERR_EXPIRED and it sends SETCLIENTID. >> >> > >> > if (conf && client_has_state(conf)) { >> > ... >> > status = nfserr_clid_inuse; >> > ... >> > if (!same_creds(&conf->cl_cred, &rqstp-.rq_cred)) { >> > ... >> > goto out; >> > } >> > } >> > >> > So if you're seeing SETCLIENTID succeed then maybe same_creds() or >> > client_has_state() is failing. >> > >> > Maybe client_has_state()?--that will fail (and allow the setclientid) if >> > the v4.0 client doesn't currently have any opens or delegations. >> > >> > I think that's correct: >> > >> > https://tools.ietf.org/html/rfc7530#section-9.1.2 >> > >> > when the server gets a SETCLIENTID for a client ID that >> > currently has no state, or it has state but the lease has >> > expired, rather than returning NFS4ERR_CLID_INUSE, the server >> > MUST allow the SETCLIENTID and confirm the new client ID if >> > followed by the appropriate SETCLIENTID_CONFIRM. >> >> At the time the "competing" SETCLIENTID arrives the other client >> definitely does not have an expired lease. > > Right, but does it have opens or delegations? That's what I took as the > definition of "state" for the code above. Could be wrong, I don't know. Ah I see. no I didn't have anything opens. I just did mounts. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/utils/mount/stropts.c b/utils/mount/stropts.c index 1217823..982927e 100644 --- a/utils/mount/stropts.c +++ b/utils/mount/stropts.c @@ -242,11 +242,21 @@ static int nfs_append_clientaddr_option(const struct sockaddr *sap, struct sockaddr *my_addr = &address.sa; socklen_t my_len = sizeof(address); - if (po_contains(options, "clientaddr") == PO_FOUND) - return 1; - nfs_callback_address(sap, salen, my_addr, &my_len); + if (po_contains(options, "clientaddr") == PO_FOUND) { + char *addr = po_get(options, "clientaddr"); + char address[NI_MAXHOST]; + + if (!nfs_present_sockaddr(my_addr, my_len, address, + sizeof(address))) + goto out; + + if (strcmp(addr, address)) + goto out; + return 1; + } +out: return nfs_append_generic_address_option(my_addr, my_len, "clientaddr", options);