Patchwork scsi: fix the issue that iscsi_if_rx doesn't parse nlmsg properly

login
register
mail settings
Submitter Xin Long
Date Aug. 27, 2017, 12:25 p.m.
Message ID <0c9fd6fbc0f5fa7b72e6ae5b82d5499a38fd375e.1503836726.git.lucien.xin@gmail.com>
Download mbox | patch
Permalink /patch/9923803/
State Accepted, archived
Headers show

Comments

Xin Long - Aug. 27, 2017, 12:25 p.m.
ChunYu found a kernel crash by syzkaller:

[  651.617875] kasan: CONFIG_KASAN_INLINE enabled
[  651.618217] kasan: GPF could be caused by NULL-ptr deref or user memory access
[  651.618731] general protection fault: 0000 [#1] SMP KASAN
[  651.621543] CPU: 1 PID: 9539 Comm: scsi Not tainted 4.11.0.cov #32
[  651.621938] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[  651.622309] task: ffff880117780000 task.stack: ffff8800a3188000
[  651.622762] RIP: 0010:skb_release_data+0x26c/0x590
[...]
[  651.627260] Call Trace:
[  651.629156]  skb_release_all+0x4f/0x60
[  651.629450]  consume_skb+0x1a5/0x600
[  651.630705]  netlink_unicast+0x505/0x720
[  651.632345]  netlink_sendmsg+0xab2/0xe70
[  651.633704]  sock_sendmsg+0xcf/0x110
[  651.633942]  ___sys_sendmsg+0x833/0x980
[  651.637117]  __sys_sendmsg+0xf3/0x240
[  651.638820]  SyS_sendmsg+0x32/0x50
[  651.639048]  entry_SYSCALL_64_fastpath+0x1f/0xc2

It's caused by skb_shared_info at the end of sk_buff was overwritten by
ISCSI_KEVENT_IF_ERROR when parsing nlmsg info from skb in iscsi_if_rx.

During the loop if skb->len == nlh->nlmsg_len and both are sizeof(*nlh),
ev = nlmsg_data(nlh) will acutally get skb_shinfo(SKB) instead and set a
new value to skb_shinfo(SKB)->nr_frags by ev->type.

This patch is to fix it by checking nlh->nlmsg_len properly there to
avoid over accessing sk_buff.

Reported-by: ChunYu Wang <chunwang@redhat.com>
Signed-off-by: Xin Long <lucien.xin@gmail.com>
---
 drivers/scsi/scsi_transport_iscsi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Xin Long - Aug. 27, 2017, 12:37 p.m.
cc jejb@linux.vnet.ibm.com

On Mon, Aug 28, 2017 at 12:25 AM, Xin Long <lucien.xin@gmail.com> wrote:
> ChunYu found a kernel crash by syzkaller:
>
> [  651.617875] kasan: CONFIG_KASAN_INLINE enabled
> [  651.618217] kasan: GPF could be caused by NULL-ptr deref or user memory access
> [  651.618731] general protection fault: 0000 [#1] SMP KASAN
> [  651.621543] CPU: 1 PID: 9539 Comm: scsi Not tainted 4.11.0.cov #32
> [  651.621938] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
> [  651.622309] task: ffff880117780000 task.stack: ffff8800a3188000
> [  651.622762] RIP: 0010:skb_release_data+0x26c/0x590
> [...]
> [  651.627260] Call Trace:
> [  651.629156]  skb_release_all+0x4f/0x60
> [  651.629450]  consume_skb+0x1a5/0x600
> [  651.630705]  netlink_unicast+0x505/0x720
> [  651.632345]  netlink_sendmsg+0xab2/0xe70
> [  651.633704]  sock_sendmsg+0xcf/0x110
> [  651.633942]  ___sys_sendmsg+0x833/0x980
> [  651.637117]  __sys_sendmsg+0xf3/0x240
> [  651.638820]  SyS_sendmsg+0x32/0x50
> [  651.639048]  entry_SYSCALL_64_fastpath+0x1f/0xc2
>
> It's caused by skb_shared_info at the end of sk_buff was overwritten by
> ISCSI_KEVENT_IF_ERROR when parsing nlmsg info from skb in iscsi_if_rx.
>
> During the loop if skb->len == nlh->nlmsg_len and both are sizeof(*nlh),
> ev = nlmsg_data(nlh) will acutally get skb_shinfo(SKB) instead and set a
> new value to skb_shinfo(SKB)->nr_frags by ev->type.
>
> This patch is to fix it by checking nlh->nlmsg_len properly there to
> avoid over accessing sk_buff.
>
> Reported-by: ChunYu Wang <chunwang@redhat.com>
> Signed-off-by: Xin Long <lucien.xin@gmail.com>
> ---
>  drivers/scsi/scsi_transport_iscsi.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c
> index e4b3d8f..bb4ed7b 100644
> --- a/drivers/scsi/scsi_transport_iscsi.c
> +++ b/drivers/scsi/scsi_transport_iscsi.c
> @@ -3697,7 +3697,7 @@ iscsi_if_rx(struct sk_buff *skb)
>                 uint32_t group;
>
>                 nlh = nlmsg_hdr(skb);
> -               if (nlh->nlmsg_len < sizeof(*nlh) ||
> +               if (nlh->nlmsg_len < sizeof(*nlh) + sizeof(*ev) ||
>                     skb->len < nlh->nlmsg_len) {
>                         break;
>                 }
> --
> 2.1.0
>
Chris Leech - Sept. 13, 2017, 3:51 p.m.
Acked-by: Chris Leech <cleech@redhat.com>

On Sun, Aug 27, 2017 at 08:25:26PM +0800, Xin Long wrote:
> ChunYu found a kernel crash by syzkaller:
> 
> [  651.617875] kasan: CONFIG_KASAN_INLINE enabled
> [  651.618217] kasan: GPF could be caused by NULL-ptr deref or user memory access
> [  651.618731] general protection fault: 0000 [#1] SMP KASAN
> [  651.621543] CPU: 1 PID: 9539 Comm: scsi Not tainted 4.11.0.cov #32
> [  651.621938] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
> [  651.622309] task: ffff880117780000 task.stack: ffff8800a3188000
> [  651.622762] RIP: 0010:skb_release_data+0x26c/0x590
> [...]
> [  651.627260] Call Trace:
> [  651.629156]  skb_release_all+0x4f/0x60
> [  651.629450]  consume_skb+0x1a5/0x600
> [  651.630705]  netlink_unicast+0x505/0x720
> [  651.632345]  netlink_sendmsg+0xab2/0xe70
> [  651.633704]  sock_sendmsg+0xcf/0x110
> [  651.633942]  ___sys_sendmsg+0x833/0x980
> [  651.637117]  __sys_sendmsg+0xf3/0x240
> [  651.638820]  SyS_sendmsg+0x32/0x50
> [  651.639048]  entry_SYSCALL_64_fastpath+0x1f/0xc2
> 
> It's caused by skb_shared_info at the end of sk_buff was overwritten by
> ISCSI_KEVENT_IF_ERROR when parsing nlmsg info from skb in iscsi_if_rx.
> 
> During the loop if skb->len == nlh->nlmsg_len and both are sizeof(*nlh),
> ev = nlmsg_data(nlh) will acutally get skb_shinfo(SKB) instead and set a
> new value to skb_shinfo(SKB)->nr_frags by ev->type.
> 
> This patch is to fix it by checking nlh->nlmsg_len properly there to
> avoid over accessing sk_buff.
> 
> Reported-by: ChunYu Wang <chunwang@redhat.com>
> Signed-off-by: Xin Long <lucien.xin@gmail.com>
> ---
>  drivers/scsi/scsi_transport_iscsi.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c
> index e4b3d8f..bb4ed7b 100644
> --- a/drivers/scsi/scsi_transport_iscsi.c
> +++ b/drivers/scsi/scsi_transport_iscsi.c
> @@ -3697,7 +3697,7 @@ iscsi_if_rx(struct sk_buff *skb)
>  		uint32_t group;
>  
>  		nlh = nlmsg_hdr(skb);
> -		if (nlh->nlmsg_len < sizeof(*nlh) ||
> +		if (nlh->nlmsg_len < sizeof(*nlh) + sizeof(*ev) ||
>  		    skb->len < nlh->nlmsg_len) {
>  			break;
>  		}
> -- 
> 2.1.0
>
Vladis Dronov - Sept. 25, 2017, 10:56 a.m.
hello,

an additional research shows that the very latest kernels are not showing
a crash with a reproducer. git bisect showed that:

commit 7f564528a480084e2318cd48caba7aef4a54a77f is the first commit (between
v4.11 and v4.12-rc1) a crash is not reproduced with:

commit 7f564528a480084e2318cd48caba7aef4a54a77f
Author: Steffen Klassert <steffen.klassert@secunet.com>
Date:   Sat Apr 8 20:36:24 2017 +0200
skbuff: Extend gso_type to unsigned int.

i.e. this is commit which fixed the crash. checking the code, it looks like
struct skb_shared_info's fields were reordered, so a field which overwrite
was causing a panic has been moved. nevertheless, the buffer overwrite is still
there, so a suggested patch 9923803 (or its later version) is still needed.

for a proof compare a flaw description:

> ev = nlmsg_data(nlh) will acutally get skb_shinfo(SKB) instead and set a
> new value to skb_shinfo(SKB)->nr_frags by ev->type.

and the commit message:

>    The remaining two byte hole is moved to the
>    beginning of the structure, this protects us
>    from immediate overwites on out of bound writes
>    to the sk_buff head.
> 
>    Structure layout on x86-64 before the change:
> 
>    struct skb_shared_info {
>            unsigned char              nr_frags;
>            __u8                       tx_flags;
> 
>    Structure layout on x86-64 after the change:
> 
>    struct skb_shared_info {
>            short unsigned int         _unused;
>            unsigned char              nr_frags;
>            __u8                       tx_flags;

Best regards,
Vladis Dronov | Red Hat, Inc. | Product Security Engineer

----- Original Message -----
From: Xin Long <lucien.xin@gmail.com>
To: linux-scsi@vger.kernel.org
Sent: Sun, 27 Aug 2017 20:25:26 +0800
Subject: scsi: fix the issue that iscsi_if_rx doesn't parse nlmsg properly

> ChunYu found a kernel crash by syzkaller:
> 
> [  651.617875] kasan: CONFIG_KASAN_INLINE enabled
> [  651.618217] kasan: GPF could be caused by NULL-ptr deref or user memory access
> [  651.618731] general protection fault: 0000 [#1] SMP KASAN
> [  651.621543] CPU: 1 PID: 9539 Comm: scsi Not tainted 4.11.0.cov #32
> [  651.621938] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
> [  651.622309] task: ffff880117780000 task.stack: ffff8800a3188000
> [  651.622762] RIP: 0010:skb_release_data+0x26c/0x590
Martin Petersen - Sept. 25, 2017, 7:28 p.m.
Xin,

> ChunYu found a kernel crash by syzkaller:

[...]

> It's caused by skb_shared_info at the end of sk_buff was overwritten by
> ISCSI_KEVENT_IF_ERROR when parsing nlmsg info from skb in iscsi_if_rx.
>
> During the loop if skb->len == nlh->nlmsg_len and both are sizeof(*nlh),
> ev = nlmsg_data(nlh) will acutally get skb_shinfo(SKB) instead and set a
> new value to skb_shinfo(SKB)->nr_frags by ev->type.
>
> This patch is to fix it by checking nlh->nlmsg_len properly there to
> avoid over accessing sk_buff.

Applied to 4.14/scsi-fixes. Thank you!
Ewan Milne - Sept. 29, 2017, 1:33 p.m.
On Mon, 2017-09-25 at 15:28 -0400, Martin K. Petersen wrote:
> Xin,
> 
> > ChunYu found a kernel crash by syzkaller:
> 
> [...]
> 
> > It's caused by skb_shared_info at the end of sk_buff was overwritten by
> > ISCSI_KEVENT_IF_ERROR when parsing nlmsg info from skb in iscsi_if_rx.
> >
> > During the loop if skb->len == nlh->nlmsg_len and both are sizeof(*nlh),
> > ev = nlmsg_data(nlh) will acutally get skb_shinfo(SKB) instead and set a
> > new value to skb_shinfo(SKB)->nr_frags by ev->type.
> >
> > This patch is to fix it by checking nlh->nlmsg_len properly there to
> > avoid over accessing sk_buff.
> 
> Applied to 4.14/scsi-fixes. Thank you!
> 

Should this be considered for -stable?  (Despite not being reproduced
after 7f564528a4).

Patch

diff --git a/drivers/scsi/scsi_transport_iscsi.c b/drivers/scsi/scsi_transport_iscsi.c
index e4b3d8f..bb4ed7b 100644
--- a/drivers/scsi/scsi_transport_iscsi.c
+++ b/drivers/scsi/scsi_transport_iscsi.c
@@ -3697,7 +3697,7 @@  iscsi_if_rx(struct sk_buff *skb)
 		uint32_t group;
 
 		nlh = nlmsg_hdr(skb);
-		if (nlh->nlmsg_len < sizeof(*nlh) ||
+		if (nlh->nlmsg_len < sizeof(*nlh) + sizeof(*ev) ||
 		    skb->len < nlh->nlmsg_len) {
 			break;
 		}