diff mbox

[GIT,PULL] Please pull NFS client changes for Linux 4.13

Message ID 1500245845.13893.3.camel@primarydata.com (mailing list archive)
State New, archived
Headers show

Commit Message

Trond Myklebust July 16, 2017, 10:57 p.m. UTC
Hi Dave,

On Sun, 2017-07-16 at 17:15 -0400, Dave Jones wrote:
> On Fri, Jul 14, 2017 at 10:25:43AM -0400, Dave Jones wrote:
>  > On Thu, Jul 13, 2017 at 05:16:24PM -0400, Anna Schumaker wrote:
>  >  > Hi Linus,
>  >  > 
>  >  > The following changes since commit
> 32c1431eea4881a6b17bd7c639315010aeefa452:
>  >  > 
>  >  >   Linux 4.12-rc5 (2017-06-11 16:48:20 -0700)
>  >  > 
>  >  > are available in the git repository at:
>  >  > 
>  >  >   git://git.linux-nfs.org/projects/anna/linux-nfs.git tags/nfs-
> for-4.13-1
>  >  > 
>  >  > for you to fetch changes up to
> b4f937cffa66b3d56eb8f586e620d0b223a281a3:
>  >  > 
>  >  >   NFS: Don't run wake_up_bit() when nobody is waiting... (2017-
> 07-13 16:57:18 -0400)
>  > 
>  > Since this landed, I'm seeing this during boot..
>  > 
>  >  =================================================================
> =
>  >  BUG: KASAN: global-out-of-bounds in strscpy+0x4a/0x230
>  >  Read of size 8 at addr ffffffffb4eeaf20 by task nfsd/688
> 
> Now that this one got fixed, this one fell out instead..
> Will dig deeper tomorrow.
> 
> ==================================================================
> BUG: KASAN: global-out-of-bounds in call_start+0x93/0x100
> Read of size 8 at addr ffffffff8d582588 by task kworker/0:1/22
> 
> CPU: 0 PID: 22 Comm: kworker/0:1 Not tainted 4.13.0-rc1-firewall+ #1 
> Workqueue: rpciod rpc_async_schedule
> Call Trace:
>  dump_stack+0x68/0x94
>  print_address_description+0x2c/0x270
>  ? call_start+0x93/0x100
>  kasan_report+0x239/0x350
>  __asan_load8+0x55/0x90
>  call_start+0x93/0x100
>  ? rpc_default_callback+0x10/0x10
>  ? rpc_default_callback+0x10/0x10
>  __rpc_execute+0x170/0x740
>  ? rpc_wake_up_queued_task+0x50/0x50
>  ? __lock_is_held+0x9f/0x110
>  rpc_async_schedule+0x12/0x20
>  process_one_work+0x4ba/0xb10
>  ? process_one_work+0x401/0xb10
>  ? pwq_dec_nr_in_flight+0x120/0x120
>  worker_thread+0x91/0x670
>  ? __sched_text_start+0x8/0x8
>  kthread+0x1ab/0x200
>  ? process_one_work+0xb10/0xb10
>  ? __kthread_create_on_node+0x340/0x340
>  ret_from_fork+0x27/0x40
> 
> The buggy address belongs to the variable:
>  nfs_cb_version+0x8/0x740

Does the following patch fix it?

Cheers
  Trond

8<--------------------------------------
From b9230cdfbbee90178a1318d20cd3373ffb758788 Mon Sep 17 00:00:00 2001
From: Trond Myklebust <trond.myklebust@primarydata.com>
Date: Sun, 16 Jul 2017 18:52:18 -0400
Subject: [PATCH] nfsd: Fix a memory scribble in the callback channel

The offset of the entry in struct rpc_version has to match the version
number.

Reported-by: Dave Jones <davej@codemonkey.org.uk>
Fixes: 1c5876ddbdb4 ("sunrpc: move p_count out of struct rpc_procinfo")
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfsd/nfs4callback.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

-- 
2.13.3

-- 
Trond Myklebust
Linux NFS client maintainer, PrimaryData
trond.myklebust@primarydata.com

Comments

Dave Jones July 17, 2017, 3:05 a.m. UTC | #1
On Sun, Jul 16, 2017 at 10:57:27PM +0000, Trond Myklebust wrote:
 
 > > BUG: KASAN: global-out-of-bounds in call_start+0x93/0x100
 > > Read of size 8 at addr ffffffff8d582588 by task kworker/0:1/22
 > 
 > Does the following patch fix it?

Yep, seems to do the trick!

	Dave
Linus Torvalds July 17, 2017, 7:02 p.m. UTC | #2
On Sun, Jul 16, 2017 at 8:05 PM, davej@codemonkey.org.uk
<davej@codemonkey.org.uk> wrote:
> On Sun, Jul 16, 2017 at 10:57:27PM +0000, Trond Myklebust wrote:
>
>  > > BUG: KASAN: global-out-of-bounds in call_start+0x93/0x100
>  > > Read of size 8 at addr ffffffff8d582588 by task kworker/0:1/22
>  >
>  > Does the following patch fix it?
>
> Yep, seems to do the trick!

I'm assuming I'll get this fix through a future pull request, and am
not applying the patch as-is. Just FYI.

                     Linus
J. Bruce Fields July 18, 2017, 2:20 p.m. UTC | #3
Please pull:

  git://linux-nfs.org/~bfields/linux.git tags/nfsd-4.13-1

One fix for a problem introduced in the most recent merge window and
found by Dave Jones and KASAN.

--b.

Trond Myklebust (1):
      nfsd: Fix a memory scribble in the callback channel

 fs/nfsd/nfs4callback.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Dave Jones July 31, 2017, 3:43 p.m. UTC | #4
Another NFSv4 KASAN splat, this time from rc3.


==================================================================
BUG: KASAN: use-after-free in nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]
Read of size 8 at addr ffff8804508af528 by task kworker/2:1/34

CPU: 2 PID: 34 Comm: kworker/2:1 Not tainted 4.13.0-rc3-think+ #1 
Workqueue: rpciod rpc_async_schedule [sunrpc]
Call Trace:
 dump_stack+0x68/0xa1
 print_address_description+0xd9/0x270
 kasan_report+0x257/0x370
 ? nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]
 check_memory_region+0x13a/0x1a0
 __asan_loadN+0xf/0x20
 nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]
 ? nfs4_exchange_id_release+0xb0/0xb0 [nfsv4]
 rpc_exit_task+0x69/0x110 [sunrpc]
 ? rpc_destroy_wait_queue+0x20/0x20 [sunrpc]
 ? rpc_destroy_wait_queue+0x20/0x20 [sunrpc]
 __rpc_execute+0x1a0/0x840 [sunrpc]
 ? rpc_wake_up_queued_task+0x50/0x50 [sunrpc]
 ? __lock_is_held+0x9a/0x100
 ? debug_lockdep_rcu_enabled.part.16+0x1a/0x30
 rpc_async_schedule+0x12/0x20 [sunrpc]
 process_one_work+0x4d5/0xa70
 ? flush_delayed_work+0x70/0x70
 ? lock_acquire+0xfc/0x220
 worker_thread+0x88/0x630
 ? pci_mmcfg_check_reserved+0xc0/0xc0
 kthread+0x1a6/0x1f0
 ? process_one_work+0xa70/0xa70
 ? kthread_create_on_node+0xc0/0xc0
 ret_from_fork+0x27/0x40

Allocated by task 1:
 save_stack_trace+0x1b/0x20
 save_stack+0x46/0xd0
 kasan_kmalloc+0xad/0xe0
 kasan_slab_alloc+0x12/0x20
 kmem_cache_alloc+0xe0/0x2f0
 getname_flags+0x43/0x220
 getname+0x12/0x20
 do_sys_open+0x14c/0x2b0
 SyS_open+0x1e/0x20
 do_syscall_64+0xea/0x260
 return_from_SYSCALL_64+0x0/0x7a

Freed by task 1:
 save_stack_trace+0x1b/0x20
 save_stack+0x46/0xd0
 kasan_slab_free+0x72/0xc0
 kmem_cache_free+0xa8/0x300
 putname+0x80/0x90
 do_sys_open+0x22f/0x2b0
 SyS_open+0x1e/0x20
 do_syscall_64+0xea/0x260
 return_from_SYSCALL_64+0x0/0x7a

The buggy address belongs to the object at ffff8804508aeac0\x0a which belongs to the cache names_cache of size 4096
The buggy address is located 2664 bytes inside of\x0a 4096-byte region [ffff8804508aeac0, ffff8804508afac0)
The buggy address belongs to the page:
page:ffffea0011422a00 count:1 mapcount:0 mapping:          (null) index:0x0
[CONT START]  compound_mapcount: 0
flags: 0x8000000000008100(slab|head)
raw: 8000000000008100 0000000000000000 0000000000000000 0000000100070007
raw: ffffea00113d6020 ffffea001136e220 ffff8804664f8040 0000000000000000
page dumped because: kasan: bad access detected

Memory state around the buggy address:
 ffff8804508af400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff8804508af480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff8804508af500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                                  ^
 ffff8804508af580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff8804508af600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================
Linus Torvalds Aug. 1, 2017, 5:35 a.m. UTC | #5
On Mon, Jul 31, 2017 at 8:43 AM, davej@codemonkey.org.uk
<davej@codemonkey.org.uk> wrote:
> Another NFSv4 KASAN splat, this time from rc3.
>
> BUG: KASAN: use-after-free in nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]

Ugh. It's really hard to tell what access that it - KASAN doesn't
actually give enough information. There's lots of 8-byte accesses
there in that function.

Any chance of getting the output from

   ./scripts/faddr2line vmlinux nfs4_exchange_id_done+0x3d7/0x8e0

or something? That would be extremely useful in general for
stacktraces, but it's doubly useful for KASAN because most *other*
stacktraces tend to have a very limited number of things that can warn
(ie there's one or two WARN_ON() calls in a function), but KASAN can
have tens or hundreds..

                       Linus


> Read of size 8 at addr ffff8804508af528 by task kworker/2:1/34
>
> CPU: 2 PID: 34 Comm: kworker/2:1 Not tainted 4.13.0-rc3-think+ #1
> Workqueue: rpciod rpc_async_schedule [sunrpc]
> Call Trace:
>  dump_stack+0x68/0xa1
>  print_address_description+0xd9/0x270
>  kasan_report+0x257/0x370
>  ? nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]
>  check_memory_region+0x13a/0x1a0
>  __asan_loadN+0xf/0x20
>  nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]
>  ? nfs4_exchange_id_release+0xb0/0xb0 [nfsv4]
>  rpc_exit_task+0x69/0x110 [sunrpc]
>  ? rpc_destroy_wait_queue+0x20/0x20 [sunrpc]
>  ? rpc_destroy_wait_queue+0x20/0x20 [sunrpc]
>  __rpc_execute+0x1a0/0x840 [sunrpc]
>  ? rpc_wake_up_queued_task+0x50/0x50 [sunrpc]
>  ? __lock_is_held+0x9a/0x100
>  ? debug_lockdep_rcu_enabled.part.16+0x1a/0x30
>  rpc_async_schedule+0x12/0x20 [sunrpc]
>  process_one_work+0x4d5/0xa70
>  ? flush_delayed_work+0x70/0x70
>  ? lock_acquire+0xfc/0x220
>  worker_thread+0x88/0x630
>  ? pci_mmcfg_check_reserved+0xc0/0xc0
>  kthread+0x1a6/0x1f0
>  ? process_one_work+0xa70/0xa70
>  ? kthread_create_on_node+0xc0/0xc0
>  ret_from_fork+0x27/0x40
>
> Allocated by task 1:
>  save_stack_trace+0x1b/0x20
>  save_stack+0x46/0xd0
>  kasan_kmalloc+0xad/0xe0
>  kasan_slab_alloc+0x12/0x20
>  kmem_cache_alloc+0xe0/0x2f0
>  getname_flags+0x43/0x220
>  getname+0x12/0x20
>  do_sys_open+0x14c/0x2b0
>  SyS_open+0x1e/0x20
>  do_syscall_64+0xea/0x260
>  return_from_SYSCALL_64+0x0/0x7a
>
> Freed by task 1:
>  save_stack_trace+0x1b/0x20
>  save_stack+0x46/0xd0
>  kasan_slab_free+0x72/0xc0
>  kmem_cache_free+0xa8/0x300
>  putname+0x80/0x90
>  do_sys_open+0x22f/0x2b0
>  SyS_open+0x1e/0x20
>  do_syscall_64+0xea/0x260
>  return_from_SYSCALL_64+0x0/0x7a
>
> The buggy address belongs to the object at ffff8804508aeac0\x0a which belongs to the cache names_cache of size 4096
> The buggy address is located 2664 bytes inside of\x0a 4096-byte region [ffff8804508aeac0, ffff8804508afac0)
> The buggy address belongs to the page:
> page:ffffea0011422a00 count:1 mapcount:0 mapping:          (null) index:0x0
> [CONT START]  compound_mapcount: 0
> flags: 0x8000000000008100(slab|head)
> raw: 8000000000008100 0000000000000000 0000000000000000 0000000100070007
> raw: ffffea00113d6020 ffffea001136e220 ffff8804664f8040 0000000000000000
> page dumped because: kasan: bad access detected
>
> Memory state around the buggy address:
>  ffff8804508af400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  ffff8804508af480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>>ffff8804508af500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>                                   ^
>  ffff8804508af580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  ffff8804508af600: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==================================================================
>
Dave Jones Aug. 1, 2017, 3:51 p.m. UTC | #6
On Mon, Jul 31, 2017 at 10:35:45PM -0700, Linus Torvalds wrote:
 > On Mon, Jul 31, 2017 at 8:43 AM, davej@codemonkey.org.uk
 > <davej@codemonkey.org.uk> wrote:
 > > Another NFSv4 KASAN splat, this time from rc3.
 > >
 > > BUG: KASAN: use-after-free in nfs4_exchange_id_done+0x3d7/0x8e0 [nfsv4]
 > 
 > Ugh. It's really hard to tell what access that it - KASAN doesn't
 > actually give enough information. There's lots of 8-byte accesses
 > there in that function.
 > 
 > Any chance of getting the output from
 > 
 >    ./scripts/faddr2line vmlinux nfs4_exchange_id_done+0x3d7/0x8e0
 

Hm, that points to this..

7463                 /* Save the EXCHANGE_ID verifier session trunk tests */
7464                 memcpy(clp->cl_confirm.data, cdata->args.verifier->data,
7465                        sizeof(clp->cl_confirm.data));

	Dave
diff mbox

Patch

diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index b45083c0f9ae..49b0a9e7ff18 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -720,8 +720,8 @@  static const struct rpc_version nfs_cb_version4 = {
 	.counts			= nfs4_cb_counts,
 };
 
-static const struct rpc_version *nfs_cb_version[] = {
-	&nfs_cb_version4,
+static const struct rpc_version *nfs_cb_version[2] = {
+	[1] = &nfs_cb_version4,
 };
 
 static const struct rpc_program cb_program;
@@ -795,7 +795,7 @@  static int setup_callback_client(struct nfs4_client *clp, struct nfs4_cb_conn *c
 		.saddress	= (struct sockaddr *) &conn->cb_saddr,
 		.timeout	= &timeparms,
 		.program	= &cb_program,
-		.version	= 0,
+		.version	= 1,
 		.flags		= (RPC_CLNT_CREATE_NOPING | RPC_CLNT_CREATE_QUIET),
 	};
 	struct rpc_clnt *client;