From patchwork Wed May 3 11:44:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 9709493 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AA6716021C for ; Wed, 3 May 2017 11:45:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89BDD28469 for ; Wed, 3 May 2017 11:45:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B7F728609; Wed, 3 May 2017 11:45:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_TVD_MIME_EPI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D9E728469 for ; Wed, 3 May 2017 11:45:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750981AbdECLo7 (ORCPT ); Wed, 3 May 2017 07:44:59 -0400 Received: from mx2.suse.de ([195.135.220.15]:34206 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751264AbdECLo6 (ORCPT ); Wed, 3 May 2017 07:44:58 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 3B617AB09; Wed, 3 May 2017 11:44:56 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 2BC5B1E34C6; Wed, 3 May 2017 13:44:54 +0200 (CEST) Date: Wed, 3 May 2017 13:44:54 +0200 From: Jan Kara To: Corentin Labbe Cc: Jan Kara , anna.schumaker@netapp.com, hch@lst.de, trond.myklebust@primarydata.com, axboe@fb.com, linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [BUG] cannot mount nfs, bisected to 0db10944a76ba ("nfs: Convert to separately allocated bdi") Message-ID: <20170503114454.GE10084@quack2.suse.cz> References: <20170428095624.GA22965@Red> <20170502162704.GM13916@quack2.suse.cz> <20170502172222.GA28711@Red> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170502172222.GA28711@Red> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Tue 02-05-17 19:22:22, Corentin Labbe wrote: > On Tue, May 02, 2017 at 06:27:04PM +0200, Jan Kara wrote: > > Hello, > > > > On Fri 28-04-17 11:56:24, Corentin Labbe wrote: > > > Since linux next-20170421, mounting nfs give me: > > > [ 774.994934] ------------[ cut here ]------------ > > > [ 774.994975] WARNING: CPU: 1 PID: 10284 at /linux-next/fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x74 > > > [ 774.994985] sysfs: cannot create duplicate filename '/devices/virtual/bdi/0:32' > > > [ 774.994992] Modules linked in: axp20x_usb_power gpio_axp209 nvmem_sunxi_sid sun4i_dma sun4i_ss virt_dma > > > [ 774.995047] CPU: 1 PID: 10284 Comm: mount.nfs Not tainted 4.11.0-rc4+ #14 > > > [ 774.995054] Hardware name: Allwinner sun7i (A20) Family > > > [ 774.995085] [] (unwind_backtrace) from [] (show_stack+0x10/0x14) > > > [ 774.995104] [] (show_stack) from [] (dump_stack+0x78/0x8c) > > > [ 774.995121] [] (dump_stack) from [] (__warn+0xe8/0x100) > > > [ 774.995135] [] (__warn) from [] (warn_slowpath_fmt+0x38/0x48) > > > [ 774.995150] [] (warn_slowpath_fmt) from [] (sysfs_warn_dup+0x64/0x74) > > > [ 774.995167] [] (sysfs_warn_dup) from [] (sysfs_create_dir_ns+0x84/0x94) > > > [ 774.995184] [] (sysfs_create_dir_ns) from [] (kobject_add_internal+0x9c/0x2ec) > > > [ 774.995199] [] (kobject_add_internal) from [] (kobject_add+0x48/0x98) > > > [ 774.995217] [] (kobject_add) from [] (device_add+0xe4/0x5a0) > > > [ 774.995232] [] (device_add) from [] (device_create_groups_vargs+0xac/0xbc) > > > [ 774.995247] [] (device_create_groups_vargs) from [] (device_create_vargs+0x20/0x28) > > > [ 774.995263] [] (device_create_vargs) from [] (bdi_register_va+0x44/0xfc) > > > [ 774.995280] [] (bdi_register_va) from [] (super_setup_bdi_name+0x48/0xa4) > > > [ 774.995299] [] (super_setup_bdi_name) from [] (nfs_fill_super+0x1a4/0x204) > > > [ 774.995315] [] (nfs_fill_super) from [] (nfs_fs_mount_common+0x140/0x1e8) > > > [ 774.995333] [] (nfs_fs_mount_common) from [] (nfs4_remote_mount+0x50/0x58) > > > [ 774.995349] [] (nfs4_remote_mount) from [] (mount_fs+0x14/0xa4) > > > [ 774.995368] [] (mount_fs) from [] (vfs_kern_mount+0x54/0x128) > > > [ 774.995385] [] (vfs_kern_mount) from [] (nfs_do_root_mount+0x80/0xa0) > > > [ 774.995400] [] (nfs_do_root_mount) from [] (nfs4_try_mount+0x28/0x3c) > > > [ 774.995415] [] (nfs4_try_mount) from [] (nfs_fs_mount+0x2cc/0x8c4) > > > [ 774.995430] [] (nfs_fs_mount) from [] (mount_fs+0x14/0xa4) > > > [ 774.995445] [] (mount_fs) from [] (vfs_kern_mount+0x54/0x128) > > > [ 774.995461] [] (vfs_kern_mount) from [] (do_mount+0x158/0xc7c) > > > [ 774.995475] [] (do_mount) from [] (SyS_mount+0x8c/0xb4) > > > [ 774.995491] [] (SyS_mount) from [] (ret_fast_syscall+0x0/0x3c) > > > [ 774.995501] ---[ end trace 0665e451f8864ff0 ]--- > > ... > > > The mount command is > > > mount -t nfs -o tcp,hard,intr,async,rsize=4096,wsize=4096 192.168.1.100:/mnt/local_kernel /usr/src/ > > > > > > the mount command failling with: "mount.nfs: Cannot allocate memory" > > > > I've tried reproducing this (both with NFSv3 and NFSv4) and failed. Also I > > have looked through the code and I fail to see how this could happen. Is > > this the only NFS mount that you have on your system? Didn't also the > > WARN_ON in super_setup_bdi_name() trigger? > > > > Can you run with the attached debug patch and post full dmesg after the > > failure? > > > > Hello > > I forgot to said that I have 6 nfs mount point. > mount -t nfs -o ro,tcp,hard,intr,async,vers=3 192.168.1.100:/usr/portage /usr/portage/ > mount -t nfs -o tcp,hard,intr,async,vers=3 192.168.1.100:/usr/local/portage /usr/local/portage/ > mount -t nfs -o tcp,hard,intr,async,vers=3 192.168.1.100:/mnt/tempo/portages/distfiles /usr/portage/distfiles > mount -t nfs -o tcp,hard,intr,async,vers=4 192.168.1.100:/mnt/tempo/portages/cubiedev/packages /usr/portage/packages > mount -t nfs -o tcp,hard,intr,async,vers=4,rsize=4096,wsize=4096 192.168.1.100:/var/tmp/portage/cubie /var/tmp/portage/ > mount -t nfs -o tcp,hard,intr,async,rsize=4096,wsize=4096 192.168.1.100:/mnt/tempo/cubie/local_kernel /usr/src/ > > It's always the last which fail, but strangely inverting the two last > made everything ok. Thanks for more info and the debug data. I still was not able to reproduce the issue but after pondering my head on this for some time I have found a bug in NFS conversion which can possibly cause the behavior you observe. Does the attached patch fix the problem for you? If it does not, can you also post /proc/mounts and /etc/exports from you NFS server? Honza From 3367ba6af234038874d0b8e0a1638d3451ee9b4d Mon Sep 17 00:00:00 2001 From: Jan Kara Date: Wed, 3 May 2017 13:02:34 +0200 Subject: [PATCH] nfs: Fix bdi handling for cloned superblocks In commit 0d3b12584972 "nfs: Convert to separately allocated bdi" I have wrongly cloned bdi reference in nfs_clone_super(). Further inspection has shown that originally the code was actually allocating a new bdi (in ->clone_server callback) which was later registered in nfs_fs_mount_common() and used for sb->s_bdi in nfs_initialise_sb(). This could later result in bdi for the original superblock not getting unregistered when that superblock got shutdown (as the cloned sb still held bdi reference) and later when a new superblock was created under the same anonymous device number, a clash in sysfs has happened on bdi registration: ------------[ cut here ]------------ WARNING: CPU: 1 PID: 10284 at /linux-next/fs/sysfs/dir.c:31 sysfs_warn_dup+0x64/0x74 sysfs: cannot create duplicate filename '/devices/virtual/bdi/0:32' Modules linked in: axp20x_usb_power gpio_axp209 nvmem_sunxi_sid sun4i_dma sun4i_ss virt_dma CPU: 1 PID: 10284 Comm: mount.nfs Not tainted 4.11.0-rc4+ #14 Hardware name: Allwinner sun7i (A20) Family [] (unwind_backtrace) from [] (show_stack+0x10/0x14) [] (show_stack) from [] (dump_stack+0x78/0x8c) [] (dump_stack) from [] (__warn+0xe8/0x100) [] (__warn) from [] (warn_slowpath_fmt+0x38/0x48) [] (warn_slowpath_fmt) from [] (sysfs_warn_dup+0x64/0x74) [] (sysfs_warn_dup) from [] (sysfs_create_dir_ns+0x84/0x94) [] (sysfs_create_dir_ns) from [] (kobject_add_internal+0x9c/0x2ec) [] (kobject_add_internal) from [] (kobject_add+0x48/0x98) [] (kobject_add) from [] (device_add+0xe4/0x5a0) [] (device_add) from [] (device_create_groups_vargs+0xac/0xbc) [] (device_create_groups_vargs) from [] (device_create_vargs+0x20/0x28) [] (device_create_vargs) from [] (bdi_register_va+0x44/0xfc) [] (bdi_register_va) from [] (super_setup_bdi_name+0x48/0xa4) [] (super_setup_bdi_name) from [] (nfs_fill_super+0x1a4/0x204) [] (nfs_fill_super) from [] (nfs_fs_mount_common+0x140/0x1e8) [] (nfs_fs_mount_common) from [] (nfs4_remote_mount+0x50/0x58) [] (nfs4_remote_mount) from [] (mount_fs+0x14/0xa4) [] (mount_fs) from [] (vfs_kern_mount+0x54/0x128) [] (vfs_kern_mount) from [] (nfs_do_root_mount+0x80/0xa0) [] (nfs_do_root_mount) from [] (nfs4_try_mount+0x28/0x3c) [] (nfs4_try_mount) from [] (nfs_fs_mount+0x2cc/0x8c4) [] (nfs_fs_mount) from [] (mount_fs+0x14/0xa4) [] (mount_fs) from [] (vfs_kern_mount+0x54/0x128) [] (vfs_kern_mount) from [] (do_mount+0x158/0xc7c) [] (do_mount) from [] (SyS_mount+0x8c/0xb4) [] (SyS_mount) from [] (ret_fast_syscall+0x0/0x3c) Fix the problem by always creating new bdi for a superblock as we used to do. Reported-by: Corentin Labbe Fixes: 0d3b12584972ce5781179ad3f15cca3cdb5cae05 Signed-off-by: Jan Kara --- fs/nfs/internal.h | 6 +++--- fs/nfs/super.c | 28 ++++++++++------------------ 2 files changed, 13 insertions(+), 21 deletions(-) diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 9dc65d7ae754..7b38fedb7e03 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -139,7 +139,7 @@ struct nfs_mount_request { }; struct nfs_mount_info { - int (*fill_super)(struct super_block *, struct nfs_mount_info *); + void (*fill_super)(struct super_block *, struct nfs_mount_info *); int (*set_security)(struct super_block *, struct dentry *, struct nfs_mount_info *); struct nfs_parsed_mount_data *parsed; struct nfs_clone_mount *cloned; @@ -407,7 +407,7 @@ struct dentry *nfs_fs_mount(struct file_system_type *, int, const char *, void * struct dentry * nfs_xdev_mount_common(struct file_system_type *, int, const char *, struct nfs_mount_info *); void nfs_kill_super(struct super_block *); -int nfs_fill_super(struct super_block *, struct nfs_mount_info *); +void nfs_fill_super(struct super_block *, struct nfs_mount_info *); extern struct rpc_stat nfs_rpcstat; @@ -458,7 +458,7 @@ extern void nfs_read_prepare(struct rpc_task *task, void *calldata); extern void nfs_pageio_reset_read_mds(struct nfs_pageio_descriptor *pgio); /* super.c */ -int nfs_clone_super(struct super_block *, struct nfs_mount_info *); +void nfs_clone_super(struct super_block *, struct nfs_mount_info *); void nfs_umount_begin(struct super_block *); int nfs_statfs(struct dentry *, struct kstatfs *); int nfs_show_options(struct seq_file *, struct dentry *); diff --git a/fs/nfs/super.c b/fs/nfs/super.c index dc69314d455e..2f3822a4a7d5 100644 --- a/fs/nfs/super.c +++ b/fs/nfs/super.c @@ -2321,11 +2321,10 @@ inline void nfs_initialise_sb(struct super_block *sb) /* * Finish setting up an NFS2/3 superblock */ -int nfs_fill_super(struct super_block *sb, struct nfs_mount_info *mount_info) +void nfs_fill_super(struct super_block *sb, struct nfs_mount_info *mount_info) { struct nfs_parsed_mount_data *data = mount_info->parsed; struct nfs_server *server = NFS_SB(sb); - int ret; sb->s_blocksize_bits = 0; sb->s_blocksize = 0; @@ -2343,21 +2342,13 @@ int nfs_fill_super(struct super_block *sb, struct nfs_mount_info *mount_info) } nfs_initialise_sb(sb); - - ret = super_setup_bdi_name(sb, "%u:%u", MAJOR(server->s_dev), - MINOR(server->s_dev)); - if (ret) - return ret; - sb->s_bdi->ra_pages = server->rpages * NFS_MAX_READAHEAD; - return 0; - } EXPORT_SYMBOL_GPL(nfs_fill_super); /* * Finish setting up a cloned NFS2/3/4 superblock */ -int nfs_clone_super(struct super_block *sb, struct nfs_mount_info *mount_info) +void nfs_clone_super(struct super_block *sb, struct nfs_mount_info *mount_info) { const struct super_block *old_sb = mount_info->cloned->sb; struct nfs_server *server = NFS_SB(sb); @@ -2377,10 +2368,6 @@ int nfs_clone_super(struct super_block *sb, struct nfs_mount_info *mount_info) } nfs_initialise_sb(sb); - - sb->s_bdi = bdi_get(old_sb->s_bdi); - - return 0; } static int nfs_compare_mount_options(const struct super_block *s, const struct nfs_server *b, int flags) @@ -2600,14 +2587,19 @@ struct dentry *nfs_fs_mount_common(struct nfs_server *server, nfs_free_server(server); server = NULL; } else { + error = super_setup_bdi_name(s, "%u:%u", MAJOR(server->s_dev), + MINOR(server->s_dev)); + if (error) { + mntroot = ERR_PTR(error); + goto error_splat_super; + } + s->s_bdi->ra_pages = server->rpages * NFS_MAX_READAHEAD; server->super = s; } if (!s->s_root) { /* initial superblock/root creation */ - error = mount_info->fill_super(s, mount_info); - if (error) - goto error_splat_super; + mount_info->fill_super(s, mount_info); nfs_get_cache_cookie(s, mount_info->parsed, mount_info->cloned); } -- 2.12.0