| Message ID | 20240910030713.3716318-1-chao@kernel.org (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 930c6ab93492c4b15436524e704950b364b2930c |
| Headers | show |
| Series | [f2fs-dev] f2fs: fix to don't set SB_RDONLY in f2fs_handle_critical_error() | expand |
On Tue, Sep 10, 2024 at 11:07:13AM GMT, Chao Yu wrote: > syzbot reports a f2fs bug as below: > > ------------[ cut here ]------------ > WARNING: CPU: 1 PID: 58 at kernel/rcu/sync.c:177 rcu_sync_dtor+0xcd/0x180 kernel/rcu/sync.c:177 > CPU: 1 UID: 0 PID: 58 Comm: kworker/1:2 Not tainted 6.10.0-syzkaller-12562-g1722389b0d86 #0 > Workqueue: events destroy_super_work > RIP: 0010:rcu_sync_dtor+0xcd/0x180 kernel/rcu/sync.c:177 > Call Trace: > percpu_free_rwsem+0x41/0x80 kernel/locking/percpu-rwsem.c:42 > destroy_super_work+0xec/0x130 fs/super.c:282 > process_one_work kernel/workqueue.c:3231 [inline] > process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312 > worker_thread+0x86d/0xd40 kernel/workqueue.c:3390 > kthread+0x2f0/0x390 kernel/kthread.c:389 > ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 > > As Christian Brauner pointed out [1]: the root cause is f2fs sets > SB_RDONLY flag in internal function, rather than setting the flag > covered w/ sb->s_umount semaphore via remount procedure, then below > race condition causes this bug: > > - freeze_super() > - sb_wait_write(sb, SB_FREEZE_WRITE) > - sb_wait_write(sb, SB_FREEZE_PAGEFAULT) > - sb_wait_write(sb, SB_FREEZE_FS) > - f2fs_handle_critical_error > - sb->s_flags |= SB_RDONLY > - thaw_super > - thaw_super_locked > - sb_rdonly() is true, so it skips > sb_freeze_unlock(sb, SB_FREEZE_FS) > - deactivate_locked_super > > Since f2fs has almost the same logic as ext4 [2] when handling critical > error in filesystem if it mounts w/ errors=remount-ro option: > - set CP_ERROR_FLAG flag which indicates filesystem is stopped > - record errors to superblock > - set SB_RDONLY falg > Once we set CP_ERROR_FLAG flag, all writable interfaces can detect the > flag and stop any further updates on filesystem. So, it is safe to not > set SB_RDONLY flag, let's remove the logic and keep in line w/ ext4 [3]. > > [1] https://lore.kernel.org/all/20240729-himbeeren-funknetz-96e62f9c7aee@brauner > [2] https://lore.kernel.org/all/20240729132721.hxih6ehigadqf7wx@quack3 > [3] https://lore.kernel.org/linux-ext4/20240805201241.27286-1-jack@suse.cz > > Fixes: b62e71be2110 ("f2fs: support errors=remount-ro|continue|panic mountoption") > Reported-by: syzbot+20d7e439f76bbbd863a7@syzkaller.appspotmail.com > Closes: https://lore.kernel.org/all/000000000000b90a8e061e21d12f@google.com/ > Cc: Jan Kara <jack@suse.cz> > Cc: Christian Brauner <brauner@kernel.org> > Signed-off-by: Chao Yu <chao@kernel.org> > --- Thanks! Reviewed-by: Christian Brauner <brauner@kernel.org>
Another problem is that it sets SB_RDONLY instead of an internal shutdown flag. But that can be solved later.
Hello: This patch was applied to jaegeuk/f2fs.git (dev) by Jaegeuk Kim <jaegeuk@kernel.org>: On Tue, 10 Sep 2024 11:07:13 +0800 you wrote: > syzbot reports a f2fs bug as below: > > ------------[ cut here ]------------ > WARNING: CPU: 1 PID: 58 at kernel/rcu/sync.c:177 rcu_sync_dtor+0xcd/0x180 kernel/rcu/sync.c:177 > CPU: 1 UID: 0 PID: 58 Comm: kworker/1:2 Not tainted 6.10.0-syzkaller-12562-g1722389b0d86 #0 > Workqueue: events destroy_super_work > RIP: 0010:rcu_sync_dtor+0xcd/0x180 kernel/rcu/sync.c:177 > Call Trace: > percpu_free_rwsem+0x41/0x80 kernel/locking/percpu-rwsem.c:42 > destroy_super_work+0xec/0x130 fs/super.c:282 > process_one_work kernel/workqueue.c:3231 [inline] > process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312 > worker_thread+0x86d/0xd40 kernel/workqueue.c:3390 > kthread+0x2f0/0x390 kernel/kthread.c:389 > ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 > > [...] Here is the summary with links: - [f2fs-dev] f2fs: fix to don't set SB_RDONLY in f2fs_handle_critical_error() https://git.kernel.org/jaegeuk/f2fs/c/930c6ab93492 You are awesome, thank you!
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index e774bdf875b2..acfd000c6bb0 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -4207,12 +4207,14 @@ void f2fs_handle_critical_error(struct f2fs_sb_info *sbi, unsigned char reason, } f2fs_warn(sbi, "Remounting filesystem read-only"); + /* - * Make sure updated value of ->s_mount_flags will be visible before - * ->s_flags update + * We have already set CP_ERROR_FLAG flag to stop all updates + * to filesystem, so it doesn't need to set SB_RDONLY flag here + * because the flag should be set covered w/ sb->s_umount semaphore + * via remount procedure, otherwise, it will confuse code like + * freeze_super() which will lead to deadlocks and other problems. */ - smp_wmb(); - sb->s_flags |= SB_RDONLY; } static void f2fs_record_error_work(struct work_struct *work)
syzbot reports a f2fs bug as below: ------------[ cut here ]------------ WARNING: CPU: 1 PID: 58 at kernel/rcu/sync.c:177 rcu_sync_dtor+0xcd/0x180 kernel/rcu/sync.c:177 CPU: 1 UID: 0 PID: 58 Comm: kworker/1:2 Not tainted 6.10.0-syzkaller-12562-g1722389b0d86 #0 Workqueue: events destroy_super_work RIP: 0010:rcu_sync_dtor+0xcd/0x180 kernel/rcu/sync.c:177 Call Trace: percpu_free_rwsem+0x41/0x80 kernel/locking/percpu-rwsem.c:42 destroy_super_work+0xec/0x130 fs/super.c:282 process_one_work kernel/workqueue.c:3231 [inline] process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312 worker_thread+0x86d/0xd40 kernel/workqueue.c:3390 kthread+0x2f0/0x390 kernel/kthread.c:389 ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 As Christian Brauner pointed out [1]: the root cause is f2fs sets SB_RDONLY flag in internal function, rather than setting the flag covered w/ sb->s_umount semaphore via remount procedure, then below race condition causes this bug: - freeze_super() - sb_wait_write(sb, SB_FREEZE_WRITE) - sb_wait_write(sb, SB_FREEZE_PAGEFAULT) - sb_wait_write(sb, SB_FREEZE_FS) - f2fs_handle_critical_error - sb->s_flags |= SB_RDONLY - thaw_super - thaw_super_locked - sb_rdonly() is true, so it skips sb_freeze_unlock(sb, SB_FREEZE_FS) - deactivate_locked_super Since f2fs has almost the same logic as ext4 [2] when handling critical error in filesystem if it mounts w/ errors=remount-ro option: - set CP_ERROR_FLAG flag which indicates filesystem is stopped - record errors to superblock - set SB_RDONLY falg Once we set CP_ERROR_FLAG flag, all writable interfaces can detect the flag and stop any further updates on filesystem. So, it is safe to not set SB_RDONLY flag, let's remove the logic and keep in line w/ ext4 [3]. [1] https://lore.kernel.org/all/20240729-himbeeren-funknetz-96e62f9c7aee@brauner [2] https://lore.kernel.org/all/20240729132721.hxih6ehigadqf7wx@quack3 [3] https://lore.kernel.org/linux-ext4/20240805201241.27286-1-jack@suse.cz Fixes: b62e71be2110 ("f2fs: support errors=remount-ro|continue|panic mountoption") Reported-by: syzbot+20d7e439f76bbbd863a7@syzkaller.appspotmail.com Closes: https://lore.kernel.org/all/000000000000b90a8e061e21d12f@google.com/ Cc: Jan Kara <jack@suse.cz> Cc: Christian Brauner <brauner@kernel.org> Signed-off-by: Chao Yu <chao@kernel.org> --- fs/f2fs/super.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)