| Message ID | 20170704114906.8419-1-asarai@suse.de (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 4.07.2017 14:49, Aleksa Sarai wrote: > Several distributions mount the "proper root" as ro during initrd and > then remount it as rw before pivot_root(2). Thus, if a rescan had been > aborted by a previous shutdown, the rescan would never be resumed. > > This issue would manifest itself as several btrfs ioctl(2)s causing the > entire machine to hang when btrfs_qgroup_wait_for_completion was hit > (due to the fs_info->qgroup_rescan_running flag being set but the rescan > itself not being resumed). Notably, Docker's btrfs storage driver makes > regular use of BTRFS_QUOTA_CTL_DISABLE and BTRFS_IOC_QUOTA_RESCAN_WAIT > (causing this problem to be manifested on boot for some machines). > > Cc: <stable@vger.kernel.org> # v3.11+ > Cc: Jeff Mahoney <jeffm@suse.com> > Fixes: b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") > Signed-off-by: Aleksa Sarai <asarai@suse.de> Indeed, looking at the code it seems that b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") missed adding the qgroup_rescan_resume in the remount path. One thing which I couldn't verify though is whether reading fs_info->qgroup_flags without any locking is safe from remount context. During remount I don't see any locks taken that prevent operations which can modify qgroup_flags. > --- > fs/btrfs/super.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c > index 6346876c97ea..ff6690389343 100644 > --- a/fs/btrfs/super.c > +++ b/fs/btrfs/super.c > @@ -1821,6 +1821,8 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) > goto restore; > } > > + btrfs_qgroup_rescan_resume(fs_info); > + > if (!fs_info->uuid_root) { > btrfs_info(fs_info, "creating UUID tree"); > ret = btrfs_create_uuid_tree(fs_info); > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10.07.2017 16:12, Nikolay Borisov wrote: > > > On 4.07.2017 14:49, Aleksa Sarai wrote: >> Several distributions mount the "proper root" as ro during initrd and >> then remount it as rw before pivot_root(2). Thus, if a rescan had been >> aborted by a previous shutdown, the rescan would never be resumed. >> >> This issue would manifest itself as several btrfs ioctl(2)s causing the >> entire machine to hang when btrfs_qgroup_wait_for_completion was hit >> (due to the fs_info->qgroup_rescan_running flag being set but the rescan >> itself not being resumed). Notably, Docker's btrfs storage driver makes >> regular use of BTRFS_QUOTA_CTL_DISABLE and BTRFS_IOC_QUOTA_RESCAN_WAIT >> (causing this problem to be manifested on boot for some machines). >> >> Cc: <stable@vger.kernel.org> # v3.11+ >> Cc: Jeff Mahoney <jeffm@suse.com> >> Fixes: b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") >> Signed-off-by: Aleksa Sarai <asarai@suse.de> > > Indeed, looking at the code it seems that b382a324b60f ("Btrfs: fix > qgroup rescan resume on mount") missed adding the qgroup_rescan_resume > in the remount path. One thing which I couldn't verify though is whether > reading fs_info->qgroup_flags without any locking is safe from remount > context. > > During remount I don't see any locks taken that prevent operations which > can modify qgroup_flags. > > Further inspection reveals that the access rules to qgroup_flags are somewhat broken so this patch doesn't really make things any worse than they are. As such: Reviewed-by: Nikolay Borisov <nborisov@suse.com> Tested-by: Nikolay Borisov <nborisov@suse.com> > >> --- >> fs/btrfs/super.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c >> index 6346876c97ea..ff6690389343 100644 >> --- a/fs/btrfs/super.c >> +++ b/fs/btrfs/super.c >> @@ -1821,6 +1821,8 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) >> goto restore; >> } >> >> + btrfs_qgroup_rescan_resume(fs_info); >> + >> if (!fs_info->uuid_root) { >> btrfs_info(fs_info, "creating UUID tree"); >> ret = btrfs_create_uuid_tree(fs_info); >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jul 10, 2017 at 04:56:36PM +0300, Nikolay Borisov wrote: > On 10.07.2017 16:12, Nikolay Borisov wrote: > > On 4.07.2017 14:49, Aleksa Sarai wrote: > >> Several distributions mount the "proper root" as ro during initrd and > >> then remount it as rw before pivot_root(2). Thus, if a rescan had been > >> aborted by a previous shutdown, the rescan would never be resumed. > >> > >> This issue would manifest itself as several btrfs ioctl(2)s causing the > >> entire machine to hang when btrfs_qgroup_wait_for_completion was hit > >> (due to the fs_info->qgroup_rescan_running flag being set but the rescan > >> itself not being resumed). Notably, Docker's btrfs storage driver makes > >> regular use of BTRFS_QUOTA_CTL_DISABLE and BTRFS_IOC_QUOTA_RESCAN_WAIT > >> (causing this problem to be manifested on boot for some machines). > >> > >> Cc: <stable@vger.kernel.org> # v3.11+ > >> Cc: Jeff Mahoney <jeffm@suse.com> > >> Fixes: b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") > >> Signed-off-by: Aleksa Sarai <asarai@suse.de> > > > > Indeed, looking at the code it seems that b382a324b60f ("Btrfs: fix > > qgroup rescan resume on mount") missed adding the qgroup_rescan_resume > > in the remount path. One thing which I couldn't verify though is whether > > reading fs_info->qgroup_flags without any locking is safe from remount > > context. > > > > During remount I don't see any locks taken that prevent operations which > > can modify qgroup_flags. > > Further inspection reveals that the access rules to qgroup_flags are > somewhat broken so this patch doesn't really make things any worse than > they are. The usage follows a pattern for a bitfield, updated by set_bit/clear_bit etc. The updates to the state or inconsistency is not safe, so some updates could get lost under some circumstances. Patch added to devel queue, possibly will be submitted to 4.13 so stable can pick it. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 07/12/2017 03:03 AM, David Sterba wrote: > On Mon, Jul 10, 2017 at 04:56:36PM +0300, Nikolay Borisov wrote: >> On 10.07.2017 16:12, Nikolay Borisov wrote: >>> On 4.07.2017 14:49, Aleksa Sarai wrote: >>>> Several distributions mount the "proper root" as ro during initrd and >>>> then remount it as rw before pivot_root(2). Thus, if a rescan had been >>>> aborted by a previous shutdown, the rescan would never be resumed. >>>> >>>> This issue would manifest itself as several btrfs ioctl(2)s causing the >>>> entire machine to hang when btrfs_qgroup_wait_for_completion was hit >>>> (due to the fs_info->qgroup_rescan_running flag being set but the rescan >>>> itself not being resumed). Notably, Docker's btrfs storage driver makes >>>> regular use of BTRFS_QUOTA_CTL_DISABLE and BTRFS_IOC_QUOTA_RESCAN_WAIT >>>> (causing this problem to be manifested on boot for some machines). >>>> >>>> Cc: <stable@vger.kernel.org> # v3.11+ >>>> Cc: Jeff Mahoney <jeffm@suse.com> >>>> Fixes: b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") >>>> Signed-off-by: Aleksa Sarai <asarai@suse.de> >>> >>> Indeed, looking at the code it seems that b382a324b60f ("Btrfs: fix >>> qgroup rescan resume on mount") missed adding the qgroup_rescan_resume >>> in the remount path. One thing which I couldn't verify though is whether >>> reading fs_info->qgroup_flags without any locking is safe from remount >>> context. >>> >>> During remount I don't see any locks taken that prevent operations which >>> can modify qgroup_flags. >> >> Further inspection reveals that the access rules to qgroup_flags are >> somewhat broken so this patch doesn't really make things any worse than >> they are. > > The usage follows a pattern for a bitfield, updated by set_bit/clear_bit > etc. The updates to the state or inconsistency is not safe, so some > updates could get lost under some circumstances. > > Patch added to devel queue, possibly will be submitted to 4.13 so stable > can pick it. Looks like it wasn't merged in the 4.13 window (so stable hasn't picked it), will this be submitted for 4.14? Thanks.
On Sat, Aug 26, 2017 at 04:39:02PM +1000, Aleksa Sarai wrote: > On 07/12/2017 03:03 AM, David Sterba wrote: > > On Mon, Jul 10, 2017 at 04:56:36PM +0300, Nikolay Borisov wrote: > >> On 10.07.2017 16:12, Nikolay Borisov wrote: > >>> On 4.07.2017 14:49, Aleksa Sarai wrote: > >>>> Several distributions mount the "proper root" as ro during initrd and > >>>> then remount it as rw before pivot_root(2). Thus, if a rescan had been > >>>> aborted by a previous shutdown, the rescan would never be resumed. > >>>> > >>>> This issue would manifest itself as several btrfs ioctl(2)s causing the > >>>> entire machine to hang when btrfs_qgroup_wait_for_completion was hit > >>>> (due to the fs_info->qgroup_rescan_running flag being set but the rescan > >>>> itself not being resumed). Notably, Docker's btrfs storage driver makes > >>>> regular use of BTRFS_QUOTA_CTL_DISABLE and BTRFS_IOC_QUOTA_RESCAN_WAIT > >>>> (causing this problem to be manifested on boot for some machines). > >>>> > >>>> Cc: <stable@vger.kernel.org> # v3.11+ > >>>> Cc: Jeff Mahoney <jeffm@suse.com> > >>>> Fixes: b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") > >>>> Signed-off-by: Aleksa Sarai <asarai@suse.de> > >>> > >>> Indeed, looking at the code it seems that b382a324b60f ("Btrfs: fix > >>> qgroup rescan resume on mount") missed adding the qgroup_rescan_resume > >>> in the remount path. One thing which I couldn't verify though is whether > >>> reading fs_info->qgroup_flags without any locking is safe from remount > >>> context. > >>> > >>> During remount I don't see any locks taken that prevent operations which > >>> can modify qgroup_flags. > >> > >> Further inspection reveals that the access rules to qgroup_flags are > >> somewhat broken so this patch doesn't really make things any worse than > >> they are. > > > > The usage follows a pattern for a bitfield, updated by set_bit/clear_bit > > etc. The updates to the state or inconsistency is not safe, so some > > updates could get lost under some circumstances. > > > > Patch added to devel queue, possibly will be submitted to 4.13 so stable > > can pick it. > > Looks like it wasn't merged in the 4.13 window (so stable hasn't picked > it), will this be submitted for 4.14? Thanks. Sorry this didn't make it to 4.13, the patch is in the queue for 4.14. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 6346876c97ea..ff6690389343 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1821,6 +1821,8 @@ static int btrfs_remount(struct super_block *sb, int *flags, char *data) goto restore; } + btrfs_qgroup_rescan_resume(fs_info); + if (!fs_info->uuid_root) { btrfs_info(fs_info, "creating UUID tree"); ret = btrfs_create_uuid_tree(fs_info);
Several distributions mount the "proper root" as ro during initrd and then remount it as rw before pivot_root(2). Thus, if a rescan had been aborted by a previous shutdown, the rescan would never be resumed. This issue would manifest itself as several btrfs ioctl(2)s causing the entire machine to hang when btrfs_qgroup_wait_for_completion was hit (due to the fs_info->qgroup_rescan_running flag being set but the rescan itself not being resumed). Notably, Docker's btrfs storage driver makes regular use of BTRFS_QUOTA_CTL_DISABLE and BTRFS_IOC_QUOTA_RESCAN_WAIT (causing this problem to be manifested on boot for some machines). Cc: <stable@vger.kernel.org> # v3.11+ Cc: Jeff Mahoney <jeffm@suse.com> Fixes: b382a324b60f ("Btrfs: fix qgroup rescan resume on mount") Signed-off-by: Aleksa Sarai <asarai@suse.de> --- fs/btrfs/super.c | 2 ++ 1 file changed, 2 insertions(+)