| Message ID | 1379605688-987-2-git-send-email-jbacik@fusionio.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
* Josef Bacik <jbacik@fusionio.com> wrote: > We can starve out the transaction commit with a bunch of caching threads > all running at the same time. This is because we will only drop the > extent_commit_sem if we need_resched(), which isn't likely to happen > since we will be reading a lot from the disk so have already > schedule()'ed plenty. Alex observed that he could starve out a > transaction commit for up to a minute with 32 caching threads all > running at once. This will allow us to drop the extent_commit_sem to > allow the transaction commit to swap the commit_root out and then all > the cachers will start back up. Thanks, > > Signed-off-by: Josef Bacik <jbacik@fusionio.com> > --- > fs/btrfs/extent-tree.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > index cfb3cf7..cc074c34 100644 > --- a/fs/btrfs/extent-tree.c > +++ b/fs/btrfs/extent-tree.c > @@ -442,7 +442,8 @@ next: > if (ret) > break; > > - if (need_resched()) { > + if (need_resched() || > + rwsem_is_contended(&fs_info->extent_commit_sem)) { > caching_ctl->progress = last; > btrfs_release_path(path); > up_read(&fs_info->extent_commit_sem); So, just to fill in what happens in this loop: mutex_unlock(&caching_ctl->mutex); cond_resched(); goto again; where 'again:' takes caching_ctl->mutex and fs_info->extent_commit_sem again: again: mutex_lock(&caching_ctl->mutex); /* need to make sure the commit_root doesn't disappear */ down_read(&fs_info->extent_commit_sem); So, if I'm reading the code correct, there can be a fair amount of concurrency here: there may be multiple 'caching kthreads' per filesystem active, while there's one fs_info->extent_commit_sem per filesystem AFAICS. So, what happens if there are a lot of CPUs all busy holding the ->extent_commit_sem rwsem read-locked and a writer arrives? They'd all rush to try to release the fs_info->extent_commit_sem, and they'd block in the down_read() because there's a writer waiting. So there's a guarantee of forward progress. This should answer akpm's concern I think. If this analysis is correct then: Acked-by: Ingo Molnar <mingo@kernel.org> Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Sep 20, 2013 at 07:12:47AM +0200, Ingo Molnar wrote: > > * Josef Bacik <jbacik@fusionio.com> wrote: > > > We can starve out the transaction commit with a bunch of caching threads > > all running at the same time. This is because we will only drop the > > extent_commit_sem if we need_resched(), which isn't likely to happen > > since we will be reading a lot from the disk so have already > > schedule()'ed plenty. Alex observed that he could starve out a > > transaction commit for up to a minute with 32 caching threads all > > running at once. This will allow us to drop the extent_commit_sem to > > allow the transaction commit to swap the commit_root out and then all > > the cachers will start back up. Thanks, > > > > Signed-off-by: Josef Bacik <jbacik@fusionio.com> > > --- > > fs/btrfs/extent-tree.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > > index cfb3cf7..cc074c34 100644 > > --- a/fs/btrfs/extent-tree.c > > +++ b/fs/btrfs/extent-tree.c > > @@ -442,7 +442,8 @@ next: > > if (ret) > > break; > > > > - if (need_resched()) { > > + if (need_resched() || > > + rwsem_is_contended(&fs_info->extent_commit_sem)) { > > caching_ctl->progress = last; > > btrfs_release_path(path); > > up_read(&fs_info->extent_commit_sem); > > So, just to fill in what happens in this loop: > > mutex_unlock(&caching_ctl->mutex); > cond_resched(); > goto again; > > where 'again:' takes caching_ctl->mutex and fs_info->extent_commit_sem > again: > > again: > mutex_lock(&caching_ctl->mutex); > /* need to make sure the commit_root doesn't disappear */ > down_read(&fs_info->extent_commit_sem); > > So, if I'm reading the code correct, there can be a fair amount of > concurrency here: there may be multiple 'caching kthreads' per filesystem > active, while there's one fs_info->extent_commit_sem per filesystem > AFAICS. > > So, what happens if there are a lot of CPUs all busy holding the > ->extent_commit_sem rwsem read-locked and a writer arrives? They'd all > rush to try to release the fs_info->extent_commit_sem, and they'd block in > the down_read() because there's a writer waiting. > > So there's a guarantee of forward progress. This should answer akpm's > concern I think. > > If this analysis is correct then: > > Acked-by: Ingo Molnar <mingo@kernel.org> > Yup this is correct, thank you, I'll add your ack'ed by to the next iteration. Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
* Josef Bacik <jbacik@fusionio.com> wrote: > On Fri, Sep 20, 2013 at 07:12:47AM +0200, Ingo Molnar wrote: > > > > * Josef Bacik <jbacik@fusionio.com> wrote: > > > > > We can starve out the transaction commit with a bunch of caching threads > > > all running at the same time. This is because we will only drop the > > > extent_commit_sem if we need_resched(), which isn't likely to happen > > > since we will be reading a lot from the disk so have already > > > schedule()'ed plenty. Alex observed that he could starve out a > > > transaction commit for up to a minute with 32 caching threads all > > > running at once. This will allow us to drop the extent_commit_sem to > > > allow the transaction commit to swap the commit_root out and then all > > > the cachers will start back up. Thanks, > > > > > > Signed-off-by: Josef Bacik <jbacik@fusionio.com> > > > --- > > > fs/btrfs/extent-tree.c | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > > > index cfb3cf7..cc074c34 100644 > > > --- a/fs/btrfs/extent-tree.c > > > +++ b/fs/btrfs/extent-tree.c > > > @@ -442,7 +442,8 @@ next: > > > if (ret) > > > break; > > > > > > - if (need_resched()) { > > > + if (need_resched() || > > > + rwsem_is_contended(&fs_info->extent_commit_sem)) { > > > caching_ctl->progress = last; > > > btrfs_release_path(path); > > > up_read(&fs_info->extent_commit_sem); > > > > So, just to fill in what happens in this loop: > > > > mutex_unlock(&caching_ctl->mutex); > > cond_resched(); > > goto again; > > > > where 'again:' takes caching_ctl->mutex and fs_info->extent_commit_sem > > again: > > > > again: > > mutex_lock(&caching_ctl->mutex); > > /* need to make sure the commit_root doesn't disappear */ > > down_read(&fs_info->extent_commit_sem); > > > > So, if I'm reading the code correct, there can be a fair amount of > > concurrency here: there may be multiple 'caching kthreads' per filesystem > > active, while there's one fs_info->extent_commit_sem per filesystem > > AFAICS. > > > > So, what happens if there are a lot of CPUs all busy holding the > > ->extent_commit_sem rwsem read-locked and a writer arrives? They'd all > > rush to try to release the fs_info->extent_commit_sem, and they'd block in > > the down_read() because there's a writer waiting. > > > > So there's a guarantee of forward progress. This should answer akpm's > > concern I think. > > > > If this analysis is correct then: > > > > Acked-by: Ingo Molnar <mingo@kernel.org> > > > > Yup this is correct, thank you, I'll add your ack'ed by to the next > iteration. You might also want to stick the explanation into the changelog - it wasn't really obvious to someone not versed in btrfs internals. Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index cfb3cf7..cc074c34 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -442,7 +442,8 @@ next: if (ret) break; - if (need_resched()) { + if (need_resched() || + rwsem_is_contended(&fs_info->extent_commit_sem)) { caching_ctl->progress = last; btrfs_release_path(path); up_read(&fs_info->extent_commit_sem);
We can starve out the transaction commit with a bunch of caching threads all running at the same time. This is because we will only drop the extent_commit_sem if we need_resched(), which isn't likely to happen since we will be reading a lot from the disk so have already schedule()'ed plenty. Alex observed that he could starve out a transaction commit for up to a minute with 32 caching threads all running at once. This will allow us to drop the extent_commit_sem to allow the transaction commit to swap the commit_root out and then all the cachers will start back up. Thanks, Signed-off-by: Josef Bacik <jbacik@fusionio.com> --- fs/btrfs/extent-tree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)