[v2,0/2] ceph: fix long stalls on sync/syncfs

Message

Jeffrey Layton April 2, 2020, 11:29 a.m. UTC

This is v2 of the patch I sent the other day to fix the problem of long
stalls when calling sync or syncfs.

This set converts the mdsc->cap_dirty list to a per-session list, and
then has the only caller that looks at cap_dirty walk the list of
sessions and issue flushes for each session in turn.

With this, we can use an empty s_cap_dirty list as an indicator that the
cap flush is the last one going to the session and can mark that one as
one we're waiting on so the MDS can expedite it.

This also attempts to clarify some of the locking around s_cap_dirty,
and adds a FIXME comment to raise the question about locking around
s_cap_flushing.

Jeff Layton (2):
  ceph: convert mdsc->cap_dirty to a per-session list
  ceph: request expedited service on session's last cap flush

 fs/ceph/caps.c       | 72 +++++++++++++++++++++++++++++++++++++++-----
 fs/ceph/mds_client.c |  2 +-
 fs/ceph/mds_client.h |  5 +--
 fs/ceph/super.h      | 21 +++++++++++--
 4 files changed, 87 insertions(+), 13 deletions(-)

Comments

Gregory Farnum April 2, 2020, 3:54 p.m. UTC | #1

Didn't look at the code too hard, but this sounds good to me.

I also realized that the userspace client isn't quite complete because
it didn't send out FLAG_SYNC on a per-MDS-session basis, just for the
last cap in its global dirty list. So thanks for that. :)
https://tracker.ceph.com/issues/44916

On Thu, Apr 2, 2020 at 4:29 AM Jeff Layton <[e-mail]> wrote:
>
> This is v2 of the patch I sent the other day to fix the problem of long
> stalls when calling sync or syncfs.
>
> This set converts the mdsc->cap_dirty list to a per-session list, and
> then has the only caller that looks at cap_dirty walk the list of
> sessions and issue flushes for each session in turn.
>
> With this, we can use an empty s_cap_dirty list as an indicator that the
> cap flush is the last one going to the session and can mark that one as
> one we're waiting on so the MDS can expedite it.
>
> This also attempts to clarify some of the locking around s_cap_dirty,
> and adds a FIXME comment to raise the question about locking around
> s_cap_flushing.
>
> Jeff Layton (2):
>   ceph: convert mdsc->cap_dirty to a per-session list
>   ceph: request expedited service on session's last cap flush
>
>  fs/ceph/caps.c       | 72 +++++++++++++++++++++++++++++++++++++++-----
>  fs/ceph/mds_client.c |  2 +-
>  fs/ceph/mds_client.h |  5 +--
>  fs/ceph/super.h      | 21 +++++++++++--
>  4 files changed, 87 insertions(+), 13 deletions(-)
>
> --
> 2.25.1
>