[0/9,v2] ceph: auto reconnect after blacklisted
mbox series

Message ID 20190724122120.17438-1-zyan@redhat.com
Headers show
Series
  • ceph: auto reconnect after blacklisted
Related show

Message

Yan, Zheng July 24, 2019, 12:21 p.m. UTC
This series add support for auto reconnect after blacklisted.

Auto reconnect is controlled by recover_session=<clean|no> mount option.
So far only clean mode is supported and it is the default mode. In this
mode, client drops any dirty data/metadata, invalidates page caches and
invalidates all writable file handles. After reconnect, file locks become
stale because MDS lose track of them. If an inode contains any stale file
lock, read/write on the indoe are not allowed until all stale file locks
are released by applications.

v2: remove force_remount mount option
    no enabled auto reconnect by default
    remove unfinished recover_session=brute code

Yan, Zheng (9):
  libceph: add function that reset client's entity addr
  libceph: add function that clears osd client's abort_err
  ceph: allow closing session in restarting/reconnect state
  ceph: track and report error of async metadata operation
  ceph: pass filp to ceph_get_caps()
  ceph: add helper function that forcibly reconnects to ceph cluster.
  ceph: return -EIO if read/write against filp that lost file locks
  ceph: invalidate all write mode filp after reconnect
  ceph: auto reconnect after blacklisted

 Documentation/filesystems/ceph.txt | 10 ++++
 fs/ceph/addr.c                     | 37 ++++++++----
 fs/ceph/caps.c                     | 93 +++++++++++++++++++++---------
 fs/ceph/file.c                     | 50 +++++++++-------
 fs/ceph/inode.c                    |  2 +
 fs/ceph/locks.c                    |  8 ++-
 fs/ceph/mds_client.c               | 89 ++++++++++++++++++++++------
 fs/ceph/mds_client.h               |  6 +-
 fs/ceph/super.c                    | 45 ++++++++++++++-
 fs/ceph/super.h                    | 21 +++++--
 include/linux/ceph/libceph.h       |  1 +
 include/linux/ceph/messenger.h     |  1 +
 include/linux/ceph/mon_client.h    |  1 +
 include/linux/ceph/osd_client.h    |  2 +
 net/ceph/ceph_common.c             |  8 +++
 net/ceph/messenger.c               |  5 ++
 net/ceph/mon_client.c              |  7 +++
 net/ceph/osd_client.c              | 24 ++++++++
 18 files changed, 324 insertions(+), 86 deletions(-)

Comments

Jeff Layton July 24, 2019, 12:56 p.m. UTC | #1
On Wed, 2019-07-24 at 20:21 +0800, Yan, Zheng wrote:
> This series add support for auto reconnect after blacklisted.
> 
> Auto reconnect is controlled by recover_session=<clean|no> mount option.
> So far only clean mode is supported and it is the default mode. In this
> mode, client drops any dirty data/metadata, invalidates page caches and
> invalidates all writable file handles. After reconnect, file locks become
> stale because MDS lose track of them. If an inode contains any stale file
> lock, read/write on the indoe are not allowed until all stale file locks
> are released by applications.
> 
> v2: remove force_remount mount option
>     no enabled auto reconnect by default
>     remove unfinished recover_session=brute code
> 

I've looked over the set and I think this looks reasonable modulo a few
small nits that can be cleaned up during or after merging.

I don't see the knob that forces a reconnect in this set any longer. Did
you decide that that wasn't needed or are you planning to add it in a
later set?


> Yan, Zheng (9):
>   libceph: add function that reset client's entity addr
>   libceph: add function that clears osd client's abort_err
>   ceph: allow closing session in restarting/reconnect state
>   ceph: track and report error of async metadata operation
>   ceph: pass filp to ceph_get_caps()
>   ceph: add helper function that forcibly reconnects to ceph cluster.
>   ceph: return -EIO if read/write against filp that lost file locks
>   ceph: invalidate all write mode filp after reconnect
>   ceph: auto reconnect after blacklisted
> 
>  Documentation/filesystems/ceph.txt | 10 ++++
>  fs/ceph/addr.c                     | 37 ++++++++----
>  fs/ceph/caps.c                     | 93 +++++++++++++++++++++---------
>  fs/ceph/file.c                     | 50 +++++++++-------
>  fs/ceph/inode.c                    |  2 +
>  fs/ceph/locks.c                    |  8 ++-
>  fs/ceph/mds_client.c               | 89 ++++++++++++++++++++++------
>  fs/ceph/mds_client.h               |  6 +-
>  fs/ceph/super.c                    | 45 ++++++++++++++-
>  fs/ceph/super.h                    | 21 +++++--
>  include/linux/ceph/libceph.h       |  1 +
>  include/linux/ceph/messenger.h     |  1 +
>  include/linux/ceph/mon_client.h    |  1 +
>  include/linux/ceph/osd_client.h    |  2 +
>  net/ceph/ceph_common.c             |  8 +++
>  net/ceph/messenger.c               |  5 ++
>  net/ceph/mon_client.c              |  7 +++
>  net/ceph/osd_client.c              | 24 ++++++++
>  18 files changed, 324 insertions(+), 86 deletions(-)
>
Yan, Zheng July 25, 2019, 1:31 a.m. UTC | #2
On 7/24/19 8:56 PM, Jeff Layton wrote:
> On Wed, 2019-07-24 at 20:21 +0800, Yan, Zheng wrote:
>> This series add support for auto reconnect after blacklisted.
>>
>> Auto reconnect is controlled by recover_session=<clean|no> mount option.
>> So far only clean mode is supported and it is the default mode. In this
>> mode, client drops any dirty data/metadata, invalidates page caches and
>> invalidates all writable file handles. After reconnect, file locks become
>> stale because MDS lose track of them. If an inode contains any stale file
>> lock, read/write on the indoe are not allowed until all stale file locks
>> are released by applications.
>>
>> v2: remove force_remount mount option
>>      no enabled auto reconnect by default
>>      remove unfinished recover_session=brute code
>>
> 
> I've looked over the set and I think this looks reasonable modulo a few
> small nits that can be cleaned up during or after merging.
> 
> I don't see the knob that forces a reconnect in this set any longer. Did
> you decide that that wasn't needed or are you planning to add it in a
> later set?

add it later

Regards
Yan, Zheng

> 
> 
>> Yan, Zheng (9):
>>    libceph: add function that reset client's entity addr
>>    libceph: add function that clears osd client's abort_err
>>    ceph: allow closing session in restarting/reconnect state
>>    ceph: track and report error of async metadata operation
>>    ceph: pass filp to ceph_get_caps()
>>    ceph: add helper function that forcibly reconnects to ceph cluster.
>>    ceph: return -EIO if read/write against filp that lost file locks
>>    ceph: invalidate all write mode filp after reconnect
>>    ceph: auto reconnect after blacklisted
>>
>>   Documentation/filesystems/ceph.txt | 10 ++++
>>   fs/ceph/addr.c                     | 37 ++++++++----
>>   fs/ceph/caps.c                     | 93 +++++++++++++++++++++---------
>>   fs/ceph/file.c                     | 50 +++++++++-------
>>   fs/ceph/inode.c                    |  2 +
>>   fs/ceph/locks.c                    |  8 ++-
>>   fs/ceph/mds_client.c               | 89 ++++++++++++++++++++++------
>>   fs/ceph/mds_client.h               |  6 +-
>>   fs/ceph/super.c                    | 45 ++++++++++++++-
>>   fs/ceph/super.h                    | 21 +++++--
>>   include/linux/ceph/libceph.h       |  1 +
>>   include/linux/ceph/messenger.h     |  1 +
>>   include/linux/ceph/mon_client.h    |  1 +
>>   include/linux/ceph/osd_client.h    |  2 +
>>   net/ceph/ceph_common.c             |  8 +++
>>   net/ceph/messenger.c               |  5 ++
>>   net/ceph/mon_client.c              |  7 +++
>>   net/ceph/osd_client.c              | 24 ++++++++
>>   18 files changed, 324 insertions(+), 86 deletions(-)
>>
>
Jeff Layton July 25, 2019, 11:02 a.m. UTC | #3
On Thu, 2019-07-25 at 09:31 +0800, Yan, Zheng wrote:
> On 7/24/19 8:56 PM, Jeff Layton wrote:
> > On Wed, 2019-07-24 at 20:21 +0800, Yan, Zheng wrote:
> > > This series add support for auto reconnect after blacklisted.
> > > 
> > > Auto reconnect is controlled by recover_session=<clean|no> mount option.
> > > So far only clean mode is supported and it is the default mode. In this
> > > mode, client drops any dirty data/metadata, invalidates page caches and
> > > invalidates all writable file handles. After reconnect, file locks become
> > > stale because MDS lose track of them. If an inode contains any stale file
> > > lock, read/write on the indoe are not allowed until all stale file locks
> > > are released by applications.
> > > 
> > > v2: remove force_remount mount option
> > >      no enabled auto reconnect by default
> > >      remove unfinished recover_session=brute code
> > > 
> > 
> > I've looked over the set and I think this looks reasonable modulo a few
> > small nits that can be cleaned up during or after merging.
> > 
> > I don't see the knob that forces a reconnect in this set any longer. Did
> > you decide that that wasn't needed or are you planning to add it in a
> > later set?
> 
> add it later
> 
> 

Ok. I went ahead and pushed these to the testing branch, and added a few
small patches on top to clean up the documentation and some sparse
warnings. Let me know if you have issues with any of those.

> > 
> > > Yan, Zheng (9):
> > >    libceph: add function that reset client's entity addr
> > >    libceph: add function that clears osd client's abort_err
> > >    ceph: allow closing session in restarting/reconnect state
> > >    ceph: track and report error of async metadata operation
> > >    ceph: pass filp to ceph_get_caps()
> > >    ceph: add helper function that forcibly reconnects to ceph cluster.
> > >    ceph: return -EIO if read/write against filp that lost file locks
> > >    ceph: invalidate all write mode filp after reconnect
> > >    ceph: auto reconnect after blacklisted
> > > 
> > >   Documentation/filesystems/ceph.txt | 10 ++++
> > >   fs/ceph/addr.c                     | 37 ++++++++----
> > >   fs/ceph/caps.c                     | 93 +++++++++++++++++++++---------
> > >   fs/ceph/file.c                     | 50 +++++++++-------
> > >   fs/ceph/inode.c                    |  2 +
> > >   fs/ceph/locks.c                    |  8 ++-
> > >   fs/ceph/mds_client.c               | 89 ++++++++++++++++++++++------
> > >   fs/ceph/mds_client.h               |  6 +-
> > >   fs/ceph/super.c                    | 45 ++++++++++++++-
> > >   fs/ceph/super.h                    | 21 +++++--
> > >   include/linux/ceph/libceph.h       |  1 +
> > >   include/linux/ceph/messenger.h     |  1 +
> > >   include/linux/ceph/mon_client.h    |  1 +
> > >   include/linux/ceph/osd_client.h    |  2 +
> > >   net/ceph/ceph_common.c             |  8 +++
> > >   net/ceph/messenger.c               |  5 ++
> > >   net/ceph/mon_client.c              |  7 +++
> > >   net/ceph/osd_client.c              | 24 ++++++++
> > >   18 files changed, 324 insertions(+), 86 deletions(-)
> > >