mbox series

[00/20] filelock: split struct file_lock into file_lock and file_lease structs

Message ID 20240116-flsplit-v1-0-c9d0f4370a5d@kernel.org (mailing list archive)
Headers show
Series filelock: split struct file_lock into file_lock and file_lease structs | expand

Message

Jeff Layton Jan. 16, 2024, 7:45 p.m. UTC
Long ago, file locks used to hang off of a singly-linked list in struct
inode. Because of this, when leases were added, they were added to the
same list and so they had to be tracked using the same sort of
structure.

Several years ago, we added struct file_lock_context, which allowed us
to use separate lists to track different types of file locks. Given
that, leases no longer need to be tracked using struct file_lock.

That said, a lot of the underlying infrastructure _is_ the same between
file leases and locks, so we can't completely separate everything.

This patchset first splits a group of fields used by both file locks and
leases into a new struct file_lock_core, that is then embedded in struct
file_lock. Coccinelle was then used to convert a lot of the callers to
deal with the move, with the remaining 25% or so converted by hand.

It then converts several internal functions in fs/locks.c to work
with struct file_lock_core. Lastly, struct file_lock is split into
struct file_lock and file_lease, and the lease-related APIs converted to
take struct file_lease.

After the first few patches (which I left split up for easier review),
the set should be bisectable. I'll plan to squash the first few
together to make sure the resulting set is bisectable before merge.

Finally, I left the coccinelle scripts I used in tree. I had heard it
was preferable to merge those along with the patches that they
generate, but I wasn't sure where they go. I can either move those to a
more appropriate location or we can just drop that commit if it's not
needed.

I'd like to have this considered for inclusion in v6.9. Christian, would
you be amenable to shepherding this into mainline (assuming there are no
major objections, of course)?

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
Jeff Layton (20):
      filelock: split common fields into struct file_lock_core
      filelock: add coccinelle scripts to move fields to struct file_lock_core
      filelock: the results of the coccinelle conversion
      filelock: fixups after the coccinelle changes
      filelock: convert some internal functions to use file_lock_core instead
      filelock: convert more internal functions to use file_lock_core
      filelock: make posix_same_owner take file_lock_core pointers
      filelock: convert posix_owner_key to take file_lock_core arg
      filelock: make locks_{insert,delete}_global_locks take file_lock_core arg
      filelock: convert locks_{insert,delete}_global_blocked
      filelock: convert the IS_* macros to take file_lock_core
      filelock: make __locks_delete_block and __locks_wake_up_blocks take file_lock_core
      filelock: convert __locks_insert_block, conflict and deadlock checks to use file_lock_core
      filelock: convert fl_blocker to file_lock_core
      filelock: clean up locks_delete_block internals
      filelock: reorganize locks_delete_block and __locks_insert_block
      filelock: make assign_type helper take a file_lock_core pointer
      filelock: convert locks_wake_up_blocks to take a file_lock_core pointer
      filelock: convert locks_insert_lock_ctx and locks_delete_lock_ctx
      filelock: split leases out of struct file_lock

 cocci/filelock.cocci            |  81 +++++
 cocci/filelock2.cocci           |   6 +
 cocci/nlm.cocci                 |  81 +++++
 fs/9p/vfs_file.c                |  38 +-
 fs/afs/flock.c                  |  55 +--
 fs/ceph/locks.c                 |  74 ++--
 fs/dlm/plock.c                  |  44 +--
 fs/fuse/file.c                  |  14 +-
 fs/gfs2/file.c                  |  16 +-
 fs/libfs.c                      |   2 +-
 fs/lockd/clnt4xdr.c             |  14 +-
 fs/lockd/clntlock.c             |   2 +-
 fs/lockd/clntproc.c             |  60 +--
 fs/lockd/clntxdr.c              |  14 +-
 fs/lockd/svc4proc.c             |  10 +-
 fs/lockd/svclock.c              |  64 ++--
 fs/lockd/svcproc.c              |  10 +-
 fs/lockd/svcsubs.c              |  24 +-
 fs/lockd/xdr.c                  |  14 +-
 fs/lockd/xdr4.c                 |  14 +-
 fs/locks.c                      | 785 ++++++++++++++++++++++------------------
 fs/nfs/delegation.c             |   4 +-
 fs/nfs/file.c                   |  22 +-
 fs/nfs/nfs3proc.c               |   2 +-
 fs/nfs/nfs4_fs.h                |   2 +-
 fs/nfs/nfs4file.c               |   2 +-
 fs/nfs/nfs4proc.c               |  39 +-
 fs/nfs/nfs4state.c              |   6 +-
 fs/nfs/nfs4trace.h              |   4 +-
 fs/nfs/nfs4xdr.c                |   8 +-
 fs/nfs/write.c                  |   8 +-
 fs/nfsd/filecache.c             |   4 +-
 fs/nfsd/nfs4callback.c          |   2 +-
 fs/nfsd/nfs4layouts.c           |  34 +-
 fs/nfsd/nfs4state.c             |  98 ++---
 fs/ocfs2/locks.c                |  12 +-
 fs/ocfs2/stack_user.c           |   2 +-
 fs/smb/client/cifsfs.c          |   2 +-
 fs/smb/client/cifssmb.c         |   8 +-
 fs/smb/client/file.c            |  74 ++--
 fs/smb/client/smb2file.c        |   2 +-
 fs/smb/server/smb2pdu.c         |  44 +--
 fs/smb/server/vfs.c             |  14 +-
 include/linux/filelock.h        |  58 ++-
 include/linux/fs.h              |   5 +-
 include/linux/lockd/lockd.h     |   8 +-
 include/trace/events/afs.h      |   4 +-
 include/trace/events/filelock.h |  54 +--
 48 files changed, 1119 insertions(+), 825 deletions(-)
---
base-commit: 052d534373b7ed33712a63d5e17b2b6cdbce84fd
change-id: 20240116-flsplit-bdb46824db68

Best regards,

Comments

Christian Brauner Jan. 17, 2024, 12:48 p.m. UTC | #1
> I'd like to have this considered for inclusion in v6.9. Christian, would
> you be amenable to shepherding this into mainline (assuming there are no
> major objections, of course)?

Yes, of course I will be happy to.
Jeff Layton Jan. 17, 2024, 12:59 p.m. UTC | #2
On Wed, 2024-01-17 at 13:48 +0100, Christian Brauner wrote:
> > I'd like to have this considered for inclusion in v6.9. Christian, would
> > you be amenable to shepherding this into mainline (assuming there are no
> > major objections, of course)?
> 
> Yes, of course I will be happy to.

Great! I probably have at least another version or two to send before
it's ready for linux-next, but hopefully we can get it there soon after
the merge window closes.

Thanks,
Chuck Lever III Jan. 17, 2024, 3:12 p.m. UTC | #3
On Tue, Jan 16, 2024 at 02:45:56PM -0500, Jeff Layton wrote:
> Long ago, file locks used to hang off of a singly-linked list in struct
> inode. Because of this, when leases were added, they were added to the
> same list and so they had to be tracked using the same sort of
> structure.
> 
> Several years ago, we added struct file_lock_context, which allowed us
> to use separate lists to track different types of file locks. Given
> that, leases no longer need to be tracked using struct file_lock.
> 
> That said, a lot of the underlying infrastructure _is_ the same between
> file leases and locks, so we can't completely separate everything.

Naive question: locks and leases are similar. Why do they need to be
split apart? The cover letter doesn't address that, and I'm new
enough at this that I don't have that context.


> This patchset first splits a group of fields used by both file locks and
> leases into a new struct file_lock_core, that is then embedded in struct
> file_lock. Coccinelle was then used to convert a lot of the callers to
> deal with the move, with the remaining 25% or so converted by hand.
> 
> It then converts several internal functions in fs/locks.c to work
> with struct file_lock_core. Lastly, struct file_lock is split into
> struct file_lock and file_lease, and the lease-related APIs converted to
> take struct file_lease.
> 
> After the first few patches (which I left split up for easier review),
> the set should be bisectable. I'll plan to squash the first few
> together to make sure the resulting set is bisectable before merge.
> 
> Finally, I left the coccinelle scripts I used in tree. I had heard it
> was preferable to merge those along with the patches that they
> generate, but I wasn't sure where they go. I can either move those to a
> more appropriate location or we can just drop that commit if it's not
> needed.
> 
> I'd like to have this considered for inclusion in v6.9. Christian, would
> you be amenable to shepherding this into mainline (assuming there are no
> major objections, of course)?
> 
> Signed-off-by: Jeff Layton <jlayton@kernel.org>
> ---
> Jeff Layton (20):
>       filelock: split common fields into struct file_lock_core
>       filelock: add coccinelle scripts to move fields to struct file_lock_core
>       filelock: the results of the coccinelle conversion
>       filelock: fixups after the coccinelle changes
>       filelock: convert some internal functions to use file_lock_core instead
>       filelock: convert more internal functions to use file_lock_core
>       filelock: make posix_same_owner take file_lock_core pointers
>       filelock: convert posix_owner_key to take file_lock_core arg
>       filelock: make locks_{insert,delete}_global_locks take file_lock_core arg
>       filelock: convert locks_{insert,delete}_global_blocked
>       filelock: convert the IS_* macros to take file_lock_core
>       filelock: make __locks_delete_block and __locks_wake_up_blocks take file_lock_core
>       filelock: convert __locks_insert_block, conflict and deadlock checks to use file_lock_core
>       filelock: convert fl_blocker to file_lock_core
>       filelock: clean up locks_delete_block internals
>       filelock: reorganize locks_delete_block and __locks_insert_block
>       filelock: make assign_type helper take a file_lock_core pointer
>       filelock: convert locks_wake_up_blocks to take a file_lock_core pointer
>       filelock: convert locks_insert_lock_ctx and locks_delete_lock_ctx
>       filelock: split leases out of struct file_lock
> 
>  cocci/filelock.cocci            |  81 +++++
>  cocci/filelock2.cocci           |   6 +
>  cocci/nlm.cocci                 |  81 +++++
>  fs/9p/vfs_file.c                |  38 +-
>  fs/afs/flock.c                  |  55 +--
>  fs/ceph/locks.c                 |  74 ++--
>  fs/dlm/plock.c                  |  44 +--
>  fs/fuse/file.c                  |  14 +-
>  fs/gfs2/file.c                  |  16 +-
>  fs/libfs.c                      |   2 +-
>  fs/lockd/clnt4xdr.c             |  14 +-
>  fs/lockd/clntlock.c             |   2 +-
>  fs/lockd/clntproc.c             |  60 +--
>  fs/lockd/clntxdr.c              |  14 +-
>  fs/lockd/svc4proc.c             |  10 +-
>  fs/lockd/svclock.c              |  64 ++--
>  fs/lockd/svcproc.c              |  10 +-
>  fs/lockd/svcsubs.c              |  24 +-
>  fs/lockd/xdr.c                  |  14 +-
>  fs/lockd/xdr4.c                 |  14 +-
>  fs/locks.c                      | 785 ++++++++++++++++++++++------------------
>  fs/nfs/delegation.c             |   4 +-
>  fs/nfs/file.c                   |  22 +-
>  fs/nfs/nfs3proc.c               |   2 +-
>  fs/nfs/nfs4_fs.h                |   2 +-
>  fs/nfs/nfs4file.c               |   2 +-
>  fs/nfs/nfs4proc.c               |  39 +-
>  fs/nfs/nfs4state.c              |   6 +-
>  fs/nfs/nfs4trace.h              |   4 +-
>  fs/nfs/nfs4xdr.c                |   8 +-
>  fs/nfs/write.c                  |   8 +-
>  fs/nfsd/filecache.c             |   4 +-
>  fs/nfsd/nfs4callback.c          |   2 +-
>  fs/nfsd/nfs4layouts.c           |  34 +-
>  fs/nfsd/nfs4state.c             |  98 ++---
>  fs/ocfs2/locks.c                |  12 +-
>  fs/ocfs2/stack_user.c           |   2 +-
>  fs/smb/client/cifsfs.c          |   2 +-
>  fs/smb/client/cifssmb.c         |   8 +-
>  fs/smb/client/file.c            |  74 ++--
>  fs/smb/client/smb2file.c        |   2 +-
>  fs/smb/server/smb2pdu.c         |  44 +--
>  fs/smb/server/vfs.c             |  14 +-
>  include/linux/filelock.h        |  58 ++-
>  include/linux/fs.h              |   5 +-
>  include/linux/lockd/lockd.h     |   8 +-
>  include/trace/events/afs.h      |   4 +-
>  include/trace/events/filelock.h |  54 +--
>  48 files changed, 1119 insertions(+), 825 deletions(-)
> ---
> base-commit: 052d534373b7ed33712a63d5e17b2b6cdbce84fd
> change-id: 20240116-flsplit-bdb46824db68
> 
> Best regards,
> -- 
> Jeff Layton <jlayton@kernel.org>
>
Jeff Layton Jan. 17, 2024, 5:32 p.m. UTC | #4
On Wed, 2024-01-17 at 10:12 -0500, Chuck Lever wrote:
> On Tue, Jan 16, 2024 at 02:45:56PM -0500, Jeff Layton wrote:
> > Long ago, file locks used to hang off of a singly-linked list in struct
> > inode. Because of this, when leases were added, they were added to the
> > same list and so they had to be tracked using the same sort of
> > structure.
> > 
> > Several years ago, we added struct file_lock_context, which allowed us
> > to use separate lists to track different types of file locks. Given
> > that, leases no longer need to be tracked using struct file_lock.
> > 
> > That said, a lot of the underlying infrastructure _is_ the same between
> > file leases and locks, so we can't completely separate everything.
> 
> Naive question: locks and leases are similar. Why do they need to be
> split apart? The cover letter doesn't address that, and I'm new
> enough at this that I don't have that context.
> 

Leases and locks do have some similarities, but it's mostly the
internals (stuff like the blocker/waiter handling) where they are
similar. Superficially they are very different objects, and handling
them with the same struct is unintuitive.

So, for now this is just about cleaning up the lock and lease handling
APIs for better type safety and clarity. It's also nice to separate out
things like the kasync handling, which only applies to leases, as well
as splitting up the lock_manager_operations, which don't share any
operations between locks and leases.

Longer term, we're also considering adding things like directory
delegations, which may need to either expand struct file_lease, or add
a new variant (dir_deleg ?). I'd rather not add that complexity to
struct file_lock. 

> 
> > This patchset first splits a group of fields used by both file locks and
> > leases into a new struct file_lock_core, that is then embedded in struct
> > file_lock. Coccinelle was then used to convert a lot of the callers to
> > deal with the move, with the remaining 25% or so converted by hand.
> > 
> > It then converts several internal functions in fs/locks.c to work
> > with struct file_lock_core. Lastly, struct file_lock is split into
> > struct file_lock and file_lease, and the lease-related APIs converted to
> > take struct file_lease.
> > 
> > After the first few patches (which I left split up for easier review),
> > the set should be bisectable. I'll plan to squash the first few
> > together to make sure the resulting set is bisectable before merge.
> > 
> > Finally, I left the coccinelle scripts I used in tree. I had heard it
> > was preferable to merge those along with the patches that they
> > generate, but I wasn't sure where they go. I can either move those to a
> > more appropriate location or we can just drop that commit if it's not
> > needed.
> > 
> > I'd like to have this considered for inclusion in v6.9. Christian, would
> > you be amenable to shepherding this into mainline (assuming there are no
> > major objections, of course)?
> > 
> > Signed-off-by: Jeff Layton <jlayton@kernel.org>
> > ---
> > Jeff Layton (20):
> >       filelock: split common fields into struct file_lock_core
> >       filelock: add coccinelle scripts to move fields to struct file_lock_core
> >       filelock: the results of the coccinelle conversion
> >       filelock: fixups after the coccinelle changes
> >       filelock: convert some internal functions to use file_lock_core instead
> >       filelock: convert more internal functions to use file_lock_core
> >       filelock: make posix_same_owner take file_lock_core pointers
> >       filelock: convert posix_owner_key to take file_lock_core arg
> >       filelock: make locks_{insert,delete}_global_locks take file_lock_core arg
> >       filelock: convert locks_{insert,delete}_global_blocked
> >       filelock: convert the IS_* macros to take file_lock_core
> >       filelock: make __locks_delete_block and __locks_wake_up_blocks take file_lock_core
> >       filelock: convert __locks_insert_block, conflict and deadlock checks to use file_lock_core
> >       filelock: convert fl_blocker to file_lock_core
> >       filelock: clean up locks_delete_block internals
> >       filelock: reorganize locks_delete_block and __locks_insert_block
> >       filelock: make assign_type helper take a file_lock_core pointer
> >       filelock: convert locks_wake_up_blocks to take a file_lock_core pointer
> >       filelock: convert locks_insert_lock_ctx and locks_delete_lock_ctx
> >       filelock: split leases out of struct file_lock
> > 
> >  cocci/filelock.cocci            |  81 +++++
> >  cocci/filelock2.cocci           |   6 +
> >  cocci/nlm.cocci                 |  81 +++++
> >  fs/9p/vfs_file.c                |  38 +-
> >  fs/afs/flock.c                  |  55 +--
> >  fs/ceph/locks.c                 |  74 ++--
> >  fs/dlm/plock.c                  |  44 +--
> >  fs/fuse/file.c                  |  14 +-
> >  fs/gfs2/file.c                  |  16 +-
> >  fs/libfs.c                      |   2 +-
> >  fs/lockd/clnt4xdr.c             |  14 +-
> >  fs/lockd/clntlock.c             |   2 +-
> >  fs/lockd/clntproc.c             |  60 +--
> >  fs/lockd/clntxdr.c              |  14 +-
> >  fs/lockd/svc4proc.c             |  10 +-
> >  fs/lockd/svclock.c              |  64 ++--
> >  fs/lockd/svcproc.c              |  10 +-
> >  fs/lockd/svcsubs.c              |  24 +-
> >  fs/lockd/xdr.c                  |  14 +-
> >  fs/lockd/xdr4.c                 |  14 +-
> >  fs/locks.c                      | 785 ++++++++++++++++++++++------------------
> >  fs/nfs/delegation.c             |   4 +-
> >  fs/nfs/file.c                   |  22 +-
> >  fs/nfs/nfs3proc.c               |   2 +-
> >  fs/nfs/nfs4_fs.h                |   2 +-
> >  fs/nfs/nfs4file.c               |   2 +-
> >  fs/nfs/nfs4proc.c               |  39 +-
> >  fs/nfs/nfs4state.c              |   6 +-
> >  fs/nfs/nfs4trace.h              |   4 +-
> >  fs/nfs/nfs4xdr.c                |   8 +-
> >  fs/nfs/write.c                  |   8 +-
> >  fs/nfsd/filecache.c             |   4 +-
> >  fs/nfsd/nfs4callback.c          |   2 +-
> >  fs/nfsd/nfs4layouts.c           |  34 +-
> >  fs/nfsd/nfs4state.c             |  98 ++---
> >  fs/ocfs2/locks.c                |  12 +-
> >  fs/ocfs2/stack_user.c           |   2 +-
> >  fs/smb/client/cifsfs.c          |   2 +-
> >  fs/smb/client/cifssmb.c         |   8 +-
> >  fs/smb/client/file.c            |  74 ++--
> >  fs/smb/client/smb2file.c        |   2 +-
> >  fs/smb/server/smb2pdu.c         |  44 +--
> >  fs/smb/server/vfs.c             |  14 +-
> >  include/linux/filelock.h        |  58 ++-
> >  include/linux/fs.h              |   5 +-
> >  include/linux/lockd/lockd.h     |   8 +-
> >  include/trace/events/afs.h      |   4 +-
> >  include/trace/events/filelock.h |  54 +--
> >  48 files changed, 1119 insertions(+), 825 deletions(-)
> > ---
> > base-commit: 052d534373b7ed33712a63d5e17b2b6cdbce84fd
> > change-id: 20240116-flsplit-bdb46824db68
> > 
> > Best regards,
> > -- 
> > Jeff Layton <jlayton@kernel.org>
> > 
>
NeilBrown Jan. 17, 2024, 6:59 p.m. UTC | #5
On Thu, 18 Jan 2024, Chuck Lever wrote:
> On Tue, Jan 16, 2024 at 02:45:56PM -0500, Jeff Layton wrote:
> > Long ago, file locks used to hang off of a singly-linked list in struct
> > inode. Because of this, when leases were added, they were added to the
> > same list and so they had to be tracked using the same sort of
> > structure.
> > 
> > Several years ago, we added struct file_lock_context, which allowed us
> > to use separate lists to track different types of file locks. Given
> > that, leases no longer need to be tracked using struct file_lock.
> > 
> > That said, a lot of the underlying infrastructure _is_ the same between
> > file leases and locks, so we can't completely separate everything.
> 
> Naive question: locks and leases are similar. Why do they need to be
> split apart? The cover letter doesn't address that, and I'm new
> enough at this that I don't have that context.

For me, the big win was in the last patch where we got separate
lock_manager_operations and lease_manager_operations.  There is zero
overlap between the two.  This highlights that one one level these are
different things with different behaviours.

NeilBrown