| Message ID | 20190404064537.4031-2-wqu@suse.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | btrfs: Refactor delayed ref parameter list | expand |
On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: > Current delayed ref interface has several problems: > - Longer and longer parameter lists > bytenr > num_bytes > parent > ---------- so far so good > ref_root > owner > offset > ---------- I don't feel good now > > - Different interpretation for the same parameter > Above @owner for data ref is inode number (u64), > while for tree ref, it's level (int). > > They are even in different size range. > For level we only need 0~8, while for ino it's > BTRFS_FIRST_FREE_OBJECTID~BTRFS_LAST_FREE_OBJECTID. > > And @offset doesn't even makes sense for tree ref. > > Such parameter reuse may look clever as an hidden union, but it > destroys code readability. > > To solve both problems, we introduce a new structure, btrfs_ref to solve > them: > > - Structure instead of long parameter list > This makes later expansion easier, and better documented. > > - Use btrfs_ref::type to distinguish data and tree ref > > - Use proper union to store data/tree ref specific structures. > > - Use separate functions to fill data/tree ref data, with a common generic > function to fill common bytenr/num_bytes members. > > All parameters will find its place in btrfs_ref, and an extra member, > @real_root, inspired by ref-verify code, is newly introduced for later > qgroup code, to record which tree is triggered this extent modification. > > This patch doesn't touch any code, but provides the basis for incoming > refactors. > > Signed-off-by: Qu Wenruo <wqu@suse.com> > --- > fs/btrfs/delayed-ref.h | 116 +++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 116 insertions(+) > > diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h > index 70606da440aa..8eb5b3576759 100644 > --- a/fs/btrfs/delayed-ref.h > +++ b/fs/btrfs/delayed-ref.h > @@ -176,6 +176,90 @@ struct btrfs_delayed_ref_root { > u64 qgroup_to_skip; > }; > > +enum btrfs_ref_type { > + BTRFS_REF_NOT_SET, > + BTRFS_REF_DATA, > + BTRFS_REF_METADATA, > + BTRFS_REF_LAST, > +}; > + > +struct btrfs_data_ref { > + /* For EXTENT_DATA_REF */ > + > + /* Root who refers to this data extent */ > + u64 ref_root; > + > + /* Inode who refers to this data extent */ > + u64 ino; > + > + /* > + * file_offset - extent_offset > + * > + * file_offset is the key.offset of the EXTENT_DATA key. > + * extent_offset is btrfs_file_extent_offset() of the EXTENT_DATA data. > + */ > + u64 offset; > +}; > + > +struct btrfs_tree_ref { > + /* > + * Level of this tree block > + * > + * Shared for skinny (TREE_BLOCK_REF) and normal tree ref. > + */ > + int level; > + > + /* > + * Root who refers to this tree block. > + * > + * For TREE_BLOCK_REF (skinny metadata, either inline or keyed) > + */ > + u64 root; > + > + /* For non-skinny metadata, no special member needed */ > +}; > + > +struct btrfs_ref { The structure name sounds a bit generic, but I think we can keep it short. There are no other btrfs-specific references that could be confused, there are 'backrefs', 'delayed-refs' all refering to the b-tree references. > + enum btrfs_ref_type type; > + int action; > + > + /* > + * Only use parent pointers as backref (SHARED_BLOCK_REF or > + * SHARED_DATA_REF) for this extent and its children. > + * Set for reloc trees. > + */ > + bool only_backreferences:1; No bool bitfields please, wasn't this mentioned last time?
On 2019/4/5 下午8:09, David Sterba wrote: > On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: >> Current delayed ref interface has several problems: >> - Longer and longer parameter lists >> bytenr >> num_bytes >> parent >> ---------- so far so good >> ref_root >> owner >> offset >> ---------- I don't feel good now >> >> - Different interpretation for the same parameter >> Above @owner for data ref is inode number (u64), >> while for tree ref, it's level (int). >> >> They are even in different size range. >> For level we only need 0~8, while for ino it's >> BTRFS_FIRST_FREE_OBJECTID~BTRFS_LAST_FREE_OBJECTID. >> >> And @offset doesn't even makes sense for tree ref. >> >> Such parameter reuse may look clever as an hidden union, but it >> destroys code readability. >> >> To solve both problems, we introduce a new structure, btrfs_ref to solve >> them: >> >> - Structure instead of long parameter list >> This makes later expansion easier, and better documented. >> >> - Use btrfs_ref::type to distinguish data and tree ref >> >> - Use proper union to store data/tree ref specific structures. >> >> - Use separate functions to fill data/tree ref data, with a common generic >> function to fill common bytenr/num_bytes members. >> >> All parameters will find its place in btrfs_ref, and an extra member, >> @real_root, inspired by ref-verify code, is newly introduced for later >> qgroup code, to record which tree is triggered this extent modification. >> >> This patch doesn't touch any code, but provides the basis for incoming >> refactors. >> >> Signed-off-by: Qu Wenruo <wqu@suse.com> >> --- >> fs/btrfs/delayed-ref.h | 116 +++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 116 insertions(+) >> >> diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h >> index 70606da440aa..8eb5b3576759 100644 >> --- a/fs/btrfs/delayed-ref.h >> +++ b/fs/btrfs/delayed-ref.h >> @@ -176,6 +176,90 @@ struct btrfs_delayed_ref_root { >> u64 qgroup_to_skip; >> }; >> >> +enum btrfs_ref_type { >> + BTRFS_REF_NOT_SET, >> + BTRFS_REF_DATA, >> + BTRFS_REF_METADATA, >> + BTRFS_REF_LAST, >> +}; >> + >> +struct btrfs_data_ref { >> + /* For EXTENT_DATA_REF */ >> + >> + /* Root who refers to this data extent */ >> + u64 ref_root; >> + >> + /* Inode who refers to this data extent */ >> + u64 ino; >> + >> + /* >> + * file_offset - extent_offset >> + * >> + * file_offset is the key.offset of the EXTENT_DATA key. >> + * extent_offset is btrfs_file_extent_offset() of the EXTENT_DATA data. >> + */ >> + u64 offset; >> +}; >> + >> +struct btrfs_tree_ref { >> + /* >> + * Level of this tree block >> + * >> + * Shared for skinny (TREE_BLOCK_REF) and normal tree ref. >> + */ >> + int level; >> + >> + /* >> + * Root who refers to this tree block. >> + * >> + * For TREE_BLOCK_REF (skinny metadata, either inline or keyed) >> + */ >> + u64 root; >> + >> + /* For non-skinny metadata, no special member needed */ >> +}; >> + >> +struct btrfs_ref { > > The structure name sounds a bit generic, but I think we can keep it > short. There are no other btrfs-specific references that could be > confused, there are 'backrefs', 'delayed-refs' all refering to the > b-tree references. > >> + enum btrfs_ref_type type; >> + int action; >> + >> + /* >> + * Only use parent pointers as backref (SHARED_BLOCK_REF or >> + * SHARED_DATA_REF) for this extent and its children. >> + * Set for reloc trees. >> + */ >> + bool only_backreferences:1; > > No bool bitfields please, wasn't this mentioned last time? > Oh, I forgot that one. Do I need to resend or just edit that commit in my github branch? Thanks, Qu
On Fri, Apr 05, 2019 at 09:18:24PM +0800, Qu Wenruo wrote: > > > On 2019/4/5 下午8:09, David Sterba wrote: > > On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: > >> Current delayed ref interface has several problems: > >> - Longer and longer parameter lists > >> bytenr > >> num_bytes > >> parent > >> ---------- so far so good > >> ref_root > >> owner > >> offset > >> ---------- I don't feel good now > >> > >> - Different interpretation for the same parameter > >> Above @owner for data ref is inode number (u64), > >> while for tree ref, it's level (int). > >> > >> They are even in different size range. > >> For level we only need 0~8, while for ino it's > >> BTRFS_FIRST_FREE_OBJECTID~BTRFS_LAST_FREE_OBJECTID. > >> > >> And @offset doesn't even makes sense for tree ref. > >> > >> Such parameter reuse may look clever as an hidden union, but it > >> destroys code readability. > >> > >> To solve both problems, we introduce a new structure, btrfs_ref to solve > >> them: > >> > >> - Structure instead of long parameter list > >> This makes later expansion easier, and better documented. > >> > >> - Use btrfs_ref::type to distinguish data and tree ref > >> > >> - Use proper union to store data/tree ref specific structures. > >> > >> - Use separate functions to fill data/tree ref data, with a common generic > >> function to fill common bytenr/num_bytes members. > >> > >> All parameters will find its place in btrfs_ref, and an extra member, > >> @real_root, inspired by ref-verify code, is newly introduced for later > >> qgroup code, to record which tree is triggered this extent modification. > >> > >> This patch doesn't touch any code, but provides the basis for incoming > >> refactors. > >> > >> Signed-off-by: Qu Wenruo <wqu@suse.com> > >> --- > >> fs/btrfs/delayed-ref.h | 116 +++++++++++++++++++++++++++++++++++++++++ > >> 1 file changed, 116 insertions(+) > >> > >> diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h > >> index 70606da440aa..8eb5b3576759 100644 > >> --- a/fs/btrfs/delayed-ref.h > >> +++ b/fs/btrfs/delayed-ref.h > >> @@ -176,6 +176,90 @@ struct btrfs_delayed_ref_root { > >> u64 qgroup_to_skip; > >> }; > >> > >> +enum btrfs_ref_type { > >> + BTRFS_REF_NOT_SET, > >> + BTRFS_REF_DATA, > >> + BTRFS_REF_METADATA, > >> + BTRFS_REF_LAST, > >> +}; > >> + > >> +struct btrfs_data_ref { > >> + /* For EXTENT_DATA_REF */ > >> + > >> + /* Root who refers to this data extent */ > >> + u64 ref_root; > >> + > >> + /* Inode who refers to this data extent */ > >> + u64 ino; > >> + > >> + /* > >> + * file_offset - extent_offset > >> + * > >> + * file_offset is the key.offset of the EXTENT_DATA key. > >> + * extent_offset is btrfs_file_extent_offset() of the EXTENT_DATA data. > >> + */ > >> + u64 offset; > >> +}; > >> + > >> +struct btrfs_tree_ref { > >> + /* > >> + * Level of this tree block > >> + * > >> + * Shared for skinny (TREE_BLOCK_REF) and normal tree ref. > >> + */ > >> + int level; > >> + > >> + /* > >> + * Root who refers to this tree block. > >> + * > >> + * For TREE_BLOCK_REF (skinny metadata, either inline or keyed) > >> + */ > >> + u64 root; > >> + > >> + /* For non-skinny metadata, no special member needed */ > >> +}; > >> + > >> +struct btrfs_ref { > > > > The structure name sounds a bit generic, but I think we can keep it > > short. There are no other btrfs-specific references that could be > > confused, there are 'backrefs', 'delayed-refs' all refering to the > > b-tree references. > > > >> + enum btrfs_ref_type type; > >> + int action; > >> + > >> + /* > >> + * Only use parent pointers as backref (SHARED_BLOCK_REF or > >> + * SHARED_DATA_REF) for this extent and its children. > >> + * Set for reloc trees. > >> + */ > >> + bool only_backreferences:1; > > > > No bool bitfields please, wasn't this mentioned last time? > > > Oh, I forgot that one. > > Do I need to resend or just edit that commit in my github branch? No need to resend, I'll edit that as it's a trivial change. I'll have another look at the whole structure if the layout could be optimized, eg. the enum takes 4 bytes but we'd be fine with a byte.
>>>> + /* For non-skinny metadata, no special member needed */ >>>> +}; >>>> + >>>> +struct btrfs_ref { >>> >>> The structure name sounds a bit generic, but I think we can keep it >>> short. There are no other btrfs-specific references that could be >>> confused, there are 'backrefs', 'delayed-refs' all refering to the >>> b-tree references. >>> >>>> + enum btrfs_ref_type type; >>>> + int action; >>>> + >>>> + /* >>>> + * Only use parent pointers as backref (SHARED_BLOCK_REF or >>>> + * SHARED_DATA_REF) for this extent and its children. >>>> + * Set for reloc trees. >>>> + */ >>>> + bool only_backreferences:1; >>> >>> No bool bitfields please, wasn't this mentioned last time? >>> >> Oh, I forgot that one. >> >> Do I need to resend or just edit that commit in my github branch? > > No need to resend, I'll edit that as it's a trivial change. I'll have > another look at the whole structure if the layout could be optimized, > eg. the enum takes 4 bytes but we'd be fine with a byte. > Maybe I have already said this before, but I still really hope either we have some external tool to do that for us, or some attribute to do it. Thanks, Qu
On 6.04.19 г. 2:47 ч., Qu Wenruo wrote: > >>>>> + /* For non-skinny metadata, no special member needed */ >>>>> +}; >>>>> + >>>>> +struct btrfs_ref { >>>> >>>> The structure name sounds a bit generic, but I think we can keep it >>>> short. There are no other btrfs-specific references that could be >>>> confused, there are 'backrefs', 'delayed-refs' all refering to the >>>> b-tree references. >>>> >>>>> + enum btrfs_ref_type type; >>>>> + int action; >>>>> + >>>>> + /* >>>>> + * Only use parent pointers as backref (SHARED_BLOCK_REF or >>>>> + * SHARED_DATA_REF) for this extent and its children. >>>>> + * Set for reloc trees. >>>>> + */ >>>>> + bool only_backreferences:1; >>>> >>>> No bool bitfields please, wasn't this mentioned last time? >>>> >>> Oh, I forgot that one. >>> >>> Do I need to resend or just edit that commit in my github branch? >> >> No need to resend, I'll edit that as it's a trivial change. I'll have >> another look at the whole structure if the layout could be optimized, >> eg. the enum takes 4 bytes but we'd be fine with a byte. >> > Maybe I have already said this before, but I still really hope either we > have some external tool to do that for us, or some attribute to do it. Pahole[0] does show the layout of structures. Not entirely automatic but better than nothing. [0]https://git.kernel.org/pub/scm/devel/pahole/pahole.git > > Thanks, > Qu >
On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: > +struct btrfs_ref { > + enum btrfs_ref_type type; > + int action; > + > + /* > + * Only use parent pointers as backref (SHARED_BLOCK_REF or > + * SHARED_DATA_REF) for this extent and its children. > + * Set for reloc trees. > + */ > + bool only_backreferences:1; I renamed this to only_backrefs and was surprised that there were no compilation errors, ie. this member is not used at all ...
On 2019/4/12 下午11:46, David Sterba wrote: > On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: >> +struct btrfs_ref { >> + enum btrfs_ref_type type; >> + int action; >> + >> + /* >> + * Only use parent pointers as backref (SHARED_BLOCK_REF or >> + * SHARED_DATA_REF) for this extent and its children. >> + * Set for reloc trees. >> + */ >> + bool only_backreferences:1; > > I renamed this to only_backrefs and was surprised that there were no > compilation errors, ie. this member is not used at all ... Yep, for callers who really uses this member, they just set @parent, and that's all. Thanks, Qu
On Sat, Apr 13, 2019 at 07:48:51AM +0800, Qu Wenruo wrote: > > > On 2019/4/12 下午11:46, David Sterba wrote: > > On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: > >> +struct btrfs_ref { > >> + enum btrfs_ref_type type; > >> + int action; > >> + > >> + /* > >> + * Only use parent pointers as backref (SHARED_BLOCK_REF or > >> + * SHARED_DATA_REF) for this extent and its children. > >> + * Set for reloc trees. > >> + */ > >> + bool only_backreferences:1; > > > > I renamed this to only_backrefs and was surprised that there were no > > compilation errors, ie. this member is not used at all ... > > Yep, for callers who really uses this member, they just set @parent, and > that's all. So there's nothing in the old and new code that uses it, then why do you add it? If this is for some existing patchset then ok, keep it there but otherwise remove it.
On 2019/4/16 上午12:50, David Sterba wrote: > On Sat, Apr 13, 2019 at 07:48:51AM +0800, Qu Wenruo wrote: >> >> >> On 2019/4/12 下午11:46, David Sterba wrote: >>> On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: >>>> +struct btrfs_ref { >>>> + enum btrfs_ref_type type; >>>> + int action; >>>> + >>>> + /* >>>> + * Only use parent pointers as backref (SHARED_BLOCK_REF or >>>> + * SHARED_DATA_REF) for this extent and its children. >>>> + * Set for reloc trees. >>>> + */ >>>> + bool only_backreferences:1; >>> >>> I renamed this to only_backrefs and was surprised that there were no >>> compilation errors, ie. this member is not used at all ... >> >> Yep, for callers who really uses this member, they just set @parent, and >> that's all. > > So there's nothing in the old and new code that uses it, then why do you > add it? If this is for some existing patchset then ok, keep it there but > otherwise remove it. It should be removed. My bad, at the time of writing, I didn't get the point that @parent is enough to info to use SHARED_BLOCK_REF_KEY. Would you mind to fold this removal? Thanks, Qu
On Tue, Apr 16, 2019 at 08:01:42AM +0800, Qu Wenruo wrote: > > > On 2019/4/16 上午12:50, David Sterba wrote: > > On Sat, Apr 13, 2019 at 07:48:51AM +0800, Qu Wenruo wrote: > >> > >> > >> On 2019/4/12 下午11:46, David Sterba wrote: > >>> On Thu, Apr 04, 2019 at 02:45:29PM +0800, Qu Wenruo wrote: > >>>> +struct btrfs_ref { > >>>> + enum btrfs_ref_type type; > >>>> + int action; > >>>> + > >>>> + /* > >>>> + * Only use parent pointers as backref (SHARED_BLOCK_REF or > >>>> + * SHARED_DATA_REF) for this extent and its children. > >>>> + * Set for reloc trees. > >>>> + */ > >>>> + bool only_backreferences:1; > >>> > >>> I renamed this to only_backrefs and was surprised that there were no > >>> compilation errors, ie. this member is not used at all ... > >> > >> Yep, for callers who really uses this member, they just set @parent, and > >> that's all. > > > > So there's nothing in the old and new code that uses it, then why do you > > add it? If this is for some existing patchset then ok, keep it there but > > otherwise remove it. > > It should be removed. > My bad, at the time of writing, I didn't get the point that @parent is > enough to info to use SHARED_BLOCK_REF_KEY. > > Would you mind to fold this removal? If it's sufficient to remove the struct member then I'll do that, no need to resend.
diff --git a/fs/btrfs/delayed-ref.h b/fs/btrfs/delayed-ref.h index 70606da440aa..8eb5b3576759 100644 --- a/fs/btrfs/delayed-ref.h +++ b/fs/btrfs/delayed-ref.h @@ -176,6 +176,90 @@ struct btrfs_delayed_ref_root { u64 qgroup_to_skip; }; +enum btrfs_ref_type { + BTRFS_REF_NOT_SET, + BTRFS_REF_DATA, + BTRFS_REF_METADATA, + BTRFS_REF_LAST, +}; + +struct btrfs_data_ref { + /* For EXTENT_DATA_REF */ + + /* Root who refers to this data extent */ + u64 ref_root; + + /* Inode who refers to this data extent */ + u64 ino; + + /* + * file_offset - extent_offset + * + * file_offset is the key.offset of the EXTENT_DATA key. + * extent_offset is btrfs_file_extent_offset() of the EXTENT_DATA data. + */ + u64 offset; +}; + +struct btrfs_tree_ref { + /* + * Level of this tree block + * + * Shared for skinny (TREE_BLOCK_REF) and normal tree ref. + */ + int level; + + /* + * Root who refers to this tree block. + * + * For TREE_BLOCK_REF (skinny metadata, either inline or keyed) + */ + u64 root; + + /* For non-skinny metadata, no special member needed */ +}; + +struct btrfs_ref { + enum btrfs_ref_type type; + int action; + + /* + * Only use parent pointers as backref (SHARED_BLOCK_REF or + * SHARED_DATA_REF) for this extent and its children. + * Set for reloc trees. + */ + bool only_backreferences:1; + + /* + * Whether this extent should go through qgroup record. + * + * Normally false, but for certain case like delayed subtree scan, + * setting this flag can hugely reduce qgroup overhead. + */ + bool skip_qgroup:1; + + /* + * Optional. To which root this modification is for. + * Mostly used for qgroup optimization. + * + * When unset, data/tree ref init code will populate it. + * In certain case, we're modifying reference for a different root. + * E.g. Cow fs tree blocks for balance. + * In that case, tree_ref::root will be fs tree, but we're doing this + * for reloc tree, then we should set @real_root to reloc tree. + */ + u64 real_root; + u64 bytenr; + u64 len; + + /* Bytenr of the parent tree block */ + u64 parent; + union { + struct btrfs_data_ref data_ref; + struct btrfs_tree_ref tree_ref; + }; +}; + extern struct kmem_cache *btrfs_delayed_ref_head_cachep; extern struct kmem_cache *btrfs_delayed_tree_ref_cachep; extern struct kmem_cache *btrfs_delayed_data_ref_cachep; @@ -184,6 +268,38 @@ extern struct kmem_cache *btrfs_delayed_extent_op_cachep; int __init btrfs_delayed_ref_init(void); void __cold btrfs_delayed_ref_exit(void); +static inline void btrfs_init_generic_ref(struct btrfs_ref *generic_ref, + int action, u64 bytenr, u64 len, u64 parent) +{ + generic_ref->action = action; + generic_ref->bytenr = bytenr; + generic_ref->len = len; + generic_ref->parent = parent; +} + +static inline void btrfs_init_tree_ref(struct btrfs_ref *generic_ref, + int level, u64 root) +{ + /* If @real_root not set, use @root as fallback */ + if (!generic_ref->real_root) + generic_ref->real_root = root; + generic_ref->tree_ref.level = level; + generic_ref->tree_ref.root = root; + generic_ref->type = BTRFS_REF_METADATA; +} + +static inline void btrfs_init_data_ref(struct btrfs_ref *generic_ref, + u64 ref_root, u64 ino, u64 offset) +{ + /* If @real_root not set, use @root as fallback */ + if (!generic_ref->real_root) + generic_ref->real_root = ref_root; + generic_ref->data_ref.ref_root = ref_root; + generic_ref->data_ref.ino = ino; + generic_ref->data_ref.offset = offset; + generic_ref->type = BTRFS_REF_DATA; +} + static inline struct btrfs_delayed_extent_op * btrfs_alloc_delayed_extent_op(void) {
Current delayed ref interface has several problems: - Longer and longer parameter lists bytenr num_bytes parent ---------- so far so good ref_root owner offset ---------- I don't feel good now - Different interpretation for the same parameter Above @owner for data ref is inode number (u64), while for tree ref, it's level (int). They are even in different size range. For level we only need 0~8, while for ino it's BTRFS_FIRST_FREE_OBJECTID~BTRFS_LAST_FREE_OBJECTID. And @offset doesn't even makes sense for tree ref. Such parameter reuse may look clever as an hidden union, but it destroys code readability. To solve both problems, we introduce a new structure, btrfs_ref to solve them: - Structure instead of long parameter list This makes later expansion easier, and better documented. - Use btrfs_ref::type to distinguish data and tree ref - Use proper union to store data/tree ref specific structures. - Use separate functions to fill data/tree ref data, with a common generic function to fill common bytenr/num_bytes members. All parameters will find its place in btrfs_ref, and an extra member, @real_root, inspired by ref-verify code, is newly introduced for later qgroup code, to record which tree is triggered this extent modification. This patch doesn't touch any code, but provides the basis for incoming refactors. Signed-off-by: Qu Wenruo <wqu@suse.com> --- fs/btrfs/delayed-ref.h | 116 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+)