| Message ID | 20231207-btrfs-raid-v5-0-44aa1affe856@wdc.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | fstests: add tests for btrfs' raid-stripe-tree feature | expand |
On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn <johannes.thumshirn@wdc.com> wrote: > > Add tests for btrfs' raid-stripe-tree feature. All of these test work by > writing a specific pattern to a newly created filesystem and afterwards > using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify > the placement and the layout of the metadata. > > The md5sum of each file will be compared as well after a re-mount of the > filesystem. > > --- > Changes in v5: > - add _require_btrfs_free_space_tree helper and use in tests > - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com > > Changes in v4: > - add _require_btrfs_no_compress to all tests > - add _require_btrfs_no_nodatacow helper and add to btrfs/308 > - add _require_btrfs_feature "free_space_tree" to all tests > - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com > > Changes in v3: > - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it > automatically > - Rename test cases as btrfs/302 and btrfs/303 already exist upstream > - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com > > Changes in v2: > - Re-ordered series so the newly introduced group is added before the > tests > - Changes Filipe requested to the tests. > - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com > > --- > Johannes Thumshirn (9): > fstests: doc: add new raid-stripe-tree group > common: add filter for btrfs raid-stripe dump > common: add _require_btrfs_no_nodatacow helper > common: add _require_btrfs_free_space_tree > btrfs: add fstest for stripe-tree metadata with 4k write > btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree > btrfs: add fstest for writing to a file at an offset with RST > btrfs: add fstests to write 128k to a RST filesystem > btrfs: add fstest for overwriting a file partially with RST > > common/btrfs | 17 +++++++++ > common/filter.btrfs | 14 +++++++ > doc/group-names.txt | 1 + > tests/btrfs/304 | 56 +++++++++++++++++++++++++++ > tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++ > tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++ > tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++ > tests/btrfs/306 | 59 +++++++++++++++++++++++++++++ > tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++ > tests/btrfs/307 | 56 +++++++++++++++++++++++++++ > tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++ > tests/btrfs/308 | 60 +++++++++++++++++++++++++++++ > tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > 13 files changed, 710 insertions(+) > --- > base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8 Btw this base commit does not exist in the official fstests repo. That commit is from the staging branch at https://github.com/kdave/xfstests A "git am" will fail because the official fstests repo doesn't have _require_btrfs_no_block_group_tree() at common/btrfs, so it needs to be manually adjusted when applying the 3rd patch. I tried the tests and they look good, so: Reviewed-by: Filipe Manana <fdmanana@suse.com> One question I missed before. Test 304 for example does a 4K write and expects in the golden output to get a 4K raid stripe item. What happens on a machine with 64K page size? There the default sector size is 64K, will the write result in a 64K raid stripe item or will it be 4K? In the former case, it will make the test fail. Thanks. > change-id: 20231204-btrfs-raid-75975797f97d > > Best regards, > -- > Johannes Thumshirn <johannes.thumshirn@wdc.com> > >
On 12/7/23 17:41, Filipe Manana wrote: > On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn > <johannes.thumshirn@wdc.com> wrote: >> >> Add tests for btrfs' raid-stripe-tree feature. All of these test work by >> writing a specific pattern to a newly created filesystem and afterwards >> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify >> the placement and the layout of the metadata. >> >> The md5sum of each file will be compared as well after a re-mount of the >> filesystem. >> >> --- >> Changes in v5: >> - add _require_btrfs_free_space_tree helper and use in tests >> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com >> >> Changes in v4: >> - add _require_btrfs_no_compress to all tests >> - add _require_btrfs_no_nodatacow helper and add to btrfs/308 >> - add _require_btrfs_feature "free_space_tree" to all tests >> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com >> >> Changes in v3: >> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it >> automatically >> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream >> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com >> >> Changes in v2: >> - Re-ordered series so the newly introduced group is added before the >> tests >> - Changes Filipe requested to the tests. >> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com >> >> --- >> Johannes Thumshirn (9): >> fstests: doc: add new raid-stripe-tree group >> common: add filter for btrfs raid-stripe dump >> common: add _require_btrfs_no_nodatacow helper >> common: add _require_btrfs_free_space_tree >> btrfs: add fstest for stripe-tree metadata with 4k write >> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree >> btrfs: add fstest for writing to a file at an offset with RST >> btrfs: add fstests to write 128k to a RST filesystem >> btrfs: add fstest for overwriting a file partially with RST >> >> common/btrfs | 17 +++++++++ >> common/filter.btrfs | 14 +++++++ >> doc/group-names.txt | 1 + >> tests/btrfs/304 | 56 +++++++++++++++++++++++++++ >> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++ >> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++ >> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++ >> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++ >> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++ >> tests/btrfs/307 | 56 +++++++++++++++++++++++++++ >> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++ >> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++ >> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ >> 13 files changed, 710 insertions(+) >> --- >> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8 > > Btw this base commit does not exist in the official fstests repo. > That commit is from the staging branch at https://github.com/kdave/xfstests > > A "git am" will fail because the official fstests repo doesn't have > _require_btrfs_no_block_group_tree() at common/btrfs, > so it needs to be manually adjusted when applying the 3rd patch. > > I tried the tests and they look good, so: > > Reviewed-by: Filipe Manana <fdmanana@suse.com> > > One question I missed before. Test 304 for example does a 4K write and > expects in the golden output to get a 4K raid stripe item. > What happens on a machine with 64K page size? There the default sector > size is 64K, will the write result in a 64K raid stripe item or will > it be 4K? In the former case, it will make the test fail. > Testing on a 64K pagesize. Will run it. Apologies for intermittent responses; OOO until December 21. Thanks Anand > Thanks. > > >> change-id: 20231204-btrfs-raid-75975797f97d >> >> Best regards, >> -- >> Johannes Thumshirn <johannes.thumshirn@wdc.com> >> >>
On 08.12.23 02:19, Anand Jain wrote: > > > On 12/7/23 17:41, Filipe Manana wrote: >> On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn >> <johannes.thumshirn@wdc.com> wrote: >>> >>> Add tests for btrfs' raid-stripe-tree feature. All of these test work by >>> writing a specific pattern to a newly created filesystem and afterwards >>> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify >>> the placement and the layout of the metadata. >>> >>> The md5sum of each file will be compared as well after a re-mount of the >>> filesystem. >>> >>> --- >>> Changes in v5: >>> - add _require_btrfs_free_space_tree helper and use in tests >>> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com >>> >>> Changes in v4: >>> - add _require_btrfs_no_compress to all tests >>> - add _require_btrfs_no_nodatacow helper and add to btrfs/308 >>> - add _require_btrfs_feature "free_space_tree" to all tests >>> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com >>> >>> Changes in v3: >>> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it >>> automatically >>> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream >>> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com >>> >>> Changes in v2: >>> - Re-ordered series so the newly introduced group is added before the >>> tests >>> - Changes Filipe requested to the tests. >>> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com >>> >>> --- >>> Johannes Thumshirn (9): >>> fstests: doc: add new raid-stripe-tree group >>> common: add filter for btrfs raid-stripe dump >>> common: add _require_btrfs_no_nodatacow helper >>> common: add _require_btrfs_free_space_tree >>> btrfs: add fstest for stripe-tree metadata with 4k write >>> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree >>> btrfs: add fstest for writing to a file at an offset with RST >>> btrfs: add fstests to write 128k to a RST filesystem >>> btrfs: add fstest for overwriting a file partially with RST >>> >>> common/btrfs | 17 +++++++++ >>> common/filter.btrfs | 14 +++++++ >>> doc/group-names.txt | 1 + >>> tests/btrfs/304 | 56 +++++++++++++++++++++++++++ >>> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++ >>> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++ >>> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++ >>> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++ >>> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++ >>> tests/btrfs/307 | 56 +++++++++++++++++++++++++++ >>> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++ >>> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++ >>> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> 13 files changed, 710 insertions(+) >>> --- >>> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8 >> >> Btw this base commit does not exist in the official fstests repo. >> That commit is from the staging branch at https://github.com/kdave/xfstests >> >> A "git am" will fail because the official fstests repo doesn't have >> _require_btrfs_no_block_group_tree() at common/btrfs, >> so it needs to be manually adjusted when applying the 3rd patch. >> >> I tried the tests and they look good, so: >> >> Reviewed-by: Filipe Manana <fdmanana@suse.com> >> >> One question I missed before. Test 304 for example does a 4K write and >> expects in the golden output to get a 4K raid stripe item. >> What happens on a machine with 64K page size? There the default sector >> size is 64K, will the write result in a 64K raid stripe item or will >> it be 4K? In the former case, it will make the test fail. >> > > Testing on a 64K pagesize. Will run it. Apologies for intermittent > responses; OOO until December 21. Thanks Anand! I don't have a 64k page size system to test, but I _think_ Filipe is right, that will fail. I think we should skip these tests on non 4k sectors.
On Fri, Dec 8, 2023 at 4:19 AM Johannes Thumshirn <Johannes.Thumshirn@wdc.com> wrote: > > On 08.12.23 02:19, Anand Jain wrote: > > > > > > On 12/7/23 17:41, Filipe Manana wrote: > >> On Thu, Dec 7, 2023 at 9:03 AM Johannes Thumshirn > >> <johannes.thumshirn@wdc.com> wrote: > >>> > >>> Add tests for btrfs' raid-stripe-tree feature. All of these test work by > >>> writing a specific pattern to a newly created filesystem and afterwards > >>> using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify > >>> the placement and the layout of the metadata. > >>> > >>> The md5sum of each file will be compared as well after a re-mount of the > >>> filesystem. > >>> > >>> --- > >>> Changes in v5: > >>> - add _require_btrfs_free_space_tree helper and use in tests > >>> - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com > >>> > >>> Changes in v4: > >>> - add _require_btrfs_no_compress to all tests > >>> - add _require_btrfs_no_nodatacow helper and add to btrfs/308 > >>> - add _require_btrfs_feature "free_space_tree" to all tests > >>> - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com > >>> > >>> Changes in v3: > >>> - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it > >>> automatically > >>> - Rename test cases as btrfs/302 and btrfs/303 already exist upstream > >>> - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com > >>> > >>> Changes in v2: > >>> - Re-ordered series so the newly introduced group is added before the > >>> tests > >>> - Changes Filipe requested to the tests. > >>> - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com > >>> > >>> --- > >>> Johannes Thumshirn (9): > >>> fstests: doc: add new raid-stripe-tree group > >>> common: add filter for btrfs raid-stripe dump > >>> common: add _require_btrfs_no_nodatacow helper > >>> common: add _require_btrfs_free_space_tree > >>> btrfs: add fstest for stripe-tree metadata with 4k write > >>> btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree > >>> btrfs: add fstest for writing to a file at an offset with RST > >>> btrfs: add fstests to write 128k to a RST filesystem > >>> btrfs: add fstest for overwriting a file partially with RST > >>> > >>> common/btrfs | 17 +++++++++ > >>> common/filter.btrfs | 14 +++++++ > >>> doc/group-names.txt | 1 + > >>> tests/btrfs/304 | 56 +++++++++++++++++++++++++++ > >>> tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++ > >>> tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++ > >>> tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++ > >>> tests/btrfs/306 | 59 +++++++++++++++++++++++++++++ > >>> tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++ > >>> tests/btrfs/307 | 56 +++++++++++++++++++++++++++ > >>> tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++ > >>> tests/btrfs/308 | 60 +++++++++++++++++++++++++++++ > >>> tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>> 13 files changed, 710 insertions(+) > >>> --- > >>> base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8 > >> > >> Btw this base commit does not exist in the official fstests repo. > >> That commit is from the staging branch at https://github.com/kdave/xfstests > >> > >> A "git am" will fail because the official fstests repo doesn't have > >> _require_btrfs_no_block_group_tree() at common/btrfs, > >> so it needs to be manually adjusted when applying the 3rd patch. > >> > >> I tried the tests and they look good, so: > >> > >> Reviewed-by: Filipe Manana <fdmanana@suse.com> > >> > >> One question I missed before. Test 304 for example does a 4K write and > >> expects in the golden output to get a 4K raid stripe item. > >> What happens on a machine with 64K page size? There the default sector > >> size is 64K, will the write result in a 64K raid stripe item or will > >> it be 4K? In the former case, it will make the test fail. > >> > > > > Testing on a 64K pagesize. Will run it. Apologies for intermittent > > responses; OOO until December 21. > > Thanks Anand! > > I don't have a 64k page size system to test, but I _think_ Filipe is > right, that will fail. I think we should skip these tests on non 4k sectors. > Once we land the patch to default to 4k sector size[1] regardless of page size, this should all work across all architectures, no? [1]: https://lore.kernel.org/linux-btrfs/20231116160235.2708131-2-neal@gompa.dev/ -- 真実はいつも一つ！/ Always, there's only one truth!
On 09.12.23 20:16, Neal Gompa wrote: > > Once we land the patch to default to 4k sector size[1] regardless of > page size, this should all work across all architectures, no? > > [1]: https://lore.kernel.org/linux-btrfs/20231116160235.2708131-2-neal@gompa.dev/ > Per default, yes. But we can still set 64k sector size manually.
Add tests for btrfs' raid-stripe-tree feature. All of these test work by writing a specific pattern to a newly created filesystem and afterwards using `btrfs inspect-internal -t raid-stripe $SCRATCH_DEV_POOL` to verify the placement and the layout of the metadata. The md5sum of each file will be compared as well after a re-mount of the filesystem. --- Changes in v5: - add _require_btrfs_free_space_tree helper and use in tests - Link to v4: https://lore.kernel.org/r/20231206-btrfs-raid-v4-0-578284dd3a70@wdc.com Changes in v4: - add _require_btrfs_no_compress to all tests - add _require_btrfs_no_nodatacow helper and add to btrfs/308 - add _require_btrfs_feature "free_space_tree" to all tests - Link to v3: https://lore.kernel.org/r/20231205-btrfs-raid-v3-0-0e857a5439a2@wdc.com Changes in v3: - added 'raid-stripe-tree' to mkfs options, as only zoned raid gets it automatically - Rename test cases as btrfs/302 and btrfs/303 already exist upstream - Link to v2: https://lore.kernel.org/r/20231205-btrfs-raid-v2-0-25f80eea345b@wdc.com Changes in v2: - Re-ordered series so the newly introduced group is added before the tests - Changes Filipe requested to the tests. - Link to v1: https://lore.kernel.org/r/20231204-btrfs-raid-v1-0-b254eb1bcff8@wdc.com --- Johannes Thumshirn (9): fstests: doc: add new raid-stripe-tree group common: add filter for btrfs raid-stripe dump common: add _require_btrfs_no_nodatacow helper common: add _require_btrfs_free_space_tree btrfs: add fstest for stripe-tree metadata with 4k write btrfs: add fstest for 8k write spanning two stripes on raid-stripe-tree btrfs: add fstest for writing to a file at an offset with RST btrfs: add fstests to write 128k to a RST filesystem btrfs: add fstest for overwriting a file partially with RST common/btrfs | 17 +++++++++ common/filter.btrfs | 14 +++++++ doc/group-names.txt | 1 + tests/btrfs/304 | 56 +++++++++++++++++++++++++++ tests/btrfs/304.out | 58 ++++++++++++++++++++++++++++ tests/btrfs/305 | 61 ++++++++++++++++++++++++++++++ tests/btrfs/305.out | 82 ++++++++++++++++++++++++++++++++++++++++ tests/btrfs/306 | 59 +++++++++++++++++++++++++++++ tests/btrfs/306.out | 75 +++++++++++++++++++++++++++++++++++++ tests/btrfs/307 | 56 +++++++++++++++++++++++++++ tests/btrfs/307.out | 65 ++++++++++++++++++++++++++++++++ tests/btrfs/308 | 60 +++++++++++++++++++++++++++++ tests/btrfs/308.out | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 13 files changed, 710 insertions(+) --- base-commit: baca8a2b5cb6e798ce3a07e79a081031370c6cb8 change-id: 20231204-btrfs-raid-75975797f97d Best regards,