mbox series

[PATCHv3,0/4] generic: Add some tests around journal replay/recoveryloop

Message ID cover.1648730443.git.ritesh.list@gmail.com (mailing list archive)
Headers show
Series generic: Add some tests around journal replay/recoveryloop | expand

Message

Ritesh Harjani (IBM) March 31, 2022, 12:54 p.m. UTC
Hello,

The ext4 fast_commit kernel fix has landed into mainline tree [1].
In this v3, I have addressed review comments from Darrick.
Does this looks good to be picked up?

I have tested ext4 1k, 4k (w & w/o fast_commit). Also tested other FS with
default configs (like xfs, btrfs, f2fs). No surprises were seen.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bfdc502a4a4c058bf4cbb1df0c297761d528f54d

-ritesh

Changelogs:
===========

v2 => v3
=========
1. Addressed review comments from Darrick.
2. Rebased to latest master.

v1 => v2
=========
Sending v2 with tests/ext4/ converted to tests/generic/
(although I had not received any review comments on v1).
It seems all of the tests which I had sent in v1 are not ext4 specific anyways.
So in v2, I have made those into tests/generic/.

Summary
=========
These are some of the tests which when tested with ext4 fast_commit feature
w/o kernel fixes, could cause tests failures and/or KASAN bug (generic/486).

I gave these tests a run with default xfs, btrfs and f2fs configs (along with
ext4). No surprises observed.

[v2]: https://lore.kernel.org/all/cover.1647342932.git.riteshh@linux.ibm.com/
[v1]: https://lore.kernel.org/all/cover.1644070604.git.riteshh@linux.ibm.com/


Ritesh Harjani (4):
  generic/468: Add another falloc test entry
  common/punch: Add block_size argument to _filter_fiemap_**
  generic/678: Add a new shutdown recovery test
  generic/679: Add a test to check unwritten extents tracking

 common/punch          |  9 +++---
 tests/generic/468     |  8 +++++
 tests/generic/468.out |  2 ++
 tests/generic/678     | 72 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/678.out |  7 +++++
 tests/generic/679     | 65 ++++++++++++++++++++++++++++++++++++++
 tests/generic/679.out |  6 ++++
 7 files changed, 165 insertions(+), 4 deletions(-)
 create mode 100755 tests/generic/678
 create mode 100644 tests/generic/678.out
 create mode 100755 tests/generic/679
 create mode 100644 tests/generic/679.out

--
2.31.1

Comments

Zorro Lang March 31, 2022, 2:59 p.m. UTC | #1
On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> Hello,

Hi,

Your below patches looks like not pure text format, they might contain
binary character or some special characers, looks like the "^M" [1].

So please check how you generate/edit/send these patches, make sure
they're pure text for Linux then send again.

Thanks,
Zorro


[1]
# cat -A /path/to/your_patch
index 95752d3b..5e73cff9 100755^M$
--- a/tests/generic/468^M$
+++ b/tests/generic/468^M$
@@ -34,6 +34,13 @@ _scratch_mkfs >/dev/null 2>&1^M$
 _require_metadata_journaling $SCRATCH_DEV^M$
 _scratch_mount^M$
 ^M$
+# blocksize and fact are used in the last case of the fsync/fdatasync test.^M$
+# This is mainly trying to test recovery operation in case where the data^M$
+# blocks written, exceeds the default flex group size (32768*4096*16) in ext4.^M$
+blocks=32768^M$
+blocksize=4096^M$
+fact=18^M$
+^M$
...
...

> 
> The ext4 fast_commit kernel fix has landed into mainline tree [1].
> In this v3, I have addressed review comments from Darrick.
> Does this looks good to be picked up?
> 
> I have tested ext4 1k, 4k (w & w/o fast_commit). Also tested other FS with
> default configs (like xfs, btrfs, f2fs). No surprises were seen.
> 
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bfdc502a4a4c058bf4cbb1df0c297761d528f54d
> 
> -ritesh
> 
> Changelogs:
> ===========
> 
> v2 => v3
> =========
> 1. Addressed review comments from Darrick.
> 2. Rebased to latest master.
> 
> v1 => v2
> =========
> Sending v2 with tests/ext4/ converted to tests/generic/
> (although I had not received any review comments on v1).
> It seems all of the tests which I had sent in v1 are not ext4 specific anyways.
> So in v2, I have made those into tests/generic/.
> 
> Summary
> =========
> These are some of the tests which when tested with ext4 fast_commit feature
> w/o kernel fixes, could cause tests failures and/or KASAN bug (generic/486).
> 
> I gave these tests a run with default xfs, btrfs and f2fs configs (along with
> ext4). No surprises observed.
> 
> [v2]: https://lore.kernel.org/all/cover.1647342932.git.riteshh@linux.ibm.com/
> [v1]: https://lore.kernel.org/all/cover.1644070604.git.riteshh@linux.ibm.com/
> 
> 
> Ritesh Harjani (4):
>   generic/468: Add another falloc test entry
>   common/punch: Add block_size argument to _filter_fiemap_**
>   generic/678: Add a new shutdown recovery test
>   generic/679: Add a test to check unwritten extents tracking
> 
>  common/punch          |  9 +++---
>  tests/generic/468     |  8 +++++
>  tests/generic/468.out |  2 ++
>  tests/generic/678     | 72 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/678.out |  7 +++++
>  tests/generic/679     | 65 ++++++++++++++++++++++++++++++++++++++
>  tests/generic/679.out |  6 ++++
>  7 files changed, 165 insertions(+), 4 deletions(-)
>  create mode 100755 tests/generic/678
>  create mode 100644 tests/generic/678.out
>  create mode 100755 tests/generic/679
>  create mode 100644 tests/generic/679.out
> 
> --
> 2.31.1
>
Ritesh Harjani (IBM) March 31, 2022, 4:19 p.m. UTC | #2
On 22/03/31 10:59PM, Zorro Lang wrote:
> On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> > Hello,
>
> Hi,
>
> Your below patches looks like not pure text format, they might contain
> binary character or some special characers, looks like the "^M" [1].

Ohh, is it. Might be some settings needed to be tweaked for using
gmail for community patches.

>
> So please check how you generate/edit/send these patches, make sure
> they're pure text for Linux then send again.

Sure Zorro, I will check it again at my end. However, when I tried using
b4 am -o - cover.1648730443.git.ritesh.list@gmail.com | git am

It worked fine for me and applied all 4 patches to my local tree.

I also didn't notice any of such characters in my other inbox
("riteshh@in.ibm.com"), which was cc'd too. hmm, strange.

Sure, I will check it once again at my end and resend this.

-ritesh


>
> Thanks,
> Zorro
>
>
> [1]
> # cat -A /path/to/your_patch
> index 95752d3b..5e73cff9 100755^M$
> --- a/tests/generic/468^M$
> +++ b/tests/generic/468^M$
> @@ -34,6 +34,13 @@ _scratch_mkfs >/dev/null 2>&1^M$
>  _require_metadata_journaling $SCRATCH_DEV^M$
>  _scratch_mount^M$
>  ^M$
> +# blocksize and fact are used in the last case of the fsync/fdatasync test.^M$
> +# This is mainly trying to test recovery operation in case where the data^M$
> +# blocks written, exceeds the default flex group size (32768*4096*16) in ext4.^M$
> +blocks=32768^M$
> +blocksize=4096^M$
> +fact=18^M$
> +^M$
> ...
> ...
>
> >
> > The ext4 fast_commit kernel fix has landed into mainline tree [1].
> > In this v3, I have addressed review comments from Darrick.
> > Does this looks good to be picked up?
> >
> > I have tested ext4 1k, 4k (w & w/o fast_commit). Also tested other FS with
> > default configs (like xfs, btrfs, f2fs). No surprises were seen.
> >
> > [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bfdc502a4a4c058bf4cbb1df0c297761d528f54d
> >
> > -ritesh
> >
> > Changelogs:
> > ===========
> >
> > v2 => v3
> > =========
> > 1. Addressed review comments from Darrick.
> > 2. Rebased to latest master.
> >
> > v1 => v2
> > =========
> > Sending v2 with tests/ext4/ converted to tests/generic/
> > (although I had not received any review comments on v1).
> > It seems all of the tests which I had sent in v1 are not ext4 specific anyways.
> > So in v2, I have made those into tests/generic/.
> >
> > Summary
> > =========
> > These are some of the tests which when tested with ext4 fast_commit feature
> > w/o kernel fixes, could cause tests failures and/or KASAN bug (generic/486).
> >
> > I gave these tests a run with default xfs, btrfs and f2fs configs (along with
> > ext4). No surprises observed.
> >
> > [v2]: https://lore.kernel.org/all/cover.1647342932.git.riteshh@linux.ibm.com/
> > [v1]: https://lore.kernel.org/all/cover.1644070604.git.riteshh@linux.ibm.com/
> >
> >
> > Ritesh Harjani (4):
> >   generic/468: Add another falloc test entry
> >   common/punch: Add block_size argument to _filter_fiemap_**
> >   generic/678: Add a new shutdown recovery test
> >   generic/679: Add a test to check unwritten extents tracking
> >
> >  common/punch          |  9 +++---
> >  tests/generic/468     |  8 +++++
> >  tests/generic/468.out |  2 ++
> >  tests/generic/678     | 72 +++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/678.out |  7 +++++
> >  tests/generic/679     | 65 ++++++++++++++++++++++++++++++++++++++
> >  tests/generic/679.out |  6 ++++
> >  7 files changed, 165 insertions(+), 4 deletions(-)
> >  create mode 100755 tests/generic/678
> >  create mode 100644 tests/generic/678.out
> >  create mode 100755 tests/generic/679
> >  create mode 100644 tests/generic/679.out
> >
> > --
> > 2.31.1
> >
>
Ritesh Harjani (IBM) March 31, 2022, 4:53 p.m. UTC | #3
On 22/03/31 09:49PM, Ritesh Harjani wrote:
> On 22/03/31 10:59PM, Zorro Lang wrote:
> > On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> > > Hello,
> >
> > Hi,
> >
> > Your below patches looks like not pure text format, they might contain
> > binary character or some special characers, looks like the "^M" [1].

Sorry to bother you. But here is what I tried.
1. Download the mbx file using b4 am. I didn't see any such character ("^M") in
   the patches.
2. Saved the patch using mutt. Again didn't see such character while doing
	cat -A /patch/to/patch
3. Downloaded the mail using eml format from webmail. Here I do see this
   character appended. But that happens not just for my patch, but for all
   other patches too.

So could this be related to the way you are downloading these patches.
Please let me know, if I need to resend these patches again? Because, I don't
see this behavior at my end. But I would happy to correct it, if that's not the
case.

-ritesh
Zorro Lang April 1, 2022, 5:30 a.m. UTC | #4
On Thu, Mar 31, 2022 at 10:23:35PM +0530, Ritesh Harjani wrote:
> On 22/03/31 09:49PM, Ritesh Harjani wrote:
> > On 22/03/31 10:59PM, Zorro Lang wrote:
> > > On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> > > > Hello,
> > >
> > > Hi,
> > >
> > > Your below patches looks like not pure text format, they might contain
> > > binary character or some special characers, looks like the "^M" [1].
> 
> Sorry to bother you. But here is what I tried.
> 1. Download the mbx file using b4 am. I didn't see any such character ("^M") in
>    the patches.
> 2. Saved the patch using mutt. Again didn't see such character while doing
> 	cat -A /patch/to/patch
> 3. Downloaded the mail using eml format from webmail. Here I do see this
>    character appended. But that happens not just for my patch, but for all
>    other patches too.
> 
> So could this be related to the way you are downloading these patches.
> Please let me know, if I need to resend these patches again? Because, I don't
> see this behavior at my end. But I would happy to correct it, if that's not the
> case.

Hmm... weird, When I tried to open your patch emails, my mutt show me:

  [-- application/octet-stream is unsupported (use 'v' to view this part) --]

Then I have to input 'v' to see the patch content. I'm not sure what's wrong,
this's the 2nd time I hit this "octet-stream is unsupported" issue yesterday.

Hi Darrick, or any other forks, can you open above 4 patches normally? If that's
only my personal issue, I'll check my side.

Thanks,
Zorro

> 
> -ritesh
>
Ojaswin Mujoo April 1, 2022, 5:55 a.m. UTC | #5
On Fri, Apr 01, 2022 at 01:30:47PM +0800, Zorro Lang wrote:
> On Thu, Mar 31, 2022 at 10:23:35PM +0530, Ritesh Harjani wrote:
> > On 22/03/31 09:49PM, Ritesh Harjani wrote:
> > > On 22/03/31 10:59PM, Zorro Lang wrote:
> > > > On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> > > > > Hello,
> > > >
> > > > Hi,
> > > >
> > > > Your below patches looks like not pure text format, they might contain
> > > > binary character or some special characers, looks like the "^M" [1].
> > 
> > Sorry to bother you. But here is what I tried.
> > 1. Download the mbx file using b4 am. I didn't see any such character ("^M") in
> >    the patches.
> > 2. Saved the patch using mutt. Again didn't see such character while doing
> > 	cat -A /patch/to/patch
> > 3. Downloaded the mail using eml format from webmail. Here I do see this
> >    character appended. But that happens not just for my patch, but for all
> >    other patches too.
> > 
> > So could this be related to the way you are downloading these patches.
> > Please let me know, if I need to resend these patches again? Because, I don't
> > see this behavior at my end. But I would happy to correct it, if that's not the
> > case.
> 
> Hmm... weird, When I tried to open your patch emails, my mutt show me:
> 
>   [-- application/octet-stream is unsupported (use 'v' to view this part) --]
> 
> Then I have to input 'v' to see the patch content. I'm not sure what's wrong,
> this's the 2nd time I hit this "octet-stream is unsupported" issue yesterday.
> 
> Hi Darrick, or any other forks, can you open above 4 patches normally? If that's
> only my personal issue, I'll check my side.
Hi Zorro,

The patchset seems to open normally at my end.

Thanks,
Ojaswin
> 
> Thanks,
> Zorro
> 
> > 
> > -ritesh
> > 
>
Darrick J. Wong April 1, 2022, 5:04 p.m. UTC | #6
On Fri, Apr 01, 2022 at 01:30:47PM +0800, Zorro Lang wrote:
> On Thu, Mar 31, 2022 at 10:23:35PM +0530, Ritesh Harjani wrote:
> > On 22/03/31 09:49PM, Ritesh Harjani wrote:
> > > On 22/03/31 10:59PM, Zorro Lang wrote:
> > > > On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> > > > > Hello,
> > > >
> > > > Hi,
> > > >
> > > > Your below patches looks like not pure text format, they might contain
> > > > binary character or some special characers, looks like the "^M" [1].
> > 
> > Sorry to bother you. But here is what I tried.
> > 1. Download the mbx file using b4 am. I didn't see any such character ("^M") in
> >    the patches.
> > 2. Saved the patch using mutt. Again didn't see such character while doing
> > 	cat -A /patch/to/patch
> > 3. Downloaded the mail using eml format from webmail. Here I do see this
> >    character appended. But that happens not just for my patch, but for all
> >    other patches too.
> > 
> > So could this be related to the way you are downloading these patches.
> > Please let me know, if I need to resend these patches again? Because, I don't
> > see this behavior at my end. But I would happy to correct it, if that's not the
> > case.
> 
> Hmm... weird, When I tried to open your patch emails, my mutt show me:
> 
>   [-- application/octet-stream is unsupported (use 'v' to view this part) --]
> 
> Then I have to input 'v' to see the patch content. I'm not sure what's wrong,
> this's the 2nd time I hit this "octet-stream is unsupported" issue yesterday.
> 
> Hi Darrick, or any other forks, can you open above 4 patches normally? If that's
> only my personal issue, I'll check my side.

There's no application/octet anywhere in the email that I received.
Has your IT department gone rogue^W^Wincreased value-add again?

--D

> Thanks,
> Zorro
> 
> > 
> > -ritesh
> > 
>
Zorro Lang April 2, 2022, 3:40 a.m. UTC | #7
On Fri, Apr 01, 2022 at 10:04:51AM -0700, Darrick J. Wong wrote:
> On Fri, Apr 01, 2022 at 01:30:47PM +0800, Zorro Lang wrote:
> > On Thu, Mar 31, 2022 at 10:23:35PM +0530, Ritesh Harjani wrote:
> > > On 22/03/31 09:49PM, Ritesh Harjani wrote:
> > > > On 22/03/31 10:59PM, Zorro Lang wrote:
> > > > > On Thu, Mar 31, 2022 at 06:24:19PM +0530, Ritesh Harjani wrote:
> > > > > > Hello,
> > > > >
> > > > > Hi,
> > > > >
> > > > > Your below patches looks like not pure text format, they might contain
> > > > > binary character or some special characers, looks like the "^M" [1].
> > > 
> > > Sorry to bother you. But here is what I tried.
> > > 1. Download the mbx file using b4 am. I didn't see any such character ("^M") in
> > >    the patches.
> > > 2. Saved the patch using mutt. Again didn't see such character while doing
> > > 	cat -A /patch/to/patch
> > > 3. Downloaded the mail using eml format from webmail. Here I do see this
> > >    character appended. But that happens not just for my patch, but for all
> > >    other patches too.
> > > 
> > > So could this be related to the way you are downloading these patches.
> > > Please let me know, if I need to resend these patches again? Because, I don't
> > > see this behavior at my end. But I would happy to correct it, if that's not the
> > > case.
> > 
> > Hmm... weird, When I tried to open your patch emails, my mutt show me:
> > 
> >   [-- application/octet-stream is unsupported (use 'v' to view this part) --]
> > 
> > Then I have to input 'v' to see the patch content. I'm not sure what's wrong,
> > this's the 2nd time I hit this "octet-stream is unsupported" issue yesterday.
> > 
> > Hi Darrick, or any other forks, can you open above 4 patches normally? If that's
> > only my personal issue, I'll check my side.
> 
> There's no application/octet anywhere in the email that I received.
> Has your IT department gone rogue^W^Wincreased value-add again?

Thanks Darrick and Ojaswin! I'll check with our IT department. But it's weird, only
this patchset and 2 of another patch[1](it's sent 3 times) has this problem, I see
all other patches normally. And Ritesh, sorry for taking your time to check it :-D

Thanks,
Zorro

[1]
[PATCH v2] generic/674: replace _require_scratch_reflink with _require_scratch_dedupe

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > > 
> > > -ritesh
> > > 
> > 
>