| Message ID | 20210127072143.247849-1-zlang@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | generic/233,270: unlimit the max locked memory size for io_uring | expand |
On Wed, Jan 27, 2021 at 03:21:43PM +0800, Zorro Lang wrote: > The ltp/fsstress always fails on io_uring_queue_init() by returnning > ENOMEM. Due to io_uring accounts memory it needs under the rlimit > memlocked option, which can be quite low on some setups, especially > on 64K pagesize machine. root isn't under this restriction, but > regular users are. So only g/233 and g/270 which use $qa_user to run > fsstress are failed. > > To avoid this failure, set max locked memory to unlimited before doing > fsstress, then restore it after test done. > > Signed-off-by: Zorro Lang <zlang@redhat.com> > --- > tests/generic/233 | 8 ++++++++ > tests/generic/270 | 8 ++++++++ > 2 files changed, 16 insertions(+) > > diff --git a/tests/generic/233 b/tests/generic/233 > index 7eda5774..342690c2 100755 > --- a/tests/generic/233 > +++ b/tests/generic/233 > @@ -43,6 +43,13 @@ _fsstress() > -f rename=10 -f fsync=2 -f write=15 -f dwrite=15 \ > -n $count -d $out -p 7` > > + # io_uring accounts memory it needs under the rlimit memlocked option, > + # which can be quite low on some setups (especially 64K pagesize). root > + # isn't under this restriction, but regular users are. To avoid the > + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited > + # temporarily. > + lmem=`ulimit -l` > + ulimit -l unlimited Should this apply to all the other tests that run fsstress? $ grep -l FSSTRESS_PROG tests/ | wc -l 94 --D > echo "fsstress $args" >> $seqres.full > if ! su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num > then > @@ -50,6 +57,7 @@ _fsstress() > cat $tmp.out | tee -a $seqres.full > status=1 > fi > + ulimit -l $lmem > } > > # real QA test starts here > diff --git a/tests/generic/270 b/tests/generic/270 > index 3d8656d4..bd52d56e 100755 > --- a/tests/generic/270 > +++ b/tests/generic/270 > @@ -37,6 +37,13 @@ _workout() > cp $FSSTRESS_PROG $tmp.fsstress.bin > $SETCAP_PROG cap_chown=epi $tmp.fsstress.bin > > + # io_uring accounts memory it needs under the rlimit memlocked option, > + # which can be quite low on some setups (especially 64K pagesize). root > + # isn't under this restriction, but regular users are. To avoid the > + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited > + # temporarily. > + lmem=`ulimit -l` > + ulimit -l unlimited > (su $qa_user -c "$tmp.fsstress.bin $args" &) > /dev/null 2>&1 > > echo "Run dd writers in parallel" > @@ -50,6 +57,7 @@ _workout() > done > > $KILLALL_PROG -w $tmp.fsstress.bin > + ulimit -l $lmem > } > > # real QA test starts here > -- > 2.29.2 >
On Wed, Jan 27, 2021 at 03:43:40PM -0800, Darrick J. Wong wrote: > On Wed, Jan 27, 2021 at 03:21:43PM +0800, Zorro Lang wrote: > > The ltp/fsstress always fails on io_uring_queue_init() by returnning > > ENOMEM. Due to io_uring accounts memory it needs under the rlimit > > memlocked option, which can be quite low on some setups, especially > > on 64K pagesize machine. root isn't under this restriction, but > > regular users are. So only g/233 and g/270 which use $qa_user to run > > fsstress are failed. > > > > To avoid this failure, set max locked memory to unlimited before doing > > fsstress, then restore it after test done. > > > > Signed-off-by: Zorro Lang <zlang@redhat.com> > > --- > > tests/generic/233 | 8 ++++++++ > > tests/generic/270 | 8 ++++++++ > > 2 files changed, 16 insertions(+) > > > > diff --git a/tests/generic/233 b/tests/generic/233 > > index 7eda5774..342690c2 100755 > > --- a/tests/generic/233 > > +++ b/tests/generic/233 > > @@ -43,6 +43,13 @@ _fsstress() > > -f rename=10 -f fsync=2 -f write=15 -f dwrite=15 \ > > -n $count -d $out -p 7` > > > > + # io_uring accounts memory it needs under the rlimit memlocked option, > > + # which can be quite low on some setups (especially 64K pagesize). root > > + # isn't under this restriction, but regular users are. To avoid the > > + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited > > + # temporarily. > > + lmem=`ulimit -l` > > + ulimit -l unlimited > > Should this apply to all the other tests that run fsstress? > > $ grep -l FSSTRESS_PROG tests/ | wc -l > 94 The root user isn't under this restriction, only g/233 and g/270 run fsstress with non-root user ($qa_user). Hmm... is that possible to run xfstests with a non-root user? Or you'd like to write a common helper _fsstress() (if run fsx with non-root user, I doubt it's needed too)... Thanks, Zorro > > --D > > > echo "fsstress $args" >> $seqres.full > > if ! su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num > > then > > @@ -50,6 +57,7 @@ _fsstress() > > cat $tmp.out | tee -a $seqres.full > > status=1 > > fi > > + ulimit -l $lmem > > } > > > > # real QA test starts here > > diff --git a/tests/generic/270 b/tests/generic/270 > > index 3d8656d4..bd52d56e 100755 > > --- a/tests/generic/270 > > +++ b/tests/generic/270 > > @@ -37,6 +37,13 @@ _workout() > > cp $FSSTRESS_PROG $tmp.fsstress.bin > > $SETCAP_PROG cap_chown=epi $tmp.fsstress.bin > > > > + # io_uring accounts memory it needs under the rlimit memlocked option, > > + # which can be quite low on some setups (especially 64K pagesize). root > > + # isn't under this restriction, but regular users are. To avoid the > > + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited > > + # temporarily. > > + lmem=`ulimit -l` > > + ulimit -l unlimited > > (su $qa_user -c "$tmp.fsstress.bin $args" &) > /dev/null 2>&1 > > > > echo "Run dd writers in parallel" > > @@ -50,6 +57,7 @@ _workout() > > done > > > > $KILLALL_PROG -w $tmp.fsstress.bin > > + ulimit -l $lmem > > } > > > > # real QA test starts here > > -- > > 2.29.2 > > >
On Thu, Jan 28, 2021 at 11:02:09AM +0800, Zorro Lang wrote: > On Wed, Jan 27, 2021 at 03:43:40PM -0800, Darrick J. Wong wrote: > > On Wed, Jan 27, 2021 at 03:21:43PM +0800, Zorro Lang wrote: > > > The ltp/fsstress always fails on io_uring_queue_init() by returnning > > > ENOMEM. Due to io_uring accounts memory it needs under the rlimit > > > memlocked option, which can be quite low on some setups, especially > > > on 64K pagesize machine. root isn't under this restriction, but > > > regular users are. So only g/233 and g/270 which use $qa_user to run > > > fsstress are failed. > > > > > > To avoid this failure, set max locked memory to unlimited before doing > > > fsstress, then restore it after test done. > > > > > > Signed-off-by: Zorro Lang <zlang@redhat.com> > > > --- > > > tests/generic/233 | 8 ++++++++ > > > tests/generic/270 | 8 ++++++++ > > > 2 files changed, 16 insertions(+) > > > > > > diff --git a/tests/generic/233 b/tests/generic/233 > > > index 7eda5774..342690c2 100755 > > > --- a/tests/generic/233 > > > +++ b/tests/generic/233 > > > @@ -43,6 +43,13 @@ _fsstress() > > > -f rename=10 -f fsync=2 -f write=15 -f dwrite=15 \ > > > -n $count -d $out -p 7` > > > > > > + # io_uring accounts memory it needs under the rlimit memlocked option, > > > + # which can be quite low on some setups (especially 64K pagesize). root > > > + # isn't under this restriction, but regular users are. To avoid the > > > + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited > > > + # temporarily. > > > + lmem=`ulimit -l` > > > + ulimit -l unlimited > > > > Should this apply to all the other tests that run fsstress? > > > > $ grep -l FSSTRESS_PROG tests/ | wc -l > > 94 > > The root user isn't under this restriction, only g/233 and g/270 run fsstress > with non-root user ($qa_user). > Hmm... is that possible to run xfstests with a non-root user? Or you'd like to No, fstests is required to be run by root user, there's a check in 'check' script :) > write a common helper _fsstress() (if run fsx with non-root user, I doubt it's > needed too)... Perhaps we should raise the limit in fsstress and fsx while setting up io_uring_queue_init()? So we don't have to add this ulimit settings to every test that runs fsstress/fsx with regular user. Thanks, Eryu
On Mon, Feb 01, 2021 at 12:34:13AM +0800, Zorro Lang wrote: > On Sun, Jan 31, 2021 at 11:24:52PM +0800, Eryu Guan wrote: > > On Thu, Jan 28, 2021 at 11:02:09AM +0800, Zorro Lang wrote: > > > On Wed, Jan 27, 2021 at 03:43:40PM -0800, Darrick J. Wong wrote: > > > > On Wed, Jan 27, 2021 at 03:21:43PM +0800, Zorro Lang wrote: > > > > > The ltp/fsstress always fails on io_uring_queue_init() by returnning > > > > > ENOMEM. Due to io_uring accounts memory it needs under the rlimit > > > > > memlocked option, which can be quite low on some setups, especially > > > > > on 64K pagesize machine. root isn't under this restriction, but > > > > > regular users are. So only g/233 and g/270 which use $qa_user to run > > > > > fsstress are failed. > > > > > > > > > > To avoid this failure, set max locked memory to unlimited before doing > > > > > fsstress, then restore it after test done. > > > > > > > > > > Signed-off-by: Zorro Lang <zlang@redhat.com> > > > > > --- > > > > > tests/generic/233 | 8 ++++++++ > > > > > tests/generic/270 | 8 ++++++++ > > > > > 2 files changed, 16 insertions(+) > > > > > > > > > > diff --git a/tests/generic/233 b/tests/generic/233 > > > > > index 7eda5774..342690c2 100755 > > > > > --- a/tests/generic/233 > > > > > +++ b/tests/generic/233 > > > > > @@ -43,6 +43,13 @@ _fsstress() > > > > > -f rename=10 -f fsync=2 -f write=15 -f dwrite=15 \ > > > > > -n $count -d $out -p 7` > > > > > > > > > > + # io_uring accounts memory it needs under the rlimit memlocked option, > > > > > + # which can be quite low on some setups (especially 64K pagesize). root > > > > > + # isn't under this restriction, but regular users are. To avoid the > > > > > + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited > > > > > + # temporarily. > > > > > + lmem=`ulimit -l` > > > > > + ulimit -l unlimited > > > > > > > > Should this apply to all the other tests that run fsstress? > > > > > > > > $ grep -l FSSTRESS_PROG tests/ | wc -l > > > > 94 > > > > > > The root user isn't under this restriction, only g/233 and g/270 run fsstress > > > with non-root user ($qa_user). > > > Hmm... is that possible to run xfstests with a non-root user? Or you'd like to > > > > No, fstests is required to be run by root user, there's a check in > > 'check' script :) > > > > > write a common helper _fsstress() (if run fsx with non-root user, I doubt it's > > > needed too)... > > > > Perhaps we should raise the limit in fsstress and fsx while setting up > > io_uring_queue_init()? So we don't have to add this ulimit settings to > > every test that runs fsstress/fsx with regular user. > > Hmm... I don't know if it's the best idea, to change a sysconfig in fsstress. > > If by any chance, in the future, someone case uses fsstress to test a sysconfig > which is changed by fsstress :) And fsstress is invoked by a regular user, and doesn't have the permission to change rlimit.. Thanks, Eryu
On Wed, Jan 27, 2021 at 03:21:43PM +0800, Zorro Lang wrote: > The ltp/fsstress always fails on io_uring_queue_init() by returnning > ENOMEM. Due to io_uring accounts memory it needs under the rlimit > memlocked option, which can be quite low on some setups, especially > on 64K pagesize machine. root isn't under this restriction, but > regular users are. So only g/233 and g/270 which use $qa_user to run > fsstress are failed. > > To avoid this failure, set max locked memory to unlimited before doing > fsstress, then restore it after test done. I'm wondering if we could just set 'ulimit -l unlimited' in check? so all tests inherits the unlimited mlock setting. For root, this doesn't change anything, but command runs by regular user could get unlimited lockable memory. And new tests won't have to worry about running fsstress/fsx as $qa_user. Thanks, Eryu
diff --git a/tests/generic/233 b/tests/generic/233 index 7eda5774..342690c2 100755 --- a/tests/generic/233 +++ b/tests/generic/233 @@ -43,6 +43,13 @@ _fsstress() -f rename=10 -f fsync=2 -f write=15 -f dwrite=15 \ -n $count -d $out -p 7` + # io_uring accounts memory it needs under the rlimit memlocked option, + # which can be quite low on some setups (especially 64K pagesize). root + # isn't under this restriction, but regular users are. To avoid the + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited + # temporarily. + lmem=`ulimit -l` + ulimit -l unlimited echo "fsstress $args" >> $seqres.full if ! su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num then @@ -50,6 +57,7 @@ _fsstress() cat $tmp.out | tee -a $seqres.full status=1 fi + ulimit -l $lmem } # real QA test starts here diff --git a/tests/generic/270 b/tests/generic/270 index 3d8656d4..bd52d56e 100755 --- a/tests/generic/270 +++ b/tests/generic/270 @@ -37,6 +37,13 @@ _workout() cp $FSSTRESS_PROG $tmp.fsstress.bin $SETCAP_PROG cap_chown=epi $tmp.fsstress.bin + # io_uring accounts memory it needs under the rlimit memlocked option, + # which can be quite low on some setups (especially 64K pagesize). root + # isn't under this restriction, but regular users are. To avoid the + # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited + # temporarily. + lmem=`ulimit -l` + ulimit -l unlimited (su $qa_user -c "$tmp.fsstress.bin $args" &) > /dev/null 2>&1 echo "Run dd writers in parallel" @@ -50,6 +57,7 @@ _workout() done $KILLALL_PROG -w $tmp.fsstress.bin + ulimit -l $lmem } # real QA test starts here
The ltp/fsstress always fails on io_uring_queue_init() by returnning ENOMEM. Due to io_uring accounts memory it needs under the rlimit memlocked option, which can be quite low on some setups, especially on 64K pagesize machine. root isn't under this restriction, but regular users are. So only g/233 and g/270 which use $qa_user to run fsstress are failed. To avoid this failure, set max locked memory to unlimited before doing fsstress, then restore it after test done. Signed-off-by: Zorro Lang <zlang@redhat.com> --- tests/generic/233 | 8 ++++++++ tests/generic/270 | 8 ++++++++ 2 files changed, 16 insertions(+)