| Message ID | 1563553816-148827-1-git-send-email-andrey.shinkevich@virtuozzo.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Allow Valgrind checking all QEMU processes | expand |
PINGING... On 19/07/2019 19:30, Andrey Shinkevich wrote: > In the current implementation of the QEMU bash iotests, only qemu-io > processes may be run under the Valgrind, which is a useful tool for > finding memory usage issues. Let's allow the common.rc bash script > runing all the QEMU processes, such as qemu-kvm, qemu-img, qemu-ndb > and qemu-vxhs, under the Valgrind tool. > > v5: > 01: The patch "block/nbd: NBDReply is used being uninitialized" was detached > and taken into account in the patch "nbd: Initialize reply on failure" > by Eric Blake. > > v4: > 01: The patch "iotests: Set read-zeroes on in null block driver for Valgrind" > was extended with new cases and issued as a separate series. > 02: The new patch "block/nbd: NBDReply is used being uninitialized" was > added to resolve the failure of the iotest 083 run under Valgrind. > > v3: > 01: The new function _casenotrun() was added to the common.rc bash > script to notify the user of test cases dropped for some reason. > Suggested by Kevin. > Particularly, the notification about the nonexistent TMPDIR in > the test 051 was added (noticed by Vladimir). > 02: The timeout in some test cases was extended for Valgrind because > it differs when running on the ramdisk. > 03: Due to the common.nbd script has been changed with the commit > b28f582c, the patch "iotests: amend QEMU NBD process synchronization" > is actual no more. Note that QEMU_NBD is launched in the bash nested > shell in the _qemu_nbd_wrapper() as it was before in common.rc. > 04: The patch "iotests: new file to suppress Valgrind errors" was dropped > due to my superficial understanding of the work of the function > blk_pread_unthrottled(). Special thanks to Kevin who shed the light > on the null block driver involved. Now, the parameter 'read-zeroes=on' > is passed to the null block driver to initialize the buffer in the > function guess_disk_lchs() that the Valgrind was complaining to. > > v2: > 01: The patch 2/7 of v1 was merged into the patch 1/7, suggested by Daniel. > 02: Another patch 7/7 was added to introduce the Valgrind error suppression > file into the QEMU project. > Discussed in the email thread with the message ID: > <1560276131-683243-1-git-send-email-andrey.shinkevich@virtuozzo.com> > > Andrey Shinkevich (6): > iotests: allow Valgrind checking all QEMU processes > iotests: exclude killed processes from running under Valgrind > iotests: Add casenotrun report to bash tests > iotests: Valgrind fails with nonexistent directory > iotests: extended timeout under Valgrind > iotests: extend sleeping time under Valgrind > > tests/qemu-iotests/028 | 6 +++- > tests/qemu-iotests/039 | 5 +++ > tests/qemu-iotests/039.out | 30 +++-------------- > tests/qemu-iotests/051 | 4 +++ > tests/qemu-iotests/061 | 2 ++ > tests/qemu-iotests/061.out | 12 ++----- > tests/qemu-iotests/137 | 1 + > tests/qemu-iotests/137.out | 6 +--- > tests/qemu-iotests/183 | 9 +++++- > tests/qemu-iotests/192 | 6 +++- > tests/qemu-iotests/247 | 6 +++- > tests/qemu-iotests/common.rc | 76 +++++++++++++++++++++++++++++++++----------- > 12 files changed, 101 insertions(+), 62 deletions(-) >
On Fri, Jul 19, 2019 at 07:30:10PM +0300, Andrey Shinkevich wrote: > In the current implementation of the QEMU bash iotests, only qemu-io > processes may be run under the Valgrind, which is a useful tool for > finding memory usage issues. Let's allow the common.rc bash script > runing all the QEMU processes, such as qemu-kvm, qemu-img, qemu-ndb > and qemu-vxhs, under the Valgrind tool. > FIY, this looks very similar (in purpose) to: https://avocado-framework.readthedocs.io/en/71.0/WrapProcess.html And in fact Valgrind was one of the original motivations: https://github.com/avocado-framework/avocado/blob/master/examples/wrappers/valgrind.sh Maybe this can be helpful for the Python based iotests. - Cleber. > v5: > 01: The patch "block/nbd: NBDReply is used being uninitialized" was detached > and taken into account in the patch "nbd: Initialize reply on failure" > by Eric Blake. > > v4: > 01: The patch "iotests: Set read-zeroes on in null block driver for Valgrind" > was extended with new cases and issued as a separate series. > 02: The new patch "block/nbd: NBDReply is used being uninitialized" was > added to resolve the failure of the iotest 083 run under Valgrind. > > v3: > 01: The new function _casenotrun() was added to the common.rc bash > script to notify the user of test cases dropped for some reason. > Suggested by Kevin. > Particularly, the notification about the nonexistent TMPDIR in > the test 051 was added (noticed by Vladimir). > 02: The timeout in some test cases was extended for Valgrind because > it differs when running on the ramdisk. > 03: Due to the common.nbd script has been changed with the commit > b28f582c, the patch "iotests: amend QEMU NBD process synchronization" > is actual no more. Note that QEMU_NBD is launched in the bash nested > shell in the _qemu_nbd_wrapper() as it was before in common.rc. > 04: The patch "iotests: new file to suppress Valgrind errors" was dropped > due to my superficial understanding of the work of the function > blk_pread_unthrottled(). Special thanks to Kevin who shed the light > on the null block driver involved. Now, the parameter 'read-zeroes=on' > is passed to the null block driver to initialize the buffer in the > function guess_disk_lchs() that the Valgrind was complaining to. > > v2: > 01: The patch 2/7 of v1 was merged into the patch 1/7, suggested by Daniel. > 02: Another patch 7/7 was added to introduce the Valgrind error suppression > file into the QEMU project. > Discussed in the email thread with the message ID: > <1560276131-683243-1-git-send-email-andrey.shinkevich@virtuozzo.com> > > Andrey Shinkevich (6): > iotests: allow Valgrind checking all QEMU processes > iotests: exclude killed processes from running under Valgrind > iotests: Add casenotrun report to bash tests > iotests: Valgrind fails with nonexistent directory > iotests: extended timeout under Valgrind > iotests: extend sleeping time under Valgrind > > tests/qemu-iotests/028 | 6 +++- > tests/qemu-iotests/039 | 5 +++ > tests/qemu-iotests/039.out | 30 +++-------------- > tests/qemu-iotests/051 | 4 +++ > tests/qemu-iotests/061 | 2 ++ > tests/qemu-iotests/061.out | 12 ++----- > tests/qemu-iotests/137 | 1 + > tests/qemu-iotests/137.out | 6 +--- > tests/qemu-iotests/183 | 9 +++++- > tests/qemu-iotests/192 | 6 +++- > tests/qemu-iotests/247 | 6 +++- > tests/qemu-iotests/common.rc | 76 +++++++++++++++++++++++++++++++++----------- > 12 files changed, 101 insertions(+), 62 deletions(-) > > -- > 1.8.3.1 > >
On 16/08/2019 23:05, Cleber Rosa wrote: > On Fri, Jul 19, 2019 at 07:30:10PM +0300, Andrey Shinkevich wrote: >> In the current implementation of the QEMU bash iotests, only qemu-io >> processes may be run under the Valgrind, which is a useful tool for >> finding memory usage issues. Let's allow the common.rc bash script >> runing all the QEMU processes, such as qemu-kvm, qemu-img, qemu-ndb >> and qemu-vxhs, under the Valgrind tool. >> > > FIY, this looks very similar (in purpose) to: > > https://avocado-framework.readthedocs.io/en/71.0/WrapProcess.html > > And in fact Valgrind was one of the original motivations: > > https://github.com/avocado-framework/avocado/blob/master/examples/wrappers/valgrind.sh > > Maybe this can be helpful for the Python based iotests. > > - Cleber. > Thank you Cleber for the advice. That is the way I actually ran Python iotests under Valgrind on my host and discovered some issues with them already. Andrey >> v5: >> 01: The patch "block/nbd: NBDReply is used being uninitialized" was detached >> and taken into account in the patch "nbd: Initialize reply on failure" >> by Eric Blake. >> >> v4: >> 01: The patch "iotests: Set read-zeroes on in null block driver for Valgrind" >> was extended with new cases and issued as a separate series. >> 02: The new patch "block/nbd: NBDReply is used being uninitialized" was >> added to resolve the failure of the iotest 083 run under Valgrind. >> >> v3: >> 01: The new function _casenotrun() was added to the common.rc bash >> script to notify the user of test cases dropped for some reason. >> Suggested by Kevin. >> Particularly, the notification about the nonexistent TMPDIR in >> the test 051 was added (noticed by Vladimir). >> 02: The timeout in some test cases was extended for Valgrind because >> it differs when running on the ramdisk. >> 03: Due to the common.nbd script has been changed with the commit >> b28f582c, the patch "iotests: amend QEMU NBD process synchronization" >> is actual no more. Note that QEMU_NBD is launched in the bash nested >> shell in the _qemu_nbd_wrapper() as it was before in common.rc. >> 04: The patch "iotests: new file to suppress Valgrind errors" was dropped >> due to my superficial understanding of the work of the function >> blk_pread_unthrottled(). Special thanks to Kevin who shed the light >> on the null block driver involved. Now, the parameter 'read-zeroes=on' >> is passed to the null block driver to initialize the buffer in the >> function guess_disk_lchs() that the Valgrind was complaining to. >> >> v2: >> 01: The patch 2/7 of v1 was merged into the patch 1/7, suggested by Daniel. >> 02: Another patch 7/7 was added to introduce the Valgrind error suppression >> file into the QEMU project. >> Discussed in the email thread with the message ID: >> <1560276131-683243-1-git-send-email-andrey.shinkevich@virtuozzo.com> >> >> Andrey Shinkevich (6): >> iotests: allow Valgrind checking all QEMU processes >> iotests: exclude killed processes from running under Valgrind >> iotests: Add casenotrun report to bash tests >> iotests: Valgrind fails with nonexistent directory >> iotests: extended timeout under Valgrind >> iotests: extend sleeping time under Valgrind >> >> tests/qemu-iotests/028 | 6 +++- >> tests/qemu-iotests/039 | 5 +++ >> tests/qemu-iotests/039.out | 30 +++-------------- >> tests/qemu-iotests/051 | 4 +++ >> tests/qemu-iotests/061 | 2 ++ >> tests/qemu-iotests/061.out | 12 ++----- >> tests/qemu-iotests/137 | 1 + >> tests/qemu-iotests/137.out | 6 +--- >> tests/qemu-iotests/183 | 9 +++++- >> tests/qemu-iotests/192 | 6 +++- >> tests/qemu-iotests/247 | 6 +++- >> tests/qemu-iotests/common.rc | 76 +++++++++++++++++++++++++++++++++----------- >> 12 files changed, 101 insertions(+), 62 deletions(-) >> >> -- >> 1.8.3.1 >> >>
In the current implementation of the QEMU bash iotests, only qemu-io processes may be run under the Valgrind, which is a useful tool for finding memory usage issues. Let's allow the common.rc bash script runing all the QEMU processes, such as qemu-kvm, qemu-img, qemu-ndb and qemu-vxhs, under the Valgrind tool. v5: 01: The patch "block/nbd: NBDReply is used being uninitialized" was detached and taken into account in the patch "nbd: Initialize reply on failure" by Eric Blake. v4: 01: The patch "iotests: Set read-zeroes on in null block driver for Valgrind" was extended with new cases and issued as a separate series. 02: The new patch "block/nbd: NBDReply is used being uninitialized" was added to resolve the failure of the iotest 083 run under Valgrind. v3: 01: The new function _casenotrun() was added to the common.rc bash script to notify the user of test cases dropped for some reason. Suggested by Kevin. Particularly, the notification about the nonexistent TMPDIR in the test 051 was added (noticed by Vladimir). 02: The timeout in some test cases was extended for Valgrind because it differs when running on the ramdisk. 03: Due to the common.nbd script has been changed with the commit b28f582c, the patch "iotests: amend QEMU NBD process synchronization" is actual no more. Note that QEMU_NBD is launched in the bash nested shell in the _qemu_nbd_wrapper() as it was before in common.rc. 04: The patch "iotests: new file to suppress Valgrind errors" was dropped due to my superficial understanding of the work of the function blk_pread_unthrottled(). Special thanks to Kevin who shed the light on the null block driver involved. Now, the parameter 'read-zeroes=on' is passed to the null block driver to initialize the buffer in the function guess_disk_lchs() that the Valgrind was complaining to. v2: 01: The patch 2/7 of v1 was merged into the patch 1/7, suggested by Daniel. 02: Another patch 7/7 was added to introduce the Valgrind error suppression file into the QEMU project. Discussed in the email thread with the message ID: <1560276131-683243-1-git-send-email-andrey.shinkevich@virtuozzo.com> Andrey Shinkevich (6): iotests: allow Valgrind checking all QEMU processes iotests: exclude killed processes from running under Valgrind iotests: Add casenotrun report to bash tests iotests: Valgrind fails with nonexistent directory iotests: extended timeout under Valgrind iotests: extend sleeping time under Valgrind tests/qemu-iotests/028 | 6 +++- tests/qemu-iotests/039 | 5 +++ tests/qemu-iotests/039.out | 30 +++-------------- tests/qemu-iotests/051 | 4 +++ tests/qemu-iotests/061 | 2 ++ tests/qemu-iotests/061.out | 12 ++----- tests/qemu-iotests/137 | 1 + tests/qemu-iotests/137.out | 6 +--- tests/qemu-iotests/183 | 9 +++++- tests/qemu-iotests/192 | 6 +++- tests/qemu-iotests/247 | 6 +++- tests/qemu-iotests/common.rc | 76 +++++++++++++++++++++++++++++++++----------- 12 files changed, 101 insertions(+), 62 deletions(-)