| Message ID | 20181204155146.9726-6-kieran.bingham@ideasonboard.com (mailing list archive) |
|---|---|
| State | New |
| Delegated to: | Kieran Bingham |
| Headers | show |
| Series | Reset controls and unloved patches | expand |
Hi Kieran, Thank you for the patch. On Tue, Dec 04, 2018 at 03:51:44PM +0000, Kieran Bingham wrote: > Provide a means for the tester to request pixel perfect matches on tests. > This can be either through setting the environment variable VSP_PIXEL_PERFECT, or > by passing either '-p' or '--pixel-perfect' on the test command line. What's the use case for this ? > Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> > --- > scripts/vsp-lib.sh | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/scripts/vsp-lib.sh b/scripts/vsp-lib.sh > index cf15a045ea8c..9140254c7459 100755 > --- a/scripts/vsp-lib.sh > +++ b/scripts/vsp-lib.sh > @@ -287,7 +287,11 @@ compare_frames() { > params=${params//)/_} > params=$pipe-$in_fmt-$out_fmt-$size$params > > - if [ x$__vsp_pixel_perfect != xtrue ] ; then > + # The system can hint when pixel-perfection is not supported, > + # however the user can override to force this requirement with > + # VSP_PIXEL_PERFECT=1 in the environment or by passing -p, or > + # --pixel-perfect on the commandline. > + if [ x$__vsp_pixel_perfect != xtrue -a x$VSP_PIXEL_PERFECT != x1 ] ; then > method=fuzzy > fi > > @@ -1113,10 +1117,15 @@ case $1 in > export VSP_KEEP_FRAMES=1 > shift > ;; > + -p|--pixel-perfect) > + export VSP_PIXEL_PERFECT=1 > + shift > + ;; > -h|--help) > echo "$(basename $0): VSP Test library" > echo " -x|--debug enable shell debug" > echo " -k|--keep-frames keep generated and captured frames" > + echo " -p|--pixel-perfect frames must match with pixel perfection" > echo " -h|--help this help" > exit > shift
Hi Laurent, On 17/02/2019 11:59, Laurent Pinchart wrote: > Hi Kieran, > > Thank you for the patch. > > On Tue, Dec 04, 2018 at 03:51:44PM +0000, Kieran Bingham wrote: >> Provide a means for the tester to request pixel perfect matches on tests. >> This can be either through setting the environment variable VSP_PIXEL_PERFECT, or >> by passing either '-p' or '--pixel-perfect' on the test command line. > > What's the use case for this ? Some of the tests (i.e. scaling) provide 'fuzzy comparisons' to allow "a close match" to pass. It can be desirable to 'turn that off' so that you can generate a set of frames (using the '-k' option) to diff the expected against the actual frames. This helps while improving the system without making changes to the specific tests. It could be called 'disable fuzzy matching' if you like? > >> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> >> --- >> scripts/vsp-lib.sh | 11 ++++++++++- >> 1 file changed, 10 insertions(+), 1 deletion(-) >> >> diff --git a/scripts/vsp-lib.sh b/scripts/vsp-lib.sh >> index cf15a045ea8c..9140254c7459 100755 >> --- a/scripts/vsp-lib.sh >> +++ b/scripts/vsp-lib.sh >> @@ -287,7 +287,11 @@ compare_frames() { >> params=${params//)/_} >> params=$pipe-$in_fmt-$out_fmt-$size$params >> >> - if [ x$__vsp_pixel_perfect != xtrue ] ; then >> + # The system can hint when pixel-perfection is not supported, >> + # however the user can override to force this requirement with >> + # VSP_PIXEL_PERFECT=1 in the environment or by passing -p, or >> + # --pixel-perfect on the commandline. >> + if [ x$__vsp_pixel_perfect != xtrue -a x$VSP_PIXEL_PERFECT != x1 ] ; then >> method=fuzzy >> fi >> >> @@ -1113,10 +1117,15 @@ case $1 in >> export VSP_KEEP_FRAMES=1 >> shift >> ;; >> + -p|--pixel-perfect) >> + export VSP_PIXEL_PERFECT=1 >> + shift >> + ;; >> -h|--help) >> echo "$(basename $0): VSP Test library" >> echo " -x|--debug enable shell debug" >> echo " -k|--keep-frames keep generated and captured frames" >> + echo " -p|--pixel-perfect frames must match with pixel perfection" >> echo " -h|--help this help" >> exit >> shift >
Hi Kieran, On Sun, Feb 17, 2019 at 08:51:52PM +0000, Kieran Bingham wrote: > On 17/02/2019 11:59, Laurent Pinchart wrote: > > On Tue, Dec 04, 2018 at 03:51:44PM +0000, Kieran Bingham wrote: > >> Provide a means for the tester to request pixel perfect matches on tests. > >> This can be either through setting the environment variable VSP_PIXEL_PERFECT, or > >> by passing either '-p' or '--pixel-perfect' on the test command line. > > > > What's the use case for this ? > > Some of the tests (i.e. scaling) provide 'fuzzy comparisons' to allow "a > close match" to pass. > > It can be desirable to 'turn that off' so that you can generate a set of > frames (using the '-k' option) to diff the expected against the actual > frames. Ideally we should improve the reference frame generation tool to generate pixel-perfect frames :-) In the meantime I agree this could be useful, but I think we should then integrate this better by marking a test run as reference, saving frames to a good location, and then reusing them for the subsequent runs. It's a bit too manual in its proposed form for an automated test suite :-) > This helps while improving the system without making changes to the > specific tests. > > It could be called 'disable fuzzy matching' if you like? > > >> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> > >> --- > >> scripts/vsp-lib.sh | 11 ++++++++++- > >> 1 file changed, 10 insertions(+), 1 deletion(-) > >> > >> diff --git a/scripts/vsp-lib.sh b/scripts/vsp-lib.sh > >> index cf15a045ea8c..9140254c7459 100755 > >> --- a/scripts/vsp-lib.sh > >> +++ b/scripts/vsp-lib.sh > >> @@ -287,7 +287,11 @@ compare_frames() { > >> params=${params//)/_} > >> params=$pipe-$in_fmt-$out_fmt-$size$params > >> > >> - if [ x$__vsp_pixel_perfect != xtrue ] ; then > >> + # The system can hint when pixel-perfection is not supported, > >> + # however the user can override to force this requirement with > >> + # VSP_PIXEL_PERFECT=1 in the environment or by passing -p, or > >> + # --pixel-perfect on the commandline. > >> + if [ x$__vsp_pixel_perfect != xtrue -a x$VSP_PIXEL_PERFECT != x1 ] ; then > >> method=fuzzy > >> fi > >> > >> @@ -1113,10 +1117,15 @@ case $1 in > >> export VSP_KEEP_FRAMES=1 > >> shift > >> ;; > >> + -p|--pixel-perfect) > >> + export VSP_PIXEL_PERFECT=1 > >> + shift > >> + ;; > >> -h|--help) > >> echo "$(basename $0): VSP Test library" > >> echo " -x|--debug enable shell debug" > >> echo " -k|--keep-frames keep generated and captured frames" > >> + echo " -p|--pixel-perfect frames must match with pixel perfection" > >> echo " -h|--help this help" > >> exit > >> shift
diff --git a/scripts/vsp-lib.sh b/scripts/vsp-lib.sh index cf15a045ea8c..9140254c7459 100755 --- a/scripts/vsp-lib.sh +++ b/scripts/vsp-lib.sh @@ -287,7 +287,11 @@ compare_frames() { params=${params//)/_} params=$pipe-$in_fmt-$out_fmt-$size$params - if [ x$__vsp_pixel_perfect != xtrue ] ; then + # The system can hint when pixel-perfection is not supported, + # however the user can override to force this requirement with + # VSP_PIXEL_PERFECT=1 in the environment or by passing -p, or + # --pixel-perfect on the commandline. + if [ x$__vsp_pixel_perfect != xtrue -a x$VSP_PIXEL_PERFECT != x1 ] ; then method=fuzzy fi @@ -1113,10 +1117,15 @@ case $1 in export VSP_KEEP_FRAMES=1 shift ;; + -p|--pixel-perfect) + export VSP_PIXEL_PERFECT=1 + shift + ;; -h|--help) echo "$(basename $0): VSP Test library" echo " -x|--debug enable shell debug" echo " -k|--keep-frames keep generated and captured frames" + echo " -p|--pixel-perfect frames must match with pixel perfection" echo " -h|--help this help" exit shift
Provide a means for the tester to request pixel perfect matches on tests. This can be either through setting the environment variable VSP_PIXEL_PERFECT, or by passing either '-p' or '--pixel-perfect' on the test command line. Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> --- scripts/vsp-lib.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)