From patchwork Thu Jul 6 13:26:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Kocialkowki X-Patchwork-Id: 9828329 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F225660317 for ; Thu, 6 Jul 2017 13:26:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E23C12861F for ; Thu, 6 Jul 2017 13:26:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D691A2862B; Thu, 6 Jul 2017 13:26:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C145A2861F for ; Thu, 6 Jul 2017 13:26:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B0C096E5FD; Thu, 6 Jul 2017 13:26:21 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id C674F6E5FD for ; Thu, 6 Jul 2017 13:26:20 +0000 (UTC) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Jul 2017 06:26:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,317,1496127600"; d="scan'208";a="105142122" Received: from linux.intel.com ([10.54.29.200]) by orsmga004.jf.intel.com with ESMTP; 06 Jul 2017 06:26:20 -0700 Received: from workstation.fi.intel.com (workstation.fi.intel.com [10.237.68.144]) by linux.intel.com (Postfix) with ESMTP id ADD6E580585; Thu, 6 Jul 2017 06:26:18 -0700 (PDT) From: Paul Kocialkowski To: intel-gfx@lists.freedesktop.org Date: Thu, 6 Jul 2017 16:26:14 +0300 Message-Id: <20170706132614.4163-2-paul.kocialkowski@linux.intel.com> X-Mailer: git-send-email 2.13.2 In-Reply-To: <20170706132614.4163-1-paul.kocialkowski@linux.intel.com> References: <20170706132614.4163-1-paul.kocialkowski@linux.intel.com> Cc: Lyude Subject: [Intel-gfx] [PATCH i-g-t] chamelium: Add support for VGA frame comparison testing X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP This adds support for VGA frame comparison check. Since VGA uses a DAC-ADC chain, its data cannot be expected to be pixel perfect. Thus, it is impossible to uses a CRC check and full frames have to be analyzed instead. Such an analysis is implemented, based on both an absolute error threshold and a correlation with the expected error trend for a DAC-ADC chain. It was tested with a couple encoders and provides reliable error detection with few false positives. In case the frame does not match, it is dumped to a file in a similar fashion as CRC tests. Signed-off-by: Paul Kocialkowski --- configure.ac | 5 +- lib/Makefile.am | 2 + lib/igt_chamelium.c | 235 +++++++++++++++++++++++++++++++++++++++++++++++++++- lib/igt_chamelium.h | 6 +- tests/chamelium.c | 82 ++++++++++++++++++ 5 files changed, 325 insertions(+), 5 deletions(-) diff --git a/configure.ac b/configure.ac index 0418e605..e7c3603b 100644 --- a/configure.ac +++ b/configure.ac @@ -182,8 +182,9 @@ PKG_CHECK_MODULES(GLIB, glib-2.0) # for chamelium PKG_CHECK_MODULES(XMLRPC, xmlrpc_client, [xmlrpc=yes], [xmlrpc=no]) PKG_CHECK_MODULES(PIXMAN, pixman-1, [pixman=yes], [pixman=no]) -AM_CONDITIONAL(HAVE_CHAMELIUM, [test "x$xmlrpc$pixman" = xyesyes]) -if test x"$xmlrpc$pixman" = xyesyes; then +PKG_CHECK_MODULES(GSL, gsl, [gsl=yes], [gsl=no]) +AM_CONDITIONAL(HAVE_CHAMELIUM, [test "x$xmlrpc$pixman$gsl" = xyesyesyes]) +if test x"$xmlrpc$pixman$gsl" = xyesyesyes; then AC_DEFINE(HAVE_CHAMELIUM, 1, [Enable Chamelium support]) fi diff --git a/lib/Makefile.am b/lib/Makefile.am index d4f41128..fb922ced 100644 --- a/lib/Makefile.am +++ b/lib/Makefile.am @@ -35,6 +35,7 @@ AM_CFLAGS = \ $(DRM_CFLAGS) \ $(PCIACCESS_CFLAGS) \ $(LIBUNWIND_CFLAGS) \ + $(GSL_CFLAGS) \ $(KMOD_CFLAGS) \ $(PROCPS_CFLAGS) \ $(DEBUG_CFLAGS) \ @@ -54,6 +55,7 @@ libintel_tools_la_LIBADD = \ $(DRM_LIBS) \ $(PCIACCESS_LIBS) \ $(PROCPS_LIBS) \ + $(GSL_LIBS) \ $(KMOD_LIBS) \ $(CAIRO_LIBS) \ $(LIBUDEV_LIBS) \ diff --git a/lib/igt_chamelium.c b/lib/igt_chamelium.c index 9aca6842..79ea8b4f 100644 --- a/lib/igt_chamelium.c +++ b/lib/igt_chamelium.c @@ -34,6 +34,8 @@ #include #include #include +#include +#include #include "igt.h" @@ -1035,6 +1037,235 @@ void chamelium_write_frame_to_png(const struct chamelium *chamelium, } /** + * chamelium_analogue_frame_crop: + * @chamelium: The Chamelium instance to use + * @dump: The chamelium frame dump to crop + * @width: The cropped frame width + * @height: The cropped frame height + * + * Detects the corners of a chamelium frame and crops it to the requested + * width/height. This is useful for VGA frame dumps that also contain the + * pixels dumped during the blanking intervals. + * + * The detection is done on a brightness-threshold-basis, that is adapted + * to the reference frame used by i-g-t. It may not be as relevant for other + * frames. + */ +void chamelium_crop_analogue_frame(struct chamelium_frame_dump *dump, int width, + int height) +{ + unsigned char *bgr; + unsigned char *p; + unsigned char *q; + int top, left; + int x, y, xx, yy; + int score; + + if (dump->width == width && dump->height == height) + return; + + /* Start with the most bottom-right position. */ + top = dump->height - height; + left = dump->width - width; + + igt_assert(top >= 0 && left >= 0); + + igt_debug("Cropping analogue frame from %dx%d to %dx%d\n", dump->width, + dump->height, width, height); + + /* Detect the top-left corner of the frame. */ + for (x = 0; x < dump->width; x++) { + for (y = 0; y < dump->height; y++) { + p = &dump->bgr[(x + y * dump->width) * 3]; + + /* Detect significantly bright pixels. */ + if (p[0] < 50 && p[1] < 50 && p[2] < 50) + continue; + + /* + * Make sure close-by pixels are also significantly + * bright. + */ + score = 0; + for (xx = x; xx < x + 10; xx++) { + for (yy = y; yy < y + 10; yy++) { + p = &dump->bgr[(xx + yy * dump->width) * 3]; + + if (p[0] > 50 && p[1] > 50 && p[2] > 50) + score++; + } + } + + /* Not enough pixels are significantly bright. */ + if (score < 25) + continue; + + if (x < left) + left = x; + + if (y < top) + top = y; + + if (left == x || top == y) + continue; + } + } + + igt_debug("Detected analogue frame edges at %dx%d\n", left, top); + + /* Crop the frame given the detected top-left corner. */ + bgr = malloc(width * height * 3); + + for (y = 0; y < height; y++) { + p = &dump->bgr[(left + (top + y) * dump->width) * 3]; + q = &bgr[(y * width) * 3]; + memcpy(q, p, width * 3); + } + + free(dump->bgr); + dump->width = width; + dump->height = height; + dump->bgr = bgr; +} + +/** + * chamelium_check_analogue_frame_eq: + * @chamelium: The Chamelium instance to use + * @dump: The chamelium frame dump to check + * @fb: The framebuffer to check against + * + * Checks that the analogue image contained in the chamelium frame dump matches + * the given framebuffer. + * + * In order to determine whether the frame matches the reference, the following + * reasoning is implemented: + * 1. The absolute error for each color value of the reference is collected. + * 2. The average absolute error is calculated for each color value of the + * reference and must not go above 60 (23.5 % of the total range). + * 3. A linear fit for the average absolute error from the pixel value is + * calculated, as a DAC-ADC chain is expected to have a linear error curve. + * 4. The linear fit is correlated with the actual average absolute error for + * the frame and the correlation coefficient is checked to be > 0.985, + * indicating a match with the expected error trend. + * + * Most errors (e.g. due to scaling, rotation, color space, etc) can be + * reliably detected this way, with a minimized number of false-positives. + * However, the brightest values (250 and up) are ignored as the error trend + * is often not linear there in practice due to clamping. + * + * Returns: a boolean indicating whether the frames match + */ +bool chamelium_check_analogue_frame_match(const struct chamelium *chamelium, + const struct chamelium_frame_dump *dump, + struct igt_fb *fb) +{ + cairo_t *cr; + cairo_surface_t *fb_surface; + pixman_image_t *reference_src, *reference_bgr; + int w = dump->width, h = dump->height; + int error_count[3][256][2] = { 0 }; + double error_average[4][250]; + double error_trend[250]; + double c0, c1, cov00, cov01, cov11, sumsq; + double correlation; + unsigned char *bgr; + unsigned char *p; + unsigned char *q; + bool match = true; + int diff; + int x, y; + int i, j; + + /* Get the cairo surface for the framebuffer */ + cr = igt_get_cairo_ctx(chamelium->drm_fd, fb); + fb_surface = cairo_get_target(cr); + cairo_surface_reference(fb_surface); + cairo_destroy(cr); + + /* + * Convert the reference image into the same format as the chamelium + * image + */ + reference_src = pixman_image_create_bits( + PIXMAN_x8r8g8b8, w, h, + (void*)cairo_image_surface_get_data(fb_surface), + cairo_image_surface_get_stride(fb_surface)); + reference_bgr = convert_frame_format(reference_src, PIXMAN_b8g8r8); + pixman_image_unref(reference_src); + + bgr = (unsigned char *) pixman_image_get_data(reference_bgr); + + /* Collect the absolute error for each color value */ + for (x = 0; x < w; x++) { + for (y = 0; y < h; y++) { + p = &dump->bgr[(x + y * w) * 3]; + q = &bgr[(x + y * w) * 3]; + + for (i = 0; i < 3; i++) { + diff = (int) p[i] - q[i]; + if (diff < 0) + diff = -diff; + + error_count[i][q[i]][0] += diff; + error_count[i][q[i]][1]++; + } + } + } + + /* Calculate the average absolute error for each color value */ + for (i = 0; i < 250; i++) { + error_average[0][i] = i; + + for (j = 1; j < 4; j++) { + error_average[j][i] = (double) error_count[j-1][i][0] / + error_count[j-1][i][1]; + + if (error_average[j][i] > 60) { + igt_warn("Error average too high (%f)\n", + error_average[j][i]); + + match = false; + goto complete; + } + } + } + + /* + * Calculate error trend from linear fit. + * A DAC-ADC chain is expected to have a linear absolute error on + * most of its range + */ + for (i = 1; i < 4; i++) { + gsl_fit_linear((const double *) &error_average[0], 1, + (const double *) &error_average[i], 1, 250, + &c0, &c1, &cov00, &cov01, &cov11, &sumsq); + + for (j = 0; j < 250; j++) + error_trend[j] = c0 + j * c1; + + correlation = gsl_stats_correlation((const double *) &error_trend, + 1, + (const double *) &error_average[i], + 1, 250); + + if (correlation < 0.985) { + igt_warn("Error with reference not correlated (%f)\n", + correlation); + + match = false; + goto complete; + } + } + +complete: + + pixman_image_unref(reference_bgr); + cairo_surface_destroy(fb_surface); + + return match; +} + +/** * chamelium_get_frame_limit: * @chamelium: The Chamelium instance to use * @port: The port to check the frame limit on @@ -1482,7 +1713,7 @@ igt_constructor { /* Frame dumps can be large, so we need to be able to handle very large * responses * - * Limit here is 10MB + * Limit here is 15MB */ - xmlrpc_limit_set(XMLRPC_XML_SIZE_LIMIT_ID, 10485760); + xmlrpc_limit_set(XMLRPC_XML_SIZE_LIMIT_ID, 15728640); } diff --git a/lib/igt_chamelium.h b/lib/igt_chamelium.h index aa881971..df1ebd12 100644 --- a/lib/igt_chamelium.h +++ b/lib/igt_chamelium.h @@ -102,13 +102,17 @@ int chamelium_get_captured_frame_count(struct chamelium *chamelium); int chamelium_get_frame_limit(struct chamelium *chamelium, struct chamelium_port *port, int w, int h); - void chamelium_assert_frame_eq(const struct chamelium *chamelium, const struct chamelium_frame_dump *dump, struct igt_fb *fb); void chamelium_write_frame_to_png(const struct chamelium *chamelium, const struct chamelium_frame_dump *dump, const char *filename); +void chamelium_crop_analogue_frame(struct chamelium_frame_dump *dump, int width, + int height); +bool chamelium_check_analogue_frame_match(const struct chamelium *chamelium, + const struct chamelium_frame_dump *dump, + struct igt_fb *fb); void chamelium_destroy_frame_dump(struct chamelium_frame_dump *dump); #endif /* IGT_CHAMELIUM_H */ diff --git a/tests/chamelium.c b/tests/chamelium.c index 3d95c05c..5ccdee7c 100644 --- a/tests/chamelium.c +++ b/tests/chamelium.c @@ -360,6 +360,10 @@ enable_output(data_t *data, BROADCAST_RGB_FULL); igt_display_commit(display); + + if (chamelium_port_get_type(port) == DRM_MODE_CONNECTOR_VGA) + usleep(250000); + chamelium_port_wait_video_input_stable(data->chamelium, port, HOTPLUG_TIMEOUT); @@ -470,6 +474,81 @@ test_display_crc(data_t *data, struct chamelium_port *port, int count) } static void +test_analogue_frame_dump(data_t *data, struct chamelium_port *port) +{ + igt_display_t display; + igt_output_t *output; + igt_plane_t *primary; + struct igt_fb fb; + struct chamelium_frame_dump *frame; + drmModeModeInfo *mode; + drmModeConnector *connector; + int fb_id, i; + const char *connector_name; + char *frame_dump_path; + char path[PATH_MAX]; + bool eq; + + output = prepare_output(data, &display, port); + connector = chamelium_port_get_connector(data->chamelium, port, false); + primary = igt_output_get_plane_type(output, DRM_PLANE_TYPE_PRIMARY); + igt_assert(primary); + + connector_name = kmstest_connector_type_str(connector->connector_type); + frame_dump_path = chamelium_get_frame_dump_path(data->chamelium); + + for (i = 0; i < connector->count_modes; i++) { + mode = &connector->modes[i]; + + fb_id = igt_create_color_pattern_fb(data->drm_fd, + mode->hdisplay, mode->vdisplay, + DRM_FORMAT_XRGB8888, + LOCAL_DRM_FORMAT_MOD_NONE, + 0, 0, 0, &fb); + igt_assert(fb_id > 0); + + enable_output(data, port, output, mode, &fb); + + igt_debug("Reading frame dumps from Chamelium...\n"); + + frame = chamelium_port_dump_pixels(data->chamelium, port, 0, 0, + 0, 0); + + chamelium_crop_analogue_frame(frame, mode->hdisplay, + mode->vdisplay); + + eq = chamelium_check_analogue_frame_match(data->chamelium, + frame, &fb); + if (!eq && frame_dump_path) { + igt_debug("Dumping reference and chamelium frames to %s...\n", + frame_dump_path); + + snprintf(path, PATH_MAX, "%s/frame-reference-%s.png", + frame_dump_path, connector_name); + igt_write_fb_to_png(data->drm_fd, &fb, path); + + snprintf(path, PATH_MAX, "%s/frame-chamelium-%s.png", + frame_dump_path, connector_name); + chamelium_write_frame_to_png(data->chamelium, + frame, path); + + chamelium_destroy_frame_dump(frame); + } + + igt_fail_on_f(!eq, + "Chamelium analogue frame mismatch with reference\n"); + + chamelium_destroy_frame_dump(frame); + + disable_output(data, port, output); + igt_remove_fb(data->drm_fd, &fb); + } + + drmModeFreeConnector(connector); + igt_display_fini(&display); +} + +static void test_hpd_without_ddc(data_t *data, struct chamelium_port *port) { struct udev_monitor *mon = igt_watch_hotplug(); @@ -706,6 +785,9 @@ igt_main connector_subtest("vga-hpd-without-ddc", VGA) test_hpd_without_ddc(&data, port); + + connector_subtest("vga-frame-dump", VGA) + test_analogue_frame_dump(&data, port); } igt_subtest_group {