From bb403086e6e4127e4ea81a48583aba838d9fc3f0 Mon Sep 17 00:00:00 2001 From: Mike Reiche Date: Mon, 17 Jan 2022 08:40:09 +0100 Subject: [PATCH 1/4] Fixed documentation about RGB deviation #195 --- docs/src/docs/modules/layout-check.adoc | 26 ++++++------------- .../docs/properties/layout-check-probs.adoc | 4 +-- .../docs/properties/property-attributes.adoc | 2 +- 3 files changed, 11 insertions(+), 21 deletions(-) diff --git a/docs/src/docs/modules/layout-check.adoc b/docs/src/docs/modules/layout-check.adoc index bd1cd047f..b59718e42 100644 --- a/docs/src/docs/modules/layout-check.adoc +++ b/docs/src/docs/modules/layout-check.adoc @@ -15,10 +15,6 @@ At this point an automatic comparison of reference screenshots can help. This te == Description A layout test with the Testerra utilities is actually a comparison between a reference and the actual state. This is done via the screenshot functionality of Selenium, in which both screenshots (reference and actual) are compared pixel by pixel from top to bottom. In this comparison, a third image is created in which differences (ReferencePixel != ActualPixel) are marked in red. -The LayoutCheck can perform a check in two ways: - -- *PIXEL*: Pixel-based image comparison of the entire reference image without exception. You get a quantified value for pixel displacement in percent (0-100%) -- *ANNOTATED*: The image comparison is based on annotations in the reference image. The annotations are rectangles in a certain color, which include the areas to be checked. You get a quantified value for displaced annotations in percent. === Prerequisites The following prerequisites must be met @@ -49,18 +45,16 @@ In order to get the layout check to work, you need at least a reference image an ---- The directory for reference image may result in `src/test/resources/layout-references/chrome/WelcomePage.png` for example. -== PIXEL Mode - The comparison is generally carried out over the entire reference image. In this case, it is a prerequisite that the reference screenshot and the screenshot created during the test runtime have the same resolution (height and width are identical). -Mostly the website consists of many more elements than should be checked. Ingredients such as the header, -menus or footer may falsify the test result if there are deviations from the reference, -but these are not currently the subject of the test. For this reason, -the `{layoutcheck_ignore_color}` property can be used to determine that a particular color in the image can be excluded from comparison. +To control the threshold, you can use the following properties. + +* `{layoutcheck_pixel_rgb_deviation_percent}` defines, which color difference in percentage of a single pixel is count as a match. You can use that value to handle slightly changes in color or alpha values. +* `{layoutcheck_match_threshold}` is the overall percentage amount of pixels that need to match, to pass the layout check based test. -To validate the layout in Pixel Mode, you can check the whole page or a single <>. +To validate the layout, you can check the whole page or a single <>. -=== Check the whole page +== Check the whole page [source, java] ---- @@ -68,7 +62,7 @@ import eu.tsystems.mms.tic.testframework.layout.LayoutCheck; LayoutCheck.assertScreenshot(WebDriverManager.getWebDriver(), "WelcomePage", 1); ---- -=== Check a single GuiElement +== Check a single GuiElement To check the layout of a single GuiElement only, you can use the standard asserts implementation. @@ -81,7 +75,7 @@ final GuiElement guiElement; guiElement.asserts().assertScreenshot(guiElementImage, 10); ---- -=== Take reference screenshots on first run +== Take reference screenshots on first run When you have configured the reference screenshots location and implemented the tests, you can now set the option @@ -92,7 +86,3 @@ When you have configured the reference screenshots location and implemented the to enable taking automatically screenshots based on the current browser and size configuration and storing them to the reference image's location. WARNING: All concrete distance values in this {layoutcheck_takereference}-mode will return `0` (zero) and always pass the tests. - -== ANNOTATED Mode - -For this mode, an arbitrary number of rectangles with a line width of one pixel are defined in the reference images. The color of the rectangles is to be communicated to Testerra via the upper left pixel (x=1, y=1). In the current screenshot, Testerra searches for the area within the annotated selection. The resolution of the current screenshot is irrelevant. diff --git a/docs/src/docs/properties/layout-check-probs.adoc b/docs/src/docs/properties/layout-check-probs.adoc index 90a275c25..ffb0216bd 100644 --- a/docs/src/docs/properties/layout-check-probs.adoc +++ b/docs/src/docs/properties/layout-check-probs.adoc @@ -14,8 +14,8 @@ include::property-attributes.adoc[] | {layoutcheck_distance_template} | `Distance%s.png` | Filename scheme for saving distance images. The value must contain a '%s' which is replaced by the specified target file name during test execution. | {layoutcheck_distance_path} | `src/test/resources/screenreferences/distance` | Directory path under which the calculated distance images are stored. | {layoutcheck_actual_path} | `src/test/resources/screenreferences/actual` | Directory path under which the current screenshots for the comparison are saved -| {layoutcheck_pixel_rgb_deviation_percent} | 0 | Max allowed difference in rgb values between actual and reference image in percentage. If R, G, and B percentages are higher than {layoutcheck_pixel_rgb_deviation_percent} the corresponding pixel is marked as false (=red color in distance image) -| {layoutcheck_pixel_rgb_deviation_percent} | 0.95d | Defines at which score a region is considered a match. Should be as high as possible and as low as needed. +| {layoutcheck_pixel_rgb_deviation_percent} | 0 | Max allowed difference in rgb values between actual and reference image in percentage. If R, G, and B percentages are higher than the given value, the corresponding pixel is marked as false (=red color in distance image) +| {layoutcheck_match_threshold} | 0.95d | Defines how many percent of pixels are considered a match. Should be as high as possible and as low as needed. | {layoutcheck_displacement_threshold} | 5 | Displacement distance of matches that is considered as error (when distance > {layoutcheck_displacement_threshold}) | {layoutcheck_intra_grouping_threshold} | 5 | Threshold for grouping movement errors (errors <= {layoutcheck_intra_grouping_threshold} are grouped) | {layoutcheck_min_match_distance} | 5 | Max distance for grouping multiple matches as single match (distance of matches < {layoutcheck_min_match_distance} are marked as single match) diff --git a/docs/src/docs/properties/property-attributes.adoc b/docs/src/docs/properties/property-attributes.adoc index 51e8a812b..e3c5c7a2d 100644 --- a/docs/src/docs/properties/property-attributes.adoc +++ b/docs/src/docs/properties/property-attributes.adoc @@ -81,7 +81,7 @@ :layoutcheck_actual_path: tt.layoutcheck.actual.path :layoutcheck_pixel_rgb_deviation_percent: tt.layout.pixel.rgb.deviation.percent -:layoutcheck_pixel_rgb_deviation_percent: tt.layoutcheck.match.threshold +:layoutcheck_match_threshold: tt.layoutcheck.match.threshold :layoutcheck_displacement_threshold: tt.layoutcheck.displacement.threshold :layoutcheck_intra_grouping_threshold: tt.layoutcheck.intra.grouping.threshold :layoutcheck_min_match_distance: tt.layoutcheck.min.match.distance From f25d75f9dad020e2b1a3657d97da7cf03ad76938 Mon Sep 17 00:00:00 2001 From: Mike Reiche Date: Thu, 20 Jan 2022 08:49:42 +0100 Subject: [PATCH 2/4] Updated documentation to make it clear --- docs/src/docs/modules/layout-check.adoc | 19 +++++++++---------- .../docs/properties/layout-check-probs.adoc | 2 +- 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/docs/src/docs/modules/layout-check.adoc b/docs/src/docs/modules/layout-check.adoc index b59718e42..b3ca49b26 100644 --- a/docs/src/docs/modules/layout-check.adoc +++ b/docs/src/docs/modules/layout-check.adoc @@ -47,10 +47,7 @@ The directory for reference image may result in `src/test/resources/layout-refer The comparison is generally carried out over the entire reference image. In this case, it is a prerequisite that the reference screenshot and the screenshot created during the test runtime have the same resolution (height and width are identical). -To control the threshold, you can use the following properties. - -* `{layoutcheck_pixel_rgb_deviation_percent}` defines, which color difference in percentage of a single pixel is count as a match. You can use that value to handle slightly changes in color or alpha values. -* `{layoutcheck_match_threshold}` is the overall percentage amount of pixels that need to match, to pass the layout check based test. +The property `{layoutcheck_pixel_rgb_deviation_percent}` controls, which color difference in percentage of a single pixel is count as a match. You can use that value to handle slightly changes in color or alpha values. To validate the layout, you can check the whole page or a single <>. @@ -58,8 +55,10 @@ To validate the layout, you can check the whole page or a single < {layoutcheck_displacement_threshold}) | {layoutcheck_intra_grouping_threshold} | 5 | Threshold for grouping movement errors (errors <= {layoutcheck_intra_grouping_threshold} are grouped) | {layoutcheck_min_match_distance} | 5 | Max distance for grouping multiple matches as single match (distance of matches < {layoutcheck_min_match_distance} are marked as single match) From abb8492c23938893383c7243e6ff5f1b53c35dce Mon Sep 17 00:00:00 2001 From: Mike Reiche Date: Thu, 20 Jan 2022 08:52:47 +0100 Subject: [PATCH 3/4] Removed useless default resolution and ANNOTATED info. --- docs/src/docs/common-attributes.adoc | 2 -- docs/src/docs/modules/layout-check.adoc | 10 +++++----- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/src/docs/common-attributes.adoc b/docs/src/docs/common-attributes.adoc index 68c988b44..e92620a65 100644 --- a/docs/src/docs/common-attributes.adoc +++ b/docs/src/docs/common-attributes.adoc @@ -1,5 +1,3 @@ // Platform badge with link // image:https://img.shields.io/badge/-PLATFORM-blue[link="index.html#_opensource_and_platform" :platform_badge: image:PLATFORM-blue.svg[link="index.html#_opensource_and_platform"] - -:default_resolution: 1920x1200 (16:10) diff --git a/docs/src/docs/modules/layout-check.adoc b/docs/src/docs/modules/layout-check.adoc index b3ca49b26..2cf150577 100644 --- a/docs/src/docs/modules/layout-check.adoc +++ b/docs/src/docs/modules/layout-check.adoc @@ -21,11 +21,11 @@ The following prerequisites must be met . Concrete execution environment: Layout tests should run in a concrete browser environment to ensure the comparability of the screenshots. ** Size of browser window: Define fixed size to exclude different sizes of the images at different VM resolutions -** Screen resolution and scaling: Make sure you have the same screen resolution and scaling on each presentation device (ea. Testerra uses a resolution of {default_resolution} with a scaling of 1 per default in headless environments) -. Build Server: The library underlying ANNOTATED mode supports the operating systems -** Windows 32 - and 64 - Bit -** Linux Debian 32 - and 64 - Bit (glibc >= 2.15; Jenkins) -** MacOSX (untested) +** Screen resolution and scaling: Make sure you have the same screen resolution and scaling on each presentation device. +//. Build Server: The library underlying ANNOTATED mode supports the operating systems +//** Windows 32 - and 64 - Bit +//** Linux Debian 32 - and 64 - Bit (glibc >= 2.15; Jenkins) +//** MacOSX (untested) == Configuration From 4347a659ab3e52587a85ef964f0d10ff08354a9e Mon Sep 17 00:00:00 2001 From: Mike Reiche Date: Thu, 20 Jan 2022 08:54:17 +0100 Subject: [PATCH 4/4] Consistency in examples --- docs/src/docs/modules/layout-check.adoc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/src/docs/modules/layout-check.adoc b/docs/src/docs/modules/layout-check.adoc index 2cf150577..8085da19c 100644 --- a/docs/src/docs/modules/layout-check.adoc +++ b/docs/src/docs/modules/layout-check.adoc @@ -69,9 +69,8 @@ To check the layout of a single GuiElement only, you can use the standard assert ---- // 10% of the pixels are allowed to be different float pixelDistanceToleranceThresholdPercent = 10.0; -String guiElementImage = "HeaderComponent"; -guiElement.asserts().assertScreenshot(guiElementImage, pixelDistanceToleranceThresholdPercent); +guiElement.asserts().assertScreenshot("HeaderComponent", pixelDistanceToleranceThresholdPercent); ---- == Take reference screenshots on first run