Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed documentation about RGB deviation #195 #196

Merged
merged 4 commits into from
Jan 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions docs/src/docs/common-attributes.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
// Platform badge with link
// image:https://img.shields.io/badge/-PLATFORM-blue[link="index.html#_opensource_and_platform"
:platform_badge: image:PLATFORM-blue.svg[link="index.html#_opensource_and_platform"]

:default_resolution: 1920x1200 (16:10)
46 changes: 17 additions & 29 deletions docs/src/docs/modules/layout-check.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,21 +15,17 @@ At this point an automatic comparison of reference screenshots can help. This te
== Description

A layout test with the Testerra utilities is actually a comparison between a reference and the actual state. This is done via the screenshot functionality of Selenium, in which both screenshots (reference and actual) are compared pixel by pixel from top to bottom. In this comparison, a third image is created in which differences (ReferencePixel != ActualPixel) are marked in red.
The LayoutCheck can perform a check in two ways:

- *PIXEL*: Pixel-based image comparison of the entire reference image without exception. You get a quantified value for pixel displacement in percent (0-100%)
- *ANNOTATED*: The image comparison is based on annotations in the reference image. The annotations are rectangles in a certain color, which include the areas to be checked. You get a quantified value for displaced annotations in percent.

=== Prerequisites
The following prerequisites must be met

. Concrete execution environment: Layout tests should run in a concrete browser environment to ensure the comparability of the screenshots.
** Size of browser window: Define fixed size to exclude different sizes of the images at different VM resolutions
** Screen resolution and scaling: Make sure you have the same screen resolution and scaling on each presentation device (ea. Testerra uses a resolution of {default_resolution} with a scaling of 1 per default in headless environments)
. Build Server: The library underlying ANNOTATED mode supports the operating systems
** Windows 32 - and 64 - Bit
** Linux Debian 32 - and 64 - Bit (glibc >= 2.15; Jenkins)
** MacOSX (untested)
** Screen resolution and scaling: Make sure you have the same screen resolution and scaling on each presentation device.
//. Build Server: The library underlying ANNOTATED mode supports the operating systems
//** Windows 32 - and 64 - Bit
//** Linux Debian 32 - and 64 - Bit (glibc >= 2.15; Jenkins)
//** MacOSX (untested)

== Configuration

Expand All @@ -49,39 +45,35 @@ In order to get the layout check to work, you need at least a reference image an
----
The directory for reference image may result in `src/test/resources/layout-references/chrome/WelcomePage.png` for example.

== PIXEL Mode

The comparison is generally carried out over the entire reference image. In this case, it is a prerequisite that the reference screenshot and the screenshot created during the test runtime have the same resolution (height and width are identical).

Mostly the website consists of many more elements than should be checked. Ingredients such as the header,
menus or footer may falsify the test result if there are deviations from the reference,
but these are not currently the subject of the test. For this reason,
the `{layoutcheck_ignore_color}` property can be used to determine that a particular color in the image can be excluded from comparison.
The property `{layoutcheck_pixel_rgb_deviation_percent}` controls, which color difference in percentage of a single pixel is count as a match. You can use that value to handle slightly changes in color or alpha values.

To validate the layout in Pixel Mode, you can check the whole page or a single <<GuiElements,GuiElement>>.
To validate the layout, you can check the whole page or a single <<GuiElements,GuiElement>>.

=== Check the whole page
== Check the whole page

[source, java]
----
import eu.tsystems.mms.tic.testframework.layout.LayoutCheck;
LayoutCheck.assertScreenshot(WebDriverManager.getWebDriver(), "WelcomePage", 1);
// 1% of the pixels are allowed to be different
float pixelDistanceToleranceThresholdPercent = 1.0;

LayoutCheck.assertScreenshot(webDriver, "WelcomePage", pixelDistanceToleranceThresholdPercent);
----

=== Check a single GuiElement
== Check a single GuiElement

To check the layout of a single GuiElement only, you can use the standard asserts implementation.

[source, java]
----
import eu.tsystems.mms.tic.testframework.pageobjects.GuiElement;
// 10% of the pixels are allowed to be different
float pixelDistanceToleranceThresholdPercent = 10.0;

final String guiElementImage = "HeaderComponent";
final GuiElement guiElement;
guiElement.asserts().assertScreenshot(guiElementImage, 10);
guiElement.asserts().assertScreenshot("HeaderComponent", pixelDistanceToleranceThresholdPercent);
----

=== Take reference screenshots on first run
== Take reference screenshots on first run

When you have configured the reference screenshots location and implemented the tests, you can now set the option

Expand All @@ -92,7 +84,3 @@ When you have configured the reference screenshots location and implemented the
to enable taking automatically screenshots based on the current browser and size configuration and storing them to the reference image's location.

WARNING: All concrete distance values in this {layoutcheck_takereference}-mode will return `0` (zero) and always pass the tests.

== ANNOTATED Mode

For this mode, an arbitrary number of rectangles with a line width of one pixel are defined in the reference images. The color of the rectangles is to be communicated to Testerra via the upper left pixel (x=1, y=1). In the current screenshot, Testerra searches for the area within the annotated selection. The resolution of the current screenshot is irrelevant.
4 changes: 2 additions & 2 deletions docs/src/docs/properties/layout-check-probs.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ include::property-attributes.adoc[]
| {layoutcheck_distance_template} | `Distance%s.png` | Filename scheme for saving distance images. The value must contain a '%s' which is replaced by the specified target file name during test execution.
| {layoutcheck_distance_path} | `src/test/resources/screenreferences/distance` | Directory path under which the calculated distance images are stored.
| {layoutcheck_actual_path} | `src/test/resources/screenreferences/actual` | Directory path under which the current screenshots for the comparison are saved
| {layoutcheck_pixel_rgb_deviation_percent} | 0 | Max allowed difference in rgb values between actual and reference image in percentage. If R, G, and B percentages are higher than {layoutcheck_pixel_rgb_deviation_percent} the corresponding pixel is marked as false (=red color in distance image)
| {layoutcheck_pixel_rgb_deviation_percent} | 0.95d | Defines at which score a region is considered a match. Should be as high as possible and as low as needed.
| {layoutcheck_pixel_rgb_deviation_percent} | 0 | Max allowed difference in rgb values between actual and reference image in percentage. If R, G, and B percentages are higher than the given value, the corresponding pixel is marked as false (=red color in distance image)
//| {layoutcheck_match_threshold} | 0.95d | Defines how many percent of pixels are considered a match. Should be as high as possible and as low as needed.
| {layoutcheck_displacement_threshold} | 5 | Displacement distance of matches that is considered as error (when distance > {layoutcheck_displacement_threshold})
| {layoutcheck_intra_grouping_threshold} | 5 | Threshold for grouping movement errors (errors <= {layoutcheck_intra_grouping_threshold} are grouped)
| {layoutcheck_min_match_distance} | 5 | Max distance for grouping multiple matches as single match (distance of matches < {layoutcheck_min_match_distance} are marked as single match)
Expand Down
2 changes: 1 addition & 1 deletion docs/src/docs/properties/property-attributes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
:layoutcheck_actual_path: tt.layoutcheck.actual.path
:layoutcheck_pixel_rgb_deviation_percent: tt.layout.pixel.rgb.deviation.percent

:layoutcheck_pixel_rgb_deviation_percent: tt.layoutcheck.match.threshold
:layoutcheck_match_threshold: tt.layoutcheck.match.threshold
:layoutcheck_displacement_threshold: tt.layoutcheck.displacement.threshold
:layoutcheck_intra_grouping_threshold: tt.layoutcheck.intra.grouping.threshold
:layoutcheck_min_match_distance: tt.layoutcheck.min.match.distance
Expand Down