Skip to content

Commit

Permalink
added integrations tests for error thresholds
Browse files Browse the repository at this point in the history
  • Loading branch information
@plutasnyy
plutasnyy committed Aug 1, 2018
1 parent e073458 commit a0a97af
Show file tree
Hide file tree
Showing 2 changed files with 113 additions and 1 deletion.
2 changes: 1 addition & 1 deletion documentation/src/main/wiki/LayoutComparator.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Resource name: screen
| --------- | ----- | ----------- | --------- |
| `pixelThreshold` | int (bigger than 0) | The value to set the error threshold in pixels e.g if difference between photos is smaller or equal to `pixelThreshold`, the test will pass. In case of difference is bigger than `pixelThreshold`, the test will fail. | no |
| `percentageThreshold` | double (0 to 100) | It works as `pixelThreshold` but values are in percentages | no |
When you provide `pixelThreshold` and `percentageThreshold` test will pass only when pixel difference will be smaller or equal than `pixelThreshold` and percentage difference will be smaller or equal than `percentageThreshold`.
When you provide `pixelThreshold` and `percentageThreshold` test will pass only if pixel difference is smaller or equal than `pixelThreshold` and percentage difference is smaller or equal than `percentageThreshold`.

##### Example Usage

Expand Down
112 changes: 112 additions & 0 deletions integration-tests/test-suite/partials/layout.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -148,5 +148,117 @@
<url href="comparators/layout/success.jsp" />
</urls>
</test>

<test name="F-comparator-Layout-pixel-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen pixelThreshold="1" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="S-comparator-Layout-pixel-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen pixelThreshold="2500" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="F-comparator-Layout-percentage-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen percentageThreshold="0.001" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="S-comparator-Layout-percentage-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen percentageThreshold="0.5" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="F-comparator-Layout-small-pixel-threshold-big-percentage-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen pixelThreshold="1" percentageThreshold="0.5" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="F-comparator-Layout-big-pixel-threshold-small-percentage-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen pixelThreshold="2500" percentageThreshold="0.001" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="F-comparator-Layout-small-pixel-threshold-small-percentage-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen pixelThreshold="1000" percentageThreshold="0.001" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>

<test name="S-comparator-Layout-big-pixel-threshold-and-big-percentage-threshold">
<collect>
<open />
<sleep duration="3000" />
<screen />
</collect>
<compare>
<screen pixelThreshold="2500" percentageThreshold="0.5" comparator="layout" />
</compare>
<urls>
<url href="comparators/layout/failed.jsp" />
</urls>
</test>
<!-- Layout-Comparator END -->
</suite>

0 comments on commit a0a97af

Please sign in to comment.