Generate test images at different output scales. #14545

brendandahl · 2022-02-07T22:42:06Z

This will default to generating test images at the device pixel
ratio of the machine the tests are created on unless the
test explicitly defines and output scale using the
outputScale setting. This makes the test look visually
like they would on the machine they are running on. It
also allows us to test different output scales.

brendandahl · 2022-02-07T22:43:41Z

As for performance locally,
DPR of 1 takes ~500s
DPR of 2 takes 680s

Snuffleupagus · 2022-02-08T09:42:09Z

test/driver.js

-            this.canvas.height = viewport.height;
+            // Restrict the test from creating a canvas that is too big.
+            const MAX_CANVAS_PIXEL_DIMENSION = 4096;
+            const largetDimension = Math.max(viewport.width, viewport.height);


Spelling-nit: largetDimension -> largestDimension.

Snuffleupagus · 2022-02-08T09:48:18Z

test/driver.js

+              MAX_CANVAS_PIXEL_DIMENSION
+            ) {
+              const rescale = MAX_CANVAS_PIXEL_DIMENSION / largetDimension;
+              viewport = page.getViewport({


While it doesn't matter in this particular case, please note that you generally want to use viewport.clone({ ... }) instead to ensure that only the scale changes (while any other possible parameters stay the same).

Snuffleupagus · 2022-02-08T09:53:24Z

test/test.js

@@ -970,7 +974,7 @@ async function startBrowser(browserName, startUrl = "") {
 }

 function startBrowsers(initSessionCallback, makeStartUrl = null) {
-  const browserNames = options.noChrome ? ["firefox"] : ["firefox", "chrome"];


This looks like a left-over from debugging, please revert this.

brendandahl · 2022-02-08T22:47:54Z

/botio test

pdfjsbot · 2022-02-08T22:47:55Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.241.84.105:8877/c95d58933ce0345/output.txt

pdfjsbot · 2022-02-08T22:47:55Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.193.163.58:8877/029a5f9097493fc/output.txt

pdfjsbot · 2022-02-08T23:11:31Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/c95d58933ce0345/output.txt

Total script time: 23.58 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: Passed
Regression tests: FAILED

  different ref/snapshot: 14

Image differences available at: http://54.241.84.105:8877/c95d58933ce0345/reftest-analyzer.html#web=eq.log

pdfjsbot · 2022-02-08T23:13:24Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/029a5f9097493fc/output.txt

Total script time: 25.46 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: Passed
Regression tests: FAILED

  different ref/snapshot: 4

Image differences available at: http://54.193.163.58:8877/029a5f9097493fc/reftest-analyzer.html#web=eq.log

brendandahl · 2022-02-08T23:44:43Z

Should note I expect a few changes because we will now be drawing the PDF's at a smaller size.

Also, we may want to add a scale option the reftest analyzer. Looking at the results from a lowdpi device on a highdpi device makes the images pretty small.

timvandermeij · 2022-02-09T18:31:56Z

How does this relate to #14534? Does it supersede that one?

Snuffleupagus · 2022-02-09T18:40:13Z

How does this relate to #14534? Does it supersede that one?

Unless I'm misunderstanding something I don't believe that it does, since this PR is essentially just simulating differing window.devicePixelRatio values (i.e. normal vs hiDPI screens).
The other PR is basically simulating different zoom levels (in the viewer), hence the two PRs ought to implement complimentary functionality.

brendandahl · 2022-02-22T23:14:44Z

@Snuffleupagus anything else you want me to address?

Snuffleupagus · 2022-02-23T08:56:15Z

/botio browsertest

pdfjsbot · 2022-02-23T08:56:16Z

From: Bot.io (Windows)

Received

Command cmd_browsertest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/e56abd3efa174a5/output.txt

pdfjsbot · 2022-02-23T08:56:16Z

From: Bot.io (Linux m4)

Received

Command cmd_browsertest from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.241.84.105:8877/d890aedfb968fc6/output.txt

pdfjsbot · 2022-02-23T09:15:33Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/e56abd3efa174a5/output.txt

Total script time: 19.28 mins

Regression tests: FAILED

  different ref/snapshot: 4

Image differences available at: http://54.193.163.58:8877/e56abd3efa174a5/reftest-analyzer.html#web=eq.log

pdfjsbot · 2022-02-23T09:16:19Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/d890aedfb968fc6/output.txt

Total script time: 20.04 mins

Regression tests: FAILED

  different ref/snapshot: 17

Image differences available at: http://54.241.84.105:8877/d890aedfb968fc6/reftest-analyzer.html#web=eq.log

Snuffleupagus

r=me, with the one comment addressed; thank you!

Snuffleupagus · 2022-02-23T09:26:20Z

test/resources/reftest-analyzer.js

@@ -228,10 +228,15 @@ window.onload = function () {
        });
        continue;
      }
-      match = line.match(/^ {2}IMAGE[^:]*: (.*)$/);
+      match = line.match(/^ {2}IMAGE[^:]*\((\d+)x(\d+)x(\d)+\): (.*)$/);


Is there a typo in the last digit group, i.e. did you mean the following instead?

Suggested change

match = line.match(/^ {2}IMAGE[^:]*$(\d+)x(\d+)x(\d)+$: (.*)$/);

match = line.match(/^ {2}IMAGE[^:]*$(\d+)x(\d+)x(\d+)$: (.*)$/);

This will default to generating test images at the device pixel ratio of the machine the tests are created on unless the test explicitly defines and output scale using the `outputScale` setting. This makes the test look visually like they would on the machine they are running on. It also allows us to test different output scales.

brendandahl · 2022-02-24T19:28:01Z

/botio test

pdfjsbot · 2022-02-24T19:28:03Z

From: Bot.io (Linux m4)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.241.84.105:8877/43574260aa2b895/output.txt

pdfjsbot · 2022-02-24T19:28:03Z

From: Bot.io (Windows)

Received

Command cmd_test from @brendandahl received. Current queue size: 0

Live output at: http://54.193.163.58:8877/bdebea6e9049d99/output.txt

pdfjsbot · 2022-02-24T19:51:57Z

From: Bot.io (Linux m4)

Failed

Full output at http://54.241.84.105:8877/43574260aa2b895/output.txt

Total script time: 23.89 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 18
  different first/second rendering: 1

Image differences available at: http://54.241.84.105:8877/43574260aa2b895/reftest-analyzer.html#web=eq.log

pdfjsbot · 2022-02-24T19:52:42Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/bdebea6e9049d99/output.txt

Total script time: 24.64 mins

Font tests: Passed
Unit tests: Passed
Integration Tests: FAILED
Regression tests: FAILED

  different ref/snapshot: 4

Image differences available at: http://54.193.163.58:8877/bdebea6e9049d99/reftest-analyzer.html#web=eq.log

Snuffleupagus · 2022-02-24T20:57:39Z

/botio makeref

pdfjsbot · 2022-02-24T20:57:40Z

From: Bot.io (Linux m4)

Received

Command cmd_makeref from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.241.84.105:8877/d4b53f2147810c2/output.txt

pdfjsbot · 2022-02-24T20:57:40Z

From: Bot.io (Windows)

Received

Command cmd_makeref from @Snuffleupagus received. Current queue size: 1

Live output at: http://54.193.163.58:8877/a4ab951443760d3/output.txt

pdfjsbot · 2022-02-24T21:18:11Z

From: Bot.io (Linux m4)

Success

Full output at http://54.241.84.105:8877/d4b53f2147810c2/output.txt

Total script time: 20.50 mins

Lint: Passed
Make references: Passed
Check references: Passed

pdfjsbot · 2022-02-24T22:00:41Z

From: Bot.io (Windows)

Failed

Full output at http://54.193.163.58:8877/a4ab951443760d3/output.txt

Total script time: 60.00 mins

Lint: Passed

Snuffleupagus · 2022-02-24T22:06:59Z

/botio-windows makeref

pdfjsbot · 2022-02-24T22:07:01Z

From: Bot.io (Windows)

Received

Command cmd_makeref from @Snuffleupagus received. Current queue size: 0

Live output at: http://54.193.163.58:8877/f5f6af7ce9c601e/output.txt

pdfjsbot · 2022-02-24T22:27:10Z

From: Bot.io (Windows)

Success

Full output at http://54.193.163.58:8877/f5f6af7ce9c601e/output.txt

Total script time: 20.15 mins

Lint: Passed
Make references: Passed
Check references: Passed

Snuffleupagus reviewed Feb 8, 2022

View reviewed changes

Snuffleupagus added the test label Feb 8, 2022

brendandahl force-pushed the output-scale branch from ea40f0e to 2fcdc91 Compare February 8, 2022 22:47

Snuffleupagus approved these changes Feb 23, 2022

View reviewed changes

brendandahl force-pushed the output-scale branch from 2fcdc91 to f5c3abb Compare February 24, 2022 19:27

Snuffleupagus merged commit 889b761 into mozilla:master Feb 24, 2022

quaoaris mentioned this pull request Feb 25, 2022

Implement reference test support for rendering at different zoom levels (issue 14505) #14534

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate test images at different output scales. #14545

Generate test images at different output scales. #14545

brendandahl commented Feb 7, 2022

brendandahl commented Feb 7, 2022

Snuffleupagus Feb 8, 2022

Snuffleupagus Feb 8, 2022 •

edited

Loading

Snuffleupagus Feb 8, 2022

brendandahl commented Feb 8, 2022

pdfjsbot commented Feb 8, 2022

pdfjsbot commented Feb 8, 2022

pdfjsbot commented Feb 8, 2022

pdfjsbot commented Feb 8, 2022

brendandahl commented Feb 8, 2022

timvandermeij commented Feb 9, 2022

Snuffleupagus commented Feb 9, 2022

brendandahl commented Feb 22, 2022

Snuffleupagus commented Feb 23, 2022

pdfjsbot commented Feb 23, 2022

pdfjsbot commented Feb 23, 2022

pdfjsbot commented Feb 23, 2022

pdfjsbot commented Feb 23, 2022

Snuffleupagus left a comment

Snuffleupagus Feb 23, 2022 •

edited

Loading

brendandahl commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

Snuffleupagus commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

Snuffleupagus commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

	match = line.match(/^ {2}IMAGE[^:]\((\d+)x(\d+)x(\d)+\): (.)$/);
	match = line.match(/^ {2}IMAGE[^:]\((\d+)x(\d+)x(\d+)\): (.)$/);

Generate test images at different output scales. #14545

Generate test images at different output scales. #14545

Conversation

brendandahl commented Feb 7, 2022

brendandahl commented Feb 7, 2022

Snuffleupagus Feb 8, 2022

Choose a reason for hiding this comment

Snuffleupagus Feb 8, 2022 • edited Loading

Choose a reason for hiding this comment

Snuffleupagus Feb 8, 2022

Choose a reason for hiding this comment

brendandahl commented Feb 8, 2022

pdfjsbot commented Feb 8, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Feb 8, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Feb 8, 2022

From: Bot.io (Linux m4)

Failed

pdfjsbot commented Feb 8, 2022

From: Bot.io (Windows)

Failed

brendandahl commented Feb 8, 2022

timvandermeij commented Feb 9, 2022

Snuffleupagus commented Feb 9, 2022

brendandahl commented Feb 22, 2022

Snuffleupagus commented Feb 23, 2022

pdfjsbot commented Feb 23, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Feb 23, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Feb 23, 2022

From: Bot.io (Windows)

Failed

pdfjsbot commented Feb 23, 2022

From: Bot.io (Linux m4)

Failed

Snuffleupagus left a comment

Choose a reason for hiding this comment

Snuffleupagus Feb 23, 2022 • edited Loading

Choose a reason for hiding this comment

brendandahl commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Feb 24, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Feb 24, 2022

From: Bot.io (Linux m4)

Failed

pdfjsbot commented Feb 24, 2022

From: Bot.io (Windows)

Failed

Snuffleupagus commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

From: Bot.io (Linux m4)

Received

pdfjsbot commented Feb 24, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Feb 24, 2022

From: Bot.io (Linux m4)

Success

pdfjsbot commented Feb 24, 2022

From: Bot.io (Windows)

Failed

Snuffleupagus commented Feb 24, 2022

pdfjsbot commented Feb 24, 2022

From: Bot.io (Windows)

Received

pdfjsbot commented Feb 24, 2022

From: Bot.io (Windows)

Success

Snuffleupagus Feb 8, 2022 •

edited

Loading

Snuffleupagus Feb 23, 2022 •

edited

Loading