Skip to content

Commit

Permalink
[rfw, ci] Regenerate goldens, manually roll flutter#4835 (flutter#4862)
Browse files Browse the repository at this point in the history
This PR updates the goldens for `package:rfw`.

This PR also:

* Modifies `run_tests.sh` so it forwards all its command-line parameters
to the inner `test_coverage.dart` bin. (This enables passing
`--update-goldens` without much fuss.)
* Updates the documentation on how to update goldens in the package.
  • Loading branch information
ditman authored Sep 7, 2023
1 parent 8e36deb commit 1da1175
Show file tree
Hide file tree
Showing 10 changed files with 80 additions and 30 deletions.
2 changes: 1 addition & 1 deletion .ci/flutter_master.version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
c175cf87a6a3d797fdc3f81cbd54419f639cd3d9
685ce14b2d0f4046bd596842208d0c17f9a21e57
3 changes: 3 additions & 0 deletions packages/rfw/.gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Ignore golden test failure files
test/failures/*.png

# Miscellaneous
*.class
*.log
Expand Down
5 changes: 5 additions & 0 deletions packages/rfw/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
## 1.0.15

* Updates README.md to point to the CONTRIBUTING.md file.
* Introduces CONTRIBUTING.md, and adds more information about golden testing.

## 1.0.14

* Adds pub topics to package metadata.
Expand Down
61 changes: 61 additions & 0 deletions packages/rfw/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Contributing

If you run into any problems, please file a [new
bug](https://github.com/flutter/flutter/issues/new?labels=p:%20rfw,package,P2).
Rather than waiting for a fix, we encourage you to consider submitting
a PR yourself. See our [contributing
guide](https://github.com/flutter/packages/blob/master/CONTRIBUTING.md)
for details.

Adding more widgets to `lib/flutter/core_widgets.dart` and
`lib/flutter/material_widgets.dart` is welcome.

When contributing code, ensure that `flutter test --coverage; lcov
--list coverage/lcov.info` continues to show 100% test coverage, and
update `test_coverage/bin/test_coverage.dart` with the appropriate
expectations to prevent future coverage regressions. (That program is
run by `run_tests.sh`.)

## Tests

To run the tests, use `flutter test`.

To run the web tests, use `flutter test --platform=chrome`. If there
is code that only runs on the web target, mark each such line by
ending it with the string `// dead code on VM target`. This will
exclude that line from the coverage calculation.

Golden tests are only run against the Flutter master channel and only
run on Linux, since minor rendering differences are expected on
different platforms and on different versions of Flutter.

## Golden test failures

When golden tests fail, several files will be created in a `test/failures`
directory.

The suffix of each file briefly describes its contents:

* `_masterImage`: The current golden against which comparisons are being made.
* `_testImage`: The image generated by this run of the test.
* `_isolatedDiff`: An image where only the pixels that differ between the
master and test images are rendered.
* `_maskedDiff`: An image where the pixels that differ between the master and
test images are rendered in black (isolatedDiff + maskedDiff = testImage).

Check the `_testImage` and the `Diff`s and see if the changes are valid. From
time to time, a new dithering or anti-aliasing algorithm will land in Flutter
that will invalidate a bunch of goldens, but not really break the rendering.

In this case, you may update the golden files.

### Updating golden files

Once you've validated that the new goldens are correct, you may update them by
calling `run_tests.sh --update-goldens`.

That command updates the reference golden images in the `test/goldens` directory.

Read more about Golden matching [in `package:flutter_test` API docs][package-test].

[package-test]: https://api.flutter.dev/flutter/flutter_test/matchesGoldenFile.html
27 changes: 1 addition & 26 deletions packages/rfw/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -965,29 +965,4 @@ concerns on iOS, anyway).

## Contributing

If you run into any problems, please file a [new
bug](https://github.com/flutter/flutter/issues/new?labels=p:%20rfw,package,P2).
Rather than waiting for a fix, we encourage you to consider submitting
a PR yourself. See our [contributing
guide](https://github.com/flutter/packages/blob/master/CONTRIBUTING.md)
for details.

Adding more widgets to `lib/flutter/core_widgets.dart` and
`lib/flutter/material_widgets.dart` is welcome.

When contributing code, ensure that `flutter test --coverage; lcov
--list coverage/lcov.info` continues to show 100% test coverage, and
update `test_coverage/bin/test_coverage.dart` with the appropriate
expectations to prevent future coverage regressions. (That program is
run by `run_tests.sh`.)

To run the tests, use `flutter test`.

To run the web tests, use `flutter test --platform=chrome`. If there
is code that only runs on the web target, mark each such line by
ending it with the string `// dead code on VM target`. This will
exclude that line from the coverage calculation.

Golden tests are only run against the Flutter master channel and only
run on Linux, since minor rendering differences are expected on
different platforms and on different versions of Flutter.
See [CONTRIBUTING.md](https://github.com/flutter/packages/blob/main/packages/rfw/CONTRIBUTING.md)
2 changes: 1 addition & 1 deletion packages/rfw/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: rfw
description: "Remote Flutter widgets: a library for rendering declarative widget description files at runtime."
repository: https://github.com/flutter/packages/tree/main/packages/rfw
issue_tracker: https://github.com/flutter/flutter/issues?q=is%3Aissue+is%3Aopen+label%3A%22p%3A+rfw%22
version: 1.0.14
version: 1.0.15

environment:
sdk: ">=3.0.0 <4.0.0"
Expand Down
3 changes: 2 additions & 1 deletion packages/rfw/run_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@

set -e
pushd test_coverage; dart pub get; popd
dart --enable-asserts test_coverage/bin/test_coverage.dart
set -x
dart --enable-asserts test_coverage/bin/test_coverage.dart "$@"
Binary file modified packages/rfw/test/goldens/argument_decoders_test.containers.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified packages/rfw/test/goldens/argument_decoders_test.text.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 6 additions & 1 deletion packages/rfw/test_coverage/bin/test_coverage.dart
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,13 @@ Future<void> main(List<String> arguments) async {

final ProcessResult result = Process.runSync(
'flutter',
<String>['test', '--coverage'],
<String>[
'test',
'--coverage',
if (arguments.isNotEmpty) ...arguments,
],
);

if (result.exitCode != 0) {
print(result.stdout);
print(result.stderr);
Expand Down

0 comments on commit 1da1175

Please sign in to comment.