Dart CI package tests are run with the noisy compact reporter for package:test #54929
Labels
analyzer-test
area-infrastructure
Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes.
area-test
Cross-cutting test issues (use area- labels for specific failures; not used for package:test).
The Dart CI and CQ run tests for the packages in the Dart SDK source on builders like
https://ci.chromium.org/ui/p/dart/builders/luci.dart.ci.sandbox/pkg-linux-release
These builders run package tests written using package:test framework the same way they run any
Dart test on the runtime VM, by running the program using the Dart VM, with no special arguments because it uses the package:test framework.
The default reporter for package:test is
--reporter compact
which prints ANSI console control codes in stdout to tryand make a console keep printing progress on the same line without scrolling. In our test logs, they appear as strings like
�[32m+10�[0m�[31m -1�[0m
on every line.It would be better to use the expanded, github, or json reporters, to produce better logs on stdout for users to see using our CI infrastructure. This could be done by passing the --reporter flag in the commands created by package:test_runner, or by setting the option in a dart_test.yaml file at the root of each package.
The json report can also be sent to a file, in addition to another report format printed on stdout.
There has been discussion of a failures-only reporting format as well on dart-lang/test#829
If we reported a structured format, then our test results systems could even read that format and track individual tests in a test file separately in our CI databases, but that would be an extensive feature addition.
So we should decide on the format to report on stdout, and change package:test_runner or else add dart_test.yaml files to our individual packages, to add the correct --reporter option when running these tests.
The text was updated successfully, but these errors were encountered: