Skip to content

Commit 3504533

Browse files
committed
Bug 1701674 - [remote] Add handling of expectation data to readme. r=remote-protocol-reviewers,jdescottes
Differential Revision: https://phabricator.services.mozilla.com/D113499
1 parent c575a89 commit 3504533

File tree

2 files changed

+74
-17
lines changed

2 files changed

+74
-17
lines changed

remote/README

-17
This file was deleted.

remote/README.md

+74
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
The Firefox remote agent is a low-level debugging interface based
2+
on the CDP protocol.
3+
4+
With it, you can inspect the state and control execution of documents
5+
running in web content, instrument Gecko in interesting ways,
6+
simulate user interaction for automation purposes, and debug
7+
JavaScript execution.
8+
9+
This component provides an experimental and partial implementation
10+
of a remote devtools interface using the CDP protocol and transport
11+
layer.
12+
13+
See https://firefox-source-docs.mozilla.org/remote/ for documentation.
14+
15+
It is available in Firefox and is started this way:
16+
17+
% ./mach run --remote-debugging-port
18+
19+
20+
Puppeteer
21+
=========
22+
Puppeteer is a Node library which provides a high-level API to control Chrome,
23+
Chromium, and Firefox over the Chrome DevTools Protocol. Puppeteer runs headless
24+
by default, but can be configured to run full (non-headless) browsers.
25+
26+
To verify that our implementation of the CDP protocol is valid we do not only
27+
run xpcshell and browser-chrome mochitests in Firefox CI but also the Puppeteer
28+
unit tests.
29+
30+
Expectation Data
31+
----------------
32+
33+
With the tests coming from upstream, it is not guaranteed that they
34+
all pass in Gecko-based browsers. For this reason it is necessary to
35+
provide metadata about the expected results of each test. This is
36+
provided in a manifest file under `test/puppeteer-expected.json`.
37+
38+
For each test of the Puppeteer unit test suite an equivalent entry will exist
39+
in this manifest file. By default tests are expected to `PASS`.
40+
41+
Tests that are intermittent may be marked with multiple statuses using
42+
a list of possibilities e.g. for a test that usually passes, but
43+
intermittently fails:
44+
45+
"Page.click should click the button (click.spec.ts)": [
46+
"PASS", "FAIL"
47+
],
48+
49+
Disabling Tests
50+
---------------
51+
52+
Tests are disabled by using the manifest file `test/puppeteer-expected.json`.
53+
For example, if a test is unstable, it can be disabled using `SKIP`:
54+
55+
"Workers Page.workers (worker.spec.ts)": [
56+
"SKIP"
57+
],
58+
59+
For intermittents it's generally preferable to give the test multiple
60+
expectations rather than disable it.
61+
62+
Autogenerating Expectation Data
63+
-------------------------------
64+
65+
After changing some code it may be necessary to update the expectation
66+
data for the relevant tests. This can of course be done manually, but
67+
`mach` is able to automate the process:
68+
69+
mach puppeteer-test --write-results
70+
71+
By default it writes the output to `test/puppeteer-expected.json`.
72+
73+
Given that the unit tests run in Firefox CI only for Linux it is advised to
74+
download the expectation data (available as artifact) from the TaskCluster job.

0 commit comments

Comments
 (0)