Skip to content

Commit 5993729

Browse files
committed
chore(cli): fix recording env
1 parent a486c85 commit 5993729

File tree

2 files changed

+25
-58
lines changed

2 files changed

+25
-58
lines changed

README.md

+7-50
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ When running locally the action uses A11yWatch Lite.
2121
LIST: true
2222
FIX: false
2323
UPGRADE: false
24-
RECORD: false
24+
RECORD: ./recordings
2525
COMPUTER_VISION_SUBSCRIPTION_KEY: ${{ secrets.COMPUTER_VISION_SUBSCRIPTION_KEY }}
2626
COMPUTER_VISION_ENDPOINT: ${{ secrets.COMPUTER_VISION_ENDPOINT }}
2727
```
@@ -39,7 +39,7 @@ All inputs are **optional** except $WEBSITE_URL.
3939
| `SITEMAP` | Extend crawl with sitemap links (required SITE_WIDE=true). | true |
4040
| `TLD` | Include all tld extensions (required SITE_WIDE=true). | true |
4141
| `LIST` | Report the results to github as a pass or fail list or detailed report. | false |
42-
| `RECORD` | Record the audit as video to a directory. | |
42+
| `RECORD` | Record the audit as video to a directory. | |
4343
| `FAIL_TOTAL_COUNT` | Determine whether to fail the CI if total issues warnings and errors exceed the counter. Takes precedence over the other FAIL inputs. | 0 |
4444
| `FAIL_ERRORS_COUNT` | Determine whether to fail the CI if total issues with errors exceed the counter. | 0 |
4545
| `FAIL_WARNINGS_COUNT` | Determine whether to fail the CI if total issues with warnings exceed the counter. | 0 |
@@ -58,61 +58,18 @@ All inputs are **optional** except $WEBSITE_URL.
5858
| -------- | -------------------------- | ------- |
5959
| `issues` | The amount of issues found | |
6060

61-
## Benches
6261

63-
The hardware specs for results:
62+
## Performance
6463

65-
```sh
66-
----------------------
67-
linux ubuntu-latest
68-
2-core CPU
69-
7 GB of RAM memory
70-
14 GB of SSD disk space
71-
-----------------------
72-
```
73-
74-
### Benchmark Results
75-
76-
The results for crawling a website that is small - large.
77-
78-
#### crawl-speed
79-
80-
Test url: `https://a11ywatch.com`
81-
82-
25 pages
83-
84-
runs with 10 samples:
85-
86-
| | `libraries` |
87-
| :--------------------- | :--------------------- |
88-
| **`A11yWatch: crawl`** | `0.4 s` (✅ **1.00x**) |
89-
| **`Pa11y-CI: crawl`** | `45 s` (✅ **1.00x**) |
90-
| **`Axe: crawl`** | `N/A` (✅ **1.00x**) |
91-
92-
#### crawl-speed (Large Website)
93-
94-
Test url: `https://www.hbo.com`
95-
96-
7500 pages.
97-
98-
runs with 10 samples:
99-
100-
| | `libraries` |
101-
| :--------------------- | :------------------------ |
102-
| **`A11yWatch: crawl`** | `2.5 mins` (✅ **1.00x**) |
103-
| **`Pa11y-CI: crawl`** | `50+ hr` (✅ **1.00x**) |
104-
| **`Axe: crawl`** | `N/A` (✅ **1.00x**) |
105-
106-
### Benchmark Info
107-
108-
On a larger website A11yWatch action runs over 60x-10,000x+ faster depending on CPUs/hardware.
64+
On a larger website A11yWatch action runs over 60x-10,000x+ faster depending on CPUs/hardware. After the first installation you should have faster setup time between runs.
65+
You can expect to handle at least 1k pages per minute on a 2-core CPU 7 GB of RAM memory shared github action. If you enable `RECORD` the output time may increase a bit.
10966

11067
When `AI_DISABLED` is set to true the run for `A11yWatch` may increase.
11168

11269
## Common Issues
11370

114-
If you experience issues on your CI you may have to toggle the `UPGRADE` input to true in order to get the latest docker images.
115-
We need docker in order to build the appliciation in quickly since we have some services that need to compile that may take awhile.
71+
If you experience issues on your CI you may have to toggle the `UPGRADE` input to true in order to get the latest installs. If you see
72+
a playwright error try adding `PLAYWRIGHT_VERSION` env variable with the newest version to install chrome.
11673

11774
## CLI
11875

action.yml

+18-8
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ author: A11yWatch
88
branding:
99
icon: "shield"
1010
color: "gray-dark"
11-
description: Powerful web accessibility testing.
11+
description: Fast and feature-packged web accessibility testing.
1212
inputs:
1313
WEBSITE_URL:
1414
description: Website domain to scan
@@ -89,6 +89,9 @@ outputs:
8989
issues:
9090
description: "The amount of issues found on the page"
9191
value: ${{ steps.a11ywatch-results-generator.outputs.issues }}
92+
recording_dir:
93+
description: "The directory that holds recordings of audits. Only enabled when inputs.RECORD is set."
94+
value: ${{ steps.a11ywatch-recordings-directory.outputs.recording_dir }}
9295
runs:
9396
using: composite
9497
steps:
@@ -148,15 +151,15 @@ runs:
148151
uses: baptiste0928/cargo-install@v1
149152
with:
150153
crate: a11ywatch_cli
151-
version: "0.10.7"
154+
version: "0.10.8"
152155
features: "grpc"
153156

154157
- name: A11yWatch CLI Install
155158
if: ${{ inputs.SLIM == 'false' }}
156159
uses: baptiste0928/cargo-install@v1
157160
with:
158161
crate: a11ywatch_cli
159-
version: "0.10.7"
162+
version: "0.10.8"
160163

161164
- name: Configure CLI creds
162165
if: ${{ inputs.A11YWATCH_TOKEN }}
@@ -178,11 +181,6 @@ runs:
178181
shell: bash
179182
run: a11ywatch --set-recording ${{ inputs.RECORD }}
180183

181-
- name: Configure Recording
182-
if: ${{ inputs.RECORD }}
183-
shell: bash
184-
run: a11ywatch --set-recording ${{ inputs.RECORD }}
185-
186184
- uses: FedericoCarboni/setup-ffmpeg@v2
187185
if: ${{ inputs.RECORD && inputs.RECORD != 'false' }}
188186

@@ -313,6 +311,18 @@ runs:
313311
name: Recordings
314312
path: ${{ inputs.RECORD && inputs.RECORD != 'false' && inputs.RECORD || './recordings' }}
315313

314+
- name: Get recording directory
315+
id: a11ywatch-recordings-directory
316+
if: ${{ inputs.RECORD && inputs.RECORD != 'false' && failure() || success() }}
317+
shell: bash
318+
run: echo "recording_dir=$( echo "$(a11ywatch --find-tmp-dir)""/recordings" )" >> $GITHUB_OUTPUT
319+
320+
- uses: actions/upload-artifact@v3
321+
if: ${{ inputs.RECORD && inputs.RECORD != 'false' && failure() || success() }}
322+
with:
323+
name: Recordings
324+
path: ${{ steps.a11ywatch-recordings-directory.outputs.recording_dir }}
325+
316326
- name: Stop A11yWatch Service
317327
if: ${{ !inputs.EXTERNAL }}
318328
shell: bash

0 commit comments

Comments
 (0)