[CI] Add a11y checks to our visual testing suite (#7047)

* Add a VISUAL_URL constant and remove all vestiges of hide inspector and tree * hide timer and add concurrency * turn off concurrency * factor out old appAction * Add expand button to panes * remove old slow annotations * fix fault * update domcontentloaded * missed refactor * driveby: setTimeBounds private * add comments to the percyCSS * suggest MISSION_TIME * more notes * regen * clean up test * driveby: clean up order * restructure * add new suite now that i'ts hidden * use mission time everywhere possible * driveby * rerun generatedata * comments * lint fix * add inital pass of a11y tests * first pass for fixing a11y problems * update build * add copyright * check for slashes * rename files * update testcases * update to latest * updates * section 508 * final version * remove leftover * comments * documentation * bad merge * comment * use current ruleset * typo * feedback * remove time conductor due to false positives * default to closed tabs * add some more accessiblity checking * change to a condition widget and update search * lint fix * turns this into a single function * update doc to match single function * update to single function * update to new function * lint * update locator for search input * fix extra page * why * comments * comments * refacotr * wrong paths and fixes
2023-12-19 14:16:08 -08:00
parent ec910dcbdc
commit 0d97675a0a
31 changed files with 480 additions and 381 deletions
--- a/e2e/playwright-visual-a11y.config.js
+++ b/e2e/playwright-visual-a11y.config.js
@@ -0,0 +1,54 @@
+/* eslint-disable no-undef */
+// playwright.config.js
+// @ts-check
+
+/** @type {import('@playwright/test').PlaywrightTestConfig<{ theme: string }>} */
+const config = {
+  retries: 0, // Visual tests should never retry due to snapshot comparison errors. Leaving as a shim
+  testDir: 'tests/visual-a11y',
+  testMatch: '**/*.visual.spec.js', // only run visual tests
+  timeout: 60 * 1000,
+  workers: 1, //Lower stress on Circle CI Agent for Visual tests https://github.com/percy/cli/discussions/1067
+  webServer: {
+    command: 'npm run start:coverage',
+    url: 'http://localhost:8080/#',
+    timeout: 200 * 1000,
+    reuseExistingServer: !process.env.CI
+  },
+  use: {
+    baseURL: 'http://localhost:8080/',
+    headless: true, // this needs to remain headless to avoid visual changes due to GPU rendering in headed browsers
+    ignoreHTTPSErrors: true,
+    screenshot: 'only-on-failure',
+    trace: 'on-first-retry',
+    video: 'off'
+  },
+  projects: [
+    {
+      name: 'chrome',
+      use: {
+        browserName: 'chromium'
+      }
+    },
+    {
+      name: 'chrome-snow-theme', //Runs the same visual tests but with snow-theme enabled
+      use: {
+        browserName: 'chromium',
+        theme: 'snow'
+      }
+    }
+  ],
+  reporter: [
+    ['list'],
+    ['junit', { outputFile: '../test-results/results.xml' }],
+    [
+      'html',
+      {
+        open: 'on-failure',
+        outputFolder: '../html-test-results' //Must be in different location due to https://github.com/microsoft/playwright/issues/12840
+      }
+    ]
+  ]
+};
+
+module.exports = config;