feat(e2e): add Playwright E2E test suite for UI regression coverage #399
ci.yml
on: pull_request
UI smoke (Ubuntu)
2m 37s
Matrix: source-build
UI functional and visual regression (Ubuntu)
10m 22s
Annotations
10 errors and 5 notices
|
[electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition:
e2e/visual/visual-helpers.ts#L85
5) [electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
1464 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: workspace-folder.png
Call log:
- Expect "toHaveScreenshot(workspace-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1464 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1464 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/workspace.spec.ts:26:11
|
|
[electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition:
e2e/visual/visual-helpers.ts#L85
5) [electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition
Error: expect(page).toHaveScreenshot(expected) failed
1464 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: workspace-folder.png
Call log:
- Expect "toHaveScreenshot(workspace-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1464 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1464 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/workspace.spec.ts:26:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible:
e2e/visual/visual-helpers.ts#L85
4) [electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
1859 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: library-search-cross-folder.png
Call log:
- Expect "toHaveScreenshot(library-search-cross-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1859 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1859 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:76:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible:
e2e/visual/visual-helpers.ts#L85
4) [electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible
Error: expect(page).toHaveScreenshot(expected) failed
1859 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: library-search-cross-folder.png
Call log:
- Expect "toHaveScreenshot(library-search-cross-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1859 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1859 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:76:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition:
e2e/visual/visual-helpers.ts#L85
3) [electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
1258 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: settings-appearance-light.png
Call log:
- Expect "toHaveScreenshot(settings-appearance-light.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1258 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1258 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:25:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition:
e2e/visual/visual-helpers.ts#L85
3) [electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition
Error: expect(page).toHaveScreenshot(expected) failed
1258 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: settings-appearance-light.png
Call log:
- Expect "toHaveScreenshot(settings-appearance-light.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1258 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1258 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:25:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface:
e2e/visual/visual-helpers.ts#L85
2) [electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
5691 pixels (ratio 0.02 of all image pixels) are different.
Snapshot: compact-document.png
Call log:
- Expect "toHaveScreenshot(compact-document.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5476 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 662 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5691 pixels (ratio 0.02 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:64:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface:
e2e/visual/visual-helpers.ts#L85
2) [electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface
Error: expect(page).toHaveScreenshot(expected) failed
5691 pixels (ratio 0.02 of all image pixels) are different.
Snapshot: compact-document.png
Call log:
- Expect "toHaveScreenshot(compact-document.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5476 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 662 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5691 pixels (ratio 0.02 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:64:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading:
e2e/visual/visual-helpers.ts#L85
1) [electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading, writing, and valid JSON keep distinct document surfaces
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
5783 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: markdown-reading.png
Call log:
- Expect "toHaveScreenshot(markdown-reading.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5783 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5783 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:29:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading:
e2e/visual/visual-helpers.ts#L85
1) [electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading, writing, and valid JSON keep distinct document surfaces
Error: expect(page).toHaveScreenshot(expected) failed
5783 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: markdown-reading.png
Call log:
- Expect "toHaveScreenshot(markdown-reading.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5783 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5783 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:29:11
|
|
🎭 Playwright Run Summary
5 passed (1.5m)
|
|
🎭 Playwright Run Summary
5 did not run
|
|
🎭 Playwright Run Summary
6 failed
[electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading, writing, and valid JSON keep distinct document surfaces
[electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface
[electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition
[electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible
[electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition
[electron-visual-linux] › e2e/visual/workspace.spec.ts:32:1 › empty library keeps the redesigned zero-state composition
|
|
🎭 Playwright Run Summary
25 passed (6.3m)
|
|
🎭 Playwright Run Summary
31 did not run
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
ui-regression-31636584401-1
|
20.8 MB |
sha256:26d0927fb7a7a48d7ac8a161f401bea1d3e46c01d085e727a312aff5c5d1c159
|
|