feat(e2e): add Playwright E2E test suite for UI regression coverage #401
ci.yml
on: pull_request
UI smoke (Ubuntu)
2m 52s
Matrix: source-build
UI functional and visual regression (Ubuntu)
12m 5s
Annotations
11 errors and 6 notices
|
[electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition:
e2e/visual/visual-helpers.ts#L85
5) [electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
1464 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: workspace-folder.png
Call log:
- Expect "toHaveScreenshot(workspace-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1464 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1464 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/workspace.spec.ts:26:11
|
|
[electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition:
e2e/visual/visual-helpers.ts#L85
5) [electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition
Error: expect(page).toHaveScreenshot(expected) failed
1464 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: workspace-folder.png
Call log:
- Expect "toHaveScreenshot(workspace-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1464 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1464 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/workspace.spec.ts:26:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible:
e2e/visual/visual-helpers.ts#L85
4) [electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
1859 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: library-search-cross-folder.png
Call log:
- Expect "toHaveScreenshot(library-search-cross-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 7277 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5601 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1859 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:76:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible:
e2e/visual/visual-helpers.ts#L85
4) [electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible
Error: expect(page).toHaveScreenshot(expected) failed
1859 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: library-search-cross-folder.png
Call log:
- Expect "toHaveScreenshot(library-search-cross-folder.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1859 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1859 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:76:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition:
e2e/visual/visual-helpers.ts#L85
3) [electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
1258 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: settings-appearance-light.png
Call log:
- Expect "toHaveScreenshot(settings-appearance-light.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1258 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1258 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:25:11
|
|
[electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition:
e2e/visual/visual-helpers.ts#L85
3) [electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition
Error: expect(page).toHaveScreenshot(expected) failed
1258 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: settings-appearance-light.png
Call log:
- Expect "toHaveScreenshot(settings-appearance-light.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 1258 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 1258 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/overlays.spec.ts:25:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface:
e2e/visual/visual-helpers.ts#L85
2) [electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
5691 pixels (ratio 0.02 of all image pixels) are different.
Snapshot: compact-document.png
Call log:
- Expect "toHaveScreenshot(compact-document.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5476 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 662 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5691 pixels (ratio 0.02 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:64:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface:
e2e/visual/visual-helpers.ts#L85
2) [electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface
Error: expect(page).toHaveScreenshot(expected) failed
5691 pixels (ratio 0.02 of all image pixels) are different.
Snapshot: compact-document.png
Call log:
- Expect "toHaveScreenshot(compact-document.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5476 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 662 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5691 pixels (ratio 0.02 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:64:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading:
e2e/visual/visual-helpers.ts#L85
1) [electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading, writing, and valid JSON keep distinct document surfaces
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(page).toHaveScreenshot(expected) failed
5783 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: markdown-reading.png
Call log:
- Expect "toHaveScreenshot(markdown-reading.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5783 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5783 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:29:11
|
|
[electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading:
e2e/visual/visual-helpers.ts#L85
1) [electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading, writing, and valid JSON keep distinct document surfaces
Error: expect(page).toHaveScreenshot(expected) failed
5783 pixels (ratio 0.01 of all image pixels) are different.
Snapshot: markdown-reading.png
Call log:
- Expect "toHaveScreenshot(markdown-reading.png)" with timeout 10000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 5783 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 5783 pixels (ratio 0.01 of all image pixels) are different.
at visual/visual-helpers.ts:85
83 | export async function expectLinuxScreenshot(page: Page, name: string): Promise<void> {
84 | if (process.platform !== 'linux') return;
> 85 | await expect(page).toHaveScreenshot(name, {
| ^
86 | animations: 'disabled',
87 | caret: 'hide',
88 | });
at expectLinuxScreenshot (/home/runner/work/stashbase/stashbase/e2e/visual/visual-helpers.ts:85:22)
at /home/runner/work/stashbase/stashbase/e2e/visual/document.spec.ts:29:11
|
|
Source build (windows-latest)
Process completed with exit code 1.
|
|
🎭 Playwright Run Summary
5 passed (1.6m)
|
|
🎭 Playwright Run Summary
5 did not run
|
|
🎭 Playwright Run Summary
6 passed (1.5m)
|
|
🎭 Playwright Run Summary
6 failed
[electron-visual-linux] › e2e/visual/document.spec.ts:13:1 › Markdown reading, writing, and valid JSON keep distinct document surfaces
[electron-visual-linux] › e2e/visual/document.spec.ts:50:1 › compact document layout preserves the active reading surface
[electron-visual-linux] › e2e/visual/overlays.spec.ts:12:1 › Settings and quick-access overlays preserve light and dark composition
[electron-visual-linux] › e2e/visual/overlays.spec.ts:57:1 › cross-folder exact search keeps grouped result identity visible
[electron-visual-linux] › e2e/visual/workspace.spec.ts:11:1 › active-folder workspace shell keeps its redesigned composition
[electron-visual-linux] › e2e/visual/workspace.spec.ts:32:1 › empty library keeps the redesigned zero-state composition
|
|
🎭 Playwright Run Summary
25 passed (6.4m)
|
|
🎭 Playwright Run Summary
31 did not run
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
ui-regression-31636871899-1
|
38.4 MB |
sha256:dc5b4c19a5a464a1cb8a1e9b0b63fe99e557b15a509a562d540b9f4defddc934
|
|