Behavioral analyzers compare across different LiveViews: 5 serious findings (and exit 1) on a healthy timeline

[lessthanseventy]

Summary

mix excessibility.review --timeline produced 5 :serious behavioral findings on a completely healthy LiveView journey test, and because Mix.Tasks.Excessibility.Review.maybe_exit/2 treats any serious behavioral finding as blocking, the run exited 1 on a PR with zero newly introduced accessibility findings.

I dug into the timeline rather than just muting them, and all of the serious ones trace back to one root cause: the analyzers compare consecutive timeline events without grouping by LiveView. A journey test drives several LiveViews, so the timeline interleaves unrelated processes.

Version: 0.15.3. Real Phoenix app, 12-event timeline from one journey test (200 ms total).

The timeline (numbers only — no assigns)

idx event               view            total_memory  evt_ms  since_ms  list_keys
 0  mount               Index                  24,012      40      —          102
 1  handle_params       Index                  24,030       0      1          102
 2  mount               Index                  24,012       4     93          102
 3  handle_params       Index                  24,030       0      0          102
 4  render              Index                  24,030       2      3          102
 5  render              Index                  36,831      14     38          148
 6  mount               UserLoginLive             220       0     10            3
 7  render              Index                 111,919       0     27          469
 8  mount               UserLoginLive             220       0     12            3
 9  handle_event:event  Index                 111,989       0      3          469
10  render              Index                 111,989       7      8          469
11  mount               UserLoginLive             220       0      5            3

What was reported, and why each is an artifact

1. [serious] memory: Memory grew 507.7x between events (220 B → 109.3 KB) (×2)

  6->7: 220B -> 111919B (508.7x)  same_view=False  [UserLoginLive -> Index]
  8->9: 220B -> 111989B (509.0x)  same_view=False  [UserLoginLive -> Index]

This is a freshly mounted UserLoginLive (220 B of assigns) followed by a loaded MarketplaceLive.Index render. Nothing grew; two different views sit next to each other in the list. Note also that the "large" side is 109 KB, which is an ordinary LiveView heap — the ratio is doing all the work here, off a 220-byte denominator.

2. [moderate] data_growth: List 'order_form.source.changes.items' growing: 0 → 0 → … → 1 (∞x)

The actual series across the 12 events, where None means the key is absent (the UserLoginLive events):

0 → 0 → 0 → 0 → 0 → 0 → None → 1 → None → 1 → 1 → None

So ∞x is 0 → 1 with cross-view gaps in between. A list going from empty to one element is normal, and an unbounded ratio from a zero baseline isn't a useful severity input.

3. [serious] performance: Very slow event (40ms, 6.7x average) / Performance bottleneck: event took 40ms (60% of total time)

The 40 ms event is index 0 — the first mount of the test. On a 12-event, 200 ms timeline, some event is always going to be the majority of total time; "60% of total time" is a statement about sample size, not about the code. 40 ms for a first mount under Ecto.Adapters.SQL.Sandbox is also unremarkable.

4. [serious] render_efficiency: 2 of 4 renders (50%) had no state changes

Percentage over 4 samples, and the render list mixes views.

Bonus noise source: assign traversal descends into library internals

list_sizes tracked 488 distinct keys for this one test, of which:

• 204 reach into Ecto changeset internals — .types., .mappings, .repo_opts, .empty_values, .validations, .constraints, .prepare
• 182 are microsecond precision tuples — e.g. …product.inserted_at.microsecond = 2

Representative key actually being tracked as app state:

order_form.source.changes.items[0].types.carrier_code[1][1].mappings[7] = 2

That's Ecto.Enum mapping metadata, not something a developer can act on, and it's what data_growth is drawing candidates from.

Possible directions

1. Group events by view_module (or the LiveView pid) before any consecutive-event comparison. This alone removes findings 1, 2 and most of 4.
2. Absolute floors alongside ratios. e.g. don't emit a memory finding under a few hundred KB regardless of ratio; treat 0 → n as "appeared", not ∞x.
3. Minimum sample size for percentage/aggregate claims (render efficiency, "% of total time"), or downgrade severity when n is small.
4. Stop the assign traversal at %Ecto.Changeset{}, %Ecto.Association.NotLoaded{}, DateTime/Time structs, and skip microsecond tuples — they're never actionable app state.
5. Don't let behavioral findings gate the exit code by default. They have no baseline (unlike the accessibility finding-delta, which is a true delta), so they're inherently absolute measurements of a single run. Advisory by default with an opt-in like --fail-on-behavioral would match how much confidence they can carry.

Happy to test patches — I have a replay harness that reproduces this timeline in about two minutes.

Note for whoever picks this up

test/excessibility/timeline.json and latest_debug.md contain serialized assigns, which in a real app includes user records. Everything quoted above is deliberately numbers, key paths and view module names only.