diff --git a/docs/reference/jspsych.md b/docs/reference/jspsych.md index 97aa093098..b7a25dc6fa 100644 --- a/docs/reference/jspsych.md +++ b/docs/reference/jspsych.md @@ -34,6 +34,7 @@ The settings object can contain several parameters. None of the parameters are r | override_safe_mode | boolean | Running a jsPsych experiment directly in a web browser (e.g., by double clicking on a local HTML file) will load the page using the `file://` protocol. Some features of jsPsych don't work with this protocol. By default, when jsPsych detects that it's running on a page loaded via the `file://` protocol, it runs in _safe mode_, which automatically disables features that don't work in this context. Specifically, the use of Web Audio is disabled (audio will be played using HTML5 audio instead, even if `use_webaudio` is `true`) and video preloading is disabled. The `override_safe_mode` parameter defaults to `false`, but you can set it to `true` to force these features to operate under the `file://` protocol. In order for this to work, you will need to disable web security (CORS) features in your browser - this is safe to do if you know what you are doing. Note that this parameter has no effect when you are running the experiment on a web server, because the page will be loaded via the `http://` or `https://` protocol. | | case_sensitive_responses | boolean | If `true`, then jsPsych will make a distinction between uppercase and lowercase keys when evaluating keyboard responses, e.g. "A" (uppercase) will not be recognized as a valid response if the trial only accepts "a" (lowercase). If false, then jsPsych will not make a distinction between uppercase and lowercase keyboard responses, e.g. both "a" and "A" responses will be valid when the trial's key choice parameter is "a". Setting this parameter to false is useful if you want key responses to be treated the same way when CapsLock is turned on or the Shift key is held down. The default value is `false`. | extensions | array | Array containing information about one or more jsPsych extensions that are used during the experiment. Each extension should be specified as an object with `type` (required), which is the name of the extension, and `params` (optional), which is an object containing any parameter-value pairs to be passed to the extension's `initialize` function. Default value is an empty array. | +| record_session | boolean | If `true`, jsPsych captures a high-fidelity recording of the session — DOM mutations within the display element, mouse/touch/keyboard/clipboard input, scroll position (window and per-element), video and audio playback events, viewport changes (including pinch zoom), and every `Math.random()` output — sufficient to reconstruct a replay of what the participant saw and did. The recording is retrieved at the end of the experiment via `jsPsych.getSessionRecording()` and can be `JSON.stringify`'d and saved alongside the trial data. While enabled, `Math.random` is wrapped to log every call into the recording; this is reverted when the experiment ends. The default value is `false`. **Note:** text typed into form inputs (e.g. survey responses) is captured verbatim. Inform participants accordingly. | ### Return value @@ -460,6 +461,39 @@ alert('You have completed approximately '+progress.percent_complete+'% of the ex ``` +--- + +## jsPsych.getSessionRecording + +```javascript +jsPsych.getSessionRecording() +``` + +### Parameters + +None. + +### Return value + +Returns a `SessionRecording` object when the experiment was initialized with `record_session: true`, or `undefined` otherwise. The object is JSON-serializable. + +### Description + +Returns the high-fidelity session recording produced by the `record_session` option. The recording includes the rendered DOM at the start of every trial, all DOM mutations within `#jspsych-content`, mouse, touch, keyboard, and clipboard events, scroll position (window and per-element), video and audio playback events, viewport changes, and every `Math.random()` output. The returned object is versioned (`schema_version: 1`); see [Session Recording Schema](./session-recording-schema.md) for the full format reference. + +### Example + +```javascript +var jsPsych = initJsPsych({ + record_session: true, + on_finish: function() { + var recording = jsPsych.getSessionRecording(); + var blob = new Blob([JSON.stringify(recording)], { type: "application/json" }); + // ...persist or upload `blob` alongside the trial data + } +}); +``` + --- ## jsPsych.getStartTime diff --git a/docs/reference/session-recording-schema.md b/docs/reference/session-recording-schema.md new file mode 100644 index 0000000000..2f3a8b93af --- /dev/null +++ b/docs/reference/session-recording-schema.md @@ -0,0 +1,410 @@ +# Session Recording Schema + +When `initJsPsych` is called with `record_session: true`, jsPsych captures a session recording sufficient to reconstruct a visual replay of the participant's experience. This page documents the JSON-serializable shape returned by `jsPsych.getSessionRecording()`. + +The schema is versioned. Schemas with the same major version are read-compatible; consumers should branch on `schema_version` and reject anything they don't understand. + +**Current version: `1`.** + +## Replayer model + +A replayer treats this recording as observational. It does not re-execute trial code. The replay is reconstructed from three things: + +1. The DOM snapshot captured at each trial's `on_load` (`trial.initial_dom`). +2. The chronological mutation and input event log scoped to that trial (`trial.events`). +3. The session-level stylesheet snapshot (`stylesheets`) so the reconstructed DOM is styled identically to the original. +4. Session-level metadata for context (viewport, scroll, RNG outputs, focus/blur, fullscreen). + +Each trial is a self-contained replay unit because jsPsych wipes the display element between trials. + +## Top-level shape + +```ts +interface SessionRecording { + schema_version: 1; + jspsych_version: string; + recording_started_at: string; // ISO 8601 wall-clock anchor + recording_started_at_perf: number; // performance.now() at start + user_agent: string; + viewport: ViewportState; + rng: { seed: string | null; math_random_patched: boolean }; + display_element_id: string; + stylesheets: StylesheetSnapshot[]; + stylesheet_events: StylesheetEvent[]; + trials: TrialRecording[]; + viewport_changes: ViewportChange[]; + rng_calls: RngCall[]; + ended_at_perf: number | null; + end_reason: "finished" | "aborted" | "unload" | null; +} +``` + +| Field | Description | +| ----- | ----------- | +| `schema_version` | Always `1` for recordings produced by this codebase. | +| `jspsych_version` | The jsPsych package version that produced the recording. | +| `recording_started_at` | ISO 8601 timestamp of when `start()` was called. | +| `recording_started_at_perf` | The `performance.now()` value at recording start. All event `t` values are relative to this. | +| `user_agent` | `navigator.userAgent` at recording start. | +| `viewport` | The initial viewport state. See [`ViewportState`](#viewportstate). | +| `rng.seed` | The seed installed via `jsPsych.randomization.setSeed` for the session, or `null` if `Math.random` was already non-native at recording start. | +| `rng.math_random_patched` | `true` while recording is active; `Math.random` is wrapped to log every call into `rng_calls`. | +| `display_element_id` | The `id` attribute of the display element (`#jspsych-content` by default). | +| `stylesheets` | Snapshot of every stylesheet attached to the document at session start. See [Stylesheets](#stylesheets). | +| `stylesheet_events` | Chronological log of `` stylesheet changes that occurred after `start()`. See [Stylesheet events](#stylesheet-events). | +| `trials` | Per-trial recordings, in chronological order. See [`TrialRecording`](#trialrecording). | +| `viewport_changes` | Session-level log of viewport changes (window resize, page zoom, pinch zoom, pinch pan). | +| `rng_calls` | Chronological log of every `Math.random` output. Includes calls outside trial boundaries (parameter eval, ITI, `on_finish`). | +| `ended_at_perf` | The `performance.now()` at `stop()`. | +| `end_reason` | How the session ended. `"aborted"` when `abortExperiment()` was called. | + +All `t` fields elsewhere in the document are floats in milliseconds, relative to `recording_started_at_perf` (i.e. `performance.now() - recording_started_at_perf`). + +## TrialRecording + +```ts +interface TrialRecording { + trial_index: number; + t_start: number; + t_dom_ready: number | null; + t_end: number | null; + plugin: string; + initial_dom: DomNode | null; + events: RecordedEvent[]; + trial_data: JsonValue; +} +``` + +| Field | Description | +| ----- | ----------- | +| `trial_index` | The jsPsych trial index. Matches `trial_data.trial_index` and the row in `data.csv`. | +| `t_start` | When the trial entered `onTrialStart` (before parameter evaluation completes). | +| `t_dom_ready` | When the plugin's initial render completed (`on_load` fired). `initial_dom` was sampled at this moment. `null` if a recording was stopped before this point. | +| `t_end` | When the trial ended. `null` if the recording was stopped before the trial finished. | +| `plugin` | The plugin name (e.g. `"html-keyboard-response"`). For labeling and filtering only; replay does not depend on it. | +| `initial_dom` | A "spine" DOM tree at `t_dom_ready` that walks from the display container down to the display element and includes the trial subtree. See [DOM representation](#dom-representation) and [Display spine](#display-spine). | +| `events` | Chronological log of mutations and input events from `t_dom_ready` to `t_end`. Empty arrays are valid. | +| `trial_data` | The data row written to `jsPsych.data` for this trial (the same object exported via `data.csv` / `data.json`). Includes `rt`, `response`, `trial_type`, `trial_index`, etc. | + +### Replay procedure for a single trial + +1. Clear the host element you render into. +2. Instantiate `initial_dom` into the host. The recording's `initial_dom` is itself rooted at the experiment's display container (carrying the `jspsych-display-element` class), so you do not need to add wrapper classes yourself. +3. Apply each entry in `events` at its `t`, in order. + +### Display spine + +`initial_dom` is not just the trial content — it is the chain of layout-bearing ancestors that wrap the trial content, with non-essential siblings stripped out. The shape is fixed for any jsPsych experiment: + +``` +display container (e.g. or the user's host) +└──
+ └──
+ └── (trial-rendered content) +``` + +The recorder captures each ancestor's tag and attributes, but only the descendant on the path to the display element appears as a child. So when `display_element` defaults to ``, the captured `` node carries body's classes/style but has only the wrapper as a child — sibling content elsewhere in `` is intentionally omitted. + +Why this matters: the CSS that vertically centers experiment content lives on `.jspsych-content-wrapper` (`margin: auto; flex: 1 1 100%`) and `.jspsych-display-element` (`display: flex; flex-direction: column`). Without the wrappers, `.jspsych-content`'s `margin: auto` has no flex parent to center against and the layout collapses to top-aligned, unstyled content. + +DOM mutations during the trial reference ids assigned during the spine serialization. The `MutationObserver` is scoped to the display element (`#jspsych-content`), so only mutations within that subtree are tracked at runtime — the wrappers are static and don't produce mutation events. + +## DOM representation + +```ts +type DomNode = ElementNode | TextNode | CommentNode; + +interface ElementNode { + id: number; + kind: "element"; + tag: string; // lowercased tag name + attrs: Record; + children: DomNode[]; + canvas_size?: { w: number; h: number }; + media_src?: string; +} + +interface TextNode { id: number; kind: "text"; text: string; } +interface CommentNode { id: number; kind: "comment"; text: string; } +``` + +Every node in `initial_dom` is assigned a monotonically-increasing integer `id`. Mutation events reference these ids. New nodes added later (via `dom.add`) carry their own id and may have child nodes that recursively carry ids of their own. + +**Per-trial scope.** Node ids are reset at every `t_dom_ready`. Ids in `trials[i]` have no relationship to ids in `trials[j]`. + +**Element-specific extras.** + +- `canvas_size`: present on `` elements; carries `width`/`height` attributes at snapshot time. The pixel contents at any later moment are recorded as separate [`canvas.snapshot`](#canvas-snapshots) events keyed by node id. +- `media_src`: present on `