VJ-9000

by GANTASMO

VJ-9000 is a browser-based, audio-reactive visual engine for live performance. It renders live cameras, video clips, generated sources, and still images through a real-time WebGL effects stack with MIDI-mappable controls and audio reactivity. It runs standalone in a browser and serves as the live-visuals engine embedded in theDAW.

How a frame is built

flowchart LR
  subgraph Sources["Source (one of)"]
    CAM["Camera<br/>webcam, phone, Quest"]
    CLIP["Video clip or still"]
    SHADER["GLSL shader<br/>fractals, 8 materials"]
    DEPTH["Depth cloud<br/>akvj, spectra, cymatics"]
    QUEST["Quest passthrough<br/>questcast, queststitch"]
  end
  subgraph Banks["Source banks"]
    BANK["Snapshot and recall slots<br/>click to place"]
  end
  Sources --> MIX["LIVE / CLIP crossfader"]
  Banks --> MIX
  MIX --> FX["GPU effect chain"]
  FX --> ASCII["ASCII effect (optional)"]
  ASCII --> OUT["Canvas output"]
  AUDIO["Audio levels or mic"] --> FX
  AUDIO --> SHADER
  MIDI["MIDI, SLIDE, Sway pose"] --> MIX
  MIDI --> FX
  OUT --> REC["WebM record / transcode"]
  OUT --> WATCH["WebRTC watch-link"]

Loading

Sources

A single active source feeds the effect chain, and the LIVE/CLIP crossfader blends a live source against a loaded clip.

• Cameras. A local webcam or capture card, and a phone, tablet, or headset camera reached over the LAN through a URL and QR code.
• Clips and stills. Video clips and image backdrops, loaded by drag-and-drop or a file picker.
• GLSL shader. A generic shader source on the atzedent uniform convention, currently a Menger flythrough plus four distance-field fractals (Mandelbulb, Julia, Mandelbox, kaleidoscopic IFS). Each exposes editable, audio-mappable params and a Hue control, and a global Material picker (Neon, Chrome, Matte, Glass, Gold, Iridescent, Velvet, Plasma) reshades the scene.
• Depth and generative. Cymatics, an akvj depth point cloud, and a spectra source, all reactive to the audio.
• Quest passthrough. A Quest 3 headset view arrives as a live source through theDAW's questcast and queststitch bridges.
• Source banks. Snapshot the live source into a bank slot, or click an empty slot to place a clip, a source, or a local file at the click point, then recall it during a set.

Effects

A composable GPU chain, every node MIDI-mappable, with a SOLO mode that isolates one effect for setup. The ASCII effect renders the live output through a glyph atlas as a post pass.

• Color and optics: hue, saturation, contrast, brightness, invert, edge detect.
• Geometry: mirror X/Y, kaleidoscope, radial mirror, tiling, equirectangular 360, stereo side-by-side and top-bottom, soft edges.
• Generative: reaction-diffusion (Gray-Scott), SDF raymarch portal, topographic isolines, fluid displacement.
• Depth: depth fog, tilt-shift miniature, z-quantized plane splits, and depth-edge outline from a luminance depth proxy.
• Distortion and glitch: feedback, glitch, RGB ghost and split, chromatic aberration, buffer backskip, strobe, pixelate, wave warp.
• Time: playback speed, reverse, posterize-time, echo trails, slit-scan, time displacement.
• Post: scanlines, vignette, CRT, the ASCII effect, and CSS-filter looks for sepia, grayscale, and blur.

Reactivity, control, and capture

• Reactivity. Audio reactivity from the host player's levels or a local microphone, BPM sync, an auto-LFO, and an Autopilot that sequences effects and clip switches.
• Control. Every control is MIDI-mappable, the host's SLIDE surface stays in two-way sync, and the Sway pose control bus drives parameters from camera-tracked motion.
• Recording. Captures the live canvas to WebM (VP9 and Opus) at 720p, 1080p, or 4K. Embedded in theDAW, the take transcodes to H.264, H.265, ProRes, or a PNG sequence.
• Broadcast. A WebRTC watch-link streams the live output and audio to remote viewers.
• Performance. Render-scale tiers trade sharpness for frame rate, and the render loop parks near zero GPU when the host tab is backgrounded.

theDAW integration

Inside theDAW's VJ tab, VJ-9000 runs in an iframe and communicates with the host over postMessage:

• theDAW streams its master-player audio levels at about 30 fps so the visuals react to whatever is playing, forwards MIDI, and pushes track metadata and play/pause state.
• The host's SLIDE tab stays in two-way sync with VJ-9000's controls, so a fader moved in either place updates the other.
• Clips and images imported into VJ-9000 upload to theDAW's library, and their session blob: URLs are replaced with stable library URLs so a cue survives a reload.
• A LAN URL and QR code make the output reachable from a phone or tablet on the same network for a second screen or a camera source.

Running locally

Prerequisites: Node.js.

npm install

# Development with HMR:
npm run dev

# Production build and serve (how theDAW runs it):
npm run build
npm run preview

theDAW spawns this app automatically as a sidecar, so a manual run is only needed when developing VJ-9000 itself.

---

_{VJ-9000 is part of theDAW, made by Daniel Joaquin Trujillo and Josh Valenzuela as GANTASMO.}