A music collection organizer that ingests music and concert files, fingerprints and analyzes them, uses AI to propose better filenames and destination paths, and provides a web UI to review and approve renames. All file operations use a safe copy-verify-delete protocol with full audit trails.

🚀 Quick Start | 📖 Documentation | 🌟 Features | 💬 Community

🎯 What is Phaze?

Phaze is a music collection organizer for managing a large personal archive of music and live concert recordings. It provides:

• 🎵 Audio Analysis: BPM, key, mood, and style detection via essentia-tensorflow
• 🔍 Audio Fingerprinting: Dual-engine deduplication with audfprint (landmark) and Panako (tempo-robust)
• 🤖 AI-Powered Renaming: LLM-generated filename and path proposals via litellm
• 🎧 Tracklist Matching: Concert set identification from 1001Tracklists
• 👀 Human-in-the-Loop: Web UI to review, approve, or reject every proposed change
• 🔒 Safe File Operations: Copy-verify-delete protocol with full audit trails -- nothing moves without review

Perfect for DJs, music collectors, and live recording enthusiasts who want their messy archives properly named, organized, and deduplicated.

🏛️ Architecture Overview

🧭 The Console (v7.0 DAG-Centric Shell)

Phaze's admin UI is a three-column "Hybrid Console" shell — a single screen with no content tabs. The pipeline DAG rail on the left is the navigation spine: it shows every stage with a live count, and clicking a stage swaps the center stage workspace in place over HTMX (GET /s/<stage>, no full-page reload and no tab bar). The right column is the per-file pane. / renders the shell with Analyze selected by default.

• DAG rail = navigation. Discover → Enrich (Metadata · Fingerprint · Analyze) → Identify (Track-ID · Tracklist) → Propose → Review & Apply (Rename · Tag write · Move · Dedupe · Cue), with Audit log and the Agents/Compute page below the line. Live per-stage counts ride the single /pipeline/stats 5-second poll.
• ⌘K command palette. A Cmd-K command palette unifies search across files, tracklists, and artists plus quick commands — it replaces the old global-search tab.
• Header status strip. Compute/agent liveness surfaces in a header status strip alongside the force-local pill (the runtime routing override, below); Kueue lanes are modeled as ephemeral Job-based identities, so they are never shown as perpetually dead.
• N-lane Analyze grid. The Analyze workspace renders a server-derived N-lane grid — one card per registry backend, sorted rank-ascending — showing each lane's kind, cost-tier rank, per-lane in-flight/cap, live availability, and a Kueue admission caption (quota-waiting / inadmissible counts). The grid is built from get_backend_lane_snapshot() and OOB-swaps as a unit off the shared poll.
• Per-file record slide-in. Opening a file row (or picking it from ⌘K) slides a full per-file record — windowed analysis timeline, metadata/identity, and this file's pending approvals — in over the shell.

This is an information-architecture and presentation layer over the existing routers and services — the analysis, identify, proposal, and execution behavior is unchanged.

☁️ Multi-Cloud Backends (2026.7.1)

Analysis dispatch runs over a pluggable backend registry. Backends are declared in a backends.toml file (loaded via PHAZE_BACKENDS_CONFIG_FILE, default /etc/phaze/backends.toml); an absent file means an implicit local-only registry (a single kind=local backend at rank=99, cap=1). The file is a typed [[backends]] list, discriminated by kind — local | compute | kueue — where each backend carries a cost-tier rank (ascending = preferred; local sorts last at 99) and a concurrency cap. A compute backend is one rsync-over-Tailscale OCI A1 agent (≤1); a kueue backend is a Kueue cluster, and N Kueue clusters are supported simultaneously. Each non-local backend stages payloads through a per-backend S3 bucket declared in a [[buckets]] registry.

Cloud is derived, not toggled — cloud_enabled is simply "the registry has any non-local backend," so there is no env master switch (the old PHAZE_CLOUD_TARGET selector was removed in Phase 67). Dispatch is a tiered drain: rank-first, spilling to the next rank, and only spilling to slow local after cloud_spill_to_local_after_seconds when higher ranks are online-but-full (immediate to local only when every non-local backend is offline). A runtime force-local pill (POST /pipeline/routing/force-local, a durable route_control row) lets an operator route everything local during an incident — reversible, no redeploy.

Deep detail lives in Architecture Overview and the Runbook.

⚙️ Services

Service	Port	Purpose	Key Technologies
API	8000	FastAPI application server	FastAPI, SQLAlchemy, asyncpg
Worker	--	SAQ async background task processor	SAQ, PostgreSQL, essentia, mutagen
Postgres	5432	Primary database + SAQ queue broker	PostgreSQL 18, Alembic, psycopg
Redis	6379	Cache, rate-limit, counters	Redis 8
Audfprint	8001	Landmark-based audio fingerprinting	audfprint
Panako	8002	Tempo-robust audio fingerprinting	Panako

📐 System Architecture

graph TD
    subgraph Frontend ["🌐 Frontend"]
        UI["🖥️ DAG-Centric Console<br/>HTMX + Tailwind + Alpine<br/>DAG rail nav · stage workspaces · ⌘K palette"]
    end

    subgraph Backend ["⚙️ Backend"]
        API["🚀 FastAPI<br/>async, :8000<br/>/api/v1/* · /proposals · /pipeline"]
        WORKER["🔧 SAQ Worker<br/>async background jobs"]
    end

    subgraph Storage ["💾 Storage"]
        PG[("🐘 PostgreSQL 18<br/>:5432<br/>DB + SAQ broker")]
        REDIS[("🔴 Redis 8<br/>:6379<br/>cache only")]
    end

    subgraph Fingerprint ["🎵 Fingerprinting"]
        AUD["🎯 Audfprint :8001<br/>landmark"]
        PAN["🎼 Panako :8002<br/>tempo-robust"]
    end

    subgraph Backends ["☁️ Analysis Backends — rank-tiered drain"]
        LOCAL["🖥️ Local<br/>essentia · rank 99"]
        COMPUTE["☁️ Compute agent<br/>OCI A1 · rsync/Tailscale · ≤1"]
        KUEUE["⎈ Kueue clusters<br/>N ClusterQueues"]
        S3[("🪣 Per-backend S3 staging")]
    end

    UI --> API
    API --> PG
    API --> REDIS
    API --> WORKER
    WORKER --> AUD
    WORKER --> PAN
    WORKER --> PG

    API -->|dispatch by rank| LOCAL
    API -->|dispatch by rank| COMPUTE
    API -->|dispatch by rank| KUEUE
    COMPUTE --> S3
    KUEUE --> S3

    style UI fill:#e3f2fd,stroke:#0d47a1,stroke-width:2px
    style API fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px
    style WORKER fill:#fff3e0,stroke:#e65100,stroke-width:2px
    style PG fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    style REDIS fill:#ffebee,stroke:#b71c1c,stroke-width:2px
    style AUD fill:#e0f2f1,stroke:#004d40,stroke-width:2px
    style PAN fill:#fce4ec,stroke:#880e4f,stroke-width:2px
    style LOCAL fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px
    style COMPUTE fill:#e3f2fd,stroke:#0d47a1,stroke-width:2px
    style KUEUE fill:#fff8e1,stroke:#ff8f00,stroke-width:2px
    style S3 fill:#ede7f6,stroke:#4527a0,stroke-width:2px

Loading

See Architecture Overview for detailed diagrams covering data flow, service communication, and the approval pipeline.

🔄 File Processing Pipeline

After discovery, metadata extraction, fingerprinting, and analysis run as independent per-file stages (each reads only the file on disk). Metadata extraction is operator-triggered from its stage workspace in the DAG-centric console — it is no longer auto-enqueued at discovery. Proposal generation joins on analysis and metadata only.

stateDiagram-v2
    [*] --> DISCOVERED
    DISCOVERED --> METADATA_EXTRACTED : mutagen (operator‑triggered)
    DISCOVERED --> FINGERPRINTED : audfprint + panako
    DISCOVERED --> ANALYZED : essentia
    METADATA_EXTRACTED --> PROPOSAL_GENERATED : LLM via litellm
    ANALYZED --> PROPOSAL_GENERATED : LLM via litellm
    FINGERPRINTED --> DUPLICATE_RESOLVED
    PROPOSAL_GENERATED --> APPROVED : human review
    PROPOSAL_GENERATED --> REJECTED : human review
    APPROVED --> EXECUTED : copy‑verify‑delete
    APPROVED --> FAILED

Loading

The v7.0 console renders this topology as the DAG rail — the left-hand navigation spine — where each stage is a node with a live count; clicking a node swaps its workspace into the center pane over HTMX (/s/<stage>), and that workspace carries the stage's trigger (gated by its upstream dependencies and by agent/controller availability). DB-truth stage counts drive every rendered count; the rail is seeded once and kept live by a 5s /pipeline/stats poll. The three agent stages (Metadata, Analyze, Fingerprint) additionally have per-stage Pause/Resume + priority control endpoints (▲ Higher / ▼ Lower, "lower number runs first"; see below); their live pause/priority state is seeded into $store.pipeline and rides the same 5s poll. Discovery is display-only — scanning is initiated solely from the Discover workspace's Trigger Scan card (the redundant "Rescan Files" anchor was removed in Phase 38). The legacy standalone dashboard page was removed in the v7.0 cutover, so /pipeline/ now 302-redirects into the shell root /.

Recovery-only automation (Phase 42). Steady state produces zero automatic enqueues — every stage advances only on an operator click. The single exception is a gated recovery pass on controller startup: it no-ops unless it detects genuine queue-loss (empty saq_jobs while the DB still shows pending work). Because the SAQ broker is durable Postgres (Phase 36), a normal restart loses nothing, so the old every-5-min auto-advance cron was removed. A global Recover button in the DAG header (POST /pipeline/recover) runs the same idempotent producer on demand (forced cold-boot safety net); its deterministic-key dedup means a forced reconcile can never double the backlog.

📬 Task Queue Routing

Every control-plane enqueue (API endpoint or admin-UI action) routes to a named SAQ queue that a worker actually consumes — the control plane never produces onto an unnamed default queue (which has no consumer). A single chokepoint, resolve_queue_for_task in src/phaze/services/enqueue_router.py, maps each task name to its destination:

• Controller-bound tasks — generate_proposals, search_tracklist, scrape_and_store_tracklist, match_tracklist_to_discogs, refresh_tracklists — route to the controller queue, consumed by the application-server phaze-worker.
• Per-agent tasks — process_file, extract_file_metadata, fingerprint_file, scan_live_set, scan_directory, execute_approved_batch, s3_upload, push_file — route via the AgentTaskRouter to a per-lane phaze-agent-<id>-<lane> queue, consumed by the matching file-server lane worker. The target agent is chosen by active-agent selection (the most-recently-seen, non-revoked agent). When no active agent is available, the operation surfaces a clear error / empty-state instead of silently enqueuing nothing.

Per-lane agent workers (quick-260707-dh1): the file-server agent runs four lane workers from one image — analyze (process_file), fingerprint (fingerprint_file), meta (extract_file_metadata / scan_directory / scan_live_set / execute_approved_batch), and io (s3_upload / push_file) — so I/O offload and cheap analysis are never head-of-line-blocked behind CPU-bound essentia backlog. The task→lane map is the single source of truth LANE_TASKS in enqueue_router.py (with AGENT_TASKS its derived union); queue_for(agent_id, lane) requires an explicit lane (no silent default). CPU-bound lanes (analyze 4 + fingerprint 2) sum within nox's 8 cores with essentia/TF pinned single-threaded; io is off the CPU budget. Exactly one liveness heartbeat runs per agent (the analyze lane) — so the heartbeat's queue_depth is analyze-lane-only, while the dashboard's get_queue_activity (which sums all four lanes + the legacy base) is the authoritative all-lane figure. The compute (cloud/x86) agent consumes the single analyze lane (its only task is process_file). See docs/agent-queue-lanes.md for the topology table, concurrency knobs, and the legacy-queue drain runbook.

Unknown task names fail loud (ValueError) — they are never silently sent to any queue. A static guard test (tests/test_no_default_queue_producers.py) scans the router and service trees on every CI run and fails if anyone reintroduces a default-queue producer (a *.state.queue reference or an unnamed Queue.from_url(...)), so this bug class cannot regress unnoticed.

Schedule-safe by construction: every routable task is keyed deterministically as <task>:<natural_id> at a single central SAQ before_enqueue hook (apply_deterministic_key), so a re-enqueue dedups against an in-flight job instead of doubling the queue — no call site can drift back to a random-uuid key. Task DB writes upsert (ON CONFLICT DO UPDATE); proposals in particular are idempotent via a partial unique index (uq_proposals_file_id_pending) so re-runs never duplicate rows.

Operational note: Jobs stranded on the legacy default queue from before this routing fix (e.g. the saq:job:default:* keys from the v4.0.6 incident) are cleared as a one-time deploy step after redeploy — re-triggering analysis re-enqueues them correctly onto their named queues. This is an operations task, not application code.

🎚️ Per-Stage Pause & Priority

Each of the three agent pipeline stages — metadata (extract_file_metadata), analyze (process_file), and fingerprint (fingerprint_file) — can be paused, resumed, and reprioritized at runtime. The durable operator intent lives in the pipeline_stage_control table; a before_enqueue hook stamps every NEW stage job from that table, while the control endpoints additionally mutate the EXISTING queued backlog (raw saq_jobs UPDATEs) so an action takes effect immediately. The control-table read is cached for 5s (TTL), so a just-changed priority reaches newly-enqueued jobs within that bounded window.

Each endpoint mutates the control row and the live backlog in a single transaction and returns {stage, priority, paused} sourced from the control row (the durable intent — a raw saq_jobs priority UPDATE reorders the dequeue column but does not rewrite a job's serialized priority, so the control row is the source of truth for the response):

Endpoint	Action
POST /pipeline/stages/{stage}/priority	Apply a signed { "delta": int } to the stage priority. The UI steps by ±10 (10 discrete levels across 0–100); the result is clamped to [0, 100] and the new absolute value is returned. Reorders the queued backlog.
POST /pipeline/stages/{stage}/pause	Drain-pause: in-flight (active) jobs finish; the queued backlog is parked (scheduled = SENTINEL, far-future) so it fails the dequeue's now >= scheduled gate. Sets paused = true.
POST /pipeline/stages/{stage}/resume	Un-park only the pause-parked rows (sentinel-guarded, so genuine retry backoffs are preserved). Sets paused = false. Restores schedulability but not priority.

Priority semantics: priority maps directly onto SAQ's saq_jobs.priority — lower number = higher priority = dequeues sooner (no inversion). The default is 50; a DB CHECK keeps every stage inside [0, 100], well within SAQ's 0–32767 dequeue window.

Adopted defaults (locked): control-state cache TTL = 5s; pause persists across reboots and re-applies to re-enqueued jobs (the hook stamps the parked state onto restarts — Phase 32 resilience); resume un-parks only (it never restores a pre-pause priority); priority is a delta op with a default UI step of ±10.

An unknown stage returns 422 (validated against the metadata/analyze/fingerprint allowlist before any backlog filter is built). These endpoints add no app-layer auth — like the rest of /pipeline/* and the /saq UI, they sit behind the reverse proxy's internal-realm auth on the private LAN.

Per-stage controls (Phase 38): each control (Pause/Resume, plus a ▲ Higher / ▼ Lower priority step — the UI steps by ±10; ▲ decrements the number, lower runs sooner) POSTs to the endpoints above with hx-swap="none"; an Alpine @htmx:after-request handler writes the authoritative {priority, paused} from the JSON response into $store.pipeline, and the 5s /pipeline/stats poll re-pushes the live per-stage state so every refresh reconciles. The control read is degrade-safe: if pipeline_stage_control is unreadable, the store falls back to the defaults (running, priority 50) and the poll still returns 200 — it never 500s. The interactive control surface that lived on the removed v6.x DAG canvas is gone with the standalone dashboard page (CUT-02); the endpoints, durable intent, and live poll state are unchanged. Scanning is initiated solely from the Discover workspace's Trigger Scan card (the redundant "Rescan Files" anchor was removed in Phase 38).

🌟 Key Features

• 🎵 Broad Format Support: mp3, m4a, ogg, flac, wav, aiff, wma, aac, opus, plus video (mp4, mkv, avi, webm, mov) and companion files (cue, nfo, m3u)
• 🔄 Dual Fingerprinting: Landmark-based (audfprint) and tempo-robust (Panako) engines for comprehensive deduplication
• 🤖 AI Rename Proposals: LLM-generated filenames and paths with structured validation via Pydantic
• 🎧 Tracklist Integration: Automatic concert set identification from 1001Tracklists with fuzzy matching
• 👀 Approval Workflow: Every rename requires human review through the web UI
• 🔒 Safe Operations: Copy-verify-delete protocol ensures no data loss
• 📊 Full Audit Trail: Every file operation is tracked in PostgreSQL
• 🧭 DAG-Centric Console: A three-column shell where the pipeline DAG rail is the navigation spine — clicking a stage swaps the center workspace over HTMX (/s/<stage>, no tab bar), a ⌘K command palette replaces the old search tab, a header status strip shows compute/agent liveness, and a per-file record slide-in opens over any row (Agents and Audit pages reachable from the rail)
• 🗺️ Pipeline Observability: The DAG rail is the navigation spine — every stage is a live-count node whose workspace carries a dependency-gated trigger, all kept live by a 5s DB-truth poll
• ⚡ Async Processing: SAQ task queue on PostgreSQL for parallel file analysis — deterministic per-task keys and idempotent re-runs (Redis backs caching/rate-limiting only)
• 📝 Type Safety: Full type hints with strict mypy validation and Bandit security scanning

🚀 Quick Start

Prerequisites

• Docker and Docker Compose v2
• uv (Python package manager)
• just (command runner)
• Python 3.14

Setup

git clone https://github.com/SimplicityGuy/phaze.git
cd phaze
uv sync
cp .env.example .env          # Edit to configure paths and API keys
just download-models           # Required for audio analysis
just up-all                    # Start all services (core + agent stacks)
just db-upgrade                # Run database migrations
curl http://localhost:8000/health   # Verify: {"status": "ok"}

Service	URL	Default Credentials
🌐 Web UI	http://localhost:8000	None
🐘 PostgreSQL	localhost:5432	phaze / phaze
🔴 Redis	localhost:6379	None
🎵 Audfprint	audfprint:8001 (agent stack)	None
🎧 Panako	panako:8002 (agent stack)	None

Fingerprint sidecars: audfprint and panako run in the agent stack (docker-compose.agent.yml) with no published host ports. Reach them on the Docker network at audfprint:8001 / panako:8002; start them with just up-agent (agent stack only) or just up-all (both stacks on one host).

Logging: all processes log through one structlog pipeline (JSON when not a TTY, console otherwise). Tune with PHAZE_LOG_LEVEL (DEBUG|INFO|WARNING|ERROR, default INFO) and PHAZE_LOG_JSON (true|false, default auto); set PHAZE_LOG_LEVEL=DEBUG to watch a running scan or model download in detail. See Configuration → Logging / observability.

Scan activity & stall reaping: RUNNING scans show a live activity indicator (a green pulsing dot + "·Ns ago" in the Recent Scans table and the in-progress card) and flip to an amber "stalled?" warning when quiet (at half the reap window — 12h). A control-side cron auto-fails scans that make no progress for PHAZE_SCAN_STALL_SECONDS (default 86400 — 24h). See the PHAZE_SCAN_STALL_SECONDS configuration row.

Deleting a scan: terminal scans (completed / failed) carry a delete control in the Recent Scans table that removes the scan batch and every row associated with its files in one transaction (scoped strictly to that batch — no other scan's data is touched). Running scans and the live watcher sentinel cannot be deleted.

See the Quick Start Guide for prerequisites, local development setup, and environment configuration.

📖 Documentation

🏁 Getting Started

Document	Purpose
Quick Start Guide	🚀 Get Phaze running in minutes
Configuration	⚙️ Environment variables and settings reference

📐 Reference

Document	Purpose
API Reference	🔌 REST API endpoints and usage
Database Schema & Migrations	🗄️ PostgreSQL schema and Alembic migrations
Project Structure	📁 Codebase layout and module organization

See docs/README.md for the full documentation index.

👨‍💻 Development

just install          # Install dependencies
just up / just down   # Start / stop services
just test             # Run tests
just test-cov         # Tests with coverage (90% min)
just check            # Lint + typecheck + test
just pre-commit       # Run all pre-commit hooks

See just --list for the full command reference.

🧪 Running integration tests locally

The full suite needs a real PostgreSQL and Redis. just integration-test spins up self-contained, disposable containers, runs the entire suite (including tests/test_migrations/), and tears them down automatically:

just integration-test   # one-shot: start ephemeral Postgres + Redis, run full suite, clean up
just test-db            # start the ephemeral services and leave them running (iterative work)
just test-db-down       # stop and remove the ephemeral services

The ephemeral services listen on 5433 (Postgres) and 6380 (Redis) to avoid colliding with a dev database/cache on the default 5432/6379. Override the ports with PHAZE_TEST_DB_PORT and PHAZE_TEST_REDIS_PORT. The test database URLs honor the TEST_DATABASE_URL and MIGRATIONS_TEST_DATABASE_URL env vars (Redis via PHAZE_REDIS_URL); with nothing set they default to localhost:5432, matching CI.

🔍 Code Quality

• Linter/Formatter: Ruff (150-char line length, double quotes)
• Type checker: mypy (strict mode, excludes tests)
• Pre-commit hooks: ruff, bandit, mypy, shellcheck, yamllint, actionlint, jsonschema validation
• All hooks use frozen SHAs for reproducibility

🚀 CI/CD

GitHub Actions runs on every push and PR:

Job	Description
Quality	Pre-commit hooks (ruff, mypy, yamllint, etc.)
Test	pytest with PostgreSQL, coverage upload to Codecov
Security	pip-audit, bandit, Semgrep, TruffleHog, Trivy

🛠️ Technology Stack

Category	Technology	Purpose
Runtime	Python 3.14	Application runtime
Web	FastAPI + Uvicorn	Async API server
Database	PostgreSQL 18 + SQLAlchemy + asyncpg	Primary data store (async ORM)
Migrations	Alembic (async template)	Database schema management
Task Queue	SAQ on PostgreSQL (psycopg3)	Async background job processing
Cache	Redis	LLM rate-limiting + pipeline counters
Audio Tags	mutagen	Read/write audio metadata
Analysis	essentia-tensorflow	BPM, key, mood, style detection
Fingerprint	audfprint + Panako	Audio deduplication + identification
AI/LLM	litellm (pinned >=1.85.6,<1.86.0)	Unified LLM API for rename proposals (capped after the 1.82.7/1.82.8 supply-chain incident)
Scraping	BeautifulSoup4 + lxml	1001Tracklists integration
Matching	rapidfuzz	Fuzzy string matching
UI	Jinja2 + HTMX + Tailwind CSS + Alpine.js	Server-rendered interactive UI
Deploy	Docker Compose	Container orchestration

💬 Support & Community

• 🐛 Bug Reports: GitHub Issues
• 💡 Feature Requests: GitHub Discussions
• ❓ Questions: Discussions Q&A
• 📖 Full Documentation: docs/README.md

📄 License

This project is licensed under the MIT License -- see the LICENSE file for details.

🙏 Acknowledgments

• 🎼 discogsography for the CI/CD patterns, project conventions, and HTTP API integration target that shaped this project
• 🎵 Discogs and AcoustID for music identification services
• 🎧 1001Tracklists for concert tracklist data
• 🚀 uv for blazing-fast package management
• 🔥 Ruff for lightning-fast linting
• 🐍 The Python community for excellent libraries and tools

---

Made with ❤️ in the Pacific Northwest