Skip to content

CLAUDE.md

Latest commit

 

History

History
131 lines (107 loc) · 5.34 KB

CLAUDE.md

File metadata and controls

131 lines (107 loc) · 5.34 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

NeuroForge is a Python toolkit for building autonomous AI agents, voice assistants, and integrated audio-visual applications. It integrates LLMs, audio pipelines (ASR/TTS/VAD), vision models, and deployment tools for edge and cloud environments.

Development Commands

Setup

uv venv && source .venv/bin/activate

# Install core package
uv pip install -e .

# Install with specific feature groups
uv pip install -e ".[audio,text,lm,torch,langchain]"

# Sync all dependencies
uv sync

Code Quality

ruff check .          # Lint
ruff format .         # Format
black .               # Formatter (installed in venv via [test] extra)
pre-commit run --all-files  # Run all pre-commit hooks

Testing

pytest tests/harness/                    # Run all harness tests
pytest tests/harness/ -v                 # Verbose output
pytest tests/harness/ -k "test_name"     # Run tests matching pattern
pytest tests/harness/ -m unit            # Run only unit-marked tests

# With coverage (fail_under=80 gate on llm/core/ and tools/)
pytest tests/harness/ --cov=neuro_forge/llm/core --cov=neuro_forge/tools --cov-report=term-missing

# Or use Makefile shortcuts
make test        # Run tests
make test-cov    # Tests + coverage gate
make check       # lint + test-cov (mirrors CI)

Test Infrastructure

  • Location: tests/harness/ — 88 tests across 6 files
  • Shared fixtures: tests/conftest.py — mock_logger, mock_llm_client
  • Markers: unit (no I/O), integration (cross-module), slow
  • Coverage gate: 80% on neuro_forge/llm/core/ + neuro_forge/tools/ (excludes providers/, models/)
  • CI: .github/workflows/ci.yml — lint → test matrix (py3.10/3.11/3.12) → coverage gate

Configuration

  • Line length: 120 characters (.ruff.toml)
  • Python target: 3.10+ (.ruff.toml)
  • Custom PyPI indexes: Tsinghua mirror + PyPI + PyTorch CPU (see uv.toml)

Architecture

Voice Assistant Pipeline

Mic Input → Wake Word (OpenWakeWord/Porcupine)
         → VAD (Silero/WebRTC/FSMN)
         → ASR (Faster-Whisper/SenseVoice/Paraformer)
         → Language Detection (langid)
         → LLM + Tool Calling (OpenAI/DeepSeek/Qwen/Bedrock)
         → TTS (Piper/Edge-TTS/GPT-SoVITS)
         → Audio Output

Entry point: neuro_forge/apps/voice_assistant/

LLM Integration Layer

  • neuro_forge/llm/core/llm_chat.py — Core orchestrator combining LLM with tool execution
  • neuro_forge/llm/core/llm_chat_oai.py — Unified OpenAI-compatible interface (abstracts OpenAI, DeepSeek, DashScope/Qwen, Ollama)
  • neuro_forge/llm/core/llm_chat_local.py — Local model inference interface
  • neuro_forge/llm/providers/ — Provider-specific demo scripts (deepseek, qwen, aws, openai)
  • neuro_forge/llm/models/ — Local model loading scripts (bert, transformer)
  • Tool schemas follow the OpenAI function-calling format; definitions live in neuro_forge/tools/funcs.py

Computer Vision (neuro_forge/ccv/)

Git submodule: cggos/ccv — Chenguang Computer Vision library. Zero-dependency C++ core covering maths, kinematics/dynamics (rotation, quaternion, Euler), state estimation (EKF, LM, Bundle Adjustment), and 2D/3D CV algorithms. Cloned via:

git submodule update --init --recursive

Agent Tools & MCP

  • neuro_forge/tools/funcs.py — Core tool definitions with OpenAI-compatible JSON schemas
  • neuro_forge/tools/mcp/ — Model Context Protocol server/client implementations
  • neuro_forge/tools/acp/ — Agent Context Protocol implementations

Web Applications

  • neuro_forge/apps/web_fastapi_rag/ — Production RAG system with FastAPI + vector storage (Chroma)
  • neuro_forge/apps/fastapi/ — Additional FastAPI endpoints
  • neuro_forge/apps/web_flask/ — Flask-based web interface
  • neuro_forge/apps/tui/ — Textual-based TUI applications

Framework Integrations (neuro_forge/framework/)

Subdir Purpose
pytorch/ PyTorch training utilities
libtorch/ LibTorch C++ inference
onnx/ ONNX model export/inference
llama.cpp/ Quantized local model inference
rknn/ Rockchip NPU edge deployment
tensor_rt/ NVIDIA TensorRT optimization
tflite/ TFLite conversion utilities (incl. ONNX→TFLite)
vllm/ vLLM inference engine
langchain/ LangChain/LangGraph RAG pipelines

ML Learning & Research (neuro_forge/mlb/)

Subdir Purpose
ml_in_action/ 《机器学习实战》书籍代码
dl_with_pytorch/ 《Deep Learning with PyTorch》书籍代码
ml_base/ 经典 ML 算法 notebook
gml/ 图神经网络 (PyG)
rl/ 强化学习

Optional Dependency Groups (in pyproject.toml)

Key groups: torch, tf, rknn, audio, text, lm, langchain, lg, api, tui, aws, cpp, benchmark, test, docs

Key Conventions

  • Both Chinese (zh) and English (en) are first-class throughout — LLM prompts, TTS, and ASR all handle bilingual flows
  • Streaming is supported for both ASR and TTS; prefer streaming APIs where available
  • Agent identity and memory configurations live in neuro_forge/agents/
  • Shared utilities (logging, HPC helpers) are in neuro_forge/common/