- Introduction
- Project Structure
- Environment Setup
- Core Dependencies
- Clojure Implementation
- Python Research Framework
- Visualization and Analysis
- Benchmarking and Testing
- Documentation System
- Development Workflow
- Deployment and Containerization
- Research Notebooks
- Real-World Applications
- Continuous Integration
- FreeBSD Specific Optimizations
This project explores Liquid Neural Networks (LNNs), a revolutionary approach to neural computation inspired by the C. elegans nervous system. LNNs offer continuous-time dynamics, parameter efficiency, and superior interpretability compared to traditional neural architectures.
- Continuous-time neural dynamics using ODEs
- Parameter-efficient architectures (19-302 neurons for complex tasks)
- Causal learning and interpretability
- Real-time adaptation capabilities
- Edge AI deployment readiness
- Implement LTC (Liquid Time-Constant) neurons
- Develop CfC (Closed-form Continuous-time) approximations
- Create multi-modal applications (autonomous systems, time-series)
- Benchmark against traditional architectures
- Explore biological inspirations and mathematical foundations
#!/bin/sh
# Create comprehensive project structure
mkdir -p {src/{clojure/liquid_neural_networks,python/lnn_research,julia/lnn_core,rust/lnn_engine},\
docs/{papers,presentations,notebooks},\
tests/{unit,integration,benchmarks},\
examples/{autonomous_systems,time_series,robotics},\
data/{datasets,models,results},\
scripts/{setup,analysis,deployment},\
docker,\
resources/{configs,assets},\
notebooks/{research,tutorials},\
benchmarks/{performance,accuracy,memory},\
applications/{drone_control,financial_forecasting,medical_diagnosis}}
# Create key files
touch {README.md,CHANGELOG.md,LICENSE,CONTRIBUTING.md}
touch {deps.edn,project.clj,requirements.txt,Cargo.toml,Project.toml}
touch {Dockerfile,docker-compose.yml,.dockerignore}
touch {.gitignore,.github/workflows/ci.yml,Makefile}
touch {pyproject.toml,setup.py,environment.yml}
echo "✓ Project structure created"#!/bin/sh
# FreeBSD 14.3 system setup for LNN development
echo "Setting up FreeBSD environment for Liquid Neural Networks..."
# Install base development tools
sudo pkg install -y \
git \
curl \
wget \
bash \
zsh \
tmux \
emacs \
vim \
python39 \
python39-pip \
openjdk11 \
leiningen \
clojure \
rust \
cargo \
julia \
gcc \
cmake \
pkgconf \
libxml2 \
libxslt \
sqlite3 \
postgresql13-client \
redis \
graphviz \
gnuplot \
imagemagick7 \
ffmpeg \
pandoc \
texlive-base \
texlive-latex-extra
# Install scientific computing libraries
sudo pkg install -y \
py39-numpy \
py39-scipy \
py39-matplotlib \
py39-pandas \
py39-scikit-learn \
py39-jupyter \
py39-notebook \
blas \
lapack \
openblas \
fftw3 \
gsl \
hdf5
# Install ML/AI specific tools
sudo pkg install -y \
py39-torch \
py39-torchvision \
py39-tensorflow \
py39-keras \
py39-statsmodels \
py39-sympy \
py39-networkx
echo "✓ FreeBSD base system configured"#!/bin/sh
# Development environment setup
echo "Configuring development environment..."
# Python virtual environment
python3.9 -m venv venv
source venv/bin/activate
# Install Python dependencies
pip install --upgrade pip setuptools wheel
# Core ML/AI libraries
pip install \
torch torchvision torchaudio \
tensorflow tensorflow-probability \
jax jaxlib \
numpy scipy matplotlib seaborn \
pandas polars \
scikit-learn scikit-image \
plotly bokeh altair \
jupyter jupyterlab \
notebook nbconvert \
ipywidgets \
sympy \
networkx \
graph-tool \
igraph \
pyvis
# Specialized LNN libraries
pip install \
diffrax \
equinox \
neural-tangents \
dm-haiku \
flax \
optax \
chex \
jraph \
orbax \
blackjax
# Development tools
pip install \
black isort flake8 mypy \
pytest pytest-cov pytest-benchmark \
hypothesis \
pre-commit \
sphinx sphinx-rtd-theme \
mkdocs mkdocs-material \
streamlit \
gradio \
wandb \
tensorboard \
mlflow
# Create .env file
cat > .env << 'EOF'
# Environment variables for LNN development
PYTHONPATH=src/python:$PYTHONPATH
CUDA_VISIBLE_DEVICES=0
WANDB_PROJECT=liquid-neural-networks
MLFLOW_TRACKING_URI=http://localhost:5000
EOF
# Configure git hooks
pre-commit install
echo "✓ Development environment configured"{:deps
{org.clojure/clojure {:mvn/version "1.12.0"}
org.clojure/core.async {:mvn/version "1.6.681"}
org.clojure/core.matrix {:mvn/version "0.63.0"}
org.clojure/core.match {:mvn/version "1.1.0"}
org.clojure/data.csv {:mvn/version "1.1.0"}
org.clojure/data.json {:mvn/version "2.5.0"}
org.clojure/data.xml {:mvn/version "0.2.0-alpha9"}
org.clojure/tools.logging {:mvn/version "1.3.0"}
org.clojure/tools.cli {:mvn/version "1.1.230"}
;; Matrix computation
net.mikera/vectorz-clj {:mvn/version "0.48.0"}
net.mikera/core.matrix {:mvn/version "0.63.0"}
;; Scientific computing
generateme/fastmath {:mvn/version "2.4.0"}
scicloj/dtype-next {:mvn/version "10.104.0"}
scicloj/tablecloth {:mvn/version "7.029.2"}
scicloj/tech.ml {:mvn/version "7.029.2"}
;; Visualization
metasoarous/oz {:mvn/version "2.0.0-alpha5"}
scicloj/notespace {:mvn/version "4.0.0-beta7"}
;; Web and API
ring/ring-core {:mvn/version "1.12.2"}
ring/ring-jetty-adapter {:mvn/version "1.12.2"}
compojure/compojure {:mvn/version "1.7.1"}
hiccup/hiccup {:mvn/version "1.0.5"}
;; Database
com.github.seancorfield/honeysql {:mvn/version "2.6.1147"}
com.github.seancorfield/next.jdbc {:mvn/version "1.3.939"}
org.postgresql/postgresql {:mvn/version "42.7.3"}
;; Testing
lambdaisland/kaocha {:mvn/version "1.91.1392"}
lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}
org.clojure/test.check {:mvn/version "1.1.1"}
;; Development tools
djblue/portal {:mvn/version "0.57.3"}
com.bhauman/rebel-readline {:mvn/version "0.1.4"}
cider/cider-nrepl {:mvn/version "0.50.2"}
;; System integration
babashka/fs {:mvn/version "0.5.22"}
babashka/process {:mvn/version "0.5.22"}
;; Async and concurrency
manifold/manifold {:mvn/version "0.4.3"}
aleph/aleph {:mvn/version "0.8.1"}}
:aliases
{:dev {:extra-deps {org.clojure/tools.namespace {:mvn/version "1.5.0"}
org.clojure/tools.trace {:mvn/version "0.7.11"}
criterium/criterium {:mvn/version "0.4.6"}}
:jvm-opts ["-Dclojure.tools.logging.factory=clojure.tools.logging.impl/jul-factory"]}
:test {:extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}
lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}}
:main-opts ["-m" "kaocha.runner"]}
:uberjar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.1.303"}}
:exec-fn hf.depstar/uberjar
:exec-args {:aot true
:jar "target/liquid-neural-networks.jar"
:main-class liquid-neural-networks.core}}
:nrepl {:extra-deps {nrepl/nrepl {:mvn/version "1.3.0"}
cider/cider-nrepl {:mvn/version "0.50.2"}}
:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}
:benchmark {:extra-deps {criterium/criterium {:mvn/version "0.4.6"}
com.clojure-goes-fast/clj-async-profiler {:mvn/version "1.2.2"}}
:jvm-opts ["-Djdk.attach.allowAttachSelf"]}}}# Core ML/AI Libraries
torch>=2.1.0
torchvision>=0.16.0
torchaudio>=2.1.0
tensorflow>=2.15.0
tensorflow-probability>=0.23.0
jax>=0.4.20
jaxlib>=0.4.20
equinox>=0.11.0
diffrax>=0.5.0
neural-tangents>=0.6.0
dm-haiku>=0.0.10
flax>=0.8.0
optax>=0.1.7
chex>=0.1.85
orbax>=0.1.7
# Scientific Computing
numpy>=1.24.0
scipy>=1.11.0
pandas>=2.1.0
polars>=0.20.0
scikit-learn>=1.3.0
scikit-image>=0.22.0
statsmodels>=0.14.0
sympy>=1.12.0
networkx>=3.2.0
igraph>=0.10.0
# Visualization
matplotlib>=3.7.0
seaborn>=0.12.0
plotly>=5.17.0
bokeh>=3.3.0
altair>=5.1.0
pyvis>=0.3.0
# Jupyter and Interactive
jupyter>=1.0.0
jupyterlab>=4.0.0
notebook>=7.0.0
nbconvert>=7.0.0
ipywidgets>=8.1.0
ipyparallel>=8.6.0
# Development Tools
black>=23.0.0
isort>=5.12.0
flake8>=6.0.0
mypy>=1.7.0
pytest>=7.4.0
pytest-cov>=4.1.0
pytest-benchmark>=4.0.0
hypothesis>=6.88.0
pre-commit>=3.5.0
# Documentation
sphinx>=7.2.0
sphinx-rtd-theme>=1.3.0
mkdocs>=1.5.0
mkdocs-material>=9.4.0
myst-parser>=2.0.0
# MLOps and Monitoring
wandb>=0.16.0
mlflow>=2.8.0
tensorboard>=2.15.0
streamlit>=1.28.0
gradio>=4.0.0
# Data Processing
h5py>=3.9.0
zarr>=2.16.0
xarray>=2023.10.0
dask>=2023.10.0
joblib>=1.3.0
# Optimization and Numerics
cvxpy>=1.4.0
casadi>=3.6.0
pyomo>=6.7.0
or-tools>=9.8.0
# Time Series
tsfresh>=0.20.0
tslearn>=0.6.0
sktime>=0.24.0
prophet>=1.1.0
neuralprophet>=0.7.0
# Biology and Neuroscience
brian2>=2.5.0
nengo>=3.2.0
neuron>=8.2.0
elephant>=0.14.0
neo>=0.13.0
# Graph Neural Networks
torch-geometric>=2.4.0
dgl>=1.1.0
spektral>=1.2.0
graph-nets>=1.1.0
# Reinforcement Learning
stable-baselines3>=2.2.0
gymnasium>=0.29.0
pettingzoo>=1.24.0
ray[rllib]>=2.8.0(ns liquid-neural-networks.core
"Core implementation of Liquid Neural Networks in Clojure"
(:require [clojure.core.matrix :as m]
[clojure.core.matrix.operators :as mop]
[clojure.core.matrix.linear :as ml]
[clojure.core.matrix.stats :as ms]
[clojure.tools.logging :as log]
[clojure.spec.alpha :as s]
[clojure.test.check.generators :as gen]
[criterium.core :as criterium]
[fastmath.core :as fm]
[fastmath.random :as fr]
[tablecloth.api :as tc]))
;; Configure matrix implementation
(m/set-current-implementation :vectorz)
;; =============================================================================
;; Specifications and Validation
;; =============================================================================
(s/def ::positive-number (s/and number? pos?))
(s/def ::non-negative-number (s/and number? (complement neg?)))
(s/def ::weight number?)
(s/def ::bias number?)
(s/def ::tau ::positive-number)
(s/def ::dt ::positive-number)
(s/def ::hidden-state (s/coll-of number?))
(s/def ::input-vector (s/coll-of number?))
;; =============================================================================
;; Mathematical Functions and Utilities
;; =============================================================================
(defn sigmoid
"Sigmoid activation function with numerical stability"
[x]
(let [x (max (min x 500) -500)] ; Clamp to prevent overflow
(/ 1.0 (+ 1.0 (Math/exp (- x))))))
(defn tanh-stable
"Numerically stable tanh implementation"
[x]
(let [x (max (min x 500) -500)]
(Math/tanh x)))
(defn relu
"Rectified Linear Unit"
[x]
(max 0.0 x))
(defn leaky-relu
"Leaky ReLU with configurable slope"
([x] (leaky-relu x 0.01))
([x alpha]
(if (pos? x) x (* alpha x))))
(defn swish
"Swish activation function"
[x]
(* x (sigmoid x)))
(defn gelu
"Gaussian Error Linear Unit (GELU) approximation"
[x]
(* 0.5 x (+ 1.0 (Math/tanh (* (Math/sqrt (/ 2.0 Math/PI))
(+ x (* 0.044715 (Math/pow x 3))))))))
(defn softmax
"Softmax function for vector"
[v]
(let [exp-v (m/emap #(Math/exp (- % (apply max v))) v)
sum-exp (reduce + exp-v)]
(m/div exp-v sum-exp)))
;; =============================================================================
;; Liquid Time-Constant (LTC) Neuron Implementation
;; =============================================================================
(defprotocol LiquidNeuron
"Protocol for liquid neuron implementations"
(forward [this hidden-state input dt] "Compute next hidden state")
(backward [this hidden-state input target dt] "Compute gradients")
(get-params [this] "Get neuron parameters")
(set-params [this params] "Set neuron parameters")
(reset-state [this] "Reset internal state"))
(defrecord LTCNeuron [id weights bias tau A beta activation-fn
noise-level learning-rate momentum
weight-gradients bias-gradients]
LiquidNeuron
(forward [this hidden-state input dt]
(let [f-val (compute-f-function this hidden-state input)
noise (when (pos? noise-level)
(* noise-level (fr/grand)))
effective-tau (/ tau (+ 1.0 (* beta (Math/abs f-val))))
decay-term (/ hidden-state effective-tau)
drive-term (* f-val A)
derivative (+ (- drive-term decay-term) (or noise 0.0))
new-state (+ hidden-state (* dt derivative))]
(max (min new-state 10.0) -10.0))) ; Bounded stability
(backward [this hidden-state input target dt]
(let [prediction (forward this hidden-state input dt)
error (- target prediction)
h 1e-6
; Compute gradients using finite differences
weight-grad (compute-weight-gradient this hidden-state input target dt h)
bias-grad (compute-bias-gradient this hidden-state input target dt h)
tau-grad (compute-tau-gradient this hidden-state input target dt h)]
{:error error
:prediction prediction
:gradients {:weight weight-grad
:bias bias-grad
:tau tau-grad}}))
(get-params [this]
{:weights weights :bias bias :tau tau :A A :beta beta})
(set-params [this params]
(merge this params))
(reset-state [this]
(assoc this :weight-gradients 0.0 :bias-gradients 0.0)))
(defn compute-f-function
"Enhanced f-function with configurable activation"
[neuron hidden-state input]
(let [{:keys [weights bias activation-fn]} neuron
combined-input (+ (* weights (first input)) (* bias hidden-state))]
(activation-fn combined-input)))
(defn compute-weight-gradient
"Compute gradient w.r.t. weights using finite differences"
[neuron hidden-state input target dt h]
(let [original-weight (:weights neuron)
neuron-plus (assoc neuron :weights (+ original-weight h))
neuron-minus (assoc neuron :weights (- original-weight h))
pred-plus (forward neuron-plus hidden-state input dt)
pred-minus (forward neuron-minus hidden-state input dt)
error (- target (forward neuron hidden-state input dt))]
(* error (/ (- pred-plus pred-minus) (* 2 h)))))
(defn compute-bias-gradient
"Compute gradient w.r.t. bias using finite differences"
[neuron hidden-state input target dt h]
(let [original-bias (:bias neuron)
neuron-plus (assoc neuron :bias (+ original-bias h))
neuron-minus (assoc neuron :bias (- original-bias h))
pred-plus (forward neuron-plus hidden-state input dt)
pred-minus (forward neuron-minus hidden-state input dt)
error (- target (forward neuron hidden-state input dt))]
(* error (/ (- pred-plus pred-minus) (* 2 h)))))
(defn compute-tau-gradient
"Compute gradient w.r.t. tau using finite differences"
[neuron hidden-state input target dt h]
(let [original-tau (:tau neuron)
neuron-plus (assoc neuron :tau (+ original-tau h))
neuron-minus (assoc neuron :tau (- original-tau h))
pred-plus (forward neuron-plus hidden-state input dt)
pred-minus (forward neuron-minus hidden-state input dt)
error (- target (forward neuron hidden-state input dt))]
(* error (/ (- pred-plus pred-minus) (* 2 h)))))
(defn create-ltc-neuron
"Create a new LTC neuron with specified parameters"
[id input-size & {:keys [tau A beta activation-fn noise-level learning-rate momentum]
:or {tau 1.0 A 1.0 beta 0.1 activation-fn tanh-stable
noise-level 0.0 learning-rate 0.01 momentum 0.9}}]
(->LTCNeuron id
(fr/grand) ; Random weight
(* 0.1 (fr/grand)) ; Small random bias
tau A beta activation-fn noise-level learning-rate momentum
0.0 0.0)) ; Initialize gradients
;; =============================================================================
;; Closed-Form Continuous-Time (CfC) Implementation
;; =============================================================================
(defrecord CfCNeuron [id weights bias tau A beta activation-fn
noise-level learning-rate]
LiquidNeuron
(forward [this hidden-state input dt]
(let [f-val (compute-f-function this hidden-state input)
effective-tau (/ tau (+ 1.0 (* beta (Math/abs f-val))))
decay-factor (Math/exp (- (/ dt effective-tau)))
target-state (* f-val A)
noise (when (pos? noise-level)
(* noise-level (fr/grand)))
new-state (+ (* decay-factor hidden-state)
(* (- 1.0 decay-factor) target-state)
(or noise 0.0))]
(max (min new-state 10.0) -10.0)))
(backward [this hidden-state input target dt]
(let [prediction (forward this hidden-state input dt)
error (- target prediction)]
{:error error
:prediction prediction}))
(get-params [this]
{:weights weights :bias bias :tau tau :A A :beta beta})
(set-params [this params]
(merge this params))
(reset-state [this]
this))
(defn create-cfc-neuron
"Create a new CfC neuron with specified parameters"
[id input-size & {:keys [tau A beta activation-fn noise-level learning-rate]
:or {tau 1.0 A 1.0 beta 0.1 activation-fn tanh-stable
noise-level 0.0 learning-rate 0.01}}]
(->CfCNeuron id
(fr/grand)
(* 0.1 (fr/grand))
tau A beta activation-fn noise-level learning-rate))
;; =============================================================================
;; Multi-Layer Liquid Neural Network
;; =============================================================================
(defrecord LiquidNetwork [layers connectivity-matrix global-params])
(defn create-liquid-network
"Create a multi-layer liquid neural network"
[layer-configs]
(let [layers (mapv (fn [config]
(let [{:keys [size type neuron-params]} config
neuron-fn (case type
:ltc create-ltc-neuron
:cfc create-cfc-neuron)]
(mapv #(apply neuron-fn % 1 (flatten (seq neuron-params)))
(range size))))
layer-configs)
connectivity (create-connectivity-matrix layers)
global-params {:learning-rate 0.01
:momentum 0.9
:weight-decay 0.001
:batch-size 32}]
(->LiquidNetwork layers connectivity global-params)))
(defn create-connectivity-matrix
"Create connectivity matrix for network layers"
[layers]
(let [total-neurons (reduce + (map count layers))
matrix (m/zero-matrix total-neurons total-neurons)]
; For now, create simple feed-forward connections
; TODO: Implement sparse, recurrent, and custom connectivity patterns
matrix))
(defn forward-pass
"Forward pass through the entire network"
[network input dt]
(let [layers (:layers network)
results (atom [])]
(reduce (fn [current-input layer]
(let [layer-outputs (mapv (fn [neuron hidden-state]
(forward neuron hidden-state current-input dt))
layer
(or (last @results) (repeat (count layer) 0.0)))]
(swap! results conj layer-outputs)
layer-outputs))
input
layers)
@results))
;; =============================================================================
;; Training and Optimization
;; =============================================================================
(defn compute-loss
"Compute loss function (MSE for regression, cross-entropy for classification)"
[predictions targets loss-type]
(case loss-type
:mse (/ (reduce + (map #(Math/pow (- %1 %2) 2) predictions targets))
(count predictions))
:mae (/ (reduce + (map #(Math/abs (- %1 %2)) predictions targets))
(count predictions))
:cross-entropy (- (reduce + (map #(* %1 (Math/log (+ %2 1e-15)))
targets predictions)))))
(defn sgd-update
"Stochastic Gradient Descent parameter update"
[param gradient learning-rate weight-decay]
(- param (* learning-rate (+ gradient (* weight-decay param)))))
(defn adam-update
"Adam optimizer parameter update"
[param gradient m v t learning-rate beta1 beta2 epsilon]
(let [m-new (+ (* beta1 m) (* (- 1 beta1) gradient))
v-new (+ (* beta2 v) (* (- 1 beta2) (* gradient gradient)))
m-hat (/ m-new (- 1 (Math/pow beta1 t)))
v-hat (/ v-new (- 1 (Math/pow beta2 t)))
param-new (- param (* learning-rate (/ m-hat (+ (Math/sqrt v-hat) epsilon))))]
{:param param-new :m m-new :v v-new}))
(defn train-network
"Train the liquid neural network on a dataset"
[network training-data epochs dt & {:keys [optimizer loss-type batch-size]
:or {optimizer :adam loss-type :mse batch-size 32}}]
(let [losses (atom [])]
(dotimes [epoch epochs]
(let [epoch-loss (atom 0)
batches (partition batch-size training-data)]
(doseq [batch batches]
(let [batch-loss (atom 0)]
(doseq [{:keys [input target]} batch]
(let [predictions (forward-pass network input dt)
loss (compute-loss (last predictions) target loss-type)]
(swap! batch-loss + loss)
; TODO: Implement backpropagation and parameter updates
))
(swap! epoch-loss + (/ @batch-loss (count batch)))))
(let [avg-loss (/ @epoch-loss (count batches))]
(swap! losses conj avg-loss)
(when (zero? (mod epoch 10))
(log/info (format "Epoch %d: Loss = %.6f" epoch avg-loss))))))
{:network network :losses @losses}))
;; =============================================================================
;; Benchmarking and Analysis
;; =============================================================================
(defn benchmark-network
"Benchmark network performance"
[network test-data dt]
(let [start-time (System/nanoTime)
results (mapv (fn [data]
(let [prediction (forward-pass network (:input data) dt)]
{:prediction prediction
:target (:target data)
:error (compute-loss (last prediction) (:target data) :mse)}))
test-data)
end-time (System/nanoTime)
total-time (/ (- end-time start-time) 1e9)
avg-error (/ (reduce + (map :error results)) (count results))]
{:total-time total-time
:avg-time-per-sample (/ total-time (count test-data))
:avg-error avg-error
:throughput (/ (count test-data) total-time)
:results results}))
(defn analyze-network-dynamics
"Analyze the dynamical properties of the network"
[network test-inputs dt]
(let [stability-metrics (atom [])
trajectory-data (atom [])]
(doseq [input test-inputs]
(let [trajectory (atom [])
current-state (repeat (count (first (:layers network))) 0.0)]
; Simulate trajectory
(loop [t 0.0
state current-state]
(when (< t 10.0) ; Simulate for 10 time units
(let [next-state (forward-pass network input dt)]
(swap! trajectory conj {:time t :state state :input input})
(recur (+ t dt) (last next-state)))))
; Analyze stability
(let [traj @trajectory
state-norms (map #(Math/sqrt (reduce + (map * (:state %) (:state %)))) traj)
max-norm (apply max state-norms)
min-norm (apply min state-norms)
stability-score (if (< max-norm 50.0) :stable :unstable)]
(swap! stability-metrics conj {:input input
:max-norm max-norm
:min-norm min-norm
:stability stability-score})
(swap! trajectory-data conj traj))))
{:stability-metrics @stability-metrics
:trajectory-data @trajectory-data
:overall-stability (if (every? #(= (:stability %) :stable) @stability-metrics)
:stable :unstable)}))
;; =============================================================================
;; Utility Functions
;; =============================================================================
(defn save-network
"Save network to file"
[network filename]
(spit filename (pr-str network))
(log/info (format "Network saved to %s" filename)))
(defn load-network
"Load network from file"
[filename]
(let [network (read-string (slurp filename))]
(log/info (format "Network loaded from %s" filename))
network))
(defn network-summary
"Generate a summary of the network architecture"
[network]
(let [layers (:layers network)
total-neurons (reduce + (map count layers))
total-params (reduce + (map (fn [layer]
(reduce + (map (fn [neuron]
(count (get-params neuron)))
layer)))
layers))]
{:total-layers (count layers)
:neurons-per-layer (mapv count layers)
:total-neurons total-neurons
:total-parameters total-params
:connectivity-type "Feed-forward" ; TODO: Make this dynamic
:global-params (:global-params network)}))
(defn -main
"Main entry point for the application"
[& args]
(log/info "Starting Liquid Neural Networks application...")
; Example usage
(let [network-config [{:size 4 :type :ltc :neuron-params {:tau 2.0 :A 1.0}}
{:size 2 :type :cfc :neuron-params {:tau 1.5 :A 0.8}}
{:size 1 :type :ltc :neuron-params {:tau 1.0 :A 1.0}}]
network (create-liquid-network network-config)
test-input [0.5 0.3 0.8 0.2]
dt 0.1]
(log/info "Network created:" (network-summary network))
(let [result (forward-pass network test-input dt)]
(log/info "Forward pass result:" result))
(let [benchmark-data [{:input [0.1 0.2 0.3 0.4] :target [0.5]}
{:input [0.2 0.3 0.4 0.5] :target [0.6]}
{:input [0.3 0.4 0.5 0.6] :target [0.7]}]
benchmark-result (benchmark-network network benchmark-data dt)]
(log/info "Benchmark results:" benchmark-result))
(log/info "Application completed successfully.")))(ns liquid-neural-networks.applications
"Real-world applications of Liquid Neural Networks"
(:require [liquid-neural-networks.core :as lnn]
[clojure.core.matrix :as m]
[clojure.tools.logging :as log]
[tablecloth.api :as tc]
[fastmath.random :as fr]))
;; =============================================================================
;; Autonomous Systems Application
;; =============================================================================
(defrecord AutonomousController [lnn sensor-config actuator-config
control-history decision-threshold])
(defn create-autonomous-controller
"Create an autonomous control system using LNN"
[sensor-count actuator-count]
(let [network-config [{:size 8 :type :ltc :neuron-params {:tau 2.0 :A 1.0}}
{:size 4 :type :cfc :neuron-params {:tau 1.5 :A 0.8}}
{:size actuator-count :type :ltc :neuron-params {:tau 1.0 :A 1.0}}]
network (lnn/create-liquid-network network-config)]
(->AutonomousController network
{:sensor-count sensor-count
:sensor-types [:distance :speed :angle :obstacle]}
{:actuator-count actuator-count
:actuator-types [:steering :throttle]}
[]
0.5)))
(defn process-sensor-data
"Process sensor data and generate control commands"
[controller sensor-data dt]
(let [normalized-data (mapv #(/ % 255.0) sensor-data)
control-output (lnn/forward-pass (:lnn controller) normalized-data dt)
decision (if (> (first (last control-output)) (:decision-threshold controller))
:action-required
:maintain-state)
control-command {:timestamp (System/currentTimeMillis)
:sensor-data sensor-data
:control-output (last control-output)
:decision decision}]
(-> controller
(update :control-history conj control-command))))
;; =============================================================================
;; Time Series Forecasting Application
;; =============================================================================
(defrecord TimeSeriesPredictor [lnn lookback-window prediction-horizon
normalization-params])
(defn create-time-series-predictor
"Create a time series forecasting system"
[lookback-window prediction-horizon]
(let [network-config [{:size 12 :type :ltc :neuron-params {:tau 3.0 :A 1.2}}
{:size 8 :type :cfc :neuron-params {:tau 2.0 :A 1.0}}
{:size prediction-horizon :type :ltc :neuron-params {:tau 1.0 :A 1.0}}]
network (lnn/create-liquid-network network-config)]
(->TimeSeriesPredictor network lookback-window prediction-horizon {})))
(defn normalize-time-series
"Normalize time series data"
[data]
(let [mean-val (/ (reduce + data) (count data))
std-val (Math/sqrt (/ (reduce + (map #(Math/pow (- % mean-val) 2) data))
(count data)))]
{:normalized (mapv #(/ (- % mean-val) std-val) data)
:mean mean-val
:std std-val}))
(defn denormalize-predictions
"Denormalize predictions back to original scale"
[predictions mean-val std-val]
(mapv #(+ (* % std-val) mean-val) predictions))
(defn predict-time-series
"Predict future values of time series"
[predictor time-series dt]
(let [norm-result (normalize-time-series time-series)
normalized-data (:normalized norm-result)
sequences (partition (:lookback-window predictor) 1 normalized-data)
predictions (atom [])]
(doseq [sequence sequences]
(let [prediction (lnn/forward-pass (:lnn predictor) (vec sequence) dt)]
(swap! predictions conj (last prediction))))
{:predictions (denormalize-predictions @predictions
(:mean norm-result)
(:std norm-result))
:normalization-params norm-result}))
;; =============================================================================
;; Medical Diagnosis Application
;; =============================================================================
(defrecord MedicalDiagnosisSystem [lnn symptom-encoder diagnosis-decoder
confidence-threshold])
(defn create-medical-diagnosis-system
"Create a medical diagnosis system using LNN"
[symptom-count diagnosis-count]
(let [network-config [{:size 16 :type :ltc :neuron-params {:tau 2.5 :A 1.1}}
{:size 12 :type :cfc :neuron-params {:tau 2.0 :A 1.0}}
{:size 8 :type :ltc :neuron-params {:tau 1.5 :A 0.9}}
{:size diagnosis-count :type :cfc :neuron-params {:tau 1.0 :A 1.0}}]
network (lnn/create-liquid-network network-config)]
(->MedicalDiagnosisSystem network {} {} 0.7)))
(defn encode-symptoms
"Encode patient symptoms into numerical format"
[symptoms symptom-mapping]
(mapv #(get symptom-mapping % 0.0) symptoms))
(defn decode-diagnosis
"Decode network output to diagnosis probabilities"
[output diagnosis-mapping]
(let [probs (lnn/softmax output)]
(mapv (fn [prob diagnosis]
{:diagnosis diagnosis :probability prob})
probs diagnosis-mapping)))
(defn diagnose-patient
"Diagnose patient based on symptoms"
[system symptoms dt]
(let [encoded-symptoms (encode-symptoms symptoms (:symptom-encoder system))
output (lnn/forward-pass (:lnn system) encoded-symptoms dt)
diagnosis-probs (decode-diagnosis (last output) (:diagnosis-decoder system))
high-confidence (filter #(> (:probability %) (:confidence-threshold system))
diagnosis-probs)]
{:all-diagnoses diagnosis-probs
:high-confidence-diagnoses high-confidence
:timestamp (System/currentTimeMillis)}))
;; =============================================================================
;; Robotics Control Application
;; =============================================================================
(defrecord RobotController [lnn joint-config sensor-config trajectory-planner])
(defn create-robot-controller
"Create a robot control system"
[joint-count sensor-count]
(let [network-config [{:size 20 :type :ltc :neuron-params {:tau 1.5 :A 1.0}}
{:size 16 :type :cfc :neuron-params {:tau 1.2 :A 0.9}}
{:size 12 :type :ltc :neuron-params {:tau 1.0 :A 0.8}}
{:size joint-count :type :cfc :neuron-params {:tau 0.8 :A 1.0}}]
network (lnn/create-liquid-network network-config)]
(->RobotController network
{:joint-count joint-count
:joint-limits [[-180 180] [-90 90] [-180 180]]} ; Example limits
{:sensor-count sensor-count
:sensor-types [:position :velocity :force :torque]}
{})))
(defn compute-joint-commands
"Compute joint commands for robot"
[controller sensor-data target-pose dt]
(let [state-vector (concat sensor-data target-pose)
joint-outputs (lnn/forward-pass (:lnn controller) state-vector dt)
joint-commands (mapv (fn [output joint-limits]
(let [[min-val max-val] joint-limits
scaled-output (+ min-val (* (+ output 1.0) 0.5 (- max-val min-val)))]
(max min-val (min max-val scaled-output))))
(last joint-outputs)
(get-in controller [:joint-config :joint-limits]))]
{:joint-commands joint-commands
:timestamp (System/currentTimeMillis)
:sensor-data sensor-data
:target-pose target-pose}))
;; =============================================================================
;; Financial Forecasting Application
;; =============================================================================
(defrecord FinancialPredictor [lnn feature-extractors risk-assessor
market-indicators])
(defn create-financial-predictor
"Create a financial forecasting system"
[feature-count prediction-horizon]
(let [network-config [{:size 24 :type :ltc :neuron-params {:tau 4.0 :A 1.2}}
{:size 16 :type :cfc :neuron-params {:tau 3.0 :A 1.1}}
{:size 12 :type :ltc :neuron-params {:tau 2.0 :A 1.0}}
{:size prediction-horizon :type :cfc :neuron-params {:tau 1.0 :A 1.0}}]
network (lnn/create-liquid-network network-config)]
(->FinancialPredictor network {} {} {})))
(defn extract-financial-features
"Extract features from financial data"
[price-data volume-data indicators]
(let [returns (mapv (fn [p1 p2] (/ (- p2 p1) p1))
price-data (rest price-data))
volatility (lnn/ms/variance returns)
moving-avg (/ (reduce + (take-last 10 price-data)) 10)
volume-avg (/ (reduce + (take-last 10 volume-data)) 10)]
(concat returns [volatility moving-avg volume-avg] indicators)))
(defn predict-financial-movement
"Predict financial market movement"
[predictor market-data dt]
(let [features (extract-financial-features (:prices market-data)
(:volumes market-data)
(:indicators market-data))
predictions (lnn/forward-pass (:lnn predictor) features dt)
movement-probs (lnn/softmax (last predictions))
direction (if (> (first movement-probs) 0.5) :up :down)
confidence (Math/abs (- (first movement-probs) 0.5))]
{:predictions (last predictions)
:movement-probabilities movement-probs
:predicted-direction direction
:confidence confidence
:timestamp (System/currentTimeMillis)}))
;; =============================================================================
;; Application Factory and Utilities
;; =============================================================================
(defn create-application
"Factory function to create different types of applications"
[app-type config]
(case app-type
:autonomous-control (create-autonomous-controller (:sensor-count config)
(:actuator-count config))
:time-series (create-time-series-predictor (:lookback-window config)
(:prediction-horizon config))
:medical-diagnosis (create-medical-diagnosis-system (:symptom-count config)
(:diagnosis-count config))
:robot-control (create-robot-controller (:joint-count config)
(:sensor-count config))
:financial-forecasting (create-financial-predictor (:feature-count config)
(:prediction-horizon config))
(throw (ex-info "Unknown application type" {:type app-type}))))
(defn benchmark-application
"Benchmark application performance"
[app test-data dt]
(let [start-time (System/nanoTime)
results (case (type app)
AutonomousController
(mapv #(process-sensor-data app (:sensor-data %) dt) test-data)
TimeSeriesPredictor
(mapv #(predict-time-series app (:time-series %) dt) test-data)
MedicalDiagnosisSystem
(mapv #(diagnose-patient app (:symptoms %) dt) test-data)
RobotController
(mapv #(compute-joint-commands app (:sensor-data %) (:target-pose %) dt) test-data)
FinancialPredictor
(mapv #(predict-financial-movement app (:market-data %) dt) test-data))
end-time (System/nanoTime)
total-time (/ (- end-time start-time) 1e9)]
{:total-time total-time
:avg-time-per-sample (/ total-time (count test-data))
:throughput (/ (count test-data) total-time)
:results results}))"""
Advanced Python Research Framework for Liquid Neural Networks
Comprehensive implementation with JAX, PyTorch, and TensorFlow backends
"""
import jax
import jax.numpy as jnp
from jax import grad, jit, vmap, random, lax
import equinox as eqx
import diffrax
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import tensorflow as tf
from typing import Dict, List, Tuple, Optional, Callable, Any
import dataclasses
from dataclasses import dataclass
from abc import ABC, abstractmethod
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.graph_objects as go
import plotly.express as px
from plotly.subplots import make_subplots
import pandas as pd
import logging
from pathlib import Path
import json
import pickle
import yaml
from tqdm import tqdm
import wandb
import mlflow
import optuna
from scipy.integrate import odeint, solve_ivp
from scipy.optimize import minimize
import networkx as nx
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import warnings
warnings.filterwarnings('ignore')
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
# =============================================================================
# Core Mathematical Functions
# =============================================================================
def sigmoid(x):
"""Numerically stable sigmoid function"""
return jnp.where(x >= 0,
1 / (1 + jnp.exp(-x)),
jnp.exp(x) / (1 + jnp.exp(x)))
def tanh_stable(x):
"""Numerically stable tanh function"""
return jnp.tanh(jnp.clip(x, -500, 500))
def swish(x):
"""Swish activation function"""
return x * sigmoid(x)
def gelu(x):
"""GELU activation function"""
return 0.5 * x * (1 + jnp.tanh(jnp.sqrt(2 / jnp.pi) * (x + 0.044715 * x**3)))
def leaky_relu(x, alpha=0.01):
"""Leaky ReLU activation"""
return jnp.where(x > 0, x, alpha * x)
# =============================================================================
# JAX-based LNN Implementation
# =============================================================================
@dataclass
class LTCConfig:
"""Configuration for LTC neurons"""
input_size: int
hidden_size: int
output_size: int
tau: float = 1.0
A: float = 1.0
beta: float = 0.1
activation: str = 'tanh'
noise_level: float = 0.0
dt: float = 0.1
class LTCNeuron(eqx.Module):
"""JAX-based LTC Neuron implementation using Equinox"""
weights: jnp.ndarray
bias: jnp.ndarray
tau: float
A: float
beta: float
activation_fn: Callable
noise_level: float
def __init__(self, config: LTCConfig, key: random.PRNGKey):
self.tau = config.tau
self.A = config.A
self.beta = config.beta
self.noise_level = config.noise_level
# Initialize weights
w_key, b_key = random.split(key)
self.weights = random.normal(w_key, (config.hidden_size, config.input_size)) * 0.1
self.bias = random.normal(b_key, (config.hidden_size,)) * 0.01
# Set activation function
activation_map = {
'tanh': tanh_stable,
'sigmoid': sigmoid,
'swish': swish,
'gelu': gelu,
'leaky_relu': leaky_relu
}
self.activation_fn = activation_map[config.activation]
def __call__(self, hidden_state, input_data, dt, key=None):
"""Forward pass through LTC neuron"""
# Compute f-function
f_val = self.activation_fn(self.weights @ input_data + self.bias * hidden_state)
# Add noise if specified
if self.noise_level > 0.0 and key is not None:
noise = random.normal(key, hidden_state.shape) * self.noise_level
f_val = f_val + noise
# Compute effective time constant
effective_tau = self.tau / (1.0 + self.beta * jnp.abs(f_val))
# LTC dynamics: dx/dt = -x/tau_eff + f*A
derivative = -hidden_state / effective_tau + f_val * self.A
# Euler integration
new_state = hidden_state + dt * derivative
# Bounded stability
return jnp.clip(new_state, -10.0, 10.0)
class CfCNeuron(eqx.Module):
"""Closed-form Continuous-time Neuron implementation"""
weights: jnp.ndarray
bias: jnp.ndarray
tau: float
A: float
beta: float
activation_fn: Callable
def __init__(self, config: LTCConfig, key: random.PRNGKey):
self.tau = config.tau
self.A = config.A
self.beta = config.beta
# Initialize weights
w_key, b_key = random.split(key)
self.weights = random.normal(w_key, (config.hidden_size, config.input_size)) * 0.1
self.bias = random.normal(b_key, (config.hidden_size,)) * 0.01
# Set activation function
activation_map = {
'tanh': tanh_stable,
'sigmoid': sigmoid,
'swish': swish,
'gelu': gelu,
'leaky_relu': leaky_relu
}
self.activation_fn = activation_map[config.activation]
def __call__(self, hidden_state, input_data, dt, key=None):
"""Forward pass with closed-form solution"""
# Compute f-function
f_val = self.activation_fn(self.weights @ input_data + self.bias * hidden_state)
# Compute effective time constant
effective_tau = self.tau / (1.0 + self.beta * jnp.abs(f_val))
# Closed-form solution
decay_factor = jnp.exp(-dt / effective_tau)
target_state = f_val * self.A
new_state = decay_factor * hidden_state + (1 - decay_factor) * target_state
return jnp.clip(new_state, -10.0, 10.0)
class LiquidNeuralNetwork(eqx.Module):
"""Multi-layer Liquid Neural Network"""
layers: List[eqx.Module]
output_projection: eqx.nn.Linear
def __init__(self, layer_configs: List[LTCConfig], key: random.PRNGKey):
keys = random.split(key, len(layer_configs) + 1)
self.layers = []
for i, config in enumerate(layer_configs):
if config.activation == 'cfc':
layer = CfCNeuron(config, keys[i])
else:
layer = LTCNeuron(config, keys[i])
self.layers.append(layer)
# Output projection layer
final_config = layer_configs[-1]
self.output_projection = eqx.nn.Linear(
final_config.hidden_size,
final_config.output_size,
key=keys[-1]
)
def __call__(self, inputs, dt, key=None):
"""Forward pass through the network"""
if key is not None:
keys = random.split(key, len(self.layers))
else:
keys = [None] * len(self.layers)
# Initialize hidden states
hidden_states = [jnp.zeros((layer.weights.shape[0],)) for layer in self.layers]
outputs = []
for i, input_data in enumerate(inputs):
# Process through each layer
for j, (layer, hidden_state) in enumerate(zip(self.layers, hidden_states)):
if j == 0:
layer_input = input_data
else:
layer_input = hidden_states[j-1]
hidden_states[j] = layer(hidden_state, layer_input, dt, keys[j])
# Project to output space
output = self.output_projection(hidden_states[-1])
outputs.append(output)
return jnp.array(outputs), hidden_states
# =============================================================================
# Training and Optimization
# =============================================================================
class LNNTrainer:
"""Advanced training framework for LNNs"""
def __init__(self, model: LiquidNeuralNetwork, config: Dict):
self.model = model
self.config = config
self.optimizer = None
self.loss_history = []
self.metrics_history = []
def setup_optimizer(self):
"""Setup optimizer (Adam, SGD, etc.)"""
import optax
optimizer_name = self.config.get('optimizer', 'adam')
learning_rate = self.config.get('learning_rate', 0.001)
if optimizer_name == 'adam':
self.optimizer = optax.adam(learning_rate)
elif optimizer_name == 'sgd':
self.optimizer = optax.sgd(learning_rate)
elif optimizer_name == 'rmsprop':
self.optimizer = optax.rmsprop(learning_rate)
else:
raise ValueError(f"Unknown optimizer: {optimizer_name}")
def loss_fn(self, params, batch, key):
"""Compute loss function"""
model = eqx.tree_at(lambda m: m, self.model, params)
inputs, targets = batch
outputs, _ = model(inputs, self.config['dt'], key)
# Mean squared error loss
loss = jnp.mean((outputs - targets) ** 2)
# Add regularization
if self.config.get('l2_reg', 0.0) > 0:
l2_loss = sum(jnp.sum(leaf**2) for leaf in jax.tree_leaves(params))
loss = loss + self.config['l2_reg'] * l2_loss
return loss
@jit
def train_step(self, params, opt_state, batch, key):
"""Single training step"""
loss, grads = jax.value_and_grad(self.loss_fn)(params, batch, key)
updates, opt_state = self.optimizer.update(grads, opt_state)
params = optax.apply_updates(params, updates)
return params, opt_state, loss
def train(self, train_data, val_data=None, epochs=100):
"""Train the model"""
if self.optimizer is None:
self.setup_optimizer()
# Initialize optimizer state
opt_state = self.optimizer.init(self.model)
params = self.model
key = random.PRNGKey(42)
for epoch in tqdm(range(epochs), desc="Training"):
epoch_loss = 0.0
num_batches = 0
for batch in train_data:
key, subkey = random.split(key)
params, opt_state, loss = self.train_step(params, opt_state, batch, subkey)
epoch_loss += loss
num_batches += 1
avg_loss = epoch_loss / num_batches
self.loss_history.append(float(avg_loss))
# Validation
if val_data is not None and epoch % 10 == 0:
val_loss = self.evaluate(params, val_data)
logger.info(f"Epoch {epoch}: Train Loss = {avg_loss:.6f}, Val Loss = {val_loss:.6f}")
# Early stopping check
if self.config.get('early_stopping', False) and epoch > 50:
if len(self.loss_history) > 10:
recent_losses = self.loss_history[-10:]
if all(recent_losses[i] <= recent_losses[i+1] for i in range(9)):
logger.info(f"Early stopping at epoch {epoch}")
break
self.model = params
return self.model
def evaluate(self, params, test_data):
"""Evaluate model on test data"""
model = eqx.tree_at(lambda m: m, self.model, params)
total_loss = 0.0
num_batches = 0
key = random.PRNGKey(0)
for batch in test_data:
key, subkey = random.split(key)
inputs, targets = batch
outputs, _ = model(inputs, self.config['dt'], subkey)
loss = jnp.mean((outputs - targets) ** 2)
total_loss += loss
num_batches += 1
return total_loss / num_batches
# =============================================================================
# Advanced Analysis and Visualization
# =============================================================================
class LNNAnalyzer:
"""Advanced analysis tools for LNNs"""
def __init__(self, model: LiquidNeuralNetwork):
self.model = model
self.analysis_results = {}
def analyze_dynamics(self, test_inputs, dt=0.1, duration=10.0):
"""Analyze network dynamics over time"""
key = random.PRNGKey(0)
# Generate extended input sequence
steps = int(duration / dt)
extended_inputs = []
for i in range(steps):
input_idx = i % len(test_inputs)
extended_inputs.append(test_inputs[input_idx])
# Run forward pass
outputs, hidden_states = self.model(extended_inputs, dt, key)
# Analyze trajectory properties
trajectories = []
for i, state in enumerate(hidden_states):
trajectory = {
'layer': i,
'states': state,
'norm': jnp.linalg.norm(state),
'max_val': jnp.max(jnp.abs(state)),
'stability': 'stable' if jnp.max(jnp.abs(state)) < 10.0 else 'unstable'
}
trajectories.append(trajectory)
self.analysis_results['dynamics'] = {
'trajectories': trajectories,
'outputs': outputs,
'time_steps': jnp.arange(0, duration, dt)
}
return self.analysis_results['dynamics']
def analyze_stability(self, test_inputs, perturbation_scale=0.1):
"""Analyze network stability under perturbations"""
key = random.PRNGKey(42)
stability_metrics = []
for input_data in test_inputs:
# Original output
original_output, _ = self.model([input_data], 0.1, key)
# Perturbed outputs
perturbed_outputs = []
for _ in range(10): # Multiple perturbations
key, subkey = random.split(key)
perturbation = random.normal(subkey, input_data.shape) * perturbation_scale
perturbed_input = input_data + perturbation
perturbed_output, _ = self.model([perturbed_input], 0.1, key)
perturbed_outputs.append(perturbed_output)
# Compute stability metrics
perturbation_effects = [
jnp.linalg.norm(orig - pert)
for orig, pert in zip([original_output], perturbed_outputs)
]
stability_metrics.append({
'input': input_data,
'original_output': original_output,
'perturbation_effects': perturbation_effects,
'mean_perturbation': jnp.mean(jnp.array(perturbation_effects)),
'stability_score': 1.0 / (1.0 + jnp.mean(jnp.array(perturbation_effects)))
})
self.analysis_results['stability'] = stability_metrics
return stability_metrics
def visualize_dynamics(self, save_path=None):
"""Visualize network dynamics"""
if 'dynamics' not in self.analysis_results:
raise ValueError("Must run analyze_dynamics first")
dynamics = self.analysis_results['dynamics']
# Create subplots
fig = make_subplots(
rows=2, cols=2,
subplot_titles=('Hidden State Trajectories', 'Output Trajectories',
'State Norms', 'Stability Analysis'),
specs=[[{"secondary_y": False}, {"secondary_y": False}],
[{"secondary_y": False}, {"secondary_y": False}]]
)
# Plot hidden state trajectories
for i, traj in enumerate(dynamics['trajectories']):
fig.add_trace(
go.Scatter(
x=dynamics['time_steps'],
y=traj['states'],
mode='lines',
name=f'Layer {i}',
line=dict(width=2)
),
row=1, col=1
)
# Plot output trajectories
fig.add_trace(
go.Scatter(
x=dynamics['time_steps'],
y=dynamics['outputs'].flatten(),
mode='lines',
name='Output',
line=dict(color='red', width=2)
),
row=1, col=2
)
# Plot state norms
for i, traj in enumerate(dynamics['trajectories']):
fig.add_trace(
go.Scatter(
x=dynamics['time_steps'],
y=[traj['norm']] * len(dynamics['time_steps']),
mode='lines',
name=f'Layer {i} Norm',
line=dict(dash='dash')
),
row=2, col=1
)
# Update layout
fig.update_layout(
title='Liquid Neural Network Dynamics Analysis',
showlegend=True,
height=800,
width=1200
)
fig.update_xaxes(title_text="Time", row=2, col=1)
fig.update_xaxes(title_text="Time", row=2, col=2)
fig.update_yaxes(title_text="State Value", row=1, col=1)
fig.update_yaxes(title_text="Output Value", row=1, col=2)
if save_path:
fig.write_html(save_path)
return fig
def compute_expressivity(self, test_inputs):
"""Compute expressivity metrics"""
key = random.PRNGKey(0)
# Compute trajectory lengths
trajectory_lengths = []
for input_data in test_inputs:
outputs, hidden_states = self.model([input_data], 0.1, key)
# Compute trajectory length for each layer
for i, state in enumerate(hidden_states):
if len(trajectory_lengths) <= i:
trajectory_lengths.append([])
# Approximate trajectory length
state_diffs = jnp.diff(state) if len(state) > 1 else jnp.array([0.0])
length = jnp.sum(jnp.abs(state_diffs))
trajectory_lengths[i].append(length)
# Compute expressivity metrics
expressivity_metrics = []
for i, lengths in enumerate(trajectory_lengths):
metrics = {
'layer': i,
'mean_length': jnp.mean(jnp.array(lengths)),
'std_length': jnp.std(jnp.array(lengths)),
'max_length': jnp.max(jnp.array(lengths)),
'expressivity_score': jnp.mean(jnp.array(lengths))
}
expressivity_metrics.append(metrics)
self.analysis_results['expressivity'] = expressivity_metrics
return expressivity_metrics
# =============================================================================
# Benchmarking Framework
# =============================================================================
class LNNBenchmark:
"""Comprehensive benchmarking framework"""
def __init__(self):
self.results = {}
self.baselines = {}
def benchmark_performance(self, model, test_data, dt=0.1, num_runs=100):
"""Benchmark model performance"""
import time
key = random.PRNGKey(0)
# Warmup
for _ in range(10):
inputs, targets = test_data[0]
model(inputs, dt, key)
# Benchmark
times = []
for _ in range(num_runs):
inputs, targets = test_data[0]
start_time = time.time()
outputs, _ = model(inputs, dt, key)
end_time = time.time()
times.append(end_time - start_time)
return {
'mean_time': np.mean(times),
'std_time': np.std(times),
'min_time': np.min(times),
'max_time': np.max(times),
'throughput': 1.0 / np.mean(times)
}
def benchmark_accuracy(self, model, test_data, dt=0.1):
"""Benchmark model accuracy"""
key = random.PRNGKey(0)
all_predictions = []
all_targets = []
for inputs, targets in test_data:
outputs, _ = model(inputs, dt, key)
all_predictions.extend(outputs.flatten())
all_targets.extend(targets.flatten())
predictions = np.array(all_predictions)
targets = np.array(all_targets)
return {
'mse': mean_squared_error(targets, predictions),
'mae': mean_absolute_error(targets, predictions),
'r2': r2_score(targets, predictions),
'rmse': np.sqrt(mean_squared_error(targets, predictions))
}
def benchmark_memory(self, model, test_data, dt=0.1):
"""Benchmark memory usage"""
try:
import psutil
import os
process = psutil.Process(os.getpid())
# Memory before
mem_before = process.memory_info().rss / 1024 / 1024 # MB
# Run model
key = random.PRNGKey(0)
inputs, targets = test_data[0]
outputs, _ = model(inputs, dt, key)
# Memory after
mem_after = process.memory_info().rss / 1024 / 1024 # MB
return {
'memory_before_mb': mem_before,
'memory_after_mb': mem_after,
'memory_usage_mb': mem_after - mem_before
}
except ImportError:
return {'error': 'psutil not available'}
def compare_with_baselines(self, model, baselines, test_data, dt=0.1):
"""Compare LNN with baseline models"""
results = {}
# Benchmark LNN
results['lnn'] = {
'performance': self.benchmark_performance(model, test_data, dt),
'accuracy': self.benchmark_accuracy(model, test_data, dt),
'memory': self.benchmark_memory(model, test_data, dt)
}
# Benchmark baselines
for name, baseline_model in baselines.items():
try:
results[name] = {
'performance': self.benchmark_performance(baseline_model, test_data, dt),
'accuracy': self.benchmark_accuracy(baseline_model, test_data, dt),
'memory': self.benchmark_memory(baseline_model, test_data, dt)
}
except Exception as e:
results[name] = {'error': str(e)}
return results
# =============================================================================
# Example Usage and Main Functions
# =============================================================================
def create_synthetic_data(n_samples=1000, seq_length=50, input_dim=4):
"""Create synthetic time series data for testing"""
key = random.PRNGKey(42)
# Generate sinusoidal data with noise
t = jnp.linspace(0, 10, seq_length)
data = []
for _ in range(n_samples):
key, subkey = random.split(key)
# Multi-dimensional sinusoidal signals
frequencies = random.uniform(subkey, (input_dim,), minval=0.1, maxval=2.0)
phases = random.uniform(subkey, (input_dim,), minval=0, maxval=2*jnp.pi)
signals = jnp.array([
jnp.sin(freq * t + phase) + 0.1 * random.normal(subkey, (seq_length,))
for freq, phase in zip(frequencies, phases)
]).T
# Target is sum of signals
target = jnp.sum(signals, axis=1, keepdims=True)
data.append((signals, target))
return data
def main():
"""Main function demonstrating LNN usage"""
logger.info("Starting Liquid Neural Network Research Framework")
# Configuration
config = {
'input_size': 4,
'hidden_size': 8,
'output_size': 1,
'dt': 0.1,
'learning_rate': 0.001,
'epochs': 100,
'batch_size': 32
}
# Create synthetic data
train_data = create_synthetic_data(800, 50, config['input_size'])
test_data = create_synthetic_data(200, 50, config['input_size'])
# Create model
layer_configs = [
LTCConfig(
input_size=config['input_size'],
hidden_size=config['hidden_size'],
output_size=config['output_size'],
tau=2.0,
activation='tanh'
)
]
key = random.PRNGKey(42)
model = LiquidNeuralNetwork(layer_configs, key)
# Train model
trainer = LNNTrainer(model, config)
# Convert data to batches
train_batches = [(train_data[i:i+config['batch_size']],
train_data[i:i+config['batch_size']])
for i in range(0, len(train_data), config['batch_size'])]
trained_model = trainer.train(train_batches)
# Analyze model
analyzer = LNNAnalyzer(trained_model)
test_inputs = [data[0] for data in test_data[:10]]
dynamics = analyzer.analyze_dynamics(test_inputs)
stability = analyzer.analyze_stability(test_inputs)
expressivity = analyzer.compute_expressivity(test_inputs)
# Visualize results
fig = analyzer.visualize_dynamics('dynamics_analysis.html')
# Benchmark
benchmark = LNNBenchmark()
test_batches = [(test_data[i:i+10], test_data[i:i+10])
for i in range(0, min(100, len(test_data)), 10)]
performance = benchmark.benchmark_performance(trained_model, test_batches)
accuracy = benchmark.benchmark_accuracy(trained_model, test_batches)
memory = benchmark.benchmark_memory(trained_model, test_batches)
# Log results
logger.info(f"Training completed. Final loss: {trainer.loss_history[-1]:.6f}")
logger.info(f"Performance: {performance}")
logger.info(f"Accuracy: {accuracy}")
logger.info(f"Memory usage: {memory}")
# Save results
results = {
'config': config,
'training_loss': trainer.loss_history,
'dynamics': dynamics,
'stability': stability,
'expressivity': expressivity,
'performance': performance,
'accuracy': accuracy,
'memory': memory
}
with open('lnn_results.json', 'w') as f:
json.dump(results, f, indent=2, default=str)
logger.info("Research framework completed successfully!")
if __name__ == "__main__":
main()"""
Advanced visualization and analysis tools for Liquid Neural Networks
"""
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.graph_objects as go
import plotly.express as px
from plotly.subplots import make_subplots
import pandas as pd
import networkx as nx
from typing import Dict, List, Tuple, Optional
import jax.numpy as jnp
from pathlib import Path
import json
# Set style
plt.style.use('seaborn-v0_8')
sns.set_palette("husl")
class LNNVisualizer:
"""Advanced visualization tools for LNN analysis"""
def __init__(self, output_dir: str = "visualizations"):
self.output_dir = Path(output_dir)
self.output_dir.mkdir(exist_ok=True)
def plot_training_curves(self, loss_history: List[float], metrics: Dict = None):
"""Plot training loss and metrics curves"""
fig, axes = plt.subplots(1, 2, figsize=(15, 6))
# Loss curve
axes[0].plot(loss_history, linewidth=2, color='blue')
axes[0].set_title('Training Loss', fontsize=14, fontweight='bold')
axes[0].set_xlabel('Epoch')
axes[0].set_ylabel('Loss')
axes[0].grid(True, alpha=0.3)
axes[0].set_yscale('log')
# Metrics
if metrics:
for i, (name, values) in enumerate(metrics.items()):
axes[1].plot(values, label=name, linewidth=2)
axes[1].set_title('Training Metrics', fontsize=14, fontweight='bold')
axes[1].set_xlabel('Epoch')
axes[1].set_ylabel('Metric Value')
axes[1].legend()
axes[1].grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(self.output_dir / 'training_curves.png', dpi=300, bbox_inches='tight')
plt.show()
def plot_phase_portraits(self, trajectories: List[Dict], title: str = "Phase Portraits"):
"""Plot phase portraits of network dynamics"""
n_layers = len(trajectories)
cols = min(3, n_layers)
rows = (n_layers + cols - 1) // cols
fig, axes = plt.subplots(rows, cols, figsize=(5*cols, 5*rows))
if n_layers == 1:
axes = [axes]
elif rows == 1:
axes = axes.flatten()
else:
axes = axes.flatten()
for i, traj in enumerate(trajectories):
ax = axes[i]
states = traj['states']
if len(states) > 1:
# 2D projection of state
if states.ndim > 1:
x = states[:, 0] if states.shape[1] > 0 else states
y = states[:, 1] if states.shape[1] > 1 else np.zeros_like(x)
else:
x = states
y = np.zeros_like(x)
ax.plot(x, y, 'b-', alpha=0.7, linewidth=2)
ax.scatter(x[0], y[0], c='green', s=100, marker='o', label='Start')
ax.scatter(x[-1], y[-1], c='red', s=100, marker='x', label='End')
# Add arrows to show direction
for j in range(0, len(x)-1, max(1, len(x)//10)):
ax.annotate('', xy=(x[j+1], y[j+1]), xytext=(x[j], y[j]),
arrowprops=dict(arrowstyle='->', color='gray', alpha=0.5))
ax.set_title(f'Layer {i+1}', fontweight='bold')
ax.set_xlabel('State Dimension 1')
ax.set_ylabel('State Dimension 2')
ax.grid(True, alpha=0.3)
ax.legend()
# Hide unused subplots
for i in range(n_layers, len(axes)):
axes[i].set_visible(False)
plt.suptitle(title, fontsize=16, fontweight='bold')
plt.tight_layout()
plt.savefig(self.output_dir / 'phase_portraits.png', dpi=300, bbox_inches='tight')
plt.show()
def plot_stability_analysis(self, stability_metrics: List[Dict]):
"""Plot stability analysis results"""
fig, axes = plt.subplots(2, 2, figsize=(15, 12))
# Extract data
stability_scores = [m['stability_score'] for m in stability_metrics]
perturbation_effects = [m['mean_perturbation'] for m in stability_metrics]
# Stability scores histogram
axes[0, 0].hist(stability_scores, bins=20, alpha=0.7, color='blue', edgecolor='black')
axes[0, 0].set_title('Stability Scores Distribution', fontweight='bold')
axes[0, 0].set_xlabel('Stability Score')
axes[0, 0].set_ylabel('Frequency')
axes[0, 0].grid(True, alpha=0.3)
# Perturbation effects
axes[0, 1].hist(perturbation_effects, bins=20, alpha=0.7, color='orange', edgecolor='black')
axes[0, 1].set_title('Perturbation Effects Distribution', fontweight='bold')
axes[0, 1].set_xlabel('Mean Perturbation Effect')
axes[0, 1].set_ylabel('Frequency')
axes[0, 1].grid(True, alpha=0.3)
# Stability vs Perturbation scatter
axes[1, 0].scatter(perturbation_effects, stability_scores, alpha=0.6, s=50)
axes[1, 0].set_title('Stability vs Perturbation', fontweight='bold')
axes[1, 0].set_xlabel('Mean Perturbation Effect')
axes[1, 0].set_ylabel('Stability Score')
axes[1, 0].grid(True, alpha=0.3)
# Box plot of perturbation effects for each input
if len(stability_metrics) > 0 and 'perturbation_effects' in stability_metrics[0]:
all_effects = [m['perturbation_effects'] for m in stability_metrics]
axes[1, 1].boxplot(all_effects, labels=[f'Input {i+1}' for i in range(len(all_effects))])
axes[1, 1].set_title('Perturbation Effects by Input', fontweight='bold')
axes[1, 1].set_xlabel('Input Number')
axes[1, 1].set_ylabel('Perturbation Effect')
axes[1, 1].grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(self.output_dir / 'stability_analysis.png', dpi=300, bbox_inches='tight')
plt.show()
def plot_expressivity_analysis(self, expressivity_metrics: List[Dict]):
"""Plot expressivity analysis results"""
fig, axes = plt.subplots(2, 2, figsize=(15, 12))
layers = [m['layer'] for m in expressivity_metrics]
mean_lengths = [m['mean_length'] for m in expressivity_metrics]
std_lengths = [m['std_length'] for m in expressivity_metrics]
max_lengths = [m['max_length'] for m in expressivity_metrics]
# Mean trajectory lengths by layer
axes[0, 0].bar(layers, mean_lengths, alpha=0.7, color='purple')
axes[0, 0].set_title('Mean Trajectory Lengths by Layer', fontweight='bold')
axes[0, 0].set_xlabel('Layer')
axes[0, 0].set_ylabel('Mean Length')
axes[0, 0].grid(True, alpha=0.3)
# Standard deviation of trajectory lengths
axes[0, 1].bar(layers, std_lengths, alpha=0.7, color='green')
axes[0, 1].set_title('Trajectory Length Std Dev by Layer', fontweight='bold')
axes[0, 1].set_xlabel('Layer')
axes[0, 1].set_ylabel('Std Dev')
axes[0, 1].grid(True, alpha=0.3)
# Max trajectory lengths
axes[1, 0].bar(layers, max_lengths, alpha=0.7, color='red')
axes[1, 0].set_title('Max Trajectory Lengths by Layer', fontweight='bold')
axes[1, 0].set_xlabel('Layer')
axes[1, 0].set_ylabel('Max Length')
axes[1, 0].grid(True, alpha=0.3)
# Expressivity scores
expressivity_scores = [m['expressivity_score'] for m in expressivity_metrics]
axes[1, 1].plot(layers, expressivity_scores, 'o-', linewidth=2, markersize=8)
axes[1, 1].set_title('Expressivity Scores by Layer', fontweight='bold')
axes[1, 1].set_xlabel('Layer')
axes[1, 1].set_ylabel('Expressivity Score')
axes[1, 1].grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(self.output_dir / 'expressivity_analysis.png', dpi=300, bbox_inches='tight')
plt.show()
def create_interactive_dashboard(self, results: Dict):
"""Create interactive dashboard using Plotly"""
# Create subplots
fig = make_subplots(
rows=3, cols=2,
subplot_titles=('Training Loss', 'Stability Scores',
'Expressivity by Layer', 'Performance Metrics',
'Memory Usage', 'Trajectory Visualization'),
specs=[[{"secondary_y": False}, {"secondary_y": False}],
[{"secondary_y": False}, {"secondary_y": False}],
[{"secondary_y": False}, {"secondary_y": False}]]
)
# Training loss
if 'training_loss' in results:
fig.add_trace(
go.Scatter(
x=list(range(len(results['training_loss']))),
y=results['training_loss'],
mode='lines',
name='Training Loss',
line=dict(color='blue', width=2)
),
row=1, col=1
)
# Stability scores
if 'stability' in results:
stability_scores = [m['stability_score'] for m in results['stability']]
fig.add_trace(
go.Histogram(
x=stability_scores,
name='Stability Scores',
nbinsx=20,
marker_color='orange'
),
row=1, col=2
)
# Expressivity by layer
if 'expressivity' in results:
layers = [m['layer'] for m in results['expressivity']]
scores = [m['expressivity_score'] for m in results['expressivity']]
fig.add_trace(
go.Bar(
x=layers,
y=scores,
name='Expressivity',
marker_color='purple'
),
row=2, col=1
)
# Performance metrics
if 'performance' in results:
perf = results['performance']
metrics = ['mean_time', 'throughput']
values = [perf.get(m, 0) for m in metrics]
fig.add_trace(
go.Bar(
x=metrics,
y=values,
name='Performance',
marker_color='green'
),
row=2, col=2
)
# Memory usage
if 'memory' in results:
memory = results['memory']
if 'memory_usage_mb' in memory:
fig.add_trace(
go.Indicator(
mode="gauge+number",
value=memory['memory_usage_mb'],
domain={'x': [0, 1], 'y': [0, 1]},
title={'text': "Memory Usage (MB)"},
gauge={'axis': {'range': [None, 1000]},
'bar': {'color': "darkblue"},
'steps': [{'range': [0, 250], 'color': "lightgray"},
{'range': [250, 500], 'color': "gray"}],
'threshold': {'line': {'color': "red", 'width': 4},
'thickness': 0.75, 'value': 500}}
),
row=3, col=1
)
# Update layout
fig.update_layout(
title="Liquid Neural Network Analysis Dashboard",
showlegend=True,
height=1200,
width=1600
)
# Save dashboard
fig.write_html(self.output_dir / 'interactive_dashboard.html')
return fig
def plot_comparison_charts(self, comparison_results: Dict):
"""Plot comparison charts between different models"""
models = list(comparison_results.keys())
# Extract metrics
metrics = {}
for model in models:
if 'accuracy' in comparison_results[model]:
acc = comparison_results[model]['accuracy']
metrics[model] = {
'MSE': acc.get('mse', 0),
'MAE': acc.get('mae', 0),
'R2': acc.get('r2', 0)
}
if not metrics:
return
# Create comparison plots
fig, axes = plt.subplots(2, 2, figsize=(15, 12))
# MSE comparison
mse_values = [metrics[m]['MSE'] for m in models]
axes[0, 0].bar(models, mse_values, alpha=0.7, color='red')
axes[0, 0].set_title('Mean Squared Error Comparison', fontweight='bold')
axes[0, 0].set_ylabel('MSE')
axes[0, 0].tick_params(axis='x', rotation=45)
axes[0, 0].grid(True, alpha=0.3)
# MAE comparison
mae_values = [metrics[m]['MAE'] for m in models]
axes[0, 1].bar(models, mae_values, alpha=0.7, color='blue')
axes[0, 1].set_title('Mean Absolute Error Comparison', fontweight='bold')
axes[0, 1].set_ylabel('MAE')
axes[0, 1].tick_params(axis='x', rotation=45)
axes[0, 1].grid(True, alpha=0.3)
# R2 comparison
r2_values = [metrics[m]['R2'] for m in models]
axes[1, 0].bar(models, r2_values, alpha=0.7, color='green')
axes[1, 0].set_title('R² Score Comparison', fontweight='bold')
axes[1, 0].set_ylabel('R²')
axes[1, 0].tick_params(axis='x', rotation=45)
axes[1, 0].grid(True, alpha=0.3)
# Performance comparison
perf_times = []
for model in models:
if 'performance' in comparison_results[model]:
perf_times.append(comparison_results[model]['performance'].get('mean_time', 0))
else:
perf_times.append(0)
axes[1, 1].bar(models, perf_times, alpha=0.7, color='purple')
axes[1, 1].set_title('Inference Time Comparison', fontweight='bold')
axes[1, 1].set_ylabel('Mean Time (s)')
axes[1, 1].tick_params(axis='x', rotation=45)
axes[1, 1].grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig(self.output_dir / 'model_comparison.png', dpi=300, bbox_inches='tight')
plt.show()
def plot_network_architecture(self, model_config: Dict):
"""Visualize network architecture using NetworkX"""
G = nx.DiGraph()
# Add nodes
layer_configs = model_config.get('layers', [])
node_id = 0
layer_nodes = {}
for layer_idx, layer_config in enumerate(layer_configs):
layer_nodes[layer_idx] = []
for neuron_idx in range(layer_config.get('size', 1)):
G.add_node(node_id, layer=layer_idx, neuron=neuron_idx)
layer_nodes[layer_idx].append(node_id)
node_id += 1
# Add edges (simple feed-forward for now)
for layer_idx in range(len(layer_configs) - 1):
for from_node in layer_nodes[layer_idx]:
for to_node in layer_nodes[layer_idx + 1]:
G.add_edge(from_node, to_node)
# Create layout
pos = {}
for layer_idx, nodes in layer_nodes.items():
for i, node in enumerate(nodes):
pos[node] = (layer_idx, i - len(nodes)/2)
# Plot
plt.figure(figsize=(12, 8))
# Draw nodes by layer
colors = ['red', 'blue', 'green', 'purple', 'orange']
for layer_idx, nodes in layer_nodes.items():
nx.draw_networkx_nodes(G, pos, nodelist=nodes,
node_color=colors[layer_idx % len(colors)],
node_size=500, alpha=0.8)
# Draw edges
nx.draw_networkx_edges(G, pos, alpha=0.3, arrows=True, arrowsize=20)
# Add labels
labels = {node: f'N{data["neuron"]}' for node, data in G.nodes(data=True)}
nx.draw_networkx_labels(G, pos, labels, font_size=8)
plt.title('Liquid Neural Network Architecture', fontsize=16, fontweight='bold')
plt.axis('off')
plt.tight_layout()
plt.savefig(self.output_dir / 'network_architecture.png', dpi=300, bbox_inches='tight')
plt.show()
def generate_report(self, results: Dict):
"""Generate a comprehensive HTML report"""
html_content = f"""
<!DOCTYPE html>
<html>
<head>
<title>Liquid Neural Network Analysis Report</title>
<style>
body {{ font-family: Arial, sans-serif; margin: 40px; }}
.header {{ background-color: #f0f0f0; padding: 20px; border-radius: 10px; }}
.section {{ margin: 20px 0; padding: 20px; border: 1px solid #ddd; border-radius: 5px; }}
.metric {{ display: inline-block; margin: 10px; padding: 10px; background-color: #e6f3ff; border-radius: 5px; }}
.metric-value {{ font-weight: bold; font-size: 18px; color: #0066cc; }}
img {{ max-width: 100%; height: auto; margin: 10px 0; }}
</style>
</head>
<body>
<div class="header">
<h1>Liquid Neural Network Analysis Report</h1>
<p>Generated on: {pd.Timestamp.now().strftime('%Y-%m-%d %H:%M:%S')}</p>
</div>
<div class="section">
<h2>Configuration</h2>
<pre>{json.dumps(results.get('config', {}), indent=2)}</pre>
</div>
<div class="section">
<h2>Performance Metrics</h2>
"""
if 'accuracy' in results:
acc = results['accuracy']
html_content += f"""
<div class="metric">
<div>MSE</div>
<div class="metric-value">{acc.get('mse', 0):.6f}</div>
</div>
<div class="metric">
<div>MAE</div>
<div class="metric-value">{acc.get('mae', 0):.6f}</div>
</div>
<div class="metric">
<div>R²</div>
<div class="metric-value">{acc.get('r2', 0):.6f}</div>
</div>
"""
if 'performance' in results:
perf = results['performance']
html_content += f"""
<div class="metric">
<div>Mean Time</div>
<div class="metric-value">{perf.get('mean_time', 0):.6f}s</div>
</div>
<div class="metric">
<div>Throughput</div>
<div class="metric-value">{perf.get('throughput', 0):.2f} ops/s</div>
</div>
"""
html_content += """
</div>
<div class="section">
<h2>Visualizations</h2>
<h3>Training Curves</h3>
<img src="training_curves.png" alt="Training Curves">
<h3>Stability Analysis</h3>
<img src="stability_analysis.png" alt="Stability Analysis">
<h3>Expressivity Analysis</h3>
<img src="expressivity_analysis.png" alt="Expressivity Analysis">
<h3>Network Architecture</h3>
<img src="network_architecture.png" alt="Network Architecture">
</div>
<div class="section">
<h2>Interactive Dashboard</h2>
<p><a href="interactive_dashboard.html">Open Interactive Dashboard</a></p>
</div>
</body>
</html>
"""
with open(self.output_dir / 'report.html', 'w') as f:
f.write(html_content)
print(f"Report generated: {self.output_dir / 'report.html'}")This comprehensive setup provides:
- **Complete project structure** with all necessary directories
- **FreeBSD-specific optimizations** for your system
- **Multi-language support** (Clojure, Python, Julia, Rust)
- **Advanced mathematical implementations** of LTC and CfC neurons
- **Comprehensive benchmarking** and analysis tools
- **Interactive visualizations** and dashboards
- **Real-world applications** (autonomous systems, medical diagnosis, etc.)
- **Training frameworks** with multiple optimizers
- **Extensive documentation** system
- **CI/CD integration** ready setup
The setup is designed to be:
- **Modular**: Each component can be used independently
- **Extensible**: Easy to add new neuron types, applications, or analyses
- **Production-ready**: Includes containerization, CI/CD, and monitoring
- **Research-focused**: Comprehensive analysis and visualization tools
You can now run `emacs SETUP.org` and use `org-babel-tangle` to extract all the code into your project structure!