Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

Interactive 3D visualization platform for exploring transformer architectures, tensors, and real-time LLM inference.

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

Android inference engine running 20B+ parameter LLMs on 4GB-8GB RAM devices. Features proprietary Layer-by-Layer (LBL) streaming, zero-copy mmap loading, and native C++/Kotlin architecture.

Splinter cohabitates inference and semantic governance in L3 cache and memory lanes, while simultaneously providing standardized POSIX-friendly tooling as building blocks on top of the provided library. Splinter is essentially a semantic "breadboard" that can be directly deployed at scale.

Running GGUF-Format ML Models in Pure Go

A simple Gradio app for local translation using the GGUF versions of MADLAD-400

Convert and quantize llm models

Privacy-first Local RAG Server: Chat with PDF & DOCX using GGUF models via llama.cpp and Qdrant. A lightweight, standalone FastAPI server with a clean HTML UI. High-performance, fully offline document intelligence. No Ollama, no cloud, no API keys.

Emotica AI is a compassionate and therapeutic virtual assistant designed to provide empathetic and supportive conversations. It integrates a local LLaMA model for text generation, a vision model for image captioning, a RAG system for information retrieval, and emotion detection to tailor its responses.

Containerized LLM for any use-case big or small

AI tool to help users research using local LLMs and automated web search.

Nectar-X-Studio is a powerful, Local AI-Inferencing application that allows the user download, create, run agents and run large language models on their own machine. With no internet connection required, Nectar ensures privacy-first, high-performance inference using cutting-edge open-source models from Hugging Face, Ollama, and beyond.

GGUF file format for dotnet