Skip to content

21latte1007/sayou-fabric

 
 

Repository files navigation

Sayou Fabric

PyPI Docs Python License Downloads

The Data-Centric Framework for Building Enterprise RAG Pipelines
Observe, Structure, and Assemble your Knowledge Graph.

한국어 (README.ko.md)


1. Philosophy

Sayou Fabric is a framework for Deterministic Knowledge Graph Construction.

We focus on transforming unstructured data into structured assets by analyzing their intrinsic topology—syntax trees, file hierarchies, and metadata. This approach ensures that the constructed Knowledge Graph is precise, reproducible, and structurally sound, strictly following the data's native organization.

1.1. Structure-First Architecture

We prioritize the native structure of data to build a solid skeleton before processing the content.

  • Code: Parsed via AST to deterministically link Classes, Methods, and Import dependencies.
  • Media: Structured via Temporal Timeline and Semantic Segmentation.
  • Docs: Hierarchically organized by Headers, Tables, and Layout Coordinates.

1.2. The 3-Tier Design Pattern

Inspired by robust enterprise architectures, every component adheres to a strict hierarchy to ensure stability at scale:

  • Tier 1 (Interface): The immutable contract defining behavior.
  • Tier 2 (Template): Standardized logic for ETL pipelines (Battery-included).
  • Tier 3 (Plugin): Vendor-specific implementations (e.g., YouTube, GitHub, Notion).
graph TD
    %% 노드 정의
    Brain[Sayou Brain
    < Orchestrator >]:::brain
    
    subgraph Libs [Data Libraries]
        direction LR
        Conn[Connector]
        Doc[Document]
        Ref[Refinery]
        Chunk[Chunking]
        Wrap[Wrapper]
        Assem[Assembler]
    end
    
    Core[Sayou Core
    < Ontology & Schema >]:::core

    %% 연결
    Brain --> Libs
    Libs --> Core
    
    %% 정렬 힌트 (Libs 내부)
    Conn ~~~ Doc ~~~ Ref ~~~ Chunk ~~~ Wrap ~~~ Assem

    %% 스타일 적용
    class Conn,Doc,Ref,Chunk,Wrap,Assem modules
Loading

2. The Ecosystem

Sayou Fabric consists of independent, loosely coupled libraries that work together seamlessly.

Package Version Description
sayou-core PyPI Foundational layer defining Schemas, Ontology, and base components.
sayou-brain PyPI Manages the entire lifecycle of data processing pipelines.
sayou-connector PyPI Universal ingestor with auto-source detection and streaming support.
sayou-document PyPI Parses documents while preserving layout, styles, and spatial coordinates.
sayou-refinery PyPI Normalizes data formats and removes noise or PII.
sayou-chunking PyPI Splits text intelligently based on context (Code, Markdown, Time).
sayou-wrapper PyPI Maps raw data to the Sayou Ontology and creates standardized Nodes.
sayou-assembler PyPI Constructs semantic relationships (Edges) between Nodes.
sayou-loader PyPI Exports the constructed Knowledge Graph to Databases or Files.
sayou-extractor PyPI version Retrieves context using Hybrid Search (Vector + Graph).
sayou-llm PyPI version A unified adapter interface for various LLM providers.
sayou-visualizer PyPI Visualizes the pipeline flow and renders 3D Knowledge Graphs.

3. Installation

You can install the entire suite via the orchestrator package:

pip install sayou-brain

Or install individual components as needed for a lightweight setup:

pip install sayou-chunking sayou-document
pip install sayou-visualizer

4. Quick Start

The sayou-brain package provides high-level facades that abstract away the complexity of underlying modules. Choose the pipeline that fits your data source.

Case A: Document Processing (PDF, Office)

Use StandardPipeline for layout-preserving document analysis.

flowchart LR
    %% 노드 스타일
    classDef input fill:#fff3e0,stroke:#e65100;
    classDef process fill:#f3e5f5,stroke:#7b1fa2;
    classDef output fill:#e8f5e9,stroke:#2e7d32;

    Input[PDF / Office]:::input
    
    subgraph SP [Standard Pipeline]
        direction LR
        C(Connector) --> D(Document Layout Parse)
        D --> R(Refinery)
        R --> CH(Chunking)
        CH --> W(Wrapper)
        W --> A(Assembler)
    end
    
    KG[("Knowledge Graph")]:::output

    Input --> C
    A --> L(Loader)
    L --> KG

    %% 강조
    style D stroke-width:3px,stroke:#d32f2f
Loading
from sayou.brain import StandardPipeline

result = StandardPipeline().process(
    source="./reports/financial_q1.pdf",
    destination="knowledge_graph.json",
)

print(f"Ingestion Complete. Processed: {result['processed']}")

Case B: Multimedia & Code Analysis

Use NormalPipeline for logic-based extraction from Video, Code repositories, or Web sources.

flowchart LR
    %% 노드 스타일
    classDef input fill:#fff3e0,stroke:#e65100;
    classDef process fill:#e1f5fe,stroke:#0277bd;
    classDef output fill:#e8f5e9,stroke:#2e7d32;

    Input[YouTube / Code]:::input
    
    subgraph NP [Normal Pipeline]
        direction LR
        C(Connector) --> R(Refinery)
        R --> CH(Chunking)
        CH --> W(Wrapper)
        W --> A(Assembler)
    end
    
    KG[("Knowledge Graph")]:::output

    Input --> C
    A --> L(Loader)
    L --> KG
    
    %% 강조
    style CH stroke-width:3px,stroke:#0288d1
Loading
from sayou.brain import NormalPipeline

result = NormalPipeline().process(
    source="youtube://YOUTUBE_VIDEO_ID",
    destination="./output/graph_data.json"
)

print(f"Graph Construction Complete. Nodes: {len(result['nodes'])}")

Output Format (JSON)

The output is a structured JSON strictly following the Sayou Ontology, ready for Graph Databases or Vector Stores.

Click to expand JSON example
{
  "nodes": [
    {
      "node_id": "sayou:doc:1_h_0",
      "node_class": "sayou:Topic",
      "attributes": { "schema:text": "Financial Summary Q1" },
      "relationships": {}
    },
    { "……" }
  ],
  "edges": [
    { "……" }
  ]
}

5. Visualizer Showcase

Observability is crucial for data engineering. Sayou Fabric provides built-in rendering tools to inspect your data topology and pipeline flow.

📊 Pipeline Telemetry (DAG Tracer)

Monitor the execution flow. Visualize which plugins are activated and verify how data transforms across the pipeline.

🌌 Holographic Knowledge Graph

Transform raw data into an interactive KG city.

  • Analytics Knowledge View: Inspect strict node properties and relationships for debugging.

  • Showcase Knowledge View: Visualize the cluster and semantic density of your knowledge base.


6. Documentation & Contribution

Sayou Fabric is currently in Public Beta (v0.4.0). We are actively stabilizing the core engine and expanding the adapter ecosystem to support more enterprise data sources.


7. License

Apache 2.0 License © 2026 Sayouzone

8. Agent List

Agent Example Description
Code Diff AI Agent
Code Analysis AI Agent

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 82.2%
  • Jupyter Notebook 17.8%