GLM-OCR (Inference)

Docs\EN | Docs\ID | Model Card

Inference-only runtime for GLM-OCR, designed for low-latency OCR on GPUs and deployed on Modal

---

Features

• High Precision: SOTA document parsing on OmniDocBench v1.5 (94.6).
• Versatile Recognition: Optimized for text, complex tables, mathematical formulas, handwriting, and multilingual content.
• Efficient & Fast: Lightweight 0.9B model; supports low-latency inference.
• Developer Friendly: OpenAI-compatible vision API endpoints.
• Serverless Scaling: Powered by Modal for automatic scaling and zero-idle costs.

---

Quick Start

1. Install Modal CLI

pip install modal
modal setup

2. Configure API Secret

modal secret create neo_api_key NEO_API_KEY=your-secret-key

3. Deploy Service

modal run inference/zai_ocr.py::download
modal deploy inference/zai_ocr.py

---

Samples

Sample images for testing are located in the samples/ directory.

• samples/stock.jpg: A dense table containing stock market data, used for verifying tabular and numerical accuracy.

---

Prompt Modes & Output Formats

GLM-OCR is highly sensitive to the prompt prefix. All outputs are returned as strings and may include model-specific tokens like <|user|> at the end.

1. Text Recognition

Extracts raw text while maintaining the visual line structure.

• Prompt Prefix: Text Recognition:

View Example Output

Nasdaq & AMEX
Stocks in bold rose or fell 5% or more.
Track your investments with our continuously updated stocks.
...
9.19 6.89 ABX Air n 7.52 -0.10
33.25 12.40 ACMoore 13.58 -1.57
<|user|>

2. Table Recognition

Converts tabular data into clean HTML <table> format.

• Prompt Prefix: Table Recognition:

View Example Output

<table class="table table-bordered">
  <thead>
    <tr>
      <th colspan="5">52-week</th>
      <th colspan="5">52-week</th>
    </tr>
    <tr>
      <th>High</th>
      <th>Low</th>
      <th>Stock</th>
      <th>Last Change</th>
      <th></th>
      <th>High</th>
      <th>Low</th>
      <th>Stock</th>
      <th>Last Change</th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>9.19</td>
      <td>6.89</td>
      <td>ABX Air n</td>
      <td>7.52</td>
      <td>-0.10</td>
      <td>45.71</td>
      <td>32.50</td>
      <td>Biomet</td>
      <td>36.71</td>
      <td>-0.42</td>
    </tr>
  </tbody>
</table>
<|user|>

3. Formula Recognition

Converts mathematical expressions or structured arrays into LaTeX.

• Prompt Prefix: Formula Recognition:

View Example Output

$$
\begin{array}{l} 4 5. 7 1 3 2. 5 0 \mathrm {B i o m e t} 3 6. 7 1 - 0. 4 2 \\
2. 7 6 1. 2 0 \mathrm {B i o m i r a} 1. 4 6 + 0. 0 3
\end{array}
$$
<|user|>

4. Structured JSON Extraction

Extracts specific fields into a valid JSON object.

• Prompt Pattern: Please output the information in JSON with fields: {"name": "", "last": ""}

View Example Output

{"name": "ABX Air n", "last": "7.52"}
<|user|>

---

API Endpoints

POST /v1/ocr

Direct OCR endpoint for rapid text extraction.

curl -X POST https://your-workspace--zai-ocr-fastapi-service.modal.run/v1/ocr \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "image": "https://example.com/image.jpg",
    "image_type": "url",
    "prompt": "Text Recognition:"
  }'

POST /v1/chat/completions

OpenAI Vision API compatible endpoint.

curl -X POST https://your-workspace--zai-ocr-fastapi-service.modal.run/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "zai-ocr",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Table Recognition:"},
          {"type": "image_url", "image_url": {"url": "https://example.com/table.png"}}
        ]
      }
    ]
  }'

---

Global Access (via Neosantara)

This model is also available via the Neosantara AI Gateway.

• Model ID: glm-ocr
• Base URL: https://api.neosantara.xyz/v1

---

Python Example

import requests

url = "https://your-workspace--zai-ocr-fastapi-service.modal.run/v1/ocr"
payload = {
    "image": "https://example.com/document.jpg",
    "image_type": "url",
    "prompt": "Text Recognition:"
}

response = requests.post(url, json=payload, headers={"Authorization": "Bearer YOUR_API_KEY"})
print(response.json()["extracted_text"])

---

Configuration

Edit inference/ocr_config.py to customize the deployment:

• MODEL_NAME: The HuggingFace model path.
• GPU_TYPE: Modal GPU selection (e.g., L4, A10G).
• SCALEDOWN_WINDOW: Time in seconds to wait before scaling to zero.

---

Project Structure

.
├── inference/
│  ├── zai_ocr.py       # Main app & FastAPI endpoints
│  ├── ocr_config.py     # Config & dependencies
│  └── ocr_utils.py      # Image processing utils
├── samples/            # Test images
├── setup.sh            # Download helper
├── deploy.sh           # Deployment helper
├── test.sh             # Test helper
└── README.md           # Documentation

---

FAQ

Does it work with the OpenAI SDK?

Yes. The /v1/chat/completions endpoint is compatible with the OpenAI Python SDK.

Why use Modal?

Modal provides serverless GPU compute and scales to zero when idle, so you only pay for active usage.

Is streaming supported?

Not currently. OCR results are returned as a single complete block.

---

References

• GLM-OCR Model Card
• Z.ai Documentation
• Modal Documentation

---

License

Apache License 2.0.