UVA Research Computing GenAI Service

How to leverage the UVA Research Generative AI Interface in your SLURM and interactive jobs.

In March of 2026 UVA Research Computing announced the release of an internal, research-driven LLM available to faculty. This repository demonstrates ways to incorporate this new service into new and existing workflows.

This high‑powered generative AI platform runs Kimi K2.5, an open-source, 1-trillion parameter multimodal model well suited for visual analysis, agentic applications, and code‑based tasks. Open to Research Computing users, the tool runs on eight NVIDIA H200 graphics processing units (GPUs) and allows researchers to submit up to 60 queries per minute at no charge.

This service is available as both a web interface and an API.

• Read the User Guide
• Access the Web Interface

Limitations

Note that this service provides no history, memory, or context-awareness. Every interaction you have with the web UI or API is a fresh new encounter.

Setup

To incorporate the RC LLM into your workflows, you need to sign into the web interface and generate an API key. Follow these instructions to complete that.

Your key will look something like this:

sk-6defbb8f883649b4bb29a32dbb23a1c9

Since you should never store a sensitive key or password in your code itself, the best practice is to export this key as an environment variable in your Rivanna account.

1. Sign into Rivanna using either SSH or OpenOnDemand. If using the latter, open a terminal.
2. Determine what shell you are using by running this command:

   echo $0
   

3. In most cases users sign into the bash shell. Edit the appropriate resource file for your shell, i.e. ~/.bashrc for bash users, or ~/.zshrc for zsh users, etc.
4. Add an export line to that file to inject your key into all future sessions. You can assign the ENV variable any name you like:

   export UVARC_LLM_TOKEN="sk-6defbb8f883649b4bb29a32dbb23a1c9"
   

5. Either source ~/.bashrc / source ~/.zshrc or logout and back in again to pick up your new environment variable.
6. Confirm it works by attempting to echo out the variable:

   echo $UVARC_LLM_TOKEN
   

Implementation

The basics:

• Your jobs need to POST your prompt and other media (if required) to the API endpoint of the LLM.
• Every request must reference your token for authentication and transmit a well-formed JSON payload.
• Responses are streamed and must be parsed.

Simple Text Prompt with Python

See prompt-only.py for an example. A key concept is that the HTTP response from this API is streamed back to the client, not delivered as a JSON blob to deserialize. Streaming cannot be disabled.

Streams are delivered in chunks (this can be seen in the HTTP header 'Transfer-Encoding': 'chunked' from this API), which means segments of the response are sent in order back to the client, which then must be parsed and reassembled.

In the script below, the request is configured and POSTed to the API endpoint, after which the content and reasoning of the response are reassembled.

To use this prompt-only example:

python3 prompt-only.py "What is the longest navigable river in South America?"

import os
import sys
import json
import requests

prompt = sys.argv[1] if len(sys.argv) > 1 else "What is 2 cubed?"

response = requests.post(
    "https://open-webui.rc.virginia.edu/api/chat/completions",
    headers={"Authorization": f"Bearer {os.environ.get('UVARC_LLM_TOKEN')}"},
    json={
        "model": "Kimi K2.5",
        "messages": [{"role": "user", "content": prompt}],
        "stream": True,
    },
    stream=True,
)
response.raise_for_status()

content = ""
reasoning = ""
for line in response.iter_lines(decode_unicode=True):
    if not line or not line.startswith("data: "):
        continue
    payload = line[len("data: "):]
    if payload == "[DONE]":
        break
    delta = json.loads(payload)["choices"][0]["delta"]
    content += delta.get("content") or ""
    reasoning += delta.get("reasoning") or ""

if reasoning:
    print(f"[reasoning]\n{reasoning}\n")
print(content)

The same method can be used for more sophisticated queries that attach one or more files to the prompt via the files API endpoint. See prompt-datafile.py for an example.

To use this script:

python3 prompt-datafile.py "Prompt goes here" datafile.xyz

# real example
python3 prompt-datafile.py "Are these restaurant reviews generally positive or negative?" ./restaurants.csv

import os
import sys
import json
import requests

prompt = sys.argv[1] if len(sys.argv) > 1 else "Are the reviews in the attached data generally positive or negative?"
file_path = sys.argv[2] if len(sys.argv) > 2 else "restaurants.csv"

base_url = "https://open-webui.rc.virginia.edu"
headers = {"Authorization": f"Bearer {os.environ.get('UVARC_LLM_TOKEN')}"}

with open(file_path, "rb") as f:
    upload = requests.post(
        f"{base_url}/api/v1/files/",
        headers=headers,
        files={"file": (os.path.basename(file_path), f, "text/csv")},
    )
upload.raise_for_status()
file_id = upload.json()["id"]

response = requests.post(
    f"{base_url}/api/chat/completions",
    headers=headers,
    json={
        "model": "Kimi K2.5",
        "messages": [{"role": "user", "content": prompt}],
        "files": [{"type": "file", "id": file_id}],
        "stream": True,
    },
    stream=True,
)
response.raise_for_status()

content = ""
reasoning = ""
for line in response.iter_lines(decode_unicode=True):
    if not line or not line.startswith("data: "):
        continue
    payload = line[len("data: "):]
    if payload == "[DONE]":
        break
    data = json.loads(payload)
    choices = data.get("choices")
    if not choices:
        continue
    delta = choices[0].get("delta") or {}
    content += delta.get("content") or ""
    reasoning += delta.get("reasoning") or ""

if reasoning:
    print(f"[reasoning]\n{reasoning}\n")
print(content)