All issues

Search results

[FEATURE] Truncation support for non vllm_server providers

enhancementNew feature or requestNew feature or request

Status: Open.

#273 In allenai/olmo-eval;

· mgmorgan23 opened on Aug 3, 2026

Is this repo intended to replace OLMES?

Status: Open.

#266 In allenai/olmo-eval;

· BrownianNotion opened on Jul 28, 2026

[FEATURE] Add FrontierScience benchmark

enhancementNew feature or requestNew feature or request

Status: Open.

#263 In allenai/olmo-eval;

· donovanr opened on Jul 24, 2026

[BUG] Deepseek Model Output Processing

bugSomething isn't workingSomething isn't working

Status: Open.

#241 In allenai/olmo-eval;

· mgmorgan23 opened on Jul 2, 2026

[FEATURE] Cleanup magic flags for olmobase:code

enhancementNew feature or requestNew feature or request

Status: Open.

#144 In allenai/olmo-eval;

· davidheineman opened on Apr 20, 2026

Fix Deprecated Modal tunnel implementation in swe-rex

Status: Open.

#141 In allenai/olmo-eval;

· undfined opened on Apr 16, 2026

[BUG] vLLM venv mismatch between harness preset and model preset

bugSomething isn't workingSomething isn't working

Status: Open.

#123 In allenai/olmo-eval;

· rlebras opened on Apr 9, 2026

Add weighted aggregation strategy for Suite() (including nested weights)

Status: Open.

#84 In allenai/olmo-eval;

· kyleclo opened on Mar 3, 2026

Task spec API is ambiguous; propose canonical <dataset>:<format>:<metric> schema

Status: Open.

#82 In allenai/olmo-eval;

· kyleclo opened on Mar 3, 2026