[BUG] Deepseek Model Output Processing

[mgmorgan23]

Bug Description

When running evals on Deepseek models from HuggingFace, the output has undecoded tokens left throughout.

Example:

\u010aOkay,\u0120so\u0120I\u0120have\u0120this\u0120multiplechoice\u0120question\u0120about\u0120toxic\u0120proteins\u0120from\u0120bacteria,\u0120and\u0120I\u0120need\u0120to\u0120figure\u0120out\u0120the\u0120right\u0120answer.\u0120Let\u0120me\u0120think\u0120thro

This behavior is new -- output was processed correctly around June 5th but this error appeared around June 17th. I replicated this with deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B and deepseek-ai/DeepSeek-R1-Distill-Qwen-7B.

To Reproduce

Steps to reproduce the behavior:

From the main branch:

olmo-eval beaker launch \
  -m deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
  -t "ifeval_ood@urgent" \
  -w "ai2/olmo-eval-debug" \
  -B "ai2/oe-other" \
  --gpus 2 \
  --cluster h100 \
  --inspect

The error is visible in the predictions.jsonl file.