[Feature] Async translation + translate_batch with concurrency control

[soumendrak]

What

Async / batch / concurrent variants of the translation functions.

import asyncio
from openodia import odia_to_other_lang_async, translate_batch

async def main():
    out = await odia_to_other_lang_async("ନମସ୍କାର")
    outs = await translate_batch(["ନମସ୍କାର", "ଭଲ ଅଛି"], to="en", concurrency=8)

asyncio.run(main())

Why

• Translating one string at a time over the network is slow.
• Real-world ingestion pipelines translate millions of strings in batch.
• deep-translator is sync-only; each call blocks. Batching 10k strings sequentially can take an hour; async with concurrency 16 takes ~3 minutes.
• Modern Python codebases (FastAPI, aiohttp, Discord bots, RAG retrievers) are async-first.

Impact

User

• Faster batch translation by 10–20x with controlled concurrency.
• Natural integration into async webserver code.

Developer

• Foundation for [[fastapi-server]] (/translate-batch endpoint).
• Encourages clean separation of I/O from CPU work.

Before / After

flowchart LR
    subgraph Before
      L1["[10000 strings]"] --> Loop1[Sequential loop]
      Loop1 --> Net1[Sequential network calls]
      Net1 --> X1[Slow - 1 hour]
    end
    subgraph After
      L2["[10000 strings]"] --> B2[translate_batch concurrency=16]
      B2 --> Async2[Async pool with bounded concurrency]
      Async2 --> R2[Fast - 3 minutes]
    end

Loading

Comparison

Mode	Throughput	Backpressure	Cancellable?
Sync (current)	1 RPS	manual	❌
Threaded pool	~10 RPS	manual	partial
Async with semaphore	~50 RPS	built-in	✅

Detailed explanation

Architecture:

1. New _async variants for odia_to_other_lang, other_lang_to_odia, universal_translation.
2. translate_batch(texts, *, concurrency=8, **kwargs) — accepts a list, returns a list in the same order.
3. Concurrency bounded by asyncio.Semaphore.
4. Retry with exponential backoff on transient errors (5xx, network timeouts).
5. Shared cache between sync and async paths (same LRU keyed by (text, target, backend)).
6. Cancellation: asyncio.CancelledError propagates correctly; partial results not stored.

For Google backend (today), wrap deep-translator calls in asyncio.to_thread. For IndicTrans2 backend (issue [[indictrans2-backend]]), use the model's native batch API.

Proposed solutions

Option A — asyncio.to_thread wrappers + semaphore (recommended)

• Minimal code; works with current sync deep-translator.
• Pros: zero new deps; backward compatible.
• Cons: thread overhead small but non-zero.

Option B — Switch translation deps to async-native

• Use httpx async + Google Translate REST.
• Pros: true async I/O.
• Cons: replaces existing deep-translator semantics; risk.

Option C — Background queue (Celery / Arq)

• For very large batch jobs.
• Pros: persistent, retryable.
• Cons: deployment burden; overkill in-library.

Recommended: A; expose B as a follow-up.

Acceptance criteria

• await odia_to_other_lang_async(x) == odia_to_other_lang(x).
• translate_batch preserves input order.
• Concurrency limit honored (verify with a fake-slow backend).
• Cancellation does not corrupt the cache.

Backward compatibility

Existing sync functions unchanged. Async variants are net-new.

Labels: enhancement, help wanted