rag-retrieval / library
Batched
Symptoms: too many sequential embed calls; provider rate limits on unbatched traffic.
Use it when
- batch embedding requests to cut API cost
- dynamic batching for inference callables
What it solves
Not the fit when
- not a model host
Install
pip install batched
Invoke
Wrap embed or generate callables with batched.aio.BatchProcessor to cut cost-per-call under load.
Alternatives
No reviewed alternatives recorded yet.