FastChat

Use it when

• serve and compare chat models behind an OpenAI-compatible endpoint
• run a multi-model evaluation or chatbot arena

What it solves

Not the fit when

• new deployments should compare newer high-throughput serving engines

Install

pip install fschat

Invoke

Start the controller, model worker, and OpenAI-compatible API server, then send chat-completion requests to the local endpoint.

Alternatives

No reviewed alternatives recorded yet.