MerchantryTidbits

llm-inference / library

litellm

Capability: litellm

Use it when

  • model router fail over between providers
  • multi provider OpenAI-compatible proxy
  • spend caps and fallbacks across models
  • one API many LLM providers

What it solves

rate-limit-providercost-per-call-too-highmulti-model-routing

Not the fit when

  • eliminates all provider outages
  • creates new model weights
  • not a diffusion/fast specialty model
  • not a specialty diffusion/fast model

Install

pip install litellm

Invoke

Proxy/router across providers and models under a harness (Hermes, Claude Code, OpenClaw). Spend caps and fallbacks. Does not invent new model weights.

Alternatives

instructor, outlines