LiteLLM

LiteLLM is an open-source Python SDK and self-hostable gateway that lets you call more than 100 LLM providers through a single OpenAI-compatible format. It collapses each model's different calling conventions into one interface, and its proxy server centralizes cost tracking, load balancing, fallbacks, guardrails, and virtual key provisioning. It is infrastructure built for backend teams and platform engineers juggling many models at once.

Its biggest strength is that you can self-host the MIT-licensed core inside your own environment. Rather than routing traffic through an external SaaS, you can run virtual keys, budgets, and cost tracking without data ever leaving your walls, which suits organizations with strict security requirements.

That said, there is no Korean UI or documentation, so the entire setup and operation must be handled in English. Self-hosting also trades convenience for control, meaning you take on server provisioning, updates, and incident response yourself, which can be a real barrier for small teams without ops capacity.

In short, LiteLLM is a strong pick for teams that want to unify many LLMs behind one gateway while controlling cost and keys. If you would rather have a ready-to-use Korean managed service or a solution with no operational burden, another tool will serve you better.