Paddler
Open Source Apache-2.0Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm...
Description
Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Preview
Platforms
Linux macOS Docker
Replaces the following proprietary tools
ChatGPT API → Paddler
Midjourney → Paddler
DALL-E → Paddler
Self-Hosting
Paddler can be self-hosted on your own infrastructure. Visit the official documentation for installation instructions.
Documentation →Categories
Tags
#ai
#llamacpp
#llm
#llmops
#load-balancer
Similar Tools
Ollama
Run large language models locally with ease
MIT ● Easy Self-hostable
ChatGPT Midjourney DALL-E
Last: Feb 2026 105.0k
View details →
LocalAI
Local OpenAI-compatible API for various AI models
MIT ●● Medium Self-hostable
ChatGPT Midjourney DALL-E
Last: Feb 2026 26.0k
View details →
Stable Diffusion WebUI
Powerful user interface for AI image generation
AGPL-3.0 ●● Medium Self-hostable
ChatGPT Midjourney DALL-E
Last: Feb 2026 145.0k
View details →