Paddler
Open Source Apache-2.0LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm
Description
Open-source LLM load balancer and serving platform for self-hosting LLMs at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Preview
Platforms
Linux macOS Docker
Replaces the following proprietary tools
ChatGPT API → Paddler
Midjourney → Paddler
DALL-E → Paddler
Self-Hosting
Paddler can be self-hosted on your own infrastructure. Visit the official documentation for installation instructions.
Documentation →Categories
Tags
#ai
#llamacpp
#llm
#llmops
#load-balancer
Similar Tools
Ollama
Run large language models locally with ease
105.0k
↳ ChatGPT Midjourney +1
MIT ● Easy
LocalAI
Local OpenAI-compatible API for various AI models
26.0k
↳ ChatGPT Midjourney +1
MIT ●● Medium
Stable Diffusion WebUI
Powerful user interface for AI image generation
145.0k
↳ ChatGPT Midjourney +1
AGPL-3.0 ●● Medium