Parley transforms every machine in your network—whether it's Apple Silicon Macs, NVIDIA workstations, or spare CPUs—into a unified, private large language model (LLM) cluster. With an easy, zero-configuration setup, Parley ensures seamless integration and operation without cloud dependency or data leakage. Compatible with OpenAI and Ollama, this tool enhances your existing workflows effortlessly.

Key Features

• Automatic Discovery: Nodes autonomously identify themselves across your local area network. New machines integrate into the cluster immediately and exit gracefully when powered down.
• Intelligent Routing: Requests are directed to the optimal node that has the model ready, ensuring efficiency by prioritizing the shortest queue.
• Capability-Based Naming: Utilize aliases like parley:code, parley:fast, or parley:best to dynamically resolve to the most suitable model across your nodes.
• Built-In Security: All inter-node communications are safeguarded with TLS encryption and per-machine identity verification, ensuring complete privacy.

Quick Start

Run Parley with simple commands:

$ parley serve
parley serve
  local:   http://localhost
  network: http://192.168.1.42

$ parley pull qwen3-coder:30b
$ parley status

APIs Supported

Parley effectively supports multiple API families, allowing existing applications and tools to connect directly:

Usage Example

Invoke the API seamlessly with:

curl http://<node>/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"parley:code","messages":[{"role":"user","content":"hello"}]}'

Chat Interfaces

Parley implements both OpenAI and Ollama APIs, allowing compatibility with numerous front-end frameworks and chat interfaces:

Web Applications

• Open WebUI
• LibreChat
• Lobe Chat
• NextChat
• big-AGI
• ...

Desktop Clients

• Dify
• Maid
• Witsy
• ...

Parley simplifies the management of AI models, enabling the download of models using the parley pull command for effective weight management across the network. This versatile tool maximizes resource utilization while preserving privacy and security.