Parley — Run LLM with zero configuration, zero data…

Pitch

Description

Comments

About this project

Parley enables the seamless operation of local AI models across various machines in a network, providing a shared and private large language model cluster. With automatic discovery, routing, and built-in security, it simplifies AI deployment without the need for cloud services or complicated configurations.

Parley transforms every machine in your network—whether it's Apple Silicon Macs, NVIDIA workstations, or spare CPUs—into a unified, private large language model (LLM) cluster. With an easy, zero-configuration setup, Parley ensures seamless integration and operation without cloud dependency or data leakage. Compatible with OpenAI and Ollama, this tool enhances your existing workflows effortlessly.

Key Features

Automatic Discovery: Nodes autonomously identify themselves across your local area network. New machines integrate into the cluster immediately and exit gracefully when powered down.
Intelligent Routing: Requests are directed to the optimal node that has the model ready, ensuring efficiency by prioritizing the shortest queue.
Capability-Based Naming: Utilize aliases like parley:code, parley:fast, or parley:best to dynamically resolve to the most suitable model across your nodes.
Built-In Security: All inter-node communications are safeguarded with TLS encryption and per-machine identity verification, ensuring complete privacy.

Quick Start

Run Parley with simple commands:

$ parley serve
parley serve
  local:   http://localhost
  network: http://192.168.1.42

$ parley pull qwen3-coder:30b
$ parley status

APIs Supported

Parley effectively supports multiple API families, allowing existing applications and tools to connect directly:

API	Endpoints	Connect by
OpenAI	`POST /v1/chat/completions`, `GET /v1/models`	`OPENAI_BASE_URL=http://<node>/v1`
Ollama	`/api/chat`, `/api/generate`, `/api/tags`, `/api/show`, `/api/ps`	`OLLAMA_HOST=http://<node>`
Anthropic	`POST /v1/messages`	base URL `http://<node>`

Usage Example

Invoke the API seamlessly with:

curl http://<node>/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"parley:code","messages":[{"role":"user","content":"hello"}]}'

Chat Interfaces

Parley implements both OpenAI and Ollama APIs, allowing compatibility with numerous front-end frameworks and chat interfaces:

Web Applications

Desktop Clients

Dify
Maid
Witsy
...

Parley simplifies the management of AI models, enabling the download of models using the parley pull command for effective weight management across the network. This versatile tool maximizes resource utilization while preserving privacy and security.