GreyFox Community Edition

GreyFox Community Edition is a powerful self-hosted AI traffic proxy and local operator console designed for teams aiming to manage LLM token usage efficiently. This tool enables users to enforce individual usage limits, leverage exact cached responses, and maintain comprehensive visibility of AI traffic within their infrastructure.

Key Features :

• Provides an OpenAI-compatible proxy endpoint based at /v1/chat/completions
• Local Admin User Interface hosted directly from the container for ease of access
• Per-user token quota enforcement using the X-App-User-Id header
• Mock mode for seamless onboarding and demonstrations without costs
• Provider mode compatible with OpenAI upstream APIs
• Caches exact responses for improved efficiency on repeated non-streaming requests
• Utilizes local SQLite storage for settings, user management, logging, caching, and metrics tracking
• Offers traffic history, token analytics, manual cost calculation tools, and safe maintenance features

Community Edition Limitations:

• Supports a maximum of 5 active managed users
• Token monitoring acts as the primary signal for usage tracking
• Cost estimates are provided as manual information only
• Lacks a hosted GreyFox cloud control plane
• Does not include automatic update checks or updates to the container
• Excludes sophisticated request detail assessments, exports, in-depth diagnostics, and live traffic metrics

System Requirements:

• Docker Desktop or Docker Engine paired with Docker Compose
• An available host port, defaults to 8080
• A Provider API key is necessary only for utilizing live provider mode; no other dependencies like Node.js, npm, or Angular are required to run this release.

Quick Start Guide: To initiate GreyFox, create a compose.yaml file as shown below:

services:
  greyfox:
    image: ghcr.io/skillful-fox-studio/grey-fox-community:0.1.0
    container_name: greyfox-community
    environment:
      OPENAI_BASE_URL: ${OPENAI_BASE_URL:-https://api.openai.com/v1}
      GREYFOX_DB_PATH: ${GREYFOX_DB_PATH:-data/greyfox.db}
      PORT: 3000
      GREYFOX_STATIC_ROOT: /app/public/admin-ui
    ports:
      - "${GREYFOX_HTTP_PORT:-8080}:3000"
    volumes:
      - greyfox-data:/app/data
    restart: unless-stopped

volumes:
  greyfox-data:

After creating the configuration, start GreyFox using:

docker compose up -d

Access the Admin UI via:

http://localhost:8080

Conduct a health check using:

curl http://localhost:8080/api/health

Expected output:

{"status":"ok","service":"proxy-api"}

Integration Process:
GreyFox serves as a proxy layer without needing browser extensions or intercepting personal usage data. Integrate it by directing your application's requests through GreyFox instead of the upstream provider. For example:

Direct Setup:

Your application
      |
      | HTTPS request with provider API key
      v
OpenAI-compatible provider

GreyFox Setup:

Your application
      |
      | OpenAI-compatible request
      | Base URL: http://<greyfox-host>:<port>/v1
      | Header: X-App-User-Id: <your-end-user-id>
      v
GreyFox Community Edition
      |
      | Local checks:
      | - user token quota
      | - exact response cache
      | - prompt injection guard
      | - traffic logging
      v
OpenAI-compatible provider

Deployment Scenarios:

• For local demo or evaluation, refer to:

App or curl -> http://localhost:8080/v1/chat/completions -> GreyFox -> Provider

• When both the application and GreyFox run on the same Docker host, access using its service name:

http://greyfox:3000/v1/chat/completions

• In a team environment, GreyFox should be run on an internal server:

http://greyfox.internal:8080/v1/chat/completions

Client Configuration Example: Most SDKs allow overriding the base URL. Change:

baseURL = "https://api.openai.com/v1"

to:

baseURL = "http://localhost:8080/v1"

Ensure to add the end-user identifier header:

X-App-User-Id: user-123

Data Privacy:
GreyFox Community Edition is designed with a strong emphasis on privacy, ensuring that prompts, completions, logs, settings, provider keys, and metrics remain within the local infrastructure. Manual update checks only involve one request to GitHub Releases, preserving data confidentiality.

For more information or any queries regarding GreyFox, please refer to the repository or reach out to the support team.