TensorWall serves as an open-source control plane for LLM APIs, providing essential tools for governance, security, policy management, and budget oversight. With features like a multi-provider architecture, policy engine, and real-time analytics, it simplifies API integrations while enhancing security and compliance.
TensorWall is an innovative, developer-focused API gateway tailored for large language model (LLM) services. It provides an integrated solution that prioritizes governance and security, enabling seamless connections to multiple LLM providers.
Key Features
| Feature | Description |
|---|---|
| OpenAI Compatible | Acts as a drop-in replacement for /v1/chat/completions and /v1/embeddings. |
| Multi-Provider Support | Supports various LLM providers including OpenAI, Anthropic, Ollama, LM Studio, and AWS Bedrock. |
| Policy Engine | Implements fine-grained access control before LLM calls. |
| Budget Control | Enforces per-application spending limits with alert functionalities. |
| Audit Logging | Ensures full traceability for all requests and responses. |
| Cost Tracking | Provides real-time analytics on usage per application and feature. |
| Security Guards | Detects prompt injection and secures sensitive information. |
Architecture Overview
┌─────────────────────────────────────────────────────────────┐
│ Client App │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ TensorWall │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Auth │→ │ Policy │→ │ Budget │→ │ Audit │ │
│ │ Check │ │ Engine │ │ Check │ │ Log │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ OpenAI │ │Anthropic│ │ Ollama │
└─────────┘ └─────────┘ └─────────┘
API Endpoints Overview
LLM Proxy Endpoints
POST /v1/chat/completions # Chat completion
POST /v1/embeddings # Text embeddings
Admin API Endpoints
GET/POST /admin/applications # Manage applications
GET/POST /admin/policies # Manage policies
GET/POST /admin/budgets # Manage budgets
GET /admin/analytics # Access usage analytics
GET /admin/audit # Retrieve audit logs
Authentication Endpoints
POST /auth/login # Login (returns JWT)
POST /auth/logout # Logout
GET /auth/me # Retrieve current user
TensorWall is designed to simplify the management of LLM APIs by offering essential tools such as budget control, audit logging, and multi-provider capabilities within a single platform. The combination of a robust policy engine and security features makes it an ideal choice for developers looking to enhance their application’s interaction with language models.
No comments yet.
Sign in to be the first to comment.