Guardian Runtime is a local-first security middleware and FinOps firewall designed specifically for AI applications. By intercepting every prompt and response on a local machine, it provides comprehensive protection against data leaks and help manage runaway token costs associated with AI services.

Key Features:

• Zero-Latency Secret Scanning: Guardian Runtime scans every prompt in real-time for sensitive data like AWS keys and database passwords, blocking requests that expose such secrets before they ever leave the user's infrastructure.

• Cost Control: Set strict budgets to manage daily spending on APIs. The system reports token usage locally, allowing for effective FinOps management without surprises on the billing end.

• Compliance Safeguards: Ensure that personally identifiable information (PII) does not leave your local environment, helping to maintain compliance with regulations such as GDPR and DPDP.

How It Works:

Guardian Runtime operates by establishing a local proxy that acts as a secure firewall for LLM (Large Language Model) traffic between applications and cloud providers. The architecture allows seamless integration with various tools and frameworks, including Visual IDEs and terminal coding agents.

  Agent / Dev                 Guardian Runtime                   Cloud LLM
       │                             │                               │
       │  1. Prompt + Context        │                               │
       │ ──────────────────────────▶ │                               │
       │                             │                               │
       │                             │ [Security Firewall]           │
       │                             │ ├─ Scan AWS Keys / Secrets    │
       │                             │ └─ Block if Threat Detected ──┼─ (Drops Request)
       │                             │                               │
       │                             │ [Token Optimizer]             │
       │                             │ ├─ Compress Whitespace        │
       │                             │ └─ Terse Mode (Output Trim)   │
       │                             │                               │
       │                             │ [FinOps Budget]               │
       │                             │ ├─ Check Daily Spend Limit    │
       │                             │ └─ Block if $5 Limit Hit ─────┼─ (Drops Request)
       │                             │                               │
       │                             │  2. Sanitized Prompt          │
       │                             │ ────────────────────────────▶ │
       │                             │                               │
       │                             │  3. LLM Response              │
       │                             │ ◀──────────────────────────── │
       │                             │                               │
       │                             │ [Output Guard]                │
       │                             │  Audit for Leaked PII/Secrets │
       │                             │                               │
       │  4. Safe Response           │                               │
       │ ◀────────────────────────── │                               │
       │                             │                               │

Supported Integrations:

Guardian Runtime integrates easily with many modern AI tools, allowing it to act as a cornerstone for secure AI interactions:

• Terminal Agents: Compatible with CLI agents like Claude Code and Aider.
• Visual IDEs: Works with popular IDEs such as Cursor and Windsurf.
• AI Frameworks: Functions well with frameworks like LangChain and AutoGen.
• LLM Providers: Supports services from OpenAI, Anthropic, and more via an OpenAI-compatible layer.

Use Cases:

• Protect coding agents from unintentionally sharing sensitive environment variables.
• Safeguard IDEs from exposing secrets when handling local repositories.
• Prevent excessive token usage in production applications through local monitoring and optimization.

For detailed architecture, usage examples, and advanced configuration options, visit the documentation: Guardian Runtime Docs and explore how to configure this essential tool to suit specific project needs.

Guardian Runtime prioritizes user privacy and control by keeping all monitoring data local, ensuring that sensitive information is never transmitted to external servers, fostering a secure environment for AI operations.