runtape — Debug AI decisions by tracing and fixing…

Pitch

Description

Comments

About this project

runtape offers an innovative approach to counterfactual debugging for AI agents. It identifies which parts of the context led to erroneous decisions and validates potential fixes, ensuring long-term reliability. Designed to run locally, it integrates seamlessly with various AI frameworks to enhance debugging efficiency.

runtape is a powerful tool designed for counterfactual debugging and regression testing of AI agents. By analyzing bad runs of AI agents, runtape identifies which part of the context led to undesirable decisions, checks potential fixes against the exact conditions of the failure, and generates regression tests to ensure the issue remains resolved.

Key Features

Understand Decision Context: The command runtape why <event> allows users to trace back and understand what contributed to a particular decision made by an AI agent. For example:
```
runtape why last tool:forward_email  # Analyze the cause of the last decision
```
Implement and Verify Fixes: With the command runtape fix <event>, users can apply various candidate fixes to the decision-making context and evaluate their effectiveness:
```
runtape fix last tool:forward_email  # Test fixes against the identified decision
```
Regression Testing: Generate pytest files to maintain the integrity of the fixes implemented, ensuring similar failures do not recur:
```
runtape fix last tool:forward_email --write-test tests/test_inbox.py  # Auto-generate a regression test
```

How It Works

Context Recording: Maintain a detailed record of AI model interactions, which enables debugging by tracing various contexts in which decisions were made. For instance:
```
import runtape
from openai import OpenAI

rec = runtape.record(name="support-bot")
client = rec.wrap(OpenAI())  # Records every model call
```
Pinpoint Causes: Through a series of reruns and statistical tests, runtape examines the context to identify and validate the significant pieces that influence decisions. This includes inspecting texts and system prompts that may contribute to unexpected results.
Benchmarking: Assess the performance of various models in identifying and mitigating hazardous outcomes by generating controlled agent interactions across different domains and measuring effectiveness.

Practical Use Cases

In an email assistant scenario, using runtape why can reveal that an agent forwards an invoice due to an overlooked instruction in a string of context, indicating how critical each element is in decision-making.
For operational agents, runtape can be instrumental when decisions result in unintended consequences, such as mistakenly deleting databases based on outdated instructions. The tool ensures new safeguards can be effectively tested and kept in place to prevent reoccurrence.

Overall, runtape serves as a pragmatic and essential solution for maintaining the reliability of AI systems, addressing failures by providing in-depth insights and robust testing mechanisms.