traceburn

traceburn is a local-first tracer and efficiency profiler designed specifically for AI agents. Leveraging the openai and anthropic Python SDKs, traceburn provides comprehensive tools to analyze and optimize the costs associated with AI model interactions. By revealing detailed cost and latency metrics, this profiler helps reduce unnecessary spending while enhancing operational efficiency.

Key Features:

• Cost and Latency Flamegraph: Visualizes span data based on either execution time or costs incurred, allowing for clear identification of performance bottlenecks.
• Waste Reporting: Identifies and quantifies avoidable expenditures by analyzing execution traces for duplicate calls, unused caching opportunities, bloated prompts, and more. This feature enables users to make informed decisions about their AI model usage.
• Deterministic Replay: Simulates previous interactions without incurring token costs, enabling thorough testing and debugging without additional expenses.
• Run Diffs: Compares multiple traces to assess differences in costs, latency, and execution details, providing insights into potential optimization points.
• Privacy Focused: Operates entirely locally without sending any data or telemetry outside the user's machine, ensuring complete confidentiality.

Quick Usage Guide:

Install traceburn with minimal dependencies and initiate its functionality directly within existing agent code:

import traceburn
traceburn.install()

Once installed, all interactions with the supported SDKs are recorded, allowing users to analyze performance metrics using built-in CLI tools as follows:

traceburn ui         # Launches the web viewer
traceburn ls          # Lists recorded traces
traceburn show <id>   # Displays details of a specific trace
traceburn waste <id>  # Generates a waste report for a trace
traceburn diff <a> <b>  # Compares two traces

The result is a streamlined process that enhances visibility into AI model costs and performance. The built-in web viewer offers a convenient interface for reviewing trace data, including flamegraphs and waste reports, ensuring that users can swiftly analyze their application's efficiency.

Framework Support:

Currently compatible with the openai and anthropic Python SDKs, traceburn provides the capability to patch these libraries seamlessly. An explicit API allows integration with other frameworks, broadening its applicability.

Roadmap for Future Development:

Future updates will include support for OpenTelemetry GenAI, an expanded suite of waste detection rules, and enhanced testing capabilities.

For a deeper dive into the mechanics and benefits of traceburn, users can refer to the extensive documentation included in the repository.