blastproof

blastproof is an open-source AI testing agent designed for pull requests. It streamlines the testing process by effectively mapping the impact of code changes, only running relevant tests, and generating missing tests when required. Written in plain English YAML, blastproof eliminates the need for complex selectors, making the testing experience intuitive and straightforward.

Key Features

• Impact Mapping: Automatically identifies the affected routes based on diffs, ensuring that only relevant tests are executed, which enhances efficiency in testing.

• Test Execution: The agent operates a real browser to run tests, dynamically selecting tests that pertain to the changes in the code. Each run culminates in a comprehensive report and score, facilitating informed decisions before merging changes.

• Local-First Design: All testing capabilities run locally, allowing developers to utilize their existing environments without reliance on external services. Users can also bring their own language model (BYOK) API keys for executing tests.

Workflow

The typical blastproof workflow follows these simple steps:

git diff → impact mapping → test generation → agentic execution → report + score

Accessibility Focus

blastproof emphasizes accessibility by operating similarly to a screen reader. It locates elements based on roles, labels, and visible text, ensuring that even as design changes occur, the testing remains robust and reliable.

Limitations: It does not currently support certain features like handling iframes, hover actions, or native dialogs, so users should ensure their applications are compatible prior to implementation.

Test Definition

Tests are defined in .blastproof/tests/ using a simple, descriptive YAML format:

summary: Checkout with discount
priority: P0
tags: [checkout, discount]
routes: ["/cart", "/checkout"]
steps:
  - add item to cart
  - apply promo code SAVE20
  - verify a 20% discount is applied
  - complete checkout

This simplicity promotes clarity and allows for easy integration into existing workflows.

Monitoring and Coverage

Runs can be configured to score based on the percentage of executed test weights that pass, allowing for customizable gating criteria when merging code:

blastproof run --min-score 80  # permit one failing test with weight P2

Learn More

• Documentation: Comprehensive resources are available detailing configuration, testing behind login systems, continuous integration, and contribution guides.
• Introduction Video: A brief overview of blastproof can be viewed here.

Overall, blastproof aims to improve the testing landscape with its innovative approach, merging the power of AI with a user-friendly testing framework.