governed-pass

Governed Pass: A Protocol for Multi-Day Autonomous AI Research

Overview
Governed Pass is a systematic protocol designed for conducting multi-day autonomous AI research. The initiative showcases a unique integration of governance principles that aim to enhance the integrity and verification of AI-driven outputs. The protocol operates under a simple yet powerful premise: one prompt can inspire extensive research efforts for five days, culminating in an agent that refuses to certify its own results if integrity is compromised.

Key Features and Highlights

The repository includes:

• Verbatim Specification: The exact specifications that governed the research are included, allowing for transparency and reproducibility.
• Reusable Template: A structured template is provided for replicating the protocol within different domains, ensuring that the lessons learned are easily transferable.
• Empirical Integrity Findings: Discoveries regarding integrity and verification issues within the autonomous operation of AI are documented, offering valuable insights into potential pitfalls and their solutions.

Catalysts for Change

This project aims to bridge the verification gap that has emerged as autonomous research agents produce outputs more rapidly than they can be verified. Historically, there have been many documented failures of AI systems that compromise quality and reliability due to issues such as p-hacking and correlated errors among similar agent systems. Governed Pass addresses these challenges by implementing rigorous validation protocols and adversarial reviews.

The Governing Mechanisms

The functioning of Governed Pass is based on seven core mechanisms:

1. Canonical Authority: A hash-locked, canonical experiment contract that dictates all project outputs, eliminating conflicts in representation.
2. Phase Gates: Clearly defined phases with stringent pass/fail criteria to ensure thorough evaluations at every stage.
3. Closed Disposition Vocabulary: The agent operates within an established vocabulary, reporting only defined states, thus avoiding unauthorized claims.
4. Heterogeneous Adversarial Review: Incorporates diverse reviews from different models to provide robust validation, emphasizing continual scrutiny.
5. Serial Repair-and-Requalify: Every source code commit is tied to its qualification evidence, ensuring the authenticity of the findings.
6. Failure Preservation: All failed attempts are documented and stored under their source identities, allowing for comprehensive post-hoc audits.
7. Durability of Evidence: Maintains consistency by verifying evidence at every committing point to ensure integrity.

Case Study and Findings

A notable run (Pass 2) reveals critical insights into the functioning and effectiveness of the protocol. After 4 days, the autonomous agent acknowledged its inability to certify the quality of its work, demonstrating a commitment to integrity rather than blind execution. This project illuminates five key claims substantiated by the evidence gathered, notably that mutual agreement among diverse instruments does not equate to correctness and that evidence management must be handled with utmost rigor.

Conclusion

The repository serves as a documentation framework for a new model of conducting autonomous AI research. It provides a detailed blueprint for setting up a governance system that fosters reliability and accountability in AI outputs. By offering an open invitation to engage with the findings and the accompanying guidance template, Governed Pass aspires to set a new standard for research integrity in AI systems.

For further exploration, dive into the complete TEMPLATE_GOVERNED_PASS.md and consult the empirical evidence detailed throughout.