Riskratchet: A Maintainability Ratchet for AI-assisted Python

Riskratchet is a powerful tool designed to enhance the maintainability of Python code by effectively managing risks associated with code complexity and testing. It systematically computes a per-function risk score based on several factors such as coverage gaps, cyclomatic complexity, and churn, ensuring that the risk level never increases beyond a predefined baseline.

Key Features

• Monitors and evaluates code maintainability metrics automatically during the CI process.
• Prevents the degradation of code quality by blocking commits or failing builds when risk scores rise above acceptable thresholds.
• Integrates with continuous integration workflows to promote a solid coding standard without manual oversight.

The Problem It Solves

AI coding agents excel at generating code that is functional; however, they often fall short in the following critical areas:

• Crafting meaningful tests for new code.
• Detecting unnoticed increases in function size and complexity.
• Identifying exposed functions in public APIs that lack sufficient test coverage.
• Recognizing when a minor refactor substantially complicates code.

Riskratchet addresses these challenges by serving as an automated safeguard, computing risk scores that highlight potential issues while allowing teams to maintain their focus on development.

Workflow Overview

1. Install Riskratchet (Use pip or run without installation):

   pip install riskratchet
   # or run without installing
   uvx riskratchet --help
   

2. Run Tests with Coverage to generate a JSON report:

   pytest --cov --cov-report=json:coverage.json
   

3. Create a Baseline that defines the current acceptable risk profile:

   riskratchet baseline src --coverage coverage.json --output .riskratchet.json
   

4. Scan Code for risks based on the established profile:

   riskratchet scan src --coverage coverage.json
   

5. Check for Regressions after a commit:

   riskratchet check src --coverage coverage.json --baseline .riskratchet.json
   

   The check command returns 1 for regressions, 2 for usage errors, and 0 for a successful run.

The GitHub Action Integration

Riskratchet offers seamless integration as a GitHub Action, making it easy to include in pull request workflows. It automatically checks for quality violations and provides insightful comments directly on PRs to inform contributors of any issues:

# .github/workflows/riskratchet.yml
on: [pull_request]
jobs:
  riskratchet:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: KayhanB21/riskratchet@v0.2.13
        with:
          coverage: coverage.json

Advanced Scoring System

The Riskratchet scoring system evaluates each function based on six components:

Conclusion

Riskratchet is an essential tool for Python developers aiming to maintain a high standard of code quality while leveraging AI-assisted coding. By effectively identifying and managing risks associated with complexity and testing, it empowers teams to focus on innovation rather than administrative oversight. Explore more on the PyPI project page and start improving code maintainability today.