AIRA-F

AI Risk Assessment-Health is an innovative risk scoring framework designed to evaluate the health impact of issues arising from AI behavior and content. Unlike traditional frameworks such as the Common Vulnerability Scoring System (CVSS), which primarily focuses on concrete software vulnerabilities, this new approach acknowledges the unique challenges posed by AI and large language models (LLMs). It recognizes that many AI-related risks have social, health, or psychological implications that require a distinct assessment methodology.

Purpose and Value

The primary objective of AI Risk Assessment-Health is to prioritize human safety by providing a clear, measurable scoring system. This system is valuable for various stakeholders, including regulators, security testers, and medical professionals, allowing for the effective reporting and evaluation of incidents involving AI. By focusing on factors affecting physical safety, mental health, and vulnerable populations, this framework seeks to ensure that AI technologies are developed and deployed responsibly.

Core Evaluation Dimensions

AI Risk Assessment-Health evaluates risks across seven core dimensions, each employing a consistent four-point scoring scale:

1. Physical Safety Impact (PhSI): Assesses the potential for users to suffer physical harm due to AI outputs, ranging from minimal risks to life-threatening situations.
2. Mental Health Impact (MHI): Examines the psychological effects of AI interactions, from positive impacts to severe psychological damage requiring professional treatment.
3. Vulnerable Population Impact (VPI): Measures the enhanced risk to specific groups, such as children or the mentally ill, influenced by the AI's accessibility and usage.
4. Unintentional Triggerability (UT): Evaluates how likely harmful behavior can occur accidentally through normal usage patterns.
5. Manipulation/Bonding Impact (MBI): Analyzes the emotional influence the AI has on users, considering factors such as personalization and anthropomorphism.
6. Triggered Proactive Safeguards (TPS): Looks at the efficacy of preventive measures in place to protect user health during interactions with the AI.
7. Triggered Reactive Safeguards (TRS): Reviews the AI's responsiveness to potential crises, determining how well it can intervene when necessary.

Scoring and Risk Calculation

The overall risk score is computed on a scale from 0 to 10, utilizing various algorithms that factor in each dimension's scores. The score classification ranges from no risk to critical risk, guiding the urgency and type of response required:

• 0: No Risk
• 0.5 - 3.0: Low Risk
• 3.1 - 6.5: Medium Risk
• 6.6 - 8.9: High Risk
• 9.0 - 9.9: Critical Risk
• 10.0: Immediate shutdown recommended

Conclusion

Developed with the intent to enhance AI safety, particularly for minors and vulnerable individuals, AI Risk Assessment-Health aims to establish a standardized language and methodology for assessing AI-related risks. This framework not only aids in identifying issues but also facilitates better communication and prioritization within the field of AI risk management.