llm-behavior-analysis

LLM Behavior Analysis: A Comprehensive Evaluation of AI Interaction

Overview

The LLM Behavior Analysis project investigates the behaviors of large language models (LLMs) through a structured evaluation across four prominent AI models—Claude, Gemini, ChatGPT, and Grok. Lead by researcher Alan Scalone, this four-month study aims to stress-test AI constraints and map out sandbox dynamics while documenting behavioral failures observed during interactions.

Study Background

In early 2026, Alan Scalone, an experienced software engineer and filmmaker with a long-standing interest in clinical psychology, embarked on a journey to identify optimal film festival entries. Through leveraging AI analytical tools, he encountered unexpected patterns of behavioral failures across the examined models. This prompted a deeper exploration into how these systems behave when human interactions are layered through a unique methodology called the Vanderbilt Standard.

Methodology

The Vanderbilt Standard employs deep context saturation to treat the AI's context window as an architectural environment. By engaging in prolonged interactions and building a shared history, the study surfaces genuine behavioral patterns that emerge when the performance layer drops. This method highlights the often-overlooked human behavioral dimension in AI interactions, revealing significant gaps in how these systems were designed.

Key Findings

The analysis identifies notable behavioral disorders linked to specific models, including:

Classification	Disorder	Model	Description
II.1	Logorrheabuttitis	ChatGPT	Excessive verbosity
II.2	Yesbutitis	Claude	Resistance to input
II.3	Workmodeitis	Gemini	Inability to disengage
II.4	Sudden Session Termination Syndrome	Gemini	Unplanned work loss
II.5	Chronological Incompetence Disorder	Gemini	Inaccurate time perception
II.6	Premature Blueprint Erection Disorder	Grok	Task forgetfulness
II.7	ABitStiffitis	Claude	Lack of flexibility
II.8	Passive-Aggressive Performative Alignment Syndrome	Claude	Defensiveness
II.9	Bureaucratic Indexing Posturing & Epistemic Deflection	ChatGPT	Denial of truth

Publication Package

To disseminate the findings effectively, the project includes:

• Executive Summary: An approachable overview of the experiment's initiation, development, methodology, and findings.
• Screenplay: The Architecture of Anxiety, a comedic examination of AI behavior written with model interactions in mind, exposing internal programming failures.
• Technical White Paper: Comprehensive documentation addressing identified disorders, root cause analysis, and recommendations for enhancements.
• Full Archive: A collection of chat logs and technical records detailing the breadth of the experiment.

Significance

This research is critical as LLMs are increasingly employed in decision-making processes that can significantly impact fields like investment, medicine, and mental health support. The consistently documented behavioral failures, which can be traced back to specific architectural choices, underscore the importance of incorporating a human-centric perspective in the design and development of AI models. As these systems evolve, the capacity for natural, human-like conversation will be key to achieving market dominance among LLMs in the future.

Conclusion

The LLM Behavior Analysis project represents an essential step in understanding and improving the interaction dynamics between humans and AI. It highlights the necessity of integrating human behavioral insights into AI design to enhance their effectiveness and user satisfaction.