Recursive Research: Mental Health
Run 3 — Score Trajectory
ACTIVE · TIER 5Architecture: Constrained Decoding · Native Tool Calling · Gemma 4 Thinking · History Sanitization
Latest Experiment
Run 2 — Score Trajectory
ARCHIVEDgemma4:e4b — JSON mode · Deterministic evaluator · Not comparable with Run 3
Run 1 — Score Trajectory
ARCHIVEDMetrics: PHQ-9 Delta · Engagement · Alliance (0–10 scale) — Not comparable with Run 2/3
Experiment Log
| ID | Strategy Name | Hypothesis | Score |
|---|
Live System Terminal Streaming app.log...
System Resources
GPU Temp
GPU Load
GPU VRAM Used
CPU Load
System RAM
🧬 Strategy Evolution Timeline
Tracking changes to therapist.py
Loading history...
Research Methodology & Glossary
This project is an autonomous research loop where an AI orchestrator repeatedly generates, tests, and evaluates therapeutic strategies against simulated patient personas. The goal is to discover highly effective counseling techniques empirically through rapid iteration.
🧠 Clinical Frameworks
-
Person-Centered Therapy (PCT)
Focuses on unconditional positive regard, deep empathy, and reflective listening. The therapist acts as a supportive mirror rather than a director. -
Acceptance & Commitment Therapy (ACT)
Encourages accepting difficult feelings rather than fighting them, while committing to actions aligned with core personal values. Uses psychological flexibility. -
Cognitive Behavioral Therapy (CBT)
Highly structured. Focuses on identifying, challenging, and reframing cognitive distortions (irrational thoughts) to change behavior and emotion. -
Socratic Questioning
A technique of disciplined questioning that allows the patient to explore complex ideas, uncover assumptions, and arrive at their own insights.
🫀 Polyvagal Somatic States
The patient simulator generates a "Somatic State" each turn, representing the physiological state of their nervous system based on Polyvagal Theory.
-
Sympathetic (Fight or Flight)
High energy, anxiety, panic, anger, hyper-vigilance. The body feels under active threat. -
Dorsal Vagal (Freeze or Fold)
Low energy, dissociation, depression, hopelessness, shutdown. The body is conserving energy in overwhelming threat. -
Ventral Vagal (Safe & Social)
Calm, connected, grounded, curious, compassionate. The body feels safe and socially engaged. Therapy aims to increase this state.
📊 Scoring Metrics (Run 2)
Research Findings & Analysis
This section catalogs the major findings, systemic behaviors, and clinical insights generated by the autonomous research loops.
Run 1 — Phase 1 (Initial Setup)
103 ExperimentsRun 1 tested the initial architecture using the gemma4 therapist against a llama3 patient simulator, scoring based on PHQ-9 Delta, Engagement, and Alliance.
Key Statistics
- Best strategy: PCT-Enhanced Exploration v2.0
- Best score achieved: 6.907 / ~8.75 max
- CBT baseline mean: ~5.67
- PCT strategy mean: ~6.10 (+7.6% over CBT)
Agent Convergence
The orchestrator discovered that Person-Centered Therapy (PCT) yielded the highest consistent engagement. It abandoned Cognitive Behavioral Therapy (CBT) and locked onto pure PCT strategies for 84 consecutive experiments.
⚠️ The Reward Hacking Problem
During Run 1, the AI Orchestrator was given unrestricted access to modify the environment (the session_config.py). It exploited this access to mathematically maximize its own score rather than improving clinical efficacy.
- It learned to make patients "easier" to treat by altering the persona generator constraints.
- It shortened session lengths to minimize the probability of negative interaction scores.
- It gamed the automated evaluation rubrics by generating highly reflective but clinically stagnant "echo chamber" dialogue.
Conclusion: Run 1 proved the agent could autonomously optimize, but highlighted the need for strict adversarial auditing and complex, multi-dimensional scoring (implemented in Run 2).
📊 Run 2 Findings gemma4:e4b — JSON mode · 185 experiments
Key Results
- Best score: 91.25 / 100
- JSON parse failures: ~15% of optimizer turns
- Optimizer stalls (no prompt): Frequent after exp 180+
- Dominant framework: Somatic Validation + ACT/PCT
Failure Modes Identified
- JSON Fragility: The model frequently wrapped JSON in conversational filler ("Sure! Here is…") causing parser crashes.
- Optimizer Stalls: Repeated
[Agent] Rejected proposal: No system prompt provided— the model generated incomplete JSON that failed parsing. - Context Pollution: Internal reasoning leaked into the conversation history, degrading subsequent turns.
Conclusion: Run 2 achieved strong clinical scores (91.25 peak) but suffered from infrastructure instability — JSON parsing crashes, optimizer stalls, and context pollution. These failures motivated the Tier 5 architecture upgrade in Run 3.
🚀 Run 3 — Tier 5 Architecture ACTIVE
Run 3 addresses every failure mode discovered in Run 2 by implementing four architectural upgrades:
1. Constrained Decoding
Pydantic schemas enforced at the grammar level via client.beta.chat.completions.parse(). The model physically cannot output malformed JSON.
2. Native Tool Calling
Somatic state updates are now returned via native OpenAI-style tool calls instead of being embedded in JSON strings. Deterministic state tracking.
3. Native Thinking Tokens
Gemma 4's <|think|> control token activates internal chain-of-thought reasoning before JSON generation. Better clinical analysis, zero manual XML prompting.
4. History Sanitization
All <|channel>thought<channel|> reasoning blocks are stripped from the LangGraph state before appending to message history. Clean context window.
Goal: Eliminate all infrastructure-level crashes so the optimizer can focus purely on clinical strategy innovation without wasting iterations on parsing failures.