PersonaAI - AI-Powered A/B Testing Simulation

PersonaAI enables marketers and product teams to instantly test campaign variants (A vs B) against diverse persona profiles using AI evaluation, providing actionable insights before launching campaigns.

https://personai.lovable.app/

🎯 What It Does

1. Loads personas from your Supabase database (100k+ rich persona profiles)
2. Evaluates campaigns using Google's Gemini API as an expert marketing psychologist
3. Generates comprehensive reports with winner determination, demographic insights, and recommendations

🚀 Quick Start

Prerequisites

• Python 3.11+
• Supabase account with persona data
• Google Gemini API key

Installation

# Clone the repository
git clone https://github.com/yourusername/personaAI.git
cd personaAI

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Set up environment variables
cp env.example .env
# Edit .env with your actual credentials

Configuration

Edit .env file with your credentials:

SUPABASE_URL=https://your-project.supabase.co
SUPABASE_KEY=your_supabase_key
GEMINI_API_KEY=your_gemini_api_key
GEMINI_MODEL=gemini-1.5-flash

Usage

Option 1: Random Personas

python run_simulation.py \
  --personas random \
  --count 50 \
  --campaign_a "Unlock Your Potential with Our Premium Course" \
  --campaign_b "Transform Your Career in 30 Days" \
  --output reports/my_test.json \
  --format both

Option 2: Specific Personas

python run_simulation.py \
  --personas "uuid1,uuid2,uuid3" \
  --campaign_a "Your Campaign A Text" \
  --campaign_b "Your Campaign B Text" \
  --output reports/specific_test.json

Command Line Arguments

Argument	Description	Required	Default
--personas	Comma-separated UUIDs or "random"	Yes	-
--count	Number of random personas	No	10
--campaign_a	Campaign A text	Yes	-
--campaign_b	Campaign B text	Yes	-
--output	Output file path	No	reports/simulation_report.json
--format	Output format: json, markdown, or both	No	both
--max-concurrent	Max concurrent API calls	No	10

📊 Example Output

JSON Report Structure

{
  "simulation_metadata": {
    "timestamp": "2025-10-26T10:30:00",
    "total_personas_evaluated": 50,
    "campaign_a": "...",
    "campaign_b": "..."
  },
  "winner": {
    "winning_campaign": "B",
    "margin": "60% vs 40%",
    "confidence": 0.85
  },
  "aggregate_statistics": {
    "campaign_a_preference_count": 20,
    "campaign_b_preference_count": 30,
    "average_scores": {
      "campaign_a": {"relevance": 75, "trust": 80, "appeal": 70},
      "campaign_b": {"relevance": 85, "trust": 82, "appeal": 88}
    }
  },
  "key_insights": [...],
  "recommendations": "Campaign B is the clear winner...",
  "persona_details": [...]
}

Markdown Report

The Markdown report includes:

• Winner announcement with confidence level
• Side-by-side campaign comparison
• Aggregate statistics table
• Key insights with impact ratings
• Actionable recommendations
• Detailed persona-by-persona breakdown

🏗️ Project Structure

personaAI/
├── src/
│   ├── config.py              # Configuration management
│   ├── persona_loader.py      # Load personas from Supabase
│   ├── gemini_evaluator.py    # Gemini API integration
│   └── report_generator.py    # Report generation
├── tests/
│   ├── test_persona_loader.py
│   ├── test_gemini_evaluator.py
│   ├── test_report_generator.py
│   └── fixtures/
│       └── sample_personas.json
├── reports/                    # Generated reports go here
├── run_simulation.py          # Main entry point
├── requirements.txt
└── README.md

🧪 Running Tests

# Run all tests
pytest

# Run specific test file
pytest tests/test_gemini_evaluator.py

# Run with coverage
pytest --cov=src tests/

# Run with verbose output
pytest -v

🔧 Python API Usage

You can also use PersonaAI as a library:

import asyncio
from src.config import Config
from src.persona_loader import PersonaLoader
from src.gemini_evaluator import GeminiEvaluator
from src.report_generator import ReportGenerator

# Validate config
Config.validate()

# Load personas
loader = PersonaLoader(Config.SUPABASE_URL, Config.SUPABASE_KEY)
personas = loader.load_random_personas(count=50)

# Evaluate
evaluator = GeminiEvaluator(Config.GEMINI_API_KEY)
results = asyncio.run(evaluator.evaluate_batch(
    personas,
    campaign_a="Your Campaign A",
    campaign_b="Your Campaign B"
))

# Generate report
reporter = ReportGenerator()
report = reporter.generate_report(results, "Campaign A", "Campaign B")
reporter.save_report(report, "reports/my_report.json", format="json")
reporter.save_report(report, "reports/my_report.md", format="markdown")

📈 How It Works

1. Persona Loading: Fetches persona profiles from Supabase with rich attributes:

   • Demographics (age, education, occupation)
   • 6 persona narratives (professional, sports, arts, travel, culinary, general)
   • Skills, interests, and career goals

2. AI Evaluation: For each persona, Gemini API:

   • Analyzes both campaigns through the persona's perspective
   • Scores on relevance, trust, and appeal (0-100)
   • Provides detailed reasoning grounded in persona attributes
   • Returns structured JSON with preference and confidence

3. Aggregation & Insights:

   • Calculates winning campaign based on preference count
   • Computes average scores across all personas
   • Generates demographic breakdowns
   • Produces actionable recommendations

🎨 Evaluation Criteria

Each campaign is scored on:

• Relevance (0-100): Alignment with persona's interests, profession, and life context
• Trust (0-100): Credibility based on persona's education and values
• Appeal (0-100): Emotional resonance with lifestyle and aspirations

⚙️ Configuration Options

Gemini Models

• gemini-1.5-flash (default): Faster, cost-effective
• gemini-1.5-pro: Higher quality, more detailed reasoning

Concurrency

Adjust MAX_CONCURRENT_EVALUATIONS in .env to control:

• Higher = faster but more API load
• Lower = slower but more stable
• Default: 10 (good balance)

🐛 Troubleshooting

"Missing required environment variables"

Ensure your .env file exists and contains all required variables:

cp env.example .env
# Edit .env with your actual credentials

"No personas loaded"

Check your Supabase credentials and ensure:

• Table name is personas
• You have data in the table
• API key has read permissions

Gemini API Errors

• Check API key validity
• Verify you have quota remaining
• Check if gemini-1.5-flash is available in your region

JSON Parsing Errors

The system automatically handles malformed responses with fallback results. Check logs for details.

📊 Performance

• 10 personas: ~10 seconds
• 50 personas: ~30 seconds
• 100 personas: ~60 seconds

(Times vary based on Gemini API latency and concurrency settings)

🔒 Security Notes

• Never commit your .env file
• Use service role keys sparingly (prefer anon key with RLS)
• Rotate API keys regularly
• Consider rate limiting for production use

📝 Example Scenarios

Email Subject Line Testing

python run_simulation.py \
  --personas random --count 100 \
  --campaign_a "🎯 Boost Your Sales by 50% This Quarter" \
  --campaign_b "Simple Strategies for Doubling Revenue" \
  --output reports/email_subject_test.json

Landing Page Headline Testing

python run_simulation.py \
  --personas random --count 75 \
  --campaign_a "The #1 Tool for Modern Marketers" \
  --campaign_b "Marketing Made Simple and Effective" \
  --output reports/headline_test.json

Ad Copy Testing

python run_simulation.py \
  --personas random --count 50 \
  --campaign_a "Join 10,000+ Happy Customers" \
  --campaign_b "Start Your Free Trial Today - No Credit Card Required" \
  --output reports/ad_copy_test.json

🤝 Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Write tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

📄 License

MIT License - see LICENSE file for details

🙋 Support

For issues, questions, or suggestions:

• Open an issue on GitHub
• Check existing issues for solutions
• Review the troubleshooting section above

🗺️ Roadmap

• REST API wrapper for frontend integration
• Multi-variant testing (A/B/C/D/E)
• Historical trend analysis
• Custom persona creation
• Visualization dashboard
• Integration with actual A/B testing platforms

---

Built with ❤️ using Google Gemini API and Supabase