DermaBot - Tele-Dermatology Triage Bot

A WhatsApp-based tele-dermatology triage system that uses AI to analyze skin conditions, route cases by severity, and connect patients with medical professionals.

Built for Velora 1.0 Hackathon.

Features

• WhatsApp Integration: Patients send skin images via WhatsApp for instant AI analysis
• Dual AI Analysis: HuggingFace skin classifier + Gemini Flash multimodal analysis for accuracy
• Smart Severity Routing: Low-risk cases get AI-generated remedies; high-risk cases escalate to doctors
• Conflict Resolution: When AI models disagree, cases automatically escalate to doctors
• Multilingual Support: English, Hindi, and Marathi
• Doctor Portal: Web dashboard for case review, diagnosis correction, and patient communication
• Patient Portal: QR-code accessible case records
• Periodic Follow-ups: Automated check-ins via WhatsApp with escalation on negative feedback
• ABHA Integration: Optional Ayushman Bharat Health Account identity linking

Tech Stack

Component	Technology
Backend	FastAPI (Python)
Database	MongoDB Atlas
WhatsApp	Twilio WhatsApp API
Skin Classification	HuggingFace (DermNet-trained model)
LLM	Gemini Flash (Google AI Studio)
Image Storage	Cloudinary
Frontend	React + Tailwind CSS
QR Codes	Python qrcode library
Scheduling	APScheduler

Setup

Prerequisites

• Python 3.11+
• Node.js 18+
• MongoDB Atlas account (free M0 tier)
• Twilio account with WhatsApp sandbox
• Google AI Studio API key
• Cloudinary account

Backend

cd backend
pip install -r requirements.txt
cp ../.env.example ../.env
# Edit .env with your credentials
uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000

Frontend

cd frontend
npm install
npm run dev

Twilio Webhook

For local development, use ngrok:

ngrok http 8000

Set your Twilio WhatsApp sandbox webhook to: https://your-ngrok-url/api/whatsapp/webhook

Architecture

Patient (WhatsApp) --> Twilio --> FastAPI Webhook
                                      |
                                      v
                              Image Upload (Cloudinary)
                                      |
                         +------------+------------+
                         |                         |
                    HuggingFace              Gemini Flash
                    Classifier               Multimodal
                         |                         |
                         +------------+------------+
                                      |
                                      v
                              Severity Router
                              (agree/disagree)
                         +----------+----------+
                         |                     |
                     LOW severity          HIGH/CRITICAL
                         |                     |
                    AI Remedies           Doctor Portal
                    + Follow-up           (Escalation)
                         |                     |
                         v                     v
                   WhatsApp Reply      Doctor Reviews
                   + QR Code           + WhatsApp Update

Team

Team Collective - Velora 1.0 Hackathon