HackerRank Orchestrate

Starter repository for the HackerRank Orchestrate 24-hour hackathon.

Current router version: 3.1.0 (code/VERSION)

Message Notification Router

Build an AI-powered system for WhatsApp that decides which messages deserve immediate attention, which should wait, and which should be muted.

The system must reason over multimodal messages, including text messages, image posters/screenshots, and voice notes.

WhatsApp is noisy. A user can receive family chats, society notices, school updates, co-worker messages, business account promotions, image posters, voice notes, and scams in the same message stream. Treating every message the same creates two bad outcomes: important messages get missed, and unwanted or risky messages interrupt the user.

Read problem_statement.md for the full task spec, input/output schema, allowed values, and submission format.

---

Repository Layout

.
├── AGENTS.md                         # Rules for AI coding tools + transcript logging
├── problem_statement.md              # Full challenge statement
├── README.md                         # You are here
└── dataset/
    ├── messages.csv                  # Messages to route
    ├── output.csv                    # Blank submission template
    ├── sample_messages.csv           # Solved examples
    ├── users.csv                     # User notification behavior
    ├── groups.csv                    # Group metadata
    ├── group_members.csv             # User-group relationships
    ├── business_accounts.csv         # Business sender metadata
    ├── user_business_history.csv     # User-business history
    ├── message_history.csv           # Historical messages
    ├── message_events.csv            # User reactions to historical messages
    ├── images.csv                    # Image IDs and media file paths
    ├── voice_notes.csv               # Voice note IDs and media file paths
    ├── daily_notification_summary.csv
    └── media/
        ├── images/
        └── audio/

---

What You Need to Build

For every row in dataset/messages.csv, produce one row in output.csv with:

Column	Meaning
message_id	Incoming message ID
action	One of notify, digest, or mute
message_type	Best-fit message category
reason	Short human-readable explanation
confidence	Number from 0 to 1
evidence_message_ids	Historical message IDs used as evidence; write none if there is no useful evidence

Your system should make personalized decisions using the provided message, user, group, business, media, and historical interaction data. For image and voice-note messages, images.csv and voice_notes.csv only provide file paths; your system should inspect the media files themselves.

---

Suggested Workflow

1. Inspect dataset/sample_messages.csv to understand the expected output format.
2. Load dataset/messages.csv and all relevant context files.
3. Build your routing system using any approach: LLMs, retrieval, rules, classifiers, agents, or hybrids.
4. Write predictions to output.csv.
5. Evaluate your approach on the solved sample rows before submitting.

You may use any language or runtime. Python, JavaScript, and TypeScript are all reasonable choices.

---

Requirements

Your solution must:

• be runnable from the terminal
• read the provided files from dataset/
• produce a valid output.csv
• include one prediction for every message_id in dataset/messages.csv
• not use organizer-only files or hardcoded labels

If you use API keys or secrets, read them from environment variables. Never hardcode secrets in the repo.

---

Evaluation

Your output.csv will be compared against hidden ground-truth labels.

The scoring will consider:

• correctness of action
• correctness of message_type
• usefulness and consistency of reason
• whether evidence_message_ids point to relevant historical messages
• reasonable confidence calibration

Strong systems will combine retrieval, structured metadata, behavioral history, safety checks, OCR/ASR handling, and contextual reasoning.

---

Chat Transcript Logging

This repo includes an AGENTS.md file for AI coding tools. It asks compatible tools to append conversation summaries to:

Platform	Path
macOS / Linux	$HOME/hackerrank_orchestrate_august26/log.txt
Windows	%USERPROFILE%\hackerrank_orchestrate_august26\log.txt

Upload this log as your chat transcript at submission time. Do not paste secrets into the chat.

---

Submission

Submit the following files as instructed by HackerRank:

1. Code zip: full runnable solution, prompts/configs, README, and any evaluation files.
2. Predictions CSV: final output.csv for all rows in dataset/messages.csv.
3. Chat transcript: the log.txt described above.

Before submitting, confirm:

• output.csv has one row per row in dataset/messages.csv.
• output.csv has the exact required columns in the exact required order.
• Your runnable code and setup instructions are included in code.zip.

Final Submission Runbook

Recommended final steps from repo root:

1. Generate full predictions and markdown report:

python code/main.py --messages --write-md-report

1. Copy the newest generated messages output into the submission template path:

# Bash (Git Bash / WSL style)
latest=$(ls -1t reports/output/*/output-messages-*.csv | head -n 1)
cp "$latest" dataset/output.csv

1. Sanity-check row count and schema quickly:

python - <<'PY'
import csv
from pathlib import Path
messages = Path("dataset/messages.csv")
output = Path("dataset/output.csv")
with messages.open(newline='', encoding='utf-8') as f:
    m_rows = sum(1 for _ in csv.DictReader(f))
with output.open(newline='', encoding='utf-8') as f:
    reader = csv.DictReader(f)
    o_rows = sum(1 for _ in reader)
    cols = reader.fieldnames
print("messages rows:", m_rows)
print("output rows:", o_rows)
print("columns:", cols)
PY

Expected columns:

• message_id,action,message_type,reason,confidence,evidence_message_ids

1. Package for submission:

• code zip with runnable solution + README
• dataset/output.csv
• chat transcript log from %USERPROFILE%/hackerrank_orchestrate_august26/log.txt

Final Behavior Highlights

• Deterministic scam detection for login-link coercion and advance-fee release-funds lures.
• Legitimate verified-business promotions are classified as promotion (not generic spam) when no scam/credential-harvest cues exist.
• Promotions opt-out history is respected, enabling mute/promotion for unwanted promotional content.
• Translation observability is explicit in terminal logs and markdown reports (translation_requested, translation_applied).
• Related message context in reports now includes per-message message_events inline under each related_messages entry for faster review.
• Trusted business-domain exceptions are supported for specific accounts (business_092, business_032) so domain mismatch does not cause instant mute/spam by itself.
• Group payment-link messages from non-admin senders with screenshot-proof requests are deterministically treated as scam (mute/scam).

---

Quickstart (Python Router)

If you are using the provided Python router in code/main.py, run from repo root:

python code/main.py --sample --write-md-report

This runs against dataset/sample_messages.csv (default-friendly mode) and writes:

• output CSV to reports/output/YYYYMMDD/output-sample-<timestamp>.csv
• markdown report to reports/reports/YYYYMMDD/report-sample-<timestamp>.md

From v3.1.0 onward, when translation is triggered for non-English or romanized Hindi text, runtime logs include explicit translation events and markdown reports include translation_requested / translation_applied fields.

Run on full challenge input:

python code/main.py --messages --write-md-report

Useful mode flags:

• --sample = use dataset/sample_messages.csv
• --messages = use dataset/messages.csv
• --message-id <id> = run only one message row from the selected CSV (fast validation mode)

Optional override:

• --messages-file <path> to point to a custom CSV. If a relative filename is provided and not found in the current folder, it will be resolved under dataset/ when available.

Fast single-message validation examples:

python code/main.py --messages --message-id msg_016 --write-md-report
python code/main.py --sample --message-id sample_msg_015 --write-md-report

When --message-id is used, report filenames include the message ID after the timestamp:

• reports/reports/YYYYMMDD/report-<dataset-tag>-<timestamp>-<message-id>.md

Translation Observability

When translation runs, terminal logs include per-message events such as:

• Translation requested for message_id=...
• Translation cache hit for message_id=...
• Translation applied for message_id=...

Markdown reports also show:

• Translation requested: yes|no
• Translation applied: yes|no
• Translated message text (en): ... when a translation is available

Note: required prediction CSV columns are unchanged.