ChatLLM Web

A private local model studio, AI chat, and agent workspace powered by WebGPU.

English · 简体中文

Open ChatLLM · Latest release · WebLLM · Cloudflare Pages

Local AI as a complete product

ChatLLM Web v3.2 combines Chat, a Local Agent Workspace, and Local Model Studio in one browser-native product. It inspects the current browser, recommends a model, runs WebLLM in a dedicated worker, and keeps conversations, selected files, tool results, and artifacts on the device.

• Switch each conversation between Chat and Agent modes.
• Run native WebLLM function calling with three Hermes models.
• Inspect selected files, search context, calculate, and work with saved artifacts through sandboxed local tools.
• Review every artifact write as a line diff before it is committed to IndexedDB.
• Start with 20 curated chat, coding, reasoning, vision, and tool-capable models.
• Open the Advanced catalog to browse 65 logical models backed by 163 official WebLLM records.
• Let device features select a compatible quantization while every logical model stays a single catalog item.
• See WebGPU, memory, browser storage, compatibility, cache, and runtime status.
• Download, load, switch, unload, retry, and delete models without refreshing the page.
• Attach TXT, Markdown, JSON, and common code files as direct local context.
• Import advanced MLC manifests from approved HTTPS model sources.
• Install the PWA and reuse cached models after the first successful load.
• Use the full interface in English or Simplified Chinese, light or dark.

The static application is served by Cloudflare Pages. Model assets come directly from their declared Hugging Face and WebLLM library URLs. ChatLLM has no application backend, account, API key, analytics, or telemetry.

Model catalog

Tier	Model	Declared VRAM	Capability
Stable	SmolLM2 360M	376 MB	Fast chat
Stable	Gemma 3 1B	711 MB	Compact assistance
Stable	Llama 3.2 1B	879 MB	Lower-resource chat
Stable	Qwen 3.5 0.8B	1.6 GB	Lightweight bilingual work
Stable	Qwen 3.5 2B	2.2 GB	Default everyday assistant
Stable	Llama 3.2 3B	2.2 GB	Balanced general chat
Stable	Phi-4 Mini	3.4 GB	Instruction following
Stable	Qwen 3.5 4B	3.8 GB	Higher-quality answers
Stable	Qwen 3 8B	5.6 GB	Multilingual chat
Stable	Qwen 3.5 9B	6.3 GB	Best general quality
Stable	Qwen 2.5 Coder 0.5B	945 MB	Fast code completion
Stable	Qwen 2.5 Coder 1.5B	1.6 GB	Light code explanation
Stable	Qwen 2.5 Coder 3B	2.4 GB	Code review
Stable	Qwen 2.5 Coder 7B	5.0 GB	Complex coding
Experimental	Ministral 3 3B Reasoning	2.8 GB	Compact reasoning
Experimental	DeepSeek R1 Distill Qwen 7B	5.0 GB	Long-form reasoning
Experimental	Phi 3.5 Vision	3.9 GB	Vision preview
Experimental	Hermes 2 Pro Mistral 7B	3.9 GB	Default local agent
Experimental	Hermes 2 Pro Llama 3 8B	4.9 GB	Multi-step tool use
Experimental	Hermes 3 Llama 3.1 8B	4.8 GB	Higher-quality local agent

Every curated model uses a 4K context window. ChatLLM recommends Qwen 3.5 2B when WebGPU is available and the browser reports at least 8 GB of device memory. Unknown or lower-memory devices start with Llama 3.2 1B. WebGPU features, buffer limits, and declared memory requirements drive compatibility and high-memory states.

The Advanced switch reveals the complete official catalog for power users. Quantization and 1K variants are grouped under one logical model. ChatLLM selects q4f16 when shader-f16 is available and chooses the compatible q4f32 variant where the official catalog provides one. The original model ID remains attached to each conversation and generated message.

Chat Workspace

The Beautiful UI–inspired workspace provides streaming Markdown, observable local activity, file context cards, model selection, generation settings, and a compact agent-ready prompt bar.

• + adds local files.
• @ selects files already attached to the current conversation.
• / opens Summarize, Explain, Rewrite, and Code presets.
• Enter sends; Shift+Enter adds a line; Stop interrupts generation.
• Each conversation keeps its own model, temperature, top-p, max output, and system prompt.
• When a 4K input budget is exceeded, complete older turns are omitted and the activity trace reports it.

Local Agent Workspace

Choose Agent in the Prompt Bar to start a local tool-calling run. A conversation keeps its selected mode and model. If the current model lacks native function calling, ChatLLM asks to load Hermes 2 Pro Mistral 7B.

Local tool	Access	Approval
List and read selected context files	Current run selection	Automatic
Search selected context files	Current run selection	Automatic
Calculate arithmetic	Sandboxed parser	Automatic
List and read conversation artifacts	Current conversation	Automatic
Create an artifact	Current conversation	Diff approval
Update an artifact	Current conversation	Diff approval

The Agent Run Rail records planning, tool calls, approvals, completion, stops, and errors. Each run has an eight-step safety limit. Stop preserves finished steps; Retry starts a fresh run from the original task. Approved Markdown, code, JSON, and text artifacts are stored in IndexedDB, opened in the Artifact panel, copied, or downloaded. Each artifact is limited to 64 KB and 2,000 lines so the complete approval diff stays reviewable.

Agent tools never receive arbitrary disk access. Shell execution, direct filesystem writes, network search, MCP servers, RAG, and third-party data sources stay outside the v3.2 runtime boundary.

Local Model Studio

Open /models to manage the runtime directly:

• Device insight cards for WebGPU, reported memory, browser storage, and WebLLM.
• Evidence-based model recommendation.
• Searchable catalog with Recommended, Compatible, Cached, Coding, Reasoning, Vision, Tools, Experimental, and Custom filters.
• Stable and Experimental curated tiers plus the complete Advanced WebLLM directory.
• Cache, compatibility, download, load, active, and error states.
• Explicit approvals for model downloads, external WASM, cache deletion, and memory fallback.

Local files

ChatLLM reads supported files in the browser and inserts selected text directly into the current request.

Limit	Value
Files per request	4
Size per file	64 KB
Combined context	About 2,048 tokens
Supported content	Text, Markdown, JSON, JS/TS, Python, Go, Rust, Java, CSS/HTML, YAML/TOML, SQL, shell

JSON is validated and formatted. Requests over the context limit are rejected with a visible error. v3 uses complete file text without embeddings or a vector index.

Custom MLC models

The advanced importer accepts a strict WebLLM ModelRecord wrapper:

{
  "schemaVersion": 1,
  "label": "My Local Model",
  "record": {
    "model": "https://huggingface.co/account/model",
    "model_id": "my-model-q4f16-MLC",
    "model_lib": "https://raw.githubusercontent.com/account/repo/main/model.wasm",
    "vram_required_MB": 2048,
    "required_features": ["shader-f16"],
    "overrides": { "context_window_size": 4096 }
  }
}

URLs must use HTTPS and match the deployment CSP allowlist for Hugging Face, HF/XetHub, or GitHub Raw. ChatLLM previews the source before external WebAssembly executes.

Architecture

flowchart LR
  UI[Beautiful UI component layer] --> Chat[Chat Workspace]
  UI --> Agent[Local Agent Workspace]
  UI --> Studio[Local Model Studio]
  Chat --> Store[Zustand product state]
  Agent --> Store
  Agent --> Tools[Sandboxed local tools]
  Tools --> Approval[Artifact diff approval]
  Approval --> IDB
  Studio --> Store
  Store --> Engine[WebLLM Engine Manager]
  Engine --> Worker[Dedicated Web Worker]
  Worker --> GPU[WebGPU]
  Store --> IDB[IndexedDB]
  Engine --> Cache[Cache API]
  PWA[Service Worker] --> Shell[Offline app shell]

Loading

Data	Storage	Boundary
Conversations, messages, and Agent runs	IndexedDB	This browser
File text, artifacts, and preferences	IndexedDB	This browser
Model weights and WASM	WebLLM Cache API	This browser
App shell	Service Worker cache	This browser
Generation	Dedicated worker + WebGPU	This device

The Engine Manager owns a single worker and a single active model. Request, conversation, and model identity prevent stale worker events from overwriting newer state. OOM and GPU device-loss paths release the worker and offer a lighter model.

v3.2 uses the fresh chatllm-v3.2 IndexedDB schema. Earlier local conversation data is left untouched and is not loaded into the new Agent-capable schema.

Early runtime archive

ChatLLM began with a handwritten inference layer before the current WebLLM package existed. The v1 worker, React bridge, types, and runtime configuration are preserved in legacy/early-webllm-runtime. They remain excluded from production builds. The complete original application is available from the v1.0.0 tag.

Develop

Requirements: Node.js 20+ and a current WebGPU-capable browser.

npm install
npm run dev

Quality gates:

npm run typecheck
npm run lint
npm test
npm run build
npm audit --omit=dev --audit-level=high

Deploy to Cloudflare Pages

npm run build
npx wrangler pages deploy dist --project-name chatllm-web --branch main

Production: chatllm-web.pages.dev

Cloudflare headers restrict framing, permissions, scripts, workers, WASM execution, and model connections. The PWA caches the application shell; WebLLM manages model assets separately.

Credits

• Application code: MIT
• Runtime: MLC WebLLM
• Agent-native component source and interaction reference: Beautiful UI
• Interface architecture and motion vocabulary: Motion Lexicon
• WebGL sweep effects: Glimm
• Icons: Iconoir

Focused issues and pull requests are welcome. Keep inference local, preserve reduced-motion support, and run every quality gate before submitting.