description	BoxLang AI is the module for building AI into your applications with one fluent API for chat, agents, tools, memory, RAG, MCP, and enterprise governance.
icon	house-chimney-heart

Introduction

BoxLang AI lets you build AI into your applications with one fluent API for chat, agents, tools, memory, RAG, and MCP across many providers, with the governance controls you need in production. It is part of the BoxLang software productivity platform, built for developers, built for AI, built to ship.

🙋 What is BoxLang AI?

BoxLang AI is a module that gives you a unified API for chatbots, content generation, code assistants, RAG systems, and multi-step agent workflows. Start with one line of code and grow into agents, memory, tools, and middleware as your needs grow.

graph LR
    User["👤 User"] --> Agent["🤖 Agent"]

    Agent --> Memory["💭 Memory"]
    Agent --> Tools["🛠️ Tools"]
    Agent --> Docs["📄 Documents"]

    Docs --> Loaders["📥 Loaders"]
    Loaders --> VectorDB[("📊 Vectors")]

    Memory --> VectorDB
    VectorDB --> RAG["🔍 RAG"]
    Tools --> RAG

    RAG --> AI["🚀 AI Models"]
    AI --> Response["💬 Response"]
    Response --> Agent
    Agent --> User

    style User fill:#4CAF50,stroke:#2E7D32,stroke-width:3px,color:#fff
    style Agent fill:#2196F3,stroke:#1565C0,stroke-width:3px,color:#fff
    style AI fill:#9C27B0,stroke:#6A1B9A,stroke-width:3px,color:#fff
    style Response fill:#FF9800,stroke:#E65100,stroke-width:3px,color:#fff
    style VectorDB fill:#00BCD4,stroke:#006064,stroke-width:2px,color:#fff
    style RAG fill:#FF5722,stroke:#BF360C,stroke-width:2px,color:#fff

Loading

✨ Key Features

• Multi-Provider Support: Work with OpenAI, Claude, Gemini, Grok, Groq, DeepSeek, Cloudflare Workers AI, Ollama, and more
• Unified API: One consistent interface across all providers
• Multi-Tenant Memory: Enterprise-grade isolation with userId and conversationId across all 20 memory types
• Multimodal Content: Process images, audio, video, and documents alongside text
• Local AI Support: Run models locally with Ollama for privacy and offline use
• AI Pipelines: Chain operations together for complex multi-step workflows
• Streaming Responses: Get real-time responses as they're generated
• Tool Integration: Enable AI to call functions and access real-time data
• Async Support: Non-blocking operations for better performance
• Template System: Create reusable prompts with dynamic placeholders
• AI Agents: Autonomous agents with memory, tools, and reasoning
• Document Loaders: Load and process various file formats for RAG
• Vector Memory: Semantic search with 12 vector database integrations
• AI Skills: Composable, reusable knowledge blocks injected into agent system messages at runtime
• MCP Server Integration: Seed agents and models directly from MCP servers — tools discovered automatically
• Middleware Pipeline: Intercept and transform AI requests/responses with a powerful middleware system
• Audio & Speech: Text-to-speech, speech-to-text, and audio translation across multiple providers
• Image Generation: Generate images from text prompts with provider-agnostic API
• Web Search: Real-time web search integration with agent tools and structured results
• Global Tool Registry: Register tools by name once, reference by string everywhere
• Provider Capabilities: Type-safe capability system — providers declare what they support, BIFs enforce it
• Human-in-the-Loop: Suspend a run for human approval before sensitive tool calls, with durable "always allow" grants and batched approvals
• Gateways: Present approvals over CLI, signed HTTP webhooks, or a platform module you register yourself
• Gateway Sessions: Wire an agent to inbound gateway messages, with reject/queue/steer/interrupt dispatch policies for a busy thread
• Agent Run Control: Cancel or steer a run already in flight, addressed purely by threadId
• Normalized Reasoning: One message.reasoning/delta.reasoning envelope across every reasoning-capable provider
• Security & Guardrails: Prompt-injection scanning, untrusted-content fencing, LLM-as-judge classification, and output redaction — all opt-in, all offline-testable

📡 Supported Providers

BoxLang supports a variety of AI providers out of the box. You can also create custom providers by following our Custom Provider Guide.

Provider	Type	Chat	Stream	Tools	Embeddings	Vision	Audio
Bedrock	Cloud	✅	✅	✅	✅	❌	❌
Claude	Cloud	✅	✅	✅	❌	✅	❌
Cloudflare	Cloud	✅	✅	✅	✅	❌	❌
Cohere	Cloud	✅	✅	✅	✅	❌	❌
DeepSeek	Cloud	✅	✅	✅	✅	❌	❌
Docker Desktop	Local	✅	✅	✅	✅	❌	❌
Cartesia	Cloud	❌	❌	❌	❌	❌	✅
ElevenLabs	Cloud	❌	❌	❌	❌	❌	✅
Gemini	Cloud	✅	✅	✅	✅	✅	❌
Grok	Cloud	✅	✅	✅	❌	✅	❌
Groq	Cloud	✅	✅	✅	❌	❌	❌
HuggingFace	Cloud	✅	✅	✅	✅	❌	❌
MiniMax	Cloud	✅	✅	✅	✅	✅	❌
Mistral	Cloud	✅	✅	✅	✅	❌	❌
Ollama	Local	✅	✅	✅	✅	✅	❌
OpenAI	Cloud	✅	✅	✅	✅	✅	❌
OpenRouter	Gateway	✅	✅	✅	✅	✅	❌
Perplexity	Cloud	✅	✅	❌	❌	❌	❌
Voyage	Cloud	❌	❌	❌	✅	❌	❌

Legend: Vision support requires a multimodal model from the provider (e.g., gpt-4o, claude-3, gemini-2.0). Audio covers text-to-speech/speech-to-text/translation, shown here only for providers whose only capability is audio (ElevenLabs, Cartesia) — see Audio/Speech for the full per-provider TTS/STT matrix. OpenRouter capabilities depend on the selected underlying model.

🗃️ Supported Memory Types

BoxLang AI provides 20+ memory types for conversation history and semantic search. All memory types support multi-tenant isolation with userId and conversationId.

Memory Type	Vector	Best For	Storage	Multi-Tenant
Windowed	❌	Recent N messages, simple context	In-memory	✅
Summary	❌	Long conversations with summarization	In-memory	✅
Session	❌	Web sessions, survives page refresh	HTTP Session	✅
File	❌	Persistent storage, file-based	File System	✅
Cache	❌	Fast retrieval, distributed cache	CacheBox	✅
JDBC	❌	Enterprise database storage	Any JDBC DB	✅
BoxVector	✅	Development, prototyping	In-memory	✅
Chroma	✅	Python integration, local dev	ChromaDB	✅
Postgres	✅	Existing PostgreSQL infrastructure	PostgreSQL	✅
MySQL	✅	Existing MySQL 9+ infrastructure	MySQL	✅
MariaDB	✅	Existing MariaDB 11.7+ infrastructure	MariaDB	✅
OpenSearch	✅	AWS integration, enterprise search	OpenSearch	✅
TypeSense	✅	Fast typo-tolerant search	TypeSense	✅
Pinecone	✅	Production cloud-native	Pinecone Cloud	✅
Qdrant	✅	Self-hosted, high performance	Qdrant	✅
Weaviate	✅	GraphQL, knowledge graphs	Weaviate	✅
Milvus	✅	Enterprise, massive scale	Milvus	✅
Hybrid	✅	Recent + semantic combined	Vector + Standard	✅

📖 Learn More: Standard Memory Guide · Vector Memory Guide

📋 Supported Document Loaders

BoxLang AI provides 12 document loaders for importing content from various sources into standardized Document format for RAG workflows.

Loader	File Types	Best For	Features
TextLoader	.txt, .text	Plain text files	Simple text ingestion
MarkdownLoader	.md, .markdown	Markdown documents	Header splitting, code removal
HTMLLoader	.html, .htm	HTML pages	Tag extraction, script/style removal
CSVLoader	.csv	Tabular data	Row/column modes, filtering
JSONLoader	.json	JSON data	Field extraction, array handling
XMLLoader	.xml	XML documents	XPath queries, attribute extraction
PDFLoader	.pdf	PDF documents	Text extraction, page splitting
LogLoader	.log	Log files	Pattern matching, timestamp parsing
HTTPLoader	URLs	Web content	HTTP/S, headers, authentication
FeedLoader	RSS/Atom	RSS/Atom feeds	Feed parsing, item extraction
SQLLoader	SQL queries	Database records	JDBC datasources, query execution
DirectoryLoader	Folders	Batch loading	Recursive scanning, type detection
WebCrawlerLoader	Websites	Web crawling	Multi-page, depth control, link following

Key Features:

• ✅ Fluent API - Chain configuration methods
• ✅ Memory Integration - Direct ingestion with toMemory()
• ✅ Automatic Chunking - Split large documents
• ✅ Async Support - Load documents asynchronously
• ✅ Multi-Memory Fan-out - Ingest to multiple vector stores
• ✅ Transform/Filter - Apply transformations during load

📖 Learn More: Document Loaders Guide · Custom Loaders

🚀 Use Cases

• 💬 Chatbots: Build conversational interfaces with memory and context
• ✍️ Content Generation: Create articles, documentation, marketing copy
• 💻 Code Assistance: Generate, review, and explain code
• 📊 Data Analysis: Extract insights from text and structured data
• 📄 Document Processing: Analyze PDFs, contracts, and reports
• 🎥 Media Analysis: Process images, audio, and video content
• 🌍 Translations: Multi-language content translation
• 📋 Summarization: Condense long documents intelligently
• ❓ Question Answering: Build knowledge bases and FAQs with RAG
• 🔄 Custom Workflows: Multi-step AI processing pipelines

---

📖 Quick Start

🎓 Getting Started

Perfect for beginners - get up and running quickly

💬 Simple AI Interactions

Learn basic chat, streaming, and structured output

🔗 AI Pipelines

Build complex workflows with agents, memory, and tools

🤖 AI Agents

Build reusable and autonomous agents, sub-agents, and much more.

---

🔧 Built-In Functions (BIFs)

BoxLang AI provides a comprehensive set of BIFs for different AI operations. You can see all of our BIF reference documentation here: BIF Reference.

💬 Chat & Conversation

BIF	Purpose	Return Type	Example Use Case
aiChat()	Simple one-shot chat request	String	Quick Q&A, content generation
aiChatAsync()	Non-blocking chat request	Future	Background processing, parallel requests
aiChatRequest()	Build structured chat requests	AiChatRequest	Complex requests with tools
aiChatStream()	Real-time streaming responses	void	Live chat, progressive output
aiService()	Get AI service instances	Service	Multi-provider management

🏗️ Pipeline Components

BIF	Purpose	Return Type	Example Use Case
aiAgent()	Create AI agents with tools, memory, skills & MCP servers	AiAgent	Autonomous assistants, multi-turn
aiMessage()	Build message pipelines	AiMessage	Reusable prompts, templates
aiModel()	Create model runnables with skills & MCP servers	AiModel	Pipeline integration
aiTransform()	Create data transformers	Transformer	Pipeline data processing
aiTool()	Define callable functions	Tool	Real-time data, function calling
aiSkill()	Create or discover AI skill blocks from SKILL.md files	AiSkill	Reusable knowledge injection
aiGlobalSkills()	Access the global shared skills pool	Array	Cross-agent knowledge sharing
aiToolRegistry()	Access the global AI tool registry singleton	AIToolRegistry	Named tool resolution
aiAgentRegistry()	Access the global AI agent registry singleton	AIAgentRegistry	Named agent resolution
aiParallel()	Execute runnables (models, pipelines) in parallel	Array	Parallel model execution

🧠 Memory & Context

BIF	Purpose	Return Type	Example Use Case
aiMemory()	Create conversation or vector memory	Memory	Context-aware conversations, RAG, semantic search

📄 Document Processing

BIF	Purpose	Return Type	Example Use Case
aiDocuments()	Load documents from sources	Array/Loader	Document processing, RAG

🔢 Utilities

BIF	Purpose	Return Type	Example Use Case
aiChunk()	Split text into chunks	Array	Processing large documents
aiEmbed()	Generate vector embeddings	Array/Struct	Semantic search, similarity
aiPopulate()	Populate a class instance, struct, or array from JSON data or struct.	Any	This is useful for testing, custom workflows, or working with cached AI responses.
aiTokens()	Estimate token counts	Numeric	Cost estimation, limits

⚙️ MCP (Model Context Protocol)

BIF	Purpose	Return Type	Example Use Case
MCP()	Connect to MCP servers (HTTP URL or stdio child process)	IMCPClient	External tools, resources
MCPServer()	Create MCP server instances	MCPServer	Expose tools to agents

🛡️ Governance & Gateways

BIF	Purpose	Return Type	Example Use Case
aiFence()	Wrap untrusted content in boundary markers so the model treats it as data	String	Defending against indirect prompt injection
aiGateway()	Resolve a gateway by name	IGateway	Human approvals over CLI or signed HTTP
aiGatewayRegistry()	Access the registry external gateway modules register into	GatewayRegistry	Custom platform gateways
aiGatewaySession()	Wire an agent to inbound gateway messages	GatewaySession	Chat platform agents
aiDecisionStore()	Create the store for durable human-approval grants	IDecisionStore	Human-in-the-Loop "always allow"

🎵 Audio & Speech

BIF	Purpose	Return Type	Example Use Case
aiSpeak()	Convert text to spoken audio	Binary	Voice responses, accessibility
aiTranscribe()	Transcribe audio to text	String	Meeting notes, voice commands
aiTranslate()	Translate audio between languages	String	Multi-language audio translation

🖼️ Image Generation

BIF	Purpose	Return Type	Example Use Case
aiImage()	Generate images from text prompts	Binary/Struct	Marketing visuals, content creation

🌐 Web Search

BIF	Purpose	Return Type	Example Use Case
aiWebSearch()	Search the web in real-time	String	Live data retrieval, research
aiWebSearchAsync()	Non-blocking web search	Future	Background web research

---

🚀 Quick Examples

💬 Simple Chat

answer = aiChat( "What is BoxLang?" )
println( answer )

🎨 Simple Chat with Parameters

answer = aiChat(
    "Write a haiku about coding",
    { temperature: 0.9 }
)

🔗 Build a Pipeline

pipeline = aiMessage()
    .system( "You are a helpful assistant" )
    .user( "Explain ${topic}" )
    .toDefaultModel()
    .transform( r => r.content )

result = pipeline.run( { topic: "recursion" } )

⚡ Stream Responses

aiChatStream(
    "Tell me a story",
    ( chunk ) => print( chunk.choices?.first()?.delta?.content ?: "" )
)

📦 Get JSON Responses

// Automatically parse JSON responses
user = aiChat(
    "Create a user profile with name, age, and email for Alice",
    { returnFormat: "json" }
)

println( "Name: #user.name#" )
println( "Age: #user.age#" )
println( "Email: #user.email#" )

🛠️ Use AI Tools

// Let AI call functions for real-time data
getWeather = aiTool(
    name: "get_weather",
    description: "Get current weather for a location",
    parameters: {
        location: { type: "string", description: "City name" }
    },
    callback: ( args ) => {
        return { temp: 72, condition: "sunny", location: args.location }
    }
)

response = aiChat(
    "What's the weather in San Francisco?",
    { tools: [ getWeather ] }
)

🔢 Generate Embeddings

// Create vector embeddings for semantic search
embeddings = aiEmbed([
    "BoxLang is a modern JVM language",
    "Java is a programming language",
    "Python is popular for AI"
])

// Use embeddings for similarity comparison
println( "Generated #embeddings.len()# embeddings" )

📄 Load Documents

// Vector memory for semantic retrieval
memory = aiMemory( memory: "boxvector", config: { embeddingProvider: "openai" } )

// Load documents straight into it
aiDocuments( source: "docs/guide.md" ).toMemory( memory )

// Ask an agent that can retrieve from it
agent = aiAgent( name: "DocsBot", memory: memory )
response = agent.run( "What is covered in the guide?" )

🤖 Create an Agent

// Build an autonomous agent
agent = aiAgent(
    name        : "Research Assistant",
    instructions: "You help research and summarize topics",
    memory      : aiMemory( memory: "window", config: { maxMessages: 10 } ),
    tools       : [ searchTool, summarizeTool ]
)

// Agent handles multi-turn conversations
response = agent.run( "Research AI trends in 2025" )

---

🆘 Need Help?

📚 Resources

• 📖 Full Documentation: Explore all sections above for comprehensive guides
• 💡 Examples: Check the bx-ai/examples/ folder for 60+ runnable code samples
• 🔍 BIF Reference: See reference/built-in-functions/ for detailed function docs
• 📦 Module Components: Explore main-components/ for in-depth component guides

🤝 Community & Support

• 👥 Community: BoxLang Community Forum
• 🐛 Issues: GitHub Issues
• 💬 Discussions: GitHub Discussions
• ✉️ Email Support: ai@ortussolutions.com

🎓 Learning Paths

1. 🌱 Beginners: Start with Quick Start → Basic Chatting → Examples
2. 🏗️ Builders: Learn Pipelines → Memory → Tools
3. 🚀 Advanced: Explore Agents → MCP → Custom Providers

---

🌟 Part of the BoxLang Platform

BoxLang AI is part of BoxLang, the software productivity platform for building, modernizing and running applications, with developers and AI agents working together.

• BoxLang AI overview in the main BoxLang docs
• Agentic Development to set up skills, MCP servers, and agents that write BoxLang code with you

BoxLang and BoxLang AI are professional open source projects. Most modules are open source and free to use, and every BoxLang+ module is free to try for 60 days. The trial starts automatically the first time you start a BoxLang server or CLI with a BoxLang+ module installed, with no sign-up and no key. When you want enterprise support, SLAs, and premium modules, join us.