DataFoundry v0.2.0

v0.2.0

DataFoundry 0.2 expands the initial governed data-agent workbench into a stateful workflow for concurrent analysis, evidence-backed follow-ups, reusable outputs, and production-facing Web deployment.

Highlights

Data Link: first-party semantics for data agents

Data Link, our newly open-sourced data knowledge graph, now joins the DataFoundry ecosystem as a first-party semantic foundation. A schema tells an agent what columns exist; Data Link helps it understand what those columns mean, how datasets connect, and which relationships are reliable enough to use.

• Build a dual-layer graph: structural nodes such as tables and columns remain grounded in source data, while shared Concept and Entity nodes organize business meaning.
• Connect explicit relationships such as foreign keys and lineage with inferred relationships such as joinable, semantic_synonym, correlated, and distribution_similar, each carrying confidence information.
• Explore unfamiliar data through an agent-oriented retrieval interface instead of injecting an entire catalog into the prompt.
• Query Data Link through MCP and REST. DataFoundry discovers a configured Data Link/DataGraph server and exposes its graph, exploration, table-management, and rebuild workflows inside the Web workbench.
• Give complex analysis a stronger semantic starting point: fewer blind joins, less field guessing, and a clearer path from physical schema to business concepts.

Data Link is designed as a universal graph for tables, documents, slides, and images. DataFoundry 0.2 integrates its graph and agent retrieval surface; the Data Link service is deployed and configured separately, so both projects can evolve independently.

Stateful analysis and conversation control

• Run multiple data-task sessions without coupling their live state.
• Queue, edit, send immediately, or remove follow-up prompts while a run is active.
• Restore persisted messages, tool calls, run events, outputs, and terminal run status after refresh or session switching.
• Re-ask from an earlier user turn or create a branch from a persisted checkpoint. The original path remains available, and branch navigation keeps the alternatives explicit.

Evidence, trace, and semantic exploration

• Reference a whole output or a selected table/text region in the next question.
• Resolve artifact, SQL audit, trace-step, schema, preview, and knowledge references on the server before they enter the governed run context.
• Inspect a semantic Trace DAG built from runs, steps, tool calls, checkpoints, outputs, and their relationships.
• Open the Data Link workspace graph to explore tables, columns, concepts, entities, and edges when a compatible Data Link MCP service is configured.

Outputs and workspace assets

• Restore session outputs independently from the live conversation.
• Preview tables, charts, reports, SQL, and files; export supported table and chart formats.
• Upload files into the active session and promote supported files or file-backed outputs into cross-session workspace assets.
• Mention reusable workspace assets in later questions so analysis can continue from prior material.

Web deployment and onboarding

• Use built-in password authentication with registration, verification, login, password reset, session management, CSRF protection, and secure cookies.
• Route browser API and CopilotKit traffic through the same-origin Next.js proxy in formal deployments.
• Use the Web workbench in English or Simplified Chinese.
• Create and test OpenAI-compatible model profiles before selecting them for a run.
• Start with the guided first-run flow and the automatically provisioned DTC Growth Review SQLite case.

Runtime and terminal workflow

• Preserve provider-compatible prompt content while keeping governed context compilation and prompt snapshot ordering deterministic.
• Keep schema-first, read-only SQL execution and audit behavior across restored and continued runs.
• Use the refreshed chat-first TUI, session resume, output browsing, improved tables, input history, completion, scrolling, and responsive terminal layout.

Reliability and bug fixes

• Deterministic model context: fixed prompt snapshot ordering and provider message compatibility so restored history, the current turn, assistant tool calls, and tool results reach the model in the intended order.
• Conversation restore: removed races around session switching, restored terminal run status from backend checkpoints, preserved tool-call/result pairing, and prevented stale sessions from overwriting the active conversation.
• Session UX: fixed chat scroll restoration, session-switch positioning, input-window overlap, queued-prompt transitions, running indicators, and stop behavior.
• SQL and audit stability: normalized BigInt and other non-JSON SQL values before transport, stabilized audit-log identifiers, and kept read-only results available as artifacts without serialization failures.
• Datasource readiness: tightened datasource selection, local file-path handling, connection-state display, and schema-first execution so runs fail earlier and more clearly when required data context is unavailable.
• Tool and trace delivery: improved streamed tool-result normalization, failure presentation, parent/step correlation, token usage display, cancellation, and recovery of outputs after refresh.
• Production Web path: reduced the initial workbench bundle through a lightweight route entry, hardened same-origin API proxying, and improved backend startup/readiness reporting.
• Release checks: repaired stale source-contract tests after the Web workbench split and aligned smoke tests with the new DTC case instead of the retired DuckDB demo.

Compatibility and known boundaries

• No intentional public REST or AG-UI breaking change is introduced for existing 0.1 integrations, but the recommended deployment and demo path changed.
• The retired api-duckdb-demo is no longer auto-created. npm install now generates the DTC Growth Review SQLite fixture, which the API provisions per user/workspace.
• Data Link requires a separately running compatible Data Link/DataGraph MCP and REST service.
• This is a pre-release. Validate authentication, secrets, datasource policy, backup, monitoring, and reverse-proxy behavior for your environment before production use.

Upgrade notes

• Node.js 22 or later is required.
• Run npm install after pulling 0.2 because workspace dependencies and the lockfile changed.
• Formal deployment uses password auth with build / start; review the required auth variables in Quick start.
• Rebuild the Web app after changing NEXT_PUBLIC_* values.
• Data Link is an integration surface, not an embedded semantic service. Configure a compatible MCP server before expecting graph data.
• Production deployments still need environment-specific secret management, access policy, audit export, monitoring, backup, and TLS termination.

Documentation map

• First deployment: Quick start
• New Web workflows: Web workbench guide
• Capability boundaries: Capabilities
• Authentication and production boundaries: Security
• Session, branch, file, output, and model endpoints: REST API

---

Full Changelog: v0.1.0...v0.2.0

DataFoundry v0.1.0 Public Preview

DataFoundry v0.1.0 Public Preview

DataFoundry is now open source.

This is the first public preview of DataFoundry: an open-source, enterprise-grade Data Agent workbench for trustworthy data analysis. It is built for teams that want AI to work with real datasources, files, knowledge, tools, and analysis artifacts without turning enterprise data tasks into an unsafe black box.

DataFoundry is not just a SQL chatbot. It combines unified semantics, read-only execution boundaries, audit/replay evidence, and Web/TUI/API surfaces so one natural-language question can become a governed, reproducible data task.

Why This Release Matters

Most data-agent demos look impressive because they reduce the workflow to:

prompt -> SQL -> answer

That is not enough for enterprise data work.

In real environments, the hard questions are different:

• Does the agent understand business definitions such as GMV, retention, active user, or conversion?
• Can it avoid guessing the wrong table, field, join path, or metric definition?
• Can it query data without exposing credentials to the model or browser?
• Can teams audit what SQL was generated, what tools were called, and why a conclusion was produced?
• Can the result become a reusable table, chart, report, or file instead of disappearing into a chat transcript?

DataFoundry v0.1.0 is the first public version of our answer to those questions.

Highlights

Enterprise Data Agent Workbench

DataFoundry provides a dedicated workbench for data tasks instead of a generic coding-agent interface. The product surface is optimized for selecting datasources, inspecting schema, managing context, running analysis, reviewing evidence, and turning outputs into reusable assets.

The current release includes:

• A Web workbench for interactive data-agent workflows.
• A TUI for terminal-native data tasks.
• A REST API for integration and embedding.
• Shared runtime behavior across Web, TUI, and API surfaces.

28 Datasource Types Out Of The Box

DataFoundry is designed to meet existing enterprise data stacks where they already are.

The current public preview includes broad datasource coverage across analytical databases, operational databases, search systems, caches, and file-like sources, including PostgreSQL, MySQL, Snowflake, BigQuery, ClickHouse, MongoDB, Redis, Elasticsearch, CSV, XLSX, DuckDB, SQLite, and more.

This matters because data agents become useful only when they can connect to the systems teams already depend on. The goal is to reduce integration friction and get from first question to real analysis faster.

Unified Semantics And Context Organization

DataFoundry treats schema, metrics, field relationships, and business definitions as first-class context.

Instead of forcing the model to re-guess meaning from table names on every run, DataFoundry is designed around a semantic operating layer: business terms should resolve to approved tables, fields, joins, and metric definitions. This is the foundation for better accuracy, fewer wrong joins, and less definition drift.

In v0.1.0, the implemented base includes schema-first analysis, metadata/context organization, datasource configuration, and runtime context controls. The broader unified semantic layer is a strategic product direction and will continue to evolve in future releases.

Safe By Default: Read-Only Data Gateway

DataFoundry puts a controlled execution boundary between the model and enterprise data.

The Data Gateway owns datasource connections, SQL guardrails, read-only execution, row limits, timeouts, and field masking. Credentials and API keys are not passed through model messages, browser state, or forwarded props.

This is a core design choice: a data agent should not need direct credential visibility to answer data questions.

Audit, Replay, And Evidence Chain

A useful enterprise data agent must be explainable after the answer is produced.

DataFoundry persists the operational evidence behind a run:

• SQL generated and executed.
• Tool calls and tool results.
• Event streams.
• Session history.
• Generated tables, charts, reports, and files.
• Run identity and task state.

This gives teams a way to review, replay, continue, and hand off analysis work instead of trusting a one-shot black-box answer.

Complex Multi-Step Data Tasks

DataFoundry is built for more than direct single-table lookup.

The runtime is designed around multi-step analysis: selecting resources, inspecting schema, budgeting context, calling tools, executing read-only queries, checking intermediate results, and materializing outputs. This helps complex questions converge into usable conclusions rather than ending as partial SQL snippets.

Open Agent Ecosystem

DataFoundry builds on and connects with modern open-source agent infrastructure:

• Mastra for agent runtime foundations.
• AG-UI for event stream protocol integration.
• CopilotKit for agent UX integration.
• Ink for terminal UI.
• MCP for tool ecosystem connectivity.

The product direction is open and composable: teams should be able to connect tools, skills, files, knowledge, and model providers without locking the data-agent workflow into one closed runtime.

Self-Hosted And Multi-Model

DataFoundry is intended to run inside your own environment.

The model side supports OpenAI-compatible providers, including GPT, Qwen, DeepSeek, and similar compatible services. This lets teams choose the right balance of security, cost, latency, and quality for each scenario.

What You Can Try Today

This public preview is best suited for:

• Local evaluation.
• Product demos.
• Integration development.
• Data-agent workflow exploration.
• Testing datasource connection and schema inspection flows.
• Trying Web, TUI, and API-based data task surfaces.

Start with the README and Quick Start:

• README: https://github.com/datagallery-lab/datafoundry
• Quick Start: https://github.com/datagallery-lab/datafoundry/blob/main/docs/en/quick-start.md
• Chinese README: https://github.com/datagallery-lab/datafoundry/blob/main/README_zh.md

Installation

The npm package is published as a public distribution anchor for the workbench:

npm view datafoundry@0.1.0

For normal usage, we recommend cloning the repository and following the Quick Start for the full workspace setup.

Known Boundaries

This is a public preview, not a production-stable v1.0.0.

The current release is suitable for local trials, demos, and integration development. Production deployments still need environment-specific design for:

• Multi-tenant authentication and authorization.
• Centralized secret management.
• Deployment topology and service operations.
• Observability, monitoring, and alerting.
• Enterprise governance policies.
• Production data-access review.

The npm package is not a stable SDK contract yet. It is published to reserve the public package name and provide a distribution anchor for the open-source workbench.

Community

We welcome feedback, issues, discussions, and real-world adoption notes.

• GitHub Issues: https://github.com/datagallery-lab/datafoundry/issues
• GitHub Discussions: https://github.com/datagallery-lab/datafoundry/discussions

Release Metadata

• Version: v0.1.0
• Status: Public Preview
• License: Apache-2.0
• Package: datafoundry@0.1.0