docs(@voltagent/rag): add README with chunker API and usage examples

[Goutham-Annem]

What is current behavior?

@voltagent/rag has no README. The package provides 13 chunkers (token, sentence, recursive, markdown, code, HTML, JSON, LaTeX, table, semantic, semantic-markdown, late, neural, slumber) plus tokenizer and similarity utilities, but there is no documentation at the package level.

What is new behavior?

Adds packages/rag/README.md covering:

• Installation
• All chunkers with a code example for each
• Chunk type reference
• Utility functions (tokenizer, cosine similarity, text utils)
• Custom tokenizer guide
• Links to the full RAG documentation

Notes for reviewers

Documentation-only change. No code was modified.

---

Summary by cubic

Adds a comprehensive README to @voltagent/rag covering installation, API, and examples for all 13 chunkers, utilities (tokenizer, cosine similarity, text helpers), the Chunk type, and custom tokenizer setup. Also fixes the custom tokenizer example by adding a missing import and correcting character offsets.

^{Written for commit f995575. Summary will update on new commits.}

Summary by CodeRabbit

• Documentation
  • Added comprehensive README documentation for the @voltagent/rag package, including installation guidance, chunker API reference, utility functions, type shape details, and configuration/examples for custom tokenizers.
  • Added an entry to the Changesets changelog to record this documentation update for a patch release.

[changeset-bot]

🦋 Changeset detected

Latest commit: f995575

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@voltagent/rag	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3d5b0bb2-2e07-4478-b973-5f37a5c18061

📥 Commits

Reviewing files that changed from the base of the PR and between ddf4b58 and f995575.

📒 Files selected for processing (1)

• packages/rag/README.md

✅ Files skipped from review due to trivial changes (1)

• packages/rag/README.md

---

📝 Walkthrough

Walkthrough

A new README is added for the @voltagent/rag package, documenting installation, all 14 chunker implementations with TypeScript examples, utility functions, the Chunk type shape, custom tokenizer configuration, and the MIT license. A patch-level changeset entry is also added.

Changes

@voltagent/rag README and changeset

Layer / File(s)	Summary
RAG README: installation, chunkers, utilities, types, and license packages/rag/README.md	Adds the complete README with banner and navigation, installation commands (npm/yarn/pnpm), documentation of 14 chunker implementations (Token, Sentence, Recursive, Markdown, Code, HTML, JSON, LaTeX, Table, Semantic, SemanticMarkdown, Late, Neural, Slumber) with TypeScript usage examples, utility functions (default tokenizer, token counting, cosine similarity, text helpers), Chunk type field definitions, custom Tokenizer configuration guide, further documentation links, and MIT license notice.
Patch changeset entry .changeset/rag-readme.md	Registers a patch bump for @voltagent/rag tied to the README addition.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐇 Hippity-hop, the docs are here at last,
Chunkers and tokenizers, a README amassed!
Token and Sentence, Markdown and Code,
Each with an example right down the road.
The rabbit stamps "patch" with a satisfied smile,
Good docs make the warren worth every mile! 📜

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding a README with documentation for the @voltagent/rag package, including the chunker API and usage examples.
Description check	✅ Passed	The description comprehensively covers current behavior, new behavior, and notes for reviewers. It addresses key template sections and clearly explains the documentation additions without requiring code changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/rag/README.md`:
- Around line 279-288: The example code uses the TokenChunker class but only
imports Tokenizer from the "`@voltagent/rag`" package. Add TokenChunker to the
import statement on the first line so that it imports both Tokenizer and
TokenChunker from "`@voltagent/rag`", making the example complete and functional.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d3922464-09da-4691-9fe4-ed5a97530c1a

📥 Commits

Reviewing files that changed from the base of the PR and between 1e1af65 and ddf4b58.

📒 Files selected for processing (2)

• .changeset/rag-readme.md
• packages/rag/README.md

[cubic-dev-ai]

2 issues found across 2 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>