Skip to content

Preserve model EOS and chat-template defaults - #128

Merged
Qubitium merged 2 commits into
mainfrom
agent/laguna-generation-contract
Jul 22, 2026
Merged

Preserve model EOS and chat-template defaults#128
Qubitium merged 2 commits into
mainfrom
agent/laguna-generation-contract

Conversation

@Qubitium

@Qubitium Qubitium commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • preserve the checkpoint's full generation_config.eos_token_id value instead of replacing it with the tokenizer's single primary EOS id
  • copy the model's loaded generation config for continuous generation and pass its default_chat_template_kwargs into chat rendering
  • stop every GSM8K variant at </assistant> and cover the assistant-boundary contract with regressions
  • bump Evalution to 0.0.9 and document the release in the README

Root cause

Laguna S 2.1 declares EOS ids [2, 24]; token 24 is </assistant>, while the tokenizer's primary eos_token_id is only 2. Evalution replaced the model value with the tokenizer value and rebuilt continuous-generation configuration from model.config, discarding both the assistant-boundary EOS and default_chat_template_kwargs={"enable_thinking": true}. GSM8K also lacked the textual assistant boundary in its stop list.

Tokenicer already loads the checkpoint correctly: </assistant> encodes to [24], and its clean main suite passes. The reusable fix therefore belongs in Evalution's transformer generation contract rather than Tokenicer.

Validation

  • CUDA_VISIBLE_DEVICES=2,3 PYTHON_GIL=0 pytest -q -m 'not integration' — 741 passed, 274 deselected
  • PYTHON_GIL=0 pytest -q tests/test_transformer.py tests/test_gsm8k.py tests/test_gsm8k_platinum.py — 86 passed
  • focused regression selection — 3 passed
  • ruff check on all changed files — passed
  • git diff --check — passed
  • pytest -q tests/test_package.py tests/test_startup_banner.py — 10 passed
  • python -m build --wheel — built evalution-0.0.9-py3-none-any.whl with packaged version 0.0.9
  • Tokenicer main (v0.0.14) full suite — 468 passed, 4 skipped
  • live Laguna S 2.1 GPTQModel checkpoint load/request-preparation smoke on physical GPUs 2 and 3 with CPU spill — EOS [2, 24], assistant token [24], GSM8K stop contains </assistant>, prompt ends in <assistant><think>

Companion GPTQModel loader/evaluation fix: ModelCloud/GPTQModel#2961

@Qubitium Qubitium reopened this Jul 22, 2026
@Qubitium
Qubitium merged commit 050693e into main Jul 22, 2026
2 checks passed
@Qubitium
Qubitium deleted the agent/laguna-generation-contract branch July 22, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant