Repository navigation
fix(engine): utf-8 encoding for all I/O — Windows cp1252 crash (#189) - #191
Merged
Merged
Conversation
…crash) Fixes #189. On Windows (and any non-UTF-8 locale) text-mode open() inherits the code page (cp1252), so writing the engine's '->' / em-dash characters to the log, arsenal map, or report crashed with UnicodeEncodeError, and reading them back could mojibake. - Pin encoding="utf-8" on every text-mode open() in engine/ (state.py, engine.py, memory.py, scope.py) and the subprocess-reading tests. - Reconfigure stdout/stderr to utf-8 in engine main() so console output of the same characters does not crash the CLI run. - Make Engagement.log()'s print non-fatal on a non-UTF-8 console: logging is a side channel and must never crash the run; the full line is already in the utf-8 log. - Parent-side decode of child engine output in the mock subprocess tests pinned to utf-8 (the two tests #189 reports failing on Windows). New engine/test_windows_encoding.py reproduces the crash deterministically on any host by forcing an ASCII default (PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C), so the regression is caught on the existing ubuntu CI without a windows runner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #189.
Problem
On Windows (default code page cp1252 — and any non-UTF-8 locale) the engine crashes as soon as the
mapphase logs a line containing→/—:Text-mode
open()calls inherit the locale code page, so writing the engine's log, arsenal map, and report blew up; 2/34 engine tests (the mock-subprocess ones) failed on Windows. Reported by @lucasaivn with the exact file:line list.Fix
encoding="utf-8"on every text-modeopen()inengine/(state.py,engine.py,memory.py,scope.py) — reads and writes both, so round-trips are locale-independent.stdout/stderrto utf-8 inengine.main()— the CLI also prints those characters, which crashed independently of the file writes.Engagement.log()'s print is now non-fatal on a non-UTF-8 console: logging is a side channel and must never crash a run; the full line is already in the utf-8 log file.Test
New
engine/test_windows_encoding.pyreproduces the crash deterministically on any host by forcing an ASCII default (PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C) — the same failure mode Windows hits with cp1252 — and asserts the full--mock --huntflow finishes clean. This catches the regression class on the existing ubuntu CI without needing awindows-latestrunner.Verified green in both normal and forced-ASCII locales:
Note
#189 also suggests adding a
windows-latestjob toruntime-tests.yml. Deferred as a separate call — the forced-ASCII regression test covers this specific class cheaply; a real Windows runner is a broader CI decision.Related: #183 / #185 cover the same encoding class in the PowerShell scripts (separate fix).
🤖 Generated with Claude Code