Skip to content

fix(engine): utf-8 encoding for all I/O — Windows cp1252 crash (#189) - #191

Merged
elementalsouls merged 1 commit into
mainfrom
fix/windows-engine-encoding
Nov 20, 2026
Merged

elementalsouls merged 1 commit into
mainfrom
fix/windows-engine-encoding

Conversation

@elementalsouls

Copy link
Copy Markdown
Owner

Fixes #189.

Problem

On Windows (default code page cp1252 — and any non-UTF-8 locale) the engine crashes as soon as the map phase logs a line containing → / —:

UnicodeEncodeError: 'charmap' codec can't encode character '→' ...

Text-mode open() calls inherit the locale code page, so writing the engine's log, arsenal map, and report blew up; 2/34 engine tests (the mock-subprocess ones) failed on Windows. Reported by @lucasaivn with the exact file:line list.

Fix

  • Pin encoding="utf-8" on every text-mode open() in engine/ (state.py, engine.py, memory.py, scope.py) — reads and writes both, so round-trips are locale-independent.
  • Reconfigure stdout/stderr to utf-8 in engine.main() — the CLI also prints those characters, which crashed independently of the file writes.
  • Engagement.log()'s print is now non-fatal on a non-UTF-8 console: logging is a side channel and must never crash a run; the full line is already in the utf-8 log file.
  • Mock subprocess tests now decode child output as utf-8 (the two tests [bug] engine crashes on Windows: UnicodeEncodeError when writing engine.log (open() without encoding) #189 reports failing on Windows).

Test

New engine/test_windows_encoding.py reproduces the crash deterministically on any host by forcing an ASCII default (PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C) — the same failure mode Windows hits with cp1252 — and asserts the full --mock --hunt flow finishes clean. This catches the regression class on the existing ubuntu CI without needing a windows-latest runner.

Verified green in both normal and forced-ASCII locales:

  • engine suite 35/35 (was 33/35 on Windows-equivalent)
  • tests suite 58/58

Note

#189 also suggests adding a windows-latest job to runtime-tests.yml. Deferred as a separate call — the forced-ASCII regression test covers this specific class cheaply; a real Windows runner is a broader CI decision.

Related: #183 / #185 cover the same encoding class in the PowerShell scripts (separate fix).

🤖 Generated with Claude Code

…crash)

Fixes #189. On Windows (and any non-UTF-8 locale) text-mode open() inherits the
code page (cp1252), so writing the engine's '->' / em-dash characters to the log,
arsenal map, or report crashed with UnicodeEncodeError, and reading them back
could mojibake.

- Pin encoding="utf-8" on every text-mode open() in engine/ (state.py, engine.py,
  memory.py, scope.py) and the subprocess-reading tests.
- Reconfigure stdout/stderr to utf-8 in engine main() so console output of the same
  characters does not crash the CLI run.
- Make Engagement.log()'s print non-fatal on a non-UTF-8 console: logging is a side
  channel and must never crash the run; the full line is already in the utf-8 log.
- Parent-side decode of child engine output in the mock subprocess tests pinned to
  utf-8 (the two tests #189 reports failing on Windows).

New engine/test_windows_encoding.py reproduces the crash deterministically on any
host by forcing an ASCII default (PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C), so
the regression is caught on the existing ubuntu CI without a windows runner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@elementalsouls
elementalsouls merged commit dbb9a59 into main Nov 20, 2026
4 checks passed
@elementalsouls
elementalsouls deleted the fix/windows-engine-encoding branch November 20, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] engine crashes on Windows: UnicodeEncodeError when writing engine.log (open() without encoding)

1 participant