595 FIX Render heredoc diffs by gutter, not leading dashes

[chris-downs]

Closes #595

Problem

When a Terraform/OpenTofu plan contains a heredoc with YAML inside (e.g. a helm_release or argocd_application resource), the - that begins a YAML list item is interpreted as a removal by the diff formatting in the runner. The mis-formatted text is stored as payload['plan'] and rendered in both PR comments and the UI, making the plan look like it is removing things it is not.

This is the runner-side root cause behind terrateamio/terrateam#339, terrateamio/terrateam#528 and terrateamio/terrateam#1062.

Cause

format_diff in terrat_runner/engine_tf.py promoted any leading +/-/~ marker (with its indentation) to column 0 so a diff syntax highlighter would colour the line. YAML list items start with - , which is indistinguishable from a Terraform removal marker, so they were coloured as removals.

Fix

Terraform renders a changed heredoc body as a line-by-line diff with a fixed 2-character gutter: real +/- markers sit in the gutter, two columns to the left of where the value content begins; unchanged lines have a blank gutter. YAML list items always sit at the content column or deeper, never in the gutter.

format_diff now detects that gutter and promotes only the markers that sit in it, leaving the heredoc value content (including its - list items) verbatim. This keeps genuine add/remove colouring inside heredocs while no longer flagging unchanged list items as removed. Non-heredoc lines keep the original behaviour.

Gutter detection: a + only ever appears as a gutter marker (YAML content never starts a line with +), so a + column is the gutter directly — this also handles a heredoc whose YAML root is a bare list. It falls back to (shallowest content line − 2) for removal-only bodies, and treats a body with no gutter marker as unchanged (emitted verbatim).

Before / after

A helm value heredoc with one added and one removed env var.

Before — unchanged YAML list items shown as removals:

              ~ values = <<-EOT
                    "env":
-                   - "name": "ENV"
                      "value": "preview"
-                   - "name": "TRACING_SERVICE_NAME"
                      "value": "xxx-preview"
                EOT

After — only the genuinely added/removed lines are marked:

!   values = <<-EOT
                    "env":
                    - "name": "ENV"
                      "value": "preview"
+                   - "name": "TEST_VAR_DONT_MERGE"
+                     "value": "deadbeef"
-                   - "name": "TRACING_SERVICE_NAME"
-                     "value": "xxx-preview"
                EOT

Testing

Verified against a real argocd_application end to end through PR comments and the UI: the two genuinely changed env vars are coloured (green add / red remove) and ~20 unchanged YAML list items are left uncoloured, with no false positives. Also checked: unchanged heredocs (emitted verbatim), non-heredoc resource diffs (unchanged behaviour), custom and multiple heredoc delimiters, and a YAML-root-list heredoc with an addition.

Note: duplicate logic in the terrateam server

The same marker-promotion logic exists in the terrateam server repo as Terrat_plan_diff.transform (code/src/terrat_plan_diff/terrat_plan_diff.ml), and has the same heredoc bug. From tracing the code, it appears to be referenced in a few places but not in the live render path for current PR comments or the UI:

• terrat_vcs_github_comment_templates / terrat_vcs_gitlab_comment_templates expose it as a plan_diff template transformer, but only the older plan_complete.tmpl uses | plan_diff. The active plan_complete2.tmpl renders the runner's already-formatted plan field raw in a ```diff fence.
• terrat_ui_js_service (the older js_of_ocaml UI) calls it, but the current iris UI highlights the runner's plan field client-side with highlight.js.

So this runner-side fix is what actually corrects the output today. The server copy may be worth fixing for consistency / the legacy paths, but I haven't confirmed whether those paths are still reachable — flagging it for maintainers who know the server better.

🤖 Generated with Claude Code

[joshpollara]

@chris-downs Thank you! Will take a look this week.

[chris-downs]

I found an edge case that isn't covered by this fix, so I'm putting this PR back in draft until it's figured out.

[chris-downs]

@joshpollara we've been using this fork on our runner for a couple of weeks now with good results. It is ready for your review whenever you get a chance. Tests have also been added.

[joshpollara]

@chris-downs Hey sorry for the late reply here... I'm going to pull this in. Thank yo.