fix(prompts): stop compile() crashing on literal $ in templates

[DevMello]

Prompt.compile() raised ValueError('Invalid placeholder in string') for any prompt containing a literal dollar sign (for example "costs $5") or a spaced placeholder like {{ name }}, because the {{var}} to $var rewrite feeds string.Template unescaped. This change escapes literal dollars before the rewrite and lets the placeholder regex absorb surrounding whitespace, so those templates now compile instead of crashing, while templates that already compiled produce identical output. Adds unit tests covering literal $, $$, ${x}, ${{amount}}, spaced placeholders, and the existing missing-variable error.

Example:

prompt = Prompt(
    name="pricing",
    prompt="Our plan costs $99/month. Write a pitch for {{ audience }}.",
    created_at="2024-01-01",
    tags=[],
    commit_id="c1",
)

prompt.compile(audience="developers")
# before: ValueError: Invalid placeholder in string: line 1, col 16
# after:  'Our plan costs $99/month. Write a pitch for developers.'

[devin-ai-integration]

Devin Review found 1 potential issue.

View 1 additional finding in Devin Review.

[devin-ai-integration]

🟡 Prompts whose placeholder is immediately followed by letters or digits fail to fill in

A placeholder is rewritten into a bare marker without delimiting braces (r"$\1" at src/judgeval/prompts/prompt.py:46), so when a placeholder is directly followed by letters, digits or an underscore the name runs into the following text and filling in the prompt fails with a "missing variable" error.
Impact: Prompts like "Hello {{name}}san" or "{{count}}x" can never be compiled, even when every variable is supplied.

Mechanism: bare $name placeholder swallows adjacent identifier characters

re.sub(r"\{\{\s*([^}]+?)\s*\}\}", r"$\1", ...) turns Hello {{name}}san into Hello $namesan. string.Template parses the identifier greedily, so substitute(name="Ada") raises KeyError('namesan'), which compile() converts to ValueError: Missing required variable: namesan (src/judgeval/prompts/prompt.py:70-74).

Now that literal dollars are escaped up front (self.prompt.replace("$", "$$")), emitting the braced form ${name} is safe and fixes the adjacency case; ${name} is unambiguous to Template.

Suggested change

	template_str = re.sub(
	r"\{\{\s*([^}]+?)\s*\}\}", r"$\1", self.prompt.replace("$", "$$")
	)
	template_str = re.sub(
	r"\{\{\s*([^}]+?)\s*\}\}", r"${\1}", self.prompt.replace("$", "$$")
	)

---

Was this helpful? React with 👍 or 👎 to provide feedback.