Skip to content

[codex] Fix template expression parsing compatibility#200

Open
xsploit wants to merge 2 commits into
microsoft:mainfrom
xsploit:codex/template-expression-compat
Open

[codex] Fix template expression parsing compatibility#200
xsploit wants to merge 2 commits into
microsoft:mainfrom
xsploit:codex/template-expression-compat

Conversation

@xsploit
Copy link
Copy Markdown

@xsploit xsploit commented May 12, 2026

Summary

  • normalize POML input before XML parsing so raw < and & in text/attribute values no longer explode before template evaluation
  • decode XML entities before evaluating template expressions, including &amp;&amp;, &lt;, and &gt;
  • support multiline wrapped attribute expressions, missing-variable fallback expressions, optional bare identifiers in if guards, and <if condition="..."> wrappers
  • add a semantic <tool-policy> component alias and regression coverage for the compatibility cases

Root Cause

POML templates are parsed as XML before template expressions are evaluated. That means common JavaScript expression characters such as < and & can fail at the XML lexer layer, while XML entity escapes can survive into the JavaScript evaluator without being decoded. Separately, missing context variables throw before fallback expressions like missing || "fallback" can work.

This is related to #197, which reports unexpected character: ->&<- while rendering a POML file.

Validation

  • npm test -- --runInBand packages/poml/tests
  • npm run build-cli
  • python -m pytest python/tests/test_poml_formats.py python/tests/test_basic.py
  • npx prettier --check packages/poml/file.tsx packages/poml/components/instructions.tsx packages/poml/tests/file.test.tsx

@xsploit
Copy link
Copy Markdown
Author

xsploit commented May 12, 2026

@microsoft-github-policy-service agree

@xsploit xsploit marked this pull request as ready for review May 12, 2026 05:25
Copilot AI review requested due to automatic review settings May 12, 2026 05:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves POML template compatibility by preventing XML parsing failures caused by XML-sensitive characters in template expressions/content, and by decoding XML entities before expression evaluation. It also extends conditional templating support and adds a <tool-policy> component alias with regression tests.

Changes:

  • Normalize POML input prior to XML parsing and unescape XML entities before evaluating template expressions / meta eval blocks.
  • Expand template conditional support (XML-sensitive operators, <if condition="..."> wrapper, multiline {{ ... }} attribute expressions, missing-variable fallbacks).
  • Add <tool-policy> component alias and regression tests covering the compatibility cases.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
packages/poml/file.tsx Normalizes XML input, unescapes entity text, extends if handling, updates expression evaluation semantics, and adds shared content processing helper.
packages/poml/components/instructions.tsx Introduces ToolPolicy component with toolPolicy / tool-policy aliases.
packages/poml/tests/file.test.tsx Adds regression tests for XML-sensitive operators/entities, <if condition>, missing-variable fallbacks, multiline attribute expressions, and <tool-policy>.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/poml/file.tsx
Comment on lines 135 to 138
private readXml(text: string) {
text = normalizePomlXmlInput(text);
const { cst, tokenVector, lexErrors, parseErrors } = parseXML(text);
const errors: ReadError[] = [];
Comment thread packages/poml/file.tsx Outdated
const ifElement = this.handleIfElement(element, globalContext, currentLocal);
if (ifElement !== undefined) {
if (ifElement) {
resultElements.push(ifElement);
Comment thread packages/poml/file.tsx
@xsploit
Copy link
Copy Markdown
Author

xsploit commented May 12, 2026

Addressed the Copilot review comments in 39593a7:

  • Switched XML-sensitive input normalization to length-preserving sentinels so parser offsets continue to map to original source ranges.
  • Wrapped looped output in keyed React fragments.
  • Refactored normal element child rendering through processXmlContents() so and normal elements share the same content-processing path.
  • Added regression coverage for inside for loops and expression ranges with raw < / && operators.

Validation rerun:

  • npm test -- --runInBand packages/poml/tests
  • npm run build-cli
  • python -m pytest python/tests/test_poml_formats.py python/tests/test_basic.py
  • npx prettier --check packages/poml/components/instructions.tsx packages/poml/file.tsx packages/poml/tests/file.test.tsx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants